I'm backfilling weeks I missed. December was rough.

I'm currently running Vikunja for my todos. I only started using it recently. It's pretty neat, but it's lacking on some of the basics. The UI feels tacky. Specifically, when switching between projects, on mobile you must press on the same project twice before it will actually open it up. There's no protection from double-entry; if you hit enter twice after typing in a todo or new project name, they will get created twice.

I tried out StandardNotes as an alternative. It looks really cool and useful, but as a self-hoster, I couldn't get it working fully. I got the server running, and could see that I could make a network connection to it; I got the client working, and was able to create a note in the local browser store. But I couldn't log in and sync from the client to the server. They were communicating, but the login failed and there weren't enough helpful messages to diagnose it. From watching the server error messages, I saw "No cookies provided for cookie-based session token." One second, the client would say that I was signed in and my workspace had been synced; the next, it said that login credentials were required to continue.

I think StandardNotes has a really nice stance on privacy, TNO, and generally on their business practices, but with a software and ecosystem that touts itself as privacy- and security-oriented from the ground up, the absolute minimum basics of being able to connect the client and server must be fully working and stable to be able to use them. The concern with this buggy, opaque behavior with insufficient error messages is that one day it could stop working and be near impossible to diagnose, losing all of your hard work.

I'm sure StandardNotes works better for someone who just says "heck it" and uses their client with their server. As long as you trust that the channel where you downloaded the client from works as advertised without any supply-chain attacks, and that you trust the software given that it is unmolested in the supply chain, I can see that this would be a beautiful piece of software.

However, I'm heavily focused on self-hosted. I carefully followed through their instructions, and everything seemed to almost be working perfectly. But I hit a roadblock with getting the client to talk to the server correctly.

Maybe I'll try again someday, but considering Vikunja is working for the moment, and the great risk of losing any data I enter into StandardNotes, I can't recommend StandardNotes as a self-hosted option for notes.

This week I worked on the podcast player's autoplay!

Previously, when you'd be listening to an episode, you'd get near the end and hit the next key to listen to the next episode.. and the next episode would start loading, but it wouldn't play. This would require you to now manually hit the play button, which especially on small devices, can be difficult (think: trying to start the next episode on your mobile device while driving..). Also, the playback speed wouldn't persist, so you'd have to click another tiny radio button to restore your playback speed. Now, whether or not the player would automatically switch to the next episode when you got to the very end of an episode without pushing the next button, I don't know. It was supposed to, but I didn't test it thoroughly because I always hit "next" manually.

Now, it automatically refreshes the playback speed and starts playing the episode! Single-click to progress!

I thought I had seen a page on MDN (Mozilla Developer Network) about the audio tag event lifecycle, but I couldn't find it after what I felt was too much time spent. Instead, I added some logging scaffolding to watch every audio element event as it occurred. This code automatically overrides console.log to add all log messages to a div element to the page instead of only to the console, enabling the viewing of logging even on mobile!

const originalLog = console.log;

const logToScreen = (...args: any[]) => {
  originalLog.apply(console, args);

  // Create debug element if it doesn't exist
  let debugDiv = document.getElementById('debug-log');
  if (!debugDiv) {
    debugDiv = document.createElement('div');
    debugDiv.id = 'debug-log';
    debugDiv.style.cssText = 'position: fixed; bottom: 0; left: 0; right: 0; background: rgba(0,0,0,0.8); color: white; padding: 10px; max-height: 200px; overflow-y: auto; font-family: monospace; font-size: 12px;';
    document.body.appendChild(debugDiv);
  }

  // Add new log entry
  const entry = document.createElement('div');
  entry.textContent = `${new Date().toISOString().split('T')[1].split('.')[0]} ${args.join(' ')}`;
  debugDiv.appendChild(entry);

  // Keep only last 5 entries
  while (debugDiv.childNodes.length > 100) {
    debugDiv.removeChild(debugDiv.firstChild);
  }
};

// console.log = logToScreen;

All you need to do is comment out the last line to enable it, and ensure your code has some console.log calls.

This allowed me to realize what I really needed to do: record speed and play state whenever they change; when metadataloaded is finished, trigger a play; when play is triggered, restore the values. There's some extra boolean logic to make sure it only restores at the correct times, but that's the basic gist of it.

This also reminds me that I'd also like to style the UI so the playback speed radio boxes look like nice big clickable buttons, increase the size of the forward/backward/previous/next buttons to overall make it more mobile-friendly.

As if that's not enough, I also want to show the date of the episode on the page, but that requires a backend change and I've been super focused on what I can do only by changing frontend code. It gives me a greater appreciation for communication and teamwork, because if you only know how to do frontend or only backend, and depend on someone else for vital functionality, if they don't do it then it simply doesn't get done and you're stuck! That's certainly a nice thing about knowing backend and frontend.

I've always been more of a backend gal, but I think I'm finally getting the hang of React, reasonably, especially with using lots of custom hooks to keep functionality nice and self-contained. The code needs another refactor after working on the autoplay, but it's worlds apart from the raw HTML/JavaScript I wrote on my dashboard/weather app back in 2017. Unfortunately that code was a bit too tied-in with my "personal stuff" and I never published it on Github. Thinking back, that "personal stuff" may be entirely irrelevant and non-sensitive today, so hopefully I'll get a burst of motivation someday to go find it, clean it up, and publish it.

Back on "As if that's not enough", I have a lot of functionality I want to implement, and I'm trying to avoid making a huge list and overwhelming myself, instead focusing on the most important usability functionality right now. The past two weeks I got playback to be really smooth, being able to play the audio as soon as you hit the page instead of waiting several seconds, and the ability to switch to playing the next episode at the same speed with a single click. I think these are massive usability improvements. I'm wanting to implement the improvement I mentioned on last week's post about the server loading segments from file in increasing sized groups, instead of one segment from the filesystem per request while running an additional backend job to load the entire file. While the algorithm is unclear, the idea is not.

As I was working on these two features, it's really making me itch to start making automated browser testing using Selenium. I've been exposed to Selenium but I actually got to use it very briefly for work at least once or twice, and it's not terribly scary. My time spent manually sliding and scrolling and skipping and clicking, all would have been a lot faster (or at least less tedious on my part) if I had an automated testing suite. There's so much I want to do!!!

In case anyone is wondering why the image of the console logs shows the "Adding event listeners to videoRef number 1" and another message each twice, it's because in non-production mode, React useEffect runs twice in a row each time to help developers catch bugs, because useEffect is supposed to be idempotent, i.e. running it twice in a row should behave exactly the same as if it had run once. See the docs if you're interested.

I've made it up to SN episode 146. Steve mentioned Black Viper (https://www.blackviper.com/), who has an "In Loving Memory" page, which I'm considering making one of my own. Additionally, I need to rename Dev Week to Tech Week.

In my home lab stuff, I struggled with Element/Synapse/Matrix a little bit. I needed to manually reset permissions on my Element client (which had gone from something like rwxr-xr-x to ------ [?!]), change the configs to stop automatically logging my dang clients out, fix router DNS settings (there was an old record that was wrong and overriding the correct record), and fixing up the Nginx server to redirect the .well-known paths for server discovery. Phew! All that just to post the image above of the logs in the playlist app, but now my entire system is much nicer, with additional comments in all the config files I touched, setting up a highway that I can drive on the next time I have to deal with something (no, I'm not using Ansible or Terraform, which I'd love to use if I didn't already have so much stuff set up without them; maybe the next time I migrate servers, or maybe I'll experiment with a raspberry pi.. if I weren't so busy building a podcast app). I'm getting much better at commenting and leaving myself breadcrumbs to get back faster.

I interact with a lot of different FOSS technologies and I'm wondering about including them in my blog posts more, hence Element/Synapse/Matrix, Nginx, Ansible, (Terraform isn't FOSS), and Raspberry Pi. I don't even know if I've mentioned anywhere on my site: I use Gentoo Linux with Xfce4 as my windowing manager. I love looking at Github repos and installing various FOSS software, setting up lots of Docker containers with different apps. I have around 27 apps installed as Docker Stacks/Compose, and I'd like to mention at least 1-2 each week.

Thanks for joining us for this week's Dev Tech Week!

Oh no, I missed another blog post this week! (This is being written after the fact.) Fear not, though, for I have not stopped work on my podcast-/playlist-listening app!

This week I implemented a dramatically improved instant-listening feature. Previously, when switching to a new episode, there would be a nontrivial delay before the episode would start playing. Now, it starts playing instantly!

It's still not perfect as I've been playing with it, but it's much better. In particular, it doesn't handle switching episodes quickly, because it's still loading an entire episode in the background while it's feeding up short segments. I have ideas to write a nice little algorithm to load up chunks in a more intelligent fashion, however. To understand the improvement, we first need to understand how it works now after the updates:

• When the first request for a given episode comes in for an audio segment, load only that segment and feed it up.
• Start a background process that begins loading the entire episode.
• If additional requests for more segments come in while the whole episode is still in the process of loading, then make additional single-segment reads while the episode is still loading.
• Finally, when the whole episode is done loading into RAM, feed segments up from memory.

There are a few scenarios this doesn't handle well:

• Switching to an episode and then scrobbling through the episode multiple times right away.
• Switching back and forth between episodes.
• Clients that request a large number of segments upfront.

In all these cases, it bogs down the server with requests for individual segments which each individually have to context switch to reading only a specific segment of the file, all stacked on top of each other. This is very painful to the end user as the server stops responding quickly.

The new process that I'll eventually write goes something like this:

• When a request for a given episode comes in, load that segment and feed it up immediately. Keep track of this request in a data structure but don't do any additional loading (note that at this point some clients such as iOS/Safari will have already sent in 5+ requests at a time).
• When we see the second request for a following contiguous segment, we see that we just had a request for the previous segment. Instead of loading the entire episode or only one segment, load the following 2 segments into the cache and serve up the next 1 segment (the request for the following segment that we're loading from file is likely already in the pipeline, so at least now we'll already have that in RAM and don't need to issue another file read).
• When the next request comes in for a segment just beyond what we just loaded, we see that they are more contiguous blocks. The first time, we loaded 1 segment; the second time, we loaded 2 segments; now load 4 segments.
• Each time this process occurs, we load 2x the number of segments as we loaded the last time. The assumption is that if the client keeps requesting segments from the same episode then they will likely continue to do so, but by not loading the entire episode all at once, we leave the server ready to make requests for segments from other episodes.

This still isn't perfect, but would be another huge improvement over the client response speed.

I'm also considering talking about non-development concepts in my blog posts, and I already renamed it from Game Week to Dev Week, but I'm thinking I'll have to rename it again, to Tech Week. There are plenty of times when listening to Security Now that I think I might like to talk about a security/privacy issue instead of development, especially on slower development weeks.

Thanks for joining me for another Tech Week! 🙂

Unfortunately I missed a post this week 🙁 please check for next week's post!

This week we'll talk a little bit about NLP, or Natural Language Processing.

ChatGPT took the world by storm, and now we have Llama, Claude, Gemini, whatever. What are they and where did they come from? I was obsessed with watching the progression of AI and Deep Learning around 2016 when Google first released TensorFlow, back when it was still only cool in the tech world, but now you'd probably have to hide under a rock to entirely avoid hearing about it.

Word2Vec

We'll start at Word2Vec since this is my first favorite machine learning algorithm. Let's borrow this image from School of Informatics at the University of Edinburgh:

King - man + woman = queen: the hidden algebraic structure of words

Word2vec is a clever algorithm used to create what are called "word embeddings". We can start with some graph paper and start writing out some clusters of words that are related and unrelated. Since our graph paper surface is 2d (2 dimensions; logically, not physically), we can consider that each word has an x,y coordinate pair, and that words that are related are close to each other, while words that are less related are far apart. We'll notice quickly that as we start writing out more than just a few words, we start having difficulty showing the far-ness of some pairs that ended up close together because of their related pairing with another common word.

However, using only 2 dimensions is limiting, so we can use 3, 4, 5, or even 100 dimensions. Since we can't easily envision more than 3-4 dimensions (space and perhaps time), we can imagine that each of these dimensions is a different aspect of meaning of words. Maybe one dimension is related to sweetness, one dimension is related to respect, one or more dimensions are related to color. We don't actually know what they are, as they are automatically "learned" by an algorithm, but this helps us get some feeling of what a "word embedding" is. Elephants and horses might be close together in the "mammal" or "animal type" dimension but far apart in the "size" dimension.

The algorithm works by first assigning a semirandom vector to each word in the training data, where each number is pulled from a random normal distribution centered around 0 with values approximately between -1 and 1. Imagine something like [0.12971, -0.2614, 0.08536, ...]. Since they are random, they don't actually contain any semantic meaning yet. Next, the algorithm runs by comparing proximity relationships between words in training data. This training data can be Shakespeare, emails, books, Wikipedia articles, or anything that contains lots of words and shows usage of those words and the relationships between words. There are multiple ways to look at words that are close to each other, for example checking every pair of words, or every 3 words, or every 5 words, but let's imagine we're looking at every group of 4 consecutive words.

We'll assume our training text data from Wikipedia etc is already cleaned: special characters are removed and the text is broken up into individual words.

The algorithm starts at the beginning of the training text data, looking at the first group of 4 words. If our first sentence is "1 2 3 4 5 6 7 8," then our first group is "1 2 3 4", and then we slide, our second group is "2 3 4 5", third "3 4 5 6", and so on.

Every time we process a group of words like this, we grab the vectors corresponding to those words, and we perform some basic mathematical options to make their vectors closer together. In our 2-dimensional example, imagine that we look at the coordinates for our 4 words on the graph, find the middle point of all of them, and then nudge each one slightly closer to the middle point. We only nudge them slightly, because if we nudge them too close, we'll "overfit" the training. We have lots of training data, so there's no need to forcefully nudge words together the first time we encounter their pairings, as if those words are actually meant to be close to each other, certainly we'll encounter them near each other in many sentences in our training data.

Nudge nudge.

There are all kinds of mathematical tricks to deciding how close to nudge the words together, how many times to train on the training data set, how to initialize the numbers in the vectors at the beginning, and so on, none of which we will concern ourselves with.

Next, we can grab some interesting word out of a dictionary, such as "ocean". Then we can perform some calculations to figure out which other vectors are closest to our "ocean" vector, and we might find similar words that we'd expect to see in the thesaurus: sea, tide, waters. We might even find the names of oceans, things that would be found in the ocean, other large parts of our planet such as "sky" and "Earth". To play with a realtime demo and search, see https://projector.tensorflow.org for an interactive version of the following:

Finally, getting back to the graphic from earlier in this blog post:

King - man + woman = queen: the hidden algebraic structure of words

After all this training, if we grab the vectors for king, man, and woman, and then we calculate king - man + woman with our vectors, receiving another vector, then compute the real vector that's closest to our calculated vector, what do we find? Queen!!! No one told the Word2Vec algorithm explicitly how these words are related. It learned it through this rather simple algorithm of nudging word vectors closer together based on where they appear in textual data. This "understanding" or meaning that is captured by our vectors is called "semantic meaning".

I want to work us up to understanding ChatGPT, but we're beginning with Word2Vec because I believe it's one of the most miraculous pieces of technology and tools along the many steps it took to get to ChatGPT. Although it's primarily used for word embeddings, it's also been applied to whole documents and dubbed "Doc2vec", and we could also use it for the topics on Wikipedia themselves (think of how all the Wikipedia articles are linking to each other in a complex web), webpages, or anything else that has some sequential relationship. While ChatGPT may not use "Word2Vec" exactly, it similarly uses "embeddings". These embeddings represented as vectors are data that can be operated on quickly in a GPU card by machine learning models, so it's an excellent way to imbue our machines with some kind of understanding about human language.

Thanks for joining us this week and we look forward to next week! We'll be continuing down this machine learning / LLM path for a while as we work our way up to using LLMs locally on your own machine (not in the ChatGPT/etc clouds).