Welcome to another dev week! Last week we realized that the Rust programming language will help us scale our codebase with confidence compared to the wishy washy landscape of C++. But how?
If you wish to make software from scratch, you must first invent the CPU.
~My parody of a beloved quote by Carl Sagan
Finding the right level of abstraction
Abstractions enable us to understand or use complex concepts with less mental overhead and burden. Abstractions are one of the most valuable tools in software programming for developing high-level functionality with less mental burden. However, finding the correct level of abstraction is a constant challenge with each step we take with software.
An example in computer networking
Internet
Most people have a fuzzy idea of what the "Internet" is. It's a place where we can "transfer data", including text, audio, and video across this magical wire (or airwaves) to someone anywhere else in the world. Put simply, it's a communication medium similar to making a phone call, but a phone call only allows audio, and early days of the telephone only allowed very few privileged parties to access and use it at once (no packet-switched networks then). And most people don't need to understand the internet at a much deeper level because websites, web services, our devices (phones, computers, operating systems), and web browsers abstract it all away for us. All a user needs to know is to open their web browser app and type a url or search into their address bar. The next thing they know, they "visited a website" and now they have information on their screen from that website. "Apps" abstract this away even further, because all you have to do is click on the app and you are presented with various functionalities backed by that organization's services/servers, so all you have to do after logging in is hit "new post" and start typing.
Packet
Most new computer networking students have a fuzzy idea of what a "packet" is. As mentioned, the telephone network only allowed a few conversations at once because of the limited amount of wires, and each wire could only transmit one conversation at a time (this is not entirely accurate, but again, the whole point of this conversas whtion is abstractions). The packet (and internet protocol, IP) is what allowed us to reuse these few wires more effectively to allow many communication streams at once, providing the backbone that led to the internet we know and love today. A packet is simply an abstraction around turning that large video file on your device into many smaller pieces suitable for transfer over the internet to the intended destination while simultaneously sharing those wires with millions of others. Without packets, the internet would be relegated to a few who could afford the rights to time slices over those network cables, and we wouldn't have the wide-open internet as we know it.
Packet Internals
Some enthusiastic computer networking students will understand deeper still what a packet is and how it works. It has various bits of information (literally bits) which specify what type of data it holds, protocol versions, source and destination IP addresses, and all kinds of information needed to get it across the internet to the intended destination. Out of everyone today who has learned or will learn what a packet is, most of them will never write a single line of code destined to create or transfer a packet over the internet. This is because we have FOSS web server software with permissive licenses such as Apache and Nginx which abstract this away for us, so we can focus on concepts such as "websites", "services", and "HTTP APIs" instead of packets.
Even the SFML game development library for C++ has a Packet class, but again, developers need not know how the packet works; they only need to know to put data inside of one and receive data out of one on the other end. The documentation even states, "Packets provide a safe and easy way to serialize data, in order to send it over the network using sockets (sf::TcpSocket, sf::UdpSocket)." For those who know what a packet is, this sf::Packet is not even a real "packet", as "you can consider there's no limit" (Laurent Gomila, sfml forums). In that way, the sf::Packet is abstracting away a block of arbitrary length data into packets for you, so you don't have to. It would be more accurately called a "PacketReadyDataStream", but that doesn't sound as nice.
Why abstractions matter
One of the main purposes of abstractions is to reduce mental burden and enable ourselves (and others) to move forward with less effort. When it comes to doors in our physical world, a doorknob provides an affordance which enables us to understand how it is meant to be used. A doorknob provides a public interface that most people will understand: "twist this to unlatch and therefore open the door". We don't need to understand how the twisting of the knob moves mechanical gears and springs in order to recede the latch bolt away from the strike plate, or even such jargon, because engineers have abstracted it away for us into a simple and understandable "doorknob" interface. There may be 10 different unique types of mechanisms used to make the doorknob work, but we don't need to concern ourselves with any of them to use it. The same goes for engines in cars for drivers and software libraries for developers.
How is this relevant to us?
I've made plenty of GitHub repositories that I "hoped would be useful to someone". When I originally got a Stream Deck, I failed to find any (linux-compatible) user-friendly GUI software that could allow it to be configured and used as nicely as Elgato's software. Instead, I found the low-level Python library python-elgato-streamdeck which has very little abstraction (sending image data and receiving button press events). I built my own abstractions on top of it, encapsulating Boards/Pages and Buttons. While these were good for my specific use case, they required a programmer who understood the code directly to use them, and I could quickly see that my abstractions were falling apart (or simply weren't nice enough) as I tried to build something more complex. I knew this meant I needed to redesign the abstractions and refactor (or start from scratch based on what I had learned), but it ultimately never happened. I'm not too heartbroken considering I eventually found an excellent software called StreamController. StreamController has very nice abstractions for pages, actions, labels, board layouts, and just about anything else you might want to use to conceptually build your Stream Deck GUI. As a result, it's easy to configure and doesn't require deep understanding to set up and use. The abstractions provide our clean interface for interaction.
Another abstraction mistake of mine was the Camera I implemented in the Python version of my gamedev adventure. Although it appeared to work as expected, I wasn't confident in the implementation details and therefore lacked confidence in my usage of the abstraction. Since I was the only user of my Camera abstraction, (also I didn't write any automated tests..yet..) I didn't have testers to reveal logical issues with the Camera's translation from world or pixel coordinates to drawing coordinates. This hindered progress of my development as I started using the camera to translate everything before drawing, not knowing for certain whether a machine or character was standing on the block that it appeared to be. It probably was, but I couldn't prove it to you; and that's what we need! Proof that our abstractions aren't deceiving us or misrepresenting their internals!
How is this relevant to switching languages?
Whenever a new piece of software is being built, the designers must decide on the abstractions and levels of abstractions. C++ aimed to provide abstractions to coders to control system resources (memory) with minimal (runtime) overhead. However, C++ didn't also implement strict rules and magical algebraic stuff to disallow various types of memory-related flaws. Rust has the concepts (abstractions!) ownership, borrowing, and lifetimes, which the compiler can then use in an automated fashion to understand and reason about your code in a way that's not possible with C++. There are plenty of tools to try to help you do just that in C++, but they still don't offer strict guarantees. Rust is your friend, but in C++, the onus is on the developer and their decades of experience to avoid making those bugs.
I'm almost certain that highly advanced C++ developers are silently citing rules in their head every time they interact with pointers:
- "Treat your pointers like a cat in a new home: make sure they have a safe place to start" (always initialize your pointers).
- "A null pointer is like a pizza box in the fridge: always check it's not empty before you get your hopes up or risk crashing the party" (always check pointers for null before dereferencing).
- "Think of dynamic memory like a subscription: someone has to pay for it, and you better cancel it when you're done or it'll keep draining your resources" (manage ownership and lifetime resources meticulously).
After looking into SFML, I really liked the general abstractions and level of control it still offered its users. But as I dug in and understood the difficulties it faced in the past and today, I recognized that SFML itself (and many C++ libraries) struggle with abstractions of the underlying language. For example, C++ has too many error handling mechanisms, which each have their own distinct camps of thought throughout the developer community all supported by decades of individual experience, which leads to heated disagreements of how error states should be thrown or handled by a library. "Even boost filesystem has support for both codes and throwing. Exceptions are too oppressive to be used everywhere. 
" https://en.sfml-dev.org/forums/index.php?topic=13240.0 And even explicitly bleeds into breaking the software design choice fourth wall (from 2014): "maybe in 20 years from now we will look back at these ideas [of how to handle errors] and wonder how we could be so crazy ;)" https://en.sfml-dev.org/forums/index.php?topic=13240.msg93032#msg93032
By providing more opinionated abstractions over resource usage and error handling, with the foresight of C++'s struggles, Rust has had time and space to develop a robust solution, giving developers safety by default, and control when they need it. Rust isn't perfect; it can still have memory leak issues such as reference counting cycles or in some cases with collections. But Rust's goal is to disallow undefined behavior, which significantly reduces the risk of "what if?"s littering your codebase.
Game Development
I haven't picked a game development framework for Rust yet, but Bevy Engine looks extremely promising. Bevy is the primary successor to Amethyst which was archived in April 2022. Carter Anderson's GitHub intro line is: "Creator of Bevy Engine | GameDev | Programmer | Artist | Previously Senior Software Engineer at Microsoft." He got to learn from the challenges of Amethyst to build "Amethyst Engine 2.0". From reading about Amethyst's history, it seems that although it was beloved and built by many, it struggled from too many cooks in the kitchen, while Carter had time to build Bevy from the ground up and learn from Amethyst's codebase and history.
There is another game engine in Rust called Macroquad, but it struggles from the abstraction level problem. It is trying too hard to support WASM (building to a game/binary that can be used directly in a web browser page), and since WASM didn't (doesn't?) support FPS limiting (frames per second), Macroquad hasn't been enhanced to handle FPS limiting on any platform. While the oldest open ticket I could find relevant to the issue was opened in October 2020, there are many tickets since then that all come to that same conclsuion. This is a failure to provide the right balance of control and abstraction to the user. Although Bevy may not directly support this capability either, there's a plugin, and there's still an open issue to have well-thought-out support for it someday (and not tacked on as a hack). That's not to say you can't do your own workaround in Macroquad, but when focusing on desktop-first and web-second, it's not promising that the library intentionally doesn't give support for frame limiting just because it can't be supported natively in WASM (also the core/original problem originates in WASM, but bleeds into Macroquad..).
Rust Progress
I spent quite a bit of time learning Rust this week. I progressed most quickly and happily when I focused on completing the Rustlings challenges, then when getting stuck, referencing Rust by Example for the most relevant code, and then if I was still stuck, referencing the Experimental Rust Programming Language Book, and then the Primary Rust Programming Language Book.
While the most important concepts of Rust (ownership, borrowing, lifetimes) may be challenging for those who have ingrained habits or otherwise have a very strong understanding of how their language should work, the basics aren't too difficult to grasp, and I consistently feel welcomed into Rust with open arms, both as a developer and as part of one or more marginalized groups.
Although there was some tumult in Rust in the 2020-2022 time range, they've since developed the Rust Foundation which provides better oversight and transparency into organizational decisions.
Regarding security, Rust has plenty of CVEs that are being tracked. There is also research, but some suggest, for example, that the Heartbleed bug couldn't have happened in Rust. David Svoboda et al show that Rust still has security limitations and potential for misuse, but is overall certainly "safer" than C++.
Side note: I've been making Anki flashcards for learning Rust; I wrote about Anki back in 2018, and as I look now, Anki's codebase is 45% Rust!
Now
As a result of beginning with a new language and wanting to get acquainted with the community and history, I spent a lot more time researching and digging around than I would have liked. I really enjoyed it, though I ended up on a lot of deep side-paths (for example learning more about compilers) that I realized might never make it into this blog series. As a result, I found my attention was too far split, and with the inundation of knowledge I've accumulated all week, I actually struggled to focus and decide a specific blog post topic. I wanted to talk about so many things. Maybe this just means that I'll never run out of topics, and maybe it means I can make better-structured versions in the future of my currently fuzzy ideas. There were several topics I did want to mention but am running out of time for (and several topics I should probably not waste any more time on, since I don't find social drama particularly interesting or fun).
Next
My intention this week will be on getting Rust to become second nature as I focus on coding first, and a blog post flowing out of that effort next. I also hope to have a game/media framework/engine chosen by next Friday. The goal with Rust and whatever engine we use is for them to reduce mental burden and get out of the way so we can focus on the goal at hand: developing some awesome software!
EDIT: Finished the Rustlings within a few hours after this post.