Book recommendation: Elements of Computing Systems

Ever since I started transitioning into a “coder” or someone that writes code, there has always been a lingering feeling that I couldn’t shake off. Maybe it’s imposter syndrome? I’m not sure.

I’ve felt that I knew how to write code to get something working or read an API/documentation on the latest Python package and get things running. It works but I’m never quite sure HOW it works.

I know I need to type python script.py in the terminal but then everything else that happens might as well be a black box to me. To me, this was a knowledge gap that I had for far too long which I’ve wanted to address without knowing how. Luckily, one day while stumbling around on hackernews reading a thread about people’s favourite books, I found one book mentioned multiple times as a highly recommended book for self taught coder/software engineers similar to myself.

The book is called “The Elements of Computing Systems” by Noam Nisan and Shimon Schocken. There is a great TED talk done by one of the author, Shimon Shocken which goes into the unique teaching style this book encourages in this video: the sense of discovery and working things out yourself instead of being spoon fed the answers and rote learning.

The book is ambitious, it starts off with basic gates such as a NAND gate and by the end of the book you are expected to have built a memory chip, ALU, assembler, virtual machine, higher level language and finally an operating system. It’s why this book is also commonly refered to as ‘Nand2Tetris’, since you literally do start off with a NAND gate and end up with the capaiblity of coding up the tetris game via a higher level language running on your own OS.

Even though the scope of the book is quite large, it manages to keep each part connected, building on top of the previous but also in a manageable “size” as not to overwhelm the reader. “How do you eat a whale? One piece at a time.” Each chapter of the book ends with a project where you are expected to implement the exact hardware or software that was described and explained in the chapter. Apaart from the design specs and requirements, the implementation details are left up to the reader to discover by themselves. The authors expect that the reader to figure things out for themselves.

A deliberate design that I really liked from the book is the fact that it provides the necessary tests and checks to ensure that the feedback loop is fast, you are able to check your implementation to see if it is correct. This is one of the critical elements for “deliberate practice”, ideally you would like “…feedback on results is continiously available…“.

I have a severe dislike of books that have questions but never provide the answers, so many math text books are guilty of this. It really doesn’t cater to the self-study crowd of readers.

So far I have not run into any major issues. I did follow this medium article and code repo to setup my environment. There was also some questions which I found to be unclear but those were all solved by searching the official forum.

Currently, I have finished the hardware chapters of the book and now up to the Virtual machine chapter. It’s been an eye opening experience for me personally, the closest analogy is that the “fog of war” feels like its lifted on the black box. Previously I didn’t really have an idea on what stack and heaps were and why it mattered, or why it was faster to access the stack rather then the heap. The mechanism of Python and how the interpreter compiles the code down to a “bytecode” representation which then runs in a VM now makes sense to me. Understanding how the computer processes instructions sequentially in a stack gave me a stronger understanding of what it meant to have a “thread” or “multi-threaded” routines were doing. Will this make me a better programmer or software engineer? I’m not sure.

I’m enjoying the journey though and will be sharing my implementation of the Nand2Tetris assembler and VM that I implemented in Rust in later posts.