Low Level

Getting some proficiency with frameworks like TensorFlow and Keras has been pretty interesting. I’m continuing my study of machine learning by trying to really understand every detail of all components. In my own mind I had the idea that I wanted to be able to understand the entire technology chain from the lowest levels to the highest, not necessarily in extreme detail but at least a rough understanding. I had the ambition to understand neural network details as well as I understood how software in general worked. Then I wondered… I do know how software works, right? I’d better check that!

I wanted to find the smallest example I could think of where a cogent change of some specific low-level ones and zeros produced a predictable change in high-level software behavior. What I came up with is a nice reminder that computers do work with just ones and zeros. Here’s my demonstration.

I start with a high level program. It is a simple program to be sure, but it is a C program just like the Linux kernel or a compiler or Quake. The way complexity increases from here is relatively well known.

This creates a program, called p_sub, which should take arguments, count them, and return the total number of them. This requires that the argument count variable be decremented by one (c-1) because the program name is considered a part of the argument count. Don’t worry about the details.

So there’s a usable C program in 28 characters. Next I wanted to see some different behavior by fiddling with the lowest level possible. Instead of subtracting one, I will add three to the argument count. I picked through the program and found the bits that would make that happen. This command line changes the necessary bits and creates a new executable, p_add.

If your hex vision is strong you may object, this is not bits! This is bytes. It turns out that the xxd program won’t allow bit level changes; you do have to organize them by bytes. But I promised ones and zeros! The xxd program can show binary. Here is a summary of the two bytes I changed. This directly shows the changed ones and zeros that effect this behavior difference.

Let’s consider these binary numbers in the reverse order that they are listed. Binary 00000011 is a one in the ones place and a one in the twos place (one one and one two, so three). 11000000 is 128+64 which is 192. And 192 divided by 16 is 12. If hex 9 is decimal 9 and a is 10 and b is 11, then 12 must be c. Therefore 192 is c0 hex. Binary 00000001 is the same in decimal and hex, just 1. Finally 11101000 is 128+64+32+8 which is 232 or e8 hex (trust me).

So I basically changed e8 01 to c0 03. The 1 and the 3 are the ammounts I’m subtracting and adding respectively. But do the op codes for e8 and c0 mean subtract and add respectively? The objdump program can tell us.

Looks right! In fact the objdump program was how I figured out what bits to change.

Do the programs work as advertised? Let’s try them out!

The first program counts the arguments (5 x’s and the program name minus one). The second program does almost the same thing but it seems to add 4 to the argument count (5 x’s, the program name, and an additional plus 3).

The point of this exercise is to show that there is no magic or mysterious hidden processes involved in general computing. One could go lower and wonder what the electrical potentials and clock pulses were doing in the chip after these bits are sent to it. But that behavior is literally carved in stone (silicon at least). Understanding that voodoo is very unlikely to have any helpful effect because there’s not much that can be done about the hardware.

Now I need to get back to studying neural networks. I don’t like the feeling that TensorFlow is fooling me into going along with something stupid simply because the details can be a little hard to understand.