Introduction To Unix For Beginners

The next issue is bash vs. tcsh. If your job puts you in a culture surrounded by people who are all happily using tcsh, then you may have to be comfortable with that and you certainly need to understand what the difference is. Everyone else can skip ahead and take bash for granted. The reason for this is that the whole world has pretty well standardized on bash as a default. Apple started out with tcsh but has since moved to bash.

Basically bash and tcsh are both programs that implement shells. They are quite similar but there are important differences. If you’re a tcsh user and find yourself using a bash shell, typing tcsh will often put you back into your favorite shell. An important tip is that this almost always works the other way, if someone’s given you a tcsh account, typing bash will get you a bash shell.

To demystify it a bit, bash is a derivative or extension of the ancient Unix shell called simply "sh", sometimes referred to as the "Bourne shell". Turns out that sh is still a valid command and you can still run this primitive shell on pretty much all Unix systems. The derivative shell, bash, is short for "Bourne Again SHell". On the other side of the spectrum is tcsh which stands for "The C SHell". Notice how both compete fiercely for snazziest pun. Unix is like that. Get used to it. Anyway, in theory, tcsh embodies more syntax elements borrowed from the C programming language.

If you’re very interested in the differences between bash and tcsch, this discussion highlighting the differences is quite detailed. Or maybe this classic discussion will be interesting to you.

Primarily what a shell does is it allows a user to specify commands which it then causes the operating system to actually execute. When you first log in or start a terminal, the shell confirms its presence with a "prompt". This is a character that is supposed to be the user’s cue or prompt to do something. If you don’t see the prompt then it is not prompting you to do something and you probably need to wait until something is finished. The prompt is conventionally $ (or > in tcsh) but it could be whatever you set it to. At the prompt, you can type commands. Here’s a nice example of that in action:

The prompt, $, said to do something, I entered the command cal 7 2011 whose function I hope is self evident by the results which then appeared. The prompt appears again inviting you to do something else.

People new to working with their computers through a command line interface or CLI (as opposed to a GUI, graphic user interface) often panic about how tedious they imagine typing in long complex commands will be. When I was starting out learning Linux I asked myself how likely it was that the smartest computer scientists and programmers, all of whom use a CLI to some extent, would do something flagrantly tedious? The answer is, of course, not too likely. It turns out that entering commands by typing them is quite efficient thanks to a couple of key tricks.

If you’re familiar with the ancient DOS command line you may not know about these tricks as command line input with that system was indeed dreadful. But all modern full-featured shells allow the user to only partially type things and, if the thing is sufficiently unambiguous, magically fill in the rest. This slashes typing by quite a bit. What typing remains is highly specific and important information you are trying to impart to your computer. Since you’re touch typing (it is helpful to learn to type well!) you’ll find that in many situations, you can tell your computer what your intentions are much faster than if you had to fumble for a mouse and go through what is essentially a little target shooting video game with the mouse pointer.

In the previous example, cal is a pretty short command. What if the command were factor. There are lots of commands that start with f so that’s not enough to be unambiguous. There are several that start with fa, so that’s not enough. But there is only one that starts with fac (on my system). Because fac is unambiguous, I can type f then a then c and then tab and the command factor will appear. Then type space and then your argument, which is just a fancy way of saying the thing you want this program to think about when it runs. In the following arbitrary example, I’m showing that the fourth root of 81 is 3.

It turns out that the arguments can also often be "tab completed", generally when they are file names. Here’s an example:

This command was specified by typing file then space then forward slash then u then tab then b then tab then ej followed by a final tab. This sequence specifies the parts of the location of this file named eject. That’s 12 keys instead of 19, or less than 5 seconds for a very slow typist. These parts are separated by forward slashes. If you’re used to DOS, you might think that something like C:\Trash contains a "slash". Not so. That is a "backslash". Get used to the terminology of "slash" meaning /. This is important in Unix which uses proper slashes to help organize files.

A very important Unix trick to know is the man command. This is short for "manual" and it is the key to not having to memorize all of the arcane details of every Unix command. Most commands and programs have a "man page" which is an explicit record of all the command’s acceptable syntax. This includes a reference list of all of the command’s options, a description of its output and a description of what is required for input. An example:

This shows the man page for the factor command. Use arrows to navigate and q to quit viewing the man page.

Another trick for getting help is to try the -h option. Many commands adhere to the convention of interpreting the -h option to mean that the user wants some information, albeit very terse, about how the program is used. Here’s an example:

As it says, using the long form option, --help produces even more help. This is very important in programs that use the -h option for something else like (often "human readable" as in ls or df).

This produces a pretty comprehensive summary of what options ls can take.

The other critical trick that the command line shell has is called "history". Even if forced to type a complicated command sequence, Unix experts would feel foolish doing that twice. The shells remember hundreds of the most recent command sequences you send to the computer and can reuse them. The easiest way to use history is to use the up and down arrows. The up arrow puts the most recent command on the command line. You can edit it and resubmit it or just resubmit it. Or you can keep pressing the up arrow until you get to the command from the past that you want. This is very useful and makes things, especially recovering from mistakes, much quicker.

You can also type the command history which will show you the history as the shell remembers it. There are much fancier history tricks for more advanced usage.

It might also be a good opportunity to advise some good habits with respect to naming files. In Linux you can make files with terrible and inconvenient names, but it will turn out to be a terrible and inconvenient idea. Since command lines are parsed looking for certain special things, it’s really not wise to have those special things as a part of your file’s name. Special things that should not go in your file names include:

To start learning how to use files we’ll start with a somewhat obscure command. This command creates files from nothing. It’s obscure because there are many other more natural ways to do this. Nonetheless, try creating a file with:

Barring some kind of problem (disk full?), you’ve just created a new file. Note that almost everything in Unix is case sensitive. This file is empty right now. The only space on the hard drive it uses is to keep a record of its own existence. It has no contents.

We can change the file’s name with this command:

This changes the name of ANewFile to CPUdata. The command here is short for "move". Notice that moving and renaming are essentially the same thing.

Similar to moving is copying which creates a completely new file and leaves the original intact. Here is an interesting way to use cp, the copy command.

This command is populating the file we created and renamed by copying the contents of another file into it. Another way to think of it is that CPUdata gets obliterated with the contents of /proc/cpuinfo, something to be careful of if CPUdata was valuable to you as it was.

It turns out that /proc/cpuinfo is an interesting example file. It is found on all Linux systems and what is interesting about it is that it is fake. All of the files that start with /proc/ are actually not files stored on the hard drive. They are messages supplied directly by the kernel in the disguise of files. In this case, we want to know what the CPU’s specifications are so we find out by checking the contents of this file. When we check, the kernel immediately synthesizes some contents. This content could change depending on the state of the kernel. But you can use normal file handling tools to interact with this information. This consistency is an example of the Unix philosophy of everything is a file.

To actually have a look at the contents of this new file we created or the source we got the information from, we need the following command:

This will cause the OS to lookup the contents of the specified file and spew it out on the screen for you. Why cat? It is short for "concatenate" and this command’s nominal purpose is to join multiple files into one big long one.

This command produces a long output of the contents of /proc/meminfo, another virtual file, followed by /proc/cpuinfo.

This output is probably longer than your terminal and when that happens, the output is pretty much useless if it runs off the top of the screen before you have a chance to see it. One thing you can do is hold the Shift key and PageUp to scroll up and see what happened. Another way which works better if you can anticipate more than a screen full of output is to use what is called a "pager". There are two generally used pagers, the classic more and the modern less. It turns out that less is more. More of those questionable Unix puns. This shows how to use both pagers on a rather big file that should exist on most systems:

You might notice that more only stops the output and waits for you to press Space or Enter every so often so you can read it and once all of the lines have been shown, it ends. More useful is less which allows you to use the arrow keys to go up or down and the Space to jump forward a page. At the end less is still active in case you want to go back to the beginning (pressing g goes to the beginning and G goes to the end). To quit using less press q.

Now you know how to create files, rename them, copy them, and look at them. The last major command to know about in the life cycle of a file is how to get rid of it when you no longer need it. The remove command, rm, deletes a file:

You can try to erase /proc/cpuinfo but it won’t work. It’s not really there physically anyway. Generally you need to be very, very careful with the rm command. There is no "Trash" or "Recycle bin" when using normal Unix command line tools. If you tell the OS to erase the whole drive, you can expect to have that done in the fastest most brutal way possible. It’s like a gun — a serious tool that can easily produce very serious accidents. Be super careful with rm. It’s good to use tab completion to make sure you’re really deleting what you think you are and not a similarly named typo file.

Not only can you take output and redirect it to a file, you can also send output of a program to be the input of another. This is called a pipe and it allows you to string together a sequence of simple commands into a very powerful complex one. Here’s a simple example.

This takes the long help message of the ls command and pipes it to the less command which allows you to scroll through it at a comfortable reading pace (q to quit less).

You can string together many programs like so:

Here I’m dumping the contents of /proc/cpuinfo into the filter program grep which will output only the lines containing the word name. That output will go to a program called wc which is a "word counter"; it also counts lines if you give it the -l option. The result shows how many CPUs the system thinks it has to work with.

Sometimes these pipe chains can get quite long and complex. A major aspect of the Unix philosophy which seems to have held up well over the decades is to try and focus on small tools that do one very small specific task very well. Pipes are the glue that bind these high quality simple tools into high quality complex ones.

Here is a simple command. (Try running it like this!)

Here is a command with an option - an option changes how the command behaves, sort of an adverb.

This makes the date come out in UTC. Useful! Note you can also achieve the same effect with this.

And in this (well-chosen?) example you can even do this.

The "--universal" and "--utc" variants are simply convenient synonyms. That’s actually not too common to see such redundancy. Ignore that.

But the "-u" and "-utc" are quite different strategies to achieve the same thing and both are quite common. The first is a classic Unix option. They are one letter. It was not envisioned that a program should have 8000 "options" that you’d control on the command line. And if you did need that, you’d do something like this:

Classic Unix one letter options tend to be shorter (obviously) and more cryptic. Both of those properties intensify when you consider multiple options. Compare these two ways of saying the same thing.

It turns out that because the Unix one letter options are guaranteed to be, by definition, one letter, you can get away with either of these.

This is why I have an alias called "rap" which I use a lot.

Which is short for the long version which is this.

These full double dash options are often called GNU long options because GNU popularized it in ancient Unix times. The benefit to long options are that they are more comprehensible. This is valid and valuable when scripting. Although I might type "-u" on the command line, if I put "--universal" in scripts, when troubleshooting my script years hence, I won’t have to look up what the hell "-u" is. It’s also helpful when you need a "--rate" option and also "--recursive", "--relative", and "--read-only" options.

So that’s "options". The next thing to understand is the concept of "arguments". A command argument is simply something you want to program to operate on. Grammatically, if the program is a "verb", the arguments are the "direct objects".

Consider this.

The + looks like some kind of option but here it is actually an argument. It is the text string describing what kind of date format you’d like (year only in this case - "2020"). I use this somewhat confusing example because some programs do use "+" for a type of option. This is not exactly standard or common, but it does show up occasionally. For example using the unruly maverick software xed.ch/h/im[ImageMagick], you can negate an option like this.

Did you notice something else non-standard about that ImageMagick identify command? They didn’t use double dash for a long option. In a normal Unix program it would be interpreted like this.

But ImageMagick is not playing along. Pretty much everything else does! Remember, how the program handles options is up to the program. Although there are exceptions like this, there are important conventions.

So that’s complex options and arguments. What about option arguments? They are arguments not to the program itself, but rather to the option. Consider this example.

this is the second "/mnt/backupvolume/backups2020/" (since it is the last argument, the destination). But what about this?

What is "ssh -p123"? That is an option argument; it is the help that the -e option needs to do the right thing. Obviously you can’t do something like this.

Because "-P" does not take an option argument. But this unfortunate set of options would work.

Here’s the actual command I tend to type in real life.

With a long option argument the equal is usually necessary (true in rsync). This style minimizes confusion. But if you type this so often that it hurts, go with the short option and reduce the command to the minimum.

That explanation goes a long way to unravelling the weird things you see on the command line. When programs break this tradition, they are doing it wrong. Programs like ffmpeg are so complex and baroque that they do break with tradition and their command lines are very difficult to use. ImageMagick does it wrong. Ironically, both of those pieces of software are very hardcore and the idiosyncratic architecture must be begrudgingly respected because they do indeed have Reasons. Normal little people like us do not and we should adhere to the conventions when creating software.

Understanding this goes a long way towards unravelling the whole mystery. After reading this, just check out some man pages to see how useful this is. Check specifically man date and man rsync. In the rsync man page - which is a lengthy beast - practice finding the thing you really want to know about by doing something like "/" (forward search) and then, for example, "--rsh".