These notes mostly cover symmetric cryptography where a single secret key is used and known by all participants (maybe you and your future self, etc.). For an excellent introduction to what "asymmetric cryptography" is check out this very helpful video.

Encrypting Files

Vim

For Vim users, the simplest way to encrypt something is to open it in Vim and use :X and enter a pass phrase. Save the file with :w as normal and the file will be encrypted:

$ file vimsecret
vimsecret: Vim encrypted file data

If you’re interested in a strong encryption, use this.

:setlocal cryptmethod=blowfish

To always use this, put this in your ~/.vimrc.

set cryptmethod=blowfish

The default cryptmethod is zip which is compatible with Pkzip’s encryption method. Note that this encryption method is breakable by a mildly motivated attacker. If it’s really important, use blowfish.

To remove any Vim encryption, use set key= and then :w the file.

See :help encryption for other exciting details.

To for other tips on using Vim securely, see my Vim notes.

GPG/PGP

PGP, or "Pretty Good Privacy" is the classic piece of software which applies strong encryption to anything that needs it with a particular emphasis on email. Don’t let the name fool you, the privacy is more than pretty good, it is superlative. There were patent and export regulation issues surrounding the original PGP, so the good folks at GNU went ahead and resynthesized the functionality from scratch and created GPG as a drop in clone unencumbered by any legal nonsense. GPG stands for "Gnu Privacy Guard". The GPG website is here.

For email applications, see my specific notes on PGP/GPG

To encrypt a file without using keys or any of the trust infrastructure GPG is capable of, use the --symmetric flag (the short option -c does the same thing).

GPG can handle many fancy cryptographic protocols. It’s default is probably fine for all normal people doing normal things, but if you want some real tin foil hat excitement, it can’t hurt to specify that GPG use AES 256 bit encryption. To do that, simply include --cipher-algo AES256 as an option.

To decrypt, just run gpg with the encrypted filename as the argument. It will know what to do.

$ cowsay "GPG is good." > myreadablefile
$ file myreadablefile
myreadablefile: ASCII text
$ gpg --output mysecretfile --cipher-algo AES256 --symmetric myreadablefile
gpg: gpg-agent is not available in this session
Enter passphrase:
Repeat passphrase:
$ file myreadablefile
myreadablefile: ASCII text
$ file mysecretfile
mysecretfile: DOS executable (COM)
$ rm myreadablefile
$ gpg mysecretfile
gpg: AES256 encrypted data
gpg: gpg-agent is not available in this session
gpg: encrypted with 1 passphrase
gpg: mysecretfile: unknown suffix
Enter new filename [myreadablefile]:
:-> [crow][~/testy]$ cat myreadablefile
 ______________
< GPG is good. >
 --------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Note that "DOS executable (COM)" just means that the system has no idea what this file is. That’s good.

This also works using pipes.

Using GPG with pipes
$ cat myreadablefile | gpg -c -o - --cipher-algo AES256  - | cat > mysecretfile
$ file mysecretfile
mysecretfile: DOS executable (COM)
$ gpg -quiet --no-use-agent -o - mysecretfile2
 ______________
< GPG is good. >
 --------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

ccrypt

I like ccrypt. This is a replacement of old fashioned Unix crypt. I think it can be installed easily on Macs with brew. Unlike Unix crypt, the algorithm provided by ccrypt is not symmetric, i.e., one must specify whether to encrypt or decrypt.

mcrypt

On CentOS and presumably Red Hat, the easiest file encryption utility to get working is mcrypt. Apparently mcrypt uses a symmetric encryption algorithm for both encryption and decryption.

Here is the normal operation.

$ file myfile
myfile: ASCII text
$ mcrypt myfile
Enter the passphrase (maximum of 512 characters)
Please use a combination of upper and lower case letters and numbers.
Enter passphrase:
Enter passphrase:
File myfile was encrypted.
$ file myfile myfile.nc
myfile:    ASCII text
myfile.nc: mcrypt 2.5 encrypted data, algorithm: rijndael-128, keysize: 32 bytes, mode: cbc,

This will produce a file named the same as the original but with a new extra .nc extension (I don’t know what the "n" is for). Note also that while it asks for some password minimum assurances, if you know what you’re doing and want a simple password, it works fine.

Note
Notice that the plain text file is also still there and in plain text. The default behavior of mcrypt is to make a new encrypted file. This is ideal for sending off to the archive while the real thing continues to do something useful, but very bad if you think you’ve protected something and you leave the original lying around.

Use mcrypt -d filename to decrypt.

Here is a way to use streams to get pretty much the same effect.

$ cat myfile | mcrypt - > myfile.nc
Enter the passphrase (maximum of 512 characters)
Please use a combination of upper and lower case letters and numbers.
Enter passphrase:
Enter passphrase:
Stdin was encrypted.
$ file myfile myfile.nc
myfile:    ASCII text
myfile.nc: mcrypt 2.5 encrypted data, algorithm: rijndael-128, keysize: 32 bytes, mode: cbc,

With this technique you can run it through compression and/or archiving or whatever and take complete control of the resulting name or destination. There are options to the mcrypt program executable that can do much of this. Here is an example of this being very useful. In this case, it is difficult to get the mcrypt executable on the target machine. Here was the solution.

read -p "Enter pass:" X; KF=keyfile.$RANDOM; \
touch $KF; chmod 600 $KF; echo $X > $KF; \
ssh ldap.example.com sudo slapcat -v -f /etc/ldap/slapd.conf |\
mcrypt  -f $KF - > \
/mnt/usbext/ldap.example.com.backup-$(date +%Y%m%d).ldif.mcrypt ; rm $KF

The trick here is to use keys with empty pass phrases or, as shown here, use the -f option to specify a file which contains the pass phrase. This prevents both programs from asking stdin for user input.

OpenSSL

While I prefer the interface of something like mcrypt, it’s hardly universal. OpenSSL, however, is and it is able to encrypt arbitrary things including files. While OpenSSL is impressively capable of many interesting things, the interface and documentation are utterly terrible. For example, there is no program enc on my system, but man enc unintuitively gives a man page on openssl enc. This is probably the man page to look at for doing general purpose encryption. And since the interface is so bad (long opts with a single dash, a cryptic openssl enc -help, etc.) the man page is pretty essential.

An important thing to note is that if you don’t specify a cipher OpenSSL will just "encrypt" with plain text, i.e. no cipher really. In other words, do not rely on a sensible default cipher. The aforementioned openssl enc -help command produces a list of supported ciphers. So too does openssl list-cipher-algorithms. Purposefully choosing one of these from the dozens on offer is not a minor task. The format of these things seems to be "cipher-key_length-mode_of_operation". This mode of operation stuff is discussed adequately on this Wikipedia page.

Based on advice from the NIST Computer Security Division, I’d say that in 2016, AES (Advanced Encryption Standard) and triple AES are the only recommended choices. Wikipedia has full information on the alternatives. Until I find contradicting information, I’d say that -aes-256-cbc is the best for general purpose tasks.

Here is a reasonable approach. These are equivalent.

$ openssl enc -aes-256-cbc < secret_agenda > secret_agenda.opensslaes256cbc
$ openssl enc -aes-256-cbc -in secret_agenda -out secret_agenda.opensslaes256cbc

It will ask for a password (twice). To decrypt the cipher text, add a -d.

$ openssl enc -aes-256-cbc -d < secret_agenda.opensslaes256cbc

Note that on Mac OS you can get annoying incompatibility errors when decrypting a file encrypted on Linux. Used this extra message digest specifier like so.

openssl enc -d -aes-256-cbc -md sha256 -in secretfile.tar.enc -out secretfile.tar

Adding a -a makes the encrypted output ASCII only (good for mail attachments, etc). This can also be useful just to ascii-armor a stream with no encryption.

Here’s another way that may just be simpler from an interface perspective. Blowfish seems pretty reasonable for normal people.

$ openssl bf < secret_agenda > secret_agenda.bf
enter bf-cbc encryption password:
Verifying - enter bf-cbc encryption password:
$ head -c 20 secret_agenda.bf | cat -A ; echo
Salted__hM-^M-^H^IM-#lM-o)^F^RM-39
$ openssl bf -d <secret_agenda.bf | head -n2
enter bf-cbc decryption password:
                            2016
      January               February               March

This also works with aes256 as an alias for aes-256-cbc just like bf is an alias for bf-cbc.

Encrypted Filesystems

LUKS stands for Linux Unified Key Setup and is a way to manage encryption of hard drive or other data volumes. The way it works generally is to create the infrastructure for encryption and when you want to use the encrypted volume, you open a virtual device that can communicate with it. Opening requires the key. When you close it, the underlying volume (which could also be virtual) is just a bunch of random stuff.

Packages

If the cryptsetup command seems missing it can simply be obtained with sudo apt-get install cryptsetup.

Troubleshooting

Note that the --debug option goes far beyond the --verbose option and is worth using sooner than later when troubleshooting.

Create An Encrypted Volume

Start by creating a volume that is organized for LUKS and password locked with your password which you will define in this step. This command will hose anything already here, so this is a one-time only kind of operation.

sudo cryptsetup -y -v luksFormat /dev/sdX

WARNING!
========
This will overwrite data on /dev/sdX irrevocably.

Are you sure? (Type 'yes' in capital letters): YES
Enter passphrase for /dev/sdX:
Verify passphrase:
Ignoring bogus optimal-io size for data device (33553920 bytes).
Key slot 0 created.
Command successful.

You can create an encrypted volume over another virtual device. This would live in /dev/mapper when it’s open. When it’s not open, it’s just fuzz on the underlying partition or logical volume.

This can be done on top of LVM partitions as shown here.

cryptsetup --verbose --verify-passphrase luksFormat /dev/xed_vg/xed_SunBackup

Open, Use, And Close Encrypted Partition

Once you have the underlying volume configured for cyrpto, you need to open it by creating another higher level virtual device that you can actually interact with. You supply a passphrase here.

cryptsetup --verbose luksOpen /dev/sdX backupLUKS

Now you can use this volume like normal.

    mkfs.ext4 -v -L XedBakLUKS /dev/mapper/backupLUKS # ONLY FOR NEW VOLUMES!!!
    mount /dev/mapper/backupLUKS /mnt/B/
      -- ADD SOME STUFF --
    umount /mnt/B

Now you want to protect the partition from unauthorized use, close it. The paritition unnervingly disappears, but the underlying partition remains with your data encrypted.

cryptsetup --verbose luksClose xed_SunBackupCRYPT

Checking For LUKS

To see if a partition is actually something that can be opened into a decrypted partition, use the isLuks sub-command.

:-> $ cryptsetup isLuks /dev/sdX
:-> $ # Notice return code is happy (zero).
:-> $ cryptsetup isLuks /dev/xed_vg/xed_MonThuBackup
/dev/xed_vg/xed_MonThuBackup is not a LUKS partition
:-< $ # And return code is sad (non-zero).

To see more of the details of what is stored in the LUKS header try the luksDump sub-command.

_ cryptsetup luksDump /dev/sdX

Note that this wants the actual raw block device, not the LUKs volume if it has been opened.

Other Useful LUKS Tricks

Resizing a crpyto partition over LVM (yikes!). It can be done: see here.

Need to change a passphrase on a LUKS volume? This should behave sensibly.

$ cryptsetup luksChangeKey /dev/sdX

Note that you can add additional keys. You will need one of the working keys.

$ cryptsetup luksAddKey /dev/sdX

I don’t mind doing this when I feel like I need a new key because my old one is too complicated or hard to type. Better to leave that as an option too in case I get confused about what the status is.

How To Open Multiple Volumes With One Passphrase Entry

Imagine you’re making a backup of something every day to 7 drives, one for each day of the week; on Monday, you plug in Monday’s drive and start the back up script; on Tuesday, you do the same for Tuesday’s drive, and so on. The host you’re plugging these into every day is not being turned off and you’d like for it to remember the passphrase for all the drives (which will share the same one). The only thing you’re concerned about is if someone breaks in and steals everything, you want to ensure that they can’t see any of the data. This means that the key can be floating around in the live system, but you want to have it encrypted when it returns from a power cycle. How can this be done?

The strategy I’m going for involves using one of the "key slots" of LUKS. A LUKS protected volume can be opened with one of 8 (I think, 0-7 at worst, maybe 0-31 now for LUKS2) slots. This allows you to basically have 8 ways to open the drive (e.g. 8 different admins with their own passwords). In the main slot, I would just put the ordinary passphrase as normal. But then we’ll add a new slot that will be accessible with a key. To do this, start by generating the key.

dd if=/dev/urandom of=/tmp/drivekey bs=512 count=1

This will be just like typing a long random 512 character passphrase. But since we want this to be opaque to someone who steals the system, it itself needs to be encrypted. To do this I’ll suppress my misgivings and used GPG. If it’s good enough for package management, well, I guess it’s better than OpenSSL (which also can work for this). The dd generation can be piped right to it.

dd if=/dev/urandom bs=512 count=1 | gpg --symmetric --pinentry-mode loopback --output /tmp/drivekey.gpg

The pinentry-mode just prevents it from trying to use some PGP keys lying around or something like that; we want it to just prompt for a passphrase. If all goes well, this should make a (binary) file called /tmp/drivekey.gpg.

Now that you have this random thing encrypted with simple symmetric GPG, anytime you point gpg at this it will want you decrypt the gpg part and it will ask you for the gpg passphrase you just used — perhaps with a GUI form box — or it will dig into some agent and see if it already knows what to do. Either way, here is a strategy that can work. The trick is sending the decrypted key right into the LUKS opening process. But first you need to create a normal LUKS system that can be opened with an annoying passphrase. So format it as such and use a master passphrase that will be like your emergency access.

_ cryptsetup -y -v luksFormat /dev/sdX  # Start with normal passphrase slot.

So it turns out that it doesn’t quite work to just pipe the decrypted GPG encrypted key into cryptsetup when adding a key. I actually deeply explored this with the developers and it really doesn’t look like it’s possible. You have to decrypt the PGP encrypted file to use it as a key file when adding another slot. So given that must happen, it makes sense to do that using the encrypted volume. This way the insecurity of having the plaintext version floating around (even briefly) is lessened by the fact that it’s already on the target volume (i.e. if attackers can get at this, they’re already done).

sudo cryptsetup luksOpen /dev/sdX xed_crypt
sudo mkfs.ext4 -v -L XedBakLUKS /dev/mapper/xed_crypt # ONLY FOR NEW VOLUMES!!!
sudo mount /dev/mapper/xed_crypt /mnt/ext

Now prepare the key.

gpg --decrypt drivekey.gpg > /mnt/ext/INSECUREdrivekey
_ cryptsetup luksAddKey /dev/sdb  --new-keyfile /mnt/ext/INSECUREdrivekey --new-key-slot 1
rm /mnt/ext/INSECUREdrivekey # Or leave a copy of it, or leave a GPG encrypted copy.

Now that those are setup, you can use this in the future when you connect it by opening it like this.

gpg --decrypt drivekey.gpg | _ cryptsetup -v luksOpen --key-file=- /dev/sdX

Now you should be able to use this key-file or the normal LUKS password. If the GPG agent showed up to work, then you only have to do this once per boot. If the power is cycled (e.g. all the hardware stolen) then everything is once again locked down.

Remember that this is putting a bonus key in slot 1 and that it is probably smart to keep the main old fashioned one in slot 0. If for some reason you need to recreate this extra key file to replace the one you’d created (e.g. you lost it, but still remember the slot 0 passphrase) you can clear a slot with luksKillSlot.

cryptsetup luksKillSlot /dev/sdX 1

Then just recreate it and luksAddKey as needed.

My Normal LUKS Backup Process

How to back up to LUKS usbflash
# sudo cryptsetup --verbose luksOpen /dev/sdX ravenbackup
# sudo mount /dev/mapper/ravenbackup /mnt
# rsync  -aP --del 192.168.0.22:/home/xed/ /mnt/xed/
# sudo rsync --verbose  -aP --del xed@192.168.0.22:/home/xed/ /mnt/xed/
# sudo umount /mnt
# sudo cryptsetup luksClose ravenbackup

Here’s the new way I’m trying with the slot 0 passphrase and then the slot 1 key encrypted with symmetric gpg. This prepares a new drive and will erase it!

lsb  # Check to make sure the drive you're about to wipe is sdX!
_ cryptsetup -y -v --type=luks luksFormat /dev/sdX
_ cryptsetup luksOpen /dev/sdX xed_crypt
_ mkfs.ext4 -v -L XedBakLUKS /dev/mapper/xed_crypt
_ mount /dev/mapper/xed_crypt /mnt/backup
_ chown xed:xed /mnt/backup
rap 192.168.1.22:/intSSD/ravenhome /mnt/backup/
cd backup/
ln -s ravenhome/xed/xfile/skelx/xeddrivekey.gpg
gpg --decrypt xeddrivekey.gpg > INSECURExeddrivekey
_ cryptsetup luksAddKey /dev/sdX  --new-keyfile INSECURExeddrivekey --new-key-slot 1
rm INSECURExeddrivekey
_ cryptsetup luksClose xed_crypt
sync

TrueCrypt

UPDATE: I received this interesting email which everyone thinking about TrueCrypt should nota bene.

I noticed you’ve shared TrueCrypt on this page http://xed.ch/help/crypto.html, as you may be aware, development of TrueCrypt was discontinued back in 2014 and has subsequently not been maintained. A number of security flaws have been uncovered and as a result we are reaching out to people to highlight a list of alternatives.

They seem to recommend LUKS and VeraCrypt and some proprietary things (which I obviously do not recommend). I’d stick with LUKS. Go ahead and "upgrade your OS" if need be; I think that was the phrase old browsers used to constantly tell me.

Old Notes

I have used TrueCrypt and it seems to work. However, there are currently (2013) some things about it that I find less than ideal. Although there is a perfectly good command line interface, most documentation does seem to focus more on GUI use. This seems like a lot of overhead to me. Most unnerving to me is the fact that "TrueCrypt is not currently available in the repositories for various reasons (possibly licensing)." according to some official looking Ubuntu documentation. I was, however able to find source code on its website which was possible to compile.

Compiling TrueCrypt From Source

As a serious crypto program, some people might want to have a quick look over the source code for obvious Eve badness. I had to resolve a lot of problems and errors and over all, I wasn’t too impressed with the compiling process.

First you need to have g++ and nasm installed. And maybe these wouldn’t hurt your chances either: libpkcs11-helper1 libpkcs11-helper1-dev libssl-dev libssl-doc zlib1g-dev libfuse-dev libselinux1-dev libsepol1-dev libwxbase2.8-0 libwxbase2.8-dev wx2.8-headers

You also may need to fix some minor problems with the source code. I got an error saying that CKR_NEW_PIN_MODE and CKR_NEXT_OTP were not defined (by /usr/include/pkcs11-helper-1.0/pkcs11.h). This doesn’t look like it’s functionally too important - just error messages, so I opened Common/SecurityToken.cpp and commented them out (lines 660 and 661).

I also had to adjust the Makefile by adding a library path (libdl.so.2) to the following line:

export LIBS="$(BASE_DIR)/$$DIR/$$PROJ.a $$LIBS /lib/x86_64-linux-gnu/libdl.so.2"; \

After all that, it was able to successfully compile with the following command:

PKCS11_INC=/usr/include/pkcs11-helper-1.0 make  NOGUI=1

Note the NOGUI=1 which only seems reasonable to me for a tool like this, but if you really love the complexity of a GUI, then have fun with the other tweaks that undoubtedly will be required.

The compiled executable lives in ./Main/truecrypt.

Setup An Encrypted Volume

To set up an encrypted volume just do something like this:

$ ./truecrypt --text --create --quick
Volume type:
 1) Normal
 2) Hidden
Select [1]: 1

Enter volume path: /dev/sdX

Encryption algorithm:
 1) AES
 2) Serpent
 3) Twofish
 4) AES-Twofish
 5) AES-Twofish-Serpent
 6) Serpent-AES
 7) Serpent-Twofish-AES
 8) Twofish-Serpent
Select [1]: 1

Hash algorithm:
 1) RIPEMD-160
 2) SHA-512
 3) Whirlpool
Select [1]: 1

Filesystem:
 1) None
 2) FAT
 3) Linux Ext2
 4) Linux Ext3
 5) Linux Ext4
Select [2]: 4

Enter password:
Re-enter password:

Enter keyfile path [none]:

Please type at least 320 randomly chosen characters and then press Enter:
Characters remaining: 274
Characters remaining: 204
Characters remaining: 150
Characters remaining: 71

Note that I just copied and pasted a block of text here since typing that is kind of onerous. However, don’t go crazy. Truecrypt doesn’t bother clearing the buffer so that when your program ends, you’ll get a ton of garbage on the command line.

If you don’t use the --quick option it will take 6 or 7 hours to set up a 1TB USB drive. With the --quick option it will say that it’s 100% done and proceed to seem like it’s hanging. What it’s really doing is a mkfs.ext3 /dev/mapper/truecrypt1. And that can take quite a while too.

Mounting

To mount a TrueCrypt volume, I just used the interactive mount command which went like this:

$ ./Main/truecrypt --mount
Enter volume path: /dev/sdX
Enter mount directory [default]: /mnt/tc
Enter password for /dev/sdX:              <- Password entered
Enter keyfile [none]:                      <- Just enter.
Protect hidden volume (if any)? (y=Yes/n=No) [No]: <- Just enter.
$ df -h /mnt/tc
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/truecrypt1  917G   41G  830G   5% /mnt/tc

Unmounting

To unmount do this kind of thing:

$ ./truecrypt --dismount /dev/sdX

tcplay

A possible alternative to TrueCrypt is tcplay. This is a completely separate implementation of the same kind of thing as TrueCrypt. I can’t quite tell if volumes created by one are compatible with the other.

I was able to install it with a simple sudo apt-get install tcplay.

Benchmarks

LUKS/cryptsetup

mkfs.ext3  617s
/dev/mapper/backupLUKS  917G  200M  871G   1% /mnt/backup
rsync     2677s
/dev/mapper/backupLUKS  917G   34G  837G   4% /mnt/backup

TrueCrypt

I couldn’t easily time the setup for TrueCrypt because of all the interactive (type 320 characters, etc) hand waving. But whatever it was, it was no worse than LUKS. If you don’t specify the --quick option, you’re looking at almost two orders of magnitude more.

/dev/mapper/truecrypt1  917G  200M  871G   1% /mnt/backup
rsync     2532s
/dev/mapper/truecrypt1  917G   34G  837G   4% /mnt/backup