Encrypting Files

Vim

For Vim users, the simplest way to encrypt something is to open it in Vim and use :X and enter a pass phrase. Save the file with :w as normal and the file will be encrypted:

$ file vimsecret
vimsecret: Vim encrypted file data

If you’re interested in a strong encryption, use this.

:setlocal cryptmethod=blowfish

To always use this, put this in your ~/.vimrc.

set cryptmethod=blowfish

The default cryptmethod is zip which is compatible with Pkzip’s encryption method. Note that this encryption method is breakable by a mildly motivated attacker. If it’s really important, use blowfish.

To remove any Vim encryption, use set key= and then :w the file.

See :help encryption for other exciting details.

GPG/PGP

PGP, or "Pretty Good Privacy" is the classic piece of software which applies strong encryption to anything that needs it with a particular emphasis on email. Don’t let the name fool you, the privacy is more than pretty good, it is superlative. There were patent and export regulation issues surrounding the original PGP, so the good folks at GNU went ahead and resynthesized the functionality from scratch and created GPG as a drop in clone unencumbered by any legal nonsense. GPG stands for "Gnu Privacy Guard". The GPG website is here.

For email applications, see my specific notes on PGP/GPG

To encrypt a file without using keys or any of the trust infrastructure GPG is capable of, use the --symmetric flag (the short option -c does the same thing).

GPG can handle many fancy cryptographic protocols. It’s default is probably fine for all normal people doing normal things, but if you want some real tin foil hat excitement, it can’t hurt to specify that GPG use AES 256 bit encryption. To do that, simply include --cipher-algo AES256 as an option.

To decrypt, just run gpg with the encrypted filename as the argument. It will know what to do.

$ cowsay "GPG is good." > myreadablefile
$ file myreadablefile
myreadablefile: ASCII text
$ gpg --output mysecretfile --cipher-algo AES256 --symmetric myreadablefile
gpg: gpg-agent is not available in this session
Enter passphrase:
Repeat passphrase:
$ file myreadablefile
myreadablefile: ASCII text
$ file mysecretfile
mysecretfile: DOS executable (COM)
$ rm myreadablefile
$ gpg mysecretfile
gpg: AES256 encrypted data
gpg: gpg-agent is not available in this session
gpg: encrypted with 1 passphrase
gpg: mysecretfile: unknown suffix
Enter new filename [myreadablefile]:
:-> [crow][~/testy]$ cat myreadablefile
 ______________
< GPG is good. >
 --------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Note that "DOS executable (COM)" just means that the system has no idea what this file is. That’s good.

This also works using pipes.

Using GPG with pipes
$ cat myreadablefile | gpg -c -o - --cipher-algo AES256  - | cat > mysecretfile
$ file mysecretfile
mysecretfile: DOS executable (COM)
$ gpg -quiet --no-use-agent -o - mysecretfile2
 ______________
< GPG is good. >
 --------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

ccrypt

I like ccrypt. This is a replacement of old fashioned Unix crypt. I think it can be installed easily on Macs with brew. Unlike Unix crypt, the algorithm provided by ccrypt is not symmetric, i.e., one must specify whether to encrypt or decrypt.

mcrypt

On CentOS and presumably Red Hat, the easiest file encryption utility to get working is mcrypt. Apparently mcrypt uses a symmetric encryption algorithm for both encryption and decryption.

Here is the normal operation.

$ file myfile
myfile: ASCII text
$ mcrypt myfile
Enter the passphrase (maximum of 512 characters)
Please use a combination of upper and lower case letters and numbers.
Enter passphrase:
Enter passphrase:
File myfile was encrypted.
$ file myfile myfile.nc
myfile:    ASCII text
myfile.nc: mcrypt 2.5 encrypted data, algorithm: rijndael-128, keysize: 32 bytes, mode: cbc,

This will produce a file named the same as the original but with a new extra .nc extension (I don’t know what the "n" is for). Note also that while it asks for some password minimum assurances, if you know what you’re doing and want a simple password, it works fine.

Note
Notice that the plain text file is also still there and in plain text. The default behavior of mcrypt is to make a new encrypted file. This is ideal for sending off to the archive while the real thing continues to do something useful, but very bad if you think you’ve protected something and you leave the original lying around.

Use mcrypt -d filename to decrypt.

Here is a way to use streams to get pretty much the same effect.

$ cat myfile | mcrypt - > myfile.nc
Enter the passphrase (maximum of 512 characters)
Please use a combination of upper and lower case letters and numbers.
Enter passphrase:
Enter passphrase:
Stdin was encrypted.
$ file myfile myfile.nc
myfile:    ASCII text
myfile.nc: mcrypt 2.5 encrypted data, algorithm: rijndael-128, keysize: 32 bytes, mode: cbc,

With this technique you can run it through compression and/or archiving or whatever and take complete control of the resulting name or destination. There are options to the mcrypt program executable that can do much of this. Here is an example of this being very useful. In this case, it is difficult to get the mcrypt executable on the target machine. Here was the solution.

read -p "Enter pass:" X; KF=keyfile.$RANDOM; \
touch $KF; chmod 600 $KF; echo $X > $KF; \
ssh ldap.example.com sudo slapcat -v -f /etc/ldap/slapd.conf |\
mcrypt  -f $KF - > \
/mnt/usbext/ldap.example.com.backup-$(date +%Y%m%d).ldif.mcrypt ; rm $KF

The trick here is to use keys with empty pass phrases or, as shown here, use the -f option to specify a file which contains the pass phrase. This prevents both programs from asking stdin for user input.

OpenSSL

While I prefer the interface of something like mcrypt, it’s hardly universal. OpenSSL, however, is and it is able to encrypt arbitrary things including files. While OpenSSL is impressively capable of many interesting things, the interface and documentation are utterly terrible. For example, there is no program enc on my system, but man enc unintuitively gives a man page on openssl enc. This is probably the man page to look at for doing general purpose encryption. And since the interface is so bad (long opts with a single dash, a cryptic openssl enc -help, etc.) the man page is pretty essential.

An important thing to note is that if you don’t specify a cipher OpenSSL will just "encrypt" with plain text, i.e. no cipher really. In other words, do not rely on a sensible default cipher. The aforementioned openssl enc -help command produces a list of supported ciphers. So too does openssl list-cipher-algorithms. Purposefully choosing one of these from the dozens on offer is not a minor task. The format of these things seems to be "cipher-key_length-mode_of_operation". This mode of operation stuff is discussed adequately on this Wikipedia page.

Based on advice from the NIST Computer Security Division, I’d say that in 2016, AES (Advanced Encryption Standard) and triple AES are the only recommended choices. Wikipedia has full information on the alternatives. Until I find contradicting information, I’d say that -aes-256-cbc is the best for general purpose tasks.

Here is a reasonable approach. These are equivalent.

$ openssl enc -aes-256-cbc < secret_agenda > secret_agenda.opensslaes256cbc
$ openssl enc -aes-256-cbc -in secret_agenda -out secret_agenda.opensslaes256cbc

It will ask for a password (twice). To decrypt the cipher text, add a -d.

$ openssl enc -aes-256-cbc -d < secret_agenda.opensslaes256cbc

Adding a -a makes the encrypted output ASCII only (good for mail attachments, etc). This can also be useful just to ascii-armor a stream with no encryption.

Here’s another way that may just be simpler from an interface perspective. Blowfish seems pretty reasonable for normal people.

$ openssl bf < secret_agenda > secret_agenda.bf
enter bf-cbc encryption password:
Verifying - enter bf-cbc encryption password:
$ head -c 20 secret_agenda.bf | cat -A ; echo
Salted__hM-^M-^H^IM-#lM-o)^F^RM-39
$ openssl bf -d <secret_agenda.bf | head -n2
enter bf-cbc decryption password:
                            2016
      January               February               March

This also works with aes256 as an alias for aes-256-cbc just like bf is an alias for bf-cbc.

Encrypted Filesystems

LUKS stands for Linux Unified Key Setup and is a way to manage encryption of harddrive or other data volumes. The way it works generally is to create the infrastructure for encryption and when you want to use the encrypted volume, you open a virtual device that can communicate with it. Opening requires the key. When you close it, the underlying volume (which could also be virtual) is just a bunch of random stuff.

Create An Encrypted Partition

First you need a new virtual device that can be built over another virtual device. This lives in /dev/mapper when it’s open. When it’s not open, it’s just fuzz on the underlying partition. This command will hose anything already here, so this is a one-time only kind of operation. This can be done on top of LVM partitions as shown here.

cryptsetup --verbose --verify-passphrase luksFormat /dev/xed_vg/xed_SunBackup

Open, Use, And Close Encrypted Partition

Once you have the underlying volume configured for cyrpto, you need to open it by creating another higher level virtual device that you can actually interact with. You supply a passphrase here.

cryptsetup --verbose luksOpen /dev/sdb backupLUKS

Now you can use this volume like normal.

    mkfs.ext3 -v -L XedBakLUKS /dev/mapper/backupLUKS # ONLY FOR NEW VOLUMES!!!
    mount /dev/mapper/backupLUKS /mnt/B/
      -- ADD SOME STUFF --
    umount /mnt/B

Now you want to protect the partition from unauthorized use, close it. The paritition unnervingly disappears, but the underlying partition remains with your data encrypted.

cryptsetup --verbose luksClose xed_SunBackupCRYPT

Other Useful LUKS Tricks

To see if a partition is actually something that can be opened into a decrypted partition, use the isLuks command.

    :-> [xed.ucsd.edu][~]$ cryptsetup isLuks /dev/sdb
    :-> [xed.ucsd.edu][~]$ # Notice return code is happy (true).
    :-> [xed.ucsd.edu][~]$ cryptsetup isLuks /dev/xed_vg/xed_MonThuBackup
    /dev/xed_vg/xed_MonThuBackup is not a LUKS partition
    :-< [xed.ucsd.edu][~]$ # And return code is sad (false).

Resizing a crpyto partition over LVM (yikes!). It can be done: see here.

TrueCrypt

UPDATE: I received this interesting email which everyone thinking about TrueCrypt should nota bene.

I noticed you’ve shared TrueCrypt on this page http://xed.ch/help/crypto.html, as you may be aware, development of TrueCrypt was discontinued back in 2014 and has subsequently not been maintained. A number of security flaws have been uncovered and as a result we are reaching out to people to highlight a list of alternatives.

They seem to recommend LUKS and VeraCrypt and some proprietary things (which I obviously do not recommend). I’d stick with LUKS. Go ahead and "upgrade your OS" if need be; I think that was the phase old browsers used to constantly tell me.

Old Notes

I have used TrueCrypt and it seems to work. However, there are currently (2013) some things about it that I find less than ideal. Although there is a perfectly good command line interface, most documentation does seem to focus more on GUI use. This seems like a lot of overhead to me. Most unnerving to me is the fact that "TrueCrypt is not currently available in the repositories for various reasons (possibly licensing)." according to some official looking Ubuntu documentation. I was, however able to find source code on its website which was possible to compile.

Compiling TrueCrypt From Source

As a serious crypto program, some people might want to have a quick look over the source code for obvious Eve badness. I had to resolve a lot of problems and errors and over all, I wasn’t too impressed with the compiling process.

First you need to have g++ and nasm installed. And maybe these wouldn’t hurt your chances either: libpkcs11-helper1 libpkcs11-helper1-dev libssl-dev libssl-doc zlib1g-dev libfuse-dev libselinux1-dev libsepol1-dev libwxbase2.8-0 libwxbase2.8-dev wx2.8-headers

You also may need to fix some minor problems with the source code. I got an error saying that CKR_NEW_PIN_MODE and CKR_NEXT_OTP were not defined (by /usr/include/pkcs11-helper-1.0/pkcs11.h). This doesn’t look like it’s functionally too important - just error messages, so I opened Common/SecurityToken.cpp and commented them out (lines 660 and 661).

I also had to adjust the Makefile by adding a library path (libdl.so.2) to the following line:

export LIBS="$(BASE_DIR)/$$DIR/$$PROJ.a $$LIBS /lib/x86_64-linux-gnu/libdl.so.2"; \

After all that, it was able to successfully compile with the following command:

PKCS11_INC=/usr/include/pkcs11-helper-1.0 make  NOGUI=1

Note the NOGUI=1 which only seems reasonable to me for a tool like this, but if you really love the complexity of a GUI, then have fun with the other tweaks that undoubtedly will be required.

The compiled executable lives in ./Main/truecrypt.

Setup An Encrypted Volume

To set up an encrypted volume just do something like this:

$ ./truecrypt --text --create --quick
Volume type:
 1) Normal
 2) Hidden
Select [1]: 1

Enter volume path: /dev/sdb

Encryption algorithm:
 1) AES
 2) Serpent
 3) Twofish
 4) AES-Twofish
 5) AES-Twofish-Serpent
 6) Serpent-AES
 7) Serpent-Twofish-AES
 8) Twofish-Serpent
Select [1]: 1

Hash algorithm:
 1) RIPEMD-160
 2) SHA-512
 3) Whirlpool
Select [1]: 1

Filesystem:
 1) None
 2) FAT
 3) Linux Ext2
 4) Linux Ext3
 5) Linux Ext4
Select [2]: 4

Enter password:
Re-enter password:

Enter keyfile path [none]:

Please type at least 320 randomly chosen characters and then press Enter:
Characters remaining: 274
Characters remaining: 204
Characters remaining: 150
Characters remaining: 71

Note that I just copied and pasted a block of text here since typing that is kind of onerous. However, don’t go crazy. Truecrypt doesn’t bother clearing the buffer so that when your program ends, you’ll get a ton of garbage on the command line.

If you don’t use the --quick option it will take 6 or 7 hours to set up a 1TB USB drive. With the --quick option it will say that it’s 100% done and proceed to seem like it’s hanging. What it’s really doing is a mkfs.ext3 /dev/mapper/truecrypt1. And that can take quite a while too.

Mounting

To mount a TrueCrypt volume, I just used the interactive mount command which went like this:

$ ./Main/truecrypt --mount
Enter volume path: /dev/sdb
Enter mount directory [default]: /mnt/tc
Enter password for /dev/sdb:              <- Password entered
Enter keyfile [none]:                      <- Just enter.
Protect hidden volume (if any)? (y=Yes/n=No) [No]: <- Just enter.
$ df -h /mnt/tc
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/truecrypt1  917G   41G  830G   5% /mnt/tc

Unmounting

To unmount do this kind of thing:

$ ./truecrypt --dismount /dev/sdb

tcplay

A possible alternative to TrueCrypt is tcplay. This is a completely separate implementation of the same kind of thing as TrueCrypt. I can’t quite tell if volumes created by one are compatible with the other.

I was able to install it with a simple sudo apt-get install tcplay.

Benchmarks

LUKS/cryptsetup

mkfs.ext3  617s
/dev/mapper/backupLUKS  917G  200M  871G   1% /mnt/backup
rsync     2677s
/dev/mapper/backupLUKS  917G   34G  837G   4% /mnt/backup

TrueCrypt

I couldn’t easily time the setup for TrueCrypt because of all the interactive (type 320 characters, etc) hand waving. But whatever it was, it was no worse than LUKS. If you don’t specify the --quick option, you’re looking at almost two orders of magnitude more.

/dev/mapper/truecrypt1  917G  200M  871G   1% /mnt/backup
rsync     2532s
/dev/mapper/truecrypt1  917G   34G  837G   4% /mnt/backup