These notes mostly cover symmetric cryptography where a single secret key is used and known by all participants (maybe you and your future self, etc.). For an excellent introduction to what "asymmetric cryptography" is check out this very helpful video.
Encrypting Files
Vim
For Vim users, the simplest way to encrypt something is to open it in
Vim and use :X
and enter a pass phrase. Save the file with :w
as
normal and the file will be encrypted:
$ file vimsecret
vimsecret: Vim encrypted file data
If you’re interested in a strong encryption, use this.
:setlocal cryptmethod=blowfish
To always use this, put this in your ~/.vimrc
.
set cryptmethod=blowfish
The default cryptmethod
is zip
which is compatible with Pkzip’s
encryption method. Note that this encryption method is breakable by a
mildly motivated attacker. If it’s really important, use blowfish
.
To remove any Vim encryption, use set key=
and then :w
the file.
See :help encryption
for other exciting details.
To for other tips on using Vim securely, see my Vim notes.
GPG/PGP
PGP, or "Pretty Good Privacy" is the classic piece of software which applies strong encryption to anything that needs it with a particular emphasis on email. Don’t let the name fool you, the privacy is more than pretty good, it is superlative. There were patent and export regulation issues surrounding the original PGP, so the good folks at GNU went ahead and resynthesized the functionality from scratch and created GPG as a drop in clone unencumbered by any legal nonsense. GPG stands for "Gnu Privacy Guard". The GPG website is here.
For email applications, see my specific notes on PGP/GPG
To encrypt a file without using keys or any of the trust
infrastructure GPG is capable of, use the --symmetric
flag (the
short option -c
does the same thing).
GPG can handle many fancy cryptographic protocols. It’s default is
probably fine for all normal people doing normal things, but if you
want some real tin foil hat excitement, it can’t hurt to specify that
GPG use AES 256 bit encryption. To do that, simply include
--cipher-algo AES256
as an option.
To decrypt, just run gpg
with the encrypted filename as the
argument. It will know what to do.
$ cowsay "GPG is good." > myreadablefile
$ file myreadablefile
myreadablefile: ASCII text
$ gpg --output mysecretfile --cipher-algo AES256 --symmetric myreadablefile
gpg: gpg-agent is not available in this session
Enter passphrase:
Repeat passphrase:
$ file myreadablefile
myreadablefile: ASCII text
$ file mysecretfile
mysecretfile: DOS executable (COM)
$ rm myreadablefile
$ gpg mysecretfile
gpg: AES256 encrypted data
gpg: gpg-agent is not available in this session
gpg: encrypted with 1 passphrase
gpg: mysecretfile: unknown suffix
Enter new filename [myreadablefile]:
:-> [crow][~/testy]$ cat myreadablefile
______________
< GPG is good. >
--------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Note that "DOS executable (COM)" just means that the system has no idea what this file is. That’s good.
This also works using pipes.
$ cat myreadablefile | gpg -c -o - --cipher-algo AES256 - | cat > mysecretfile
$ file mysecretfile
mysecretfile: DOS executable (COM)
$ gpg -quiet --no-use-agent -o - mysecretfile2
______________
< GPG is good. >
--------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
ccrypt
mcrypt
On CentOS and presumably Red Hat, the easiest file encryption utility
to get working is mcrypt. Apparently
mcrypt
uses a symmetric encryption algorithm for both encryption and
decryption.
Here is the normal operation.
$ file myfile
myfile: ASCII text
$ mcrypt myfile
Enter the passphrase (maximum of 512 characters)
Please use a combination of upper and lower case letters and numbers.
Enter passphrase:
Enter passphrase:
File myfile was encrypted.
$ file myfile myfile.nc
myfile: ASCII text
myfile.nc: mcrypt 2.5 encrypted data, algorithm: rijndael-128, keysize: 32 bytes, mode: cbc,
This will produce a file named the same as the original but with a new
extra .nc
extension (I don’t know what the "n" is for). Note also
that while it asks for some password minimum assurances, if you know
what you’re doing and want a simple password, it works fine.
Note
|
Notice that the plain text file is also still there and in plain
text. The default behavior of mcrypt is to make a new encrypted
file. This is ideal for sending off to the archive while the real
thing continues to do something useful, but very bad if you think
you’ve protected something and you leave the original lying around. |
Use mcrypt -d filename
to decrypt.
Here is a way to use streams to get pretty much the same effect.
$ cat myfile | mcrypt - > myfile.nc
Enter the passphrase (maximum of 512 characters)
Please use a combination of upper and lower case letters and numbers.
Enter passphrase:
Enter passphrase:
Stdin was encrypted.
$ file myfile myfile.nc
myfile: ASCII text
myfile.nc: mcrypt 2.5 encrypted data, algorithm: rijndael-128, keysize: 32 bytes, mode: cbc,
With this technique you can run it through compression and/or
archiving or whatever and take complete control of the resulting name
or destination. There are options to the mcrypt
program executable
that can do much of this. Here is an example of this being very
useful. In this case, it is difficult to get the mcrypt executable on
the target machine. Here was the solution.
read -p "Enter pass:" X; KF=keyfile.$RANDOM; \
touch $KF; chmod 600 $KF; echo $X > $KF; \
ssh ldap.example.com sudo slapcat -v -f /etc/ldap/slapd.conf |\
mcrypt -f $KF - > \
/mnt/usbext/ldap.example.com.backup-$(date +%Y%m%d).ldif.mcrypt ; rm $KF
The trick here is to use keys with empty pass phrases or, as shown
here, use the -f
option to specify a file which contains the pass
phrase. This prevents both programs from asking stdin for user input.
OpenSSL
While I prefer the interface of something like mcrypt, it’s hardly
universal. OpenSSL, however, is and it is able to encrypt arbitrary
things including files. While OpenSSL is impressively capable of many
interesting things, the interface and documentation are utterly
terrible. For example, there is no program enc
on my system, but
man enc
unintuitively gives
a man page on
openssl enc
. This is probably the man page to look at for doing
general purpose encryption. And since the interface is so bad (long
opts with a single dash, a cryptic openssl enc -help
, etc.) the man
page is pretty essential.
An important thing to note is that if you don’t specify a cipher
OpenSSL will just "encrypt" with plain text, i.e. no cipher really. In
other words, do not rely on a sensible default cipher. The
aforementioned openssl enc -help
command produces a list of
supported ciphers. So too does openssl list-cipher-algorithms
.
Purposefully choosing one of these from the dozens on offer is not a
minor task. The format of these things seems to be
"cipher-key_length-mode_of_operation". This mode of operation stuff is
discussed adequately on
this
Wikipedia page.
Based on
advice from
the NIST Computer Security Division, I’d say that in 2016, AES
(Advanced Encryption Standard) and triple AES are the only recommended
choices. Wikipedia has full
information on
the alternatives. Until I find contradicting information, I’d say
that -aes-256-cbc
is the best for general purpose tasks.
Here is a reasonable approach. These are equivalent.
$ openssl enc -aes-256-cbc < secret_agenda > secret_agenda.opensslaes256cbc
$ openssl enc -aes-256-cbc -in secret_agenda -out secret_agenda.opensslaes256cbc
It will ask for a password (twice). To decrypt the cipher text, add a
-d
.
$ openssl enc -aes-256-cbc -d < secret_agenda.opensslaes256cbc
Note that on Mac OS you can get annoying incompatibility errors when decrypting a file encrypted on Linux. Used this extra message digest specifier like so.
openssl enc -d -aes-256-cbc -md sha256 -in secretfile.tar.enc -out secretfile.tar
Adding a -a
makes the encrypted output ASCII only (good for mail
attachments, etc). This can also be useful just to ascii-armor a
stream with no encryption.
Here’s another way that may just be simpler from an interface perspective. Blowfish seems pretty reasonable for normal people.
$ openssl bf < secret_agenda > secret_agenda.bf
enter bf-cbc encryption password:
Verifying - enter bf-cbc encryption password:
$ head -c 20 secret_agenda.bf | cat -A ; echo
Salted__hM-^M-^H^IM-#lM-o)^F^RM-39
$ openssl bf -d <secret_agenda.bf | head -n2
enter bf-cbc decryption password:
2016
January February March
This also works with aes256
as an alias for aes-256-cbc
just like
bf
is an alias for bf-cbc
.
Encrypted Filesystems
LUKS stands for Linux Unified Key Setup and is a way to manage encryption of hard drive or other data volumes. The way it works generally is to create the infrastructure for encryption and when you want to use the encrypted volume, you open a virtual device that can communicate with it. Opening requires the key. When you close it, the underlying volume (which could also be virtual) is just a bunch of random stuff.
Packages
If the cryptsetup
command seems missing it can simply be obtained
with sudo apt-get install cryptsetup
.
Troubleshooting
Note that the --debug
option goes far beyond the --verbose
option
and is worth using sooner than later when troubleshooting.
Create An Encrypted Volume
Start by creating a volume that is organized for LUKS and password locked with your password which you will define in this step. This command will hose anything already here, so this is a one-time only kind of operation.
sudo cryptsetup -y -v luksFormat /dev/sdX
WARNING!
========
This will overwrite data on /dev/sdX irrevocably.
Are you sure? (Type 'yes' in capital letters): YES
Enter passphrase for /dev/sdX:
Verify passphrase:
Ignoring bogus optimal-io size for data device (33553920 bytes).
Key slot 0 created.
Command successful.
You can create an encrypted volume over another virtual device. This would live in /dev/mapper when it’s open. When it’s not open, it’s just fuzz on the underlying partition or logical volume.
This can be done on top of LVM partitions as shown here.
cryptsetup --verbose --verify-passphrase luksFormat /dev/xed_vg/xed_SunBackup
Open, Use, And Close Encrypted Partition
Once you have the underlying volume configured for cyrpto, you need to open it by creating another higher level virtual device that you can actually interact with. You supply a passphrase here.
cryptsetup --verbose luksOpen /dev/sdX backupLUKS
Now you can use this volume like normal.
mkfs.ext4 -v -L XedBakLUKS /dev/mapper/backupLUKS # ONLY FOR NEW VOLUMES!!!
mount /dev/mapper/backupLUKS /mnt/B/
-- ADD SOME STUFF --
umount /mnt/B
Now you want to protect the partition from unauthorized use, close it. The paritition unnervingly disappears, but the underlying partition remains with your data encrypted.
cryptsetup --verbose luksClose xed_SunBackupCRYPT
Checking For LUKS
To see if a partition is actually something that can be opened into a
decrypted partition, use the isLuks
sub-command.
:-> $ cryptsetup isLuks /dev/sdX
:-> $ # Notice return code is happy (zero).
:-> $ cryptsetup isLuks /dev/xed_vg/xed_MonThuBackup
/dev/xed_vg/xed_MonThuBackup is not a LUKS partition
:-< $ # And return code is sad (non-zero).
To see more of the details of what is stored in the LUKS header try
the luksDump
sub-command.
_ cryptsetup luksDump /dev/sdX
Note that this wants the actual raw block device, not the LUKs volume if it has been opened.
Other Useful LUKS Tricks
Resizing a crpyto partition over LVM (yikes!). It can be done: see here.
Need to change a passphrase on a LUKS volume? This should behave sensibly.
$ cryptsetup luksChangeKey /dev/sdX
Note that you can add additional keys. You will need one of the working keys.
$ cryptsetup luksAddKey /dev/sdX
I don’t mind doing this when I feel like I need a new key because my old one is too complicated or hard to type. Better to leave that as an option too in case I get confused about what the status is.
How To Open Multiple Volumes With One Passphrase Entry
Imagine you’re making a backup of something every day to 7 drives, one for each day of the week; on Monday, you plug in Monday’s drive and start the back up script; on Tuesday, you do the same for Tuesday’s drive, and so on. The host you’re plugging these into every day is not being turned off and you’d like for it to remember the passphrase for all the drives (which will share the same one). The only thing you’re concerned about is if someone breaks in and steals everything, you want to ensure that they can’t see any of the data. This means that the key can be floating around in the live system, but you want to have it encrypted when it returns from a power cycle. How can this be done?
The strategy I’m going for involves using one of the "key slots" of LUKS. A LUKS protected volume can be opened with one of 8 (I think, 0-7 at worst, maybe 0-31 now for LUKS2) slots. This allows you to basically have 8 ways to open the drive (e.g. 8 different admins with their own passwords). In the main slot, I would just put the ordinary passphrase as normal. But then we’ll add a new slot that will be accessible with a key. To do this, start by generating the key.
dd if=/dev/urandom of=/tmp/drivekey bs=512 count=1
This will be just like typing a long random 512 character passphrase.
But since we want this to be opaque to someone who steals the system,
it itself needs to be encrypted. To do this I’ll suppress my
misgivings and used GPG. If it’s good enough for package management,
well, I guess it’s better than OpenSSL (which also can work for this).
The dd
generation can be piped right to it.
dd if=/dev/urandom bs=512 count=1 | gpg --symmetric --pinentry-mode loopback --output /tmp/drivekey.gpg
The pinentry-mode
just prevents it from trying to use some PGP keys
lying around or something like that; we want it to just prompt for a
passphrase. If all goes well, this should make a (binary) file called
/tmp/drivekey.gpg
.
Now that you have this random thing encrypted with simple symmetric
GPG, anytime you point gpg
at this it will want you decrypt the gpg
part and it will ask you for the gpg passphrase you just used — perhaps with a GUI form box — or it will dig into some agent and see
if it already knows what to do. Either way, here is a strategy that
can work. The trick is sending the decrypted key right into the LUKS
opening process. But first you need to create a normal LUKS system
that can be opened with an annoying passphrase. So format it as such
and use a master passphrase that will be like your emergency access.
_ cryptsetup -y -v luksFormat /dev/sdX # Start with normal passphrase slot.
So it turns out that it doesn’t quite work to just pipe the decrypted GPG encrypted key into cryptsetup when adding a key. I actually deeply explored this with the developers and it really doesn’t look like it’s possible. You have to decrypt the PGP encrypted file to use it as a key file when adding another slot. So given that must happen, it makes sense to do that using the encrypted volume. This way the insecurity of having the plaintext version floating around (even briefly) is lessened by the fact that it’s already on the target volume (i.e. if attackers can get at this, they’re already done).
sudo cryptsetup luksOpen /dev/sdX xed_crypt
sudo mkfs.ext4 -v -L XedBakLUKS /dev/mapper/xed_crypt # ONLY FOR NEW VOLUMES!!!
sudo mount /dev/mapper/xed_crypt /mnt/ext
Now prepare the key.
gpg --decrypt drivekey.gpg > /mnt/ext/INSECUREdrivekey
_ cryptsetup luksAddKey /dev/sdb --new-keyfile /mnt/ext/INSECUREdrivekey --new-key-slot 1
rm /mnt/ext/INSECUREdrivekey # Or leave a copy of it, or leave a GPG encrypted copy.
Now that those are setup, you can use this in the future when you connect it by opening it like this.
gpg --decrypt drivekey.gpg | _ cryptsetup -v luksOpen --key-file=- /dev/sdX
Now you should be able to use this key-file or the normal LUKS password. If the GPG agent showed up to work, then you only have to do this once per boot. If the power is cycled (e.g. all the hardware stolen) then everything is once again locked down.
Remember that this is putting a bonus key in slot 1 and that it is
probably smart to keep the main old fashioned one in slot 0. If for
some reason you need to recreate this extra key file to replace the
one you’d created (e.g. you lost it, but still remember the slot 0
passphrase) you can clear a slot with luksKillSlot
.
cryptsetup luksKillSlot /dev/sdX 1
Then just recreate it and luksAddKey
as needed.
My Normal LUKS Backup Process
# sudo cryptsetup --verbose luksOpen /dev/sdX ravenbackup
# sudo mount /dev/mapper/ravenbackup /mnt
# rsync -aP --del 192.168.0.22:/home/xed/ /mnt/xed/
# sudo rsync --verbose -aP --del xed@192.168.0.22:/home/xed/ /mnt/xed/
# sudo umount /mnt
# sudo cryptsetup luksClose ravenbackup
Here’s the new way I’m trying with the slot 0 passphrase and then the slot 1 key encrypted with symmetric gpg. This prepares a new drive and will erase it!
lsb # Check to make sure the drive you're about to wipe is sdX!
_ cryptsetup -y -v --type=luks luksFormat /dev/sdX
_ cryptsetup luksOpen /dev/sdX xed_crypt
_ mkfs.ext4 -v -L XedBakLUKS /dev/mapper/xed_crypt
_ mount /dev/mapper/xed_crypt /mnt/backup
_ chown xed:xed /mnt/backup
rap 192.168.1.22:/intSSD/ravenhome /mnt/backup/
cd backup/
ln -s ravenhome/xed/xfile/skelx/xeddrivekey.gpg
gpg --decrypt xeddrivekey.gpg > INSECURExeddrivekey
_ cryptsetup luksAddKey /dev/sdX --new-keyfile INSECURExeddrivekey --new-key-slot 1
rm INSECURExeddrivekey
_ cryptsetup luksClose xed_crypt
sync
TrueCrypt
UPDATE: I received this interesting email which everyone thinking about TrueCrypt should nota bene.
I noticed you’ve shared TrueCrypt on this page http://xed.ch/help/crypto.html, as you may be aware, development of TrueCrypt was discontinued back in 2014 and has subsequently not been maintained. A number of security flaws have been uncovered and as a result we are reaching out to people to highlight a list of alternatives.
They seem to recommend LUKS and VeraCrypt and some proprietary things (which I obviously do not recommend). I’d stick with LUKS. Go ahead and "upgrade your OS" if need be; I think that was the phrase old browsers used to constantly tell me.
Old Notes
I have used TrueCrypt and it seems to work. However, there are currently (2013) some things about it that I find less than ideal. Although there is a perfectly good command line interface, most documentation does seem to focus more on GUI use. This seems like a lot of overhead to me. Most unnerving to me is the fact that "TrueCrypt is not currently available in the repositories for various reasons (possibly licensing)." according to some official looking Ubuntu documentation. I was, however able to find source code on its website which was possible to compile.
Compiling TrueCrypt From Source
As a serious crypto program, some people might want to have a quick look over the source code for obvious Eve badness. I had to resolve a lot of problems and errors and over all, I wasn’t too impressed with the compiling process.
First you need to have g++
and nasm
installed. And maybe these
wouldn’t hurt your chances either: libpkcs11-helper1
libpkcs11-helper1-dev libssl-dev libssl-doc zlib1g-dev
libfuse-dev libselinux1-dev libsepol1-dev
libwxbase2.8-0 libwxbase2.8-dev wx2.8-headers
You also may need to fix some minor problems with the source code. I
got an error saying that CKR_NEW_PIN_MODE
and CKR_NEXT_OTP
were
not defined (by /usr/include/pkcs11-helper-1.0/pkcs11.h
). This
doesn’t look like it’s functionally too important - just error
messages, so I opened Common/SecurityToken.cpp
and commented them
out (lines 660 and 661).
I also had to adjust the Makefile by adding a library path
(libdl.so.2
) to the following line:
export LIBS="$(BASE_DIR)/$$DIR/$$PROJ.a $$LIBS /lib/x86_64-linux-gnu/libdl.so.2"; \
After all that, it was able to successfully compile with the following command:
PKCS11_INC=/usr/include/pkcs11-helper-1.0 make NOGUI=1
Note the NOGUI=1
which only seems reasonable to me for a tool like
this, but if you really love the complexity of a GUI, then have fun
with the other tweaks that undoubtedly will be required.
The compiled executable lives in ./Main/truecrypt
.
Setup An Encrypted Volume
To set up an encrypted volume just do something like this:
$ ./truecrypt --text --create --quick
Volume type:
1) Normal
2) Hidden
Select [1]: 1
Enter volume path: /dev/sdX
Encryption algorithm:
1) AES
2) Serpent
3) Twofish
4) AES-Twofish
5) AES-Twofish-Serpent
6) Serpent-AES
7) Serpent-Twofish-AES
8) Twofish-Serpent
Select [1]: 1
Hash algorithm:
1) RIPEMD-160
2) SHA-512
3) Whirlpool
Select [1]: 1
Filesystem:
1) None
2) FAT
3) Linux Ext2
4) Linux Ext3
5) Linux Ext4
Select [2]: 4
Enter password:
Re-enter password:
Enter keyfile path [none]:
Please type at least 320 randomly chosen characters and then press Enter:
Characters remaining: 274
Characters remaining: 204
Characters remaining: 150
Characters remaining: 71
Note that I just copied and pasted a block of text here since typing that is kind of onerous. However, don’t go crazy. Truecrypt doesn’t bother clearing the buffer so that when your program ends, you’ll get a ton of garbage on the command line.
If you don’t use the --quick
option it will take 6 or 7 hours to set
up a 1TB USB drive. With the --quick
option it will say that it’s
100% done and proceed to seem like it’s hanging. What it’s really
doing is a mkfs.ext3 /dev/mapper/truecrypt1
. And that can take quite
a while too.
Mounting
To mount a TrueCrypt volume, I just used the interactive mount command which went like this:
$ ./Main/truecrypt --mount
Enter volume path: /dev/sdX
Enter mount directory [default]: /mnt/tc
Enter password for /dev/sdX: <- Password entered
Enter keyfile [none]: <- Just enter.
Protect hidden volume (if any)? (y=Yes/n=No) [No]: <- Just enter.
$ df -h /mnt/tc
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/truecrypt1 917G 41G 830G 5% /mnt/tc
Unmounting
To unmount do this kind of thing:
$ ./truecrypt --dismount /dev/sdX
tcplay
A possible alternative to TrueCrypt is tcplay
. This is a completely
separate implementation of the same kind of thing as TrueCrypt. I
can’t quite tell if volumes created by one are compatible with the
other.
I was able to install it with a simple sudo apt-get install tcplay
.
Benchmarks
LUKS/cryptsetup
mkfs.ext3 617s
/dev/mapper/backupLUKS 917G 200M 871G 1% /mnt/backup
rsync 2677s
/dev/mapper/backupLUKS 917G 34G 837G 4% /mnt/backup
TrueCrypt
I couldn’t easily time the setup for TrueCrypt because of all the
interactive (type 320 characters, etc) hand waving. But whatever it was,
it was no worse than LUKS. If you don’t specify the --quick
option,
you’re looking at almost two orders of magnitude more.
/dev/mapper/truecrypt1 917G 200M 871G 1% /mnt/backup
rsync 2532s
/dev/mapper/truecrypt1 917G 34G 837G 4% /mnt/backup