Sunday, December 14, 2014

A Naive Benchmark of GnuPG 2.1 Symmetric Algorithms

Some symmetric algo benchmarks already exist, but still don't answer to a typical question for a typical setup:

I do a regular backup of N (or even K) gigabytes. I don't want the backup to be readable by a random hacker form Russia (if he breaks into my server). What algo should I use to encrypt the backup as fast as possible?

This rules out many existing benchmarks.

The typical setup also includes gpg2. I don't care about synthetic algo tests (like 'I read once that Rijndael is fast & 3DES is slow'), I'm interested in a particular implementation that runs on my machines.

(Note that benchmarks below are not 'scientific' in any way; they are meant to be useful for 1 specific operation only: encrypting binary blobs through ruby-gpeme.)

gpg2 cli program

The first thing I did was to run

$ gpg2 --batch --passphrase 12345 -o out --compress-algo none \
    --cipher-algo '<ALGO>' -c < file.tar.gz

But was quickly saddened because the results weren't consistent: the deviation between runs was too big.

What we needed here was to dissociate the crypto from the IO.

libgcrypt

'Modern' versions of GnuPG have detached a big chunk of the crypto magic into a separate low-level library libgcrypt. If we want to test symmetric ciphers w/o any additional overhead, we can write a nano version of gpg2.

It'll read some bytes from /dev/urandom, pad them (if a block cipher mode requires it), generate an IV, encrypt, prepend the IV to an encrypted text, append a MAC, run that for all libgcrypt supported ciphers. Then we can draw a pretty graph & brag about it to coworkers.

The problem is that there is no any docs (at least I haven't found them) about a general format that gpg2 uses for block ciphers. And you need it because a decipher must be able to know what algo was used, its cipher mode, where to search for a stored IV, etc.

There is OpenPGP RFC 4880 of course:

The data is encrypted in CFB mode, with a CFB shift size equal to the cipher's block size. The Initial Vector (IV) is specified as all zeros. Instead of using an IV, OpenPGP prefixes a string of length equal to the block size of the cipher plus two to the data before it is encrypted.

That's better than nothing, but still leaves us w/ n hours of struggling to write & test code that will produce an encrypted stream suitable for gpg2.

GPGME

GnuPG has an official library that even has bindings for such languages as Ruby. It's an opposite of libgcrypt: it does all the work for you, where libgcrypt doesn't even provide auto padding.

The trouble w/ gpgme is that it was unusable for automated testing purposes until GnuPG hit version 2.1 this fall.

For instance,

  • Versions 2.0.x cannot read passwords w/o pinentry.
  • At the time of writing, 2.1 isn't available on any major Linux distribution (except Arch, but I'm not using it anywhere (maybe I should)).
Writing a Benchmark

ruby-gpgme has a nifty example for symmetric ciphers:

crypto = GPGME::Crypto.new password: '12345'
r = crypto.encrypt "Hello world!\n", symmetric: true

where r.read() will return an encrypted string.

We have 2 problems here:

  1. There is absolutely no way to change through the API the symmetric cipher. (The default one is CAST5.) This isn't a fault of ruby-gpgme, but the very same gpgme library under it.

    GnuPG has a concept of a 'home' directory (it has nothing to do w/ user's home directory, it just uses it as a default). Each 'home' can have its number of configuration files. We need gpg.conf file there w/ a line:

    personal-cipher-preferences <algo>
    
  2. The modest password: '12345' option does nothing unless archaic gpg1 is used. W/ gnupg 2.0.x an annoying pinentry window will pop-up.

    E.g. installing 2.1 is the only option. Instead overwriting the existing 2.0.x installation (and possibly breaking your system), install 2.1 under a separate prefix (for example, to ~/tmp/gnupg).

    Next, for each gpg 'home' directory we need to add to gpg.conf another line:

    pinentry-mode loopback
    

    & create a gpg-agent.conf file w/ a line:

    allow-loopback-pinentry
    

The benchmark works like this:

  1. Before running any crypto operations, for each cipher we create a 'home' directory & fill it w/ custom gpg.conf & gpg-agent.conf files.
  2. Start a bunch of copies of gpg-agent, each for a different 'home' dir.
  3. Add a bin directory of our fresh gnupg 2.1 installation to the PATH, for example ~/tmp/gnupg/bin.
  4. Set LD_LIBRARY_PATH to ~/tmp/gnupg/lib.
  5. Generate 'plaint text' as n bytes from /dev/urandom.
  6. Encode 'plain text' w/ a list of all supported symmetric ciphers.
  7. Print the results.

Ruby script that does this can be cloned form https://github.com/gromnitsky/gpg-algo-speed. You'll need gpgme & benchmark-ips gems. Run the file benchmark from the cloned dir.

Results

AMD Sempron 145, Linux 3.11.7-200.fc19.x86_64

$ ./benchmark /opt/tmp/gnupg $((256*1024*1024))
Plain text size: 268,435,456B
Calculating -------------------------------------
                idea     1.000  i/100ms
                3des     1.000  i/100ms
               cast5     1.000  i/100ms
            blowfish     1.000  i/100ms
                 aes     1.000  i/100ms
              aes192     1.000  i/100ms
              aes256     1.000  i/100ms
             twofish     1.000  i/100ms
         camellia128     1.000  i/100ms
         camellia192     1.000  i/100ms
         camellia256     1.000  i/100ms
-------------------------------------------------
                idea      0.051  (± 0.0%) i/s -      1.000  in  19.443114s
                3des      0.037  (± 0.0%) i/s -      1.000  in  27.137538s
               cast5      0.059  (± 0.0%) i/s -      1.000  in  16.850647s
            blowfish      0.058  (± 0.0%) i/s -      1.000  in  17.183059s
                 aes      0.059  (± 0.0%) i/s -      1.000  in  17.080337s
              aes192      0.057  (± 0.0%) i/s -      1.000  in  17.516253s
              aes256      0.057  (± 0.0%) i/s -      1.000  in  17.673528s
             twofish      0.057  (± 0.0%) i/s -      1.000  in  17.533964s
         camellia128      0.054  (± 0.0%) i/s -      1.000  in  18.359755s
         camellia192      0.053  (± 0.0%) i/s -      1.000  in  18.712756s
         camellia256      0.054  (± 0.0%) i/s -      1.000  in  18.684303s

Comparison:
               cast5:        0.1 i/s
                 aes:        0.1 i/s - 1.01x slower
            blowfish:        0.1 i/s - 1.02x slower
              aes192:        0.1 i/s - 1.04x slower
             twofish:        0.1 i/s - 1.04x slower
              aes256:        0.1 i/s - 1.05x slower
         camellia128:        0.1 i/s - 1.09x slower
         camellia256:        0.1 i/s - 1.11x slower
         camellia192:        0.1 i/s - 1.11x slower
                idea:        0.1 i/s - 1.15x slower
                3des:        0.0 i/s - 1.61x slower

Algo         Total Iterations
       idea          2
       3des          2
      cast5          2
   blowfish          2
        aes          2
     aes192          2
     aes256          2
    twofish          2
camellia128          2
camellia192          2
camellia256          2

As we see, 3DES is indeed slower that Rijndael.

(The plot is written in Grap. It doesn't really matter but I wanted to show off that I was tinkering w/ a Bell Labs language from 1984 that nobody is using anymore.)

In the repo above there is the result for 3G blob (w/ compression turned on), where Ruby garbage collector has run amok.

1 comment: