Before reading "Why Johnny Still Can't Encrypt" I'd read "Why Johnny Can't Encrypt". Boy it was hilarious!
In the original paper they asked 12 people to send en encrypted message to 5 people. In the process the participants had to stumble upon several traps like a need to distinguish a key algo type because 1 of the recipients used an 'old' style RSA key.
The results were funny to read:
'One of the 12 participants (P4) was unable to figure out how to encrypt at all. He kept attempting to find a way to "turn on" encryption, and at one point believed that he had done so by modifying the settings in the Preferences dialog in PGPKeys.'
'P1, P7 and P11 appeared to develop an understanding that they needed the team members' public keys, but still did not succeed at correctly encrypting email. P2 never appeared to understand what was wrong, even after twice receiving feedback that the team members could not decrypt his email.'
'(P5) so completely misunderstood the model that he generated key pairs for each team member rather than for himself, and then attempted to send the secret in an email encrypted with the five public keys he had generated. Even after receiving feedback that the team members were unable to decrypt his email, he did not manage to recover from this error.'
'P6 generated a test key pair and then revoked it, without sending either the key pair or its revocation to the key server. He appeared to think he had successfully completed the task.'
'P11 expressed great distress over not knowing whether or not she should trust the keys, and got no further in the remaining ten minutes of her test session.'
The new paper "Why Johnny Still Can't Encrypt" is uninspiring. They used a JS OpenPGP implementation (Mailvelope), avalivible as a Chrome/Firefox plugin. Before reading the sequel I'd installed the plugin to judge it by myself.
Mailvelope is fine if you understand that it operates just on a arbitual block of text; it doesn't (& cannot) 'hook' into GMail in any way except for trying to parse encoded text blocks & looking for editible DIVs. It can be confusing if you don't get that selecting the recipeint in the GMail compose window has nothing to do the encrypting: it's easy to sent a mail to firstname.lastname@example.org where you encoded the message with email@example.com PK.
In other aspects I've found Mailvelope pretty obvious.
Having 'achieved' the grandiose task of exchanging public keys between 2 emails & sending encrypting messages, I finally read the paper.
Boy it was disappointing.
In contrast w/ the original PGP study, they resorted to the most simplest possible tasks: user A should generate a key pair; ask user B for his PK; send an encrypted email. They got 20 pairs of A-B users. Only 1 pair successfully send/read a message.
The 1 pair.
This is why humanity is doomed.