PyAEScrypt
pyAEScrypt demonstrates reading the fields of an *.aes file encrypted with Python's AES Encrypt. The Python class uses AES to encrypt a file with some metadata. The file data is MAC'd so it uses some form of authenticated encryption. The metadata is not MAC'd so it is subject to tampering.
pyAEScrypt requires at least three keys. One is used for encrypting the content encryption key and initialization vector. Two are used for HMACs. The keys are derived from the password. It is not clear if the keys are independent at the moment.
The sample file has several shortcomings. First, it does not actually decrypt the AES key, AES IV, or the message. That's because the specification at AES File Format does not say how to do it. Second, the specification does not say how to derive a keystream from the password. Third, the checksum - described as file size modulo 16 - does not work as expected.
In general, you should avoid this format if possible. Instead, use an Integrated Encryption Scheme like Elliptic Curve Integrated Encryption Scheme (ECIES) or Discrete Logarithm Integrated Encryption Scheme (DLIES).
Sample Program
There are two programs below. The first parses the metadata in the file. The second isolates the encrypted stream so it can be decrypted using a pipeline.
Metadata
The program below parses the metadata from the test file named hello_world.txt. The file is embedded as a std::string using the variable pyAEScryptString.
The program has the downside that if the encrypted message is large, then you probably don't want to allocate memory for it. You probably want to decrypt directly from storage. The second sample shows you how to skip the metadata and pipeline just the encrypted message bytes.
#include "cryptlib.h"
#include "files.h"
#include "modes.h"
#include "misc.h"
#include "aes.h"
#include "sha.h"
#include "hex.h"
#include <iostream>
#include <string>
// Forward declaration. The string is below.
extern std::string pyAEScryptString;
int main(int argc, char* argv[])
{
using namespace CryptoPP;
try
{
// To output hex data to stdout
HexEncoder encoder(new FileSink(std::cout));
// Hex decoded pyAEScrypt sample data
StringSource decoded(pyAEScryptString, true /*pumpAll*/, new HexDecoder);
// Used for the checksum
size_t fileSize = decoded.MaxRetrievable();
////////////////////////////////////////
std::string magic(3u, '\0');
size_t len;
len = decoded.Get((byte*)&magic[0], 3);
if (len != 3 || magic != "AES")
throw Exception(Exception::OTHER_ERROR, "Wrong magic value");
std::cout << "magic: " << magic << std::endl;
////////////////////////////////////////
byte version;
len = decoded.Get((byte*)&version, 1);
if (len != 1 || version != 2)
throw Exception(Exception::OTHER_ERROR, "Wrong version number");
std::cout << "version: " << (int)version << std::endl;
////////////////////////////////////////
byte reserved;
len = decoded.Get((byte*)&reserved, 1);
if (len != 1 || reserved != 0x00)
throw Exception(Exception::OTHER_ERROR, "Wrong reserved value");
std::cout << "reserved: " << (int)reserved << std::endl;
////////////////////////////////////////
word16 extensionLength;
do
{
len = decoded.GetWord16(extensionLength, BIG_ENDIAN_ORDER);
if (len != 2)
throw Exception(Exception::OTHER_ERROR, "Bad extension length");
if (extensionLength == 0) { break; }
std::string extensionValue;
extensionValue.resize(extensionLength);
len = decoded.Get((byte*)&extensionValue[0], extensionLength);
if (len != extensionLength)
throw Exception(Exception::OTHER_ERROR, "Bad extension data length");
std::cout << "extensionValue " << "(" << extensionLength << "): ";
std::cout << extensionValue << std::endl;
}
while (extensionLength != 0x00);
////////////////////////////////////////
SecByteBlock staticIV(16);
len = decoded.Get((byte*)staticIV, staticIV.size());
if (len != staticIV.size())
throw Exception(Exception::OTHER_ERROR, "Bad static IV length");
std::cout << "static IV: ";
encoder.Put(staticIV, staticIV.size());
encoder.MessageEnd();
std::cout << std::endl;
////////////////////////////////////////
SecByteBlock encryptedIV(16);
len = decoded.Get((byte*)encryptedIV, encryptedIV.size());
if (len != encryptedIV.size())
throw Exception(Exception::OTHER_ERROR, "Bad encrypted IV length");
std::cout << "encrypted IV: ";
encoder.Put(encryptedIV, encryptedIV.size());
encoder.MessageEnd();
std::cout << std::endl;
////////////////////////////////////////
SecByteBlock encryptedKey(32);
len = decoded.Get((byte*)encryptedKey, encryptedKey.size());
if (len != encryptedKey.size())
throw Exception(Exception::OTHER_ERROR, "Bad encrypted key length");
std::cout << "encrypted Key: ";
encoder.Put(encryptedKey, encryptedKey.size());
encoder.MessageEnd();
std::cout << std::endl;
////////////////////////////////////////
SecByteBlock keyAndIvMac(32);
len = decoded.Get((byte*)keyAndIvMac, keyAndIvMac.size());
if (len != keyAndIvMac.size())
throw Exception(Exception::OTHER_ERROR, "Bad key and IV HMAC length");
std::cout << "key and IV HMAC: ";
encoder.Put(keyAndIvMac, keyAndIvMac.size());
encoder.MessageEnd();
std::cout << std::endl;
////////////////////////////////////////
// 1: file size mod 16
// 32: message HMAC
size_t encryptedSize = SaturatingSubtract(decoded.MaxRetrievable(), 33u);
if (encryptedSize == 0)
throw Exception(Exception::OTHER_ERROR, "Bad encrypted message size");
SecByteBlock encryptedMessage(encryptedSize);
len = decoded.Get((byte*)encryptedMessage, encryptedMessage.size());
if (len != encryptedMessage.size())
throw Exception(Exception::OTHER_ERROR, "Bad encrypted message length");
std::cout << "encrypted Message: ";
encoder.Put(encryptedMessage, encryptedMessage.size());
encoder.MessageEnd();
std::cout << std::endl;
////////////////////////////////////////
byte check;
len = decoded.Get((byte*)&check, 1);
if (len != 1)
throw Exception(Exception::OTHER_ERROR, "Bad check data");
// This does not work as expected... And its not due to CR/LF...
// if (check != (byte)(fileSize % 16))
// throw Exception(Exception::OTHER_ERROR, "Bad check value");
std::cout << "check: " << (int)check << std::endl;
////////////////////////////////////////
SecByteBlock messageMac(32);
len = decoded.Get((byte*)messageMac, messageMac.size());
if (len != messageMac.size())
throw Exception(Exception::OTHER_ERROR, "Bad message HMAC length");
std::cout << "message HMAC: ";
encoder.Put(messageMac, messageMac.size());
encoder.MessageEnd();
std::cout << std::endl;
// TODO... The docs don't appear to say how to do these things
// 1. derive keystream from password
// 2. verify MAC on key and IV
// 3. verify MAC on message
// 4. decrypt key and iv
// 5. decrypt message
}
catch (const Exception& ex)
{
std::cerr << ex.what() << std::endl;
exit (1);
}
return 0;
}
// This value is taken from https://www.aescrypt.com/aes_file_format.html
// It is hello_world.txt, processed with 'cut -b 12-59 hello_world.txt'.
std::string pyAEScryptString =
"41 45 53 02 00 00 18 43 52 45 41 54 45 44 5F 42"
"59 00 61 65 73 63 72 79 70 74 20 33 2E 30 35 00"
"80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
"00 00 00 DB 66 A1 F9 73 4F 97 B5 6C 0D 1D CA 0C"
"AA 13 C9 4F 01 8C 9E 33 15 E3 04 87 1F D2 59 DA"
"97 14 A4 6E 66 D5 8F F7 AF 05 44 92 D8 21 D8 82"
"6B C2 7E BC 13 D6 F7 60 7A 3B 3B 0B DE 60 A3 A4"
"39 66 34 21 65 70 97 3B A6 49 60 FD 70 E0 5D FA"
"71 B6 61 78 49 DE 7D 38 69 B4 6D 1D 9C 84 A2 57"
"77 80 61 66 43 87 15 98 AF 07 A6 AB EA A6 FF 82"
"89 2F 6F 0E 24 0C 7E 6F 35 F3 46 73 6A 9E 5A CA"
"35 F7 97 93 CF 5A 79 74 77 5A 3D 50 0A DD 29 4E"
"17 F4 F6 8C";
Running the program results in the following output.
$ ./pyAEScrypt.exe magic: AES version: 2 reserved: 0 extensionValue (24): CREATED_BYaescrypt 3.05 extensionValue (128): static IV: DB66A1F9734F97B56C0D1DCA0CAA13C9 encrypted IV: 4F018C9E3315E304871FD259DA9714A4 encrypted Key: 6E66D58FF7AF054492D821D8826BC27EBC13D6F7607A3B3B0BDE60A3A4396634 key and IV HMAC: 216570973BA64960FD70E05DFA71B6617849DE7D3869B46D1D9C84A257778061 encrypted Message: 6643871598AF07A6ABEAA6FF82892F6F check: 14 message HMAC: 240C7E6F35F346736A9E5ACA35F79793CF5A7974775A3D500ADD294E17F4F68C
File data
The first example parsed the metadata and copied the encrypted message into memory. For large files you probably want to decrypt directly from storage. This example isolates the encrypted file data so you can decrypt it using a pipeline.
Downloads
PyAEScrypt.zip - PyAEScrypt.cxx source file to parse metadata.