Details
-
Bug
-
Status: Reopened
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
This is about transparently compressing encrypted data. Compression can be optionally enabled via CryptoModuleBuilder. By default we'll use GZIP compression, but as with everything in cayenne-crypto, this is a pluggable strategy.
If compression is enabled, we should distinguish between small and large values. For small values compression only adds overhead and should be skipped. Per [1] Google recommends not compressing files smaller than 150 - 1000 bytes (this is of course for the web content). We'll use an internal threshold somewhere in this range, and then we should just do some performance tests.
Compression will happen before encryption, as it is likely to be more efficient on plaintext than randomized encrypted data [2].
We'll need to add compression flag to the "header" block of the ciphertext. To do that we'll reorganize the header to allow storing more data, and expand dynamically as needed:
- byte 0..2: "magic" number identifying the format as Cayenne-crypto encrypted sequence.
- byte 3: header length N, i.e. how many bytes the header contains, including magic number and the length indicator. N can be 0..127.
- byte 4: a bit String representing various flags, such as compression.
- byte 5..N: UTF8-encoded symbolic name of the encryption key.
[1] https://developers.google.com/speed/docs/best-practices/payload#GzipCompression
[2] http://superuser.com/questions/257782/compress-and-then-encrypt-or-vice-versa