Envelope Encryption using AWS

Please see my previous post on what is Envelope Encryption.

This post is about how to leverage AWS key management system to do envelope encryption. All code used in this post is available on GitHub.

You will need the following:

  1. An AWS account.
  2. aws_access_key and aws_secret_access_key either for the root aws account or an individual account which has full access to AWS.
  3. Create a master key to be used for this example.
    1. You can create a master key by logging into aws console , go to the IAM section and then choose Encryption Keys in the left hand menu.
    2. Pick any region you want and create a key.
    3. Note the ARN for the newly created key, it will look something like this “arn:aws:kms:us-west-1:324671914464:key/510f222f-fbb8-46aa-9408-a329fbb15575”

 

Steps to encrypt (EnvelopeEncryptionService.java)

  1. Using AWS API, send a request to generate data keys.
  2. AWS will return the data key both in plain text and encrypted format.
  3. Using the plain text, encrypt your message.
  4. Base64 encode the encrypted message.
  5. Save both the encrypted message and the encrypted data key.
private GenerateDataKeyResult generateDataKey() {
        GenerateDataKeyRequest generateDataKeyRequest = new GenerateDataKeyRequest();
        generateDataKeyRequest.setKeyId(clientMasterKeyId);
        generateDataKeyRequest.setKeySpec(DataKeySpec.AES_128);
        GenerateDataKeyResult dataKeyResult = awskmsClient.generateDataKey(generateDataKeyRequest);
        return dataKeyResult;
    }

    private EnvelopeEncryptedMessage encryptMessage(final String message, final GenerateDataKeyResult dataKey) throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException {
        SecretKeySpec key = new SecretKeySpec(dataKey.getPlaintext().array(), AES);
        Cipher cipher = Cipher.getInstance(AES);
        cipher.init(Cipher.ENCRYPT_MODE, key);

        byte[] enc = cipher.doFinal(message.getBytes());

        String cipherText = Base64.getEncoder().encodeToString(enc);

        EnvelopeEncryptedMessage envelope = new EnvelopeEncryptedMessage();
        envelope.setEncryptedKey(dataKey.getCiphertextBlob().array());
        envelope.setCiphertext(cipherText);
        return envelope;
    }

Steps to decrypt (EnvelopeEncryptionService.java)

  1. Extract the encrypted data key from the envelope.
  2. Use AWS api to decrypt the data key.
  3. Base64 decode the message.
  4. Use the plain text key obtained in step 2 to decrypt the message from step 3.

 

private String decrypt(final SecretKeySpec secretKeySpec, final String cipherText) throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException {
        byte[] decodeBase64src = Base64.getDecoder().decode(cipherText);
        Cipher cipher = Cipher.getInstance(AES);
        cipher.init(Cipher.DECRYPT_MODE, secretKeySpec);
        return new String(cipher.doFinal(decodeBase64src));
    }

    private SecretKeySpec decryptKey(final EnvelopeEncryptedMessage envelope) {
        ByteBuffer encryptedKey = ByteBuffer.wrap(envelope.getEncryptedKey());
        DecryptRequest decryptRequest = new DecryptRequest().withCiphertextBlob(encryptedKey);
        ByteBuffer plainTextKey = awskmsClient.decrypt(decryptRequest).getPlaintext();
        SecretKeySpec key = new SecretKeySpec(plainTextKey.array(), AES);
        return key;
    }

References:

http://docs.aws.amazon.com/kms/latest/developerguide/programming-encryption.html

https://github.com/awslabs/aws-dynamodb-encryption-java

https://github.com/uzresk/aws-examples

 

 

Envelope Encryption

What is it ?

Envelope Encryption is an approach/process used within many applications to encrypt data. Using this approach your data is protected two-fold.

How does this work ?

  • Typically there are many long term keys or master keys that is held in a key management system (KMS).
  • When you need to encrypt some message :
    • A request is sent to the KMS to generate a data key  based on one of the master keys.
    • KMS returns a data key, which usually contains both the plain text version and the encrypted version of the data key.
    • The message is encrypted using the plain text key.
    • Then both the encrypted message and the encrypted data key are packaged into a structure (sometimes called envelope) and written.
    • The plain text key is immediately removed from memory.
  • When it comes time to decrypt the message:
    • The encrypted data key is extracted from the envelope.
    • KMS is requested to decrypt the data key using the same master key as that was used to generate it.
    • Once the plain text version of the data key is obtained then the encrypted message itself is decrypted.

 

Using this approach if one wants to decrypt data, they need be authenticated with the KMS, since the master keys are only held there and never exported, and only the KMS can decrypt the data keys.

 

image

 

In the next post I will show how this can be done using AWS.

References:

https://d0.awsstatic.com/whitepapers/KMS-Cryptographic-Details.pdf

What’s So Special About Digital Envelope Encryption?

 

 

Conscious Recommendation Systems

We live in a world where recommendation systems are all around us examples of these : Amazon, Netflix, Dating Sites, your smart phone for that matter even your GPS is one.

What if these recommendation systems were programed to have some bit of consciousness.

Like for example Amazon could say “You have only bought high fructose drinks in the last month, do you want to take a look at our other selections”.

Or Netflix could say “Hey you have been watching tv straight for 2 hours without pausing, do you want to take a 5 min break walk around or blink your eyes”

This would probably piss off a lot of customers, but what if the Govt took the blame and wrote a law that all recommendation systems give alternative ideas that is good for the customer.

.

Help Me!! I am hooked on Galaxy of Heroes

A couple of weeks ago I was at an office lunch and noticed a colleague of mine staring into his Mobile Phone …. with more intensity than usual I should add (cause you know everyone now stares at their screens). This of-course made me curious and thus I got introduced to this game Star Wars Galaxy of Heroes.

iTunes

 

70K+ ratings with average of 4.5 or more which is really good. I have been playing it for some time now and it offers in app purchases so naturally I have spent about $50 already on it.

But here is the twist : I HATE IT and I HATE MYSELF FOR PLAYING IT.

 

The game is pretty much mindless, you keep leveling up your characters and you do that by doing battles and the battles itself have an option of auto play, so you could start a battle and you know go off get a coffee or beer depending on the time of day and come back to claim your points. It does NOTHING to improve your brain so to speak, no grand strategy… no out thinking your friends…especially when you are playing against someone that person is never involved !!

But it does do one thing effectively, it keeps my left brain’s processor running at 100% i.e it drowns out all the noise , like the fact that the best place I ever worked for got bought by a big corporate company and me and all my colleagues are in a state of limbo/despair/frustration… yeah… it does do a really good job of blocking that.

Anyways back to the game my brain’s cpu is running at 100% and is very happy cause it thinks I am accomplishing something , whoohoo I just leveled to 10 points lucky me!! In reality you have accomplished nothing but waste your precious time.

My book reading has tanked cause it is so much easier to pick this stupid game over a book and lets not even start on physical exercise that is a rabbit hole.

Ever stop to think everything in today’s world is conspiring to make you fat and dumb ? Play a game, binge watch a show and if you feel guilty watch a series of documentaries so you can pretend that you are learning something when in reality that stuff will never make it into your right brain to be ever useful.

It just feels that nowadays it takes a lot more effort to be good than it used to before, I miss the days of growing up when the TV transmission started at 5 PM. God I even miss electricity black outs…cut off all sorts of external noise and just let your right brain do its magic. Or simply read a good old fashioned book.

And finally My Ally Code for this game that I hate and hate myself for playing it is 961-748-229 hopefully I will be out of this phase soon.

 

 

 

 

Smart Databases : RethinkDB

I started playing with RethinkDB and it is very interesting and definitely worth a look.

Lets start with a typical example in applications for example the client table. The client table can get modified via UI or SalesForce or by hand or any number of ways. Hence any app cannot just read this table once and keep it in cache, we have to constantly pool to get changes.

This is very frustrating, cause this table rarely changes but if it does change we need to know about the change immediately, so far we have relied on polling or the system that makes the change inform other systems so that they can blow there local cache.

But with RethinkDB it changes all that.

RethinkDB is like a smart database, your client can listen for table changes and whenever that particular table changes the database itself will notify any clients that the table has changed and on top of it will tell you the old and new values.

This is almost revolutionary, (yes I know you can do this in mysql by installing the JDBC driver and a trigger that listens on the table and the trigger can inform other apps but it is way too complicated and good luck getting your ops to install it), but RethinkDB does it for you OUT OF THE BOX. And they now have an awesome JDK driver.

Just awesome sauce.

The n+1 problem, or why we can never truly have artificial life.

Man builds working hard drive in mine craft !

So someone was able to construct a hard drive in mine craft, good.

For someone to have a working hard-drive of capacity N, the game itself needs to be running on a computer which has a hard drive of capacity N+1, the +1 is for storing the actual game itself.

We can similarly say to have a working cpu of capacity N, the game itself needs to be running on a computer which has a cpu capacity of N+1, the +1 for actually running the game itself.

If we were to assume that we live in a virtual world, then in order for us to create new artificial life inside of our virtual world, we would first need to construct a computer that is at-least as big as the computer we are already in.

Can you simulate a computer that is bigger and faster than the computer on which the simulation is running ? I would say no, hence we can never simulate artificial life in its truest form.

I am sure we can and will create artificial life but will it ever match what we ourselves live in ? I don’t think that so.

Pairing Functions

Wow, I did not know about it till last week and when I read about it, I was like where has this been all my life🙂

Anyways its called Pairing Functions, which allows you to reduce 2 positive numbers into 1 and back.

Uses: Imagine you had a set of objects called Pair(long,long) and you wanted to search if a particular pair exists in that set, the usual way would be implement hashcode and equals . But with a pairing function you can reduce it to a set of longs.

In my experiments I saw an improvement of 30-40%.

For more info

http://sachiniscool.blogspot.com/2011/07/another-elegant-pairing-function.html

https://hbfs.wordpress.com/2011/09/27/pairing-functions/

http://szudzik.com/ElegantPairing.pdf

Ping

It has been so long since I posted anything here, kind of lost interest in blogging. Going to give it another try and see if it works out this time.