See AWS Kinesis Demo

Wanted to post at least 1 item before the year’s end.

Reading Books.

TL;DR Do yourself a favor and create an account on GoodReads, go the extra step of signing up for the yearly reading challenge.

From a young age I loved reading books and it was my mom who (inadvertently) got me into the habit. For her it was yet another way to keep me occupied, whenever she had some major work to be done or for that matter just wanted some peace and quiet she would buy me a bunch of books. You see we had just the one tv channel and no ipods/ipads/computers/gameboys etc that we now have to keep young people out of trouble and boy did I get into trouble.

I was a big fan of Hergé‘s creation TinTin, there were a lot of other comics and slowly I graduated to reading novels, it helped that my father was also a voracious reader and I could just borrow his books.  This was all great but it didn’t last, I noticed a steep decline in my reading over the years (Full time job, marriage, kids, the Internet, Age of Empires and World of Warcraft took a big chunk of my time).

And then came the Kindle sure there were other e-book readers before it, but nothing that beat the convenience of the Kindle and thus I slowly got back to reading, I already had a list of items I had to catch up on but once that list was exhausted where do I look up for more inspiration ? Enter GoodReads, if you don’t know what GoodReads is I highly recommend you stop reading this silly post and head on over to https://www.goodreads.com/.

GoodReads is an awesome website packed with great features like:

• Look up any book and read in depth reviews.
• Follow authors
• Keep track of what you have already read (and how many times)
• Get recommendations based on your past reading habits.
• And to top it all off few years ago they introduced a feature called the Yearly Reading Challenge.

This meant now you could actually commit to something at the beginning of the year and make sure you were on track, also everybody now knows what you have signed up for !!!! Let me tell you for procrastinators like me this was a god-sent.

Yes there was a year when I almost did not make it and crammed in couple of SciFi novels at the end of the year, but overall I am happy to report that this has been a great motivational tool to help me get back to reading.

Happy Reading!

Please see my previous post on what is Envelope Encryption.

This post is about how to leverage AWS key management system to do envelope encryption. All code used in this post is available on GitHub.

You will need the following:

1. An AWS account.
2. aws_access_key and aws_secret_access_key either for the root aws account or an individual account which has full access to AWS.
3. Create a master key to be used for this example.
   1. You can create a master key by logging into aws console , go to the IAM section and then choose Encryption Keys in the left hand menu.
   2. Pick any region you want and create a key.
   3. Note the ARN for the newly created key, it will look something like this “arn:aws:kms:us-west-1:324671914464:key/510f222f-fbb8-46aa-9408-a329fbb15575”

 

Steps to encrypt (EnvelopeEncryptionService.java)

1. Using AWS API, send a request to generate data keys.
2. AWS will return the data key both in plain text and encrypted format.
3. Using the plain text, encrypt your message.
4. Base64 encode the encrypted message.
5. Save both the encrypted message and the encrypted data key.

private GenerateDataKeyResult generateDataKey() {
        GenerateDataKeyRequest generateDataKeyRequest = new GenerateDataKeyRequest();
        generateDataKeyRequest.setKeyId(clientMasterKeyId);
        generateDataKeyRequest.setKeySpec(DataKeySpec.AES_128);
        GenerateDataKeyResult dataKeyResult = awskmsClient.generateDataKey(generateDataKeyRequest);
        return dataKeyResult;
    }

    private EnvelopeEncryptedMessage encryptMessage(final String message, final GenerateDataKeyResult dataKey) throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException {
        SecretKeySpec key = new SecretKeySpec(dataKey.getPlaintext().array(), AES);
        Cipher cipher = Cipher.getInstance(AES);
        cipher.init(Cipher.ENCRYPT_MODE, key);

        byte[] enc = cipher.doFinal(message.getBytes());

        String cipherText = Base64.getEncoder().encodeToString(enc);

        EnvelopeEncryptedMessage envelope = new EnvelopeEncryptedMessage();
        envelope.setEncryptedKey(dataKey.getCiphertextBlob().array());
        envelope.setCiphertext(cipherText);
        return envelope;
    }

Steps to decrypt (EnvelopeEncryptionService.java)

1. Extract the encrypted data key from the envelope.
2. Use AWS api to decrypt the data key.
3. Base64 decode the message.
4. Use the plain text key obtained in step 2 to decrypt the message from step 3.

 

private String decrypt(final SecretKeySpec secretKeySpec, final String cipherText) throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException {
        byte[] decodeBase64src = Base64.getDecoder().decode(cipherText);
        Cipher cipher = Cipher.getInstance(AES);
        cipher.init(Cipher.DECRYPT_MODE, secretKeySpec);
        return new String(cipher.doFinal(decodeBase64src));
    }

    private SecretKeySpec decryptKey(final EnvelopeEncryptedMessage envelope) {
        ByteBuffer encryptedKey = ByteBuffer.wrap(envelope.getEncryptedKey());
        DecryptRequest decryptRequest = new DecryptRequest().withCiphertextBlob(encryptedKey);
        ByteBuffer plainTextKey = awskmsClient.decrypt(decryptRequest).getPlaintext();
        SecretKeySpec key = new SecretKeySpec(plainTextKey.array(), AES);
        return key;
    }

References:

http://docs.aws.amazon.com/kms/latest/developerguide/programming-encryption.html

https://github.com/awslabs/aws-dynamodb-encryption-java

https://github.com/uzresk/aws-examples

 

 

What is it ?

Envelope Encryption is an approach/process used within many applications to encrypt data. Using this approach your data is protected two-fold.

How does this work ?

• Typically there are many long term keys or master keys that is held in a key management system (KMS).
• When you need to encrypt some message :
  • A request is sent to the KMS to generate a data key  based on one of the master keys.
  • KMS returns a data key, which usually contains both the plain text version and the encrypted version of the data key.
  • The message is encrypted using the plain text key.
  • Then both the encrypted message and the encrypted data key are packaged into a structure (sometimes called envelope) and written.
  • The plain text key is immediately removed from memory.
• When it comes time to decrypt the message:
  • The encrypted data key is extracted from the envelope.
  • KMS is requested to decrypt the data key using the same master key as that was used to generate it.
  • Once the plain text version of the data key is obtained then the encrypted message itself is decrypted.

 

Using this approach if one wants to decrypt data, they need be authenticated with the KMS, since the master keys are only held there and never exported, and only the KMS can decrypt the data keys.

 

 

In the next post I will show how this can be done using AWS.

References:

Click to access KMS-Cryptographic-Details.pdf

What’s So Special About Digital Envelope Encryption?

 

 

We live in a world where recommendation systems are all around us examples of these : Amazon, Netflix, Dating Sites, your smart phone for that matter even your GPS is one.

What if these recommendation systems were programed to have some bit of consciousness.

Like for example Amazon could say “You have only bought high fructose drinks in the last month, do you want to take a look at our other selections”.

Or Netflix could say “Hey you have been watching tv straight for 2 hours without pausing, do you want to take a 5 min break walk around or blink your eyes”

This would probably piss off a lot of customers, but what if the Govt took the blame and wrote a law that all recommendation systems give alternative ideas that is good for the customer.

.

A couple of weeks ago I was at an office lunch and noticed a colleague of mine staring into his Mobile Phone …. with more intensity than usual I should add (cause you know everyone now stares at their screens). This of-course made me curious and thus I got introduced to this game Star Wars Galaxy of Heroes.

 

70K+ ratings with average of 4.5 or more which is really good. I have been playing it for some time now and it offers in app purchases so naturally I have spent about $50 already on it.

But here is the twist : I HATE IT and I HATE MYSELF FOR PLAYING IT.

 

The game is pretty much mindless, you keep leveling up your characters and you do that by doing battles and the battles itself have an option of auto play, so you could start a battle and you know go off get a coffee or beer depending on the time of day and come back to claim your points. It does NOTHING to improve your brain so to speak, no grand strategy… no out thinking your friends…especially when you are playing against someone that person is never involved !!

But it does do one thing effectively, it keeps my left brain’s processor running at 100% i.e it drowns out all the noise , like the fact that the best place I ever worked for got bought by a big corporate company and me and all my colleagues are in a state of limbo/despair/frustration… yeah… it does do a really good job of blocking that.

Anyways back to the game my brain’s cpu is running at 100% and is very happy cause it thinks I am accomplishing something , whoohoo I just leveled to 10 points lucky me!! In reality you have accomplished nothing but waste your precious time.

My book reading has tanked cause it is so much easier to pick this stupid game over a book and lets not even start on physical exercise that is a rabbit hole.

…

…

Ever stop to think everything in today’s world is conspiring to make you fat and dumb ? Play a game, binge watch a show and if you feel guilty watch a series of documentaries so you can pretend that you are learning something when in reality that stuff will never make it into your right brain to be ever useful.

It just feels that nowadays it takes a lot more effort to be good than it used to before, I miss the days of growing up when the TV transmission started at 5 PM. God I even miss electricity black outs…cut off all sorts of external noise and just let your right brain do its magic. Or simply read a good old fashioned book.

And finally My Ally Code for this game that I hate and hate myself for playing it is 961-748-229 hopefully I will be out of this phase soon.

 

 

 

 

Man builds working hard drive in mine craft !

So someone was able to construct a hard drive in mine craft, good.

For someone to have a working hard-drive of capacity N, the game itself needs to be running on a computer which has a hard drive of capacity N+1, the +1 is for storing the actual game itself.

We can similarly say to have a working cpu of capacity N, the game itself needs to be running on a computer which has a cpu capacity of N+1, the +1 for actually running the game itself.

If we were to assume that we live in a virtual world, then in order for us to create new artificial life inside of our virtual world, we would first need to construct a computer that is at-least as big as the computer we are already in.

Can you simulate a computer that is bigger and faster than the computer on which the simulation is running ? I would say no, hence we can never simulate artificial life in its truest form.

I am sure we can and will create artificial life but will it ever match what we ourselves live in ? I don’t think that so.

Wow, I did not know about it till last week and when I read about it, I was like where has this been all my life 🙂

Anyways its called Pairing Functions, which allows you to reduce 2 positive numbers into 1 and back.

Uses: Imagine you had a set of objects called Pair(long,long) and you wanted to search if a particular pair exists in that set, the usual way would be implement hashcode and equals . But with a pairing function you can reduce it to a set of longs.

In my experiments I saw an improvement of 30-40%.

For more info

http://sachiniscool.blogspot.com/2011/07/another-elegant-pairing-function.html

https://hbfs.wordpress.com/2011/09/27/pairing-functions/

Click to access ElegantPairing.pdf