In DevOps systems we've many places were credentials are needed. Often these credentials provide access to many servers or whole datacenters. So if they get compromised, this would be one of the maximum credible security accidents.

Considerations

So having a good security concept for credentials will be essential to any DevOps system. For dda-pallet we took the following decisions:

1. We use proven tooling: We're using GnuPG (https://gnupg.org/), Bouncy Castle (http://bouncycastle.org/) and clj-pgp (https://github.com/greglook/clj-pgp). This security toolchain is also used by Leinigen in order to sign code.
2. We need tools to en- and de-crypt credentials manually: We will use the REPL for this.
3. We will use credentials only in the most secure way in dda-pallet. This means
   1. unlock secret keys at the last possible point in time before usage.
   2. don't store unencrypted credentials, keys on the target system.
   3. passphrases are never stored.
   4. should review log-output for compromising outputs.

If you're searching for a more in-depth reasoning, you will find our analysis and decisions in more detail. Let's do a roundtrip for our credentials handling:

1. Lets inspect available keys

Let's have a look at the available keys (assuming you have a linux system with GnuPG installed):

 mje@jem-ide:~/$ gpg --list-keys --fingerprint

This results in:

/home/mje/.gnupg/pubring.gpg
----------------------------

pub 2048X/xxxxxxxx 2015-12-31
Key fingerprint = xxxx xxxx xxxx xxxx xxxx xxxx ...
uid Michael Jerger <michael.jerger@meissa-gmbh.de>
sub 2048x/xxxxxxx 2015-12-31

pub 2048R/AA3ACBFE 2016-09-22
Key fingerprint = EB0B 5F7C D3E4 B6EB 45E6 3E54 26E9 0AA6 AA3A CBFE
uid Michael Jerger (rsa code sign) <michael.jerger@meissa-gmbh.de>

Now we take a look at the clojure functionality with the help of a REPL (one of our modules integration test namespaces will be a good starting point: https://github.com/DomainDrivenArchitecture/dda-pallet-commons/blob/master/src/org/domaindrivenarchitecture/pallet/commons/encrypted_credentials.clj).

Let's try out the following methods:

(s/defn load-secret-keyring [encryption-config :- EncryptionConfiguration] ...)
(s/defn get-public-key      [encryption-config :- EncryptionConfiguration] ...)
(s/defn get-secret-key      [encryption-config :- EncryptionConfiguration] ...)

An EncryptionConfiguration is defined as follows (namespace is part of plumatics schema.core - https://github.com/plumatic/schema):

(def EncryptionConfiguration
  {:user-home dir/NonRootDirectory
   (s/optional-key :pallet-home) dir/NonRootDirectory
   (s/optional-key :secring-path) s/Str
   (s/optional-key :key-id) s/Str})

So let's watch the available GnuPG keys using the REPL:

(clj-pgp.keyring/list-public-keys
  (dda.pallet.commons.encrypted-credentials/load-secret-keyring
    {:user-home "/home/mje/"}))

REPL will provide the following output:

(#<PGPPublicKey org.bouncycastle.openpgp.PGPPublicKey@76b5ffbe>
 #<PGPPublicKey org.bouncycastle.openpgp.PGPPublicKey@cd97c36>
 #<PGPPublicKey org.bouncycastle.openpgp.PGPPublicKey@1a69c3d1>
 #<PGPPublicKey org.bouncycastle.openpgp.PGPPublicKey@15b6dbe3>)

Now let's get some more verbose information out of our keys:

(map clj-pgp.core/key-info
   (clj-pgp.keyring/list-public-keys
     (dda.pallet.commons.encrypted-credentials/load-secret-keyring
       {:user-home "/home/mje/"})))

The REPL now provides:

({:master-key? true,
  :algorithm :rsa-general,
  :user-ids ["Michael Jerger (rsa code sign) <michael.jerger@meissa-gmbh.de>"],
  :key-id "26e90aa6aa3acbfe",
  ...
  :fingerprint "EB0B5F7CD3E4B6EB45E63E5426E90AA6AA3ACBFE"}

 {:master-key? true,
  :algorithm :dsa,
  :user-ids ["Michael Jerger <michael.jerger@meissa-gmbh.de>"], ...
  :fingerprint "xxxxxxxxxxxxxxxxxxxxxx"})

clj-pgp needs a RSA-key, so we will use the key 26E90AA6AA3ACBFE for the next steps.

2. Lets encrypt our first credentials

As we now have keys, encrypting secrets is now quite easy. If we enter in REPL:

(dda.pallet.commons.encrypted-credentials/encrypt
  (dda.pallet.commons.encrypted-credentials/get-public-key
    {:user-home "/home/mje/"
     :key-id "26E90AA6AA3ACBFE"})
    {:account "acnt"
     :secret "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"})

It results in:

{:account "acnt",
 :secret "-----BEGIN PGP MESSAGE-----\nVersion: BCPG v1.54\n\nhQEMAybpCqaqOsv+AQgA0pRO/Ue2C9GxiY8V5MxD/A8ypt7F9TRvyheuCLNaKcSe\nxYDToCfy09h9syGKA/8b14itEg8IpQApDr0zTUFwE6LgA5W8zqej9qBkQZHsi11R\n"+
   "oYnuTd0bq5QJO46NsHZwYYQuu+zKwPA1fd7OwPHmjsmJ2VVvrjQ5SXaL3wwniezA\nAY1M2bb1yDkrYNKbzMfBlCbcz724ifm+VehJnGRR2p38XTQFnrrOa6ZfzzxL5TKo\n"+
   "TJfolxsjFYBd7r1P5GfT8TCCLe+dHVbnt7wgQ6eK9Xmnv/H/DJDrl9QJVsSA8c0C\nNreGqEMhVcmIsrOyh/ts2HnYhVI5RE7NVXrgM+0Rf9JhAUj+piyU/QnxyzyRwAIw\n"+
   "yWSgCJ1jdXFOHfk90wTksJ3Jlmvl4XCu1h/PxIl/YBXTkAxtaEItFl1+61WFWYl3\nab0nZqR/YqNidI79nMqmGgfZAIkXiyTtju3lj2uStQHiRQ==\n=db1u\n-----END PGP MESSAGE-----\n"}

3. Store encrypted credentials

Now as we've encrypted our credentials we can store them on several places:

1. As system default: PALLET_HOME/config.clj
2. As user default: ~/.pallet/config.clj
3. Or inside of any configuration.

In our example, we store credentials in the users home directory:

(defpallet
  :services
  {:aws
    {:account "acnt",
     :secret "-----BEGIN PGP MESSAGE-----....----END PGP MESSAGE-----\n"} } )

In pallet, we can get these accounts as follows:

(get-in (pallet.configure/pallet-config) [:services :aws])

Which provides:

{:account "acnt",
 :secret "-----BEGIN PGP MESSAGE-----....-----END PGP MESSAGE-----\n"}

Now we can save the encrypted output to the file ~/.pallet/config.clj

4. Using encrypted credentials

If we provide the key-id and passphrase, we are now able to decrypt credentials as follows:

(crypto/decrypt
 (crypto/get-secret-key {:user-home "/home/mje/"
                         :key-id key-id})
   aws-encrypted-credentials
   key-passphrase)

This will provide decrypted credentials in the following schema:

(def UnencryptedCredential
   {:account s/Str
   :secret (s/pred unencrypted?)})

Decrypted credentials can now be used to create for example an aws provider

(compute/instantiate-provider
  :pallet-ec2
  :identity (get-in aws-decrypted-credentials [:account])
  :credential (get-in aws-decrypted-credentials [:secret])
  :endpoint "eu-central-1"
  :subnet-ids ["xxxxxxx"])))

5. Bringing it all together

Use GnuPg encrypted credentials from meissa GmbH on Vimeo.