Vault

Release 0.7.8 included the much-awaited Data Connections feature! This feature makes use of a new service called Vault (https://www.vaultproject.io/) to encrypt data connection credentials so that you can be sure they are always secure. Installing and initializing Vault requires a few extra manual steps which are documented here.

NOTE: If you ever restart Vault (either manually or by restarting a node), you will need to follow the steps in Unsealing Vault to unseal vault.

First installation

  1. Make sure you have jq installed

    1. For mac: brew install jq

    2. For linux: apt-get install jq

  2. Open up the KOTS admin panel

    kubectl kots admin-console --namespace hex

  3. Check for updates, and deploy the new version 0.7.8

  4. Wait until consul starts up

    1. Keep running kubectl get -n hex pods until you see 3 hex-consul-server-N 's in the Running state

  5. Run ./init-vault.sh -n hex -r hex --init once in order to initially set up vault

    1. After running, it will display a blob of json output. SAVE THIS JSON SOMEWHERE SECURE. You will need this to unseal vault any time it restarts. (last line in the below blob)

    2. A good place to store this is in AWS Secrets Manager

    ...
    --- -----
    Seal Type shamir
    Initialized true
    Sealed false
    Total Shares 5
    Threshold 3
    Version 1.4.2
    Cluster Name vault-cluster-84b72044
    Cluster ID b5a55915-c4fe-eeff-972b-84a7d7471297
    HA Enabled true
    HA Cluster <https://hex-vault-0.hex-vault-internal:8201>
    HA Mode standby
    Active Node Address <http://10.0.20.101:8200>
    Vaault unsealed!
    Success! You are now authenticated. The token information displayed below
    is already stored in the token helper. You do NOT need to run "vault login"
    again. Future Vault requests will automatically use this token.
    Key Value
    --- -----
    token s.3p9VCD3dJ1juB95BLu0expKU
    token_accessor EjmGo7YdCGV3EqsEoiMTGQkj
    token_duration ∞
    token_renewable false
    token_policies ["root"]
    identity_policies []
    policies ["root"]
    Success! Enabled kubernetes auth method at: kubernetes/
    Success! Data written to: auth/kubernetes/config
    Success! Uploaded policy: hex
    Success! Data written to: auth/kubernetes/role/hex
    Success! Enabled the kv secrets engine at: secret/
    { "unseal_keys_b64": [ "/geRTGCyhSGVxmYKePxbjzh7SljU9mxjcKFVxFUfyIeu", "u6zTOkQ7ABCE4nuJaWCN+nghqlEC4e6tspdzkWGBM5pk", "ddJ0c/BxxF/5/iTYk08BorcuFBY3SGlByOFyb5CAxwma", "+F1qgw4OtvHSB+AK762/zKzFyzHF48+2WrebZ/6vw5T1", "BLxQ7xbO0vuiTIB+Bv0hDYqZz5fGDj+WujhudaoISMse" ], "unseal_keys_hex": [ "fe07914c60b2852195c6660a78fc5b8f387b4a58d4f66c6370a155c4551fc887ae", "bbacd33a443b001084e27b8969608dfa7821aa5102e1eeadb29773916181339a64", "75d27473f071c45ff9fe24d8934f01a2b72e141637486941c8e1726f9080c7099a", "f85d6a830e0eb6f1d207e00aefadbfccacc5cb31c5e3cfb65ab79b67feafc394f5", "04bc50ef16ced2fba24c807e06fd210d8a99cf97c60e3f96ba386e75aa0848cb1e" ], "unseal_shares": 5, "unseal_threshold": 3, "recovery_keys_b64": [], "recovery_keys_hex": [], "recovery_keys_shares": 5, "recovery_keys_threshold": 3, "root_token": "s.3p9VCD3dJ1juB95BLu0expKU" }
  6. After running this script, you should see all the vault pods are in the "Running" state as well, and should now be able to use Data Connections

    1. Run kubectl get -n hex pods and check that the 3 hex-vault-N pods are all in Running

Unsealing Vault after a restart

This step needs to happen after any time that a Vault pod is restarted

  1. Navigate back to the init-vault.sh script you downloaded in the initial steps

  2. Get 3 unseal keys from wherever you stored the JSON blob from above

    1. Get 3 keys from the section titled unseal_keys_b64

  3. Run the init-vault.sh script with the 3 unseal keys

    1. ./init-vault.sh -n hex -r hex "/geRTGCyhSGVxmYKePxbjzh7SljU9mxjcKFVxFUfyIeu" "u6zTOkQ7ABCE4nuJaWCN+nghqlEC4e6tspdzkWGBM5pk" "ddJ0c/BxxF/5/iTYk08BorcuFBY3SGlByOFyb5CAxwma"