Skip to main content
  1. Addigy
  2. Advanced Topics
  3. Advanced Topics

Using Vault with Addigy

Updated:

TABLE OF CONTENTS

Overview

Vault is an open-source secrets manager maintained by HashiCorp. Secrets stored in Vault are kept encrypted and are accessed as a virtual file system, like secrets/login-creds. Vault tightly controls access to secrets such as tokens, passwords, certificates, and encryption keys, which can be accessed using a GUI, CLI, or HTTP API.

Permissions to access secrets are "deny by default," meaning that if a machine or user doesn't have explicit permission to create, read, update, or delete a secret, they can't do so. Permissions are defined in policies in HCL format. Vault also offers audit logging: secrets that are accessed are logged to track who accessed what.

Authentication in Vault can be done via passwords or by generating temporary tokens. Additional authentication methods integrate with other platforms such as AWS, Azure, Google Cloud, and GitHub.

Installation

macOS:

Download the Vault binary from the official HashiCorp site.

Unzip the binary into the /usr/local/bin folder.

Verify that Vault works by running:

vault -v

Ubuntu 20.04:

Download the Vault binary for Linux (this command downloads version 1.6.1 — check the HashiCorp site for the latest version):

wget https://releases.hashicorp.com/vault/1.6.1/vault_1.6.1_linux_amd64.zip

Unzip the file:

unzip vault_1.6.1_linux_amd64.zip

Move the vault binary to /usr/local/bin:

sudo mv vault /usr/local/bin/vault

Verify that Vault works:

vault -v

Deploying with Addigy to Macs:

To deploy the binary with Addigy to a client Mac, add the binary as Custom Software and enter the following as the installation script:

DEST=/usr/local/bin/vault
SOURCE='/Library/Addigy/ansible/packages/Vault (1.6.1)/vault'
if [ -f "$DEST" ]; then

    rm -f "$DEST"

fi

echo 'Installing Vault to bin'
cp "$SOURCE" "$DEST"
chmod +x "$DEST"
echo 'Cleaning up'
rm -rf '/Library/Addigy/ansible/packages/Vault (1.6.1)/'

Note: This script assumes Vault 1.6.1 is being deployed — be sure to change the paths if deploying a different version, like 1.7.0.

Set up Vault

To start a production-ready Vault server with TLS, go to the Start a Vault Server in Ubuntu 20.04 with TLS section.

Start a Dev Vault Server without TLS

Create the directory where you'll keep Vault's data:

mkdir -p vault/data

Configurations and policies are defined in HashiCorp Configuration Language (HCL), also used in Terraform. Policies can also be defined in JSON format.

Create a file named config.hcl with the following server configuration:

ui = true
disable_mlock = true
storage "raft" {
  path = "./vault/data"
  node_id = "node1"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = 1
}
api_addr = "http://127.0.0.1:8200"
cluster_addr = "https://127.0.0.1:8201"

This creates a server with the raft storage backend and the listener address at localhost.

Note: A production-ready environment would have TLS enabled.

Now run the server with the config.hcl file:

vault server -config=config.hcl

For more information, see the Vault configuration docs and the server command docs.

Start a Vault Server in Ubuntu 20.04 with TLS

Create a TLS Certificate with Certbot

Be sure your Ubuntu server allows TCP traffic through port 8200 and HTTP traffic through port 80.

You'll also need a DNS A record with a domain pointing to the IP of the machine — for example, vault.addigy.io pointing to the machine at 184.72.89.72.

Now install Certbot, which will provide the TLS certificate.

Ensure snapd is up to date:

sudo snap install core; sudo snap refresh core

Install Certbot:

sudo snap install --classic certbot

Run certbot:

sudo certbot certonly --standalone -d your.url.here

Note: Your domain name will be different — this example uses vault.addigy.io.

If all goes well, this creates a certificate that expires in 3 months.

This certificate is automatically renewed by default; however, Vault needs to be restarted to use the renewed certificate. A script for this is shown at the end of the Create a Vault Service section.

Set up Vault Configuration

Make a directory where you'll store your Vault data:

sudo mkdir -p /var/vault/data

Create the config.hcl file for Vault:

sudo nano /var/vault/config.hcl

Enter the following in config.hcl:

ui = true
disable_mlock = true
storage "raft" {
  path = "/var/vault/data"
  node_id = "node1"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = 0
  tls_cert_file = "/etc/letsencrypt/live/your.url.here/fullchain.pem"
  tls_key_file = "/etc/letsencrypt/live/your.url.here/privkey.pem"
}

api_addr = "https://127.0.0.1:8200"
cluster_addr = "https://127.0.0.1:8201"

Note: The tls_cert_file and tls_key_file paths will differ depending on your domain name — this example uses vault.addigy.io.

Create a Vault Service

Create a file named vault.service at /etc/systemd/system/vault.service:

sudo nano /etc/systemd/system/vault.service

Enter the following in the file:

[Unit]

Description=vault service
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/var/vault/config.hcl

[Service]
EnvironmentFile=-/etc/sysconfig/vault
Environment=GOMAXPROCS=2
Restart=on-failure
ExecStart=/usr/local/bin/vault server -config=/var/vault/config.hcl
LimitMEMLOCK=infinity
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target

Start the service and enable it to start on boot:

sudo systemctl start vault.service
sudo systemctl enable vault.service

Check the status:

sudo systemctl status vault.service

Create a script to reload the Vault service every time Certbot renews the certificate:

sudo sh -c 'printf "#!/bin/sh\nsudo systemctl reload vault.service\n"  /etc/letsencrypt/renewal-hooks/post/vault.sh'

Viewing Vault Logs

To view logs output by the Vault service, use the journalctl command:

journalctl -b --no-pager -u vault -f

The -b flag tells journalctl to output logs from the most recent boot. --no-pager prevents logs from displaying in pager format, making the output easier to use with text manipulation tools. -u filters by unit. -f follows output as it comes, similar to tail -f.

Initialize Vault

Open a new terminal and set the VAULT_ADDR environment variable. This is required for the vault command to know where the Vault server is:

export VAULT_ADDR='http://127.0.0.1:8200'

Note: The IP should point to the machine where the Vault server is running. If operating on localhost, set the address to 127.0.0.1. If operating against a server deployment elsewhere, set the address to that machine's domain.

Now initialize the Vault server to get the unseal keys and root token:

vault operator init

Note: Be sure to securely store the unseal keys and initial root token, as they will not be shown again.

Conventionally, you'd distribute the unseal keys among 5 people, with only 3 required to unseal the Vault.

For more information, see the operator init command docs.

Unseal the Vault and Login as Root

With the unseal keys, unseal the Vault one by one:

vault operator unseal

Note: By default, 3 out of 5 unseal keys are required to unseal the Vault.

After unsealing the Vault, log in with the root token:

vault login

Enter the root token when prompted — it should look similar to:

s.1X9qYtFLmUZ5a1234567834324

For more information, see the operator unseal command docs and the login command docs.

Create a Secret to be Accessed

Here you'll create a secret that other machines should be able to access.

First, enable the key-value secrets engine:

vault secrets enable kv

Note: The default path for the kv secrets engine is kv/.

Create a secret in the path kv/creds with the key/value pair password=123456:

vault kv put kv/creds password=123456

You can delete secrets in the key-value secrets engine with:

vault kv delete [PATH]

You can get secrets in the key-value secrets engine with:

vault kv get [PATH]

For more information, see the secrets command docs and the kv command docs.

Create a Policy that Grants Read Access to our Secret

Here you'll create a policy that allows a machine with your defined permissions to read your secret.

Create a file named read_secret.hcl with the following:

# Permits reading of secret in kv/creds

path "kv/creds" {

  capabilities = ["read"]

}

Now create the policy in Vault using the read_secret.hcl file:

vault policy write read_secret read_secret.hcl

For more information, see the Vault policies docs and the policy command docs.

Create a Token Associated With a Policy

Tokens authenticate with Vault, and their permissions are determined by the policy associated with the token.

For security, tokens should be ephemeral — they should have a specified max TTL and/or use limit, so a leaked token would already be revoked by the time it could be misused.

Create a token associated with the policy created earlier:

vault token create -policy=read_secret -use-limit=1 -explicit-max-ttl=1m

This creates a token associated with the read_secret policy, with a use limit of 1 and a max TTL of 1 minute. TTL can be specified in seconds, minutes, or hours (e.g. 30s, 1m, 1h).

Copy the created token to use in your Addigy custom script.

Note: It's highly recommended that the use limit be set to exactly the number of times the token should be used, and that the max TTL be set to the shortest time necessary for your custom script(s) to execute.

To manually revoke a token, use the vault token revoke command:

vault token revoke s.Ie7jV4oxoftehFeylBieeoU4

For more information, see the token create command docs.

Use in an Addigy Custom Script

Using Vault in a custom script isn't as simple as just running vault kv get kv/creds to get the secret.

The Vault server's address and the Vault client token need to be defined in the environment variables VAULT_ADDR and VAULT_TOKEN, respectively.

For VAULT_ADDR, it should look something like this:

export VAULT_ADDR="http://127.0.0.1:8200"

Replace 127.0.0.1 with the server's IP or domain name (change http to https if using TLS).

For VAULT_TOKEN, enter the token created earlier with the associated policy:

export VAULT_TOKEN=s.20Je8g0MJTJDKI80HGEEF4Qn

Your token will be different from the one used in this example.

Finally, the PATH needs to be set. When running scripts in Addigy, the PATH environment variable doesn't include /usr/local/bin, so by default it won't recognize the vault command installed there.

Modifying the PATH variable should look like this:

export PATH=/usr/local/bin:$PATH

Now you can make API calls with Vault however you choose. Here's an example script that prints out a secret:

export VAULT_ADDR="https://vault.url.here:8200"
export VAULT_TOKEN="Vault_Token_Here"
export PATH=/usr/local/bin:$PATH
PASSWORD=$(vault kv get -field=password kv/creds)
echo "$PASSWORD"

If the token's usage limit is set to the exact number of times it's used in your script, there's virtually no risk of a leaked token granting access to your secrets, since it would already be revoked. It's also good practice to set a max TTL in addition to the max usage, in case the script fails before the token reaches its usage limit — otherwise you risk leaking a token that was never used enough to be revoked.

Enable Audit Logs

One of Vault's key features is its ability to audit any request and response on the Vault server. This feature needs to be enabled to use it.

Be sure the VAULT_ADDR environment variable is set to the Vault server's address, and that you're logged in with the root token or a token with permission to enable audit methods.

Run the following to enable audit logs:

vault audit enable file file_path=/var/log/vault/vault_audit.log

This writes audit logs to the specified path on the Vault server.

The log file may eventually grow too large for the allotted space and will need to be rotated, using logrotate.

SSH into your Vault server deployment to make the following changes.

Create a file named vault in /etc/logrotate.d:

sudo nano /etc/logrotate.d/vault

Enter the following in the file:

/var/log/vault/vault_audit.log {

        rotate 30
        daily
        notifempty
        missingok
        compress
        delaycompress
        postrotate
                systemctl reload vault 2/dev/null || true
        endscript
        extension log
        dateext
        dateformat %Y-%m-%d.
}

Run logrotate with this file:

sudo logrotate /etc/logrotate.d/vault

Log rotation should now be set up with the audit logs.

For information on querying audit logs, see Querying Audit Device Logs.

References

Vault Docs

Was this article helpful?
0 out of 0 found this helpful
Return to top

Didn’t find what you’re looking for?

Contact our Support Team