Skip to content

Commit

Permalink
Initial Implementation (#1)
Browse files Browse the repository at this point in the history
+ added initial elasticache redis implementation for static-roles
  • Loading branch information
maxcoulombe authored Sep 6, 2022
1 parent a9c0de8 commit 548b4d8
Show file tree
Hide file tree
Showing 10 changed files with 628 additions and 146 deletions.
11 changes: 10 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,13 @@ fmtcheck:

.PHONY: fmt
fmt:
gofumpt -l -w .
gofumpt -l -w . && cd bootstrap/terraform && terraform fmt

.PHONY: setup-env
setup-env:
cd bootstrap/terraform && terraform init && terraform apply -auto-approve


.PHONY: teardown-env
teardown-env:
cd bootstrap/terraform && terraform init && terraform destroy -auto-approve
176 changes: 115 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,61 +1,21 @@
# Vault Plugin Scaffolding
# Vault Plugin Database Redis ElastiCache

This is a standalone backend plugin for use with [Hashicorp
This is a standalone [Database Plugin](https://www.vaultproject.io/docs/secrets/databases) for use with [Hashicorp
Vault](https://www.github.com/hashicorp/vault).

[//]: <> (Include a general statement about this plugin)
This plugin supports exclusively AWS ElastiCache for Redis. [Redis Enterprise](https://github.com/RedisLabs/vault-plugin-database-redis-enterprise)
and [Redis Open Source](https://github.com/fhitchen/vault-plugin-database-redis) use different plugins.

Please note: We take Vault's security and our users' trust very seriously. If
you believe you have found a security issue in Vault, please responsibly
disclose by contacting us at [[email protected]](mailto:[email protected]).

## Using this Template Repository

_Note: Remove this instruction sub-heading once you've created a repository from this template_

This repository is a template for a Vault secret engine and auth method plugins.
It is intended as a starting point for creating Vault plugins, containing:

- Changelog, readme, Makefile, pull request template
- Scripts for internal tooling
- Jira sync and basic testing GitHub actions
- A base `main.go` for compiling the plugin

There's some minimal GitHub Secrets setup required in order to get the Jira sync
GH action working. Install the `gh` [CLI](https://cli.github.com/manual/) and
perform the following commands to set secrets for this repository.

```sh
gh secret set JIRA_SYNC_BASE_URL
gh secret set JIRA_SYNC_USER_EMAIL
gh secret set JIRA_SYNC_API_TOKEN
```


This template repository does not include a Mozilla Public License 2.0 `LICENSE`
since plugins created this way can be internal to hashicorp and for Vault
Enterprise consumption. To add a license, follow [these GitHub
instructions](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-license-to-a-repository),
or obtain one from one of our public Vault plugins.

Please see the [GitHub template repository
documentation](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-from-a-template)
for how to create a new repository from this template on GitHub.

Things _not_ handled by this template repository:
- Repository settings, such as branch protection rules
- Memberships and permissions
- GitHub secrets for this repository

Please see the [Repository Configuration Page](https://hashicorp.atlassian.net/wiki/spaces/VAULT/pages/2103476333/Repository+Configuration)
for the setting proper repository configuration values.

## Quick Links

- [Vault Website](https://www.vaultproject.io)
- [Vault Project GitHub](https://www.github.com/hashicorp/vault)
- [Plugin System](https://www.vaultproject.io/docs/plugins)

[//]: <> (Include any other quick links relevant to your plugin)

## Getting Started

Expand All @@ -67,14 +27,17 @@ Otherwise, first read this guide on how to [get started with
Vault](https://www.vaultproject.io/intro/getting-started/install.html).


## Usage
## Development

[//]: <> (Provide usage instructions and/or links to this plugin)
If you wish to work on this plugin, you'll first need
[Go](https://www.golang.org) installed on your machine (version 1.17+ recommended)

## Developing
Make sure Go is properly installed, including setting up a [GOPATH](https://golang.org/doc/code.html#GOPATH).

If you wish to work on this plugin, you'll first need
[Go](https://www.golang.org) installed on your machine.
To run the tests locally you will need to have write permissions to an [ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/) instance.
A small Terraform project is included to provision one for you if needed. More details in the [Environment Set Up](#environment-set-up) section.

## Building

If you're developing for the first time, run `make bootstrap` to install the
necessary tools. Bootstrap will also update repository name references if that
Expand All @@ -92,6 +55,39 @@ mode will only generate the binary for your platform and is faster:
$ make dev
```

## Tests

### Environment Set Up

To test the plugin, you need access to an Elasticache for Redis Cluster.
A Terraform project is included for convenience to initialize a new cluster if needed.
If not already available, you can install Terraform by using [this documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html).

The setup script tries to find and use available AWS credentials from the environment. You can configure AWS credentials using [this documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html).
Or if you prefer you can edit the provider defined ./bootstrap/terraform/elasticache.tf with your desired set of credentials.

Note that resources created via the Terraform project cost a small amount of money per hour.

To set up the test cluster:

```sh
$ make setup-env
...
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
```

### Environment Teardown

The test cluster created via the setup-env command can be destroyed using the teardown-env command.

```sh
$ make teardown-env
...
Destroy complete! Resources: 4 destroyed.
```

### Testing Manually

Put the plugin binary into a location of your choice. This directory
will be specified as the [`plugin_directory`](https://www.vaultproject.io/docs/configuration#plugin_directory)
in the Vault config used to start the server.
Expand All @@ -112,25 +108,62 @@ $ vault server -dev -config=path/to/config.hcl ...
Once the server is started, register the plugin in the Vault server's [plugin catalog](https://www.vaultproject.io/docs/plugins/plugin-architecture#plugin-catalog):

```sh
$ SHA256=$(openssl dgst -sha256 $GOPATH/vault-plugin-secrets-myplugin | cut -d ' ' -f2)
$ vault plugin register \
-sha256=$SHA256 \
-command="vault-plugin-secrets-myplugin" \
secrets myplugin
$ SHA256=$(openssl dgst -sha256 $GOPATH/vault-plugin-database-redis-elasticache | cut -d ' ' -f2)
$ vault plugin register -sha256=$SHA256 database vault-plugin-database-redis-elasticache
...
Success! Data written to: sys/plugins/catalog/database/vault-plugin-database-redis-elasticache
```

Enable the database engine to use this plugin:

```sh
$ vault secrets enable database
...
Success! Data written to: sys/plugins/catalog/myplugin

Success! Enabled the database secrets engine at: database/
```

Enable the secrets engine to use this plugin:
Once the database engine is enabled you can configure an ElastiCache instance:

```sh
$ vault secrets enable myplugin
$ vault write database/config/redis-mydb \
plugin_name="vault-plugin-database-redis-elasticache" \
username=$USERNAME \
password=$PASSWORD \
url=$URL \
region=$REGION
...

Successfully enabled 'plugin' at 'myplugin'!
Success! Data written to: database/config/redis-mydb
```

Configure a static role:

```sh
$ vault write database/static-roles/redis-myrole \
db_name="redis-mydb" \
username="my-elasticache-username" \
rotation_period=5m
...

Success! Data written to: database/roles/redis-myrole
```

Retrieve your first set of static credentials:

```sh
$ vault read database/static-creds/redis-myrole
Key Value
--- -----
last_vault_rotation 2022-09-06T12:15:33.958413491-04:00
password PASSWORD
rotation_period 5m
ttl 4m55s
username my-elasticache-username
```

### Tests

### Automated Tests

To run the tests, invoke `make test`:

Expand All @@ -144,4 +177,25 @@ You can also specify a `TESTARGS` variable to filter tests like so:
$ make test TESTARGS='-run=TestConfig'
```

[//]: <> (Specify any other test instructions such as acceptance/integration tests)
### Acceptance Tests

The majority of tests must communicate with an existing ElastiCache instance. See the [Environment Set Up](#environment-set-up) section for instructions on how to prepare a test cluster.

Some environment variables are required to run tests expecting to communicate with an ElastiCache cluster.
The username and password should be valid IAM access key and secret key with read and write access to the ElastiCache cluster used for testing. The URL should be the complete configuration endpoint including the port, for example: `vault-plugin-elasticache-test.id.xxx.use1.cache.amazonaws.com:6379`.

```sh
$ export TEST_ELASTICACHE_USERNAME="AWS ACCESS KEY ID"
$ export TEST_ELASTICACHE_PASSWORD="AWS SECRET ACCESS KEY"
$ export TEST_ELASTICACHE_URL="vault-plugin-elasticache-test.id.xxx.use1.cache.amazonaws.com:6379"
$ export TEST_ELASTICACHE_REGION="us-east-1"
$ export TEST_ELASTICACHE_USER="vault-test"

$ make test
```

You can also specify a `TESTARGS` variable to filter tests like so:

```sh
$ make test TESTARGS='-run=TestConfig'
```
104 changes: 104 additions & 0 deletions bootstrap/terraform/elasticache.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
provider "aws" {
// Credentials and configuration derived from the environment
// Uncomment if you wish to configure the provider explicitly

// access_key = ""
// secret_key = ""
// region = ""
}

resource "random_password" "vault_plugin_elasticache_test" {
length = 16
}

resource "aws_elasticache_replication_group" "vault_plugin_elasticache_test" {
replication_group_id = "vault-plugin-elasticache-test"
description = "vault elasticache plugin generated test cluster"
engine = "REDIS"
engine_version = "6.2"
node_type = "cache.t4g.micro"
num_cache_clusters = 1
parameter_group_name = "default.redis6.x"
transit_encryption_enabled = true
user_group_ids = [aws_elasticache_user_group.vault_plugin_elasticache_test.id]

tags = {
"description" : "vault elasticache plugin generated test cluster"
}
}

resource "aws_elasticache_user_group" "vault_plugin_elasticache_test" {
engine = "REDIS"
user_group_id = "vault-test-user-group"
user_ids = ["default", aws_elasticache_user.vault_plugin_elasticache_test.user_id]
}

resource "aws_elasticache_user" "vault_plugin_elasticache_test" {
user_id = "vault-test"
user_name = "vault-test"
access_string = "on ~* +@all"
engine = "REDIS"
passwords = [random_password.vault_plugin_elasticache_test.result]
}

resource "aws_iam_user" "vault_plugin_elasticache_test" {
name = "vault-plugin-elasticache-user-test"

tags = {
"description" : "vault elasticache plugin generated test user"
}
}

resource "aws_iam_access_key" "vault_plugin_elasticache_test" {
user = aws_iam_user.vault_plugin_elasticache_test.name
}

resource "aws_iam_user_policy" "vault_plugin_elasticache_test" {
name = "vault-plugin-elasticache-policy-test"
user = aws_iam_user.vault_plugin_elasticache_test.name

policy = data.aws_iam_policy_document.vault_plugin_elasticache_test.json
}

data "aws_iam_policy_document" "vault_plugin_elasticache_test" {
statement {
actions = [
"elasticache:DescribeUsers",
"elasticache:ModifyUser",
]
resources = [
"arn:aws:elasticache:*.*:user:*",
]
}
}

// export TEST_ELASTICACHE_USERNAME=${username}
output "username" {
value = aws_iam_access_key.vault_plugin_elasticache_test.id
}

// export TEST_ELASTICACHE_PASSWORD=${password}
// Use `terraform output password` to access the value
output "password" {
sensitive = true
value = aws_iam_access_key.vault_plugin_elasticache_test.secret
}

// export TEST_ELASTICACHE_URL=${url}
output "url" {
value = format(
"%s:%s",
aws_elasticache_replication_group.vault_plugin_elasticache_test.primary_endpoint_address,
aws_elasticache_replication_group.vault_plugin_elasticache_test.port)
}

// export TEST_ELASTICACHE_REGION=${region}
data "aws_region" "current" {}
output "region" {
value = data.aws_region.current.name
}

// export TEST_ELASTICACHE_USER=${user}
output "user" {
value = aws_elasticache_user.vault_plugin_elasticache_test.user_name
}
23 changes: 23 additions & 0 deletions cmd/vault-plugin-database-redis-elasticache/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package main

import (
"log"
"os"

"github.com/hashicorp/vault-plugin-database-redis-elasticache"
"github.com/hashicorp/vault/sdk/database/dbplugin/v5"
)

func main() {
if err := Run(); err != nil {
log.Println(err)
os.Exit(1)
}
}

// Run starts serving the plugin
func Run() error {
dbplugin.ServeMultiplex(rediselasticache.New)

return nil
}
Loading

0 comments on commit 548b4d8

Please sign in to comment.