vault: unable to list subkeys of a kv1 secret

[noahehall]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. create a kv1 secret at path env/auth_approle_role_bff
2. inject this payload at the path
3. verify you can retrieve a list of keys at path env
4. verif you cant retrieve the keys at path env/auth_approle_role_bff via the vault cli
5. nor can you retrieve the keys via the http api

Expected behavior
A clear and concise description of what you expected to happen.

• the api matches the docs and we can retrieve the keys for a secret at the path provided

Environment:

• Vault Server Version (retrieve with vault status):

$ vault status
Key                     Value
---                     -----
Seal Type               shamir
Initialized             true
Sealed                  false
Total Shares            2
Threshold               2
Version                 1.12.2
Build Date              2022-11-23T12:53:46Z
Storage Type            raft
Cluster Name            vault-cluster-02897051
Cluster ID              6c41c0f4-31c1-80e8-1a23-b78198ed5ad7
HA Enabled              true
HA Cluster              https://127.0.0.1:8301
HA Mode                 active
Active Since            2022-12-28T10:55:25.940895658Z
Raft Committed Index    335
Raft Applied Index      335

• Vault CLI Version (retrieve with vault version):

05:06 AM (nomad *+$%|u=) 
$ vault version
Vault v1.12.2 (415e1fe3118eebd5df6cb60d13defdc01aa17b03), built 2022-11-23T12:53:46Z

• Server Operating System/Architecture:

05:07 AM (nomad *+$%|u=) 
$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.1 LTS
Release:	22.04
Codename:	jammy

Vault server configuration file(s):

# @see https://developer.hashicorp.com/vault/docs/configuration

default_lease_ttl = "7d"
default_max_request_duration = "30s"
disable_cahe = false
disable_mlock = true # we use integrated raft storage
enable_response_header_hostname = true
enable_response_header_raft_node_id = true
log_format= "json"
max_lease_ttl = "30d"
raw_storage_endpoint = false
ui = true # requires at least 1 listener stanza

storage "raft" {
  path    = "/vault/data"
  node_id = "node1"
}


# advertise the non-loopback interface
api_addr = "https://127.0.0.1:8300"
cluster_addr = "https://127.0.0.1:8301"

listener "tcp" {
  address = "0.0.0.0:8300" # provides access to vault UI
  tls_cert_file = "/etc/ssl/certs/live/dev.nirv.ai/fullchain.pem"
  tls_key_file = "/etc/ssl/certs/live/dev.nirv.ai/privkey.pem"
  tls_disable = false
}


############################# todo
# plugin_directory
# plugin_file_uid
# plugin_file_permissions
// telemetry {
//   statsite_address = "127.0.0.1:8125"
//   disable_hostname = true
// }
// seal "transit" { @see https://developer.hashicorp.com/vault/docs/configuration/seal/transit
// }


## doesnt work with this admin policy
## nor does it work if i use the root token

path "secret/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "patch", "sudo"]
}

path "env/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "patch", "sudo"]
}

path "sys/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "patch", "sudo" ]
}

path "auth/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "patch", "sudo" ]
}

path "database/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "patch", "sudo" ]
}

path "pki*" {
  capabilities = [ "create", "read", "update", "delete", "list", "patch", "sudo" ]
}

Additional context
Add any other context about the problem here.

via vault cli

via http api

data at env/auth_app_role_role_bff

token

[maxb]

Vault is functioning as designed here.

Confusingly, with the KV secrets engines, there are two kinds of thing which can both be called "keys":

• URL keys (e.g. auth_approle_auth_bff here) - think of these as similar to file names
• JSON keys (e.g. APP_PORT here) - think of these as similar to file content

Each type is fully distinct. There is no recursion from the URL LIST operation, which only lists the URL keys, down inside JSON structures.

Similiarly to how the ls CLI command will not introspect inside a JSON file on disk.

[noahehall]

@maxb thanks again

it says it pretty clearly in the docs too:

The input must be a folder; list on a file will not return a value.