token-lookup returns 403 when using root token

[ghost]

Environment:

• Vault Version: Vault v0.9.1 ('87b6919dea55da61d7cd444b2442cabb8ede8ab1')
• Operating System/Architecture: Linux 4.9.0-4-amd64 Initial Website Import #1 SMP Debian 4.9.65-3+deb9u1 (2017-12-23) x86_64 GNU/Linux

Vault Config File:

{
  "backend": {
    "consul": {
      "address": "127.0.0.1:8500",
      "path": "vault",
      "token": "xxx"
    }
  },
  "listener": {
    "tcp": {
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  }
}

Issue:

I am unable to lookup all tokens from v1/auth/token/accessors when using a root token:

curl -H "X-Vault-Token: $VAULT_TOKEN" --request LIST http://127.0.0.1:8200/v1/auth/token/accessors | jq .
{
  "request_id": "xxx",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "keys": [
      "...",
      "66cfb694-eb51-0b9d-fcd7-910ce6e14b6b",
      "..."
    ]
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

Expected Behavior:

vault token-lookup -accessor 66cfb694-eb51-0b9d-fcd7-910ce6e14b6b
Key               Value
---               -----
accessor          66cfb694-eb51-0b9d-fcd7-910ce6e14b6b
...

Actual Behavior:

vault token-lookup -accessor 66cfb694-eb51-0b9d-fcd7-910ce6e14b6b
error looking up token: Error making API request.

URL: POST http://127.0.0.1:8200/v1/auth/token/lookup-accessor
Code: 403. Errors:

* 1 error occurred:

* permission denied

when running in TRACE mode:

{
  "time": "***",
  "type": "response",
  "auth": {
    "client_token": "***",
    "accessor": "***",
    "display_name": "root",
    "policies": [
      "root"
    ],
    "metadata": null
  },
  "request": {
    "id": "xxx",
    "operation": "update",
    "client_token": "***",
    "client_token_accessor": "***",
    "path": "auth/token/lookup-accessor",
    "data": {
      "accessor": "hmac-sha256:061f6e9c01e54584ebc5f168e7433ce46150dcac0c14d43598e6b1af21b38995"
    },
    "policy_override": false,
    "remote_address": "127.0.0.1",
    "wrap_ttl": 0,
    "headers": {}
  },
  "response": {},
  "error": "1 error occurred:\n\n* permission denied"
}

Important Factoids:

vault token-lookup
Key               Value
---               -----
accessor          76a176fe-8e76-e6d7-1c4d-9041e25f6fea
...
meta            	<nil>
num_uses        	0
orphan          	true
path            	auth/token/root
policies        	[root]
ttl             	0

[jefferai]

I'm unable to reproduce this:

$ vault token lookup
Key                 Value
---                 -----
accessor            f4b680d0-e74d-1368-5fd3-53f2deb7a499
creation_time       1517167787
creation_ttl        0
display_name        token
entity_id           n/a
expire_time         <nil>
explicit_max_ttl    0
id                  root
issue_time          2018-01-28T14:29:47.338154512-05:00
meta                <nil>
num_uses            0
orphan              true
path                auth/token/create
policies            [root]
renewable           false
ttl                 0

$ vault token create -policy=default
Key                Value
---                -----
token              26a12e76-783f-fd67-4ad9-0685f297ac22
token_accessor     d56ea545-c582-ca50-7460-3235e0d65b6c
token_duration     768h
token_renewable    true
token_policies     [default]

$ vault list auth/token/accessors
Keys
----
d56ea545-c582-ca50-7460-3235e0d65b6c
f4b680d0-e74d-1368-5fd3-53f2deb7a499

$ vault token-lookup -accessor f4b680d0-e74d-1368-5fd3-53f2deb7a499
WARNING! The "vault token-lookup" command is deprecated. Please use "vault
token lookup" instead. This command will be removed in Vault 0.11 (or later).

Key                 Value
---                 -----
accessor            f4b680d0-e74d-1368-5fd3-53f2deb7a499
creation_time       1517167787
creation_ttl        0
display_name        token
entity_id           n/a
expire_time         <nil>
explicit_max_ttl    0
id                  n/a
issue_time          2018-01-28T14:29:47.338154512-05:00
meta                <nil>
num_uses            0
orphan              true
path                auth/token/create
policies            [root]
renewable           false
ttl                 0

$ vault token-lookup -accessor d56ea545-c582-ca50-7460-3235e0d65b6c
WARNING! The "vault token-lookup" command is deprecated. Please use "vault
token lookup" instead. This command will be removed in Vault 0.11 (or later).

Key                 Value
---                 -----
accessor            d56ea545-c582-ca50-7460-3235e0d65b6c
creation_time       1517167805
creation_ttl        2764800
display_name        token
entity_id           n/a
expire_time         2018-03-01T14:30:05.322604374-05:00
explicit_max_ttl    0
id                  n/a
issue_time          2018-01-28T14:30:05.322604163-05:00
meta                <nil>
num_uses            0
orphan              false
path                auth/token/create
policies            [default]
renewable           true
ttl                 2764557

[ghost]

This appears to be the issue seen in #2661.

The fix is to do:

curl -H "X-Vault-Token: $VAULT_TOKEN" -X POST http://127.0.0.1:8200/v1/auth/token/tidy

and the tokens which cannot be looked up are removed from /v1/auth/token/accessors.