Commit graph

768 commits

Author SHA1 Message Date
Scott Miller
42a484c6ce
Correct the post-unseal meaning of the seal status type (#23022)
* Correct the post-unseal meaning of the seal status type

And at the same time add a RecoverySealType to the response which preserves the old meaning.
Updated the CLI to display both when relevant.

* changelog

* no longer needed

* Don't need this field either, which fixes unit tests

* fix unit tests
2023-10-09 12:53:13 -05:00
Violet Hynes
f943c37a83
VAULT-19237 Add mount_type to secret response (#23047)
* VAULT-19237 Add mount_type to secret response

* VAULT-19237 changelog

* VAULT-19237 make MountType generic

* VAULT-19237 clean up comment

* VAULT-19237 update changelog

* VAULT-19237 update test, remove mounttype from wrapped responses

* VAULT-19237 fix a lot of tests

* VAULT-19237 standby test
2023-09-20 09:28:52 -04:00
Thy Ton
12b9e5dd36
add CLI commands for plugin runtime VAULT-18181 (#22819)
---------

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-09-08 10:11:48 -07:00
Thy Ton
f96ecf3800
add oci_image and runtime to VersionedPlugin (#22866)
---------

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-09-07 13:42:47 -07:00
Thy Ton
953f6cd818
Vault 18538 reference runtime on plugin register (#22744)
---------

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-09-06 23:37:10 -07:00
Josh Black
1e34eb5ef8
update vault auth submodules to api/v1.10.0 (#22772) 2023-09-05 18:57:10 +00:00
Tom Proctor
07e76196ba
Support running plugins in isolated containers (#22712)
Implements running plugins in containers to give them some degree
of isolation from the main Vault process and other plugins. It only
supports running on Linux initially, where it is easiest to manage unix
socket communication across the container boundary.

Additionally

* Adds -env arg to vault plugin register.
* Don't return env from 'vault plugin info'

Historically it's been omitted, and it could conceivably have secret information in
it, so if we want to return it in the response, it should probably only be via explicit
opt-in. Skipping for now though as it's not the main purpose of the commit.
2023-09-01 17:55:17 +00:00
Thy Ton
08574508c8
add plugin runtime API (#22469)
---------

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-08-31 13:37:04 -07:00
Nick Cabatoff
e29d98aadb
Work around issue with UNIX domain sockets in api.Client addresses (#22523)
The Host part of the URL doesn't actually get used when we initiate connections to UNIX domain sockets.  As of https://github.com/golang/go/issues/60374 (in the latest Go releases at the time of this writing), we must set it to something that looks like a hostname or requests will fail.
2023-08-23 15:05:03 +00:00
Violet Hynes
8f1bf532f6
Fix client clone with headers deadlock (#22410)
* Fix client clone with headers deadlock

* Changelog

* Typo
2023-08-17 15:13:28 -04:00
Tanmay Pereira Naik
2269369229
docs: Update outdated vaultproject.io/docs/ links (#22162)
Signed-off-by: Tanmay Pereira Naik <59953366+tanmay-pnaik@users.noreply.github.com>
Co-authored-by: Brian Shumate <brianshumate@users.noreply.github.com>
2023-08-15 12:59:34 -04:00
Mauro Leggieri
951f1fef1b
Add missing option for RSA2048 signature (#22227)
Add support to use the new RSA2048 signature to validate EC2 instances.

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
2023-08-14 15:08:43 -04:00
hashicorp-copywrite[bot]
0b12cdcfd1
[COMPLIANCE] License changes (#22290)
* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License.

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUS-1.1

* Fix test that expected exact offset on hcl file

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2023-08-10 18:14:03 -07:00
Luis (LT) Carbonell
727c73cbd1
Add dr-token Flag to Autopilot CLI (#21165)
* Add dr-token flag for raft autopilot

* Add changelog

* Cleanup bad patch apply

* Set dr token explicitly
2023-07-27 15:35:25 +00:00
Max Bowsher
25540ad222
Fix regexes for sys/raw/ and sys/leases/lookup/ to match prevailing conventions (#21760)
* Fix regexes for `sys/raw/` and `sys/leases/lookup/` to match prevailing conventions

There are several endpoints in Vault which take an arbitrary path as the
last parameter. Many of these are defined in terms of the
`framework.MatchAllRegex` helper. Some were not, and were defined using
custom regexes which gave rise to multiple OpenAPI endpoints - one with
the path parameter, and one without.

We need to fix these definitions, because they give rise to a very
unnatural result when used to generate a client API - for example, you
end up with `LeasesLookUp()` which is only capable of being used to list
the very top level of the hierarchical collection of leases, and
`LeasesLookUpWithPrefix(prefix)` which must be used for all deeper
levels.

This PR changes the regexes used for `sys/raw/` and `sys/leases/lookup/`
to be consistent with the approach used for other well-known similar
endpoints, such as `cubbyhole/`, `kv-v1/` and `kv-v2/metadata/`.

This PR does have a very small compatibility issue, which I think is
tolerable: prior to this change, `sys/raw` with no trailing slash was
considered a valid endpoint, and now it will no longer be.

One way to observe this is to try `vault path-help sys/raw` - before
this change, it would work, after, it will not. You would have to
instead use `vault path-help sys/raw/foobar` to see the help.

I also considered whether losing the ability to read/write/delete
`sys/raw` would be an issue. In each case, the precise HTTP result code
will change, but each of these were meaningless operations that make no
sense - you cannot read/write/delete a "file" at the "root directory" of
the underlying Vault storage.

In fact, during testing, I discovered that currently, `vault write
sys/raw x=y` when using Raft storage, will permanently break the Vault
instance - it causes a panic within the Raft FSM, which re-occurs
immediately on restarting the server! This PR also closes off that
footgun / DoS vector.

None of these issues apply to `sys/leases/lookup/`, as the existing
regex in that case was already not matching the path without the
trailing slash.

* changelog

* Realign hardcoded sudo paths with updated OpenAPI spec
2023-07-21 13:50:07 -04:00
Max Bowsher
188bdca4bd
Fix sudo paths missing from OpenAPI and docs (#21772)
* Fix sudo paths missing from OpenAPI and docs

Various sudo (a.k.a. root-protected) paths are implemented in
non-standard ways, and as a result:

* are not declared as x-vault-sudo in the OpenAPI spec

* and as a result of that, are not included in the hardcoded patterns
  powering the Vault CLI `-output-policy` flag

* and in some cases are missing from the table of all sudo paths in the
  docs too

Fix these problems by:

* Adding `seal` and `step-down` to the list of root paths for the system
  backend. They don't need to be there for enforcement, as those two
  special endpoints bypass the standard request handling code, but they
  do need to be there for the OpenAPI generator to be able to know they
  require sudo.

  The way in which those two endpoints do things differently can be
  observed in the code search results for `RootPrivsRequired`:
  https://github.com/search?q=repo%3Ahashicorp%2Fvault%20RootPrivsRequired&type=code

* Fix the implementation of `auth/token/revoke-orphan` to implement
  endpoint sudo requirements in the standard way. Currently, it has an
  **incorrect** path declared in the special paths metadata, and then
  compensates with custom code throwing an error within the request
  handler function itself.

* changelog

* As discussed in PR, delete test which is just testing equality of a constant

* Restore sudo check as requested, and add comment

* Update vault/token_store.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

---------

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-07-19 16:28:17 +00:00
Max Bowsher
3961d7b9cc
api: Separate two distinct areas of code that were interleaved in one file (#21906)
This is a code cleanup and addition of an explanatory comment. For some
reason, the code related to the CLI guessing whether a path requires
sudo, has been interleaved into plugin_helpers.go, which was previously
purely code used on the server side in the implementation of Vault
plugins.

This remedies that by dividing the sudo paths code to a separate file,
and adds a comment to plugin_helpers.go providing future readers with
information about the overall theme of the file.

No code has been changed - only moved and documented.
2023-07-18 12:20:34 +01:00
Max Bowsher
e47dd9df53
OpenAPI: Separate ListOperation from ReadOperation (#21723)
* OpenAPI: Separate ListOperation from ReadOperation

Historically, since Vault's ReadOperation and ListOperation both map to
the HTTP GET method, their representation in the generated OpenAPI has
been a bit confusing.

This was partially mitigated some time ago, by making the `list` query
parameter express whether it was required or optional - but only in
a way useful to human readers - the human had to know, for example, that
the schema of the response body would change depending on whether `list`
was selected.

Now that there is an effort underway to automatically generate API
clients from the OpenAPI spec, we have a need to fix this more
comprehensively. Fortunately, we do have a means to do so - since Vault
has opinionated treatment of trailing slashes, linked to operations
being list or not, we can use an added trailing slash on the URL path to
separate list operations in the OpenAPI spec.

This PR implements that, and then fixes an operation ID which becomes
duplicated, with this change applied.

See also hashicorp/vault-client-go#174, a bug which will be fixed by
this work.

* Set further DisplayAttrs in auth/github plugin

To mask out more duplicate read/list functionality, now being separately
generated to OpenAPI client libraries as a result of this change.

* Apply requested changes to operation IDs

I'm not totally convinced its worth the extra lines of code, but
equally, I don't have strong feelings about it, so I'll just make the
change.

* Adjust logic to prevent any possibility of generating OpenAPI paths with doubled final slashes

Even in the edge case of improper use of regex patterns and operations.

* changelog

* Fix TestSudoPaths to pass again... which snowballed a bit...

Once I looked hard at it, I found it was missing several sudo paths,
which led to additional bug fixing elsewhere.

I might need to pull some parts of this change out into a separate PR
for ease of review...

* Fix other tests

* More test fixing

* Undo scope creep - back away from fixing sudo paths not shown as such in OpenAPI, at least within this PR

Just add TODO comments for now.
2023-07-13 13:36:52 -04:00
Max Bowsher
2f677665b3
Fix vault path-help for selected paths with bad regexps (#18571)
* Fix `vault path-help` for selected paths with bad regexps

See the comment being added in `sdk/framework/path.go` for the
explanation of why this change is needed.

* Grammar fix and add changelog

* Also fix hardcoded expectations in a new test

* Add a couple more testcases, and some comments.

* Tweak spelling in comment
2023-06-30 08:42:30 -07:00
Ben Ash
3ca3397676
Add support for cloning a Client's tls.Config (#21424)
Additional fixes:
- handle a failed type assert in api.Config.configureTLS()

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-06-28 17:56:39 -04:00
Hamid Ghaf
f0292fa2e4
OSS part of replication sys method (#20995)
* OSS part of replication sys method

* CL
2023-06-05 14:39:40 -07:00
Violet Hynes
cb7531d3e1
Update auth/api submodules to new API version (#20809)
* Update auth/api submodules to new API version

* Update auth/api submodules to new API version
2023-05-26 14:06:31 +00:00
Márk Sági-Kazár
200f0c0e03
Upgrade go-jose library to v3 (#20559)
* upgrade go-jose library to v3

Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>

* chore: fix unnecessary import alias

Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>

* upgrade go-jose library to v2 in vault

Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>

---------

Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
2023-05-23 12:25:58 +00:00
Marc Boudreau
382d31816d
VAULT-5094: Deal with identity_policies Set to nil in Secret Data Field (#20636)
* fix: deal with identity_policies set to nil

* add changelog file
2023-05-19 09:51:52 -04:00
Anton Averchenkov
fada2d087d
Clarify deprecation comments for RawRequest* functions (#20497) 2023-05-03 19:47:54 +00:00
Mike Palmiotto
60c3527884
Capture loop var in TestClientDisabledRedirects (#20466) 2023-05-02 17:53:30 +00:00
Nick Cabatoff
bbc1b4cb44
Bump the sdk go version to 1.20, document when go versions should be updated (#20278) 2023-04-24 08:44:39 -04:00
Nick Cabatoff
5ce3054e36
Add new method to access a client config's TLSConfig. (#20265) 2023-04-19 15:14:18 -04:00
Hamid Ghaf
e55c18ed12
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Peter Wilson
a2dbd319ca
add comment to explain use of math/rand package in lifetime_watcher (#19533) 2023-03-14 13:17:26 +00:00
Tom Proctor
4df7c64d05
Update x/net and x/crypto/ssh (#19282)
* Update x/net and x/crypto/ssh in api and sdk

* go mod tidy in root go module
2023-02-22 13:46:12 +00:00
Leland Ursu
a5fb5522a8
address various issues with the output-policy flag (#19160)
* update error message and properly handle list requests

* since we do agressive sanitizes we need to optionally check trailing slash

* added changelog record

* remove redundant path formating

* Update changelog/13106.txt

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* addressed comments from review

* also remove code that duplicates efforts in kv_list

* abstracted helper func for testing

* added test cases for the policy builder

* updated the changelog to the correct one

* removed calls that apear not to do anything given test case results

* fixed spacing issue in output string

* remove const representation of list url param

* addressed comments for pr

---------

Co-authored-by: lursu <leland.ursu@hashicorp.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-02-21 10:12:45 -05:00
Peter Wilson
1c2a61ba41
Use MaxInt vs. MaxInt64 to prevent overflow on 32bit architectures (#19213) 2023-02-16 16:40:52 +00:00
Fulton Byrne
86258cd530
LifeTimeWatcher SleepDuration calculation testing (#17919)
* factor out sleep duration calc
* property based sleep duration test

Co-authored-by: peteski22 <peter.wilson@hashicorp.com>
2023-02-14 14:57:25 +00:00
Kuba Wieczorek
fffc3daa8d
update vault auth submodules to new version of API (#19127) 2023-02-10 08:12:10 -08:00
miagilepner
13caa0842e
VAULT-8436 remove <-time.After statements in for loops (#18818)
* replace time.After with ticker in loops

* add semgrep rule

* update to use timers

* remove stop
2023-02-06 17:49:01 +01:00
Nick Cabatoff
192baa88db
Make API not depend on SDK (#18962) 2023-02-06 09:41:56 -05:00
Scott Miller
bf58843b0a
Revert #18683 (#18942)
* Revert "Don't execute the seal recovery tests on ENT. (#18841)"

This reverts commit 990d3bacc2.

* Revert "Add the ability to unseal using recovery keys via an explicit seal option. (#18683)"

This reverts commit 2ffe49aab0.
2023-02-01 13:34:53 -06:00
Rachel Culpepper
d228e62b51
Vault-11713: Add the ability for transit to use managed keys (#18844)
* add the ability for transit to use managed keys

* add fields for rotate

* reorganize managed keys for better error messaging

* add aad for managed keys

* change encrypting key interface

* fix import error

* fix fmt errors

* move managed key rotation to a separate function

* fix fmt issue

* fix import error

* fix go.mod issue

* fix go.mod

* remove extra rotate param

* fix locking issue in rotation

* fix errors in policy tests

* move managed key params to fit existing policy functions

* fix sign and verify

* fix policy tests

* fix test error
2023-01-27 13:39:58 -06:00
Scott Miller
2ffe49aab0
Add the ability to unseal using recovery keys via an explicit seal option. (#18683)
* wip

* wip

* Got it 'working', but not happy about cleanliness yet

* Switch to a dedicated defaultSeal with recovery keys

This is simpler than trying to hijack SealAccess as before.  Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal.  Then everything proceeds
as if you had a shamir seal to begin with.

* Handle recovery rekeying

* changelog

* Revert go.mod redirect

* revert multi-blob info

* Dumb nil unmarshal target

* More comments

* Update vault/seal.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* pr feedback

* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split

* Better comment on recovery seal during adjustSealMigration

* Make it possible to migrate from an auto-seal in recovery mode to shamir

* Fix sealMigrated to account for a recovery seal

* comments

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Address PR feedback

* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate

* Don't shortcut the reast of seal migration

* get rid of redundant transit server cleanup

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-24 14:57:56 -06:00
Alexander Scheel
a8a984d472
Update x/crypto + x/net (#18794)
* Update golang.org/x/crypto version

go get -u golang.org/x/crypto && go mod tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update golang.org/x/crypto version in api

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update golang.org/x/crypto version in sdk

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-23 19:11:04 +00:00
Anton Averchenkov
39335cc8cd
Update vault and api/auth submodules to use api/v1.8.3 (#18773) 2023-01-20 11:44:03 -05:00
Anton Averchenkov
b6c02b15f5
Update api & vault to use sdk v0.7.0 (#18765) 2023-01-19 15:21:10 -05:00
Anton Averchenkov
ef3e3eace2
Remove timeout logic from ReadRaw functions and add ReadRawWithContext (#18708)
Removing the timeout logic from raw-response functions and adding documentation comments. The following functions are affected:

- `ReadRaw`
- `ReadRawWithContext` (newly added)
- `ReadRawWithData`
- `ReadRawWithDataWithContext`

The previous logic of using `ctx, _ = c.c.withConfiguredTimeout(ctx)` could cause a potential [context leak](https://pkg.go.dev/context):

> Failing to call the CancelFunc leaks the child and its children until the parent is canceled or the timer fires. The go vet tool checks that CancelFuncs are used on all control-flow paths.

Cancelling the context would have caused more issues since the context would be cancelled before the request body is closed.

Resolves: #18658
2023-01-17 15:41:59 -05:00
Anton Averchenkov
3f073ae3ce
Revert "Add mount path into the default generated openapi.json spec (#17926)" (#18617)
* Revert "Add mount path into the default generated openapi.json spec (UI) (#17926)"

This reverts commit db8efac708.

* Revert "Remove `generic_mount_paths` field (#18558)"

This reverts commit 79c8f626c5.
2023-01-10 11:16:59 -05:00
Anton Averchenkov
db8efac708
Add mount path into the default generated openapi.json spec (UI) (#17926) 2022-12-08 12:15:54 -05:00
akshya96
c5d0528d27
Vault 9800 Fix vault read handling for endpoints with no top-level data object (#17913)
* fix vault read

* changing json unmarshal

* adding changelog

* removing line

* removing format=raw

* add test

* fix tests

* add fix for resp.Errors

* adding comments
2022-11-17 10:51:37 -08:00
Alexander Scheel
02d265b2fa
PKI Health Check Command (#17750)
* Stub out initial health check command

This command will be used to generate health check results for the PKI
engine.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Start common health check implementation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add common health check utilities

These utilities will collect helpers not specific to PKI health checks,
such as formatting longer durations more legibly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add PKI health check common utils

Many health checks will need issuer and/or CRL information in order to
execute. We've centrally located these helpers to avoid particular
health checks from needing to reimplement them each time.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Adding ca_validity_period health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Begin using health-checks in PKI command

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow parsing raw requests afterwards

This shifts the last of the logic difference between Read(...) and
ReadRaw(...) to a new helper, allowing ReadRaw(...) requests to be
parsed into the same response structure afterwards as Read(...); this
allows API callers to fetch the raw secret and inspect the raw response
object in case something went wrong (error code &c) -- and when the
request succeeds, they can still get the api.Secret out.

This will be used with the PKI health check functionality, making both
LIST and READ operations use ReadRaw, and optionally parsing the secret
afterwards.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add crl_validity_period health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for PKI health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix bug in raw reading with contexts

When reading raw objects, don't manually call the context cancellation:
this causes timeouts and/or EOF errors when attempting to read or parse
the response body. See message in client.RawRequestWithContext(...) for
more information.

This was causing the test suite to randomly fail, due to the context
cancelling. The test suite's client usually had a default timeout,
whereas the CLI didn't, and thus didn't exhibit the same issue.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix typo in permissions message

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Move %v->%w for errs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-16 09:27:56 -05:00
Anton Averchenkov
20f66ef7dd
Revert "Add mount path into the default generated openapi.json spec (#17839)" (#17890)
This reverts commit 02064eccb4.
2022-11-10 15:39:53 -08:00
Anton Averchenkov
02064eccb4
Add mount path into the default generated openapi.json spec (#17839)
The current behaviour is to only add mount paths into the generated `opeanpi.json` spec if a `generic_mount_paths` flag is added to the request. This means that we would have to maintain two different `openapi.json` files, which is not ideal. The new solution in this PR is to add `{mount_path}` into every path with a default value specified:

```diff
--    "/auth/token/accessors/": {
++    "/auth/{mount_path}/accessors/": {
      "parameters": [
        {
          "name": "mount_path",
          "description": "....",
          "in": "path",
          "schema": {
            "type": "string",
++          "default": "token"
          }
        }
      ],
```

Additionally, fixed the logic to generate the `operationId` (used to generate method names in the code generated from OpenAPI spec). It had a bug where the ID had `mountPath` in it. The new ID will look like this:

```diff
-- "operationId": "listAuthMountpathAccessors",
++ "operationId": "listTokenAccessors",
```
2022-11-10 15:44:43 -05:00
Violet Hynes
80cc008d6b
VAULT-8703 Add warning for dangerous undocumented overrides, if used, in status response (#17855)
* VAULT-8703 Add warning for dangerous undocumented overrides, if used, in status response

* VAULT-8703 add changelog

* VAULT-8703 fix append
2022-11-09 11:04:36 -05:00
divyaac
994dbd5861
Introspection API Implementation for Router Struct (#17789)
* OSS Commit from ENT for Introspection API

* Add changelog
2022-11-04 09:39:09 -07:00
akshya96
746b089472
Vault 8305 Prevent Brute Forcing in Auth methods : Setting user lockout configuration (#17338)
* config file changes

* lockout config changes

* auth tune r/w and auth tune

* removing changes at enable

* removing q.Q

* go mod tidy

* removing comments

* changing struct name for config file

* fixing mount tune

* adding test file for user lockout

* fixing comments and add changelog

* addressing comments

* fixing mount table updates

* updating consts in auth_tune

* small fixes

* adding hcl parse test

* fixing config compare

* fixing github comments

* optimize userlockouts.go

* fixing test

* minor changes

* adding comments

* adding sort to flaky test

* fix flaky test
2022-11-01 11:02:07 -07:00
John-Michael Faircloth
c782b6678c
update vault sdk and api go-plugin version to v1.4.5 (#17734)
* update sdk's go-plugin version

* update api go.mod

* update api go.mod go version and go mod tidy
2022-10-31 08:52:04 -05:00
Alexander Scheel
057b40d6ae
Vault Raw Read Support (CLI & Client) (#14945)
* Expose raw request from client.Logical()

Not all Vault API endpoints return well-formatted JSON objects.
Sometimes, in the case of the PKI secrets engine, they're not even
printable (/pki/ca returns a binary (DER-encoded) certificate). While
this endpoint isn't authenticated, in general the API caller would
either need to use Client.RawRequestWithContext(...) directly (which
the docs advise against), or setup their own net/http client and
re-create much of Client and/or Client.Logical.

Instead, exposing the raw Request (via the new ReadRawWithData(...))
allows callers to directly consume these non-JSON endpoints like they
would nearly any other endpoint.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add raw formatter for direct []byte data

As mentioned in the previous commit, some API endpoints return non-JSON
data. We get as far as fetching this data (via client.Logical().Read),
but parsing it as an api.Secret fails (as in this case, it is non-JSON).
Given that we intend to update `vault read` to support such endpoints,
we'll need a "raw" formatter that accepts []byte-encoded data and simply
writes it to the UI.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support for reading raw API endpoints

Some endpoints, such as `pki/ca` and `pki/ca/pem` return non-JSON
objects. When calling `vault read` on these endpoints, an error
is returned because they cannot be parsed as api.Secret instances:

> Error reading pki/ca/pem: invalid character '-' in numeric literal

Indeed, we go to all the trouble of (successfully) fetching this value,
only to be unable to Unmarshal into a Secrets value. Instead, add
support for a new -format=raw option, allowing these endpoints to be
consumed by callers of `vault read` directly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove panic

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-28 09:45:32 -04:00
AnPucel
b0adb620c4
Update vault to v1.8.1 (#17516)
* Update vault to v1.8.1

* Updating api/auth to use new version of api
2022-10-20 11:56:55 -07:00
Christopher Swenson
6bede50176
Update httpguts for GO-2022-0236 fix (#17520)
[`govulncheck`](https://go.dev/blog/vuln) reports that the `api` package
has a call chain that includes
`golang.org/x/net/http/httpguts.HeaderValuesContainsToken`, a vulnerable
function.
2022-10-12 15:02:54 -07:00
Tom Proctor
4dd8fc6ed5
Plugins: Add -version flag to 'vault plugin info' (#17454)
* Add -version flag to 'vault plugin info'
* Allow specifying a builtin tag when reading a single plugin from the catalog
2022-10-07 15:28:15 +01:00
AnPucel
8ee0f80d20
Fix for KV_V2 Custom Metadata Bug (#17395) 2022-10-05 16:43:54 -07:00
Peter Wilson
06c0c6c882
Added flag and env var which will disable client redirection (#17352)
* Added flag and env var which will disable client redirection

* Added changelog

* Docs fix for unsaved file, and test single request made

* Updated test for case when redirect is enabled, updated docs based on suggestions
2022-09-30 09:29:37 +01:00
Tom Proctor
d467681e15
cli/api: Update plugin listing to always include version info in the response (#17347) 2022-09-29 18:22:33 +01:00
Alexander Scheel
7d7bcfa304
Update tlsutil to v0.1.2 for ecdsa algo naming (#17282)
* Update tlsutil in sdk

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update tlsutil in API

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update tlsutil to v0.1.2 for ecdsa algo naming

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-09-23 08:28:40 -04:00
Tom Proctor
21d13633d7
CLI: Tune plugin version for auth/secret mounts (#17277)
* Add -plugin-version flag to vault auth/secrets tune
* CLI tests for auth/secrets tune
* CLI test for plugin register
* Plugin catalog listing bug where plugins of different type with the same name could be double counted
* Use constant for -plugin-version flag name
2022-09-22 20:55:46 +01:00
Steven Clark
a156f4e4a6
update vault auth submodules to api/v1.8.0 (#17228) 2022-09-20 10:51:51 -04:00
Steven Clark
d849e51d76
update api to use sdk/v0.6.0 (#17224) 2022-09-20 10:11:29 -04:00
Tom Proctor
d6d03ac373
Plugins: Consistently use plugin_version (#17171)
* Delete Sha field, rename RunningSha -> RunningSha256
* Rename version -> plugin_version
2022-09-20 12:35:50 +01:00
Steven Clark
d7f4bc3e08
Update missing go-kms-wrapping v2 dep and address some ENT->OSS drift (#17178)
* Update missing go-kms-wrapping v2 dep and address some ENT->OSS drift

* Bump go-kms-wrapping/wrappers/gcpckms/v2 to v2.0.1
2022-09-19 10:23:40 -04:00
Mike Palmiotto
63e6cc1d66
Add deprecation status to plugin api and cli (#17077)
* api: Add deprecation status to plugin endpoints

* cli: Add -detailed flag to `plugin list`

* docs: Update plugin list/info docs
2022-09-09 16:03:07 -04:00
Josh Black
03d2be4cb9
update gofumpt to 0.3.1 and reformat the repo (#17055)
* update gofumpt to 0.3.1 and reformat the repo

* output the version of the formatter we're using
2022-09-07 17:31:20 -07:00
Hamid Ghaf
89f08de845
HCP link integration (#16939)
* HCP link integration

* update configure-git.yml

* more OSS stuff

* removing internal repos

* adding a nil check

* removing config test to be included in ENT only

* updating hcp-sdk-go to v0.22.0

* remove Hostname and AuthURL link config params

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-09-06 14:11:04 -04:00
Mike Palmiotto
4099ca7704
Add deprecation status to auth/secrets list (#16849)
* auth: Add Deprecation Status to auth list -detailed
* secrets: Add Deprecation Status to secrets list -detailed
* Add changelog entry for deprecation status list
2022-08-31 16:11:14 -04:00
Christopher Swenson
9d97decb26
Update mount table and CLI with plugin version for auth (#16856) 2022-08-31 19:23:05 +01:00
John-Michael Faircloth
07927e036c
feature: secrets/auth plugin multiplexing (#14946)
* enable registering backend muxed plugins in plugin catalog

* set the sysview on the pluginconfig to allow enabling secrets/auth plugins

* store backend instances in map

* store single implementations in the instances map

cleanup instance map and ensure we don't deadlock

* fix system backend unit tests

move GetMultiplexIDFromContext to pluginutil package

fix pluginutil test

fix dbplugin ut

* return error(s) if we can't get the plugin client

update comments

* refactor/move GetMultiplexIDFromContext test

* add changelog

* remove unnecessary field on pluginClient

* add unit tests to PluginCatalog for secrets/auth plugins

* fix comment

* return pluginClient from TestRunTestPlugin

* add multiplexed backend test

* honor metadatamode value in newbackend pluginconfig

* check that connection exists on cleanup

* add automtls to secrets/auth plugins

* don't remove apiclientmeta parsing

* use formatting directive for fmt.Errorf

* fix ut: remove tls provider func

* remove tlsproviderfunc from backend plugin tests

* use env var to prevent test plugin from running as a unit test

* WIP: remove lazy loading

* move non lazy loaded backend to new package

* use version wrapper for backend plugin factory

* remove backendVersionWrapper type

* implement getBackendPluginType for plugin catalog

* handle backend plugin v4 registration

* add plugin automtls env guard

* modify plugin factory to determine the backend to use

* remove old pluginsets from v5 and log pid in plugin catalog

* add reload mechanism via context

* readd v3 and v4 to pluginset

* call cleanup from reload if non-muxed

* move v5 backend code to new package

* use context reload for for ErrPluginShutdown case

* add wrapper on v5 backend

* fix run config UTs

* fix unit tests

- use v4/v5 mapping for plugin versions
- fix test build err
- add reload method on fakePluginClient
- add multiplexed cases for integration tests

* remove comment and update AutoMTLS field in test

* remove comment

* remove errwrap and unused context

* only support metadatamode false for v5 backend plugins

* update plugin catalog errors

* use const for env variables

* rename locks and remove unused

* remove unneeded nil check

* improvements based on staticcheck recommendations

* use const for single implementation string

* use const for context key

* use info default log level

* move pid to pluginClient struct

* remove v3 and v4 from multiplexed plugin set

* return from reload when non-multiplexed

* update automtls env string

* combine getBackend and getBrokeredClient

* update comments for plugin reload, Backend return val and log

* revert Backend return type

* allow non-muxed plugins to serve v5

* move v5 code to existing sdk plugin package

* do next export sdk fields now that we have removed extra plugin pkg

* set TLSProvider in ServeMultiplex for backwards compat

* use bool to flag multiplexing support on grpc backend server

* revert userpass main.go

* refactor plugin sdk

- update comments
- make use of multiplexing boolean and single implementation ID const

* update comment and use multierr

* attempt v4 if dispense fails on getPluginTypeForUnknown

* update comments on sdk plugin backend
2022-08-29 21:42:26 -05:00
Anton Averchenkov
c8c138f952
Add a sentinel error for missing KV secrets (#16699) 2022-08-12 19:29:42 -04:00
Eng Zer Jun
6141d61839
refactor: replace strings.Replace with strings.ReplaceAll (#15392)
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-08-03 15:22:48 -04:00
swayne275
82f1216c83
Use %q for quoted strings where appropriate (#15216)
* change '%s' to %q where single vs double quotes shouldn't matter

* replace double quotes with %q in logs and errors
2022-08-03 12:32:45 -06:00
Jason O'Donnell
4bcc7e1bb3
agent: add disable_keep_alives configurable (#16479)
agent: add disable_keep_alives config

Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
2022-07-28 12:59:49 -07:00
John-Michael Faircloth
58b88b8aca
Revert "AutoMTLS for secrets/auth plugins (#15671)" (#16377)
This reverts commit 39bcd5c715.
2022-07-20 10:36:23 -05:00
John-Michael Faircloth
39bcd5c715
AutoMTLS for secrets/auth plugins (#15671)
* use automtls for v5 secrets/auth plugins

* add automtls env guard

* start backend without metadata mode

* use PluginClientConfig for backend's NewPluginClient param

refactor

* - fix pluginutil test
- do not expect plugin to be unloaded in UT
- fix pluginutil tests --need new env var
- use require in UT
- fix lazy load test

* add changelog

* prioritize automtls; improve comments

* user multierror; refactor pluginSet for v4 unit test

* add test cases for v4 and v5 plugin versions

* remove unnecessary call to AutoMTLSSupported

* update comment on pluginSets

* use runconfig directly in sdk newpluginclient

* use automtls without metadatamode for v5 backend plugin registration

* use multierror for plugin runconfig calls

* remove some unnecessary code
2022-07-18 16:25:18 -05:00
VAL
8aac5c2057
Update to use latest api version (#16329) 2022-07-18 10:36:50 -07:00
AnPucel
e39fa4a596
Developer Quickstart docs improvements (#16199)
- Make the dev quick start link readily available on the client library documentation page
- Move the full code samples to the top of the dev quickstart page so that they're easily accessible.
- Update the api/readme to have a link to the dev quickstart
2022-06-30 08:50:35 -07:00
Marc Boudreau
d5305500e3
Improving Handling of Unix Domain Socket Addresses (#11904)
* Removed redundant checks for same env var in ReadEnvironment, extracted Unix domain socket logic to function, and made use of this logic in SetAddress.  Adjusted unit tests to verify proper Unix domain socket handling.

* Adding case to revert from Unix domain socket dial function back to TCP

* Adding changelog file

* Only adjust DialContext if RoundTripper is an http.Transport

* Switching from read lock to normal lock

* only reset transport DialContext when setting different address type

* made ParseAddress a method on Config

* Adding additional tests to cover transitions to/from TCP to Unix

* Moved Config type method ParseAddress closer to type's other methods.

* make release note more end-user focused

* adopt review feedback to add comment about holding a lock
2022-06-21 15:16:58 -07:00
Jason O'Donnell
e38f6e72a2
agent: add disable_idle_connections configurable (#15986)
* agent: add disable_keep_alives configurable

* Add empty test

* Add website doc

* Change to disable_idle_connections

* Update tests and doc

* Add note about env

* Changelog

* Change to slice

* Remove unused disable keep alive methods

* Add invalid value test
2022-06-16 18:06:22 -04:00
Steven Clark
a47b44a18a
TestLifetimeWatcher: Address race condition in test assertions (#15969)
- If the timing is correct, a delay in the test's select might see the
   doneCh signal before the renew channels signal. If that happens, the
   test fails as it assumes we will receive signals across different
   channels in order.
 - Rework the test to make sure that we read from the renew channel if expected
   and the done channel so that any errors might not be escaping from detection
   on a renew.
2022-06-14 09:44:51 -04:00
VAL
2266004001
Update minimum required go version for api (#15915)
* Update minimum required go version for api

* Update root go.mod to use latest sdk
2022-06-09 13:15:18 -07:00
VAL
111dcf194b
Make KV structs public so the KV helper methods show in godoc (#15880) 2022-06-08 11:14:55 -07:00
VAL
606ef8ffb5
Add PutMetadata and PatchMetadata KV helpers (#15755)
* Add PutMetadata and PatchMetadata KV helpers

* Refactor tests, stop needlessly converting zero values to nil

* Merge fix for delete-version-after zero value bug

* Cast nils to zero value on Put

* Use 2 different structs for put/patch metadata input, clarity fixes

* Use local constants and simplify stringification
2022-06-08 09:29:22 -07:00
VAL
ad74e074cc
KV helpers for DeleteMetadata, Undelete, Destroy, and Rollback (#15637)
* KV helpers for DeleteMetadata, Undelete, Destroy, and Rollback

* Allow rollback when no secret data on latest version, and update error messages
2022-06-03 10:42:43 -07:00
Alexander Scheel
1e8004d810
Update to fixed parseutil v0.1.6 (#15774)
Note that this only really applies to the SSH engine, nothing else uses
this helper. Other go.mod's updated for consistency.

See also: https://github.com/hashicorp/go-secure-stdlib/pull/40

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-02 17:31:45 -04:00
Christopher Swenson
df79e2c0eb
Update AWS auth method certificates (#15719)
Update AWS auth method certificates

Add tests that the `rsa2048` document can also be verified using the
`pkcs7` field for AWS auth.

Due to the use of SHA-1-based signatures for the `identity` and `pkcs7`
methods, we want to encourage moving toward using the RSA 2048 workflow,
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/verify-rsa2048.html

This doesn't require code changes for Vault necessarily, but adding in
the (many) certificates will help end users.

Also adds `rsa2048` option to API to fetch the RSA 2048 signature.

I will make a PR to update to the AWS auth docs to document the RSA 2048
flow soon after this.
2022-06-01 10:26:17 -07:00
VAL
2afaaf4384
Add Patch KV helper (#15587)
* Add Read methods for KVClient

* KV write helper

* Add changelog

* Add Delete method

* Use extractVersionMetadata inside extractDataAndVersionMetadata

* Return nil, nil for v1 writes

* Add test for extracting version metadata

* Split kv client into v1 and v2-specific clients

* Add ability to set options on Put

* Add test for KV helpers

* Add custom metadata to top level and allow for getting versions as sorted slice

* Update tests

* Separate KV v1 and v2 into different files

* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity

* Move structs and godoc comments to more appropriate files

* Add more tests for extract methods

* Rework custom metadata helper to be more consistent with other helpers

* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method

* Add Patch KV helper

* Add godoc comment and use WithOption ourselves in other KVOption functions

* Clean up options-handling and resp parsing logic; add more tests

* Add constants and more patch tests
2022-06-01 07:50:56 -07:00
VAL
242a6f984b
KV helper methods for api package (#15305)
* Add Read methods for KVClient

* KV write helper

* Add changelog

* Add Delete method

* Use extractVersionMetadata inside extractDataAndVersionMetadata

* Return nil, nil for v1 writes

* Add test for extracting version metadata

* Split kv client into v1 and v2-specific clients

* Add ability to set options on Put

* Add test for KV helpers

* Add custom metadata to top level and allow for getting versions as sorted slice

* Update tests

* Separate KV v1 and v2 into different files

* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity

* Move structs and godoc comments to more appropriate files

* Add more tests for extract methods

* Rework custom metadata helper to be more consistent with other helpers

* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method

* Return early for readability and make test value name less confusing
2022-05-25 11:17:13 -07:00
Chris Capurso
ec18dca10a
update api to use sdk/v0.5.0 (#15599) 2022-05-25 13:48:21 -04:00
Peter Wilson
338fbeaec3
Added support for VAULT_PROXY_ADDR + Updated docs (#15377)
Updated documentation to describe the behavior when supplying `VAULT_HTTP_PROXY`. Also added support for `VAULT_PROXY_ADDR` as a 'better name' for `VAULT_HTTP_PROXY`.
2022-05-24 13:38:51 -04:00
davidadeleon
6ee3fcbec4
api/monitor: Adding log format to monitor command and debug (#15536)
* Correct handling of "unspecified" log level

* Setting log-format default on monitor path

* Create changelog file

* Update website/content/api-docs/system/monitor.mdx

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-24 13:10:53 -04:00
Josh Black
de99f93820
Add autopilot automated upgrades and redundancy zones (#15521) 2022-05-20 16:49:11 -04:00
Christopher Swenson
38b0fa6b53
Remove spurious fmt.Printf calls including one of a key (#15344)
And add a semgrep for fmt.Printf/Println.
2022-05-19 12:27:02 -07:00
Tom Proctor
01fd2660ca
api: make ListPlugins parse only known plugin types (#15434) 2022-05-17 17:41:26 +01:00
Alexander Scheel
83bc726273
Update parseutil in API, SDK (#15465)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-17 11:44:32 -04:00
Alexander Scheel
91e710f495
Fix integer overflows with new parseutil (#15437)
* Use new parseutil helper: Safe variants

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update parseutil to v0.1.5

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix additional integer overflow in command/server

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:57:19 -04:00