* Move seal barrier type field from Access to autoSeal struct.
Remove method Access.SetType(), which was only being used by a single test, and
which can use the name option of NewTestSeal() to specify the type.
* Change method signatures of Access to match those of Wrapper.
* Turn seal.Access struct into an interface.
* Tweak Access implementation.
Change `access` struct to have a field of type wrapping.Wrapper, rather than
extending it.
* Add method Seal.GetShamirWrapper().
Add method Seal.GetShamirWrapper() for use by code that need to perform
Shamir-specific operations.
* add max_entry_size to sanitized config output
* add changelog entry
* add test parallelism
* add inmem test case
* use named struct fields for TestSysConfigState_Sanitized cases
* VAULT-12940 test for templating user agent
* VAULT-12940 User agent work so far
* VAULT-12940 Vault Agent uses Vault Agent specific User-Agent header when issuing requests
* VAULT-12940 Clean-up and godocs
* VAULT-12940 changelog
* VAULT-12940 Fix test checking headers
* VAULT-12940 Fix test checking headers
* VAULT-12940 Fix test checking headers
* VAULT-12940 Fix test checking headers
* VAULT-12940 copy/paste typos
* VAULT-12940 improve comments, use make(http.Header)
* VAULT-12940 small typos and clean-up
* add flag
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* handle kv paths
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* scaffold test
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* need metadata for list paths
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add (broken) test
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* fix test
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* update docs
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add changelog
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* format
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add godoc
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add test case for mount only
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* handle case of no unnamed arg
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add non-mount behavior
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add more detail to comment
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add v1 tests
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* replace use of os.Unsetenv in test with t.Setenv and remove t.Parallel from test that rely on env being modified.
* experiment with using fromJSON function
* revert previous experiment
* including double quotes in the output value for the string ubuntu-latest
* use go run to launch gofumpt
* Migrate subset of CircleCI ci workflow to GitHub Actions
Runs test-go and test-go-remote-docker with a static splitting of test packages
* [skip actions] add comment to explain the purpose of test-generate-test-package-lists.sh and what to do if it fails
* change trigger to push
---------
Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>
* Fix synopsis for PKI subcommand
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add transit command for synopsis, help text
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix nits around spacing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix Vault Transit BYOK helper argument parsing
This commit fixes the following issues with the importer:
- More than two arguments were not supported, causing the CLI to error
out and resulting in a failure to import RSA keys.
- The @file notation support was not accepted for KEY, meaning
unencrypted keys had to be manually specified on the CLI.
- Parsing of additional argument data was done in a non-standard way.
- Fix parsing of command line options and ensure only relevant
options are included.
Additionally, some error messages and help text was clarified.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing documentation on Transit CLI to website
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for Transit BYOK vault subcommand
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Appease CI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address some small issues within pki health-check
- Notify user yaml output mode is not support with --list argument
- Output pure JSON in json output mode with --list argument
- If a checker returns a nil response, convert to an empty slice
- Add handler for permission errors to too many certs checker
- Add checks for permission issues within hardware_backed_root and root_issued_leaves
* Identify the role that contained the permission issue in role based checks
- Augument the role health checks to identify the role(s) that we have
insufficient permissions to read instead of an overall read failure
- Treat the failure to list roles as a complete failure for the check
* Handle permission issue on pki health-check tune checkers
- Prior to this fix, if the end-user's Vault token did not have permission to the
mount's tune api, we would return as if the tunable params had not been set.
- Now check to see if we encountered a permission issue and report that back to
the end-user like the other checks do.
* Fix role endpoint in pki health-check warnings
- The various warning messages point to {{mount}}/role/<rolename>
which is not a valid PKI path, it should be {{mount}}/roles/<rolename>
* Add cl
* Output default config output from health-check --list as json
- Change the output of the default configuration as JSON so
it's useable as an input to the health-check command
* Add cl
* update error message and properly handle list requests
* since we do agressive sanitizes we need to optionally check trailing slash
* added changelog record
* remove redundant path formating
* Update changelog/13106.txt
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* addressed comments from review
* also remove code that duplicates efforts in kv_list
* abstracted helper func for testing
* added test cases for the policy builder
* updated the changelog to the correct one
* removed calls that apear not to do anything given test case results
* fixed spacing issue in output string
* remove const representation of list url param
* addressed comments for pr
---------
Co-authored-by: lursu <leland.ursu@hashicorp.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* pki health-check fails to read in int config values
- Go's default behavior when decoding numbers to an interface{} is to use a float64 type which parseutil.SafeParseIntRange does not handle.
- Switch to having the JSON decoder use json.Number which our parseutil library
properly handles.
* Add cl
* Allow listing health checks without mount path
This allows the bare:
$ vault pki health-check -list
without a corresponding mount path to complete. Otherwise, users would
be greeted with a prompt for the mount, which is less than ideal.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix usage, use <mount> over pki
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename files to match test suite and existing pattern
* Factor out issuer loading into a dedicated function
- Add a little more checks/validation when loading the a PKI issuer
- Factor out the issuer loading into a dedicated function
- Leverage existing health check code to parse issuer certificates
* Read parent issuer once instead of reloading it for every child
- Read in our parent issuer once instead of running it for every child
we want to compare against
- Provides clearer error message that we have failed reading from which
path to the end user
* PR Feedback
- Rename a variable for clarity
- Use readIssuer in the validation of the parent issuer within
pkiIssuer
- Add some missing return 1 statements in error handlers that had been
missed
* The verify-sign command in it's cleanest existing form.
* Working state
* Updates to proper verification syntax
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
* make fmt
* Base functionality.
* make fmt; changelog
* pki issue command.
* Make fmt. Changelog.
* Error Handling Is Almost A Tutorial
* Issue and ReIssue are Almost the Same Command
* Make Fmt + Changelog.
* Make some of the tests go.
* make fmt
* Merge fix (take 2)
* Fix existing support, add support for use_pss, max_path_length, not_after, permitted_dns_domains and skid
* Good Test which Fails
* Test-correction.
* Fix update to key_type key_bits; allow "," in OU or similar
* More specific includeCNinSANs
* Add tests around trying to use_pss on an ec key.
* GoDoc Test Paragraph thing.
---------
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.
This can be used by either `wscat` or the new
`vault events subscribe`:
e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
and
```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
- Match the existing vault kv capitalization scheme for Synopsis help of each sub-command.
- A few small tweaks as well to the messages text in a few cases
* Revert "Don't execute the seal recovery tests on ENT. (#18841)"
This reverts commit 990d3bacc2.
* Revert "Add the ability to unseal using recovery keys via an explicit seal option. (#18683)"
This reverts commit 2ffe49aab0.
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package. Consolidate on a single "noop audit" implementation.
* Revert "Remove t.Parallel() due to initialization race (#18751)"
This reverts commit ebcd653102.
We're going to fix this properly, running initCommands exactly once.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Prevent parallel testing racing in initCommands(...)
When running initCommands(...) from multiple tests, they can potentially
race, causing a panic. Test callers needing to set formatting
information must use RunCustom(...) instead of directly invoking the
test backend directly. When using t.Parallel(...) in these top-level
tests, we thus could race.
This removes the Commands global variable, making it a local variable
instead as nothing else appears to use it. We'll update Enterprise to
add in the Enterprise-specific commands to the existing list.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* wip
* Transit byok cli
* It works!
* changelog
* document return codes
* Update command/transit_import_key.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* make fmt
---------
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* The verify-sign command in it's cleanest existing form.
* Working state
* Updates to proper verification syntax
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
* make fmt
* Git CI caught some stuff.
* Base functionality.
* make fmt; changelog
* pki issue command.
* Make fmt. Changelog.
* Error Handling Is Almost A Tutorial
* What I thought empty issuers response fix would be.
* Some tests
* PR-review updates.
* make fmt.
* Fix null response data for listing empty issuers causing a crash.
* Update command/pki_list_children_command.go
Fix double specifier
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add test for pki_list_children.
* Fix tests.
* Update descriptions for correctness based on PR reviews.
* make fmt.
* Updates based on PR feedback.
* Allow multiple arguements (space separated)
* Remove bad merge-thing.
* White-space hell fix change.
* Tests, and return information for issue ca
* Fix make fmt error introduced here: https://github.com/hashicorp/vault/pull/18876
* Update command/pki_issue_intermediate.go
Puncutation.
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove smart quotes for standard quotes.
* More information as part of the help text.
* Better help text.
* Add missing "/" into error message.
---------
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Reflow pki list-intermediates help text, add args
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Reflow pki verify-sign help text, add args
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Simplify boolean checks across PKI commands
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clean up pki list-intermediate arg text
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename list_children->list_intermediate to align with command
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Base functionality.
* make fmt; changelog
* What I thought empty issuers response fix would be.
* Fix null response data for listing empty issuers causing a crash.
* Update command/pki_list_children_command.go
Fix double specifier
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add test for pki_list_children.
* Fix tests.
* Update descriptions for correctness based on PR reviews.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Shadow globally defined var to avoid race
output is defined in format_test.go as a global variable, which the
login_test.go tests were unknowingly using. Since these tests execute in
parallel, under the correct circumstances, they'd race to write/read
from the same variable in separate tests.
Shadow to avoid the race.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove global output variable from tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* The verify-sign command in it's cleanest existing form.
* Working state
* Updates to proper verification syntax
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
* make fmt
* Git CI caught some stuff.
* Some tests
* PR-review updates.
* make fmt.
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
* Parallel migration (#18815)
* flagParallel sanity check
* Attempt to use ErrGroups
* Updated docs
* Allow 'start' and 'max-parallel' together
* parallel flag renamed to max-parallel
* tests for start + parallel
* Removed permit pool
* Updated docs to make it clearer that a high setting might not be honored based on storage backend setting
* System dependent max int size
* Default max-parallel 1 => 10
* Test folder/paths updated
Co-authored-by: Tomasz Pawelczak <10206601+gites@users.noreply.github.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
* wip
* wip
* Got it 'working', but not happy about cleanliness yet
* Switch to a dedicated defaultSeal with recovery keys
This is simpler than trying to hijack SealAccess as before. Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal. Then everything proceeds
as if you had a shamir seal to begin with.
* Handle recovery rekeying
* changelog
* Revert go.mod redirect
* revert multi-blob info
* Dumb nil unmarshal target
* More comments
* Update vault/seal.go
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Update changelog/18683.txt
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* pr feedback
* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split
* Better comment on recovery seal during adjustSealMigration
* Make it possible to migrate from an auto-seal in recovery mode to shamir
* Fix sealMigrated to account for a recovery seal
* comments
* Update changelog/18683.txt
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Address PR feedback
* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate
* Don't shortcut the reast of seal migration
* get rid of redundant transit server cleanup
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* named MFA method configurations
* fix a test
* CL
* fix an issue with same config name different ID and add a test
* feedback
* feedback on test
* consistent use of passcode for all MFA methods (#18611)
* make use of passcode factor consistent for all MFA types
* improved type for MFA factors
* add method name to login CLI
* minor refactoring
* only accept MFA method name with its namespace path in the login request MFA header
* fix a bug
* fixing an ErrorOrNil return value
* more informative error message
* Apply suggestions from code review
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* feedback
* test refactor a bit
* adding godoc for a test
* feedback
* remove sanitize method name
* guard a possbile nil ref
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Using RunCommand(...) to set format to JSON for PKI HC tests results in
multiple initCommands(...) being called, overwriting the same global
variable. Nobody else calls the test suite in this way, so remove
t.Parallel() as the CLI isn't really meant to be called in parallel and
there might be other issues.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Removing the timeout logic from raw-response functions and adding documentation comments. The following functions are affected:
- `ReadRaw`
- `ReadRawWithContext` (newly added)
- `ReadRawWithData`
- `ReadRawWithDataWithContext`
The previous logic of using `ctx, _ = c.c.withConfiguredTimeout(ctx)` could cause a potential [context leak](https://pkg.go.dev/context):
> Failing to call the CancelFunc leaks the child and its children until the parent is canceled or the timer fires. The go vet tool checks that CancelFuncs are used on all control-flow paths.
Cancelling the context would have caused more issues since the context would be cancelled before the request body is closed.
Resolves: #18658
* integer values for some log flags
* Adjusted `log_flags` to expect `int` for max files and max bytes
* Updated `server` and `agent`
Renamed updateConfig (and updateLogConfig)
* Added int log params to test
* Adjust config/params so we can identify when they're not present
* Removed pointer confusion
* add core state lockd eadlock detection config option v2
* add changelog
* split out NewTestCluster function to maintain build flag
* replace long func with constant
* remove line
* rename file, and move where detect deadlock flag is set
* Add documentation on vault pki health-check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refer users to online docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update command/agent.go
* Attempt to only reload log level and certs
* Mimicked 'server' test for cert reload in 'agent'
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Left out the `c.config` tweak that meant changes to lots of lines of code within the `Run` function of Agent command. :)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
* add Link config, init, and capabilities
* add node status proto
* bump protoc version to 3.21.9
* make proto
* adding link tests
* remove wrapped link
* add changelog entry
* update changelog entry
Move version out of SDK. For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString. That will have to wait until we've removed uses of useragent.String from all builtins.
* VAULT-11510 Vault Agent can start listeners without caching
* VAULT-11510 fix order of imports
* VAULT-11510 changelog
* VAULT-11510 typo and better switch
* VAULT-11510 update name
* VAULT-11510 New api_proxy stanza to configure API proxy
* VAULT-11510 First pass at API Proxy docs
* VAULT-11510 nav data
* VAULT-11510 typo
* VAULT-11510 docs update
* Respond with data to all writes in PKI engine
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Create global quotas of each type in every NewTestCluster. Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.
NewTestCluster also now starts the cluster, and the Start method becomes a no-op. Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode. This was needed because otherwise we can't safely make the quota api call. We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start.
The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint. The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http. The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
* Work to unify log-file for agent/server and add rotation
* Updates to rotation code, tried to centralise the log config setup
* logging + tests
* Move LogFile to ShareConfig in test
* Docs
* Rename common.go->healthcheck.go
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Push handling of no resources to the health checks
This allows us to better run on empty mounts.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Exit when no issuers are found
This makes health checks less useful.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional test criteria, refactor tests
This will allow us to setup more tests.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add more OK statuses when checks are good
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test cases for all bad results
The test for too-many-certs was elided for now due to being too hard to
setup in CI.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test for missing mount
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add expected failure test on empty mount
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test for only having an issuer in the mount
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* More consistently perform permission checks
Also return them to the caller when they're relevant.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test without token
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Run health check tests in parallel
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update command/healthcheck/healthcheck.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Update command/healthcheck/healthcheck.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add shared helpers across health checks
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add audit_visibility health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add allow_if_modified_since health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add mount-related health checks to CLI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename fetch helpers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Soften language around managed key roots
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add enable_auto_tidy health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tidy_last_run health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add too_many_certs health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tidy, CRL, cert count checks to CLI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Cache stored leaf cert count
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly parse last run
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add more PKI related common utilities
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add role_allows_localhost health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add role_allows_glob_wildcards health checks
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add role_no_store_false health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new checks to the CLI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Switch to new guard style
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Provision role for test
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix invalid version check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix message with auto-rebuild enabled
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
We previously tried to extract this log into a function (shouldExit),
but semgrep doesn't expand function invocations, leading us to be forced
to add another rule to the regex.
Instead, add the extraneous `err != nil` conditional into the if
statements, even though skip/err should always be true in these cases
and it should never be evaluated.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add hardware_backed_root health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add root_issued_leaves health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new health checks to CLI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add more helpers to common PKI health-check code
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Show config when listing, stable output order
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix %v->%w
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Stub out initial health check command
This command will be used to generate health check results for the PKI
engine.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Start common health check implementation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add common health check utilities
These utilities will collect helpers not specific to PKI health checks,
such as formatting longer durations more legibly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add PKI health check common utils
Many health checks will need issuer and/or CRL information in order to
execute. We've centrally located these helpers to avoid particular
health checks from needing to reimplement them each time.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Adding ca_validity_period health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Begin using health-checks in PKI command
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow parsing raw requests afterwards
This shifts the last of the logic difference between Read(...) and
ReadRaw(...) to a new helper, allowing ReadRaw(...) requests to be
parsed into the same response structure afterwards as Read(...); this
allows API callers to fetch the raw secret and inspect the raw response
object in case something went wrong (error code &c) -- and when the
request succeeds, they can still get the api.Secret out.
This will be used with the PKI health check functionality, making both
LIST and READ operations use ReadRaw, and optionally parsing the secret
afterwards.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add crl_validity_period health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for PKI health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix bug in raw reading with contexts
When reading raw objects, don't manually call the context cancellation:
this causes timeouts and/or EOF errors when attempting to read or parse
the response body. See message in client.RawRequestWithContext(...) for
more information.
This was causing the test suite to randomly fail, due to the context
cancelling. The test suite's client usually had a default timeout,
whereas the CLI didn't, and thus didn't exhibit the same issue.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix typo in permissions message
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move %v->%w for errs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Export CreateBackendWithStorage for pkiext
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move zlint_test.go to pkiext
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix mount all test to ignore pkiext
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Started work on adding log-file support to Agent
* Allow log file to be picked up and appended
* Use NewLogFile everywhere
* Tried to pull out the config aggregation from Agent.Run
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
When running the test suite in CI (where requests are centralized from
relatively few IPs), we'd occasionally hit Dockerhub's rate limits.
Luckily Hashicorp runs a (limited) public mirror of the containers we
need, so we can switch to them here in the tests.
For consistency between developer and CI, we've opted to have the tests
always pull from the Hashicorp mirror, rather than updating the CI
runner to prefer the mirror.
We exclude nomad and influxdb as we don't presently mirror these repos.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify when -format=raw fails
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Document Vault read's new -format=raw mode
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add raw format to usage, completion
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing support for raw format field printing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Prohibit command execution with wrong formatter
This allows us to restrict the raw formatter to only commands that
understand it; otherwise, when running `vault write -format=raw`, we'd
actually hit the Vault server, but hide the output from the user. By
switching this to a flag-parse time check, we avoid running the rest of
the command if a bad formatter was specified.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Expose raw request from client.Logical()
Not all Vault API endpoints return well-formatted JSON objects.
Sometimes, in the case of the PKI secrets engine, they're not even
printable (/pki/ca returns a binary (DER-encoded) certificate). While
this endpoint isn't authenticated, in general the API caller would
either need to use Client.RawRequestWithContext(...) directly (which
the docs advise against), or setup their own net/http client and
re-create much of Client and/or Client.Logical.
Instead, exposing the raw Request (via the new ReadRawWithData(...))
allows callers to directly consume these non-JSON endpoints like they
would nearly any other endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add raw formatter for direct []byte data
As mentioned in the previous commit, some API endpoints return non-JSON
data. We get as far as fetching this data (via client.Logical().Read),
but parsing it as an api.Secret fails (as in this case, it is non-JSON).
Given that we intend to update `vault read` to support such endpoints,
we'll need a "raw" formatter that accepts []byte-encoded data and simply
writes it to the UI.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for reading raw API endpoints
Some endpoints, such as `pki/ca` and `pki/ca/pem` return non-JSON
objects. When calling `vault read` on these endpoints, an error
is returned because they cannot be parsed as api.Secret instances:
> Error reading pki/ca/pem: invalid character '-' in numeric literal
Indeed, we go to all the trouble of (successfully) fetching this value,
only to be unable to Unmarshal into a Secrets value. Instead, add
support for a new -format=raw option, allowing these endpoints to be
consumed by callers of `vault read` directly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove panic
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Store login MFA secret with tokenhelper
* Clean up and refactor tokenhelper paths
* Refactor totp test code for re-use
* Add login MFA command tests
* Use longer sleep times and sha512 for totp test
* Add changelog
Vault's new TLS devvault mode has two nits with certificate
construction:
1. The CA doesn't need to include any SANs, as these aren't checked.
Technically this means the CA could be reused as a leaf certificate
for the one specified IP SAN, which is less desirable.
2. Add hostname to SANs in addition to CNs. This is a best practice, as
(when the CN is a hostname), it is preferable to have everything in
SANs as well.
Neither of these are major changes.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add patch support to CLI
This is based off the existing write command, using the
JSONMergePatch(...) API client method rather than Write(...), allowing
us to update specific fields.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on PATCH support
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
This fixes a couple of references to loop variables in parallel tests
and deferred functions. When running a parallel test (calling
`t.Parallel()`) combined with the table-driven pattern, it's necessary
to copy the test case loop variable, otherwise only the last test case
is exercised. This is documented in the `testing` package:
https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks
`defer` statements that invoke a closure should also not reference a
loop variable directly as the referenced value will change in each
iteration of the loop.
Issues were automatically found with the `loopvarcapture` linter.
* Added flag and env var which will disable client redirection
* Added changelog
* Docs fix for unsaved file, and test single request made
* Updated test for case when redirect is enabled, updated docs based on suggestions
* Write explicit -help output to stdout
Per the consensus of most programs, and mirroring the GNU Coding
Standards for CLI design, when users request -help explicitly via the
CLI, this should be written to stdout to allow paging of output. stderr
is fine when an invalid usage triggers the help text however.
In our case, mitchellh/cli helpfully adds a HelpWriter that we
previously set to stderr explicitly. This writer is only called to print
user-requested help text; it is not called on error cases (e.g., bad
usage triggering additional help text to the user).
Thus it should safely be settable to stdout, enabling pagers without
additional redirects.
Notably, we do have to set ErrorWriter as it defaults to initializing to
the value of HelpWriter, which we want to set to stdout now, which is
less useful.
See also: https://www.gnu.org/prep/standards/html_node/_002d_002dhelp.htmlResolves: #17004
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add -plugin-version flag to vault auth/secrets tune
* CLI tests for auth/secrets tune
* CLI test for plugin register
* Plugin catalog listing bug where plugins of different type with the same name could be double counted
* Use constant for -plugin-version flag name
* core: Handle deprecated mounts on enable and unseal
* changelog: Deprecation Status handling
* core: Add Pending Removal override var
* core: Add some documentation for Pending Removal override
* HCP link integration
* update configure-git.yml
* more OSS stuff
* removing internal repos
* adding a nil check
* removing config test to be included in ENT only
* updating hcp-sdk-go to v0.22.0
* remove Hostname and AuthURL link config params
Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
* auth: Add Deprecation Status to auth list -detailed
* secrets: Add Deprecation Status to secrets list -detailed
* Add changelog entry for deprecation status list
* OSS portion of wrapper-v2
* Prefetch barrier type to avoid encountering an error in the simple BarrierType() getter
* Rename the OveriddenType to WrapperType and use it for the barrier type prefetch
* Fix unit test
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Added a small utility method to display warnings when parsing command arguments
Will print warning if flag is passed after arguments e.g.
vault <command> -a b -c
In this example -c will be interpreted as an argument which may be misleading
Add a new config option for Vault Agent's JWT auto auth
`remove_jwt_after_reading`, which defaults to true. Can stop
Agent from attempting to delete the file, which is useful in k8s
where the service account JWT is mounted as a read-only file
and so any attempt to delete it generates spammy error logs.
When leaving the JWT file in place, the read period for new
tokens is 1 minute instead of 500ms to reflect the assumption
that there will always be a file there, so finding a file does not
provide any signal that it needs to be re-read. Kubernetes
has a minimum TTL of 10 minutes for tokens, so a period of
1 minute gives Agent plenty of time to detect new tokens,
without leaving it too unresponsive. We may want to add a
config option to override these default periods in the future.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* add func to set level for specific logger
* add endpoints to modify log level
* initialize base logger with IndependentLevels
* test to ensure other loggers remain unchanged
* add DELETE loggers endpoints to revert back to config
* add API docs page
* add changelog entry
* remove extraneous line
* add log level field to Core struct
* add godoc for getLogLevel
* add some loggers to c.allLoggers
This was causing failures when running `vault server -dev`:
> panic: runtime error: invalid memory address or nil pointer dereference
> [signal SIGSEGV: segmentation violation code=0x2 addr=0x20 pc=0x105c41c1c]
>
> goroutine 1 [running]:
> github.com/hashicorp/vault/command.(*ServerCommand).parseConfig(0x140005a2180)
> .../vault/command/server.go:429 +0x5c
Interestingly, we do not have a test case for running the dev
sever.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning about EA in FIPS mode
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* CLI: Add ability to display ListResponseWithInfos
The Vault Server API includes a ListResponseWithInfo call, allowing LIST
responses to contain additional information about their keys. This is in
a key=value mapping format (both for each key, to get the additional
metadata, as well as within each metadata).
Expand the `vault list` CLI command with a `-detailed` flag (and env var
VAULT_DETAILED_LISTS) to print this additional metadata. This looks
roughly like the following:
$ vault list -detailed pki/issuers
Keys issuer_name
---- -----------
0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7 n/a
35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0 n/a
382fad1e-e99c-9c54-e147-bb1faa8033d3 n/a
8bb4a793-2ad9-460c-9fa8-574c84a981f7 n/a
8bd231d7-20e2-f21f-ae1a-7aa3319715e7 n/a
9425d51f-cb81-426d-d6ad-5147d092094e n/a
ae679732-b497-ab0d-3220-806a2b9d81ed n/a
c5a44a1f-2ae4-2140-3acf-74b2609448cc utf8
d41d2419-efce-0e36-c96b-e91179a24dc1 something
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow detailed printing of LIST responses in JSON
When using the JSON formatter, only the absolute list of keys were
returned. Reuse the `-detailed` flag value for the `-format=json` list
response printer, allowing us to show the complete API response returned
by Vault.
This returns something like the following:
{
"request_id": "e9a25dcd-b67a-97d7-0f08-3670918ef3ff",
"lease_id": "",
"lease_duration": 0,
"renewable": false,
"data": {
"key_info": {
"0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7": {
"issuer_name": ""
},
"35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0": {
"issuer_name": ""
},
"382fad1e-e99c-9c54-e147-bb1faa8033d3": {
"issuer_name": ""
},
"8bb4a793-2ad9-460c-9fa8-574c84a981f7": {
"issuer_name": ""
},
"8bd231d7-20e2-f21f-ae1a-7aa3319715e7": {
"issuer_name": ""
},
"9425d51f-cb81-426d-d6ad-5147d092094e": {
"issuer_name": ""
},
"ae679732-b497-ab0d-3220-806a2b9d81ed": {
"issuer_name": ""
},
"c5a44a1f-2ae4-2140-3acf-74b2609448cc": {
"issuer_name": "utf8"
},
"d41d2419-efce-0e36-c96b-e91179a24dc1": {
"issuer_name": "something"
}
},
"keys": [
"0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7",
"35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0",
"382fad1e-e99c-9c54-e147-bb1faa8033d3",
"8bb4a793-2ad9-460c-9fa8-574c84a981f7",
"8bd231d7-20e2-f21f-ae1a-7aa3319715e7",
"9425d51f-cb81-426d-d6ad-5147d092094e",
"ae679732-b497-ab0d-3220-806a2b9d81ed",
"c5a44a1f-2ae4-2140-3acf-74b2609448cc",
"d41d2419-efce-0e36-c96b-e91179a24dc1"
]
},
"warnings": null
}
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use field on UI rather than secret.Data
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Only include headers from visitable key_infos
Certain API endpoints return data from non-visitable key_infos, by
virtue of using a hand-rolled response. Limit our headers to those
from visitable key_infos. This means we won't return entire columns with
n/a entries, if no key matches the key_info key that includes that
header.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use setupEnv sourced detailed info
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix changelog environment variable
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix broken tests using setupEnv
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* VAULT-5885: Fix erroneous success message in case of two-phase MFA, and provide MFA information in table format
* VAULT-5885 Add changelog
* VAULT-5885 Update changelog as per PR comments
* VAULT-5885 Update changelog category to just 'auth'
* VAULT-5885 Hide useless token info in two-phase MFA case
* VAULT-5885 Update changelog to reflect token info now no longer present
* VAULT-5885 split up changelog into three blocks
* Use new parseutil helper: Safe variants
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update parseutil to v0.1.5
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix additional integer overflow in command/server
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Vault agent redacts the token and accessor for `/auth/token/lookup-self` (and `lookup`)
if the token is the auto auth token to prevent it from leaking.
Similarly, we need to redact the token and accessor from `renew-self`
and `renew`, which also leak the token and accessor.
I tested this locally by starting up a Vault agent and querying the
agent endpoints, and ensuring that the accessor and token were set to
the empty string in the response.
* Add server information as well as ability to collect metrics from DR secondary
* Update debug docs
Adding additional information around ability to gather metrics from DR secondary
* Fix broken link in updated doc
* Create 15316.txt
Create changelog entry
* Fix Formatting
* Update website/content/docs/commands/debug.mdx
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update changelog/15316.txt
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Trigger Build
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
This requires bumping https://github.com/mitchellh/go-testing-interface.
For this new version, we have to create a wrapper to convert
the stdlib `testing.TB` interface to the
`mitchellh/go-testing-interface` `T` interface, since it uses
`Parallel()` now, which is not supported by `testing.TB`. This had to be
added to a new package, `benchhelpers`, to avoid a circular dependency
in `testhelpers`.
We also have to *unbump* https://github.com/armon/go-metrics since
updating it breaks our usage of
https://github.com/google/go-metrics-stackdriver
I verified that the new `pkiCert` template function works with agent
injection using annotations like:
```yaml
vault.hashicorp.com/agent-inject-secret-sample.crt: "pki/issue/example-dot-com"
vault.hashicorp.com/agent-inject-template-sample.crt: |
{{ pkiCert "pki/issue/example-dot-com" "common_name=foo.example.com" "ttl=1h" }}
```
* WIP: output policy
* Outputs example policy HCL for given request
* Simplify conditional
* Add PATCH capability
* Use OpenAPI spec and regex patterns to determine if path is sudo
* Add test for isSudoPath
* Add changelog
* Fix broken CLI tests
* Add output-policy to client cloning code
* Smaller fixes from PR comments
* Clone client instead of saving and restoring custom values
* Fix test
* Address comments
* Don't unset output-policy flag on KV requests otherwise the preflight request will fail and not populate LastOutputPolicyError
* Print errors saved in buffer from preflight KV requests
* Unescape characters in request URL
* Rename methods and properties to improve readability
* Put KV-specificness at front of KV-specific error
* Simplify logic by doing more direct returns of strings and errors
* Use precompiled regexes and move OpenAPI call to tests
* Remove commented out code
* Remove legacy MFA paths
* Remove unnecessary use of client
* Move sudo paths map to plugin helper
* Remove unused error return
* Add explanatory comment
* Remove need to pass in address
* Make {name} regex less greedy
* Use method and path instead of info from retryablerequest
* Add test for IsSudoPaths, use more idiomatic naming
* Use precompiled regexes and move OpenAPI call to tests (#15170)
* Use precompiled regexes and move OpenAPI call to tests
* Remove commented out code
* Remove legacy MFA paths
* Remove unnecessary use of client
* Move sudo paths map to plugin helper
* Remove unused error return
* Add explanatory comment
* Remove need to pass in address
* Make {name} regex less greedy
* Use method and path instead of info from retryablerequest
* Add test for IsSudoPaths, use more idiomatic naming
* Make stderr writing more obvious, fix nil pointer deref
* add BuildDate to version base
* populate BuildDate with ldflags
* include BuildDate in FullVersionNumber
* add BuildDate to seal-status and associated status cmd
* extend core/versions entries to include BuildDate
* include BuildDate in version-history API and CLI
* fix version history tests
* fix sys status tests
* fix TestStatusFormat
* remove extraneous LD_FLAGS from build.sh
* add BuildDate to build.bat
* fix TestSysUnseal_Reset
* attempt to add build-date to release builds
* add branch to github build workflow
* add get-build-date to build-* job needs
* fix release build command vars
* add missing quote in release build command
* Revert "add branch to github build workflow"
This reverts commit b835699ecb.
* add changelog entry
* [VAULT-1618] Agent error log level is mismatched
`logLevelToStringPtr` translates `go-hclog`'s `ERROR` to `"ERROR"` for
Consul Template's runner, but that expects `ERR` and is quite strict
about it.
This will address https://github.com/hashicorp/vault-k8s/issues/223
after it is set as the default image in `vault-k8s`.
I didn't find a simple way to test this other than starting up a full
server and agent and letting them run, which is unfortunately fairly
slow.
I confirmed that this addresses the original issue by modifying the helm
chart with the values in this commit and patching the log level to `err`.
* VAULT-1618 Add changelog/14424.txt
* VAULT-1618 Update changelog/14424.txt based on @kalafut suggestion
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
* VAULT-1618 Move cancel and server stop into defer in tests
* VAULT-1618 Triggering CircleCI tests
* VAULT-1618 Replace ioutil with os functions for agent template tests
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
* Add explanation to help text and flag usage text
* KV get with new mount flag
* Clearer naming
* KV Put, Patch, Metadata Get + corresponding tests
* KV Delete, Destroy, Rollback, Undelete, MetadataDelete, MetadataPatch, MetadataPut
* Update KV-v2 docs to use mount flag syntax
* Add changelog
* Run make fmt
* Clarify deprecation message in help string
* Address style comments
* Warn on upper case in policy name
* Rename name variable to be less confusing
* Use more general solution for other string issues
* Clarify changelog
* Remove unnecessary check
* Don't throw CLI warning until after past errors
* Add before and after names with quotes to show spacing changes
* Update mholt/archiver to v3.5.0
* Bump archiver to 3.5.1
* Vendor dependencies
* Use newer go
* go mod tidy
* Remove vendor
* Rm vendor
* Revert api and sdk sums
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
These options must be specified multiple times in order to be properly
parsed. However, the present description suggests that a comma-separated
list would work as well, however this isn't the case and results in a
slice containing a single string (with all comma-separated values) in
the API request. Clarify the argument help text to make this clearer.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Full secret path in table output of get and put
* Add path output to KV patch and metadata get
* Add changelog
* Don't print secret path for kv-v1
* Make more readable
* Switch around logic to not swallow error
* Add test for secret path
* Fix metadata test
* Add unit test for padequalsigns
* Remove wonky kv get tests
* Add support for PROXY protocol v2 in TCP listener
I did not find tests for this so I added one trying to cover different
configurations to make sure I did not break something. As far as I know,
the behavior should be exactly the same as before except for one thing
when proxy_protocol_behavior is set to "deny_unauthorized", unauthorized
requests were previously silently reject because of https://github.com/armon/go-proxyproto/blob/7e956b284f0a/protocol.go#L81-L84
but it will now be logged.
Also fixes https://github.com/hashicorp/vault/issues/9462 by adding
support for `PROXY UNKNOWN` for PROXY protocol v1.
Closes https://github.com/hashicorp/vault/issues/3807
* Add changelog
* Login MFA
* ENT OSS segragation (#14088)
* Delete method id if not used in an MFA enforcement config (#14063)
* Delete an MFA methodID only if it is not used by an MFA enforcement config
* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path
* adding use_passcode field to DUO config (#14059)
* add changelog
* preventing replay attack on MFA passcodes (#14056)
* preventing replay attack on MFA passcodes
* using %w instead of %s for error
* Improve CLI command for login mfa (#14106)
CLI prints a warning message indicating the login request needs to get validated
* adding the validity period of a passcode to error messages (#14115)
* interactive CLI for mfa login
* minor fixes
* bail if no input was inserted
* change label name
* interactive CLI when single methodID is returned from login request
* minor fix
* adding changelog
* addressing feedback
* a user with a terminal should be able to choose between interactive and non-interactive. A user without a terminal should not be able to use the interactive mode.
Co-authored-by: Josh Black <raskchanky@gmail.com>
* agent/azure: adds ability to use specific user assigned managed identity for auto auth
* add changelog
* change wording in error and docs
* Update website/content/docs/agent/autoauth/methods/azure.mdx
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Update website/content/docs/agent/autoauth/methods/azure.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* docs formatting
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
This patch adds a new /agent/v1/metrics that will return metrics on the
running Vault agent. Configuration is done using the same telemetry
stanza as the Vault server. For now default runtime metrics are
returned with a few additional ones specific to the agent:
- `vault.agent.auth.failure` and `vault.agent.auth.success` to monitor
the correct behavior of the auto auth mechanism
- `vault.agent.proxy.success`, `vault.agent.proxy.client_error` and
`vault.agent.proxy.error` to check the connection with the Vault server
- `vault.agent.cache.hit` and `vault.agent.cache.miss` to monitor the
cache
Closes https://github.com/hashicorp/vault/issues/8649
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Login MFA
* ENT OSS segragation (#14088)
* Delete method id if not used in an MFA enforcement config (#14063)
* Delete an MFA methodID only if it is not used by an MFA enforcement config
* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path
* adding use_passcode field to DUO config (#14059)
* add changelog
* preventing replay attack on MFA passcodes (#14056)
* preventing replay attack on MFA passcodes
* using %w instead of %s for error
* Improve CLI command for login mfa (#14106)
CLI prints a warning message indicating the login request needs to get validated
* adding the validity period of a passcode to error messages (#14115)
* PR feedback
* duo to handle preventing passcode reuse
Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
* port SSCT OSS
* port header hmac key to ent and generate token proto without make command
* remove extra nil check in request handling
* add changelog
* add comment to router.go
* change test var to use length constants
* remove local index is 0 check and extra defer which can be removed after use of ExternalID
In future Vault Enterprise versions, we'll be building Vault with
FIPS-validated cryptography. To help operators understand their
environment, we'll want to expose information about their FIPS status
when they're running a FIPS version of Vault.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* adds development workflow to mirage config
* adds mirage handler and factory for mfa workflow
* adds mfa handling to auth service and cluster adapter
* moves auth success logic from form to controller
* adds mfa form component
* shows delayed auth message for all methods
* adds new code delay to mfa form
* adds error views
* fixes merge conflict
* adds integration tests for mfa-form component
* fixes auth tests
* updates mfa response handling to align with backend
* updates mfa-form to handle multiple methods and constraints
* adds noDefault arg to Select component
* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios
* adds tests
* flaky test fix attempt
* reverts test fix attempt
* adds changelog entry
* updates comments for todo items
* removes faker from mfa mirage factory and handler
* adds number to word helper
* fixes tests
* Revert "Merge branch 'main' into ui/mfa"
This reverts commit 8ee6a6aaa1, reversing
changes made to 2428dd6cca.
* format-ttl helper fix from main
* store version history as utc; add self-heal logic
* add sys/version-history endpoint
* change version history from GET to LIST, require auth
* add "vault version-history" CLI command
* add vault-version CLI error message for version string parsing
* adding version-history API and CLI docs
* add changelog entry
* some version-history command fixes
* remove extraneous cmd args
* fix version-history command help text
* specify in docs that endpoint was added in 1.10.0
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* enforce UTC within storeVersionTimestamp directly
* fix improper use of %w in logger.Warn
* remove extra err check and erroneous return from loadVersionTimestamps
* add >= 1.10.0 warning to version-history cmd
* move sys/version-history tests
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Add documentation for Managed Keys
- Add concept, sys/api and pki updates related to managed keys
* Review feedback
- Reworked quite a bit of the existing documentation based on feedback
and a re-reading
- Moved the managed keys out of the concepts section and into the
enterprise section
* Address broken links and a few grammar tweaks
In doing some testing I found that the listener clusteraddr isn't really used, or at least isn't as important as the top-level clusteraddr setting. As such, go-sockaddr templating needs to be implemented for the top-level `cluster_addr` setting or it's unusable for HA.
Also fix a nil pointer panic I discovered at the same time.
* go get vault-plugin-secrets-kv@vault-4290-patch-metadata
* add kv metadata patch command
* add changelog entry
* success tests for kv metadata patch flags
* add more kv metadata patch flags tests
* add kv metadata patch cas warning test
* add kv-v2 key metadata patch API docs
* add kv metadata patch to docs
* prevent unintentional field overwriting in kv metadata put cmd
* like create/update ops, prevent patch to paths ending in /
* fix kv metadata patch cmd in docs
* fix flag defaults for kv metadata put
* go get vault-plugin-secrets-kv@vault-4290-patch-metadata
* fix TestKvMetadataPatchCommand_Flags test
* doc fixes
* go get vault-plugin-secrets-kv@master; go mod tidy
* CLI changes for new mount tune config parameter allowed_managed_keys
* Correct allowed_managed_keys description in auth and secrets
* Documentation update for secrets and removed changes for auth
* Add changelog and remove documentation changes for auth
* removed changelog
* Correct the field description
* VAULT-1564 report in-flight requests
* adding a changelog
* Changing some variable names and fixing comments
* minor style change
* adding unauthenticated support for in-flight-req
* adding documentation for the listener.profiling stanza
* adding an atomic counter for the inflight requests
addressing comments
* addressing comments
* logging completed requests
* fixing a test
* providing log_requests_info as a config option to determine at which level requests should be logged
* removing a member and a method from the StatusHeaderResponseWriter struct
* adding api docks
* revert changes in NewHTTPResponseWriter
* Fix logging invalid log_requests_info value
* Addressing comments
* Fixing a test
* use an tomic value for logRequestsInfo, and moving the CreateClientID function to Core
* fixing go.sum
* minor refactoring
* protecting InFlightRequests from data race
* another try on fixing a data race
* another try to fix a data race
* addressing comments
* fixing couple of tests
* changing log_requests_info to log_requests_level
* minor style change
* fixing a test
* removing the lock in InFlightRequests
* use single-argument form for interface assertion
* adding doc for the new configuration paramter
* adding the new doc to the nav data file
* minor fix
Changes the error output for the second open of the persistent cache
file, to differentiate it from the c.UI.Error message for the initial
open of the cache file, just to make it easier to tell where a problem
occurred.
- Add the kms_library configuration stanza to Vault's command/server
- Provide validation of keys and general configuration.
- Add initial kms_library configuration documentation
- Attempt at startup to verify we can read the configured HSM Library
- Hook in KmsLibrary config into the Validate to detect typo/unused keys
* See what it looks like to replace "master key" with "root key". There are two places that would require more challenging code changes: the storage path `core/master`, and its contents (the JSON-serialized EncodedKeyringtructure.)
* Restore accidentally deleted line
* Add changelog
* Update root->recovery
* Fix test
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Native Login method, userpass and approle interfaces to implement it
* Add AWS auth interface for Login, unexported struct fields for now
* Add Kubernetes client login
* Add changelog
* Add a test for approle client login
* Return errors from LoginOptions, use limited reader for secret ID
* Fix auth comment length
* Return actual type not interface, check for client token in tests
* Require specification of secret ID location using SecretID struct as AppRole arg
* Allow password from env, file, or plaintext
* Add flexibility in how to fetch k8s service token, but still with default
* Avoid passing strings that need to be validated by just having different login options
* Try a couple real tests with approle and userpass login
* Fix method name in comment
* Add context to Login methods, remove comments about certain sources being inherently insecure
* Perform read of secret ID at login time
* Read password from file at login time
* Pass context in integ tests
* Read env var values in at login time, add extra tests
* Update api version
* Revert "Update api version"
This reverts commit 1ef3949497.
* Update api version in all go.mod files
* go get vault-plugin-secrets-kv@extend-kv-metadata-to-get-and-put
* test for custom_metadata in kv get, put, patch command output
* remove flagFormat-specific check from TestKVMetadataGetCommand
* rewrite custom metadata changelog entry
* go get vault-plugin-secrets-kv@master
* go mod tidy
* CLI makes request to incorrect URL when namespace is both provided as argument and part of the path
fixes#12675
* adding change log
* removing a switch and addressing a possibility of out of bound index
* operator generate-root -decode: allow token from stdin
Allow passing "-" as the value for -decode, causing the encoded token to
be read from stdin. This is intended to prevent leaking the encoded
token + otp into process logs in enterprise environments.
* add changelog entry for PR12881
* add check/test for empty decode value passed via stdin
Uses a bufconn listener between consul-template and vault-agent when
caching is enabled and either templates or a listener is defined. This
means no listeners need to be defined in vault-agent for just
templating. Always routes consul-template through the vault-agent
cache (instead of only when persistent cache is enabled).
Uses a local transportDialer interface in config.Cache{}.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
* handle HTTP PATCH requests as logical.PatchOperation
* update go.mod, go.sum
* a nil response for logical.PatchOperation should result in 404
* respond with 415 for incorrect MIME type in PATCH Content-Type header
* add abstraction to handle PatchOperation requests
* add ACLs for patch
* Adding JSON Merge support to the API client
* add HTTP PATCH tests to check high level response logic
* add permission-based 'kv patch' tests in prep to add HTTP PATCH
* adding more 'kv patch' CLI command tests
* fix TestHandler_Patch_NotFound
* Fix TestKvPatchCommand_StdinValue
* add audit log test for HTTP PATCH
* patch CLI changes
* add patch CLI tests
* change JSONMergePatch func to accept a ctx
* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag
* go fmt
* add a test to verify patching works by default with the root token
* add changelog entry
* get vault-plugin-secrets-kv@add-patch-support
* PR feedback
* reorder some imports; go fmt
* add doc comment for HandlePatchOperation
* add json-patch@v5.5.0 to go.mod
* remove unnecessary cancelFunc for WriteBytes
* remove default for -method
* use stable version of json-patch; go mod tidy
* more PR feedback
* temp go get vault-plugin-secrets-kv@master until official release
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
* Customizing HTTP headers in the config file
* Add changelog, fix bad imports
* fixing some bugs
* fixing interaction of custom headers and /ui
* Defining a member in core to set custom response headers
* missing additional file
* Some refactoring
* Adding automated tests for the feature
* Changing some error messages based on some recommendations
* Incorporating custom response headers struct into the request context
* removing some unused references
* fixing a test
* changing some error messages, removing a default header value from /ui
* fixing a test
* wrapping ResponseWriter to set the custom headers
* adding a new test
* some cleanup
* removing some extra lines
* Addressing comments
* fixing some agent tests
* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before
* Removing default custom headers, and renaming some function varibles
* some refacotring
* Refactoring and addressing comments
* removing a function and fixing comments
* move merge and compare states to vault core
* move MergeState, CompareStates and ParseRequiredStates to api package
* fix merge state reference in API Proxy
* move mergeStates test to api package
* add changelog
* ghost commit to trigger CI
* rename CompareStates to CompareReplicationStates
* rename MergeStates and make compareStates and parseStates private methods
* improved error messaging in parseReplicationState
* export ParseReplicationState for enterprise files
Byte slices returned from Bolt are only valid during a transaction, so
this makes a copy.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* add custom-metdata flag to "kv metadata put" command
* add kv metadata put command test for custom-metadata flag
* add custom_metadata to kv-v2 api docs
* add custom_metadata to kv-v2 cli docs
* update go.mod
* Add custom metadata limits to docs
* add changelog entry
* update vault-plugin-secrets-kv to @master
* VAULT-2285 adding capability to accept comma separated entries for auth enable/tune
* Adding changelog
* Adding logic to detect invalid input parameter for auth enable config
* Updating tune.mdx
* Updating secret enable/tune for comma separated parameters
* Adding further parameter checks for auth/secret tests
Fixing changelog
using builtin type for a switch statement
Fixing a possible panic scenario
* Changing a function name, using deep.Equal instead of what reflect package provides
* Fixing auth/secret enable/tune mdx files
* One more mdx file fix
* Only when users provide a single comma separated string in a curl command, split the entries by commas
* Fixing API docs for auth/mount enable/tune for comma separated entries
* updating docs, removing an unnecessary switch case
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
* save
* save
* save
* first round of the diagnose language pass
* capitalization
* first round of feedback
* fix bug in advise
* a few more nouns to verbs
* Diagnose warns if HTTPS is not used for ha-storage-tls-consul
* Skipping TLS verification if https is not used in ha storage tls consul
* Adding diagnose skip message for consul service registration
* `vault delete` and `vault kv delete` should allow the same output options as `vault write`, as delete operations can similarly return data. This is needed if you want to use control groups with deletion.
* diagnose: Add seal transit tls check
* Fixing the path to the config file and the path to the cert files
* Addressing comment
* Addressing seal transit tls check comments
* tls verification bugfix
* tls verification bugfix
* allow diagnose fail to report status when there are also warnings
* allow diagnose fail to report status when there are also warnings
* Update vault/diagnose/helpers_test.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* comments
Co-authored-by: swayne275 <swayne275@gmail.com>
* Fix diagnose panic when configuration file does not exist
* Addressing comments
* Update command/operator_diagnose.go
Co-authored-by: Hridoy Roy <roy@hashicorp.com>
Co-authored-by: Hridoy Roy <roy@hashicorp.com>
* agent: restart template runner on retry for unlimited retries
* template: log error message early
* template: delegate retries back to template if param is set to true
* agent: add and use the new template config stanza
* agent: fix panic, fix existing tests
* changelog: add changelog entry
* agent: add tests for exit_on_retry_failure
* agent: properly check on agent exit cases, add separate tests for missing key vs missing secrets
* agent: add note on difference between missing key vs missing secret
* docs: add docs for template_config
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* docs: fix exit_on_retry_failure, fix Functionality section
* docs: update interaction title
* template: add internal note on behavior for persist case
* docs: update agent, template, and template-config docs
* docs: update agent docs on retry stanza
* Apply suggestions from code review
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Update changelog/11775.txt
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* agent/test: rename expectExit to expectExitFromError
* agent/test: add check on early exits on the happy path
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
