vault/api/ssh_agent_test.go

138 lines
3.9 KiB
Go
Raw Normal View History

// Copyright IBM Corp. 2016, 2025
// SPDX-License-Identifier: MPL-2.0
2015-08-18 22:00:27 -04:00
package api
import (
"fmt"
"net/http"
2016-03-10 16:47:46 -05:00
"strings"
2015-08-18 22:00:27 -04:00
"testing"
VAULT-32657 deprecate duplicate attributes in HCL configs and policies (#30386) * upgrade hcl dependency on api pkg This upgrades the hcl dependency for the API pkg, and adapts its usage so users of our API pkg are not affected. There's no good way of communicating a warning via a library call so we don't. The tokenHelper which is used by all Vault CLI commands in order to create the Vault client, as well as directly used by the login and server commands, is implemented on the api pkg, so this upgrade also affects all of those commands. Seems like this was only moved to the api pkg because the Terraform provider uses it, and I thought creating a full copy of all those files back under command would be too much spaghetti. Also leaving some TODOs to make next deprecation steps easier. * upgrade hcl dependency in vault and sdk pkgs * upgrade hcl dependency in vault and sdk pkgs * add CLI warnings to commands that take a config - vault agent (unit test on CMD warning) - vault proxy (unit test on CMD warning) - vault server (no test for the warning) - vault operator diagnose (no tests at all, uses the same function as vault server * ignore duplicates on ParseKMSes function * Extend policy parsing functions and warn on policy store * Add warning on policy fmt with duplicate attributes * Add warnings when creating/updating policy with duplicate HCL attrs * Add log warning when switchedGetPolicy finds duplicate attrs Following operations can trigger this warning when they run into a policy with duplicate attributes: * replication filtered path namespaces invalidation * policy read API * building an ACL (for many different purposes like most authZ operations) * looking up DR token policies * creating a token with named policies * when caching the policies for all namespaces during unseal * Print log warnings when token inline policy has duplicate attrs No unit tests on these as new test infra would have to be built on all. Operations affected, which will now print a log warning when the retrieved token has an inline policy with duplicate attributes: * capabilities endpoints in sys mount * handing events under a subscription with a token with duplicate attrs in inline policies * token used to create another token has duplicate attrs in inline policies (sudo check) * all uses of fetchACLTokenEntryAndEntity when the request uses a token with inline policies with duplicate attrs. Almost all reqs are subject to this * when tokens are created with inline policies (unclear exactly how that can happen) * add changelog and deprecation notice * add missing copywrite notice * fix copy-paste mistake good thing it was covered by unit tests * Fix manual parsing of telemetry field in SharedConfig This commit in the hcl library was not in the v1.0.1-vault-5 version we're using but is included in v1.0.1-vault-7: https://github.com/hashicorp/hcl/commit/e80118accb521e47bc5b93104bf46c67d89d2242 This thing of reusing when parsing means that our approach of manually re-parsing fields on top of fields that have already been parsed by the hcl annotation causes strings (maybe more?) to concatenate. Fix that by removing annotation. There's actually more occurrences of this thing of automatically parsing something that is also manually parsing. In some places we could just remove the boilerplate manual parsing, in others we better remove the auto parsing, but I don't wanna pull at that thread right now. I just checked that all places at least fully overwrite the automatically parsed field instead of reusing it as the target of the decode call. The only exception is the AOP field on ent but that doesn't have maps or slices, so I think it's fine. An alternative approach would be to ensure that the auto-parsed value is discarded, like the current parseCache function does note how it's template not templates * Fix linter complaints * Update command/base_predict.go Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com> * address review * remove copywrite headers * re-add copywrite headers * make fmt * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * undo changes to deprecation.mdx * remove deprecation doc * fix conflict with changes from main --------- Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com> Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
2025-05-23 15:02:07 -04:00
"github.com/stretchr/testify/require"
2015-08-18 22:00:27 -04:00
)
VAULT-32657 deprecate duplicate attributes in HCL configs and policies (#30386) * upgrade hcl dependency on api pkg This upgrades the hcl dependency for the API pkg, and adapts its usage so users of our API pkg are not affected. There's no good way of communicating a warning via a library call so we don't. The tokenHelper which is used by all Vault CLI commands in order to create the Vault client, as well as directly used by the login and server commands, is implemented on the api pkg, so this upgrade also affects all of those commands. Seems like this was only moved to the api pkg because the Terraform provider uses it, and I thought creating a full copy of all those files back under command would be too much spaghetti. Also leaving some TODOs to make next deprecation steps easier. * upgrade hcl dependency in vault and sdk pkgs * upgrade hcl dependency in vault and sdk pkgs * add CLI warnings to commands that take a config - vault agent (unit test on CMD warning) - vault proxy (unit test on CMD warning) - vault server (no test for the warning) - vault operator diagnose (no tests at all, uses the same function as vault server * ignore duplicates on ParseKMSes function * Extend policy parsing functions and warn on policy store * Add warning on policy fmt with duplicate attributes * Add warnings when creating/updating policy with duplicate HCL attrs * Add log warning when switchedGetPolicy finds duplicate attrs Following operations can trigger this warning when they run into a policy with duplicate attributes: * replication filtered path namespaces invalidation * policy read API * building an ACL (for many different purposes like most authZ operations) * looking up DR token policies * creating a token with named policies * when caching the policies for all namespaces during unseal * Print log warnings when token inline policy has duplicate attrs No unit tests on these as new test infra would have to be built on all. Operations affected, which will now print a log warning when the retrieved token has an inline policy with duplicate attributes: * capabilities endpoints in sys mount * handing events under a subscription with a token with duplicate attrs in inline policies * token used to create another token has duplicate attrs in inline policies (sudo check) * all uses of fetchACLTokenEntryAndEntity when the request uses a token with inline policies with duplicate attrs. Almost all reqs are subject to this * when tokens are created with inline policies (unclear exactly how that can happen) * add changelog and deprecation notice * add missing copywrite notice * fix copy-paste mistake good thing it was covered by unit tests * Fix manual parsing of telemetry field in SharedConfig This commit in the hcl library was not in the v1.0.1-vault-5 version we're using but is included in v1.0.1-vault-7: https://github.com/hashicorp/hcl/commit/e80118accb521e47bc5b93104bf46c67d89d2242 This thing of reusing when parsing means that our approach of manually re-parsing fields on top of fields that have already been parsed by the hcl annotation causes strings (maybe more?) to concatenate. Fix that by removing annotation. There's actually more occurrences of this thing of automatically parsing something that is also manually parsing. In some places we could just remove the boilerplate manual parsing, in others we better remove the auto parsing, but I don't wanna pull at that thread right now. I just checked that all places at least fully overwrite the automatically parsed field instead of reusing it as the target of the decode call. The only exception is the AOP field on ent but that doesn't have maps or slices, so I think it's fine. An alternative approach would be to ensure that the auto-parsed value is discarded, like the current parseCache function does note how it's template not templates * Fix linter complaints * Update command/base_predict.go Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com> * address review * remove copywrite headers * re-add copywrite headers * make fmt * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * undo changes to deprecation.mdx * remove deprecation doc * fix conflict with changes from main --------- Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com> Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
2025-05-23 15:02:07 -04:00
// TestSSH_CanLoadDuplicateKeys verifies that during the deprecation process of duplicate HCL attributes this function
// will still allow them.
// TODO (HCL_DUP_KEYS_DEPRECATION): on full removal change this test to ensure that duplicate attributes cannot be parsed
// under any circumstances.
VAULT-32657 deprecate duplicate attributes in HCL configs and policies (#30386) * upgrade hcl dependency on api pkg This upgrades the hcl dependency for the API pkg, and adapts its usage so users of our API pkg are not affected. There's no good way of communicating a warning via a library call so we don't. The tokenHelper which is used by all Vault CLI commands in order to create the Vault client, as well as directly used by the login and server commands, is implemented on the api pkg, so this upgrade also affects all of those commands. Seems like this was only moved to the api pkg because the Terraform provider uses it, and I thought creating a full copy of all those files back under command would be too much spaghetti. Also leaving some TODOs to make next deprecation steps easier. * upgrade hcl dependency in vault and sdk pkgs * upgrade hcl dependency in vault and sdk pkgs * add CLI warnings to commands that take a config - vault agent (unit test on CMD warning) - vault proxy (unit test on CMD warning) - vault server (no test for the warning) - vault operator diagnose (no tests at all, uses the same function as vault server * ignore duplicates on ParseKMSes function * Extend policy parsing functions and warn on policy store * Add warning on policy fmt with duplicate attributes * Add warnings when creating/updating policy with duplicate HCL attrs * Add log warning when switchedGetPolicy finds duplicate attrs Following operations can trigger this warning when they run into a policy with duplicate attributes: * replication filtered path namespaces invalidation * policy read API * building an ACL (for many different purposes like most authZ operations) * looking up DR token policies * creating a token with named policies * when caching the policies for all namespaces during unseal * Print log warnings when token inline policy has duplicate attrs No unit tests on these as new test infra would have to be built on all. Operations affected, which will now print a log warning when the retrieved token has an inline policy with duplicate attributes: * capabilities endpoints in sys mount * handing events under a subscription with a token with duplicate attrs in inline policies * token used to create another token has duplicate attrs in inline policies (sudo check) * all uses of fetchACLTokenEntryAndEntity when the request uses a token with inline policies with duplicate attrs. Almost all reqs are subject to this * when tokens are created with inline policies (unclear exactly how that can happen) * add changelog and deprecation notice * add missing copywrite notice * fix copy-paste mistake good thing it was covered by unit tests * Fix manual parsing of telemetry field in SharedConfig This commit in the hcl library was not in the v1.0.1-vault-5 version we're using but is included in v1.0.1-vault-7: https://github.com/hashicorp/hcl/commit/e80118accb521e47bc5b93104bf46c67d89d2242 This thing of reusing when parsing means that our approach of manually re-parsing fields on top of fields that have already been parsed by the hcl annotation causes strings (maybe more?) to concatenate. Fix that by removing annotation. There's actually more occurrences of this thing of automatically parsing something that is also manually parsing. In some places we could just remove the boilerplate manual parsing, in others we better remove the auto parsing, but I don't wanna pull at that thread right now. I just checked that all places at least fully overwrite the automatically parsed field instead of reusing it as the target of the decode call. The only exception is the AOP field on ent but that doesn't have maps or slices, so I think it's fine. An alternative approach would be to ensure that the auto-parsed value is discarded, like the current parseCache function does note how it's template not templates * Fix linter complaints * Update command/base_predict.go Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com> * address review * remove copywrite headers * re-add copywrite headers * make fmt * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * undo changes to deprecation.mdx * remove deprecation doc * fix conflict with changes from main --------- Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com> Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
2025-05-23 15:02:07 -04:00
func TestSSH_CanLoadDuplicateKeys(t *testing.T) {
t.Run("fail parsing without env var", func(t *testing.T) {
_, err := LoadSSHHelperConfig("./test-fixtures/agent_config_duplicate_keys.hcl")
require.Error(t, err)
require.Contains(t, err.Error(), "Each argument can only be defined once")
})
t.Run("fail parsing with env var set to false", func(t *testing.T) {
t.Setenv(allowHclDuplicatesEnvVar, "false")
_, err := LoadSSHHelperConfig("./test-fixtures/agent_config_duplicate_keys.hcl")
require.Error(t, err)
require.Contains(t, err.Error(), "Each argument can only be defined once")
})
t.Run("succeed parsing with env var set to true", func(t *testing.T) {
t.Setenv(allowHclDuplicatesEnvVar, "true")
_, err := LoadSSHHelperConfig("./test-fixtures/agent_config_duplicate_keys.hcl")
require.NoError(t, err)
})
VAULT-32657 deprecate duplicate attributes in HCL configs and policies (#30386) * upgrade hcl dependency on api pkg This upgrades the hcl dependency for the API pkg, and adapts its usage so users of our API pkg are not affected. There's no good way of communicating a warning via a library call so we don't. The tokenHelper which is used by all Vault CLI commands in order to create the Vault client, as well as directly used by the login and server commands, is implemented on the api pkg, so this upgrade also affects all of those commands. Seems like this was only moved to the api pkg because the Terraform provider uses it, and I thought creating a full copy of all those files back under command would be too much spaghetti. Also leaving some TODOs to make next deprecation steps easier. * upgrade hcl dependency in vault and sdk pkgs * upgrade hcl dependency in vault and sdk pkgs * add CLI warnings to commands that take a config - vault agent (unit test on CMD warning) - vault proxy (unit test on CMD warning) - vault server (no test for the warning) - vault operator diagnose (no tests at all, uses the same function as vault server * ignore duplicates on ParseKMSes function * Extend policy parsing functions and warn on policy store * Add warning on policy fmt with duplicate attributes * Add warnings when creating/updating policy with duplicate HCL attrs * Add log warning when switchedGetPolicy finds duplicate attrs Following operations can trigger this warning when they run into a policy with duplicate attributes: * replication filtered path namespaces invalidation * policy read API * building an ACL (for many different purposes like most authZ operations) * looking up DR token policies * creating a token with named policies * when caching the policies for all namespaces during unseal * Print log warnings when token inline policy has duplicate attrs No unit tests on these as new test infra would have to be built on all. Operations affected, which will now print a log warning when the retrieved token has an inline policy with duplicate attributes: * capabilities endpoints in sys mount * handing events under a subscription with a token with duplicate attrs in inline policies * token used to create another token has duplicate attrs in inline policies (sudo check) * all uses of fetchACLTokenEntryAndEntity when the request uses a token with inline policies with duplicate attrs. Almost all reqs are subject to this * when tokens are created with inline policies (unclear exactly how that can happen) * add changelog and deprecation notice * add missing copywrite notice * fix copy-paste mistake good thing it was covered by unit tests * Fix manual parsing of telemetry field in SharedConfig This commit in the hcl library was not in the v1.0.1-vault-5 version we're using but is included in v1.0.1-vault-7: https://github.com/hashicorp/hcl/commit/e80118accb521e47bc5b93104bf46c67d89d2242 This thing of reusing when parsing means that our approach of manually re-parsing fields on top of fields that have already been parsed by the hcl annotation causes strings (maybe more?) to concatenate. Fix that by removing annotation. There's actually more occurrences of this thing of automatically parsing something that is also manually parsing. In some places we could just remove the boilerplate manual parsing, in others we better remove the auto parsing, but I don't wanna pull at that thread right now. I just checked that all places at least fully overwrite the automatically parsed field instead of reusing it as the target of the decode call. The only exception is the AOP field on ent but that doesn't have maps or slices, so I think it's fine. An alternative approach would be to ensure that the auto-parsed value is discarded, like the current parseCache function does note how it's template not templates * Fix linter complaints * Update command/base_predict.go Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com> * address review * remove copywrite headers * re-add copywrite headers * make fmt * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * Update website/content/partials/deprecation/duplicate-hcl-attributes.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> * undo changes to deprecation.mdx * remove deprecation doc * fix conflict with changes from main --------- Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com> Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
2025-05-23 15:02:07 -04:00
}
2015-08-18 22:00:27 -04:00
func TestSSH_CreateTLSClient(t *testing.T) {
// load the default configuration
2016-02-26 17:01:40 -05:00
config, err := LoadSSHHelperConfig("./test-fixtures/agent_config.hcl")
2015-08-18 22:00:27 -04:00
if err != nil {
panic(fmt.Sprintf("error loading agent's config file: %s", err))
}
client, err := config.NewClient()
if err != nil {
panic(fmt.Sprintf("error creating the client: %s", err))
}
// Provide a certificate and enforce setting of transport
config.CACert = "./test-fixtures/vault.crt"
client, err = config.NewClient()
if err != nil {
panic(fmt.Sprintf("error creating the client: %s", err))
}
if client.config.HttpClient.Transport == nil {
panic(fmt.Sprintf("error creating client with TLS transport"))
}
}
2016-03-10 16:47:46 -05:00
func TestSSH_CreateTLSClient_tlsServerName(t *testing.T) {
// Ensure that the HTTP client is associated with the configured TLS server name.
tlsServerName := "tls.server.name"
config, err := ParseSSHHelperConfig(fmt.Sprintf(`
vault_addr = "1.2.3.4"
tls_server_name = "%s"
`, tlsServerName))
if err != nil {
panic(fmt.Sprintf("error loading config: %s", err))
}
client, err := config.NewClient()
if err != nil {
panic(fmt.Sprintf("error creating the client: %s", err))
}
actualTLSServerName := client.config.HttpClient.Transport.(*http.Transport).TLSClientConfig.ServerName
if actualTLSServerName != tlsServerName {
panic(fmt.Sprintf("incorrect TLS server name. expected: %s actual: %s", tlsServerName, actualTLSServerName))
}
}
2016-03-10 16:47:46 -05:00
func TestParseSSHHelperConfig(t *testing.T) {
config, err := ParseSSHHelperConfig(`
vault_addr = "1.2.3.4"
`)
if err != nil {
t.Fatal(err)
}
if config.SSHMountPoint != SSHHelperDefaultMountPoint {
t.Errorf("expected %q to be %q", config.SSHMountPoint, SSHHelperDefaultMountPoint)
}
}
func TestParseSSHHelperConfig_missingVaultAddr(t *testing.T) {
_, err := ParseSSHHelperConfig("")
if err == nil {
t.Fatal("expected error")
}
if !strings.Contains(err.Error(), `missing config "vault_addr"`) {
2016-03-10 16:47:46 -05:00
t.Errorf("bad error: %s", err)
}
}
func TestParseSSHHelperConfig_badKeys(t *testing.T) {
_, err := ParseSSHHelperConfig(`
vault_addr = "1.2.3.4"
nope = "bad"
`)
if err == nil {
t.Fatal("expected error")
}
if !strings.Contains(err.Error(), `ssh_helper: invalid key "nope" on line 3`) {
2016-03-10 16:47:46 -05:00
t.Errorf("bad error: %s", err)
}
}
func TestParseSSHHelperConfig_tlsServerName(t *testing.T) {
tlsServerName := "tls.server.name"
config, err := ParseSSHHelperConfig(fmt.Sprintf(`
vault_addr = "1.2.3.4"
tls_server_name = "%s"
`, tlsServerName))
if err != nil {
t.Fatal(err)
}
if config.TLSServerName != tlsServerName {
t.Errorf("incorrect TLS server name. expected: %s actual: %s", tlsServerName, config.TLSServerName)
}
}