terraform/internal/command/workdir/backend_state_test.go

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

358 lines
11 KiB
Go
Raw Permalink Normal View History

command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
// Copyright IBM Corp. 2014, 2026
// SPDX-License-Identifier: BUSL-1.1
package workdir
import (
"encoding/json"
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
"strings"
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
"testing"
"github.com/google/go-cmp/cmp"
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
tfaddr "github.com/hashicorp/terraform-registry-address"
"github.com/hashicorp/terraform/version"
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
)
func TestParseBackendStateFile(t *testing.T) {
tests := map[string]struct {
Input string
Want *BackendStateFile
WantErr string
}{
"empty": {
Input: ``,
WantErr: `invalid syntax: unexpected end of JSON input`,
},
"empty but valid JSON syntax": {
Input: `{}`,
WantErr: `invalid syntax: no format version number`,
},
"older version": {
Input: `{
"version": 2,
"terraform_version": "0.3.0"
}`,
WantErr: `unsupported backend state version 2; you may need to use Terraform CLI v0.3.0 to work in this directory`,
},
"newer version": {
Input: `{
"version": 4,
"terraform_version": "54.23.9"
}`,
WantErr: `unsupported backend state version 4; you may need to use Terraform CLI v54.23.9 to work in this directory`,
},
"legacy remote state is active": {
Input: `{
"version": 3,
"terraform_version": "0.8.0",
"remote": {
"anything": "goes"
}
}`,
WantErr: `this working directory uses legacy remote state and so must first be upgraded using Terraform v0.9`,
},
"active backend": {
Input: `{
"version": 3,
"terraform_version": "0.8.0",
"backend": {
"type": "treasure_chest_buried_on_a_remote_island",
2025-07-14 08:52:27 -04:00
"config": {},
"hash" : 12345
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
}
}`,
Want: &BackendStateFile{
Version: 3,
TFVersion: "0.8.0",
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
Backend: &BackendConfigState{
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
Type: "treasure_chest_buried_on_a_remote_island",
ConfigRaw: json.RawMessage("{}"),
2025-07-14 08:52:27 -04:00
Hash: 12345,
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
},
},
},
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
"active state_store": {
Input: `{
"version": 3,
"terraform_version": "9.9.9",
"state_store": {
"type": "foobar_baz",
"config": {
"bucket": "my-bucket",
"region": "saturn"
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
},
"provider": {
"version": "1.2.3",
"source": "registry.terraform.io/my-org/foobar",
"config": {
"credentials": "./creds.json"
2025-07-14 08:52:27 -04:00
},
"hash" : 12345
},
"hash" : 12345
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
}
}`,
Want: &BackendStateFile{
Version: 3,
TFVersion: "9.9.9",
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
// Watch out - the number of tabs in the last argument here are load-bearing
Provider: getTestProviderState(t, "1.2.3", "registry.terraform.io", "my-org", "foobar", `{
"credentials": "./creds.json"
}`),
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
ConfigRaw: json.RawMessage(`{
"bucket": "my-bucket",
"region": "saturn"
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
}`),
2025-07-14 08:52:27 -04:00
Hash: 12345,
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
},
},
},
"detection of malformed state: conflicting 'backend' and 'state_store' sections": {
Input: `{
"version": 3,
"terraform_version": "9.9.9",
"backend": {
"type": "treasure_chest_buried_on_a_remote_island",
2025-07-14 08:52:27 -04:00
"config": {},
"hash" : 12345
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
},
"state_store": {
"type": "foobar_baz",
"config": {
"provider": "foobar",
"bucket": "my-bucket"
},
"provider": {
"version": "1.2.3",
2025-07-14 08:52:27 -04:00
"source": "registry.terraform.io/my-org/foobar",
"hash" : 12345
},
"hash" : 12345
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
}
}`,
WantErr: `encountered a malformed backend state file that contains state for both a 'backend' and a 'state_store' block`,
},
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
}
for name, test := range tests {
t.Run(name, func(t *testing.T) {
got, err := ParseBackendStateFile([]byte(test.Input))
if test.WantErr != "" {
if err == nil {
t.Fatalf("unexpected success\nwant error: %s", test.WantErr)
}
if got, want := err.Error(), test.WantErr; got != want {
t.Errorf("wrong error\ngot: %s\nwant: %s", got, want)
}
return
}
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
if diff := cmp.Diff(test.Want, got); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
})
}
}
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
func TestEncodeBackendStateFile(t *testing.T) {
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
noVersionData := ""
tfVersion := version.Version
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
tests := map[string]struct {
Input *BackendStateFile
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
Envs map[string]string
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
Want []byte
WantErr string
}{
"encoding a backend state file when state_store is in use": {
Input: &BackendStateFile{
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
Provider: getTestProviderState(t, "1.2.3", "registry.terraform.io", "my-org", "foobar", `{"foo": "bar"}`),
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
PSS: Let the `init` command recognise when there are no changes in configuration. (#37777) * Pull determining of PSS provider's version from current locks into a separate method * Add code for identifying when config and provider version match existing backend state (i.e. no changes) * Update test - locks are now needed before it hits expected error diag return * Add test showing successful init when no config changes are detected. * Update `getStateStorageProviderVersion` to return nil versions for builtin and re-attached providers. This makes comparison easier when determining if config has changed since last init. * Add test coverage for `getStateStorageProviderVersion` * Move testing fixtures around, preparing for different types of changed state_store config changes being tested * Add test showing that changing the state_store config is detected as a change, but handling this scenario isn't implemented yet * Update hashes in test fixture backend state file to be accurate Previously dummy values were fine, but as tests using hashes to identify changes these values need to be accurate! * Update existing test cases so that Terraform uses the same test provider version as described in the backend state file fixture for the test. * Add test showing that changing the PSS provider's config is detected as a change, but handling this scenario isn't implemented yet * Add test showing that swapping to a different state storage implementation in the same provider is detected as a change, but handling this scenario isn't implemented yet * Add test showing that changing the provider used for PSS is detected as a change, but handling this scenario isn't implemented yet * Add test showing that upgrading a provider is detected as a change, but handling this scenario isn't implemented yet * Update test to use v1.2.3 for consistency with other tests Just to avoid any confusion if copy-pasting happens in future. * More corrections to existing test fixtures - unset config should be null, and replace dummy hash values with correct values. * Fix test for using -reconfigure with state_store; the default workspace would already exist in this scenario * Update TestInit_stateStore_configUnchanged to assert that init was a no-op for backend state * Remove unused fixture * Remove test that's replaced by new tests in command/init_test.go * Replace old references to deleted "state-store-changed" test fixture & update test to not expect a value for region attr in provider config * Make test fixture coupling a little more understandable * Refactor detection of no need to migrate into a function * Add TODO about more involved provider version change tests We will allow downgrades to succeed as long as the schema version number is unchanged * Update (configs.StateStore)Hash method to return a single hash that's impacted by: state store config, provider config, state store type, provider source * Update calling code and test helper code to reflect that the nested provider block no longer has its own hash * Remove test; there is now a single hash that SHOULD be affected by the provider block! * Also use provider name, from config, in hash * Update tests to reflect changes in how hashes are made * Remove unused `stateStoreConfigNeedsMigration` function * Remove duplicate isProviderReattached function. * Fixes to affected tests * Allow provider version to impact the state storage hash, update impacted tests and test fixtures * Update tests that now require locks data to be present in test setup * Update comment for accuracy * Fixes to other test fixtures - remove excess hash field, set hash to 0 to indicate they're not set accurately. * Make upgrade test actually use upgrade code path * Add lock files to test fixture directories that represent a project that's had a successful prior init using PSS
2025-10-20 11:51:45 -04:00
Want: []byte("{\n \"version\": 3,\n \"terraform_version\": \"" + tfVersion + "\",\n \"state_store\": {\n \"type\": \"foobar_baz\",\n \"provider\": {\n \"version\": \"1.2.3\",\n \"source\": \"registry.terraform.io/my-org/foobar\",\n \"config\": {\n \"foo\": \"bar\"\n }\n },\n \"config\": {\n \"foo\": \"bar\"\n },\n \"hash\": 123\n }\n}"),
},
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
"it's valid to record no version data when a builtin provider used for state store": {
Input: &BackendStateFile{
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
Provider: getTestProviderState(t, noVersionData, string(tfaddr.BuiltInProviderHost), string(tfaddr.BuiltInProviderNamespace), "foobar", `{"foo": "bar"}`),
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
PSS: Let the `init` command recognise when there are no changes in configuration. (#37777) * Pull determining of PSS provider's version from current locks into a separate method * Add code for identifying when config and provider version match existing backend state (i.e. no changes) * Update test - locks are now needed before it hits expected error diag return * Add test showing successful init when no config changes are detected. * Update `getStateStorageProviderVersion` to return nil versions for builtin and re-attached providers. This makes comparison easier when determining if config has changed since last init. * Add test coverage for `getStateStorageProviderVersion` * Move testing fixtures around, preparing for different types of changed state_store config changes being tested * Add test showing that changing the state_store config is detected as a change, but handling this scenario isn't implemented yet * Update hashes in test fixture backend state file to be accurate Previously dummy values were fine, but as tests using hashes to identify changes these values need to be accurate! * Update existing test cases so that Terraform uses the same test provider version as described in the backend state file fixture for the test. * Add test showing that changing the PSS provider's config is detected as a change, but handling this scenario isn't implemented yet * Add test showing that swapping to a different state storage implementation in the same provider is detected as a change, but handling this scenario isn't implemented yet * Add test showing that changing the provider used for PSS is detected as a change, but handling this scenario isn't implemented yet * Add test showing that upgrading a provider is detected as a change, but handling this scenario isn't implemented yet * Update test to use v1.2.3 for consistency with other tests Just to avoid any confusion if copy-pasting happens in future. * More corrections to existing test fixtures - unset config should be null, and replace dummy hash values with correct values. * Fix test for using -reconfigure with state_store; the default workspace would already exist in this scenario * Update TestInit_stateStore_configUnchanged to assert that init was a no-op for backend state * Remove unused fixture * Remove test that's replaced by new tests in command/init_test.go * Replace old references to deleted "state-store-changed" test fixture & update test to not expect a value for region attr in provider config * Make test fixture coupling a little more understandable * Refactor detection of no need to migrate into a function * Add TODO about more involved provider version change tests We will allow downgrades to succeed as long as the schema version number is unchanged * Update (configs.StateStore)Hash method to return a single hash that's impacted by: state store config, provider config, state store type, provider source * Update calling code and test helper code to reflect that the nested provider block no longer has its own hash * Remove test; there is now a single hash that SHOULD be affected by the provider block! * Also use provider name, from config, in hash * Update tests to reflect changes in how hashes are made * Remove unused `stateStoreConfigNeedsMigration` function * Remove duplicate isProviderReattached function. * Fixes to affected tests * Allow provider version to impact the state storage hash, update impacted tests and test fixtures * Update tests that now require locks data to be present in test setup * Update comment for accuracy * Fixes to other test fixtures - remove excess hash field, set hash to 0 to indicate they're not set accurately. * Make upgrade test actually use upgrade code path * Add lock files to test fixture directories that represent a project that's had a successful prior init using PSS
2025-10-20 11:51:45 -04:00
Want: []byte("{\n \"version\": 3,\n \"terraform_version\": \"" + tfVersion + "\",\n \"state_store\": {\n \"type\": \"foobar_baz\",\n \"provider\": {\n \"version\": null,\n \"source\": \"terraform.io/builtin/foobar\",\n \"config\": {\n \"foo\": \"bar\"\n }\n },\n \"config\": {\n \"foo\": \"bar\"\n },\n \"hash\": 123\n }\n}"),
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
},
"it's valid to record no version data when a re-attached provider used for state store": {
Input: &BackendStateFile{
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
Provider: getTestProviderState(t, noVersionData, "registry.terraform.io", "hashicorp", "foobar", `{"foo": "bar"}`),
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
Envs: map[string]string{
"TF_REATTACH_PROVIDERS": `{
"foobar": {
"Protocol": "grpc",
"ProtocolVersion": 6,
"Pid": 12345,
"Test": true,
"Addr": {
"Network": "unix",
"String":"/var/folders/xx/abcde12345/T/plugin12345"
}
}
}`,
},
PSS: Let the `init` command recognise when there are no changes in configuration. (#37777) * Pull determining of PSS provider's version from current locks into a separate method * Add code for identifying when config and provider version match existing backend state (i.e. no changes) * Update test - locks are now needed before it hits expected error diag return * Add test showing successful init when no config changes are detected. * Update `getStateStorageProviderVersion` to return nil versions for builtin and re-attached providers. This makes comparison easier when determining if config has changed since last init. * Add test coverage for `getStateStorageProviderVersion` * Move testing fixtures around, preparing for different types of changed state_store config changes being tested * Add test showing that changing the state_store config is detected as a change, but handling this scenario isn't implemented yet * Update hashes in test fixture backend state file to be accurate Previously dummy values were fine, but as tests using hashes to identify changes these values need to be accurate! * Update existing test cases so that Terraform uses the same test provider version as described in the backend state file fixture for the test. * Add test showing that changing the PSS provider's config is detected as a change, but handling this scenario isn't implemented yet * Add test showing that swapping to a different state storage implementation in the same provider is detected as a change, but handling this scenario isn't implemented yet * Add test showing that changing the provider used for PSS is detected as a change, but handling this scenario isn't implemented yet * Add test showing that upgrading a provider is detected as a change, but handling this scenario isn't implemented yet * Update test to use v1.2.3 for consistency with other tests Just to avoid any confusion if copy-pasting happens in future. * More corrections to existing test fixtures - unset config should be null, and replace dummy hash values with correct values. * Fix test for using -reconfigure with state_store; the default workspace would already exist in this scenario * Update TestInit_stateStore_configUnchanged to assert that init was a no-op for backend state * Remove unused fixture * Remove test that's replaced by new tests in command/init_test.go * Replace old references to deleted "state-store-changed" test fixture & update test to not expect a value for region attr in provider config * Make test fixture coupling a little more understandable * Refactor detection of no need to migrate into a function * Add TODO about more involved provider version change tests We will allow downgrades to succeed as long as the schema version number is unchanged * Update (configs.StateStore)Hash method to return a single hash that's impacted by: state store config, provider config, state store type, provider source * Update calling code and test helper code to reflect that the nested provider block no longer has its own hash * Remove test; there is now a single hash that SHOULD be affected by the provider block! * Also use provider name, from config, in hash * Update tests to reflect changes in how hashes are made * Remove unused `stateStoreConfigNeedsMigration` function * Remove duplicate isProviderReattached function. * Fixes to affected tests * Allow provider version to impact the state storage hash, update impacted tests and test fixtures * Update tests that now require locks data to be present in test setup * Update comment for accuracy * Fixes to other test fixtures - remove excess hash field, set hash to 0 to indicate they're not set accurately. * Make upgrade test actually use upgrade code path * Add lock files to test fixture directories that represent a project that's had a successful prior init using PSS
2025-10-20 11:51:45 -04:00
Want: []byte("{\n \"version\": 3,\n \"terraform_version\": \"" + tfVersion + "\",\n \"state_store\": {\n \"type\": \"foobar_baz\",\n \"provider\": {\n \"version\": null,\n \"source\": \"registry.terraform.io/hashicorp/foobar\",\n \"config\": {\n \"foo\": \"bar\"\n }\n },\n \"config\": {\n \"foo\": \"bar\"\n },\n \"hash\": 123\n }\n}"),
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
},
"error when neither backend nor state_store config state are present": {
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
Input: &BackendStateFile{},
Want: []byte("{\n \"version\": 3,\n \"terraform_version\": \"" + tfVersion + "\"\n}"),
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
},
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
"error when the provider is neither builtin nor reattached and the provider version is missing": {
Input: &BackendStateFile{
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
Provider: getTestProviderState(t, noVersionData, "registry.terraform.io", "my-org", "foobar", ""),
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
WantErr: `state store is not valid: provider version data is missing`,
},
"error when the provider source's hostname is missing": {
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
Input: &BackendStateFile{
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
Provider: getTestProviderState(t, "1.2.3", "", "my-org", "foobar", ""),
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
},
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
WantErr: `state store is not valid: Unknown hostname: Expected hostname in the provider address to be set`,
},
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
"error when the provider source's hostname and namespace are missing ": {
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
Input: &BackendStateFile{
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
Provider: getTestProviderState(t, "1.2.3", "", "", "foobar", ""),
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
WantErr: `state store is not valid: Unknown hostname: Expected hostname in the provider address to be set`,
},
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
"error when the provider source is completely missing ": {
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
Input: &BackendStateFile{
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
Provider: getTestProviderState(t, "1.2.3", "", "", "", ""),
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
WantErr: `state store is not valid: Empty provider address: Expected address composed of hostname, provider namespace and name`,
},
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
"error when both backend and state_store config state are present": {
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
Input: &BackendStateFile{
Backend: &BackendConfigState{
Type: "foobar",
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
StateStore: &StateStoreConfigState{
Type: "foobar_baz",
Provider: getTestProviderState(t, "1.2.3", "registry.terraform.io", "my-org", "foobar", ""),
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
WantErr: `attempted to encode a malformed backend state file; it contains state for both a 'backend' and a 'state_store' block`,
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
},
}
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
for name, test := range tests {
t.Run(name, func(t *testing.T) {
PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace
2025-10-15 05:44:21 -04:00
// Some test cases depend on ENVs, not all
for k, v := range test.Envs {
t.Setenv(k, v)
}
Update backend state file so it can describe PSS state (#37179) * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency
2025-06-11 10:10:26 -04:00
got, err := EncodeBackendStateFile(test.Input)
if test.WantErr != "" {
if err == nil {
t.Fatalf("unexpected success\nwant error: %s", test.WantErr)
}
if !strings.Contains(err.Error(), test.WantErr) {
t.Errorf("wrong error\ngot: %s\nwant: %s", err.Error(), test.WantErr)
}
return
}
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
if diff := cmp.Diff(test.Want, got); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
})
command/workdir: Model the "backend state" file format Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.
2024-03-07 14:58:46 -05:00
}
}
func TestBackendStateFile_DeepCopy(t *testing.T) {
tests := map[string]struct {
file *BackendStateFile
}{
"Deep copy preserves state_store data": {
file: &BackendStateFile{
StateStore: &StateStoreConfigState{
Type: "foo_bar",
Provider: getTestProviderState(t, "1.2.3", "A", "B", "C", ""),
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
},
"Deep copy preserves backend data": {
file: &BackendStateFile{
Backend: &BackendConfigState{
Type: "foobar",
ConfigRaw: json.RawMessage([]byte(`{"foo":"bar"}`)),
Hash: 123,
},
},
},
"Deep copy preserves version and Terraform version data": {
file: &BackendStateFile{
Version: 3,
TFVersion: "9.9.9",
},
},
}
for tn, tc := range tests {
t.Run(tn, func(t *testing.T) {
copy := tc.file.DeepCopy()
2025-07-14 08:52:27 -04:00
if diff := cmp.Diff(copy, tc.file); diff != "" {
t.Fatalf("unexpected difference in backend state data:\n %s", diff)
}
})
}
}