We previously used a loader -> BuildConfig flow to load configuration.
This commit changes most (but not all yet) flows to use the new
graph-based approach. Instead of simply recursively loading the modules,
we now need to take a stepped approach:
1. Load the root module
2. Collect the variables and their values
3. Build the configuration with the graph-based approach
Because this approach relies on different parts from different packages,
it can't easliy be done within the `configload` package. So, now we do
most of in the backend or command.
This prevents a cyclic dependency and also makes sense semantically.
The arguments package will collect the unparsed variable values and
the backendrun helpers will work to collect the values and transform
them into terraform.InputValue.
* backend/inmem: Make it easier to use the backend in tests
* cloud: Make the cloud backend testable
* command/views: Introduce migration UI messages
* command/init: Add 3 tests for migrations from a backend to PSS
- TestInit_backend_to_stateStore_singleWorkspace
- TestInit_backend_to_stateStore_multipleWorkspaces
- TestInit_cloud_to_stateStore
* command/init: Implement migration from a backend to PSS
* address PR feedback
* remove local state after migration
* Add ability to parse backend blocks present in a test file's run blocks, validate configuration (#36541)
* Add ability to parse backend blocks from a run block
* Add validation to avoid multiple backend blocks across run blocks that use the same internal state file. Update tests.
* Add validation to avoid multiple backend blocks within a single run block. Update tests.
* Remove use of quotes in diagnostic messages
* Add validation to avoid backend blocks being used in plan run blocks. Update tests.
* Correct local backend blocks in new test fixtures
* Add test to show that different test files can use same backend block for same state key.
* Add validation to enforce state-storage backend types are used
* Remove TODO comment
We only need to consider one file at a time when checking if a state_key already has a backend associated with it; parallelism in `terraform test` is scoped down to individual files.
* Add validation to assert that the backend block must be in the first apply command for an internal state
* Consolidate backend block validation inside a single if statement
* Add initial version of validation that ensures a backend isn't re-used within a file
* Explicitly set the state_key at the point of parsing the config
TODO: What should be done with method (moduletest.Run).GetStateKey?
* Update test fixture now that reusing backend configs has been made invalid
* Add automated test showing validation of reused configuration blocks
* Skip test due to flakiness, minor change to test config naming
* Update test so it tolerates non-deterministic order run blocks are evaluated in
* Remove unnecessary value assignment to r.StateKey
* Replace use of GetStateKey() with accessing the state key that's now set during test config parsing
* Fix bug so that run blocks using child modules get the correct state key set at parsing time
* Update acceptance test to also cover scenario where root and child module state keys are in use
* Update test name
* Add newline to regex
* Ensure consistent place where repeat backend error is raised from
* Write leftover test state(s) to file (#36614)
* Add additional validation that the backend used in a run is a supported type (#36648)
* Prevent test run when leftover state data is present (#36685)
* `test`: Set the initial state for a state files from a backend, allow the run that defines a backend to write state to the backend (#36646)
* Allow use of backend block to set initial state for a state key
* Note about alternative place to keep 'backend factories'
* Allow the run block defining the backend to write state to it
* Fix rebase
* Change to accessing backend init functions via ContextOpts
* Add tests demonstrating how runs containing backend blocks use and update persisted state
* Fix test fixture
* Address test failure due to trouble opening the state file
This problem doesn't happen on MacOS, so I assume is due to the Linux environment of GitHub runners.
* Fix issue with paths properly
I hope
* Fix defect in test assertion
* Pivot back to approach introduced in 4afc3d7
* Let failing tests write to persistent state, add test case covering that.
I split the acceptance tests into happy/unhappy paths for this, which required some of the helper functions' declarations to be raised up to package-level.
* Change how we update internal state files, so that information about the associated backend is never lost
* Fix UpdateStateFile
* Ensure that the states map set by TestStateTransformer associates a backend with the correct run.
* Misc spelling fixes in comments and a log
* Replace state get/set functions with existing helpers (#36747)
* Replace state get/set functions with existing helpers
* Compare to string representation of state
* Compare to string representation of state
* Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729)
* Add validation to enforce skip_cleanup=false cannot be used with backend blocks (#36857)
* Integrate use of backend blocks in tests with skip_cleanup feature (#36848)
* Fix nil pointer error, update test to not be table-driven
* Make using a backend block implicitly set skip_cleanup to true
* Stop state artefacts being created when a backend is in use and no cleanup errors have occurred
* Return diagnostics so calling code knows if cleanup experienced issues or not
* Update tests to show that when cleanup fails a state artefact is created
* Add comment about why diag not returned
* Bug fix - actually pull in the state from the state manager!
* Split and simplify (?) tests to show the backend block can create and/or reuse prior state
* Update test to use new fixtures, assert about state artefact. Fix nil pointer
* Update test fixture in use, add guardrail for flakiness of forced error during cleanup
* Refactor so resource ID set in only one place
* Add documentation for using a `backend` block during `test` (#36832)
* Add backend as a documented block in a run block
* Add documentation about backend blocks in run blocks.
* Make the relationship between backends and state keys more clear, other improvements
* More test documentation (#36838)
* Terraform Test: cleanup command (#36847)
* Allow cleanup of states that depend on prior runs outputs (#36902)
* terraform test: refactor graph edge calculation
* create fake run block nodes during cleanup operation
* tidy up TODOs
* fix tests
* remove old changes
* Update internal/moduletest/graph/node_state_cleanup.go
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
* Improve diagnostics around skip_cleanup conflicts (#37385)
* Improve diagnostics around skip_cleanup conflicts
* remove unused dynamic node
* terraform test: refactor manifest file for simplicity (#37412)
* test: refactor apply and plan functions so no run block is needed
* terraform test: write and load state manifest files
* Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729)
* terraform test: add support for skip_cleanup attr
* terraform test: add cleanup command
* terraform test: add backend blocks
* pause
* fix tests
* remove commented code
* terraform test: make controlling destroy functionality experimental (#37419)
* address comments
* Update internal/moduletest/graph/node_state_cleanup.go
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
---------
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
* add experimental changelog entries
---------
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>
* Fix S3 backend test affected by making the Workspaces method return errors via diagnostics
* Address diagnostics comparison issues in test by ensuring expected diagnostics are defined in the context of the config they're triggered by
* Fix failing test case `TestBackendConfig_EC2MetadataEndpoint/envvar_invalid_mode` by making `diagnosticBase` struct comparable
* Add compile-time checks that diagnostic types fulfil interfaces
* Stop diagnosticBase implementing ComparableDiagnostic, re-add S3-specific comparer code to s3 package
* Update tests to use the S3-specific comparer again
* Fix test case missed in refactoring
* Update the backend.Backend interface to use diagnostics as return value from StateMgr method
* Fix calls to `Fatalf`
* Update Backend interface to use diagnostics for `DeleteWorkspace` and `Workspaces` methods
* Update testing helpers in backend package
* Update pluggable package to match interface changes
* Update builtin terraform provider package to match interface changes
* Update code in Meta-related files in command package; update error check logic
* Update workspace commands; update error check logic
* Update Local backend package to match interface changes
* Update Cloud backend to match interface changes
* Update Remote backend to match interface changes
* Update cos remote-state backend to match interface changes
* Update inmem remote-state backend to match interface changes
* Update pg remote-state backend to match interface changes
* Update http remote-state backend to match interface changes
* Update consul remote-state backend to match interface changes
* Update oss remote-state backend to match interface changes
* Update kubernetes remote-state backend to match interface changes
* Update gcs remote-state backend to match interface changes
* Update s3 remote-state backend to match interface changes
* Update oci remote-state backend to match interface changes
* Update azure remote-state backend to match interface changes
* Allow warnings from Workspaces and DeleteWorkspace methods to be logged or returned to calling code
* Fix defect in inmem backend test
* Change how warnings from workspace-related methods are output to view/ui
* WIP
* Reuse plan command for query CLI
* Basic CLI output
* Only fail a list request on error
* poc: store query results in separate field
* WIP: odd mixture between JSONs
* Fix list references
* Separate JSON rendering
The structured JSON now only logs a status on which list query is
currently running. The new jsonlist package can marshal the query fields
of a plan.
* Remove matcher
* Store results in an extra struct
* Structured list result logging
* Move list result output into hooks
* Add help text and additional flag
* Disable query runs with the cloud backend for now
* Review feedback
* Split code for backend state file vs backend state
* Rename BackendState to BackendConfigState
* Spelling error
* Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface.
* Split tests for backend state file vs backend config state structs
* Rename StateStorageConfigState to StateStoreConfigState
* Clarify test name, add comments
* Add tests for StateStoreConfigState methods
* Add test showing state_store in JSON is parsed correctly
* Add detection of malformed backend state files that contain both backend and state_store fields
* Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks
* Rename `state_storage` to `state_store`
* Rename `state_storage` to `state_store` in filenames
* Move`ConfigState` to its own file
* Fix test name, remove whitespace
* Update `StateStoreConfigState` comment using review suggestion
* Update error message to no longer allude to the environment TF is being run in
* Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data
* Update test helper so it doesn't accidentally supply validation in tests
* Add protection against saving an empty backend state file
* Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods
* Add Validate method to StateStoreConfigState, use in backend state encoding logic
* Refactor to use new features in registry dependency
* Update URLs from www.terraform.io to developer.hashicorp.com
* More updates to URLs linking to documentation in documentation and user-facing warnings/errors
* More updates to URLs linking to documentation, in docs, code, test fixtures
* Replace www.terraform.io/community with link to Community Forum
* Apply suggestions from code review
Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>
---------
Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>
* Fix copy-pasta
* Update some comments clarifying backend-related interfaces and "enhanced" versus "operations"
* Fix more comments that refer to types and interfaces that have moved into the backendrun package
* Add ability for TestJUnitXMLFile to access data about whether the test runner was Stopped
* Add details to XML describing why a Run was skipped
* Fix wording
* Code consistency changes
* Move all JUnit-related code down to where it's used
Away from the Views section of the code where it was relevant before
* Move JUnit-related error and warning diags to above where cancellable contexts are created
* Fix wording of user feedback
* Fix test to match updated skipped message text
* Fix test
* Update comments to reflect that the backend.Enhanced interface no longer exists
* Add compile-time check that Local backend implements backendrun.OperationsBackend
Added a comment to explain the rationale behind the guard statement, which prevents potential nil errors during backend initialization and retryLogHook call.
Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>
When key-value tags are enabled and used in the workspace, users may
define the tags attribute as a map of strings in the cloud block in order
to more precicely match workspaces using those tags.
* Remove handling of ephemeral root outputs
This is effectively reverting ~99% of https://github.com/hashicorp/terraform/pull/35676
The only changes not being reverted are some formatting and deprecation fixes which remain relevant.
The code being removed is basically dead code now in the context of root ephemeral outputs being rejected per https://github.com/hashicorp/terraform/pull/35791
* Remove unrelated changes
Some updates to staticcheck were catching more errors. Most of these
were format functions without a format string, but rather than adding
more exceptions I just fixed them all. This did also catch some
incorrectly wrapped errors.
