2023-05-02 11:33:06 -04:00
|
|
|
// Copyright IBM Corp. 2014, 2026
|
2023-08-10 18:43:27 -04:00
|
|
|
// SPDX-License-Identifier: BUSL-1.1
|
2023-05-02 11:33:06 -04:00
|
|
|
|
2021-09-01 20:01:44 -04:00
|
|
|
package workdir
|
|
|
|
|
|
|
|
|
|
import (
|
|
|
|
|
"fmt"
|
|
|
|
|
"os"
|
|
|
|
|
"path/filepath"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
// Dir represents a single Terraform working directory.
|
|
|
|
|
//
|
|
|
|
|
// "Working directory" is unfortunately a slight misnomer, because non-default
|
|
|
|
|
// options can potentially stretch the definition such that multiple working
|
|
|
|
|
// directories end up appearing to share a data directory, or other similar
|
Implement controlling destroy functionality within Terraform Test (#37359)
* Add ability to parse backend blocks present in a test file's run blocks, validate configuration (#36541)
* Add ability to parse backend blocks from a run block
* Add validation to avoid multiple backend blocks across run blocks that use the same internal state file. Update tests.
* Add validation to avoid multiple backend blocks within a single run block. Update tests.
* Remove use of quotes in diagnostic messages
* Add validation to avoid backend blocks being used in plan run blocks. Update tests.
* Correct local backend blocks in new test fixtures
* Add test to show that different test files can use same backend block for same state key.
* Add validation to enforce state-storage backend types are used
* Remove TODO comment
We only need to consider one file at a time when checking if a state_key already has a backend associated with it; parallelism in `terraform test` is scoped down to individual files.
* Add validation to assert that the backend block must be in the first apply command for an internal state
* Consolidate backend block validation inside a single if statement
* Add initial version of validation that ensures a backend isn't re-used within a file
* Explicitly set the state_key at the point of parsing the config
TODO: What should be done with method (moduletest.Run).GetStateKey?
* Update test fixture now that reusing backend configs has been made invalid
* Add automated test showing validation of reused configuration blocks
* Skip test due to flakiness, minor change to test config naming
* Update test so it tolerates non-deterministic order run blocks are evaluated in
* Remove unnecessary value assignment to r.StateKey
* Replace use of GetStateKey() with accessing the state key that's now set during test config parsing
* Fix bug so that run blocks using child modules get the correct state key set at parsing time
* Update acceptance test to also cover scenario where root and child module state keys are in use
* Update test name
* Add newline to regex
* Ensure consistent place where repeat backend error is raised from
* Write leftover test state(s) to file (#36614)
* Add additional validation that the backend used in a run is a supported type (#36648)
* Prevent test run when leftover state data is present (#36685)
* `test`: Set the initial state for a state files from a backend, allow the run that defines a backend to write state to the backend (#36646)
* Allow use of backend block to set initial state for a state key
* Note about alternative place to keep 'backend factories'
* Allow the run block defining the backend to write state to it
* Fix rebase
* Change to accessing backend init functions via ContextOpts
* Add tests demonstrating how runs containing backend blocks use and update persisted state
* Fix test fixture
* Address test failure due to trouble opening the state file
This problem doesn't happen on MacOS, so I assume is due to the Linux environment of GitHub runners.
* Fix issue with paths properly
I hope
* Fix defect in test assertion
* Pivot back to approach introduced in 4afc3d7
* Let failing tests write to persistent state, add test case covering that.
I split the acceptance tests into happy/unhappy paths for this, which required some of the helper functions' declarations to be raised up to package-level.
* Change how we update internal state files, so that information about the associated backend is never lost
* Fix UpdateStateFile
* Ensure that the states map set by TestStateTransformer associates a backend with the correct run.
* Misc spelling fixes in comments and a log
* Replace state get/set functions with existing helpers (#36747)
* Replace state get/set functions with existing helpers
* Compare to string representation of state
* Compare to string representation of state
* Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729)
* Add validation to enforce skip_cleanup=false cannot be used with backend blocks (#36857)
* Integrate use of backend blocks in tests with skip_cleanup feature (#36848)
* Fix nil pointer error, update test to not be table-driven
* Make using a backend block implicitly set skip_cleanup to true
* Stop state artefacts being created when a backend is in use and no cleanup errors have occurred
* Return diagnostics so calling code knows if cleanup experienced issues or not
* Update tests to show that when cleanup fails a state artefact is created
* Add comment about why diag not returned
* Bug fix - actually pull in the state from the state manager!
* Split and simplify (?) tests to show the backend block can create and/or reuse prior state
* Update test to use new fixtures, assert about state artefact. Fix nil pointer
* Update test fixture in use, add guardrail for flakiness of forced error during cleanup
* Refactor so resource ID set in only one place
* Add documentation for using a `backend` block during `test` (#36832)
* Add backend as a documented block in a run block
* Add documentation about backend blocks in run blocks.
* Make the relationship between backends and state keys more clear, other improvements
* More test documentation (#36838)
* Terraform Test: cleanup command (#36847)
* Allow cleanup of states that depend on prior runs outputs (#36902)
* terraform test: refactor graph edge calculation
* create fake run block nodes during cleanup operation
* tidy up TODOs
* fix tests
* remove old changes
* Update internal/moduletest/graph/node_state_cleanup.go
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
* Improve diagnostics around skip_cleanup conflicts (#37385)
* Improve diagnostics around skip_cleanup conflicts
* remove unused dynamic node
* terraform test: refactor manifest file for simplicity (#37412)
* test: refactor apply and plan functions so no run block is needed
* terraform test: write and load state manifest files
* Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729)
* terraform test: add support for skip_cleanup attr
* terraform test: add cleanup command
* terraform test: add backend blocks
* pause
* fix tests
* remove commented code
* terraform test: make controlling destroy functionality experimental (#37419)
* address comments
* Update internal/moduletest/graph/node_state_cleanup.go
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
---------
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
* add experimental changelog entries
---------
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>
2025-09-10 11:22:20 -04:00
|
|
|
// anomalies, but we continue to use this terminology both for historical
|
2021-09-01 20:01:44 -04:00
|
|
|
// reasons and because it reflects the common case without any special
|
|
|
|
|
// overrides.
|
|
|
|
|
//
|
|
|
|
|
// The naming convention for methods on this type is that methods whose names
|
|
|
|
|
// begin with "Override" affect only characteristics of the particular object
|
|
|
|
|
// they're called on, changing where it looks for data, while methods whose
|
|
|
|
|
// names begin with "Set" will write settings to disk such that other instances
|
|
|
|
|
// referring to the same directories will also see them. Given that, the
|
|
|
|
|
// "Override" methods should be used only during the initialization steps
|
|
|
|
|
// for a Dir object, typically only inside "package main", so that all
|
|
|
|
|
// subsequent work elsewhere will access consistent locations on disk.
|
|
|
|
|
//
|
|
|
|
|
// We're gradually transitioning to using this type to manage working directory
|
|
|
|
|
// settings, and so not everything in the working directory "data dir" is
|
|
|
|
|
// encapsulated here yet, but hopefully we'll gradually migrate all of those
|
|
|
|
|
// settings here over time. The working directory state not yet managed in here
|
|
|
|
|
// is typically managed directly in the "command" package, either directly
|
|
|
|
|
// inside commands or in methods of the giant command.Meta type.
|
|
|
|
|
type Dir struct {
|
|
|
|
|
// mainDir is the path to the directory that we present as the
|
|
|
|
|
// "working directory" in the user model, which is typically the
|
|
|
|
|
// current working directory when running Terraform CLI, or the
|
|
|
|
|
// directory explicitly chosen by the user using the -chdir=...
|
|
|
|
|
// global option.
|
|
|
|
|
mainDir string
|
|
|
|
|
|
|
|
|
|
// originalDir is the path to the working directory that was
|
|
|
|
|
// selected when creating the Terraform CLI process, regardless of
|
|
|
|
|
// -chdir=... being set. This is only for very limited purposes
|
|
|
|
|
// related to backward compatibility; most functionality should
|
|
|
|
|
// use mainDir instead.
|
|
|
|
|
originalDir string
|
|
|
|
|
|
|
|
|
|
// dataDir is the path to the directory where we will store our
|
|
|
|
|
// working directory settings and artifacts. This is typically a
|
|
|
|
|
// directory named ".terraform" within mainDir, but users may
|
|
|
|
|
// override it.
|
|
|
|
|
dataDir string
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// NewDir constructs a new working directory, anchored at the given path.
|
|
|
|
|
//
|
|
|
|
|
// In normal use, mainPath should be "." to reflect the current working
|
|
|
|
|
// directory, with "package main" having switched the process's current
|
|
|
|
|
// working directory if necessary prior to calling this function. However,
|
|
|
|
|
// unusual situations in tests may set mainPath to a temporary directory, or
|
|
|
|
|
// similar.
|
|
|
|
|
//
|
|
|
|
|
// WARNING: Although the logic in this package is intended to work regardless
|
|
|
|
|
// of whether mainPath is actually the current working directory, we're
|
|
|
|
|
// currently in a transitional state where this package shares responsibility
|
|
|
|
|
// for the working directory with various command.Meta methods, and those
|
|
|
|
|
// often assume that the main path of the working directory will always be
|
|
|
|
|
// ".". If you're writing test code that spans across both areas of
|
|
|
|
|
// responsibility then you must ensure that the test temporarily changes the
|
|
|
|
|
// test process's working directory to the directory returned by RootModuleDir
|
|
|
|
|
// before using the result inside a command.Meta.
|
|
|
|
|
func NewDir(mainPath string) *Dir {
|
|
|
|
|
mainPath = filepath.Clean(mainPath)
|
|
|
|
|
return &Dir{
|
|
|
|
|
mainDir: mainPath,
|
|
|
|
|
originalDir: mainPath,
|
|
|
|
|
dataDir: filepath.Join(mainPath, ".terraform"),
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// OverrideOriginalWorkingDir records a different path as the
|
|
|
|
|
// "original working directory" for the reciever.
|
|
|
|
|
//
|
|
|
|
|
// Use this only to record the original working directory when Terraform is run
|
|
|
|
|
// with the -chdir=... global option. In that case, the directory given in
|
|
|
|
|
// -chdir=... is the "main path" to pass in to NewDir, while the original
|
|
|
|
|
// working directory should be sent to this method.
|
|
|
|
|
func (d *Dir) OverrideOriginalWorkingDir(originalPath string) {
|
|
|
|
|
d.originalDir = filepath.Clean(originalPath)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// OverrideDataDir chooses a specific alternative directory to read and write
|
|
|
|
|
// the persistent working directory settings.
|
|
|
|
|
//
|
|
|
|
|
// "package main" can call this if it detects that the user has overridden
|
|
|
|
|
// the default location by setting the relevant environment variable. Don't
|
|
|
|
|
// call this when that environment variable isn't set, in order to preserve
|
|
|
|
|
// the default setting of a dot-prefixed directory directly inside the main
|
|
|
|
|
// working directory.
|
|
|
|
|
func (d *Dir) OverrideDataDir(dataDir string) {
|
|
|
|
|
d.dataDir = filepath.Clean(dataDir)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// RootModuleDir returns the directory where we expect to find the root module
|
|
|
|
|
// configuration for this working directory.
|
|
|
|
|
func (d *Dir) RootModuleDir() string {
|
|
|
|
|
// The root module configuration is just directly inside the main directory.
|
|
|
|
|
return d.mainDir
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// OriginalWorkingDir returns the true, operating-system-originated working
|
|
|
|
|
// directory that the current Terraform process was launched from.
|
|
|
|
|
//
|
|
|
|
|
// This is usually the same as the main working directory, but differs in the
|
|
|
|
|
// special case where the user ran Terraform with the global -chdir=...
|
|
|
|
|
// option. This is here only for a few backward compatibility affordances
|
|
|
|
|
// from before we had the -chdir=... option, so should typically not be used
|
|
|
|
|
// for anything new.
|
|
|
|
|
func (d *Dir) OriginalWorkingDir() string {
|
|
|
|
|
return d.originalDir
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// DataDir returns the base path where the reciever keeps all of the settings
|
|
|
|
|
// and artifacts that must persist between consecutive commands in a single
|
|
|
|
|
// session.
|
|
|
|
|
//
|
|
|
|
|
// This is exported only to allow the legacy behaviors in command.Meta to
|
|
|
|
|
// continue accessing this directory directly. Over time we should replace
|
|
|
|
|
// all of those direct accesses with methods on this type, and then remove
|
|
|
|
|
// this method. Avoid using this method for new use-cases.
|
|
|
|
|
func (d *Dir) DataDir() string {
|
|
|
|
|
return d.dataDir
|
|
|
|
|
}
|
|
|
|
|
|
Implement controlling destroy functionality within Terraform Test (#37359)
* Add ability to parse backend blocks present in a test file's run blocks, validate configuration (#36541)
* Add ability to parse backend blocks from a run block
* Add validation to avoid multiple backend blocks across run blocks that use the same internal state file. Update tests.
* Add validation to avoid multiple backend blocks within a single run block. Update tests.
* Remove use of quotes in diagnostic messages
* Add validation to avoid backend blocks being used in plan run blocks. Update tests.
* Correct local backend blocks in new test fixtures
* Add test to show that different test files can use same backend block for same state key.
* Add validation to enforce state-storage backend types are used
* Remove TODO comment
We only need to consider one file at a time when checking if a state_key already has a backend associated with it; parallelism in `terraform test` is scoped down to individual files.
* Add validation to assert that the backend block must be in the first apply command for an internal state
* Consolidate backend block validation inside a single if statement
* Add initial version of validation that ensures a backend isn't re-used within a file
* Explicitly set the state_key at the point of parsing the config
TODO: What should be done with method (moduletest.Run).GetStateKey?
* Update test fixture now that reusing backend configs has been made invalid
* Add automated test showing validation of reused configuration blocks
* Skip test due to flakiness, minor change to test config naming
* Update test so it tolerates non-deterministic order run blocks are evaluated in
* Remove unnecessary value assignment to r.StateKey
* Replace use of GetStateKey() with accessing the state key that's now set during test config parsing
* Fix bug so that run blocks using child modules get the correct state key set at parsing time
* Update acceptance test to also cover scenario where root and child module state keys are in use
* Update test name
* Add newline to regex
* Ensure consistent place where repeat backend error is raised from
* Write leftover test state(s) to file (#36614)
* Add additional validation that the backend used in a run is a supported type (#36648)
* Prevent test run when leftover state data is present (#36685)
* `test`: Set the initial state for a state files from a backend, allow the run that defines a backend to write state to the backend (#36646)
* Allow use of backend block to set initial state for a state key
* Note about alternative place to keep 'backend factories'
* Allow the run block defining the backend to write state to it
* Fix rebase
* Change to accessing backend init functions via ContextOpts
* Add tests demonstrating how runs containing backend blocks use and update persisted state
* Fix test fixture
* Address test failure due to trouble opening the state file
This problem doesn't happen on MacOS, so I assume is due to the Linux environment of GitHub runners.
* Fix issue with paths properly
I hope
* Fix defect in test assertion
* Pivot back to approach introduced in 4afc3d7
* Let failing tests write to persistent state, add test case covering that.
I split the acceptance tests into happy/unhappy paths for this, which required some of the helper functions' declarations to be raised up to package-level.
* Change how we update internal state files, so that information about the associated backend is never lost
* Fix UpdateStateFile
* Ensure that the states map set by TestStateTransformer associates a backend with the correct run.
* Misc spelling fixes in comments and a log
* Replace state get/set functions with existing helpers (#36747)
* Replace state get/set functions with existing helpers
* Compare to string representation of state
* Compare to string representation of state
* Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729)
* Add validation to enforce skip_cleanup=false cannot be used with backend blocks (#36857)
* Integrate use of backend blocks in tests with skip_cleanup feature (#36848)
* Fix nil pointer error, update test to not be table-driven
* Make using a backend block implicitly set skip_cleanup to true
* Stop state artefacts being created when a backend is in use and no cleanup errors have occurred
* Return diagnostics so calling code knows if cleanup experienced issues or not
* Update tests to show that when cleanup fails a state artefact is created
* Add comment about why diag not returned
* Bug fix - actually pull in the state from the state manager!
* Split and simplify (?) tests to show the backend block can create and/or reuse prior state
* Update test to use new fixtures, assert about state artefact. Fix nil pointer
* Update test fixture in use, add guardrail for flakiness of forced error during cleanup
* Refactor so resource ID set in only one place
* Add documentation for using a `backend` block during `test` (#36832)
* Add backend as a documented block in a run block
* Add documentation about backend blocks in run blocks.
* Make the relationship between backends and state keys more clear, other improvements
* More test documentation (#36838)
* Terraform Test: cleanup command (#36847)
* Allow cleanup of states that depend on prior runs outputs (#36902)
* terraform test: refactor graph edge calculation
* create fake run block nodes during cleanup operation
* tidy up TODOs
* fix tests
* remove old changes
* Update internal/moduletest/graph/node_state_cleanup.go
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
* Improve diagnostics around skip_cleanup conflicts (#37385)
* Improve diagnostics around skip_cleanup conflicts
* remove unused dynamic node
* terraform test: refactor manifest file for simplicity (#37412)
* test: refactor apply and plan functions so no run block is needed
* terraform test: write and load state manifest files
* Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729)
* terraform test: add support for skip_cleanup attr
* terraform test: add cleanup command
* terraform test: add backend blocks
* pause
* fix tests
* remove commented code
* terraform test: make controlling destroy functionality experimental (#37419)
* address comments
* Update internal/moduletest/graph/node_state_cleanup.go
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
---------
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
* add experimental changelog entries
---------
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>
2025-09-10 11:22:20 -04:00
|
|
|
// TestDataDir returns the path where the receiver keeps settings
|
|
|
|
|
// and artifacts related to terraform tests.
|
|
|
|
|
func (d *Dir) TestDataDir() string {
|
|
|
|
|
return filepath.Join(d.dataDir, "test")
|
|
|
|
|
}
|
|
|
|
|
|
2021-09-01 20:01:44 -04:00
|
|
|
// ensureDataDir creates the data directory and all of the necessary parent
|
|
|
|
|
// directories that lead to it, if they don't already exist.
|
|
|
|
|
//
|
|
|
|
|
// For directories that already exist ensureDataDir will preserve their
|
|
|
|
|
// permissions, while it'll create any new directories to be owned by the user
|
|
|
|
|
// running Terraform, readable and writable by that user, and readable by
|
|
|
|
|
// all other users, or some approximation of that on non-Unix platforms which
|
|
|
|
|
// have a different permissions model.
|
|
|
|
|
func (d *Dir) ensureDataDir() error {
|
|
|
|
|
err := os.MkdirAll(d.dataDir, 0755)
|
|
|
|
|
if err != nil {
|
|
|
|
|
return fmt.Errorf("failed to prepare working directory: %w", err)
|
|
|
|
|
}
|
|
|
|
|
return nil
|
|
|
|
|
}
|