We previously used a loader -> BuildConfig flow to load configuration.
This commit changes most (but not all yet) flows to use the new
graph-based approach. Instead of simply recursively loading the modules,
we now need to take a stepped approach:
1. Load the root module
2. Collect the variables and their values
3. Build the configuration with the graph-based approach
Because this approach relies on different parts from different packages,
it can't easliy be done within the `configload` package. So, now we do
most of in the backend or command.
* Add ability to parse backend blocks present in a test file's run blocks, validate configuration (#36541)
* Add ability to parse backend blocks from a run block
* Add validation to avoid multiple backend blocks across run blocks that use the same internal state file. Update tests.
* Add validation to avoid multiple backend blocks within a single run block. Update tests.
* Remove use of quotes in diagnostic messages
* Add validation to avoid backend blocks being used in plan run blocks. Update tests.
* Correct local backend blocks in new test fixtures
* Add test to show that different test files can use same backend block for same state key.
* Add validation to enforce state-storage backend types are used
* Remove TODO comment
We only need to consider one file at a time when checking if a state_key already has a backend associated with it; parallelism in `terraform test` is scoped down to individual files.
* Add validation to assert that the backend block must be in the first apply command for an internal state
* Consolidate backend block validation inside a single if statement
* Add initial version of validation that ensures a backend isn't re-used within a file
* Explicitly set the state_key at the point of parsing the config
TODO: What should be done with method (moduletest.Run).GetStateKey?
* Update test fixture now that reusing backend configs has been made invalid
* Add automated test showing validation of reused configuration blocks
* Skip test due to flakiness, minor change to test config naming
* Update test so it tolerates non-deterministic order run blocks are evaluated in
* Remove unnecessary value assignment to r.StateKey
* Replace use of GetStateKey() with accessing the state key that's now set during test config parsing
* Fix bug so that run blocks using child modules get the correct state key set at parsing time
* Update acceptance test to also cover scenario where root and child module state keys are in use
* Update test name
* Add newline to regex
* Ensure consistent place where repeat backend error is raised from
* Write leftover test state(s) to file (#36614)
* Add additional validation that the backend used in a run is a supported type (#36648)
* Prevent test run when leftover state data is present (#36685)
* `test`: Set the initial state for a state files from a backend, allow the run that defines a backend to write state to the backend (#36646)
* Allow use of backend block to set initial state for a state key
* Note about alternative place to keep 'backend factories'
* Allow the run block defining the backend to write state to it
* Fix rebase
* Change to accessing backend init functions via ContextOpts
* Add tests demonstrating how runs containing backend blocks use and update persisted state
* Fix test fixture
* Address test failure due to trouble opening the state file
This problem doesn't happen on MacOS, so I assume is due to the Linux environment of GitHub runners.
* Fix issue with paths properly
I hope
* Fix defect in test assertion
* Pivot back to approach introduced in 4afc3d7
* Let failing tests write to persistent state, add test case covering that.
I split the acceptance tests into happy/unhappy paths for this, which required some of the helper functions' declarations to be raised up to package-level.
* Change how we update internal state files, so that information about the associated backend is never lost
* Fix UpdateStateFile
* Ensure that the states map set by TestStateTransformer associates a backend with the correct run.
* Misc spelling fixes in comments and a log
* Replace state get/set functions with existing helpers (#36747)
* Replace state get/set functions with existing helpers
* Compare to string representation of state
* Compare to string representation of state
* Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729)
* Add validation to enforce skip_cleanup=false cannot be used with backend blocks (#36857)
* Integrate use of backend blocks in tests with skip_cleanup feature (#36848)
* Fix nil pointer error, update test to not be table-driven
* Make using a backend block implicitly set skip_cleanup to true
* Stop state artefacts being created when a backend is in use and no cleanup errors have occurred
* Return diagnostics so calling code knows if cleanup experienced issues or not
* Update tests to show that when cleanup fails a state artefact is created
* Add comment about why diag not returned
* Bug fix - actually pull in the state from the state manager!
* Split and simplify (?) tests to show the backend block can create and/or reuse prior state
* Update test to use new fixtures, assert about state artefact. Fix nil pointer
* Update test fixture in use, add guardrail for flakiness of forced error during cleanup
* Refactor so resource ID set in only one place
* Add documentation for using a `backend` block during `test` (#36832)
* Add backend as a documented block in a run block
* Add documentation about backend blocks in run blocks.
* Make the relationship between backends and state keys more clear, other improvements
* More test documentation (#36838)
* Terraform Test: cleanup command (#36847)
* Allow cleanup of states that depend on prior runs outputs (#36902)
* terraform test: refactor graph edge calculation
* create fake run block nodes during cleanup operation
* tidy up TODOs
* fix tests
* remove old changes
* Update internal/moduletest/graph/node_state_cleanup.go
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
* Improve diagnostics around skip_cleanup conflicts (#37385)
* Improve diagnostics around skip_cleanup conflicts
* remove unused dynamic node
* terraform test: refactor manifest file for simplicity (#37412)
* test: refactor apply and plan functions so no run block is needed
* terraform test: write and load state manifest files
* Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729)
* terraform test: add support for skip_cleanup attr
* terraform test: add cleanup command
* terraform test: add backend blocks
* pause
* fix tests
* remove commented code
* terraform test: make controlling destroy functionality experimental (#37419)
* address comments
* Update internal/moduletest/graph/node_state_cleanup.go
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
---------
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
* add experimental changelog entries
---------
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>
Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>
* Add a helper for mocking a provider with a state store
* Update tests to use the new helper
* Update other test fixtures for consistency
The related tests pass before and after this change
* Update testing helper to return the underlying type, not the interface
* Store the FQN of the provider used in PSS in representations of the parsed config.
This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places.
* Update affected tests, improve error diagnostic
* Begin enabling method to return a backend.Backend made using state_store config.
State store config can now be received via BackendOpts and there is rough validation of whether the config makes sense (does the provider offer a store with the given name?).
* Update code's logic to include possibility of a state store being in use
At this point there are no cases that actually handle the state store scenarios though!
* Add empty cases for handle all broad init scenarios involving PSS
* Update default case's error to report state store variables
* Improve how we resolve the builtin terraform provider's address
* Add test that hits the code path for adding a state store to a new (or implied local) project
* Add test for use of `-reconfigure` flag; show that it hits the code path for adding a state store for the first time
* Add test that hits the code path for removing use of a state store, migrating to (implied) local backend
* Add test that hits the code path for changing a state store's configuration
* Update existing test names to be backend-specific
* Add tests that hits the code path for migrating between PSS and backends
* Consolidate PSS-related tests at end of the file
* Fix log text
* Add test showing that using variables is disallowed with state_store and nested provider blocks
* Update test name
* Fix test cases
* Add TODOs so we remember to remove experiments from tests
* Update state store-related tests to use t.Cleanup
* Remove use of `testChdir`
* feat: update Actions parsing with current syntax
Updating actions to match the current RFC:
- config is in a block
- linked_resource(s) exist and need parsing
I added a very basic test for the decoding, and updated existing tests with the correct syntax.
* very important copyrighted test file
* Add `limit` attribute to list blocks
* Add limit and include_resource args to list RPC
* Test list arguments from config
* Collect list block related references
* Enable parsing of blocks in individual files
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* Enable handling of state stores when parsing a module from its constituent files.
This includes: validations of duplicates and clashes, supporting override files.
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* Add tests for validation that detects conflicts between state_store blocks and others
* Add tests for state_store override behavior
* Add tests for validation that detects when conflicting state-related blocks are used, either in the same file or across separate files
* Update error message summaries to explicitly say blocks conflict
* Add small changes to assertions in state_store override tests
* Update state_store block parsing to expect scoped provider block
* Update tests following syntax change
* Make config parsing experimental
* Remove testModuleFromDirWithExperiment, as testModuleFromDirWithExperiments exists!
* Update code comment
---------
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
Co-authored-by: Radek Simko <radek.simko@gmail.com>
* Add comments, rename tests, reorder tests
This commit doesn't change or add any logic
* Add test for overriding a cloud block with a backend block
* Add tests showing multiple instances of cloud or backend blocks cause an error
* Add test showing that backend and cloud blocks cannot be in the same terraform block
* Add test case asserting it's not valid to have both cloud and backend blocks across files in a module
* Clarify comments and replace some comments with test names
* Replace comments with test names
* Decode action blocks and action_triggers inside resource blocks.
This commit adds decoding of action and action_triggers inside terraform configuration. I added an Actions experiment as a hacky way of keeping the functionality out of main until we're ready for the alpha; this may never be an experiment but it's a handy feature flag so we don't have to do all the work in a long-lived feature branch.
* remove legacy shim handling
* validate that the referenced entry in actions is indeed an action
* lang: stabilise ephemeral values experiment
Stabilise the ephemeral values experiment and ephemeralasnull
experimental function.
* simplify handling of apply time input vars
As the final step in stabilising the ephemeral values experiment, we can
remove the separate code path for handling input variables supplied via
-var and -var-file during apply.
The intent here is conveyed in the tests: when applying a saved plan,
all ephemeral variables set in the plan (the "apply-time
variables"), must be set, and no others.
As per the TODO copied from the prototype, there is some more work to be
done here in making the handling of undeclared variables during apply as
sophisticated as that during plan, emitting helpful warnings (for
example) when input variables are supplied unnecessarily via environment
variables.
* experiments: officially conclude EV experiment
---------
Co-authored-by: Radek Simko <radek.simko@gmail.com>
When the ephemeral_values experiment is active, a module author can
designate individual input variables and output values as being
"ephemeral", which doesn't currently do anything but in future commits
will represent that the values can be used only in locations that don't
require Terraform to persist the value as part of state snapshots or
saved plan files.
While variable validation rules can now include other references, they
also still need to reference the variable itself. If the variable itself
wasn't referenced, then the resulting `Invalid value for variable`
error would not make sense.
Having no self-references will also cause a panic when trying to
reinsert the variable name into the evaluation context, because the
context variables map could be nil. A check for a nil map does not need
to be added however, because ensuring that a self-reference exists means
there will always be at least 1 variable in scope.
Previously we introduced a language experiment that would permit variable
validation rules to refer to other objects declared in the same module
as the variable.
Now that experiment is concluded and its behavior is available for all
modules.
This final version deviates slightly from the experiment: we learned from
the experimental implementation that we accidentally made the "validate"
command able to validate constant-valued input variables in child modules
despite the usual rule that input variables are unknown during validation,
because the previous compromise bypassed the main expression evaluator
and built its own evaluation context directly.
Even though that behavior was not intended, it's a useful behavior that is
protected by our compatibility promises and so this commit includes a
slightly hacky emulation of that behavior, in eval_variable.go, that
fetches the variable value in the same way the old implementation would
have and then modifies the hcl evaluation context to include that value,
while preserving anything else that our standard evaluation context
builder put in there. That narrowly preserves the old behavior for
expressions that compare the variable value directly to a constant, while
treating all other references (which were previously totally invalid) in
the standard way. This quirk was already covered by the existing test
TestContext2Validate_variableCustomValidationsFail, which fails if the
special workaround is removed.
When a module opts in to the variable_validation_crossref experiment,
Terraform will treat variable validation rules in a similar way as any
other dynamic expression, allowing it to refer to other objects that are
declared in the same module as the variable.
Because the check for a validation rule referring only to itself is now
experiments-sensitive we need to do it only once the full module has been
assembled, rather than during assembly of individual files, and so the
"badref" test in package configs is now classified under "invalid-modules"
rather than "invalid-files". It's otherwise unchanged, and the existing
test that a validation rule must refer to at least _something_ is still
present because that remains true even when opted in to the experiment.
This is experimental for now because although it seems pretty plausible
that it should be okay, this is allowing full evaluation in a new place
where it wasn't allowed before and so we'd like to ask those interested
in this feature to try it out with some of their realistic use-cases and
make sure this solution is a good fit before we lock it in as a
compatibility constraint.
The validation for checking if an import target is the source of a moved
block is incorrect. Besides the noted FIXME of the check being incorrect
for moved modules (and also resources scoped within modules); an import
can only be statically identified at the granularity of a resource
block, but moved block can have individual instances in the `from`
field.
Since we already validate that a resource block must _not_ exist in the
config for `moved`, we can rely on that causing `import` to fail with a
nonexistant resource block during evaluation if they have the same
target.
Because expression from JSON files are handled differently from HCL, we
need to deal with these manually when evaluating the special "to"
syntax. We take the same approach as was done for replace_triggered_by,
where the raw string is extracted from the json expression, and
re-evaluated as an HCL expression.
* [testing framework] prepare for beta phase of development
* [Testing Framework] Add module block to test run blocks
* [testing framework] allow tests to define and override providers
* command: keep our promises
* remove some nil config checks
Remove some of the safety checks that ensure plan nodes have config attached at the appropriate time.
* add GeneratedConfig to plan changes objects
Add a new GeneratedConfig field alongside Importing in plan changes.
* add config generation package
The genconfig package implements HCL config generation from provider state values.
Thanks to @mildwonkey whose implementation of terraform add is the basis for this package.
* generate config during plan
If a resource is being imported and does not already have config, attempt to generate that config during planning. The config is generated from the state as an HCL string, and then parsed back into an hcl.Body to attach to the plan graph node.
The generated config string is attached to the change emitted by the plan.
* complete config generation prototype, and add tests
* plannable import: add a provider argument to the import block
* Update internal/configs/config.go
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
* Update internal/configs/config.go
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
* Update internal/configs/config.go
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
* fix formatting and tests
---------
Co-authored-by: Katy Moe <katy@katy.moe>
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
* command: keep our promises
* remove some nil config checks
Remove some of the safety checks that ensure plan nodes have config attached at the appropriate time.
* add GeneratedConfig to plan changes objects
Add a new GeneratedConfig field alongside Importing in plan changes.
* add config generation package
The genconfig package implements HCL config generation from provider state values.
Thanks to @mildwonkey whose implementation of terraform add is the basis for this package.
* generate config during plan
If a resource is being imported and does not already have config, attempt to generate that config during planning. The config is generated from the state as an HCL string, and then parsed back into an hcl.Body to attach to the plan graph node.
The generated config string is attached to the change emitted by the plan.
* complete config generation prototype, and add tests
---------
Co-authored-by: Katy Moe <katy@katy.moe>
Applying object type defaults to null values can convert null to an
object with partial attributes. This means that even a specified default
value of null will not remain null after variable evaluation.
In turn, the result can then be invalid, if not all attributes in an
object type have defaults specified.
This commit skips the type default application step during config load
and variable evaluation if the default or given value is null of any
type. We still perform type conversion.
