* Enable parsing of blocks in individual files
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* Enable handling of state stores when parsing a module from its constituent files.
This includes: validations of duplicates and clashes, supporting override files.
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* Add tests for validation that detects conflicts between state_store blocks and others
* Add tests for state_store override behavior
* Add tests for validation that detects when conflicting state-related blocks are used, either in the same file or across separate files
* Update error message summaries to explicitly say blocks conflict
* Add small changes to assertions in state_store override tests
* Update state_store block parsing to expect scoped provider block
* Update tests following syntax change
* Make config parsing experimental
* Remove testModuleFromDirWithExperiment, as testModuleFromDirWithExperiments exists!
* Update code comment
---------
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
Co-authored-by: Radek Simko <radek.simko@gmail.com>
* terraform test: add variable definitions to test files
Currently, `terraform test` attempts to work out the type of any external variables by delaying evaluation until each run block executes so it can use the definitions within the run blocks's module. This means that the values of variables can technically change between run blocks which isn't ideal.
This commit is the first in a chain which will move the evaluation of variables into the terraform test graph. We need to give the users the option of specifying the type for external variables within the file as these variables are now going to be assessed outside of the context of a run block. To do this, we introduce the optional variable blocks that are added by this commit.
* update comment
* Add details to `<failure>` element describing which assertion failed
* Remove unused diagnostic string
* Set `message` attribute of `<failure>` element to failed assertion's error message
* Make `<failure>` contain diagnostic's message, refactor how `message` attribute is set
* Ensure that system-err is only added when needed
* Update test fixtures
* Make diags usage clearer, ensure all test failure diags in "failure" element
* Refactor how "skipped" element value is set
* Fix failing test Test_TestJUnitXMLFile_Save
* feat: add state_alias for test run blocks
* fix: move `state_alias` to run block + rename to `state_key`
* docs: add state_key changelog entry
* docs: add state_key in run block documentation
* Revert "docs: add state_key in run block documentation"
This reverts commit ccccf62a76.
* Revert "docs: add state_key changelog entry"
This reverts commit 86e2ad8dc3.
* Refactor JUnit XML output to use new concept of an Artifact
* Move JUnit-related code into new `artifact` package
* Refactor Artifact's Save method to return diagnostics, update comments
Previously TestJUnitXMLFile implemented the View interface, which cannot return errors. Now it's not a View any more we can simplify things.
* Make junitXMLTestReport output deterministic by iterating over a slice instead of a map, add test
* Provide sources to junitXMLTestReport, allowing complete error messages in the XML
We need to ensure that artifact.NewTestJUnitXMLFile is called once the config Loader is available as a non-nil pointer
* Whitespace
* Add some test coverage for JUnit XML output for `terraform test`
* Refactor how file is saved, add tests
* Move XML structs definitions outside of `junitXMLTestReport`
* Fix nil pointer bug
* Add missing file headers
* Refactor comparison of byte slices
* Rename package to `junit`, rename structs to match
* Add a test showing JUnit output when a test is skipped by the user
The CLI attempts to prevent the user from inputting non-ephemeral
variables during apply which don't match the stored plan values. Since
the user can't avoid variables parsed from auto-loaded files, we have to
ignore those values.
Non-string input variables taken from the environment initially need to
be parsed and stored as string, since there is no type associated type
information. Make sure these are correctly handled when validated during
apply.
* check root output values when determining plan applicability in refresh mode
In refresh-only mode, we do not anticipate proposing any actions; however, a plan is marked as “applyable” if there are changes in the state between runs. Currently, a plan is considered “applyable” only when there are differences in managed resources. This approach seems to overlook changes in root output values. As a result, a plan can be marked as non-applyable, even when there are changes in the root output value, due to the lack of detected changes since only managed resources were checked.
* set 'applyable' to true
* show deprecation warning if -state is used with plan, apply, refresh
* show deprecation warning if -state is used with plan, apply, refresh
* updated -state flag check condition
* added better diagnostic details view
* resolved failed test
* updated the content of the -state flag warning
* lang: stabilise ephemeral values experiment
Stabilise the ephemeral values experiment and ephemeralasnull
experimental function.
* simplify handling of apply time input vars
As the final step in stabilising the ephemeral values experiment, we can
remove the separate code path for handling input variables supplied via
-var and -var-file during apply.
The intent here is conveyed in the tests: when applying a saved plan,
all ephemeral variables set in the plan (the "apply-time
variables"), must be set, and no others.
As per the TODO copied from the prototype, there is some more work to be
done here in making the handling of undeclared variables during apply as
sophisticated as that during plan, emitting helpful warnings (for
example) when input variables are supplied unnecessarily via environment
variables.
* experiments: officially conclude EV experiment
---------
Co-authored-by: Radek Simko <radek.simko@gmail.com>
* stop invalid warning during cleanup phase
* stop invalid warning during cleanup phase
* invalid warning during cleanup phase
* remove unwanted comment
* update GetVariables parameter for including warnings
* invalid warning during cleanup phase test case
* update invalid warnings in cleanup test case to throw a warning for the validation variable before cleanup phase
* remove unwanted warnings
* terraform test: Push evaluation of variables to as late as possible
* Update internal/moduletest/hcl/variable_cache.go
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
* address comments
---------
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
These ideas are both already implied by some logic elsewhere in the system,
but until now we didn't have the decision logic centralized in a single
place that could therefore evolve over time without necessarily always
updating every caller together.
We'll now have the modules runtime produce its own boolean ruling about
each characteristic, which callers can rely on for the mechanical
decision-making of whether to offer the user an "approve" prompt, and
whether to remind the user after apply that it was an incomplete plan
that will probably therefore need at least one more plan/apply round to
converge.
The "Applyable" flag directly replaces the previous method Plan.CanApply,
with equivalent logic. Making this a field instead of a method means that
we can freeze it as part of a saved plan, rather than recalculating it
when we reload the plan, and we can export the field value in our export
formats like JSON while ensuring it'll always be consistent with what
Terraform is using internally.
Callers can (and should) still use other context in the plan to return
more tailored messages for specific situations they already know about
that might be useful to users, but with these flags as a baseline callers
can now just fall back to a generic presentation when encountering a
situation they don't yet understand, rather than making the wrong decision
and causing something strange to happen. That is: a lack of awareness of
a new rule will now cause just a generic message in the UI, rather than
incorrect behavior.
This commit mostly just deals with populating the flags, and then all of
the direct consequences of that on our various tests. Further changes to
actually make use of these flags elsewhere in the system will follow in
later commits, both in this repository and in other repositories.
* terraform fmt: add .tfmock.hcl as a supported file extension
* terraform fmt: add test cases for .tfmock.hcl files formatting
* fix: update test to run on .tfmock.hcl files
* Add logic to collect variables for terrafrom test
* Add tests for test variable collection
* Update the test variable collection implementation
* Update internal/backend/local/test.go
Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>
* Update internal/backend/local/test.go
Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>
* Move test variables into var file
* resolve diff from cross-branch switch
* go fmt
---------
Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>
Unless a user specifically requests a real operation graph using the
-type option, we'll by default present a simplified graph which only
represents the relationships between resources, since resources are the
main side-effects and so the ordering of these is more interesting than
the ordering of Terraform's internal implementation details.
* Process only relevant variables in the testing framework
* add support for variables within test provider configurations
* remove unused function
* fix consistency checks
Adds a client that encapsulates the process of downloading a cloudplugin manifest from a Terraform Cloud, downloads the appropriate binary to the specified location, and validates that it was distributed by HashiCorp
* [testing framework] prepare for beta phase of development
* [Testing Framework] Add module block to test run blocks
* [testing framework] allow tests to define and override providers
* testing framework: introduce interrupts for stopping tests
* remove panic handling, will do it properly later
* [testing framework] prepare for beta phase of development
* [Testing Framework] Add module block to test run blocks
* [testing framework] allow tests to define and override providers
