As part of our efforts to remove the large snapshot of the legacy SDK that
we've been using, this replaces the uses of that package with our new
lightweight "backendbase" package, which is a collection of helpers that
can substitute for most of the parts of the legacy SDK that backends tend
to use.
For this backend for now we'll use the "SDK like" helpers which aim to
retain some of the legacy-SDK-specific assumptions that aren't technically
true anymore, such as the idea that null and empty string are equivalent.
Hopefully one day we'll be able to wean this backend off of that more
completely, but this is a pragmatic way to get away from the legacy SDK
without a large rewrite.
This also drew attention to the fact that this backend was previously
relying on the weird context.Context value that the SDK was passing in
to the configure method, even though that context isn't connected up to
anything particularly useful. In the long run we should change the backend
APIs to pass context.Context to each method that might make network
requests, but for now we're using context.TODO so that we can more easily
find these cases and update them later once there's a real context to use.
In the very first implementation of "sensitive values" we were
unfortunately not disciplined about separating the idea of "marked value"
from the idea of "sensitive value" (where the latter is a subset of the
former). The first implementation just assumed that any marking whatsoever
meant "sensitive".
We later improved that by adding the marks package and the marks.Sensitive
value to standardize on the representation of "sensitive value" as being
a value marked with _that specific mark_.
However, we did not perform a thorough review of all of the mark-handling
codepaths to make sure they all agreed on that definition. In particular,
the state and plan models were both designed as if they supported arbitrary
marks but then in practice marks other than marks.Sensitive would be
handled in various inconsistent ways: dropped entirely, or interpreted as
if marks.Sensitive, and possibly do so inconsistently when a value is
used only in memory vs. round-tripped through a wire/file format.
The goal of this commit is to resolve those oddities so that there are now
two possible situations:
- General mark handling: some codepaths genuinely handle marks
generically, by transporting them from input value to output value in
a way consistent with how cty itself deals with marks. This is the
ideal case because it means we can add new marks in future and assume
these codepaths will handle them correctly without any further
modifications.
- Sensitive-only mark preservation: the codepaths that interact with our
wire protocols and file formats typically have only specialized support
for sensitive values in particular, and lack support for any other
marks. Those codepaths are now subject to a new rule where they must
return an error if asked to deal with any other mark, so that if we
introduce new marks in future we'll be forced either to define how we'll
avoid those markings reaching the file/wire formats or extend the
file/wire formats to support the new marks.
Some new helper functions in package marks are intended to standardize how
we deal with the "sensitive values only" situations, in the hope that
this will make it easier to keep things consistent as the codebase evolves
in future.
In practice the modules runtime only ever uses marks.Sensitive as a mark
today, so all of these checks are effectively covering "should never
happen" cases. The only other mark Terraform uses is an implementation
detail of "terraform console" and does not interact with any of the
codepaths that only support sensitive values in particular.
This helper is a slightly more convenient wrapper around cty's own
"MarkWithPaths" function that applies a single given mark at zero or more
paths in a given value.
In several parts of Terraform we preserve sensitive marks in particular
for serialization and cannot preserve anything else, and so this helper
is useful for reapplying the saved sensitive paths back to the stored
value during decoding.
This helper deals with the somewhat-common situation where a particular
codepath can handle only one kind of marking, such as in our various wire
and file format serializers which have special support for serializing
sensitive value paths but cannot preserve any other marks.
The assumption here is that the caller will probably return some sort of
error if the second return value is non-empty, but we leave thec caller to
decide exactly what to do because different situations might warrant
different treatment of the problem, such as whether it's considered to be
a Terraform language authoring error or to be a bug in Terraform.
This extends the "CmpOptions" to include a comparer for cty.Path values,
so that we don't have to hand-write that in each case where we're
comparing data structures containing those.
Our JSON state format has support for sensitive marks in particular but
lacks generalized support and so cannot deal with any other marks.
Previously it just assumed that any mark must be the sensitive mark, but
now we'll reject other marks to ensure that if any new marks are added we
must consider whether and how the JSON state format should handle them.
Everything from ioutil has been moved into other locations in modern Go.
This particular package only used ioutil.ReadAll and ioutil.ReadFile,
which have been replaced by io.ReadAll and os.ReadFile respectively.
The plan file format can only preserve the "sensitive" mark, but previously
it was just silently treating any other mark as if it were the sensitive
mark.
Now we'll reject any other marks at serialization time. There are not
currently any other marks used by the modules runtime and so in practice
this cannot fail yet, but this is here to guard against misbehavior if we
introduce new marks in future without considering whether and how they
are to be serialized. (For any mark we choose not to serialize, it'll be
the caller's responsibility to replace the values using unsupported marks
with suitable unmarked placeholders.)
The state serialization only knows how to save and restore the
marks.Sensitive mark in particular, but previously it was just assuming
that any mark it found was sensitive without actually checking.
Now we'll return an error if we're asked to save a mark we don't support.
In practice there are no other marks currently used by the modules runtime
and so this cannot fail, but this is to help notice problems sooner if we
introduce any new marks later.
