* marshal state and set in the proto req for generate resource config
* changelog
* set state in the proto req for generate resource config in plugin6
* add GRPC tests for GenerateResourceConfig
* remove changelog entry since this fix is backported to 1.14
* Implement ReadStateBytes + WriteStateBytes
* [WIP] - Testing ReadStateBytes and WriteStateBytes (#37464)
* Fix nil pointer error
* Add WIP test for ReadStateBytes
* Move test mock to separate testing file
* Update mock to send unexpected EOF when there's a problem returning data and it's not a true EOF
* Add test case for when length != expected length
* Add test for when trying to read state from a store type that doesn't exist
* Change symbol names to lowercase
* Add ability to force a diagnostic to be returned from `mockReadStateBytesClient`'s `Recv` method
* Add test showing error diagnostics raised by the ReadStateBytes client are returned
* Add missing header
* Simplify mock by using an embedded type
* Rename `mockOpts` to `mockReadStateBytesOpts`
* Update existing tests to assert what arguments are passed to the RPC method call
* Add mock WriteStateBytesClient which uses `go.uber.org/mock/gomock` to enable assertions about calls to Send
* Add a test for WriteStateBytes that makes assertions about calls to the Send method
* Update test case to explicitly test writing data smaller than the chunk size
* Implement chunking in WriteStateBytes, add test case to assert expected chunking behaviour
* Add generated mock for Provider_WriteStateBytesClient in protocol v6
* Update tests to use new `MockProvider_WriteStateBytesClient`, remove handwritten mock
* Update code comments in test
* Add tests for diagnostics and errors returned during WriteStateBytes
* Add generated mock for Provider_ReadStateBytesClient in protocol v6, replace old mock
* Add test case for grpc errors in ReadStateBytes, fix how error is returned
* Typo in comment
* Add missing warning test, rename some test cases
* Update proto file definition of Read/WriteStateBytes RPCs (#37529)
* Update Read/WriteStateBytes RPCs to match https://github.com/hashicorp/terraform-plugin-go/pull/531
* Run `make protobuf`
* Run `make generate`
* Update use of `proto.ReadStateBytes_ResponseChunk` in tests
* Fix how diagnostics are handled alongside EOF error, update ReadStateBytes test
* More fixes - test setup was incorrect
I think? I assume that a response would be returned full of zero-values when EOF is encountered.
* WIP - avoid crash if chunk is nil
* Sarah's updates to radek/pss-read-write (#37642)
* Update code to not expect a chunk when EOF encountered
* Return early if any grpc errors are encountered during ReadStateBytes
* Close the stream with CloseSend once everything's read without error. Add test case about handling grpc errors from CloseSend.
* Fix test case about warnings: We would expect to receive a chunk with data alongside the warning and have a normal closing of the stream after EOF
* Add log line, remove unneeded type info
* Implement configurable state chunk size
* handle metadata in WriteStateBytes correctly
* validate chunk sizes received from provider
* ReadStateBytes: avoid early return on warnings
---------
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
This updates the event processing logic. Terraform now allows error and
warning diagnostics for list events with all other fields set to null.
The combination of a warning with a valid result is still possible.
* Add ValidateActionConfig to provider interface and protocol
* Update internal/provider-simple-v6/provider.go
Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>
---------
Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>
* Add GetStates RPC method to Plugin Protocol v6
* Add DeleteState RPC method to Plugin Protocol v6
* Add methods to provider-simple
* Fix error messages
* Apply suggestions from code review
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* Change how GetStates RPC returns the list of state names
* Change GetStates implementation to use new data type
* `make protobuf` to accommodate new code comments
---------
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* Implement ListResource RPC via callbacks
* Implement ListResource with iterators
* Change `ListResourceResponse` to slice
* Turn ListResourceResponse into a struct
* Add a limit for ListResource
* explicitly cancel stream when listing resources
---------
Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>
* Minor auto-formatting changes
* Add list RPC and schema protobuf definitions
* make protobuf
* make generate
* Add ValidateListResourceConfig implementation
We are moving the deferred reason from plans/deferring to providers
to have a single representation of deferred reason to be used from
providers to the plan. Using the plan's representation causes circular
dependencies.
Co-authored-by: Matej Risek <matej.risek@hashicorp.com>
"ProviderMeta" is a niche feature that's used by only two providers, but
this code was written under the assumption that the caller would always
provide it for any provider that has a schema for it, and unfortunately
the Terraform SDK seems to always provide a meta schema even for providers
that don't use it, and thus it's empty.
The current phase of the "unknown_instances" language experiment is not
fully wired in to the main logic as a way to reduce the risk of it
impacting behavior for those not participating in the experiment, but
that means that right now it doesn't actually have a resolved ProviderMeta
value to present, and so was just omitting it on the assumption that it
is always optional to provide anyway.
As a pragmatic stopgap to resolve that conflict, this makes the gRPC
provider dispatching logic tolerate an absent ProviderMeta value and
synthesize a null value to use in that case. This makes that logic slightly
more robust and also means we can defer doing all of the work to weave
ProviderMeta into this experimental codepath. The new TODO comment in that
codepath is intended to remind us to consider this again should we decide
to stablize the experiment later; when we do that, we'll hopefully do it
by refactoring to share more code between the two codepaths, at which
point ProviderMeta will be easier to support.
The call site for language functions doesn't currently have a way to
handle complex diagnostics, so rather than appear to support them in the
protocol we remove the concepts of diagnostics for now. We do however
retain the argument index fields, which we can wrap in a
function.ArgError and get a little more precise hcl diagnostic from
expression.
This change ensures that providers receive the source private state and can store potentially differing private state as part of a move, rather than always copying the source private state to the target. This functionality has not yet been released and therefore there are no compatibility concerns.
* Update proto schema and provider interfaces with support for moved across resource type RPCs
* address comments
* remove unused functions
* remove support for flatmap format
Loading schemas from some providers can be particularly expensive, since
providers for large remote platforms tend to have very large schemas.
Since provider schemas are needed for many operations in Terraform,
callers sometimes end up loading schemas themselves anyway. Earlier work
tried to mitigate this by introducing a global schema cache for all
plugin-based providers, but that's problematic because it forces only a
single implementation of each distinct provider source address across the
entire lifetime of a process importing package providers.
This does not remove that global cache yet, but does add a new capability
that will hopefully eventually supplant it: callers of
terraform.NewContext can provide a set of preloaded provider schemas which
they must ensure would match what Terraform Core would find if it loaded
the schemas from an instance of the same provider instantiated through
the corresponding factory function given alongside.
A caller that wishes to avoid the potential cost of multiple schema
lookups can now therefore go look up the schemas itself before calling
terraform.NewContext, and provide frozen schemas that we'll use instead
of fetching from the associated plugins.
As of this commit no callers are actually using this mechanism. The first
caller will be the "stackeval" package, which already loads provider
schemas in order to evaluate provider configuration blocks anyway and so
should always be able to provide a full complement of preloaded schemas
to avoid Terraform Core needing to do any further lookups itself.
A typo put the check for `GetProviderSchemaOptional` into the wrong `if`
expression, and was missed partly because of the named return which we
also remove here.
Allow core to always use the global schema cache, so that providers
without GetProviderSchemaOptional are not spun up repeatedly. Rather
than conditionally setting the cache, we just conditionally use the
cache in the client to work around providers without
GetProviderSchemaOptional.
