* Update protoc version in downloader script
* go get google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1
This matched terraform-plugin-go
* make protobuf
* Run `make protobuf`
* Update generator to handle generic types from google.golang.org/grpc
Looks like this was added in v1.69.3 in https://github.com/grpc/grpc-go/pull/7057 ?
* Run `make generate`
* Fix "cannot infer Res" compile error - more usage of generics
* More fixing compile errors due to switching to use of a generic
* Make putting `google.golang.org/grpc` import into generated files conditional
* Run `make generate`
* Update more places where generics now need to be used
* Update generator to handle any types from google.golang.org/grpc in same switch case.
* Reformat Args in steps definitions
* Swap to using the new version of protoc-gen-go that doesn't include grpc support (instead it relies on protoc-gen-go-grpc).
Update command flags to reflect this change in tooling.
See this comment for explanation: https://github.com/golang/protobuf/issues/1070#issuecomment-607293496
* Run `make protobuf` with problem generate steps commented out.
* Use `require_unimplemented_servers=false` to minimise changes at the time of upgrading tooling
In future we can navigate the consequences of this in its own PR.
* WIP
* Fix out/opt typo, add remaining flags once terraform1.proto was being found ok
* Run `make protobuf`
* `go get google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.3.0` && `go mod tidy`
This version includes a feature that allows the copyright comment in the .proto file to be pulled across to generated _grpc.pb.go files.
* Run `make protobuf` with new version of protoc-gen-go-grpc that allows copyright comment to be copied to generated output files
* Potential fix for `internal/rpcapi/dynrpcserver/generator`
* Implement ReadStateBytes + WriteStateBytes
* [WIP] - Testing ReadStateBytes and WriteStateBytes (#37464)
* Fix nil pointer error
* Add WIP test for ReadStateBytes
* Move test mock to separate testing file
* Update mock to send unexpected EOF when there's a problem returning data and it's not a true EOF
* Add test case for when length != expected length
* Add test for when trying to read state from a store type that doesn't exist
* Change symbol names to lowercase
* Add ability to force a diagnostic to be returned from `mockReadStateBytesClient`'s `Recv` method
* Add test showing error diagnostics raised by the ReadStateBytes client are returned
* Add missing header
* Simplify mock by using an embedded type
* Rename `mockOpts` to `mockReadStateBytesOpts`
* Update existing tests to assert what arguments are passed to the RPC method call
* Add mock WriteStateBytesClient which uses `go.uber.org/mock/gomock` to enable assertions about calls to Send
* Add a test for WriteStateBytes that makes assertions about calls to the Send method
* Update test case to explicitly test writing data smaller than the chunk size
* Implement chunking in WriteStateBytes, add test case to assert expected chunking behaviour
* Add generated mock for Provider_WriteStateBytesClient in protocol v6
* Update tests to use new `MockProvider_WriteStateBytesClient`, remove handwritten mock
* Update code comments in test
* Add tests for diagnostics and errors returned during WriteStateBytes
* Add generated mock for Provider_ReadStateBytesClient in protocol v6, replace old mock
* Add test case for grpc errors in ReadStateBytes, fix how error is returned
* Typo in comment
* Add missing warning test, rename some test cases
* Update proto file definition of Read/WriteStateBytes RPCs (#37529)
* Update Read/WriteStateBytes RPCs to match https://github.com/hashicorp/terraform-plugin-go/pull/531
* Run `make protobuf`
* Run `make generate`
* Update use of `proto.ReadStateBytes_ResponseChunk` in tests
* Fix how diagnostics are handled alongside EOF error, update ReadStateBytes test
* More fixes - test setup was incorrect
I think? I assume that a response would be returned full of zero-values when EOF is encountered.
* WIP - avoid crash if chunk is nil
* Sarah's updates to radek/pss-read-write (#37642)
* Update code to not expect a chunk when EOF encountered
* Return early if any grpc errors are encountered during ReadStateBytes
* Close the stream with CloseSend once everything's read without error. Add test case about handling grpc errors from CloseSend.
* Fix test case about warnings: We would expect to receive a chunk with data alongside the warning and have a normal closing of the stream after EOF
* Add log line, remove unneeded type info
* Implement configurable state chunk size
* handle metadata in WriteStateBytes correctly
* validate chunk sizes received from provider
* ReadStateBytes: avoid early return on warnings
---------
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
* Add ValidateActionConfig to provider interface and protocol
* Update internal/provider-simple-v6/provider.go
Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>
---------
Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>
* Add GetStates RPC method to Plugin Protocol v6
* Add DeleteState RPC method to Plugin Protocol v6
* Add methods to provider-simple
* Fix error messages
* Apply suggestions from code review
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* Change how GetStates RPC returns the list of state names
* Change GetStates implementation to use new data type
* `make protobuf` to accommodate new code comments
---------
Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>
* Implement ListResource RPC via callbacks
* Implement ListResource with iterators
* Change `ListResourceResponse` to slice
* Turn ListResourceResponse into a struct
* Add a limit for ListResource
* explicitly cancel stream when listing resources
---------
Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>
* Minor auto-formatting changes
* Add list RPC and schema protobuf definitions
* make protobuf
* make generate
* Add ValidateListResourceConfig implementation
* go get github.com/hashicorp/go-azure-helpers@v0.72.0
* .go-version to 1.24.1
* Run `./scripts/syncdeps.sh`
This propagates the Go version upgrade
* gcs : go get google.golang.org/api@v0.155.0
* gcs: go mod tidy
* Run `./scripts/syncdeps.sh`
* make generate && make protobuf
Many of the changes between these two versions seem to be in parts of the
library that Terraform doesn't use, but there are a few sensitive parts:
- Slight changes to details of how the library implements the JSON
encoding of protobuf. Terraform CLI/Core don't use this, but some
of the backends might use it indirectly when talking to their
underlying APIs.
- Some details of the "protoreflect" package for working with
protobuf descriptors dynamically (rather than using generated stubs)
have changed, but it seems pretty marginal and unlikely to cause
problems.
The call site for language functions doesn't currently have a way to
handle complex diagnostics, so rather than appear to support them in the
protocol we remove the concepts of diagnostics for now. We do however
retain the argument index fields, which we can wrap in a
function.ArgError and get a little more precise hcl diagnostic from
expression.
This change ensures that providers receive the source private state and can store potentially differing private state as part of a move, rather than always copying the source private state to the target. This functionality has not yet been released and therefore there are no compatibility concerns.
* Update proto schema and provider interfaces with support for moved across resource type RPCs
* address comments
* remove unused functions
* remove support for flatmap format
Reference: https://github.com/hashicorp/terraform/pull/33486
This is a followup to the new provider server capability to make the `GetProviderSchema` RPC optional. While this server capability would perform its intended function when directly talking to a single provider server SDK implementation, provider servers using terraform-plugin-mux need a methodology for the mux server to determine the available resource types of each underlying provider server to properly route resource-specific RPCs. Since the only methodology available to the mux server would be calling the `GetProviderSchema` RPC to each of underlying provider servers, any memory optimization of core caching would be lost.
The choice of adding a new RPC instead of adjusting the existing `GetProviderSchema` RPC with additional request information, such as "only list the type names and not the schema information in the response", is two-fold:
- Prevents the introduction of conditional logic for the existing RPC.
- Clearly delineates the purpose of the RPC and can be documented easier.
The choice of adding this to the existing provider service is two-fold:
- Implementing a separate protocol and/or service only on the provider side of the protocol would be a novel design change. This small of a change does not warrant the potential research and testing effort that would be associated with that implementation.
- While the core implementation will not use the new RPC immediately, there is no reason why it should be restricted from doing so in the future if a valid use case surfaces. Other ecosystem tools, beyond terraform-plugin-mux, can also potentially benefit from the lightweight RPC now.
This is changing the 5.4 and 6.4 protocol versions following the guidance of this comment in the definition files, since it directly relates to the prior intention of the new minor versions:
```protobuf
// Note that only the proto files included in a release tag of Terraform are
// official protocol releases. Proto files taken from other commits may include
// incomplete changes or features that did not make it into a final release.
// In all reasonable cases, plugin developers should take the proto file from
// the tag of the most recent release of Terraform, and not from the main
// branch or any other development branch.
```
As with any Protocol Buffers definition update, protocol compatibility is guaranteed within a major version, however generated protocol source code compatibility is not guaranteed. In this case, implementing the new RPC method in protocol wrapper types and the moving of the `ServerCapabilities` message to the top namespace are considered acceptable changes.
