Add AI development guidance for Mattermost contributors working on the
Python plugin gRPC infrastructure. Includes directory structure, key
components, proto organization, common patterns, and instructions for
adding new API methods and hooks.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add ARCHITECTURE.md documenting the Python plugin gRPC system including:
- High-level overview and ASCII system architecture diagram
- Component layer documentation (Protocol, Go Infrastructure, Python SDK)
- Process lifecycle (loading, environment variables, shutdown)
- Communication flow diagrams (hooks, API calls, ServeHTTP streaming)
- Key design decisions (embedded AppError, 64KB chunks, APIServerRegistrar)
- Complete file reference table mapping functionality to source files
- Extension guide for adding new API methods and hooks
This documentation enables internal Mattermost engineers to understand,
debug, and extend the Python plugin infrastructure.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Create shell scripts for consistent test invocation across CI environments:
Python script (python-sdk/scripts/run_integration_tests.sh):
- Auto-detects virtual environment or uses system Python
- Installs package in editable mode if not present
- Runs test_integration_e2e.py with verbose output
- Supports passing extra pytest arguments
Go script (server/public/pluginapi/grpc/scripts/run_integration_tests.sh):
- Runs from the correct module directory
- Filters to TestPythonPlugin and TestIntegration tests
- Disables test caching for clean CI runs
- Supports passing extra go test arguments
Both scripts:
- Are executable (chmod +x)
- Print diagnostic information (working dir, version)
- Exit with appropriate error codes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Create integration tests that prove Go gRPC server and Python plugin
communicate correctly via simulated in-memory gRPC communication:
- TestPythonPluginLifecycle: Full lifecycle (activate, deactivate)
- TestPythonPluginAPICall: Round-trip plugin -> API server callback
- TestPythonPluginHookChain: MessageWillBePosted modification/rejection
- TestPythonPluginActivationFailure: Error propagation from plugin
- TestPythonPluginAPIErrorPropagation: AppError -> gRPC status mapping
- TestIntegrationConcurrentHookCalls: Concurrent hook invocations
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update ServeHTTPCaller to support true streaming responses:
- Full-duplex operation: plugin can respond before request body is consumed
- Flush support: best-effort flushing when underlying writer supports http.Flusher
- Status code validation (100-999 range) to prevent server panics
- Early response handling with proper goroutine cleanup
- Proper error responses for invalid status codes and plugin failures
Key changes:
- Add logger to ServeHTTPCaller for error logging
- Add context cancellation to sendRequest for early response termination
- Add receiveFirstResponse to handle first message with body chunk + flush
- Add writeResponseHeaders with status code validation
- Add flush() method for best-effort HTTP flushing
- Update streamResponseBody to handle flush flag
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add flush field to ServeHTTPResponse for best-effort HTTP flush support.
Update ServeHTTPResponse and ServeHTTPResponseInit with comprehensive
documentation of streaming invariants, message ordering, and status
code validation rules.
Changes:
- Add flush boolean field (field 4) to ServeHTTPResponse
- Document response streaming invariants (init-once, header locking)
- Document status code validation (100-999 range, 0 defaults to 200)
- Regenerate Go and Python protobuf code
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add tests for both Go and Python ServeHTTP implementations covering:
- Body chunking behavior (small, exact, multiple, large bodies)
- Context cancellation during body streaming
- Header conversion utilities
- HTTPRequest/HTTPResponseWriter helper classes
- Request body assembly from multiple chunks
- Error handling (500 for handler errors, 404 for missing handler)
Go tests:
- TestChunking_* for body chunking scenarios
- TestConvertHTTPHeaders for header conversion
- TestBuildRequestInit for request metadata building
- TestWriteResponseHeaders for response header writing
- TestContextCancellation_DuringBodyRead for cancellation
Python tests:
- TestHTTPRequest for request wrapper class
- TestHTTPResponseWriter for response writer class
- TestHeaderConversion for proto<->dict conversion
- TestServeHTTPServicer for gRPC servicer behavior
- TestChunkingBehavior for body assembly
- TestCancellation for request cancellation
Also:
- Fixed sendRequest to properly set body_complete flag on EOF
- Added ServeHTTP to HookName enum
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add ServeHTTPCaller that streams HTTP requests to Python plugins
and receives streaming responses.
Key features:
- Bidirectional streaming for efficient large body transfer
- 64KB chunked request body streaming (no full buffering)
- Context cancellation propagation from HTTP client disconnect
- HTTPHeader conversion supporting multi-value headers
- PluginContext passed through to Python handler
Implementation details:
- sendRequest: streams body chunks with completion flag
- receiveResponseInit: gets status code and headers
- streamResponseBody: writes response chunks to ResponseWriter
- buildRequestInit: converts http.Request to protobuf init
Note: Response streaming improvements deferred to 08-02.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add bidirectional streaming RPC for ServeHTTP hook to support
efficient HTTP request/response transfer between Go and Python.
Key changes:
- New hooks_http.proto with ServeHTTPRequest/Response messages
- HTTPHeader message for multi-value header support
- ServeHTTPRequestInit with full request metadata
- ServeHTTPResponseInit for status and headers
- Body chunks with completion flag for streaming
- Updated hooks.proto with ServeHTTP streaming RPC
- Regenerated Go and Python code
Design decisions:
- 64KB default chunk size per gRPC best practices
- First message carries metadata, subsequent messages carry body
- body_complete flag signals end of stream
- Headers as repeated HTTPHeader for multi-value support
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement all File-related gRPC RPC handlers:
- GetFileInfo, GetFileInfos, GetFile
- GetFileLink, ReadFile, UploadFile
- CopyFileInfos, SetFileSearchableContent
Add comprehensive unit tests for:
- GetFileInfo success and not found cases
- GetFile success and error cases
- ReadFile success and not found cases
- UploadFile success and permission error
- CopyFileInfos and SetFileSearchableContent
- GetFileInfos with options
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add conversion functions between model.Team/TeamMember/TeamUnread/TeamStats
and their protobuf equivalents. Includes:
- teamToProto/teamFromProto for Team conversion
- teamMemberToProto/teamMemberFromProto for TeamMember conversion
- teamUnreadToProto for TeamUnread conversion
- teamStatsToProto for TeamStats conversion
- teamMembersWithErrorToProto for graceful member creation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add pb <-> model conversion functions for Post, File, and KV Store types:
convert_post.go:
- Post, PostMetadata, PostEmbed, Reaction conversions
- PostList and PostSearchResults conversions
- SearchParams and SearchParameter conversions
convert_file.go:
- FileInfo conversions
- GetFileInfosOptions conversion
convert_kv.go:
- PluginKVSetOptions conversion
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Enhance package documentation to explain how subsequent plans (04-02
through 04-04) should extend the APIServer with new RPC implementations.
Includes code example and key patterns for error handling and model
conversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add in-memory gRPC testing infrastructure using bufconn. Includes:
- testHarness struct for easy test setup/teardown
- Smoke tests for GetServerVersion and IsEnterpriseReady RPCs
- Comprehensive error conversion tests for all HTTP->gRPC mappings
- Tests use plugintest.API mock from the existing codebase
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Create the APIServer struct that wraps plugin.API and embeds
UnimplementedPluginAPIServer for incremental implementation.
Includes Register helper and smoke RPCs (GetServerVersion,
IsEnterpriseReady) to validate the wiring.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add hooks_command.proto to Makefile proto mappings
- Update hooks_command.proto to import api_remaining.proto for shared types
(CommandArgs, CommandResponse, PluginClusterEvent)
- Define new types: WebSocketRequest, SyncMsgJson, SyncResponse, RemoteCluster
- Generated Go code compiles successfully
- Total hooks in PluginHooks service: 41 (excluding deferred ServeHTTP/ServeMetrics)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add hooks_user_channel.proto mapping to Makefile proto generation
- Regenerate Go protobuf code with new user/channel hook types
- Generated hooks_user_channel.pb.go with all message types
- Updated hooks.pb.go and hooks_grpc.pb.go with new RPCs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add hooks_message.proto mapping to Makefile for proto generation
- Update hooks_message.proto to import api_remaining.proto for
PushNotification and Preference types (avoiding redefinition)
- Generate Go code for all message hook messages and RPCs
Verified:
- make proto-gen succeeds
- go build ./public/pluginapi/grpc/generated/go/pluginapiv1/... succeeds
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add 12 message-related RPC definitions to the PluginHooks service:
- MessageWillBePosted - intercept/modify posts before save
- MessageWillBeUpdated - intercept/modify post updates before save
- MessageHasBeenPosted - notification after post created
- MessageHasBeenUpdated - notification after post updated
- MessagesWillBeConsumed - filter posts before client delivery
- MessageHasBeenDeleted - notification after post deleted
- FileWillBeUploaded - intercept/modify file uploads
- ReactionHasBeenAdded - notification after reaction added
- ReactionHasBeenRemoved - notification after reaction removed
- NotificationWillBePushed - intercept/modify push notifications
- EmailNotificationWillBeSent - intercept/modify email notifications
- PreferencesHaveChanged - notification after preferences changed
Each RPC includes documentation explaining:
- When the hook is called
- How to use return values (modify, reject, allow)
- The original Go signature
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create api.proto with PluginAPI service definition covering all 236
methods from server/public/plugin/api.go
- Create api_user_team.proto with User, Session, and Team request/response
messages with placeholder fields
- Create api_channel_post.proto with Channel, Post, and Emoji messages
- Create api_kv_config.proto with KV store, config, plugin, and logging
messages
- Create api_file_bot.proto with File, Upload, and Bot messages
- Create api_remaining.proto with Server, Command, Preference, OAuth,
Group, SharedChannel, Property, and Audit messages
- Add ViewUsersRestrictions to common.proto
- Add apiverify tool that parses Go API interface and proto service
to ensure parity
- Update Makefile with new proto file mappings
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Define protobuf messages and PluginHooks gRPC service for lifecycle and
system hooks:
- Implemented: returns list of hooks the plugin implements
- OnActivate/OnDeactivate: plugin lifecycle events
- OnConfigurationChange: configuration change notifications
- OnInstall: plugin installation event
- OnSendDailyTelemetry: daily telemetry hook
- RunDataRetention: data retention batch processing
- OnCloudLimitsUpdated: cloud product limit changes
- ConfigurationWillBeSaved: configuration validation/modification
Also adds model types:
- OnInstallEvent: mirrors model.OnInstallEvent
- ProductLimits/FilesLimits/MessagesLimits/TeamsLimits: typed cloud limits
- ConfigJson: JSON blob wrapper for model.Config (too large for typed proto)
Updates Makefile to include new proto file mappings.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add hooks_common.proto with PluginContext message that mirrors the
server/public/plugin/context.go Context struct. This provides session_id,
request_id, ip_address, accept_language, and user_agent fields for passing
hook invocation context from server to plugins.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RequestContext message for request metadata (plugin_id, request_id,
session_id, user_id) to enable logging correlation across Go/Python
- Add params field (google.protobuf.Struct) to AppError for error message
interpolation, matching model.AppError's params field
- Document RPC envelope conventions in proto comments:
- Request messages: {Method}Request with RequestContext as field 1
- Response messages: {Method}Response with AppError as field 1
- Error strategy: Response-embedded AppError (Option B) for full
semantic preservation, gRPC status reserved for transport errors
- Regenerate Go code with new types
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add common.proto with Empty, StringMap, and AppError messages
- Add user.proto with User message (34 fields mirroring model.User)
- Add channel.proto with Channel, ChannelType enum, ChannelBannerInfo
- Add post.proto with Post, PostMetadata, PostEmbed, PostPriority, Reaction
- Add team.proto with Team, TeamMember, TeamUnread, TeamType enum
- Add file.proto with FileInfo, FileUploadResponse, FileData
- Update Makefile to support all proto files with proper import mappings
- Remove bootstrap.proto (replaced by common.proto)
- Use google.protobuf.Struct for dynamic JSON fields (Post.props, Channel.props)
- All timestamps are int64 (milliseconds since epoch)
- All IDs are strings (26-char base32)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create server/public/pluginapi/grpc/proto/ for proto sources
- Create server/public/pluginapi/grpc/generated/go/ for generated Go code
- Add bootstrap.proto with Empty message for validating codegen pipeline
- Generate bootstrap.pb.go using protoc v6.33.4 / protoc-gen-go v1.36.6
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Implement property field limit enforcement and counting functionality in Plugin API
- Added a limit of 20 property fields per group in the CreatePropertyField method.
- Introduced CountPropertyFields method to count active and all property fields, including deleted ones.
- Enhanced tests to validate the new property field limit and counting behavior.
- Updated related API and service methods to support the new functionality.
* Update server/channels/app/properties/property_field.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix vet
* fix lint error
* fix test
* fix tests
* fix test
* count properties + targets
* Update server/channels/app/plugin_api.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* remove test for limit
* fix more tests
* improve testing messages now that the limit is removed
* Apply suggestion from @calebroseland
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
* Apply suggestion from @calebroseland
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
* Apply suggestion from @calebroseland
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
* Apply suggestion from @calebroseland
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com>
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
* MM-64486: Remove telemetry
Remove telemetry from Mattermost. We're no longer relying on Rudder upstream, and no longer making use of this information.
* recover mock for SystemStore.Get
* Fix TestClearPushNotificationSync by adding missing SystemStore mock
The test was failing because the SystemStore mock was missing the Get()
method that's required by the ServerId() function. Added the missing mock
to return a StringMap with SystemServerId.
* fix mocking issue
* Remove now-unused telemetry and constants
* Remove "Disable telemetry events" debug setting
* Remove empty functions
* Remove most "Telemetry tracking removed" comments
* Remove remains of DataPrefetch telemetry
* Remove now-unused prop from InviteMembersButton
* Remove trackDotMenuEvent
* Remove some more leftover comments
* Remove lingering logic related to trackingLocation
* Remove now-unused argument from useCopyText
* Remove lingering telemetry references from PreparingWorkspace
* fixup Remove trackDotMenuEvent
* Remove lingering telemetry references from signup page and password check
* Update snapshots and fix test broken by my changes
* Fix unintended behavior change in thread list filtering
Remove handleSetFilter wrapper that was accidentally modified during
telemetry removal. The function was calling clear() when switching to
unread filter, which was not the original behavior. Use setFilter
directly instead, restoring the original functionality.
* Remove unused useOpenDowngradeModal hook
The useOpenDowngradeModal hook was not being used anywhere in the codebase.
* Remove unused expandableLink from useExpandOverageUsersCheck
The expandableLink return value was not being used by any components.
* Re-add missing TeamLinkClicked performance telemetry
The mark(Mark.TeamLinkClicked) call was accidentally removed from the
handleSwitch function. This telemetry is needed for Looker-based
performance tracking.
* drop LogSettings.VerboseDiagnostics
---------
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
