- Add TestPythonAPIServerStartup: verify server starts and registrar called
- Add TestPythonAPIServerLifecycle: verify cleanup stops server properly
- Add TestPythonCommandWithAPIServer: verify env var not set when apiImpl is nil
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add apiServerCleanup call to supervisor Shutdown method
- Cleanup happens AFTER Python process terminates for graceful disconnect
- Fix test assertion to check for relative executable path
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add startAPIServer function to start gRPC server on random port
- Add APIServerRegistrar type to break import cycle with apiserver package
- Modify WithCommandFromManifest to accept apiImpl and registrar
- Update configurePythonCommand to start API server and set MATTERMOST_PLUGIN_API_TARGET env var
- Add apiServerCleanup field to supervisor struct for cleanup on shutdown
- Add SetAPIServerRegistrar method to Environment for dependency injection
- Wire up API server registrar in app/plugin.go
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Go side:
- Log hooks returned by Implemented()
- Log each hook name -> ID mapping
- Log OnActivate implementation status
- Log OnActivate call flow
Python side:
- Log Implemented() return value
- Log OnActivate gRPC receipt and handler invocation
This is temporary debug logging to diagnose why OnActivate
isn't being called for Python plugins.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
OnActivate, ServeHTTP, MessageWillBePosted, MessageWillBeUpdated, and
ServeMetrics are handled specially and not in client_rpc_generated.go,
but they still need to be in the hookNameToId map for the Python plugin
Implemented() mechanism to work.
Without this, Python plugins could report implementing OnActivate but
the Go side wouldn't recognize the hook name and wouldn't call it.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When pluginInfo.Path is relative (e.g., "plugins/com.mattermost.hello-python"),
the venv interpreter path was also relative. exec.Command needs an absolute
path to find the executable when cmd.Dir is set.
Fix: Convert the venv path to absolute using filepath.Abs before returning.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When cmd.Dir is set, command arguments are resolved relative to that
directory. The code was passing the full path (including plugin dir)
as the script argument while also setting cmd.Dir to the plugin dir,
causing Python to look for the script at a duplicated path like:
plugins/com.mattermost.hello-python/plugins/com.mattermost.hello-python/plugin.py
Fix: Pass just the executable name (e.g., "plugin.py") to the command
since cmd.Dir is already set to the plugin directory. The full path
is still used for the existence check before command creation.
Also removes debug logging that was added for diagnosis.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add integration tests demonstrating complete Python plugin lifecycle:
- TestPythonPluginIntegration: Tests startup, hooks, ServeHTTP, shutdown
- TestPythonPluginServeHTTP: Tests HTTP streaming with large responses
- TestPythonPluginEnvironmentIntegration: Tests Environment activation/deactivation
- TestPythonPluginCrashRecovery: Tests crash detection and restart
- TestPythonPluginImplementsChecking: Tests hook implementation tracking
All tests use fake Python interpreters (compiled Go binaries) that implement
the PluginHooks gRPC service to validate the Go-side integration without
requiring actual Python.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add TestPythonPluginHookDispatch that verifies end-to-end hook dispatch
for Python plugins. The test:
- Creates a fake Python plugin with PluginHooks gRPC service
- Verifies hooks are properly wired through the supervisor
- Tests Implemented(), OnActivate, OnDeactivate, and MessageHasBeenPosted
Also update existing Phase 5 tests to implement PluginHooks service:
- TestPythonSupervisor_HealthCheckSuccess
- TestPythonPluginEnvironmentActivation
- TestPythonSupervisor_Restart
These tests now reflect the new architecture where Python plugins
have hooks wired via hooksGRPCClient.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove Phase 5 limitation that skipped hook dispensing for Python plugins.
Now Python plugins use hooksGRPCClient to receive hook invocations through
the same infrastructure as Go plugins.
Changes:
- Extract gRPC connection from go-plugin's GRPCClient
- Create hooksGRPCClient adapter using the connection
- Wrap in hooksTimerLayer for metrics collection
- Populate implemented hooks array from gRPC client
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement the hooksGRPCClient adapter that implements the plugin.Hooks
interface by delegating to a gRPC PluginHooksClient. This enables Python
plugins to receive hook invocations through the same infrastructure as
Go plugins.
Key features:
- Task 1: Core adapter structure with constructor and lifecycle hooks
(OnActivate, OnDeactivate, OnConfigurationChange, Implemented)
- Task 2: ServeHTTP with bidirectional streaming (64KB chunks)
- Task 3: All remaining hook methods including message, user, channel,
team, command, WebSocket, and miscellaneous hooks
Technical details:
- Uses context.Background() with 30s timeout for gRPC calls
- Checks implemented array before making gRPC calls
- Converts between model types and protobuf types
- ServeHTTP uses bidirectional streaming for request/response body
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace props.runtime test cases with Server.Runtime field tests:
- python runtime from manifest field (no .py extension)
- go runtime explicit (should not be Python)
- python with full config (PythonVersion, ManifestPython)
- Keep .py extension fallback test for backward compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace transitional props.runtime hack with proper manifest field detection.
Now prioritizes Server.Runtime="python" over .py extension fallback.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add end-to-end test for restart behavior:
1. Activate healthy fake Python plugin
2. Verify health check succeeds
3. Kill process via supervisor.client.Kill() to simulate crash
4. Verify health check fails after crash
5. Call RestartPlugin and verify health check succeeds again
Uses polling loops instead of arbitrary sleeps for reliability.
Proves Environment.RestartPlugin can recover crashed Python plugins.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add tests for:
- Handshake timeout: fake interpreter blocks forever without printing
handshake, verifies supervisor times out within StartTimeout
- Invalid handshake: fake interpreter prints netrpc protocol instead of
grpc, verifies protocol mismatch error
- Malformed handshake: fake interpreter prints incomplete handshake line,
verifies parsing error
These tests use compiled Go binaries as fake "Python interpreters" to
validate supervisor error handling without requiring real Python.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- TestPythonSupervisor_HealthCheckSuccess: validates end-to-end spawning
of Python-style plugin with gRPC health check using fake interpreter
- TestPythonPluginEnvironmentActivation: validates Environment can
activate/deactivate Python plugins without panicking on nil hooks
- Update Environment.Activate to use WithCommandFromManifest which
handles both Go (netrpc) and Python (gRPC) plugins
The fake interpreter is a compiled Go binary that:
1. Starts a gRPC server on random port
2. Registers health service with "plugin" status SERVING
3. Prints go-plugin handshake line
4. Blocks until killed
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- startPluginServer: skip OnActivate when Hooks() is nil
- Deactivate: guard OnDeactivate call against nil hooks
- Shutdown: guard OnDeactivate call against nil hooks
- Add informative log when Python plugin activates without hooks
This allows Phase 5 process supervision to work while hook dispatch
is deferred to Phase 7.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Detect Python plugins in newSupervisor via isPythonPlugin helper
- Configure AllowedProtocols to include ProtocolGRPC for Python plugins
- Increase StartTimeout to 10s for Python interpreter startup
- Skip Dispense("hooks") for Python plugins (Phase 5 supervision only)
- Leave sup.hooks nil - hook dispatch deferred to Phase 7
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add helper functions for Python plugin support in the supervisor:
- isPythonPlugin: detects Python plugins via .py extension or props.runtime
- findPythonInterpreter: discovers venv-first Python with PATH fallback
- sanitizePythonScriptPath: validates script paths preventing traversal
- buildPythonCommand: creates exec.Cmd with proper working dir and WaitDelay
- WithCommandFromManifest: unified option for Go and Python plugin commands
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix regression in PluginHTTPStream where request body closed prematurely
When WriteHeader was called before reading the request body in inter-plugin
communication, the body would be closed prematurely due to defer r.Body.Close()
executing when the function returned (after starting the response goroutine).
This fix moves defer r.Body.Close() into the goroutine to ensure the request
body remains available until after the response is fully processed.
Added test case TestInterpluginPluginHTTPWithBodyAfterWriteHeader to verify
the fix and prevent future regressions.
* Fix resource leak by closing request body in all PluginHTTPStream error paths
---------
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Implement property field limit enforcement and counting functionality in Plugin API
- Added a limit of 20 property fields per group in the CreatePropertyField method.
- Introduced CountPropertyFields method to count active and all property fields, including deleted ones.
- Enhanced tests to validate the new property field limit and counting behavior.
- Updated related API and service methods to support the new functionality.
* Update server/channels/app/properties/property_field.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix vet
* fix lint error
* fix test
* fix tests
* fix test
* count properties + targets
* Update server/channels/app/plugin_api.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* remove test for limit
* fix more tests
* improve testing messages now that the limit is removed
* Apply suggestion from @calebroseland
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
* Apply suggestion from @calebroseland
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
* Apply suggestion from @calebroseland
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
* Apply suggestion from @calebroseland
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com>
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
This commit exposes audit logging functionality to plugins via the plugin API, allowing plugins to create and log audit records. Additionally, it addresses a gob encoding issue that could cause plugin crashes when audit data contains nil pointers or unregistered types.
* Always require signatures for prepackaged plugins
We have always required signatures for packages installed via the marketplace -- whether remotely satisfied, or sourced from the prepackaged plugin cache.
However, prepackaged plugins discovered and automatically installed on
startup did not require a valid signature. Since we already ship
signatures for all Mattermost-authored prepackaged plugins, it's easy to
simply start requiring this.
Distributions of Mattermost that bundle their own prepackaged plugins
will have to include their own signatures. This in turn requires
distributing and configuring Mattermost with a custom public key via
`PluginSettings.SignaturePublicKeyFiles`.
Note that this enhanced security is neutered with a deployment that uses
a file-based `config.json`, as any exploit that allows appending to the
prepackaged plugins cache probably also allows modifying `config.json`
to register a new public key. A [database-based
config](https://docs.mattermost.com/configure/configuration-in-your-database.html)
is recommended.
Finally, we already support an optional setting
`PluginSettings.RequirePluginSignature` to always require a plugin
signature, although this effectively disables plugin uploads and
requires extra effort to deploy the corresponding signature. In
environments where only prepackaged plugins are used, this setting is
ideal.
Fixes: https://mattermost.atlassian.net/browse/MM-64627
* setup dev key, expect no plugins if sig fails
* Fix shadow variable errors in test helpers
Pre-declare signaturePublicKey variable in loops to avoid shadowing
the outer err variable used in error handling.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Replace PrepackagedPlugin.Signature with SignaturePath for memory efficiency
- Changed PrepackagedPlugin struct to use SignaturePath string instead of Signature []byte
- Updated buildPrepackagedPlugin to use file descriptor instead of reading signature into memory
- Modified plugin installation and persistence to read from signature file paths
- Updated all tests to check SignaturePath instead of Signature field
- Removed unused bytes import from plugin.go
This change reduces memory usage by storing file paths instead of signature data
in memory while maintaining the same security verification functionality.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Upgrade Go to 1.24.3
Updates the following files:
- server/.go-version: 1.23.9 → 1.24.3
- server/build/Dockerfile.buildenv: golang:1.23.9-bullseye → golang:1.24.3-bullseye
- server/go.mod: go 1.23.0 → go 1.24.3, toolchain go1.23.9 → go1.24.3
- server/public/go.mod: go 1.23.0 → go 1.24.3, toolchain go1.23.9 → go1.24.3
Also fixes non-constant format string errors introduced by Go 1.24.3's stricter format string checking:
- Added response() helper function in slashcommands/util.go for simple string responses
- Removed unused responsef() function from slashcommands/util.go
- Replaced responsef() with response() for translated strings that don't need formatting
- Fixed fmt.Errorf and fmt.Fprintf calls to use proper format verbs instead of string concatenation
- Updated marketplace buildURL to handle format strings conditionally
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update generated mocks for Go 1.24.3
Regenerated mocks using mockery v2.53.4 to ensure compatibility with Go 1.24.3.
This addresses mock generation failures that occurred with the Go upgrade.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update to bookworm and fix non-existent sha
Signed-off-by: Stavros Foteinopoulos <stafot@gmail.com>
* fix non-constant format string
---------
Signed-off-by: Stavros Foteinopoulos <stafot@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Stavros Foteinopoulos <stafot@gmail.com>
There are several steps that a server runs through inside (*Server).Start
after ch.initPlugins() till the signal handler is reached which handles
the server shutdown procedure.
The issue arises when the server is shutdown after ch.initPlugins() completes
but before (*Server).Start finishes. In that case, the plugins are all started
but they won't be shut down cleanly.
To fix this edge-case, we set up an intermediate signal handler, which
attaches itself as soon as ch.initPlugins is finished, allowing us to run
the cleanup code in case the shutdown happens before (*Server).Start finishes.
And when we do reach the main signal handler, we don't need this intermediate
handler any more. So we just reset the handlers and use the main signal handler
which takes care of shutting down the whole server.
Note: This is still not 100% bug-proof because ch.initPlugins() will initialize
_all_ plugins, and the shutdown can happen just after one plugin is initialized.
To handle that case will require the need to set up signal handlers after every
plugin init which feels like overkill to me.
A sample flow diagram to visualize better:
Edge-case
server.Start()
|
ch.initPlugins()
|
<ctrl-c>
|
execute signal handler, os.Exit(1)
Happy-path
server.Start()
|
ch.initPlugins()
|
server.Start() finished
|
reset old signal handler
|
setup main signal handler
|
server runs on as usual until shutdown
https://mattermost.atlassian.net/browse/MM-49353
```release-note
NONE
```
Previously, we relied on the plugin to close the DB connections
on shutdown. While this keeps the code simple, there is no guarantee
that the plugin author will remember to close the DB.
In that case, it's better to track the connections from the server side
and close them in case they weren't closed already. This complicates
the API slightly, but it's a price we need to pay.
https://mattermost.atlassian.net/browse/MM-56402
```release-note
We close any remaining unclosed DB RPC connections
after a plugin shuts down.
```
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Revert "Revert "MM-57759: Bump mockery to version 2.42.2 to support go 1.22^" (#26772)"
This reverts commit cd3b5b46e1.
* Added the hooks.go file changes as well
```release-note
NONE
```
