- Add startAPIServer function to start gRPC server on random port
- Add APIServerRegistrar type to break import cycle with apiserver package
- Modify WithCommandFromManifest to accept apiImpl and registrar
- Update configurePythonCommand to start API server and set MATTERMOST_PLUGIN_API_TARGET env var
- Add apiServerCleanup field to supervisor struct for cleanup on shutdown
- Add SetAPIServerRegistrar method to Environment for dependency injection
- Wire up API server registrar in app/plugin.go
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- TestPythonSupervisor_HealthCheckSuccess: validates end-to-end spawning
of Python-style plugin with gRPC health check using fake interpreter
- TestPythonPluginEnvironmentActivation: validates Environment can
activate/deactivate Python plugins without panicking on nil hooks
- Update Environment.Activate to use WithCommandFromManifest which
handles both Go (netrpc) and Python (gRPC) plugins
The fake interpreter is a compiled Go binary that:
1. Starts a gRPC server on random port
2. Registers health service with "plugin" status SERVING
3. Prints go-plugin handshake line
4. Blocks until killed
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- startPluginServer: skip OnActivate when Hooks() is nil
- Deactivate: guard OnDeactivate call against nil hooks
- Shutdown: guard OnDeactivate call against nil hooks
- Add informative log when Python plugin activates without hooks
This allows Phase 5 process supervision to work while hook dispatch
is deferred to Phase 7.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Always require signatures for prepackaged plugins
We have always required signatures for packages installed via the marketplace -- whether remotely satisfied, or sourced from the prepackaged plugin cache.
However, prepackaged plugins discovered and automatically installed on
startup did not require a valid signature. Since we already ship
signatures for all Mattermost-authored prepackaged plugins, it's easy to
simply start requiring this.
Distributions of Mattermost that bundle their own prepackaged plugins
will have to include their own signatures. This in turn requires
distributing and configuring Mattermost with a custom public key via
`PluginSettings.SignaturePublicKeyFiles`.
Note that this enhanced security is neutered with a deployment that uses
a file-based `config.json`, as any exploit that allows appending to the
prepackaged plugins cache probably also allows modifying `config.json`
to register a new public key. A [database-based
config](https://docs.mattermost.com/configure/configuration-in-your-database.html)
is recommended.
Finally, we already support an optional setting
`PluginSettings.RequirePluginSignature` to always require a plugin
signature, although this effectively disables plugin uploads and
requires extra effort to deploy the corresponding signature. In
environments where only prepackaged plugins are used, this setting is
ideal.
Fixes: https://mattermost.atlassian.net/browse/MM-64627
* setup dev key, expect no plugins if sig fails
* Fix shadow variable errors in test helpers
Pre-declare signaturePublicKey variable in loops to avoid shadowing
the outer err variable used in error handling.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Replace PrepackagedPlugin.Signature with SignaturePath for memory efficiency
- Changed PrepackagedPlugin struct to use SignaturePath string instead of Signature []byte
- Updated buildPrepackagedPlugin to use file descriptor instead of reading signature into memory
- Modified plugin installation and persistence to read from signature file paths
- Updated all tests to check SignaturePath instead of Signature field
- Removed unused bytes import from plugin.go
This change reduces memory usage by storing file paths instead of signature data
in memory while maintaining the same security verification functionality.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
There are several steps that a server runs through inside (*Server).Start
after ch.initPlugins() till the signal handler is reached which handles
the server shutdown procedure.
The issue arises when the server is shutdown after ch.initPlugins() completes
but before (*Server).Start finishes. In that case, the plugins are all started
but they won't be shut down cleanly.
To fix this edge-case, we set up an intermediate signal handler, which
attaches itself as soon as ch.initPlugins is finished, allowing us to run
the cleanup code in case the shutdown happens before (*Server).Start finishes.
And when we do reach the main signal handler, we don't need this intermediate
handler any more. So we just reset the handlers and use the main signal handler
which takes care of shutting down the whole server.
Note: This is still not 100% bug-proof because ch.initPlugins() will initialize
_all_ plugins, and the shutdown can happen just after one plugin is initialized.
To handle that case will require the need to set up signal handlers after every
plugin init which feels like overkill to me.
A sample flow diagram to visualize better:
Edge-case
server.Start()
|
ch.initPlugins()
|
<ctrl-c>
|
execute signal handler, os.Exit(1)
Happy-path
server.Start()
|
ch.initPlugins()
|
server.Start() finished
|
reset old signal handler
|
setup main signal handler
|
server runs on as usual until shutdown
https://mattermost.atlassian.net/browse/MM-49353
```release-note
NONE
```
Previously, we relied on the plugin to close the DB connections
on shutdown. While this keeps the code simple, there is no guarantee
that the plugin author will remember to close the DB.
In that case, it's better to track the connections from the server side
and close them in case they weren't closed already. This complicates
the API slightly, but it's a price we need to pay.
https://mattermost.atlassian.net/browse/MM-56402
```release-note
We close any remaining unclosed DB RPC connections
after a plugin shuts down.
```
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* ProfileImageBytes for EnsureBotOptions
* leverage plugintest.NewAPI
* fix linting
* add UpdateUserRoles to plugin api
* MM-57018: support reattaching plugins
Expose a local-only API for reattaching plugins: instead of the server starting and managing the process itself, allow the plugin to be launched externally (eg within a unit test) and reattach to an existing server instance to provide the unit test with a fully functional RPC API, sidestepping the need for mocking the plugin API in most cases.
In the future, this may become the basis for running plugins in a sidecar container.
Fixes: https://mattermost.atlassian.net/browse/MM-57018
* drop unused supervisor.pid
* factor out checkMinServerVersion
* factor out startPluginServer
* restore missing setPluginState on successful reattach
* avoid passing around a stale registeredPlugin
* inline initializePluginImplementation
* have IsValid return an error
* explicitly close rpcClient
In the case of reattached plugins, the Unix socket won't necessarily disappear leaving the muxBrokers blocked indefinitely. And `Kill()` doesn't do anything if there's no process being managed.
* explicitly detachPlugin
* emphasize gRPC not being supported
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* move plugin signature verification to caller
The semantics for when plugin signature validation is required are unique to the caller, so move this logic there instead of masking it, thus simplifying some of the downstream code.
* support transitionally prepacked plugins
Transitionally prepackaged plugins are prepackaged plugins slated for unpackaging in some future release. Like prepackaged plugins, they automatically install or upgrade if the server is configured to enable that plugin, but unlike prepackaged plugins they don't add to the marketplace to allow for offline installs. In fact, if unlisted from the marketplace and not already enabled via `config.json`, a transitionally prepackaged plugin is essentially hidden.
To ensure a smooth transition in the future release when this plugin is no longer prepackaged at all, transitionally prepackaged plugins are persisted to the filestore as if they had been installed by the enduser. On the next restart, even while the plugin is still transitionally prepackaged, the version in the filestore will take priority. It remains possible for a transitionally prepackaged plugin to upgrade (and once again persist) if we ship a newer version before dropping it altogether.
Some complexity arises in a multi-server cluster, primarily because we don't want to deal with multiple servers writing the same object to the filestore. This is probably fine for S3, but has undefined semantics for regular filesystems, especially with some customers backing their files on any number of different fileshare technologies. To simplify the complexity, only the cluster leader persists transitionally prepackaged plugins.
Unfortunately, this too is complicated, since on upgrade to the first version with the transitionally prepackaged plugin, there is no guarantee that server will be the leader. In fact, as all nodes restart, there is no guarantee that any newly started server will start as the leader. So the persistence has to happen in a job-like fashion. The migration system might work, except we want the ability to run this repeatedly as we add to (or update) these transitionally prepackaged plugins. We also want to minimize the overhead required from the server to juggle any of this.
As a consequence, the persistence of transitionally prepackaged plugins occurs on every cluster leader change. Each server will try at most once to persist its collection of transitionally prepackaged plugins, and newly started servers will see the plugins in the filestore and skip this step altogether.
The current set of transitionally prepackaged plugins include the following, but this is expected to change:
* focalboard
* complete list of transitionally prepackaged plugins
* update plugin_install.go docs
* updated test plugins
* unit test transitionally prepackged plugins
* try restoring original working directory
* Apply suggestions from code review
Co-authored-by: Michael Kochell <6913320+mickmister@users.noreply.github.com>
* clarify processPrepackagedPlugins comment
---------
Co-authored-by: Michael Kochell <6913320+mickmister@users.noreply.github.com>
* remove feature flag managed plugins
* remove unneeded plugin blocklist
* remove unnecessary wrappers
* documentation and logging improvements
* avoid use of global logger
* leverage wrapped loggers (e.g. consistently log `plugin_id`)
* promote some logs from `Debug` to `Info` for better visibility.
* extract installPluginToFilestore
* rename some variables for consistency / clarity
* make generated
* Remove build references
* Remove playbooks webapp and server, and add the prepackaged plugin
* Remove translations
* Add ProductSettings to the playwright type
* Restore playbooks as a prepackaged plugin for cypress e2e tests
* Prevent boards product from being included automatically
* Fix config diff test
* Update prepackaged plugin version
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Readd boards/dist to the gitignore
* Does not enable the focalboard plugin by default
* Update plugin version to v7.10.3
---------
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
It was a good decision in hindsight to keep the public module as 0.x
because this would have been a breaking change again.
https://mattermost.atlassian.net/browse/MM-53032
```release-note
Changed the Go module path from github.com/mattermost/mattermost-server/server/v8 to github.com/mattermost/mattermost/server/v8.
For the public facing module, it's path is also changed from github.com/mattermost/mattermost-server/server/public to github.com/mattermost/mattermost/server/public
```
