Commit graph

47 commits

Author SHA1 Message Date
Agniva De Sarker
2835163b18
Fix flaky TestPlugin (#13933)
- Fix panic due to not checking for JSON marshal error
- Reorder the FileExists check which would fail if we were to pass
the -short flag to bypass the slow URL test. In which case, the plugin
would have been removed from the filesystem.
So we move the check before we remove the plugin.

https://mattermost.atlassian.net/browse/MM-22666
2020-02-26 15:11:24 +01:00
Jesús Espino
1d1ab03c38
Passing t to all tests setup functions (#13841)
* Passing t to all tests setup functions

* Fixing build
2020-02-10 19:31:41 +01:00
Ben Schumacher
cb49d560ab
Allow plugins to specify a ReleaseNotesURL via manifest (#13677) 2020-01-28 19:33:21 +01:00
Shota Gvinepadze
1e9556cc28
[MM-21572] Disable InstallFromUrl if EnableUploads is false (#13636)
* Disable InstallFromUrl if EnableUploads is false

* Break into 3 seperate lines

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-01-27 17:12:16 +04:00
Maria A Nunez
ebfd332161
MM-21374 - Fixed extracting icon for prepackaged and local plugins (#13633)
* Merge upstream

* Added extracting iconData for prepackaged and local plugins

* Fixed tests

* Removed report.xml

Co-authored-by: Jason Paul Deland <9366595+jaydeland@users.noreply.github.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-01-20 10:34:18 -05:00
Ben Schumacher
3491c3abb2
Respect HomepageURL in manifest for local plugins (#13595)
* Respect HomepageURL in manifest for local plugins

* Add documentation for updating the plugin signatures
2020-01-17 09:09:58 +01:00
Maria A Nunez
87eb7697f9
MM-19606- Rework Prepackaged Plugins (#13449)
* MM-19609 - Add new prepackage configuration settings (#13062)

* Add signatures to the prepackaged plugins (#13138)

* MM-19612 - Support querying local plugin marketplace when upst… (#13250)

* MM-19612 - Support querying local plugin marketplace when upstream unavailable or disabled

* Update translations file

* Fixed comment

* Updated to check EnableRemoteMarketplace setting and LocalOnly to get marketplace plugins

* Fixed unit tests

* Tests cleanup code

* Removed unused error message

* Updated tests

* MM-19614- Updated Marketplace Service error id (#13388)

* [MM-19610] Consume prepackaged plugins (#13005)

* consume prepackaged plugins into memory

* missing i18n

* remove spurious .gitignore changes

* return on failure to install prepackged plugins

* cleanup

* s/plugins/availablePlugins

* whitespace

* don't return extractDir when not needed

* s/plug/plugin

* error on icon, cleanup

* update armored version of testplugin signature

* honour AutomaticPrepackagedPlugins

* document getPrepackagedPlugin

* MM-19613 - Include prepackaged plugins in marketplace results (#13433)

* Added prepackaged plugins to marketplace results

* PR Feedback

* PR Feedback

* Update error where definition

* Removing unnecessary var declaration

* Updated comments

* MM-21263 - Use EnableRemoteMarketplace in marketplace install… (#13438)

* MM-21263 - Use EnableRemoteMarketplace in marketplace install endpoint

* Call updateConfig before calling NewServer in TestHelper

* Added translations

* PR feedback

* Translations

* Feedback

* s/helpers.go/download.go

* Converging env.PrepackagedPlugins

* Initial PR feedback

* Ordered imports properly

* Updated DownloadURL to return slice of bytes

* Fixed method typo

* Fixed logging

* Added read lock for prepackaged plugins list

* PR Feedback

* Added condition to only install prepackaged plugin if it was previously enabled

* Linting

* Updated to check plugin state in config

* Closing filereader

* Only add local label if remote marketplace is enabled

* Updated local tag description

* Fixed tests

Co-authored-by: Ali Farooq <ali.farooq0@pm.me>
Co-authored-by: Shota Gvinepadze <wineson@gmail.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2020-01-15 13:38:55 -05:00
Ben Schumacher
1f3f8fbf57
[MM-21123] Add Labels to upstream Marketplace response (#13425)
* Add Labels to upstream Marketplace response

* Add LOCAL label to local plugins
2020-01-14 07:16:36 +01:00
Claudio Costa
80dd2915db
[MM-19914] Fix data races in WebSocketEvent (#13039)
* Make WebSocketEvent type immutable

* Update code to use updated immutable WebSocketEvent type

* Export WebSocketEvent fields and mark them as deprecated
2019-12-24 09:32:11 +01:00
Jesús Espino
b3e49ec45c
Use require/assert.Empty instead of require/assert.Len(t, X, 0) (#13413) 2019-12-22 12:35:31 +01:00
Jesse Hallam
89c0b61bc3
MM-21103: change plugin signature path (#13360)
* MM-21103: change plugin signature path

Save as `<plugin_id>.tar.gz.sig` instead of `<plugin_id>.sig`. The latter was a relic of the previous design to support multiple plugin signatures, but now creates an inconsistency with how the original source files were supplied as `<some_name>.tar.gz` and `<some_name>.tar.gz.sig`.

Fixes: https://mattermost.atlassian.net/browse/MM-21103

* relax signature matches to avoid assuming signatures always exist
2019-12-12 13:45:55 -04:00
Shota Gvinepadze
9898f91f7e
[MM-20309] production public key (#13244)
* Add production public key

* Fix unit tests
2019-12-05 23:41:52 +04:00
Ali Farooq
4e5004ddd2 MM-20642 - Failed to install marketplace plugin on older mattermost servers (#13237)
Automatic Merge
2019-12-02 13:42:54 -05:00
Jesús Espino
a63684fcb5
Consistent license message for all the go files (#13235)
* Consistent license message for all the go files

* Fixing the last set of unconsistencies with the license headers

* Addressing PR review comments

* Fixing busy.go and busy_test.go license header
2019-11-29 12:59:40 +01:00
Miguel de la Cruz
2259b7f2a8
[MM-19948] Set version on module file and internal paths (#13186)
* [MM-19948] Set version on module file and internal paths

* Fixes after merge

* Fix i18n checker error
2019-11-28 14:39:38 +01:00
Rodrigo Villablanca Vásquez
75c4fcb87e MM-17006: An http 404 response code is returned when deleting a non existent plugin (#11631)
Automatic Merge
2019-11-26 06:36:51 -05:00
Ali Farooq
a6e992ae74
MM-16368 - Plugin Signing (#13017)
* [MM-18757] POST handler for `/plugins/marketplace` (#12372)

* Implement installMarketplacePlugin

* Add InstallMarketplacePlugin endpoint

* Fix go.mod

* merge with master

* Fix go.mod

* Fix plugin tests

* Move get plugin to marketplace client

* Fix stylistic concerns

* Add trailing newline to the go.mod

* [MM-16586] Add plugin signature settings (#12390)

* MM-17149 - Extend config.json for marketplace settings (#11933)

* MM-17149 - Extend config.json for marketplace settings

* Renamed MarketplaceUrl, tracking default marketplace url

* Added EnableMarketplace to the client config

* Revert "Added EnableMarketplace to the client config"

This reverts commit 0f982c4c66.

* MM-17149 - Added EnableMarketplace to the client config (#11958)

* Added EnableMarketplace to the client config

* Moved EnableMarketplace setting out of limited client configuration

* Add public key settings to the config.json

* Rename PublicKeys to SignaturePublicKeyFiles

* Change filepath.Split to Base

* Remove additional prints

* Force extention of a public key file

* Remove config validation

* Remove error on delete

* Remove config cloning

* Add error messages

* Add plugin public key tests

* Rename extension to PluginSignaturePublicKeyFileExtention

* Remove EnforceVerification

* Change []*PublicKeyDescription to []string

* Change .asc extension to .plugin.asc

* Change ordering of public methods

* Change plugin key commands

* Update examples in the plugin key commands

* Remove forcing extention

* Add verify signature in settings

* Fix tabbing

* Fix naming

* Remove unused text

* Remove unused text

* Update command examples

* Fix unit tests

* Change errors.New to errors.Wrap

* Fix verbose flag

* Change .asc to .gpg

* Fix }

* Change AddPublicKey signature

* Change public.key extension

* Add plugin public key command tests

* Update en.json

* Bootstrap the public keys

* Update en.json

* Fix en.json

* Fix en.json

* Bootstrap hard-coded public key

* Remove unused texts in en.json

* Change file to name

* Add license header

* Update development public key

* Remove writeFile method

* Remove .plugin.asc extension

* Rename publiKey to mattermostPublicKey

* Remove init_public_keys string

* GolangCI

* Closing file handlers

* Fixed test that was installing nps plugin

* [MM-19798] Implement plugin signature verification (#12768)

* MM-17149 - Extend config.json for marketplace settings (#11933)

* MM-17149 - Extend config.json for marketplace settings

* Renamed MarketplaceUrl, tracking default marketplace url

* Added EnableMarketplace to the client config

* Revert "Added EnableMarketplace to the client config"

This reverts commit 0f982c4c66.

* MM-17149 - Added EnableMarketplace to the client config (#11958)

* Added EnableMarketplace to the client config

* Moved EnableMarketplace setting out of limited client configuration

* Add public key settings to the config.json

* Rename PublicKeys to SignaturePublicKeyFiles

* Change filepath.Split to Base

* Remove additional prints

* Force extention of a public key file

* Remove config validation

* Remove error on delete

* Remove config cloning

* Add error messages

* Add plugin public key tests

* Rename extension to PluginSignaturePublicKeyFileExtention

* Remove EnforceVerification

* Change []*PublicKeyDescription to []string

* Change .asc extension to .plugin.asc

* Change ordering of public methods

* Change plugin key commands

* Update examples in the plugin key commands

* Remove forcing extention

* Add verify signature in settings

* Fix tabbing

* Fix naming

* Remove unused text

* Remove unused text

* Update command examples

* Fix unit tests

* Change errors.New to errors.Wrap

* Fix verbose flag

* Change .asc to .gpg

* Fix }

* Change AddPublicKey signature

* Change public.key extension

* Add plugin public key command tests

* Update en.json

* Bootstrap the public keys

* Update en.json

* Fix en.json

* Fix en.json

* Bootstrap hard-coded public key

* Remove unused texts in en.json

* Change file to name

* Add license header

* Implement plugin signature verification

* Remove benburker openpgp

* Update en.json

* Update development public key

* Add support of multiple signatures in filestore

* Update en.json

* Run go mod vendor

* Fix style

* Remove writeFile method

* Remove .plugin.asc extension

* Rename publiKey to mattermostPublicKey

* Verify plugin with mattermost public key

* Remove init_public_keys string

* Add InstallPluginWithSignature method and  Refactor

* Add signature verification on claster notification

* Remove armored signature headers

* Add error strings

* Fix en.json

* Change signatureStorePath

* Implement minor fixes

* Refactor plugin install methods

* Add installPlugin method to uploadPlugin

* Update en.json

* Refactor installPlugin

* Limit number of signatures

* Close signatures

* Fix helper function

* Fix fromReadCloseSeekerToReadSeeker

* Cleaned up ReadCloseSeeker for signatures

* Remove signature truncation on FS

* GolangCI

* Add tests for armored signatures and plugin uploads

* Fix nil slice issue

* Fix TestPluginSync

* Fixed tests

* Return io.ReadSeeker from downloadFromUrl

* Add log for the found plugins in the file store

* Remove logging plugin detection info

* [MM-20134] Consume and store single-signature for each plugin (#13081)

* Consume and store single-signature for each plugin

* Fix en.json

* Remove saveSignature method

* Remove public key hash

* PR Feedback

* refactored config

* PR feedback
2019-11-18 19:02:41 -05:00
Jesse Hallam
2571d97723
MM-18517: Plugin Marketplace (Phase 2) (#13086)
* MM-17549: use StatusNotFound when deleting plugin (#12983)

Trying to delete a plugin that does not exist should fail with a 404, not a 400.

Relates-to: https://mattermost.atlassian.net/browse/MM-17549

* MM-19630: marketplace: model ReleaseNotesURL (#13083)

Automatic Merge

* MM-20065: allow prepackaged plugin upgrade (#13076)

* MM-20065: allow prepackaged plugin upgrade

When locally installing prepackaged plugins, skip if a plugin exists with the same id and is the same or a newer version.

This is effectively a "poor man's" rework of prepackaged plugins to allow upgrade of prepackaged plugins via the marketplace. The larger plan to rework prepackaged plugins was deferred from v5.18.

Fixes: https://mattermost.atlassian.net/browse/MM-20065

* eliminate unnecessary installPlugin

* fix TestPluginSync defaults to match minio

* cleanExistingBundles

* close prepackaged filereader

* simplify
2019-11-18 13:40:49 -04:00
Ben Schumacher
e61340c54e Run gofmt -s against codebase (#12943) 2019-10-28 06:08:08 -07:00
Ali Farooq
72965d864a
MM-18881 - Attempting to configure uninstalled plugins gets st… (#12687)
* MM-18881 - Attempting to configure uninstalled plugins gets stuck at Loading... without timeout.

* Added a test

* Closing websocketclient
2019-10-10 15:01:39 -04:00
Jesse Hallam
4ce7b92283 MM-17023: Plugin Marketplace (#12183)
* MM-17149 - Extend config.json for marketplace settings (#11933)

* MM-17149 - Extend config.json for marketplace settings

* Renamed MarketplaceUrl, tracking default marketplace url

* Added EnableMarketplace to the client config

* Revert "Added EnableMarketplace to the client config"

This reverts commit 0f982c4c66.

* MM-17149 - Added EnableMarketplace to the client config (#11958)

* Added EnableMarketplace to the client config

* Moved EnableMarketplace setting out of limited client configuration

* MM-17150, MM-17545, MM-18100 - Implement GET /api/v4/plugins/m… (#11977)

* MM-17150 - Implement GET /api/v4/plugins/marketplace proxying upstream
MM-17545 - Merge locally installed plugins into GET /api/v4/plugins/marketplace

* Replaced MarketplacePluginState with Installed

* Setting InstalledVersion instead of Installed

* marketplace client setting per_page if non zero

* Creating insecure client for marketplace url

* Fixed trailing slash for default marketplace url

* Adding filtering

* Fixed function names

* Renamed Manifest() to GetManifest(), added godoc for BaseMarketplacePlugin

* Handling plugin.ErrNotFound correctly

* Checking err == nil instead when a plugin is installed

* MM-18450 - Local-only plugin search (#12152)

* MM-17846: plugin icons (#12157)

* MM-17846: add support for plugin icons

Extend the model definitions to support plugin icons from the marketplace.

* s/IconURL/IconData

* MM-18475 - Converge on snake_case responses from the marketplace (#12179)

* MM-18520 - MM-Server should forward server version to marketplace server (#12181)

* Renamed request to filter client4.GetMarketplacePlugins

* Renamed request to filter

* Guarding against bad marketplace server response
2019-09-17 15:02:26 -04:00
Ali Farooq
5754ffc320
MM-17087 - Disable plugin on removal (#11779)
* MM-17087 - Disable plugin on removal

* Updated documentation

* Got reid of notifyPluginEvents

* Updated documentation

* Added plugin installation/activatoin flow as a toplevel go doc in plugin_install.go

* Generating webapp bundle on plugin installation

* Fixed shadowing issue

* Updated doc to include unguarded race condition

* Renamed GenerateWebappBundle

* Added a debug log when peers are not ready to notify

* Updated docs

* Removed extra line
2019-08-22 15:17:47 -04:00
Maria A Nunez
53f4bf1ec1
MM-17688 - Updated http timeout for plugin install from URL (#11887)
* Bumped http request timeout for plugin install from URL

* Added unit test for timeout higher than 30 seconds. Other minor PR feedback

* Fixed spacing
2019-08-22 13:33:28 -04:00
Jesse Hallam
98ff5fab32
MM-16261: Synchronize plugins in HA (#11657)
* MM-16272 - Synchronize plugins across cluster (#11611)

* MM-16272 - Synchronize plugins across cluster

* Adding a test

* MM-16272 - Fixed tests

* MM-16272 - PR feedback

* MM-16270 - Plugin Sync (#11615)

* Initial implementation for plugin synch with file store. WIP

* Removed ListAll implementation. Used ListDirectory and change localstore to be consistent and return all items (files and folders) from directory

* Refactored plugin filestore operations out of main install/remove plugin

* Fixing error handling details

* Changes to use structured logging

* More logging fixes

* Wording and comments improvements

* Error handling and control flow improvements

* Changed managed flag check to use os.stat

* Added file store plugin dir and filename consts

* Replaced FileRead to use a the FileReader in PluginSync

* Minor styling and PR feedback changes

* Minor error handling improvements

* Added unit test for SyncPlugins. Changed SyncPlugins to use plugins environment to list available plugins

* PR Feedback improvements

* Minor err handling fix

* Removing FileStorePath from PluginEventData (#11644)

* Fix plugin path (#11654)

* tweak path, logging

Fix an issue not finding the plugins folder in S3. Tweak logging messages to add additional clarity.

* Removing FileExists check when Syncing plugins. Updated localstore to not return an error when directory does not exist

* PR Feedback

* Install prepackaged plugins locally only (#11656)

* s/uninstall/remove

* Updated ClusterMessage comment

* Updated PluginSync to test against s3 + local storage
2019-07-18 15:05:53 -03:00
Maria A Nunez
859c571558 MM-16261 - Store Plugin in File Store (#11511)
* Implemneted saving the plugin bundle on the file store upon plugin upload

* Fixed compilation error

* Fixed compilation issue

* Added deletion from file store upon plugin uninstall

* Added condition to delete from store only when exists. Added case of saving the bundle to the store when uploading from url. Added checks in plugin tests

* Fixed compilation error

* Moved storage of plugin bundle within app/installPlugin

* Moved storing to filestore before enabling the plugin

* Fixed error handling

* Code styling improvements

* Minor styling fix
2019-07-10 17:05:33 -03:00
Michael Kochell
8cdf5ffe67
[MM-16376] Allow server to download and install a plugin from… (#11372)
* Initial implementation of plugin remote source

* Implement API route

* Test API route

* Add i18n

* Handle different error cases in API route

* Include missing i18n translation

* Include AllowInsecureDownloadUrl in telemetry capture

* Updates from PR feedback

* Use HTTPService instead of http.Get

* Remove InstallPluginFromUrlForced from client4

* Use net/url library to inspect url scheme

* remove PluginDownloadUrl from web/params.go

* Allow plugin downloads from internal sources
2019-06-26 15:45:07 -04:00
Jesse Hallam
3a71709103
MM-13893: refactor config (#10230)
* refactor utils/config* to config/

* pull validateLdapFilter into app

* clean up Config/GetConfig/GetSanitizedConfig usage

Eliminate app.GetConfig() in favour of just using app.Config() directly,
but expose app.GetSanitizedConfig() for when the old behaviour was
required.

* web: isolate config setup

* TestInvitePeopleProvider: make config explicit

* regenerateClientConfig: avoid racey map access

* integrate watch flag into app.ConfigFile option

* make app.Option return an error

* release.mk: only cp static files from config/

* release.mk: fix cp static files from config/

* api4: TestPlugin cleanup

* s/c/cfg/ for clarity

* fix merge conflict

* testlib: allow customization of testlib driver name
2019-02-12 08:37:54 -05:00
Christopher Speller
8429add371
Cleanup related to context refactor (#9988) 2018-12-17 08:51:46 -08:00
Lev
0541e765ad
MM-8662: Added force flag to uploadPlugin http API (#9969)
MM-8662: Added `force` flag to uploadPlugin http API
2018-12-12 10:13:24 -08:00
Jesse Hallam
a78913178c Ensure unittest isolation (#9819)
* api4: fix TestGetUsersNotInTeam assertions

This test was relying on data from a previous test run. With the data cleared before each test, the assertions much match reality.

* *testlib: always InitSystemAdmin

Some tests implicitly relied on the basic user having system
administrator privileges because it was the first user created as such.
Eliminate `InitSystemAdmin` and explicitly create the system admin user
instead to avoid this ambiguity going forward.

* *testlib: drop all tables before each test

* api4: split up TestChannelDelete to avoid duplicate InitBasic

* api4: teardown in TestResetPassword, for when this test comes back

* invalidate cache on DropAllTables

This is necessary since the test store persists across tests.

* disable parallel tests

While tests within a package must be explicitly parallelized using `t.Parallel()`, tests across packages are run in parallel by default.  This causes problems given that the tests all currently share the same database instance.

Unfortunately, this also means that running the tests is much slower, but we can return to this later.
2018-11-20 20:16:25 -05:00
Joram Wilander
f2c1803905
Make plugin IDs case insensitive (#9117) 2018-07-16 16:56:55 -04:00
Christopher Speller
1e5c432e10
MM-10702 Moving plugins to use hashicorp go-plugin. (#8978)
* Moving plugins to use hashicorp go-plugin.

* Tweaks from feedback.
2018-06-25 12:33:13 -07:00
Jesse Hallam
9d701c7044 Fix various segfaults when running go test manually (#8448)
* failing to find i18n shouldn't segfault

The server was trying to handle the fact that it couldn't find the i18n
directory, by emitting a translated log message...

* fix utils.FindDir

The attempts to find the directory in the parent or grandparent directory
don't work if the current working directory was inside `enterprise`, with
`enterprise` itself being a symlink as per the usual developer setup.

Recurse to the root of the filesystem, cleaning the path along the way
to work around this limitation (and allow tests to be run from an
arbitrarily deep nesting level.)

Fix corresponding usages to employ filepath.Join.

* failing to find html templates shouldn't segfault

* fail fast if the test user cannot be created

* rework utils.FindDir to retain backwards compatibility
2018-03-21 18:27:14 +00:00
Chris
4c17bdff1b
Add plugin slash command support (#7941)
* add plugin slash command support

* remove unused string

* rebase
2017-12-08 13:55:41 -06:00
Joram Wilander
1012bce8bb
Remove license requirement for plugins (#7747) 2017-10-31 10:39:59 -04:00
Joram Wilander
a0bfd2885d Add config setting to configure plugins directory (#7725) 2017-10-26 13:21:35 -04:00
Joram Wilander
1d968eb55e PLT-7849 Add config setting to disable plugin uploads (#7666)
* Add config setting to disable plugin uploads

* Update unit test
2017-10-25 08:52:50 -04:00
Joram Wilander
16b845c0d7 Differentiate between installed and activated states for plugins (#7706) 2017-10-25 08:17:17 -04:00
Chris
8e19ba029f Reduce utils.Cfg references (#7650)
* app.UpdateConfig method

* test fix

* another test fix

* the config override option as-was is just error prone, remove it for now

* derp
2017-10-18 15:36:43 -07:00
Chris
9bc7af0c57 Don't use global app for api / api4 tests (#7528)
* don't use global app for api / api4 tests

* put sleep back. we're gonna have to do some goroutine wrangling

* fix oauth test config assumptions

* jobs package, i'm comin' for you next

* app test fix

* try increasing sleep a little
2017-10-02 09:50:56 +01:00
Joram Wilander
2628022275 PLT-7622 Improvements to server handling of webapp plugins (#7445)
* Improvements to server handling of webapp plugins

* Fix newline

* Update manifest function names
2017-09-15 08:51:46 -04:00
Chris
4731b8f9b9 fix plugin test (#7436)
* fix plugin test

* ugh. global state

* Fix plugin test

* Fix TestDeleteChannel

* Remove plugin config listener when shutting down
2017-09-12 15:12:29 -04:00
Chris
b066b6df13 Remove global app references (#7433)
* remove global app references

* test fix

* fix api4 test compilation
2017-09-12 09:19:52 -05:00
Chris
402491b7e5 PLT-7407: Back-end plugins (#7409)
* tie back-end plugins together

* fix comment typo

* add tests and a bit of polish

* tests and polish

* add test, don't let backend executable paths escape the plugin directory
2017-09-11 10:02:02 -05:00
Christopher Speller
29fca51821
Renaming repo 2017-09-06 23:11:59 -07:00
Chris
1adfd0e9be app type transition (#7167) 2017-09-06 17:12:54 -05:00
Joram Wilander
899ab31fff Implement experimental REST API endpoints for plugins (#7279)
* Implement experimental REST API endpoints for plugins

* Updates per feedback and rebase

* Update tests

* Further updates

* Update extraction of plugins

* Use OS temp dir for plugins instead of search path

* Fail extraction on paths that attempt to traverse upward

* Update pluginenv ActivePlugins()
2017-09-01 09:00:27 -04:00