Commit graph

128 commits

Author SHA1 Message Date
Miguel de la Cruz
809ad4f76d
Adds Remote Cluster related API endpoints (#27432)
* Adds Remote Cluster related API endpoints

New endpoints for the following routes are added:

- Get Remote Clusters at `GET /api/v4/remotecluster`
- Create Remote Cluster at `POST /api/v4/remotecluster`
- Accept Remote Cluster invite at `POST
/api/v4/remotecluster/accept_invite`
- Generate Remote Cluster invite at `POST
/api/v4/remotecluster/{remote_id}/generate_invite`
- Get Remote Cluster at `GET /api/v4/remotecluster/{remote_id}`
- Patch Remote Cluster at `PATCH /api/v4/remotecluster/{remote_id}`
- Delete Remote Cluster at `DELETE /api/v4/remotecluster/{remote_id}`

These endpoints are planned to be used from the system console, and
gated through the `manage_secure_connections` permission.

* Update server/channels/api4/remote_cluster_test.go

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>

* Fix AppError names

---------

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Mattermost Build <build@mattermost.com>
2024-07-04 10:35:26 +02:00
Agniva De Sarker
d20b55bde8
MM-57375: Update to latest minio image (#27475)
* test with latest minio image
* Update the KMS key
- Also use latest config settings. The older ones were deprecated.
2024-06-28 20:34:18 +05:30
Miguel de la Cruz
2aff84a72e
Sanitize RemoteEmail user prop (#27170)
* Sanitize RemoteEmail user prop

If the server is configured to hide user emails, the "RemoteEmail"
user property will be sanitized as well, effectively hiding the real
email of remote users.

* fix merge conflict

---------

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-26 13:48:31 -04:00
Doug Lauder
6773d13dee
MM-58255 Ensure remote users do not get valid email addresses (#27421)
* remote users don't get valid email addresses; remote users cannot have access tokens

* block notification emails for remote users

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-25 09:26:08 -04:00
Miguel de la Cruz
a181b3bc7b
Fix not enough return values in handleChannelCreation (#27430) 2024-06-24 09:27:35 +02:00
Miguel de la Cruz
d490fdc1d9
Marks the RemoteTeamId field of RemoteClusters as deprecated (#27364)
* Marks the RemoteTeamId field of RemoteClusters as deprecated

The `RemoteTeamId` was used both in the `RemoteCluster` model and as
part of remote invites. It existed so two different remotes could have
multiple secure connections between them, and have each of those
connections scoped to a team, sharing through each only the channels
that belong to their corresponding team.

The way that we're thinking on the feature currently only contemplates
one secure connection between two servers, and shares all the
channels through that secure connection, so this field is no longer
needed.

As we don't have a system in place for the user to choose in which
team a channel should be created from an invite, this change adds a
mechanism that checks the invite for a teamId, and if it's not
present, fetches a team from the database to create the channel
into. This makes the change backwards compatible for secure
connections that already have an established behavior and allows us to
move forward with the implementation of an alternative.

* Mark invite teamId field as deprecated

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-22 23:08:55 +02:00
Doug Lauder
8181a9ddff
MM-57867 Don't delete existing DM on invitation error (#27357)
* ensure channel invitations create new channels; don't delete pre-existing channels on failure cleanup

* update comment
2024-06-19 16:20:46 +02:00
Daniel Schalla
1bbc3b4e83
[MM-58560] Configurable session revocation during password resets (#27286)
* [MM-58560] Allow for configurable session revocation during password reset

* Missing i18n additions

* Update Settings Wording

* Update Settings Wording #2

* Update default_config.ts for Session Termination

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-18 21:13:29 +02:00
Doug Lauder
bf8ddb4bdc
MM-57873 Check user visibility when accepting channel invites for DMs (#27331)
* check user visibility when accepting channel invites for DMs

* stronger visibility checking for DM users

* check for correct remoteid for remote user in DM invite

* fix unit test

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-13 11:40:31 +02:00
Miguel de la Cruz
9733694854
Updates remote user creation process to preserve username and email (#27334)
This change makes sure that the original username and email are always
preserved when creating a user coming from a remote cluster.
2024-06-13 09:56:39 +02:00
Doug Lauder
594ba6e665
MM-58577 Check remote ownership for posts and reactions (#27317)
* - ensure that posts and reactions can only be added via sync when coming from a remote that the target channel is shared with.
- ensure that posts and reactions are only modified/deleted by the remote that owns them.

* check that reaction belongs to post that belongs to channel that is shared with remote;  check that posts belong to channel shared with remote

* check for correct error type in unit test

* tweak unit test
2024-06-11 17:51:00 +02:00
Aditya Pratap Singh Hada
075e0df5df
[MM-57034]: Check for s3 object is File or Directory (#26837)
* check for path when number of object is one

* add test and updated condition to check for path

* updated test and removed string trim

* using exported method

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-11 17:32:57 +05:30
Doug Lauder
a3bc73c1dc
MM-58253 Ensure remotes can only update users belonging to them (#27290)
* - ensure new users get correct remoteID
- ensure remotes can only update users belonging to them
- ensure user remoteIDs cannot be changed once set

* make modules-tidy

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-06 10:28:02 -04:00
Christopher Speller
04181247f8
Move HTTP service to public for plugin use (#27284)
* Move httpservice for use in plugins

* Adapt httpservice for plugin use

* Fix lint
2024-06-05 09:58:04 -07:00
Harrison Healey
2bcaa42dc0
MM-58275 Ensure image proxy site URL is updated when that changes (#27214)
* MM-58275 Ensure image proxy site URL is updated when that changes

* Check if proxy settings changed using reflection
2024-06-04 14:06:37 -04:00
Doug Lauder
5114c3b7cd
when sync'ing users make sure the user's remoteid matches the remote issuing the update (#27203) 2024-06-04 11:30:18 +02:00
Doug Lauder
923f6a5943
MM-57867 Don't delete existing channel on invitation error (#27143)
* Do not delete existing channels
* don't allow resharung
2024-05-29 00:46:18 -04:00
Agniva De Sarker
6f3327ce0f
MM-58038: Use context to call master for DeletePost (#27098)
Calling app.DeletePost immediately after creating a post
is susceptible to replica lag because we were calling the
replica to check for the post.

We fix this by passing a context to always query master.

https://mattermost.atlassian.net/browse/MM-58038

```release-note
NONE
```
2024-05-24 19:35:48 +05:30
Doug Lauder
2fd5725c9f
when receiving file attachments for shared channels, ensure attachments are enabled and file size does not exceed maximum configured for receiving server. (#27018) 2024-05-21 15:46:48 -04:00
Jesse Hallam
630bd40141
Eliminate MM_SERVER_PATH (#24968) 2024-05-10 18:13:05 -03:00
Harrison Healey
de3c7ad544
MM-57878 Add PerformanceReporter for clientside performance metrics (#26800)
* Define rough code for PerformanceReporter

* Create a component to manage the PerformanceReporter

* Start adding tests for PerformanceReporter

* Add test for web vitals reporting

* Update schema to more closely match the API spec

* Collect marks as counters and further update structure of API payload

* Add some outstanding TODOs about the API structure

* Add counter for long tasks

* Add EnableClientMetrics without any System Console UI

* Have PerformanceReporter use EnableClientMetrics

* Have the PerformanceReporter only report results when logged in

* Add test for having PerformanceReporter fall back to fetch

* Stop logging errors for measurements failing

* Remove buffered from observer

* Remove the Mystery Ampersand

* Still record marks with telemetry actions even if telemetry is disabled

* Add timestamps to performance reports

* Reuse the new telemetry code for the old telemetry

* The second half of the last commit

* Use Node performance libraries in all tests

* Set version of PerformanceReport

* Switch to the proper version of EnableClientMetrics

* Remove TODO for unneeded field

* Add user agent and platform detection

* Updated metrics API route
2024-05-09 12:50:52 -04:00
Nick Misasi
f1019d076e
[CLD-7567] Deprecate Self Serve: Second Pass (#26853)
* Deprecate Self Serve: First Pass

* Fix ci

* Fix more ci

* Remmove outdated server tests

* Fix a missed spot opening purchase modal in Self Hosted

* Fix i18n

* Clean up some more server code, fix webapp test

* Fix alignment of button

* Fix linter

* Fix i18n server side

* Deprecate in product true up

* Add back translation

* Remove client functions

* Put back client functions

* webapp deprecation

* Deprecate Self Serve: Second Pass

* Fix various pipeline issues

* Fix linter

* Fix pipelines

* Fix handlers_test.go

* Fix console.error around hostedCustomer in reducer

* PICKY LINTER PLEASE

* Fix webapp tests, various other fixes for the CI pipelines

* Fix i18n

* Updates to accomadate enterprise code removal

* Fix mocks

* More removal

* Fix

* Adjustments from PR

* Fixes for QA Feedback

* Update

* Add migrations to remove true up review history

* Fix migrations check

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: maria.nunez <maria.nunez@mattermost.com>
2024-05-02 09:15:15 -04:00
Robert Obermeier
322a500092
Add WebP to MediaFormats #26628 (#26805)
Changes the default content disposition for WebP attachments
from *download* to *inline*.

Add WebP codec support side-effect to decode.go. To prevent future
issues caused by possible changes in emoji.go.
2024-04-26 17:06:07 +02:00
Doug Lauder
39ba2e72c0
Fix flaky ping test for remote clusters service (#26821)
* fix flaky ping test for remote clusters service

* refactor ping test

* Update server/platform/services/remotecluster/ping_test.go
2024-04-22 17:33:37 +02:00
Doug Lauder
6aaabfb376
MM-57786 Fix Shared Channels plugin api (#26753)
* always ping on plugin registration; SharedChannel.IsValid allow no team for GM

* wait for services to start before ping

* ping plugin remotes synchronously on startup

* remove the waitForInterClusterServices stuff

* don't set remoteid when inviting remote to channel

* Update server/public/model/remote_cluster_test.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

* address review comments

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2024-04-15 16:18:25 -04:00
Agniva De Sarker
870cee30c6
MM-57759: Added mockery support for 1.22 (#26774)
* Revert "Revert "MM-57759: Bump mockery to version 2.42.2 to support go  1.22^" (#26772)"

This reverts commit cd3b5b46e1.

* Added the hooks.go file changes as well

```release-note
NONE
```
2024-04-13 08:15:59 +05:30
Agniva De Sarker
cd3b5b46e1
Revert "MM-57759: Bump mockery to version 2.42.2 to support go 1.22^" (#26772) 2024-04-12 15:31:24 +02:00
Ezekiel
98712737e6
MM-57759: Bump mockery to version 2.42.2 to support go 1.22^ (#26761) 2024-04-12 10:38:34 +02:00
Ben Schumacher
c5cf4101da
[MM-57085] Remove UseNewSAMLLibrary experimental setting (#26392)
Co-authored-by: Mattermost Build <build@mattermost.com>
2024-04-05 16:56:01 +02:00
Agniva De Sarker
dc3e5b9269
MM-57532: Improve timeouts for some tests (#26642)
- Bump timeouts for some cases
- Improve sleep with require/assert.EventuallyT
for better performance and reliability.

https://mattermost.atlassian.net/browse/MM-57532

```release-note
NONE
```
2024-04-05 19:28:49 +05:30
Claudio Costa
bc6182229b
Make S3 upload part size configurable (#26533) 2024-04-05 07:20:48 -06:00
Doug Lauder
1cb936b0b7
Create propery SC when InviteRemoteToChannel and channel should be shared. (#26469) 2024-03-18 10:54:23 -04:00
Harshil Sharma
774df37464
Rebuild channel index (#26080)
* WIP

* Added rebuild channels index functionality

* Added rough logic to send message to all sysadmins

* WIP

* WIP

* WIP

* Cleanup

* i18n fix

* reading through all pages of system admins

* Fixed webapp style

* i18n fix

* Added help text

* i18n fix

* i18n update

* Updated system console button action

* Updated snapshots

* some cleanup

* Updated snapshot

* Update server/channels/app/server.go

Co-authored-by: Daniel Espino García <larkox@gmail.com>

* fixed typo

* Refactoring to improve readibility

* moved index check to API later during config update

* Added some docs

* Updated get system bot

---------

Co-authored-by: Daniel Espino García <larkox@gmail.com>
2024-03-13 10:26:06 +05:30
Dahlton S
fd713ae9bb
[MM-56651] fix: set cache duration (#26338)
* fix: set cache duration

* fix: extra 0
2024-03-05 22:49:26 +05:30
Agniva De Sarker
f9861b8666
MM-56987: Improve cache API (#26298)
This is in preparation to make the codebase
ready to use Redis. In Redis, iterating all
the keys at once is an expensive operation.
It is recommended to work on batches of keys.

Remove the Len method as it was unused.

I tried to repurpose the Keys method to iterate
on keys rather than returning all keys at once, but
it has other complicacies because the code calls
other cache functions on those keys, so to handle
the LRU cache properly, it becomes slightly more
painful.

For now, we keep it like this and rather collect
all keys from Redis and then return.

https://mattermost.atlassian.net/browse/MM-56987

```release-note
NONE
```
2024-03-01 09:46:09 +05:30
Maria A Nunez
e9b9d4ff60
Deprecate admin advisor (#26045)
* Deprecate admin advisor

* Webapp portion

* More webapp deprecation

* More cleanup

* Linting

* emoved metric ack dialog from annoucenemet bar

* Cleanued up uninsed i18n strings

* Updated test

* fixed types

* Updating server test

* Updated i18n

* Updated cypress test:

* Updated cypress test:

---------

Co-authored-by: harshil Sharma <harshilsharma63@gmail.com>
2024-02-26 09:05:00 +05:30
Aswath S
dc8fc773dc
MM-55733 add request context to Channelstore.Save method (#26141) 2024-02-24 13:08:51 +01:00
Agniva De Sarker
c4432a5234
Remove unnecessary cache invalidation for channel (#26256)
```release-note
NONE
```
2024-02-22 10:01:17 +05:30
Doug Lauder
38bbf04e48
Metrics for shared channels (#26199)
* add metrics definitions for shared channels
2024-02-21 17:21:35 -05:00
Aditya Pratap Singh Hada
6541708a93
[MM-55735] Add Request Context to PostStore.Save (#26048) 2024-02-20 17:07:20 +01:00
Matthew Straughn
1e794136c8
MM 19654 s3 debugging logs overlap, Wrapped mlog in an io.Writer func (#25788)
* updated go mod and sum files

* Revert "updated go mod and sum files"

This reverts commit 088dd00a84.

* wrapped mlog in and io.Writer func to pass to TraceOn

* fixed var and func receiver names

* removed the underscore in the func receiver.

* passed a pointer of s3Trace to TraceOn

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-02-20 07:59:26 +05:30
Trang
f130615011
[MM-55737] Add Request Context to UserStore.Save (#26109) 2024-02-16 17:12:49 +01:00
Ben Schumacher
de3e5aab25
[MM-53156] Remove Multi-Product architecture (#25669) 2024-02-15 13:01:44 +01:00
Agniva De Sarker
bdce9f42d8
MM-56358: Unskip racy test TestUploadLicenseFile (#26009)
This was already fixed before in https://github.com/mattermost/mattermost/pull/24971.
We just enable the test now.

https://mattermost.atlassian.net/browse/MM-56358

```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-02-13 21:03:17 +05:30
Michael Kochell
4e071e861c
Webapp - Outgoing OAuth Connections (#25507)
* added store

* make generated

* add missing license headers

* fix receiver name

* i18n

* i18n sorting

* update migrations from master

* make migrations-extract

* update retrylayer tests

* replaced sql query with id pagination

* fixed flaky tests

* missing columns

* missing columns on save/update

* typo

* improved tests

* remove enum from mysql colum

* add password credentials to store

* license changes

* OAuthOutgoingConnectionInterface

* Oauth -> OAuth

* make generated

* copied over installed_oauth_apps component and renamed things to installed_outgoing_oauth_connections

* merge migrations

* renamed migrations

* model change suggestions

* refactor test functionsn

* migration typo

* refactor store table names

* updated sanitize test

* cleanup merge

* refactor symbol

* "installed outgoing oauth connections" page works

* move things into a nested folder

* add and edit page stubs work

* list endpoint

* oauthoutgoingconnection -> outgoingoauthconnection

* signature change

* i18n update

* granttype typo

* naming

* api list

* uppercase typo

* i18n

* missing license header

* fixed path in comments

* updated openapi definitions

* changes to support selecting command request url

* sanitize connections

* make generated

* test license and no feature flag

* removed t.fatal

* updated testhelper calls

* yaml schema fixes

* switched interface name

* suggested translation

* missing i18n translation

* management permission

* moved permission initalization to proper place

* endpoints

* put tests

* error check typo

* fixed specific enttity urls

* tests

* read permission check

* updated openapi definitions

* i18n

* GetConnectionByAudience method

* notes

* replaced GetConnectionsByAudience with a filter

* added custom oauth token object

* updated interface and usage

* properly set enterprise interface

* move retrieval logic to impl

* webhook tests

* translations

* i18n: updates

* address comments

* endpoint and tests

* i18n

* api docs

* fixed endpoint path

* sq.like

* use filter object instead of parameters

* set url values if not empty

* typos

* converted some components to function components, and move around files

* correctly check token url

* restore flag to previous value

* added command oauth handler

* update enterprise imports

* migrate last component to function component

* Added enterprise import

* refactor permissions and add necessary webapp code

* Check correct flag in permission tree

* allow partial updates

* sort i18n webapp

* missing test modification

* fixed webapp i18n sorting

* allow validating stored connections

* added missing translation

* fix finished adding connection link and text on result page

* added missing permission to smoke tests

* missing role in smoke test

* updated translations

* updated translations

* support editing client secret on existing connection

* fix some i18n strings

* updated translations

* better error messages

* progress on using react select for command request url while maintaining typed in value

* remove writeheader, test

* HasValidGrantType

* end early to avoid nil pointer errors

* move slash command request url input box into its own component

* wrap components related to oauth connections in config check

* fix tests

* i18n-extract

* change some i18n strings to say "Outgoing OAuth 2.0 Connections"

* remove debug code

* fixed i18n

* updated i18n file

* feature configuration backend

* typo

* add system console setting

* Revert "typo"

This reverts commit 669da23e8e.

* Revert "updated i18n file"

This reverts commit d0882c0dd7.

* Revert "fixed i18n"

This reverts commit 3108866bc1.

* fixed i18n

* updated i18n file

* typo

* updated i18n

* updated i18n

* updated i18n

* updated version to 9.6

* replace feature flag with system console configuration

* i18n

* updated tests

* pr feedback

* fix styling of disabled text box

* fix styling of action links in integration console

* server changes for validation feature

* webapp changes for validation feature

* pencil icon styling

* styling fixes for oauth audience correct configuration message

* fix sanitize test

* remove max lengths from outgoing oauth connection form

* use config var in webapp instead of feature flag

* change asterisks to bullets

* update api docs for validate endpoint

* feedback from ux review

* fix lint, types, tests

* fix stylelint

* implement validation button under the token url input

* support wildcard for matching audience urls

* updates for styling

* update snapshots

* add doc links for the outgoing oauth connections feature

* change doc links to use permalink

* add docs link to system console

* fix: use limitedreader in json decoding

* fix: form error in validation

* management permission can read now

* updated api documentation

* doc typo

* require one permission to read only

* fix api connection list audience filter

* fix audience matching and add loading indicator

* fix team permissions on outgoing oauth connection api calls

* fix api doc and test, for adding team id to query params

* handle read permissions by adding a team in the payload

* missing teamid query parameter in test

* change validate button logic to not require audience urls to be filled out

* fix redux type

---------

Co-authored-by: Felipe Martin <me@fmartingr.com>
2024-02-09 14:49:49 -05:00
Doug Lauder
7b7bcff821
Shared channels auto-share DM & group messages (#26097)
* option for auto inviting plugin to all shared channels.
* auto-invite remotes to shared channels when flag set
2024-02-09 10:47:12 -05:00
Claudio Costa
7d43712daa
[MM-54984] Update code.sajari.com/docconv to latest v2 (#25975)
* Update code.sajari.com/docconv to latest v2

* go mod tidy
2024-01-23 09:40:57 -06:00
Doug Lauder
a07097ed57
Plugin API hook for Shared Channel file attachment sync (#25874)
* option for auto inviting plugin to all shared channels.

* auto-invite remotes to shared channels when flag set

* fix unit test

* immediately ping new remotes; fix unique siteurl bug

* make i18n-extract

* fix translations

* plugin hooks for file attachments

* hook for profile image sync

* fix profile image sync

* fix unit test

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-16 15:48:51 +01:00
Agniva De Sarker
1d879ed0f4
MM-55295: Improve DB performance of some (Take 2) (#25865)
* Revert "Revert "MM-55295: Improve DB performance of some APIs (#25318)" (#25852)"

This reverts commit 077221a940.

* Fix the issue

```release-note
NONE
```

* lint fix

```release-note
NONE
```
2024-01-11 10:18:36 +05:30
Ben Cooke
bb88b92b4c
[MM-53990] Support a global retention time of less than 1 day (#25196)
* adding new MessageRetentionHours config

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-09 14:10:22 -05:00