* avoid replica race lag when remembering ServerID
In an HA environment, with a master and read replica, querying the server id from the store runs the risk of returning a value saved to master but not yet replicated. Avoid this by using the telemetry service value directly when available.
Fixes: MM-65960
* Add Get(ByName)WithContext
* explicitly use master for ServerId
* mock GetByNameWithContext
* more mocking
* more mocks
The production password hasher uses PBKDF2 with 600,000 iterations,
which is slow especially when combined with race detection. This
adds a fast test hasher (work factor 1) that can be used during tests
to speed up user creation.
The fast hasher is only available in non-production builds via build
tags, ensuring it cannot be used in production.
* Support for permissions allowing end users to create and manage their own integrations if sysadmin deems necessary
* Adjustments based on new understanding
* remove extra functions now that we've consolidated
* Fix webapp i18n
* Update snapshots
* Fix test
* Fix some tests, refactor some more, and add a few extra
* fix linter
* Update snapshots
* Fix test
* Missed some cleanup
* Fix e2e
* Fi
* Fix
* Fixes from PR feedback
* Update snapshots
* Fix tests
* Fix slash command list endpoint per PR feedback. Remove changes around OAuth Apps
* Further reversions of oauth stuff
* Update tests
* Small changes to fix when customOnly=false
* Remove extra perm from cypress
* Fixes from Eva's feedback
* Fix i18n
* More fixing
* More fixing
* MM-64486: Remove telemetry
Remove telemetry from Mattermost. We're no longer relying on Rudder upstream, and no longer making use of this information.
* recover mock for SystemStore.Get
* Fix TestClearPushNotificationSync by adding missing SystemStore mock
The test was failing because the SystemStore mock was missing the Get()
method that's required by the ServerId() function. Added the missing mock
to return a StringMap with SystemServerId.
* fix mocking issue
* Remove now-unused telemetry and constants
* Remove "Disable telemetry events" debug setting
* Remove empty functions
* Remove most "Telemetry tracking removed" comments
* Remove remains of DataPrefetch telemetry
* Remove now-unused prop from InviteMembersButton
* Remove trackDotMenuEvent
* Remove some more leftover comments
* Remove lingering logic related to trackingLocation
* Remove now-unused argument from useCopyText
* Remove lingering telemetry references from PreparingWorkspace
* fixup Remove trackDotMenuEvent
* Remove lingering telemetry references from signup page and password check
* Update snapshots and fix test broken by my changes
* Fix unintended behavior change in thread list filtering
Remove handleSetFilter wrapper that was accidentally modified during
telemetry removal. The function was calling clear() when switching to
unread filter, which was not the original behavior. Use setFilter
directly instead, restoring the original functionality.
* Remove unused useOpenDowngradeModal hook
The useOpenDowngradeModal hook was not being used anywhere in the codebase.
* Remove unused expandableLink from useExpandOverageUsersCheck
The expandableLink return value was not being used by any components.
* Re-add missing TeamLinkClicked performance telemetry
The mark(Mark.TeamLinkClicked) call was accidentally removed from the
handleSwitch function. This telemetry is needed for Looker-based
performance tracking.
* drop LogSettings.VerboseDiagnostics
---------
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* WIP
* Added post flagging properties setup
* Added tests
* Removed error in app startup when content flaghging setup fails
* Updated sync condition:
* WIP
* MOved to data migration
* lint fix
* CI
* added new migration mocks
* Used setup for tests
* some comment
* removed empty files
* Added another property field
* WIP
* Updated test
* Stored version in system key
* fixed tests
* MM-6449 - manage channel access rules permissions backend part
* add the system console changes to show the new permission
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* TestPool
* Store infra
* Store tests updates
* Bump maximum concurrent postgres connections
* More infra
* channels/jobs
* channels/app
* channels/api4
* Protect i18n from concurrent access
* Replace some use of os.Setenv
* Remove debug
* Lint fixes
* Fix more linting
* Fix test
* Remove use of Setenv in drafts tests
* Fix flaky TestWebHubCloseConnOnDBFail
* Fix merge
* [MM-62408] Add CI job to generate test coverage (#30284)
* Add CI job to generate test coverage
* Remove use of Setenv in drafts tests
* Fix flaky TestWebHubCloseConnOnDBFail
* Fix more Setenv usage
* Fix more potential flakyness
* Remove parallelism from flaky test
* Remove conflicting env var
* Fix
* Disable parallelism
* Test atomic covermode
* Disable parallelism
* Enable parallelism
* Add upload coverage step
* Fix codecov.yml
* Add codecov.yml
* Remove redundant workspace field
* Add Parallel() util methods and refactor
* Fix formatting
* More formatting fixes
* Fix reporting
* rm "No TEST_DATABASE... override" log message
Let's only log if this value is actually overridden.
* rm "(Created|Dropped) temporary database" message
* only log "Pinging SQL" on subsequent attempts
* disable morph logging from TestMain
* Fix style issues in store test files
- Add missing parameter to migrate() function calls in tests
- Remove unused log function in settings.go
- Fix formatting with go fmt
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* within sqlstore, use "enable" vs "disable" for clarity
* remove trailing newline from morph logs
---------
Co-authored-by: Claude <noreply@anthropic.com>
A recent change to the enterprise cluster code introduced a change to the enterprise API interface. GetClusterInfos() can now return an error. This commit introduces code to handle that error.
* Add config settings for additional security features on mobile
* Add system console settings for mobile security
* Update svg and link
* Fix strings
* Add test for the discovery feature
* Fix tests
* Add permission migrations
* Add relevant e2e tests
* Fix key alignment
* fix tests
* Fix lint
* Mock new migration
* Fix playwright prettier
* Add new section to delegated permissions
* Update snapshots
* Fix flakyness in playwright test
---------
Co-authored-by: Elias Nahum <nahumhbl@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
We do a cluster request to get the active and dead queues
from other nodes in the cluster to sync any missing
information.
We check the dead queue in the other nodes to see
if there's been any message loss or not. Accordingly,
we send just the active queue or both active and dead queues.
There's still an edge case that is left out where
a client could have potentially connected and reconnected
to multiple nodes leaving multiple active queues
in multiple nodes. We don't handle this scenario
because then potentially we need to create
a slice of sendQueueSize * number_of_nodes. And then
this can happen again, leading to an infinite increase
in sendQueueSize.
We leave this edge-case to Redis, acknowledging
a limitation in our architecture.
In this PR, when there's no message loss, we just
take the active queue from the last node it connected
to.
And if there's message loss where the client's
seqNum is within the last node's dead queue, we also
handle that.
But if there's severe message loss where the client's
seqNum falls within the dead queue of another node, then
we just send the data from that node to reconstruct the
data as much as possible. It could be possible to set
a new connection ID in this case, but this involves
more data transfer always from all nodes and recomputing
the state in the requestor node.
https://mattermost.atlassian.net/browse/MM-61904
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* Adds the main Property System Architecture components
This change adds the necessary migrations for the Property Groups,
Fields and Values tables to be created, the store layer and a Property
Service that can be used from the app layer.
* Update property field type to use user instead of person
* Update PropertyFields to allow for unique nondeleted fields and remove redundant indexes
* Update PropertyValues to allow for unique nondeleted fields and remove redundant indexes
* Use StringMap instead of the map[string]any on property fields
* Add i18n strings
* Revert "Use StringMap instead of the map[string]any on property fields"
This reverts commit e2735ab0f8.
* Cast JSON binary data to string and add todo note for StringMap use
* Add mocks to the retrylayer tests
* Cast JSON binary data to string in property value store
* Check for binary parameter instead of casting to string for JSON data
* Check property field type is one of the allowed ones
* Avoid reusing err variable to be explicit about the returned value
* Merge Property System Migrations into one file
* Adds NOT NULL to timestamps at the DB level
* Update stores to use tableSelectQuery instead of a slice var
* Update PropertyField model translations to be more explicit and avoid repetition
* Update PropertyValue model translations to be more explicit and avoid repetition
* Use ExecBuilder instead of ToSql&Exec
* Update property field errors to add context
* Ensure PerPage is greater than zero
* Update store errors to give more context
* Use ExecBuilder in the property stores where possible
* Add an on conflict suffix to the group register to avoid race conditions
* Remove badly used translation string
* Remove unused get in register group method
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Drop the legacy `X` suffix from `GetMasterX` and `GetReplicaX`. The
presence of the suffix suggests there's a `non-X` version: but in fact
we migrated these away a long time ago, so remove the cognitive
overhead.
As an aside, this additionally helps avoid trip up LLMs that interpret
this as "something to fix".
We create a custom histogram metric that logs the userID
when the observed value is greater or equal to the last bucket value.
This allows us to start tracking the slowest users of a system
while at the same time not polluting the Prometheus metrics
by storing a userID for every observation.
https://mattermost.atlassian.net/browse/MM-61887
```release-note
NONE
```
We were setting the user status to offline without
checking for connections on other nodes in a cluster.
Now we implement a request-response mechanism for the whole
cluster and we check that before setting a user to offline.
https://mattermost.atlassian.net/browse/MM-57153
```release-note
Fix a bug where the user status would incorrectly be set to offline
without checking for connections in other nodes in an HA cluster.
```
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
* create ChannelBookmarks table
* ChannelBookmark model
* channel bookamrks Store layer
* add GetBookmarksForAllChannelByIdSince
* add channel bookmarks to test store
* Add channel bookmarks to app layer
* remove index for createAt in channel bookmarks migrations
* remove createAt from select channel bookmark query and enable store delete bookmark test
* update reponse of UpdateBookmark
* rename db migration files
* channel bookmarks store update sort order
* channel bookmarks app layer update sort order
* fix lint & tests
* Fix lint and introduce util functions to insert / remove from slice
* remove model etag
* i18n
* defer remove file info after test run
* Fix tests passing the request context
* fix migrations
* fix TestRetry
* Add bookmark permissions (#25560)
* Adds channel bookmarks permissions
* Fix linter
* Remove unnecessary empty lines
* Remove scss change as it's not necessary anymore
* Fix mock store
* Fix mock store and add role entry
* Fix test
* Adds cypress test and update permissions migration to update admin roles
* Adds channel bookmarks roles to default admin roles
* Adds bookmark permissions to default role permissions constant in webapp
* Update mmctl test
* Update permission test after normalising the roles
* fix store tests
* fix app layer tests
* Add new bookmark endpoint (#25624)
* Adds channel bookmarks api scaffold and create endpoint
* Applies review comments to the API docs
* Adds websocket test to create channel bookmark
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-54426 exclude Channel Bookmarks files from data retention (#25656)
* Augment channel APIs to include bookmarks (#25567)
* update files docs for server 9.4
* Adds update channel bookmark endpoint (#25653)
* Adds update channel bookmark sort order endpoint (#25686)
* Adds update channel bookmark endpoint
* Updates edit app method to return the right deleted bookmark and adds tests
* Adds the update channel bookmark sort order endpoint
* Fix repeated test after merge
* Assign right permissions to each test
* Update store and app layer to return specific errors and add tests
* Adds delete channel bookmark endpoint (#25693)
* Updates edit app method to return the right deleted bookmark and adds tests
* Fix repeated test after merge
* Updates edit app method to return the right deleted bookmark and adds tests
* Adds delete channel bookmark endpoint
* Adds list channel bookmarks endpoint (#25700)
* Add channel moderation to bookmarks (#25716)
* fix migrations index
* fix getChannelsForTeamForUser
* fix getChannelsForTeamForUser
* fix bad merge client4
* fix file api with bookmark permission
* add ChannelBookmarks feature flag
* add missing translations
* Set DB column for type as enum
* use custom type for bookmark query using sqlx
* use transaction when saving bookmark
* return NewErrNotFound instead of Sql.ErrNoRows
* use squirrel for IN query
* add a limit of 1K for records in GetBookmarksForAllChannelByIdSince
* UpdateSortOrder with one single query instead of multiple updates
* fix shadow declaration
* fix channel bookmarks permission string definition in admin console
* fix another shadow declaration
* Fix model conversion
* add SplitSliceInChunks
* remove include bookmarks in channels api
* Cap amount of bookmarks per channel
* add etag back to get channels
* feedback review
* update file info when replacing a bookmark file
* return 501 not implemented when the license is not available
* add detail message when getting channel member on bookmark api
* start audit before permission check on create bookmark api
* use require.Eventuallyf for testing WS events
* remove unnecessary log in app layer
* use require instead of assert to avoid panics
* enforce limit when querying bookmarks since
* prevent to create/update bookmark if file is already attached
* fix lint
* delete file when a bookmark is deleted
* Dot allow to set a fileId and a url at the same time to a bookmark
* fix query to delete a file that belongs to a bookmark
* do not patch the bookmark type
* Server side FeatureFlag check (#26145)
* use ff in server, set ff to false
* turn on FF for unit tests
* defer unset FF for unit tests
* turn ff on for testing
* only allow attaching files that were uploaded for bookmark
* Set feature flag off as default
* fix lint
* update email templates as PR failed
* revert templates
* force the assignment of ID when creating a bookmark
* Fix unit tests
---------
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* added store
* make generated
* add missing license headers
* fix receiver name
* i18n
* i18n sorting
* update migrations from master
* make migrations-extract
* update retrylayer tests
* replaced sql query with id pagination
* fixed flaky tests
* missing columns
* missing columns on save/update
* typo
* improved tests
* remove enum from mysql colum
* add password credentials to store
* license changes
* OAuthOutgoingConnectionInterface
* Oauth -> OAuth
* make generated
* copied over installed_oauth_apps component and renamed things to installed_outgoing_oauth_connections
* merge migrations
* renamed migrations
* model change suggestions
* refactor test functionsn
* migration typo
* refactor store table names
* updated sanitize test
* cleanup merge
* refactor symbol
* "installed outgoing oauth connections" page works
* move things into a nested folder
* add and edit page stubs work
* list endpoint
* oauthoutgoingconnection -> outgoingoauthconnection
* signature change
* i18n update
* granttype typo
* naming
* api list
* uppercase typo
* i18n
* missing license header
* fixed path in comments
* updated openapi definitions
* changes to support selecting command request url
* sanitize connections
* make generated
* test license and no feature flag
* removed t.fatal
* updated testhelper calls
* yaml schema fixes
* switched interface name
* suggested translation
* missing i18n translation
* management permission
* moved permission initalization to proper place
* endpoints
* put tests
* error check typo
* fixed specific enttity urls
* tests
* read permission check
* updated openapi definitions
* i18n
* GetConnectionByAudience method
* notes
* replaced GetConnectionsByAudience with a filter
* added custom oauth token object
* updated interface and usage
* properly set enterprise interface
* move retrieval logic to impl
* webhook tests
* translations
* i18n: updates
* address comments
* endpoint and tests
* i18n
* api docs
* fixed endpoint path
* sq.like
* use filter object instead of parameters
* set url values if not empty
* typos
* converted some components to function components, and move around files
* correctly check token url
* restore flag to previous value
* added command oauth handler
* update enterprise imports
* migrate last component to function component
* Added enterprise import
* refactor permissions and add necessary webapp code
* Check correct flag in permission tree
* allow partial updates
* sort i18n webapp
* missing test modification
* fixed webapp i18n sorting
* allow validating stored connections
* added missing translation
* fix finished adding connection link and text on result page
* added missing permission to smoke tests
* missing role in smoke test
* updated translations
* updated translations
* support editing client secret on existing connection
* fix some i18n strings
* updated translations
* better error messages
* progress on using react select for command request url while maintaining typed in value
* remove writeheader, test
* HasValidGrantType
* end early to avoid nil pointer errors
* move slash command request url input box into its own component
* wrap components related to oauth connections in config check
* fix tests
* i18n-extract
* change some i18n strings to say "Outgoing OAuth 2.0 Connections"
* remove debug code
* fixed i18n
* updated i18n file
* feature configuration backend
* typo
* add system console setting
* Revert "typo"
This reverts commit 669da23e8e.
* Revert "updated i18n file"
This reverts commit d0882c0dd7.
* Revert "fixed i18n"
This reverts commit 3108866bc1.
* fixed i18n
* updated i18n file
* typo
* updated i18n
* updated i18n
* updated i18n
* updated version to 9.6
* replace feature flag with system console configuration
* i18n
* updated tests
* pr feedback
* fix styling of disabled text box
* fix styling of action links in integration console
* server changes for validation feature
* webapp changes for validation feature
* pencil icon styling
* styling fixes for oauth audience correct configuration message
* fix sanitize test
* remove max lengths from outgoing oauth connection form
* use config var in webapp instead of feature flag
* change asterisks to bullets
* update api docs for validate endpoint
* feedback from ux review
* fix lint, types, tests
* fix stylelint
* implement validation button under the token url input
* support wildcard for matching audience urls
* updates for styling
* update snapshots
* add doc links for the outgoing oauth connections feature
* change doc links to use permalink
* add docs link to system console
* fix: use limitedreader in json decoding
* fix: form error in validation
* management permission can read now
* updated api documentation
* doc typo
* require one permission to read only
* fix api connection list audience filter
* fix audience matching and add loading indicator
* fix team permissions on outgoing oauth connection api calls
* fix api doc and test, for adding team id to query params
* handle read permissions by adding a team in the payload
* missing teamid query parameter in test
* change validate button logic to not require audience urls to be filled out
* fix redux type
---------
Co-authored-by: Felipe Martin <me@fmartingr.com>
* Initial comit for ip filtering service implementation
* Add audit logs for IP Filters
* start of webapp work
* Stashing
* Updates based on Agniva's feedback around service vs einterface
* Updates completed
* Commit before refactoring, everything's working
* First pass of cleanup complete, front-end tests added
* actually add files
* Updates to some translation strings, running i18n-extract
* Lock everything behind a feature flag
* Fix tests, try to fix some linter stuff
* Fixed linter for JS, on to scss
* Fixed linter for scss
* Fix linter
* More fixes for pipeline
* Support for IPV6
* Remove tsx file that was removed in masteR
* Revert package.json and package-lock.json to master, add cidr-regex dep into channels/package.json
* Another commit to force fix Github
* Fixes around IPV6. Some suggestions from Matt re: UX review. Fixing pipelines for tests and types on new cidr-regex package
* Changes to address Matt's feedback
* A few more changes for clean up
* Add support for permissions
* Fix vet for OpenAPI spec
* Actually add the yaml file for openapi
* Add permission migration to allow support for IP Filtering
* Fix tests
* Final fixes from Matt
* Remove cancel button from page, update link outs to documentation
* Update test to account for removed cancel button
* Adjustments based on feedback from Harrison
* More fixes from PR feedback
* Add a t to fix translations that doesn't seem to be breaking anyone else?
* More fix
* updates for PR feedback
* Fix linter
* Fix types
* Now fix the linter again
* Add back tests because Harrison was able to get them running
* Adjustments for PR feedback
* Remove admin_definition.jsx
* Fix linter
* [CLD-6453] IP Filtering notification email for sysadmins (#25224)
* Initial commit for IP filtering alert email
* Updates to style for email, addition of ip_filtering email:
* Fix pipelines
* Adjustments from Matt's feedback
* Padding changes
* template diff (#25249)
Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>
* Fix hardcoded true, remove bool return value
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>
* Lock feature behind enterprise license. Drop cidr-regex in favour of ipaddr.js dependency. Refactor isIpAddressWithinRanges to use ipaddr.js
* Add a couple server tests
* fix linter
* Fix types from merge conflicts
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>
* MM-54640 Add API to get multiple emojis by name at once
* Fix status code when too many names are requested
* Address feedback
* Update unit tests
* Fix styling
* Fix more styling
* Fix mismatched i18n id
* [MM-53428] Delete empty drafts on upsert
* Add migrations to fix existing drafts
* Fix CI
* Delete empty drafts entirely from the DB
* Fix lint
* Implement batch migration for deleting drafts
* Missing store layers
* Add updated mock
* Remove unnecessary test
* PR feedback
* Add check for cluster migration
* Fix MySQL
* Don't check for len<2
* Bit of PR feedback
* Use query builder for parameters
* PR feedback
* More PR feedback
* Merge'd
* unit test GetLastCreateAtAndUserIdValuesForEmptyDraftsMigration
* simplified builder interface
* fix DeleteEmptyDraftsByCreateAtAndUserId for MySQL
* rework as batch migration worker
* fix typo
* log ip address on version mismatches too
* simplify reset semantics
* remove trace log in favour of low spam
* document parameters for clarity
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
If the store fails to initialize (e.g. run a migration), it would `log.Fatal` and then `os.Exit`. Unfortunately, this trips up `TestMain`, which happily keeps running tests, now guaranteed to fail.
Avoid this by instead returning an error from the store initialization, handling appropriately at the layer above.
It was a good decision in hindsight to keep the public module as 0.x
because this would have been a breaking change again.
https://mattermost.atlassian.net/browse/MM-53032
```release-note
Changed the Go module path from github.com/mattermost/mattermost-server/server/v8 to github.com/mattermost/mattermost/server/v8.
For the public facing module, it's path is also changed from github.com/mattermost/mattermost-server/server/public to github.com/mattermost/mattermost/server/public
```
