Commit graph

133 commits

Author SHA1 Message Date
Jesse Hallam
6ef73af2cc
MM-65960: Avoid replica race lag when accessing TelemetryID (#34586)
* avoid replica race lag when remembering ServerID

In an HA environment, with a master and read replica, querying the server id from the store runs the risk of returning a value saved to master but not yet replicated. Avoid this by using the telemetry service value directly when available.

Fixes: MM-65960

* Add Get(ByName)WithContext

* explicitly use master for ServerId

* mock GetByNameWithContext

* more mocking

* more mocks
2025-12-11 17:34:36 -04:00
Ibrahim Serdar Acikgoz
084006c0ea
[MM-61758] Burn on read feature (#34703)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Blocked by required conditions
Web App CI / check-types (push) Blocked by required conditions
Web App CI / test (platform) (push) Blocked by required conditions
Web App CI / test (mattermost-redux) (push) Blocked by required conditions
Web App CI / test (channels shard 1/4) (push) Blocked by required conditions
Web App CI / test (channels shard 2/4) (push) Blocked by required conditions
Web App CI / test (channels shard 3/4) (push) Blocked by required conditions
Web App CI / test (channels shard 4/4) (push) Blocked by required conditions
Web App CI / upload-coverage (push) Blocked by required conditions
Web App CI / build (push) Blocked by required conditions
* Add read receipt store for burn on read message types

* update mocks

* fix invalidation target

* have consistent case on index creation

* Add temporary posts table

* add mock

* add transaction support

* reflect review comments

* wip: Add reveal endpoint

* user check error id instead

* wip: Add ws events and cleanup for burn on read posts

* add burn endpoint for explicitly burning messages

* add translations

* Added logic to associate files of BoR post with the post

* Added test

* fixes

* disable pinning posts and review comments

* MM-66594 - Burn on read UI integration (#34647)

* MM-66244 - add BoR visual components to message editor

* MM-66246 - BoR visual indicator for sender and receiver

* MM-66607 - bor - add timer countdown and autodeletion

* add the system console max time to live config

* use the max expire at and create global scheduler to register bor messages

* use seconds for BoR config values in BE

* implement the read by text shown in the tooltip logic

* unestack the posts from same receiver and BoR  and fix styling

* avoid opening reply RHS

* remove unused dispatchers

* persis the BoR label in the drafts

* move expiration value to metadata

* adjust unit tests to metadata insted of props

* code clean up and some performance improvements; add period grace for deletion too

* adjust migration serie number

* hide bor messages when config is off

* performance improvements on post component and code clean up

* keep bor existing post functionality if config is disabled

* Add read receipt store for burn on read message types

* Add temporary posts table

* add transaction support

* reflect review comments

* wip: Add reveal endpoint

* user check error id instead

* wip: Add ws events and cleanup for burn on read posts

* avoid reacting to unrevealed bor messages

* adjust migration number

* Add read receipt store for burn on read message types

* have consistent case on index creation

* Add temporary posts table

* add mock

* add transaction support

* reflect review comments

* wip: Add reveal endpoint

* user check error id instead

* wip: Add ws events and cleanup for burn on read posts

* add burn endpoint for explicitly burning messages

* adjust post reveal and type with backend changes

* use real config values, adjust icon usage and style

* adjust the delete from from sender and receiver

* improve self deleting logic by placing in badge, use burn endpoint

* adjust websocket events handling for the read by sender label information

* adjust styling for concealed and error state

* update burn-on-read post event handling for improved recipient tracking and multi-device sync

* replace burn_on_read with type in database migrations and model

* remove burn_on_read metadata from PostMetadata and related structures

* Added logic to associate files of BoR post with the post

* Added test

* adjust migration name and fix linter

* Add read receipt store for burn on read message types

* update mocks

* have consistent case on index creation

* Add temporary posts table

* add mock

* add transaction support

* reflect review comments

* wip: Add reveal endpoint

* user check error id instead

* wip: Add ws events and cleanup for burn on read posts

* add burn endpoint for explicitly burning messages

* Added logic to associate files of BoR post with the post

* Added test

* disable pinning posts and review comments

* show attachment on bor reveal

* remove unused translation

* Enhance burn-on-read post handling and refine previous post ID retrieval logic

* adjust the returning chunk to work with bor messages

* read temp post from master db

* read from master

* show the copy link button to the sender

* revert unnecessary check

* restore correct json tag

* remove unused error handling  and clarify burn-on-read comment

* improve type safety and use proper selectors

* eliminate code duplication in deletion handler

* optimize performance and add documentation

* delete bor message for sender once all receivers reveal it

* add burn on read to scheduled posts

* add feature enable check

* use master to avoid  all read recipients race condition

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>

* squash migrations into single file

* add configuration for the scheduler

* don't run messagehasbeenposted hook

* remove parallel tests on burn on read

* add clean up for closing opened modals from previous tests

* simplify delete menu item rendering

* add cleanup step to close open modals after each test to prevent pollution

* streamline delete button visibility logic for Burn on Read posts

* improve reliability of closing post menu and modals by using body ESC key

---------

Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>
Co-authored-by: Pablo Vélez <pablovv2012@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-12-11 07:59:50 +01:00
Elias Nahum
1022cd44c0
MM-65756 Database Migrations, Indexes and Methods for Auto-Translation (#34047)
Some checks failed
API / build (push) Has been cancelled
Server CI / Compute Go Version (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / test (push) Has been cancelled
Web App CI / build (push) Has been cancelled
Server CI / Check mocks (push) Has been cancelled
Server CI / Check go mod tidy (push) Has been cancelled
Server CI / check-style (push) Has been cancelled
Server CI / Check serialization methods for hot structs (push) Has been cancelled
Server CI / Vet API (push) Has been cancelled
Server CI / Check migration files (push) Has been cancelled
Server CI / Generate email templates (push) Has been cancelled
Server CI / Check store layers (push) Has been cancelled
Server CI / Check mmctl docs (push) Has been cancelled
Server CI / Postgres with binary parameters (push) Has been cancelled
Server CI / Postgres (push) Has been cancelled
Server CI / Postgres (FIPS) (push) Has been cancelled
Server CI / Generate Test Coverage (push) Has been cancelled
Server CI / Run mmctl tests (push) Has been cancelled
Server CI / Run mmctl tests (FIPS) (push) Has been cancelled
Server CI / Build mattermost server app (push) Has been cancelled
* AutoTranslate config settings

* comment out Agents provider

* Add auto translate timeout config validation

* i18n messages for autotranslation config validation

* fix test

* validate url for libreTranslate

* Feedback review

* Admin Console UI for Auto-Translation

* fix admin console conditional section display

* i18n

* removed unintentional change

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* update admin.general.localization.autoTranslateProviderDescription newline

* fix lint

* Fix types

* UX feedback review

* fix typo in i18n

* Fix AutoTranslation feature flag

* feedback review

* Fix test default values

* feedback review

* re-add isHidden property to feature discovery

* Database Migrations, Indexes and Methods for Auto-Translation

* i18n

* fix retrylayer and storetest

* Fix search query

* fix lint

* remove the request.CTX and modify Translation model

* fix lint and external url

* Add settings to playwright

* Add empty as a valid value for the Provider

* Update jsonb queries

* Fix queries and add model methods

* fix go lint

* go lint fix 2

* fix db migrations

* feedback review + store cache

* increase migration number

* cleanup autotranslation store cache

* use NULL as objectType for posts

* fix bad merge

* fix tests

* add missing i18n

* Switch prop bags column to boolean

* fix lint

* fix tests

* Remove database search

* use Builder methods

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: BenCookie95 <benkcooke@gmail.com>
2025-11-22 09:32:01 +08:00
Rahim Rahman
edb05c7ea5
Magic link (passwordless) authentication for guests (#34264)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Waiting to run
Web App CI / check-types (push) Waiting to run
Web App CI / test (push) Waiting to run
Web App CI / build (push) Waiting to run
* Add EasyLogin configuration (#34217)

* add easy login config

* add easy login to the invite modal

* add to the query parameters

* Add an API to get login method for the login id (#34223)

* add an api to get login method for the login id

* do not return errors if user is not found

* Add support for Easy Login invitation link sending (#34224)

This generates Easy Login token types when requested. The server
doesn't do anything with these tokens, yet - that will come in a
future change.

* Add support for logging in with easy login (#34236)

* Fix E2E tests (#34240)

* Prevent easy login accounts to reset their password (#34262)

* Add easy login support to login api and limit token to 5 min (#34259)

* webapp easy login ui mods (#34237)

* webapp easy login ui mods
* easy login i18n
* lint issues
* getUserLoginType
* using the real API
* easylogin proper redirect
* remove unneeded functions and files
* duplicated localization
* remove easylogin
* using EnableEasyLogin setting
* localization fix
* fix lint issue
* remove excessive setIsWaiting
* changed logic to make it more readable
* renaming component to make easier editable
* password will disappear when username change
* login test
* text for easy login password

* Add app links to emails

* Update templates and always land in the landing screen

* Update svg image, improve checks on server, fix linking page and show deactivated on login type

* Update naming

* Fix mocks and imports

* Remove all sessions on disable and forbid user promotion

* Fix layer and tests

* Address feedback

* Fix tests

* Fix missing string

* Fix texts

* Fix tests

* Fix constant name

* Fix tests

* Fix test

* Address feedback

* Fix lint

* Fix test

* Address feedback

* Fix test

---------

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: David Krauser <david@krauser.org>
Co-authored-by: Daniel Espino <larkox@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-20 14:06:23 +01:00
Carlos Garcia
4ba7f7e16e
MM-66202: Migrate to aws-sdk-go-v2 (#34496)
* updated aws-sdk dependency to aws-sdk-go-v2

* simplify error handling in case of timeout errors

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-20 11:26:09 +01:00
Ibrahim Serdar Acikgoz
fc93ede640
[MM-65956] Tweak auto add to make it consistent with child policies (#33990)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Waiting to run
Web App CI / check-types (push) Waiting to run
Web App CI / test (push) Waiting to run
Web App CI / build (push) Waiting to run
2025-11-19 20:18:45 +00:00
Harshil Sharma
e8406345a5
Content flagging file downloads (#34480)
* Server change donw

* webapp changes

* Disabled file actions

* lint fixes

* Removed leftover comment

* CI

* Added tests

* lint fixes

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-19 14:22:07 +05:30
Ben Schumacher
549be3d2b1
Remove context.Context from Store (#34413)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Waiting to run
Web App CI / check-types (push) Waiting to run
Web App CI / test (push) Waiting to run
Web App CI / build (push) Waiting to run
2025-11-07 08:41:43 +01:00
Harshil Sharma
519fb5faf0
Content flagging thread fixes (#34162)
* created new API to fetch flagged posts

* lint fix

* Added new client methods

* test: add comprehensive tests for content flagging APIs

* Added new API tests

* fixed openapi spec

* Fixed DataSpillageReport tests

* Fixed PostMarkdown test

* Fixed PostPreviewPropertyRenderer test

* Added metadata to card renderer

* test fixes

* Added no comment placeholder

* Added view detail button

* Created RemoveFlaggedMessageConfirmationModal modal

* Added key and remove flag request modal

* IMplemented delete flagged post

* Handled edge cases of deleting flagged post

* keep message

* UI integration

* Added WS event for post report update and handled deleted files of flagged post

* Added error handling in keep/remove forms

* i18n fixes

* Fixed test

* Updated OpenAPI specs

* fixed types

* fixed types

* refactoring

* refactor: improve test mocking for data spillage report component

* test mock updates

* Fixed tests

* Updated reducer

* not resetting mocks

* Added migrations for content flagging tables

* Created new structure

* review fixes

* Used correct ot name

* WIP

* review fixes

* review fixes

* Added new property translations

* CI

* CI

* CI

* Improved test

* fixed test

* CI

* New UI component

* WIP

* Updated settings APIs

* cached DB data

* used cached reviewer data

* Updated tests

* Lint fixes

* test: add tests for saveContentFlaggingSettings and getContentFlaggingSettings APIs

* test fix

* test: add tests for SaveContentFlaggingConfig and GetContentFlaggingConfigReviewerIDs

* Updated tests

* test: add content flagging test for local cache layer

* test: add comprehensive tests for content flagging store cache

* Updated tests

* lint fix

* Updated mobile text

* Added content flagging SQL store mocks

* Added API specs for new APIs

* fixed tests

* feat: add TestContentFlaggingStore function for content flagging store testing

* feat: add comprehensive tests for content flagging store

* Added SQL store tests

* test: add content flagging test for local cache layer

* test: add tests for content flagging store caching

* Added cache layer tests

* Updated tests

* Fixed

* Handled JSON error

* fixes

* fixes

* Fixed retry layer test

* fixerdf i18n

* Fixed test

* CI

* building index concurrently

* CI

* fixed a test

* CI

* cleanup

* Implemented reviewer search API

* feat: add tests for SearchCommonContentFlaggingReviewers and SearchTeamContentFlaggingReviewers

* Added store tests

* test: add comprehensive tests for SearchReviewers function

* feat: add comprehensive tests for searchReviewers endpoint

* API tests

* Integrate flag post api (#33798)

* WIP

* WIP

* Added API call

* test: add test for Client4.flagPost API call in FlagPostModal

* fix: remove userEvent.setup() from flag post modal test

* test: wrap submit button click in act for proper state updates

* Updated tests

* lint fix

* CI

* Updated to allow special characters in comments

* Handled empty comment

* Used finally

* CI

* Fixed test

* Spillage card integration (#33832)

* Created getContentFlaggingFields API

* created getPostPropertyValues API

* WIP

* Created useContentFlaggingFields hook

* WIP

* WIP

* Added option to retain data for reviewers

* Displayed deleted post's preview

* DIsplayed all properties

* Adding field name i18n

* WIP - managing i18n able texts

* Finished displaying all fields

* Manual cleanup

* lint fixes

* team role filter logic fix

* Fixed tests

* created new API to fetch flagged posts

* lint fix

* Added new client methods

* test: add comprehensive tests for content flagging APIs

* Added new API tests

* fixed openapi spec

* Fixed DataSpillageReport tests

* Fixed PostMarkdown test

* Fixed PostPreviewPropertyRenderer test

* Added metadata to card renderer

* test fixes

* Added no comment placeholder

* Fixed test

* refactor: improve test mocking for data spillage report component

* test mock updates

* Updated reducer

* not resetting mocks

* WIP

* review fixes

* CI

* Fixed

* fixes

* Content flagging actions implementation (#33852)

* Added view detail button

* Created RemoveFlaggedMessageConfirmationModal modal

* Added key and remove flag request modal

* IMplemented delete flagged post

* Handled edge cases of deleting flagged post

* keep message

* UI integration

* Added WS event for post report update and handled deleted files of flagged post

* Added error handling in keep/remove forms

* i18n fixes

* Updated OpenAPI specs

* fixed types

* fixed types

* refactoring

* Fixed tests

* review fixes

* Added new property translations

* Improved test

* fixed test

* CI

* fixes

* CI

* fixed a test

* fixed  abad commit

* CI

* WIP

* IMplemented assign reviewer API

* Display reviewers

* Review fixes

* UI integration

* lint fix

* Added API docs

* test: add comprehensive tests for assignFlaggedPostReviewer function

* test: add comprehensive tests for AssignFlaggedPostReviewer

* Added tests

* Fixed test

* Sequential tests

* minor improvemenmts

* WIP

* Added keep/delete message notifications

* refactor: update AssignFlaggedPostReviewer method signature to include context

* test: add tests for getReviewerPostsForFlaggedPost and postReviewerMessage

* lint fixes

* handled reviewer updates

* Handled preference

* Implemented notifications

* test: add comprehensive tests for content flagging notification functions

* refactor: Replace th.UpdateConfig with SaveContentFlaggingConfig in tests

* test: add test case for content flagging with string comparison

* refactor: simplify content flagging test config setup

* refactor: Update content flagging notification settings types in test cases

* refactor: Update content flagging tests to use exact message matching

* Added tests

* lint fixes

* Added new hooks

* lint fixes

* feat: add API specs for getPostChannel and getPostTeam endpoints

* lint fixes

* test: add tests for getPostChannel and getPostTeam APIs

* Added API tests

* test: add empty test files for property card view loaders

* test: add comprehensive tests for property card view hooks

* refactor: replace waitForNextUpdate with waitFor in test files

* Added hook tests

* fixed test

* review fixes

* Fixed a test

* Fixed a test

* Fixed for default state

* lint fixes

* migration update

* review fixes

* Reduced code duplication

* Refactored tests to reduce duplication

* review fixes

* lint fix

* WIP

* Updated existing APIs instead of creating new API

* Lint fix

* Added new tests

* Fixed a test

* Review fixes

* WIP

* test: add comprehensive tests for sendFlaggedPostRemovalNotification and sendKeepFlaggedPostNotification

* Updated tests

* review fixes

* review fixes

* test update

* fixed a test

* Updated logs

* i18n fixes

* Restore replies when restoring root post

* Finalized the function

* Fixed threads issue

* Removed unused functions

* fixed a test

* Refactored to use properties for replies

* Updated test

* lint fix

* Test fix

* reverted unintentional refactoring

* removed a query change that is no longer used
2025-11-04 13:52:33 +05:30
Jesse Hallam
acda1fb5dd
MM-66299: type handling for ConsumeTokenOnce (#34247)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Waiting to run
Web App CI / check-types (push) Waiting to run
Web App CI / test (push) Waiting to run
Web App CI / build (push) Waiting to run
2025-10-22 18:03:33 -03:00
Harshil Sharma
79756ae1e1
Reviewer search api (#34036)
* Added another property field

* WIP

* WIP

* Added validations

* Added data validations and hidden post if confifgured to

* lint fixes

* Added API spec

* Added some tests

* Added tests for getContentReviewBot

* test: add comprehensive tests for getContentReviewChannels function

* Added more app layer tests

* Added TestCanFlagPost

* test: Add comprehensive tests for FlagPost function

* Added all app layer tests

* Removed a file that was reamoved downstream

* test: add content flagging test file

* test: add comprehensive tests for FlagContentRequest.IsValid method

* Added model tests

* test: add comprehensive tests for SqlPropertyValueStore.CreateMany

* test: add comprehensive tests for flagPost() API function

* Added API tests

* linter fix

* WIP

* sent post flagging confirmation message

* fixed i18n nissues

* fixed i18n nissues

* CI

* WIP

* WIP

* Added API call

* test: add test for Client4.flagPost API call in FlagPostModal

* fix: remove userEvent.setup() from flag post modal test

* test: wrap submit button click in act for proper state updates

* Updated tests

* lint fix

* Updated test

* fix: reset contentFlaggingGroupId for test isolation in content flagging tests

* removed cached group ID

* removed debug log

* CI

* Updated to allow special characters in comments

* Handled empty comment

* Created getContentFlaggingFields API

* created getPostPropertyValues API

* Used finally

* WIP

* Created useContentFlaggingFields hook

* WIP

* WIP

* Added option to retain data for reviewers

* Displayed deleted post's preview

* DIsplayed all properties

* Adding field name i18n

* WIP - managing i18n able texts

* Finished displaying all fields

* Manual cleanup

* lint fixes

* team role filter logic fix

* Fixed tests

* created new API to fetch flagged posts

* lint fix

* Added new client methods

* test: add comprehensive tests for content flagging APIs

* Added new API tests

* fixed openapi spec

* Fixed DataSpillageReport tests

* Fixed PostMarkdown test

* Fixed PostPreviewPropertyRenderer test

* Added metadata to card renderer

* test fixes

* Added no comment placeholder

* Added view detail button

* Created RemoveFlaggedMessageConfirmationModal modal

* Added key and remove flag request modal

* IMplemented delete flagged post

* Handled edge cases of deleting flagged post

* keep message

* UI integration

* Added WS event for post report update and handled deleted files of flagged post

* Added error handling in keep/remove forms

* i18n fixes

* Fixed test

* Updated OpenAPI specs

* fixed types

* fixed types

* refactoring

* refactor: improve test mocking for data spillage report component

* test mock updates

* Fixed tests

* Updated reducer

* not resetting mocks

* Added migrations for content flagging tables

* Created new structure

* review fixes

* Used correct ot name

* WIP

* review fixes

* review fixes

* Added new property translations

* CI

* CI

* CI

* Improved test

* fixed test

* CI

* New UI component

* WIP

* Updated settings APIs

* cached DB data

* used cached reviewer data

* Updated tests

* Lint fixes

* test: add tests for saveContentFlaggingSettings and getContentFlaggingSettings APIs

* test fix

* test: add tests for SaveContentFlaggingConfig and GetContentFlaggingConfigReviewerIDs

* Updated tests

* test: add content flagging test for local cache layer

* test: add comprehensive tests for content flagging store cache

* Updated tests

* lint fix

* Updated mobile text

* Added content flagging SQL store mocks

* Added API specs for new APIs

* fixed tests

* feat: add TestContentFlaggingStore function for content flagging store testing

* feat: add comprehensive tests for content flagging store

* Added SQL store tests

* test: add content flagging test for local cache layer

* test: add tests for content flagging store caching

* Added cache layer tests

* Updated tests

* Fixed

* Handled JSON error

* fixes

* fixes

* Fixed retry layer test

* fixerdf i18n

* Fixed test

* CI

* building index concurrently

* CI

* fixed a test

* CI

* cleanup

* Implemented reviewer search API

* feat: add tests for SearchCommonContentFlaggingReviewers and SearchTeamContentFlaggingReviewers

* Added store tests

* test: add comprehensive tests for SearchReviewers function

* feat: add comprehensive tests for searchReviewers endpoint

* API tests

* Integrate flag post api (#33798)

* WIP

* WIP

* Added API call

* test: add test for Client4.flagPost API call in FlagPostModal

* fix: remove userEvent.setup() from flag post modal test

* test: wrap submit button click in act for proper state updates

* Updated tests

* lint fix

* CI

* Updated to allow special characters in comments

* Handled empty comment

* Used finally

* CI

* Fixed test

* Spillage card integration (#33832)

* Created getContentFlaggingFields API

* created getPostPropertyValues API

* WIP

* Created useContentFlaggingFields hook

* WIP

* WIP

* Added option to retain data for reviewers

* Displayed deleted post's preview

* DIsplayed all properties

* Adding field name i18n

* WIP - managing i18n able texts

* Finished displaying all fields

* Manual cleanup

* lint fixes

* team role filter logic fix

* Fixed tests

* created new API to fetch flagged posts

* lint fix

* Added new client methods

* test: add comprehensive tests for content flagging APIs

* Added new API tests

* fixed openapi spec

* Fixed DataSpillageReport tests

* Fixed PostMarkdown test

* Fixed PostPreviewPropertyRenderer test

* Added metadata to card renderer

* test fixes

* Added no comment placeholder

* Fixed test

* refactor: improve test mocking for data spillage report component

* test mock updates

* Updated reducer

* not resetting mocks

* WIP

* review fixes

* CI

* Fixed

* fixes

* Content flagging actions implementation (#33852)

* Added view detail button

* Created RemoveFlaggedMessageConfirmationModal modal

* Added key and remove flag request modal

* IMplemented delete flagged post

* Handled edge cases of deleting flagged post

* keep message

* UI integration

* Added WS event for post report update and handled deleted files of flagged post

* Added error handling in keep/remove forms

* i18n fixes

* Updated OpenAPI specs

* fixed types

* fixed types

* refactoring

* Fixed tests

* review fixes

* Added new property translations

* Improved test

* fixed test

* CI

* fixes

* CI

* fixed a test

* fixed  abad commit

* CI

* WIP

* IMplemented assign reviewer API

* Display reviewers

* Review fixes

* UI integration

* lint fix

* Added API docs

* test: add comprehensive tests for assignFlaggedPostReviewer function

* test: add comprehensive tests for AssignFlaggedPostReviewer

* Added tests

* Fixed test

* Sequential tests

* minor improvemenmts

* WIP

* Added keep/delete message notifications

* refactor: update AssignFlaggedPostReviewer method signature to include context

* test: add tests for getReviewerPostsForFlaggedPost and postReviewerMessage

* lint fixes

* handled reviewer updates

* Handled preference

* review fixes

* Review fixes
2025-10-14 09:06:23 +05:30
Harshil Sharma
3265054ad5
Migrate content flagging settings to database (#33989)
Some checks failed
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Waiting to run
Web App CI / check-types (push) Waiting to run
Web App CI / test (push) Waiting to run
Web App CI / build (push) Waiting to run
Migration-assist Sync / Check if migration-assist have been synced (push) Has been cancelled
* lint fix

* CI

* added new migration mocks

* Used setup for tests

* some comment

* Removed unnecesseery nil check

* Form validation

* WIP tests

* WIP tests

* WIP tests

* fix: mock content flagging config selector with correct reasons format

Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>

* fix: add mock for getContentFlaggingConfig in flag post modal test

Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>

* Updated error code order in API docs

* removed empty files

* Added tests

* lint fixes

* minor tweak

* lint fix

* type fix

* fixed test

* nit

* test enhancements

* API WIP

* API WIP

* creating values

* creating content flagging channel and properties

* Able to save properties

* Added another property field

* WIP

* WIP

* Added validations

* Added data validations and hidden post if confifgured to

* lint fixes

* Added API spec

* Added some tests

* Added tests for getContentReviewBot

* test: add comprehensive tests for getContentReviewChannels function

* Added more app layer tests

* Added TestCanFlagPost

* test: Add comprehensive tests for FlagPost function

* Added all app layer tests

* Removed a file that was reamoved downstream

* test: add content flagging test file

* test: add comprehensive tests for FlagContentRequest.IsValid method

* Added model tests

* test: add comprehensive tests for SqlPropertyValueStore.CreateMany

* test: add comprehensive tests for flagPost() API function

* Added API tests

* linter fix

* WIP

* sent post flagging confirmation message

* fixed i18n nissues

* fixed i18n nissues

* CI

* WIP

* WIP

* Added API call

* test: add test for Client4.flagPost API call in FlagPostModal

* fix: remove userEvent.setup() from flag post modal test

* test: wrap submit button click in act for proper state updates

* Updated tests

* lint fix

* Updated test

* fix: reset contentFlaggingGroupId for test isolation in content flagging tests

* removed cached group ID

* removed debug log

* CI

* Updated to allow special characters in comments

* Handled empty comment

* Created getContentFlaggingFields API

* created getPostPropertyValues API

* Used finally

* WIP

* Created useContentFlaggingFields hook

* WIP

* WIP

* Added option to retain data for reviewers

* Displayed deleted post's preview

* DIsplayed all properties

* Adding field name i18n

* WIP - managing i18n able texts

* Finished displaying all fields

* Manual cleanup

* lint fixes

* team role filter logic fix

* Fixed tests

* created new API to fetch flagged posts

* lint fix

* Added new client methods

* test: add comprehensive tests for content flagging APIs

* Added new API tests

* fixed openapi spec

* Fixed DataSpillageReport tests

* Fixed PostMarkdown test

* Fixed PostPreviewPropertyRenderer test

* Added metadata to card renderer

* test fixes

* Added no comment placeholder

* Added view detail button

* Created RemoveFlaggedMessageConfirmationModal modal

* Added key and remove flag request modal

* IMplemented delete flagged post

* Handled edge cases of deleting flagged post

* keep message

* UI integration

* Added WS event for post report update and handled deleted files of flagged post

* Added error handling in keep/remove forms

* i18n fixes

* Fixed test

* Updated OpenAPI specs

* fixed types

* fixed types

* refactoring

* refactor: improve test mocking for data spillage report component

* test mock updates

* Fixed tests

* Updated reducer

* not resetting mocks

* Added migrations for content flagging tables

* Created new structure

* review fixes

* Used correct ot name

* WIP

* review fixes

* review fixes

* Added new property translations

* CI

* CI

* CI

* Improved test

* fixed test

* CI

* New UI component

* WIP

* Updated settings APIs

* cached DB data

* used cached reviewer data

* Updated tests

* Lint fixes

* test: add tests for saveContentFlaggingSettings and getContentFlaggingSettings APIs

* test fix

* test: add tests for SaveContentFlaggingConfig and GetContentFlaggingConfigReviewerIDs

* Updated tests

* test: add content flagging test for local cache layer

* test: add comprehensive tests for content flagging store cache

* Updated tests

* lint fix

* Updated mobile text

* Added content flagging SQL store mocks

* Added API specs for new APIs

* fixed tests

* feat: add TestContentFlaggingStore function for content flagging store testing

* feat: add comprehensive tests for content flagging store

* Added SQL store tests

* test: add content flagging test for local cache layer

* test: add tests for content flagging store caching

* Added cache layer tests

* Updated tests

* Fixed

* Handled JSON error

* fixes

* fixes

* Fixed retry layer test

* fixerdf i18n

* Fixed test

* CI

* building index concurrently

* CI

* fixed a test

* CI

* cleanup

* Integrate flag post api (#33798)

* WIP

* WIP

* Added API call

* test: add test for Client4.flagPost API call in FlagPostModal

* fix: remove userEvent.setup() from flag post modal test

* test: wrap submit button click in act for proper state updates

* Updated tests

* lint fix

* CI

* Updated to allow special characters in comments

* Handled empty comment

* Used finally

* CI

* Fixed test

* Spillage card integration (#33832)

* Created getContentFlaggingFields API

* created getPostPropertyValues API

* WIP

* Created useContentFlaggingFields hook

* WIP

* WIP

* Added option to retain data for reviewers

* Displayed deleted post's preview

* DIsplayed all properties

* Adding field name i18n

* WIP - managing i18n able texts

* Finished displaying all fields

* Manual cleanup

* lint fixes

* team role filter logic fix

* Fixed tests

* created new API to fetch flagged posts

* lint fix

* Added new client methods

* test: add comprehensive tests for content flagging APIs

* Added new API tests

* fixed openapi spec

* Fixed DataSpillageReport tests

* Fixed PostMarkdown test

* Fixed PostPreviewPropertyRenderer test

* Added metadata to card renderer

* test fixes

* Added no comment placeholder

* Fixed test

* refactor: improve test mocking for data spillage report component

* test mock updates

* Updated reducer

* not resetting mocks

* WIP

* review fixes

* CI

* Fixed

* fixes

* Content flagging actions implementation (#33852)

* Added view detail button

* Created RemoveFlaggedMessageConfirmationModal modal

* Added key and remove flag request modal

* IMplemented delete flagged post

* Handled edge cases of deleting flagged post

* keep message

* UI integration

* Added WS event for post report update and handled deleted files of flagged post

* Added error handling in keep/remove forms

* i18n fixes

* Updated OpenAPI specs

* fixed types

* fixed types

* refactoring

* Fixed tests

* review fixes

* Added new property translations

* Improved test

* fixed test

* CI

* fixes

* CI

* fixed a test

* CI

* Review fixes

---------

Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>
2025-10-13 12:24:01 +05:30
Harshil Sharma
c21ef29f02
Flag post API (#33765)
* Added enable/disable setting and feature flag

* added rest of notifgication settings

* Added backend for content flagging setting and populated notification values from server side defaults

* WIP user selector

* Added common reviewers UI

* Added additonal reviewers section

* WIP

* WIP

* Team table base

* Added search in teams

* Added search in teams

* Added additional settings section

* WIP

* Inbtegrated reviewers settings

* WIP

* WIP

* Added server side validation

* cleanup

* cleanup

* [skip ci]

* Some refactoring

* type fixes

* lint fix

* test: add content flagging settings test file

* test: add comprehensive unit tests for content flagging settings

* enhanced tests

* test: add test file for content flagging additional settings

* test: add comprehensive unit tests for ContentFlaggingAdditionalSettingsSection

* Added additoonal settings test

* test: add empty test file for team reviewers section

* test: add comprehensive unit tests for TeamReviewersSection component

* test: update tests to handle async data fetching in team reviewers section

* test: add empty test file for content reviewers component

* feat: add comprehensive unit tests for ContentFlaggingContentReviewers component

* Added ContentFlaggingContentReviewersContentFlaggingContentReviewers test

* test: add notification settings test file for content flagging

* test: add comprehensive unit tests for content flagging notification settings

* Added ContentFlaggingNotificationSettingsSection tests

* test: add user profile pill test file

* test: add comprehensive unit tests for UserProfilePill component

* refactor: Replace enzyme shallow with renderWithContext in user_profile_pill tests

* Added UserProfilePill tests

* test: add empty test file for content reviewers team option

* test: add comprehensive unit tests for TeamOptionComponent

* Added TeamOptionComponent tests

* test: add empty test file for reason_option component

* test: add comprehensive unit tests for ReasonOption component

* Added ReasonOption tests

* cleanup

* Fixed i18n error

* fixed e2e test lijnt issues

* Updated test cases

* Added snaoshot

* Updated snaoshot

* lint fix

* WIP

* lint fix

* Added post flagging properties setup

* review fixes

* updated snapshot

* CI

* Added base APIs

* Fetched team status data on load and team switch

* WIP

* Review fixes

* wip

* WIP

* Removed an test, updated comment

* CI

* Added tests

* Added tests

* Lint fix

* Added API specs

* Fixed types

* CI fixes

* API tests

* lint fixes

* Set env variable so API routes are regiustered

* Test update

* term renaming and disabling API tests on MySQL

* typo

* Updated store type definition

* Minor tweaks

* Added tests

* Removed error in app startup when content flaghging setup fails

* Updated sync condition:

* Flag message modal basE

* added post preview

* displaying options

* Adde comment input

* Updated tests and docs

* finction rename

* WIP

* Updated tests

* refactor

* lint fix

* MOved to data migration

* lint fix

* CI

* added new migration mocks

* Used setup for tests

* some comment

* Removed unnecesseery nil check

* Form validation

* WIP tests

* WIP tests

* WIP tests

* fix: mock content flagging config selector with correct reasons format

Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>

* fix: add mock for getContentFlaggingConfig in flag post modal test

Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>

* Updated error code order in API docs

* removed empty files

* Added tests

* lint fixes

* minor tweak

* lint fix

* type fix

* fixed test

* nit

* test enhancements

* API WIP

* API WIP

* creating values

* creating content flagging channel and properties

* Able to save properties

* Added another property field

* WIP

* WIP

* Added validations

* Added data validations and hidden post if confifgured to

* lint fixes

* Added API spec

* Added some tests

* Added tests for getContentReviewBot

* test: add comprehensive tests for getContentReviewChannels function

* Added more app layer tests

* Added TestCanFlagPost

* test: Add comprehensive tests for FlagPost function

* Added all app layer tests

* Removed a file that was reamoved downstream

* test: add content flagging test file

* test: add comprehensive tests for FlagContentRequest.IsValid method

* Added model tests

* test: add comprehensive tests for SqlPropertyValueStore.CreateMany

* test: add comprehensive tests for flagPost() API function

* Added API tests

* linter fix

* WIP

* sent post flagging confirmation message

* fixed i18n nissues

* fixed i18n nissues

* CI

* Updated test

* fix: reset contentFlaggingGroupId for test isolation in content flagging tests

* removed cached group ID

* removed debug log

* review fixes

* Used correct ot name

* CI

* Updated mobile text

* Handled JSON error

* fixerdf i18n

* CI

* Integrate flag post api (#33798)

* WIP

* WIP

* Added API call

* test: add test for Client4.flagPost API call in FlagPostModal

* fix: remove userEvent.setup() from flag post modal test

* test: wrap submit button click in act for proper state updates

* Updated tests

* lint fix

* CI

* Updated to allow special characters in comments

* Handled empty comment

* Used finally

* CI

* Fixed test

* Spillage card integration (#33832)

* Created getContentFlaggingFields API

* created getPostPropertyValues API

* WIP

* Created useContentFlaggingFields hook

* WIP

* WIP

* Added option to retain data for reviewers

* Displayed deleted post's preview

* DIsplayed all properties

* Adding field name i18n

* WIP - managing i18n able texts

* Finished displaying all fields

* Manual cleanup

* lint fixes

* team role filter logic fix

* Fixed tests

* created new API to fetch flagged posts

* lint fix

* Added new client methods

* test: add comprehensive tests for content flagging APIs

* Added new API tests

* fixed openapi spec

* Fixed DataSpillageReport tests

* Fixed PostMarkdown test

* Fixed PostPreviewPropertyRenderer test

* Added metadata to card renderer

* test fixes

* Added no comment placeholder

* Fixed test

* refactor: improve test mocking for data spillage report component

* test mock updates

* Updated reducer

* not resetting mocks

* WIP

* review fixes

* CI

* Fixed

* fixes

* Content flagging actions implementation (#33852)

* Added view detail button

* Created RemoveFlaggedMessageConfirmationModal modal

* Added key and remove flag request modal

* IMplemented delete flagged post

* Handled edge cases of deleting flagged post

* keep message

* UI integration

* Added WS event for post report update and handled deleted files of flagged post

* Added error handling in keep/remove forms

* i18n fixes

* Updated OpenAPI specs

* fixed types

* fixed types

* refactoring

* Fixed tests

* review fixes

* Added new property translations

* Improved test

* fixed test

* CI

* fixes

* CI

* fixed a test

* CI

---------

Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>
2025-10-02 20:24:29 +05:30
Ben Schumacher
18eb1347db
[MM-64900] Migrate to use request.CTX instead of context.Context (#33541)
* Migrate GetRoleByName

* Migrate users GetUsers

* Migrate Post and Thread store

* Migrate channel store

* Fix TestConvertGroupMessageToChannel

* Fix TestGetMemberCountsByGroup

* Fix TestPostStoreLastPostTimeCache
2025-09-18 16:14:24 +02:00
Jesse Hallam
8d74c6c45c
MM-64395: Remove unused searchArchivedChannelsForTeam API and implementations (#33885)
The searchArchivedChannelsForTeam functionality has been superseded by the
searchAllChannels API with include_deleted parameter. The Browse Channels modal
and other UI components now use the modern searchAllChannels approach.

Fixes: https://mattermost.atlassian.net/browse/MM-64395
2025-09-16 09:51:37 -03:00
Ben Schumacher
d78d59babe
Standardize request.CTX parameter naming to rctx (#33499)
* Standardize request.CTX parameter naming to rctx

- Migrate 886 request.CTX parameters across 147 files to use consistent 'rctx' naming
- Updated function signatures from 'c', 'ctx', and 'cancelContext' to 'rctx'
- Updated function bodies to reference the new parameter names
- Preserved underscore parameters unchanged as they are unused
- Fixed method receiver context issue in store.go

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Use request.CTX interface in batch worker

* Manual fixes

* Fix parameter naming

* Add linter check

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-10 15:11:32 +02:00
Alejandro García Montoro
553f99612e
MM-60441: Re-index public channels when a user joins a team (#33400)
* Index all public channels when a user joins a team

* Precompute team members for indexChannelsForTeam

* Refactor RequestContextWithMaster to store package

This way, we can import it from both the sqlstore and the searchlayer
packages. The alternative for this is duplicating the code in those two
packages, but that will *not* work:

The context package expects custom types for the keys stored in it, so
that different packages never clash with each other when trying to
register a new key. See the docs for the WithValue function:
https://pkg.go.dev/context#WithValue

If we try to duplicate the storeContextKey type in both the sqlstore and
searchlayer packages, although they *look* the same, they are not, and
HasMaster will fail to get the value of the storeContextKey(useMaster)
key if it's from the other package.

* Use master in call to GetTeamMembersForChannel

In GetTeamMembersForChannel, use the DB from the newly passed context,
which will be the receiving context everywhere except in the call done
from indexChannelsForTeam, to avoid the read after write issue when
saving a team member.

* Fix GetPublicChannelsForTeam paging

We were using the page and perPage arguments as is in the call to
GetPublicChannelsForTeam, but that function expects and offset and a
limit as understood by SQL. Although perPage and limit are
interchangeable, offset is not equal to page, but to page * perPage.

* Add a synchronous bulk indexer for Opensearch

* Implement Opensearch's SyncBulkIndexChannels

* Add a synchronous bulk indexer for Elasticsearch

* Implement Elasticsearch's SynkBulkIndexChannels

* Test SyncBulkIndexChannels

* make mocks

* Bulk index channels on indexChannelsForTeam

* Handle error from SyncBulkIndexChannels

* Fix style

* Revert indexChannelWithTeamMembers refactor

* Remove defensive code on sync bulk processor

* Revert "Add a synchronous bulk indexer for Opensearch"

This reverts commit bfe4671d96.

* Revert "Add a synchronous bulk indexer for Elasticsearch"

This reverts commit 6643ae3f30.

* Refactor bulk indexers with a common interface

* Test all the different implementations

Assisted by Claude

* Remove debug statements

* Refactor common code into _stop

* Rename getUserIDsFor{,Private}Channel

* Wrap error

* Make perPage a const

* Fix typos

* Call GetTeamsForUser only if needed

* Differentiate errors for sync/async processors

---------

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-08-25 19:28:19 +02:00
Henrique Machado
f418e1398d
[GH-28202]: Added GetGroupsByNames API (#33558)
* feat: Added GetGroupsByNames API
This commit implements the endpoint discussed in issue #28202.

This adds a new API endpoint to get multiple groups by a list of
names.

Previously, when the app received a post with @ mentions that it
didn't recognize, it would attempt to fetch them all as users,
then if some were still missing, it would go one by one attempting
to fetch each as a group. Now we just fetch all the groups at
once, just like we do for users.

Also added unit tests for the new API and it's respective
documentation.

* Added server version to GetGroupsByNames documentation

Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>

* fix: updated status_profile_polling tests to use new endpoint

* fix: fixed mock test

Was using get for post request

---------

Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
2025-08-20 18:20:07 -04:00
Jesse Hallam
c8d6630141
MM-63240: Always allow viewing archived channels (#32162)
* server: allow access to channel bookmarks in an archived channel

* server: allow access to posts in archived channels

* server: allow accessing channel members for archived channels

* server: allow autocompleting/searching archived channels

* server: allow access to files from archived channels

* server: fix access issue on database error

* server: allow access to archived channels

* server: remove TeamSettings.ExperimentalViewArchivedChannels from telemetry

* server: remove ExperimentalViewArchivedChannels from client config

* webapp: simplify delete channel

* webapp: simplify channel settings modal

* webapp: do not redirect away from archived channel

* webapp: rhs, always search posts from archived channels

* webapp: switch channels, always support archived channels

* webapp: search channel provider, always support archived channels

* webapp: browse channels, always support archived channels

* webapp, search results? fixup?

* webapp, confusing type issue

* webapp: unarchive, no need to report view archived

* webapp: command test, no need for ExperimentalViewArchivedChannels in config

* webapp: remove ExperimentalViewArchivedChannels from system console

* webapp: redux, do not delete posts, also fix LEAVE_CHANNEL

* update e2e tests

* server: fail startup if ExperimentalViewArchivedChannels is not enabled

* extract i18n

* updated snapshots

* update tests

* simplify posts reducer

* updated tests

* additional e2e tests

* Fix locale consistency in Jest tests

Added consistent locale environment variables (LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8)
to all Jest test scripts to prevent locale-dependent date formatting differences
across development environments.

This resolves snapshot test failures where DateTime.toLocaleString() would produce
different date formats on different systems (e.g., "6/8/2025" vs "08/06/2025" vs "2025-06-08").

Updated test scripts:
- test, test:watch, test:updatesnapshot, test:debug, test-ci

Updated snapshot to consistent en_US format.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Remove includeArchivedChannels parameter from GetMemberForPost

* Remove unnecessary includeDeleted variable assignments

* Deprecate ExperimentalViewArchivedChannels config field

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-08-15 13:50:20 -03:00
Christopher Poile
9b1e03a6b8
[MM-63557] mmctl: Add compliance export create cmd (#30594)
* Refactor job retrieval to support multiple statuses & multiple types

- Updated job retrieval functions to handle multiple job statuses.
- Renamed `GetJobsByTypeAndStatus` to `GetJobsByTypesAndStatuses` for consistency across the codebase.
- Adjusted related function signatures and implementations in the job store and retry layer to accommodate the new method.
- Updated tests to reflect changes in job retrieval logic and ensure proper functionality.

* Add compliance export create command and tests

- Introduced `ComplianceExportCreateCmd` to facilitate the creation of compliance export jobs with options for date, start, and end timestamps.
- Added unit tests for the new command, covering various scenarios including valid and invalid inputs.
- Updated documentation to include usage examples and options for the new command.
- Enhanced existing tests to ensure proper functionality of compliance export job handling.

* update docs

* update tests for new logic

* Refactor message export job tests to use DefaultPreviousJobPageSize

- Updated all test cases in worker_test.go to replace hardcoded page size of 100 with DefaultPreviousJobPageSize for consistency.
- Adjusted the worker.go file to define DefaultPreviousJobPageSize and use it in job retrieval logic.
- Ensured that the changes maintain the functionality of job data initialization and retrieval tests.

* PR comments

* PR comments, simplifications, clarifications, formatting

* prefer hypen over underscore in command names

* merge conflict

* update mmctl docs
2025-06-24 21:38:30 +00:00
Jesse Hallam
dcc72c4c61
MM-63728: simplify category store with graphql gone (#30848)
* move category permissions to api

In https://github.com/mattermost/mattermost/pull/21038, we changed the
behaviour of the channel category store to filter out deleted teams and
teams for which the user was not a member. This was necessary in part
due to querying multiple teams via GraphQL.

With GraphQL no longer supported, let's move the permissions to the
API instead and remove the `JOIN` to filter out teams in the store.

Note that we /don't/ prevent access to deleted teams. For better or
worse, deleted teams remain largely accessible via other API endpoints
anyway.

* remove ExcludeTeam / GraphQL support

As part of https://github.com/mattermost/mattermost/pull/20353, we added
`ExcludeTeam` and the associated logic to support a GraphQL API.

With GraphQL no longer supported, let's simplify this logic and remove
the filtering and associated complexity.

* Fix shadow variable declaration in channel_store_categories.go

Fixed golangci-lint error by reusing existing err variable rather than shadowing it.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix build issue

* Remove SidebarCategorySearchOpts and simplify API to use teamID string

Per code review feedback, this change removes the SidebarCategorySearchOpts
struct entirely since the Type field was never used in the store implementation.
All methods now accept a simple teamID string parameter instead of the struct,
which simplifies the API and makes the code clearer.

Changes:
- Remove SidebarCategorySearchOpts struct from store.go
- Update CreateInitialSidebarCategories and GetSidebarCategories signatures
- Update all implementations (sqlstore, retrylayer, timerlayer, mocks)
- Update all callers to pass teamID string directly
- Clean up unused imports

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-06-20 00:48:12 +00:00
Agniva De Sarker
761bc7549b
[AI assisted] MM-64298: Process setting status offline in batches (#31065)
When a user disconnects from the hub, we would spawn off a goroutine
which would make a cluster request, and then update the user status
as offline in the DB.

This was another case of unbounded concurrency where the number of
goroutines spawned was user controlled. Therefore, we would see
a clear spike in DB connections on master when a lot of users
would suddenly disconnect.

To fix this, we implement concurrency control in two areas:
1. In making the cluster request. We implement a counting semaphore
per-hub to avoid making unbounded cluster requests.
2. We use a buffered channel with a periodic flusher to process
status updates.

We also add a new store method to upsert multiple statuses
in a single query. The statusUpdateThreshold is set to 32, which means
no more than 32 rows will be upserted at one time, keeping the
SQL query load reasonable.

https://mattermost.atlassian.net/browse/MM-64298

```release-note
We improve DB connection spikes on user disconnect
by processing status updates in batches.
```
2025-06-17 09:20:34 +05:30
catalintomai
85391de22a
MM-57326: [Shared Channels] Message priority, acknowledgement and persistent notifications need to be synced (#30736) 2025-06-16 02:30:21 +02:00
catalintomai
fa1c77d9b0
MM-52600: [Shared Channels] Shared channels do not sync channel membership (#30976) 2025-06-15 10:07:56 +02:00
catalintomai
c46ed6c681
MM-62751: [Shared Channels] Allow remote users to be discoverable in the create DM/GM modal (#30918) 2025-06-13 16:51:12 +02:00
Miguel de la Cruz
43018759e5
Adds support for GMs in shared channels (#31403)
* Adds support for GMs in shared channels

* Fix linter

* Remove creatorID from slack call

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
2025-06-13 10:43:30 +00:00
Ben Cooke
aac34f6db4
[MM-64360] Data retention: optionally preserve pinned posts (#31165)
* add new config to preserve pinned posts during data retention

* more graceful error if the pinned post was a reply in a deleted thread
2025-06-09 15:41:07 -04:00
Miguel de la Cruz
e51ea025db
Deletes CPA values on CPA field type change (#31122)
* Deletes CPA values on CPA field type change

* Fix error method name reference

* Cleans the state when a CPA field's type is updated

* Fix types

* Fix linter

* Webapp no longer makes a decision on the change and server sents a flag in the WS message

* Fix linter

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
2025-05-27 13:38:05 +02:00
Ben Schumacher
c2d08b7540
[MM-63772] Add LDAP setting to re-add removed members (#30787) 2025-05-20 11:15:25 +02:00
Ibrahim Serdar Acikgoz
a344b3225b
[MM-61756] Attribute Based Access Control - Phase 1 (#30785)
Attribute Based Access Control - Base
* MM-63662

* MM-63919

* MM-63954

* MM-63955 

* MM-63425

* MM-63426

* MM-63458

* MM-63459

* MM-63603

* MM-63845

* MM-64146

* MM-64199

* MM-64201

* MM-64233

* MM-64247

* MM-64268

---------

Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>
Co-authored-by: Pablo Andrés Vélez Vidal <pablovv2012@gmail.com>
Co-authored-by: abhijit-singh <abhijitsingh0702@gmail.com>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
2025-05-15 11:33:08 +02:00
Miguel de la Cruz
6ab6a008e6
Adds the capability to retrieve a property field by name (#30859)
* Adds the capability to retrieve a property field by name

Allows to retrieve a property field by name and groupID. As the name
is only unique within the context of a group, and we can have multiple
fields with the same name in the store, for this method the groupID is
directly included in the query instead of being an optional field.

* Adds the targetID parameter to correctly filter fields

* Ensure the method only retrieves non-deleted fields

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-05-13 12:45:35 +02:00
Agniva De Sarker
4803892492
MM-56906: Remove redundant calls on team switch (#30771)
On page load, we load ALL channels and channel members from all teams.
But then, on team_switch, we would again load channels and channel
members from that team. This was redundant and mainly kept
because previously the websocket events were considered unreliable.

Now with reliable websockets, and client-side pings, we can detect
broken connections faster and recover without loss.

Additionally, the getAllChannelMembers call would page through
all responses on the client side. This was inefficient and incur
extra latency. To optimize for this, we introduce server-side
streaming of the full response if page is set to -1.

This optimizes the intial response as well.

https://mattermost.atlassian.net/browse/MM-56906

```release-note
Optimize team switch operation by removing calls to get channels
and channel members.
```


Co-authored-by: Mattermost Build <build@mattermost.com>
2025-05-12 20:05:46 +05:30
Miguel de la Cruz
ca9fd45408
Adds a mechanism to delete CPA values for a given user (#30330)
* Adds a mechanism to delete CPA values for a given user

This requires improving the Property Value service to enable delete
all values for a given target, so a new method was created that allows
to delete filtering by targetType and targetID (required) and
optionally for a specific groupID in case the caller wants to affect
all values for a target (useful in case you remove a post for example
and want to delete all values pointing to that post regardless of the
feature they belong to) or only those that belong to a specific
feature.

* Fix property value tests

* Fix after merge and update method name

* Fix linter

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-10 19:22:05 +02:00
Ibrahim Serdar Acikgoz
10b1f4c5ac
[MM-63428] add access control policy store (#30597) 2025-04-02 13:39:28 +02:00
Ben Cooke
ce9632cca3
MM-63311 (#30387)
* allow reference group changes
2025-03-31 15:49:55 -04:00
Miguel de la Cruz
1ca6f6d6fb
Adds a groupID filter to the property service methods (#30420)
* Adds a groupID filter to the property service methods

This allows the property service caller to directly ensure that a
given call is only going to affect a field or value that belongs to a
given group, instead of (for example) retrieving a property value
before deleting it by id to ensure that the value belongs to a
specific property group. The groupID filter is optional and has no
effect if called with the empty string value.

The changes also remove references to input sanitization on trimming
the whitespace for the CPA field names and validate at the API level
the input for the field patch endpoint.

* Fix linter

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-03-31 16:51:16 +00:00
Ben Schumacher
9b5d8d52bf
[MM-62427] Add message attachments validation (#30180)
* Add message attachments validation

* Add props validation

* Validate slack attachment fields

* Update tests and library usage

* Improve interactive dialog error for length checks

* Allow predefined colors for slack attachments

* Fix TestPostAction

* Use const for data source

* Add tests

* Cleanup unused props

* Add happy path tests

* lint fixes

* Add validation for PostActionOptions
2025-03-20 12:53:50 +01:00
Christopher Poile
c049748b88
[MM-63314] Fix ClaimJob in HA environments (#30383)
* ClaimJob now returns newly claimed job

* internal code affected by change

* test changes required

* two branches: for mysql, use transaction; for postgres, use returning

* two branches: for mysql, use transaction; for postgres, use returning

* use same millis value for LastActivityAt and StartAt

* blank commit

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-03-14 10:24:26 -04:00
Arya Khochare
4fbfd84957
[MM-61691] Deleting drafts when permanently deleting a user (#30233)
* deleting drafts on permanently deleting user

* verified count of drafts before deleting

* i18n fix

* use ExecBuilder

* changed error message to specify user

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>
2025-03-13 19:19:38 +05:30
Miguel de la Cruz
c5e6d9f570
Updates the property service and store method signatures (#30103)
* Updates the property service and store method signatures

Getters can now receive a `groupID` that narrows down the query if
present, so it's not necessary to check for the group ID on the
returning values from the outside layers.

The Search methods now receive the `groupID` and the `targetID`
explicitly as parameters, incentivizing the use of the indexes that
the underlying tables have on the searches.

* Fix tests

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-03-10 14:26:47 +00:00
Caleb Roseland
68c11e9ecb
MM-61375: Update file handling for bookmarks (#30034) 2025-02-13 17:11:13 -06:00
Julien Tant
632a60b332
[MM-62553]+[MM-62554] Property Architecture: cursor based pagination (#30119)
* refactor: Replace pagination with cursor-based pagination for custom profile attributes

* remove pagination loop on property value retrieval for CPA

* add migrations to optimize pagination on property fields and values

* adapt test to remove pagination check

* update migrations list

* postgres: drop index concurrently

* concurrent index manipulation must be done outside of a Tx

* fix: Correct SQL index drop syntax from "OM" to "ON" in migration files

* test: Add CountForGroup test cases for property field store

* refactor: Add CountForGroup method to PropertyFieldStore interface and implementations

* Fix style and i18n

* feat: Add optional deleted property field filtering to CountForGroup method

* refactor: Update CountForGroup to support optional deleted property fields

* test: Add comprehensive tests for CountForGroup with includeDeleted parameter

* adapt test + gen layers

* rename property service method and set the includeDelete to false

* refactor: Remove redundant constant and use CustomProfileAttributesFieldLimit directly

* fix tests

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-02-13 15:23:50 +00:00
Miguel de la Cruz
f85a8c61a4
Adds websocket messages to Custom Profile Attributes (#30163)
* Adds websocket messages to Custom Profile Attributes

The app layer now fires a websocket event as part of the operations
over Custom Profile Attribute fields and values. It updates as well
the Patch method for CPA values so all the changes are commited as
part of the same transaction.

To be able to do this last operation, the change adds methods to
upsert CPA values in both the store and the property service.

* Fix i18n strings

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
2025-02-13 11:21:46 +00:00
Arya Khochare
50258a6510
[MM-62153] Avoid SELECT * in preference_store.go (#30081)
* refractored sql queries

* GetBuilder to Select Builder as preferences is slice (fix failing test)

* test: Add test for GetCategoryAndName preference method

* fix s/GetBuilder/SelectBuilder

* linting

* make mocks

* make store-layers

---------

Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
2025-02-07 12:05:29 -04:00
Ibrahim Serdar Acikgoz
a3f22bd349
[MM-62004] update threads after a channel gets moved (#29624) 2025-02-07 15:20:13 +00:00
Agniva De Sarker
ae9e6174e5
[AI assisted]: Improve system console statistics performance (#29899)
```release-note
NONE
```

Co-authored-by: Matthew Birtch <mattbirtch@gmail.com>
2025-02-04 21:54:01 +05:30
Harshil Sharma
6e5a67caec
Feature edit attachments (#29769)
* Updated patch/update post API to allow file modification (#29447)

* WIP

* WIP

* Atatched new files ton post

* WIP: deleting removed files

* Deleted removed files and invalidated file metadata cache

* removed file ignore logif from update post API

* Added TestFindExclusives

* Added tests for DeleteForPostByIds

* Added app layer tests

* Added tests

* Added API level tests

* test enhancements

* Fixed a test

* Edit history include file metadata (#29505)

* Send file metadata in edit history metadata

* Added app tests

* Added store tests

* Added tests for populateEditHistoryFileMetadata{

* Added cache to avoid repetitigve DB calls for edits with only message changes

* Added API tests

* i18m fix

* removed commented code

* Improved test helper

* Show attachments in edit history RHS (#29519)

* Send file metadata in edit history metadata

* Added app tests

* Added store tests

* Added tests for populateEditHistoryFileMetadata{

* Added cache to avoid repetitigve DB calls for edits with only message changes

* Added API tests

* i18m fix

* WIUP: displa files in edit

* removed commented code

* Displayed file in edit history

* Handled file icon

* Fixed closing history component on clicking on file

* Simplified selector

* Simplified selector

* Improved test helper

* Disabled action menu on edit history file

* Added tests

* Improved selector

* Updated snapshot

* review Fixes

* restructured componnets

* Updated test

* Updated test

* Restore post api (#29643)

* Restore post version API WIP

* Undelete files WIP

* Added store tests

* Created post restore API

* Updated updatepost safeUpdate signature

* review fixex and improvements

* Fixed an app test

* Added API laer tests

* Added API tests and OpenAPI specs

* Fixed a typo

* Allow editing files when editing posts (#29709)

* WIP - basic view files when editing post

* Cleanup

* bg color

* Added text editor tests for files

* WIP

* WIP

* removed debug log

* Allowed admin to add and remove files on someone else's post

* Handled drafts and scheduled posts

* linter fixes

* Updated snapshot

* server test fix

* CI

* Added doc

* Restore post api integration (#29719)

* WIP - basic view files when editing post

* Cleanup

* bg color

* Added text editor tests for files

* WIP

* WIP

* removed debug log

* Allowed admin to add and remove files on someone else's post

* Handled drafts and scheduled posts

* linter fixes

* Updated snapshot

* server test fix

* Used new API to restore post

* handled edut limit and undo

* lint fix

* added comments

* Fixed edit post item tests

* Fixed buttons

* Aded snapshots

* fix test

* Updated snapshot

* Minor fixes

* fixed snapshot

* Edit file dnd area (#29763)

* dnd wip

* DND continued

* Supported multiple unbind dragster funcs

* lint fixes

* Got center channel file drop working when editing a post

* file dnd working with center channel and rhs

* file dnd working with center channel and rhs

* removed unneeded stopPropogation calls

* cleanup

* DND overlay fix

* Lint fix

* Advanced text editor test updates for file upload overlay

* fixed use upload hook tests

* Updated some more snapshots

* minor cleanup

* Updated i18n

* removed need of array for dragster unbind events

* lint fixes

* edit history cursor

* Fixed bugu causing faliure to delete empty posts (#29778)

* Files in restore confirmation (#29781)

* Added files to restore post confirmation dialog

* Fixed post restore toast colors

* Fixed restore bug

* Fixed restore confirmation toast tests

* a11y improvement and modal width fix

* Edit attachment misc fixes (#29808)

* Removed single image actions in restore post confirmation dialog

* Fixed file drop overlay size and position

* Made edit indiator accessible

* Lint fix

* Added bunch of more tests

* ANother test migrated from enzyme to react testing library

* More test enhancements

* More test enhancements

* More test enhancements

* lint fixes

* Fixed  a test

* Added missing snapshots

* Test fixes
2025-01-13 18:16:56 +05:30
Miguel de la Cruz
ecdce71fc4
Adds the main Property System Architecture components (#29644)
* Adds the main Property System Architecture components

This change adds the necessary migrations for the Property Groups,
Fields and Values tables to be created, the store layer and a Property
Service that can be used from the app layer.

* Update property field type to use user instead of person

* Update PropertyFields to allow for unique nondeleted fields and remove redundant indexes

* Update PropertyValues to allow for unique nondeleted fields and remove redundant indexes

* Use StringMap instead of the map[string]any on property fields

* Add i18n strings

* Revert "Use StringMap instead of the map[string]any on property fields"

This reverts commit e2735ab0f8.

* Cast JSON binary data to string and add todo note for StringMap use

* Add mocks to the retrylayer tests

* Cast JSON binary data to string in property value store

* Check for binary parameter instead of casting to string for JSON data

* Check property field type is one of the allowed ones

* Avoid reusing err variable to be explicit about the returned value

* Merge Property System Migrations into one file

* Adds NOT NULL to timestamps at the DB level

* Update stores to use tableSelectQuery instead of a slice var

* Update PropertyField model translations to be more explicit and avoid repetition

* Update PropertyValue model translations to be more explicit and avoid repetition

* Use ExecBuilder instead of ToSql&Exec

* Update property field errors to add context

* Ensure PerPage is greater than zero

* Update store errors to give more context

* Use ExecBuilder in the property stores where possible

* Add an on conflict suffix to the group register to avoid race conditions

* Remove badly used translation string

* Remove unused get in register group method

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-13 11:41:44 +00:00
Christopher Poile
aba4434dab
MM-59966 - Compliance Export overhaul - feature branch (#29789)
* [MM-59089] Add a compliance export constant (#27919)

* add a useful constant

* i18n

* another constant

* another i18n

* [MM-60422] Add GetChannelsWithActivityDuring (#28301)

* modify GetUsersInChannelDuring to accept a slice of channelIds

* add GetChannelsWithActivityDuring

* add compliance export progress message; remove unused custom status

* linting

* tests running too fast

* add batch size config settings

* add store tests

* linting

* empty commit

* i18n changes

* fix i18n ordering

* MM-60570 - Server-side changes consolidating the export CLI with server/ent code (#28640)

* add an i18n field; add the CLI's export directory

* int64 -> int

* Add UntilUpdateAt for MessageExport and AnalyticsPostCount

to merge

* remove now-unused i18n strings

* add TranslationsPreInitFromBuffer to allow CLI to use i18n

* use GetBuilder to simplify; rename TranslationsPreInitFromFileBytes

* [MM-59089] Improve compliance export timings (#1733 - Enterprise repo)

* MM-60422 - Performance and logic fixes for Compliance Exports (#1757 - Enterprise repo)

* MM-60570 - Enterprise-side changes consolidating the export CLI with server/ent code (#1769 - Enterprise repo)

* merge conflicts; missed file from ent branch

* MM-61038 - Add an option to sqlstore.New (#28702)

remove useless comment

add test

add an option to sqlstore.New

* MM-60976: Remove RunExport command from Mattermost binary (#28805)

* remove RunExport command from mattermost binary

* remove the code it was calling

* fix i18n

* remove test (was only testing license, not functionality)

* empty commit

* fix flaky GetChannelsWithActivityDuring test

* MM-60063: Dedicated Export Filestore fix, redo of #1772 (enterprise) (#28803)

* redo filestore fix #1772 (enterprise repo) on top of MM-59966 feature

* add new e2e tests for export filestore

* golint

* ok, note to self: shadowing bad, actually (when there's a defer)

* empty commit

* MM-61137 - Message export: Support 7.8.11 era dbs (#28824)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* blank commit

* MM-60974 - Message Export: Add performance metrics (#28836)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* cleanup unneeded named returns

* blank commit

* MM-60975 - Message export: Add startTime and endTime to export folder name (#28840)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* output startTime and endTime in export folder

* empty commit

* merge conflict

* MM-60978 - Message export: Improve xml fields; fix delete semantics (#28873)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* output startTime and endTime in export folder

* empty commit

* add xml fields, omit when empty, tests

* fix delete semantics; test (and test for update semantics)

* clarify comments

* simplify edited post detection, now there's no edge case.

* add some spacing to help fast running tests

* merge conflicts/updates needed for new deleted post semantics

* linting; fixing tests from upstream merge

* use SafeDereference

* linting

* stronger typing; better wrapped errors; better formatting

* blank commit

* goimports formatting

* fix merge mistake

* minor fixes due to changes in master

* MM-61755 - Simplifying and Support reporting to the db from the CLI (#29281)

* finally clean up JobData struct and stringMap; prep for CLI using db

* and now simplify using StringMapToJobDataWithZeroValues

* remove unused fn

* create JobDataExported; clean up errors

* MM-60176 - Message Export: Global relay cleanup (#29168)

* move global relay logic into global_relay_export

* blank commit

* blank commit

* improve errors

* MM-60693 - Refactor CSV to use same codepath as Actiance (#29191)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* MM-61696 - Refactor GlobalRelay to use same codepath as Actiance (#29225)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* remove newly unneeded function and its test. goodbye.

* refactor GetPostAttachments for csv + global relay to share

* refactor global_relay_export and fix tests (no changes to output)

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* remove unneeded nil check

* PR comments

* MM-61715 - Generalize e2e to all export types 🤖  (#29369)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* remove newly unneeded function and its test. goodbye.

* refactor GetPostAttachments for csv + global relay to share

* refactor global_relay_export and fix tests (no changes to output)

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* remove unneeded nil check

* PR comments

* refactor isDeletedMsg for all export types

* fix start and endtime, nasty csv createAt bug; bring closer to Actiance

* align unit tests with new logic (e.g. starttime / endtime)

* refactor a TimestampConvert fn for code + tests

* bug: pass templates to global relay (hurray for e2e tests, otherwise...)

* add global relay zip to allowed list (only for tests)

* test helpers

* new templates for e2e tests

* e2e tests... phew.

* linting

* merge conflicts

* unexport PostToRow; add test helper marker

* cleanup, shortening, thanks to PR comments

* MM-61972 - Generalize export data path - Actiance (#29399)

* extract and generalize the export data generation functions

* finish moving test (bc of previous extraction)

* lift a function from common -> shared (to break an import cycle)

* actiance now takes general export data, processes it into actiance data

* bring tests in line with correct sorting rules (upadateAt, messageId)

* fixups, PR comments

* turn strings.Repeat into a more descriptive const

amended: one letter fix; bad rebase

* MM-62009 - e2e clock heisenbug (#29434)

* consolidate assertions; output debuggable diffs (keeping for future)

* refactor test output generator to generators file

* waitUntilZeroPosts + pass through until to job = fix all clock issues

* simplify messages to model.NewId(); remove unneeded waitUntilZeroPosts

* model.NewId() -> storetest.NewTestID()

* MM-61980 - Generalize export data path - CSV (#29482)

* simple refactoring

* increase sleep times for (very) rare test failures

* add extra information to the generic export for CSV

* adj Actiance to handle new generic export (no difference in its output)

* no longer need mergePosts (yay), move getJoinLeavePosts for everyone

* adjust tests for new csv semantics (detailed in summary)

* and need to add the new exported data to the export_data_tests

* rearrange csv writing to happen after data export (more logical)

* linting

* remove debug statements

* figured out what was wrong with global relay e2e test 3; solid now

* PR comments

* MM-61718 - Generalize export data path - Global Relay (#29508)

* move global relay over to using the generalized export data

* performance pass -- not much can be done

* Update server/enterprise/message_export/global_relay_export/global_relay_export.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

---------

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* MM-62058 - Align CSV with Actiance (#29551)

* refactoring actiance files and var names for clarity

* bug found in exported attachments (we used to miss some start/ends)

* changes needed for actiance due to new generic exports

* bringing CSV up to actiance standards

* fixing global relay b/c of new semantics (adding a note on an edge case)

* aligning e2e tests, adding comments to clarify what is expected/tested

* necessary changes; 1 more test for added functionality (ignoreDeleted)

* comment style

* MM-62059 - Align Global Relay with Actiance/CSV; many fixes (#29665)

* core logic changes to general export_data and the specific export paths

* unit tests and e2e tests, covering all new edge cases and all logic

* linting

* better var naming, const value, and cleaning up functions calls

* MM-62436 - Temporarily skip cypress tests that require download link (#29772)

---------

Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2025-01-10 16:56:02 -05:00
Harshil Sharma
c79a8a8b4a
MM-61484 - Deleting scheduled posts when permanently deleting a user (#29152)
* Deleting scheduled posts when permanently deleting a user

* Updated tests

* CI

* Testing CI

* Restored a test change

* Skipping flaky test
2024-11-13 12:41:31 +05:30