Commit graph

42 commits

Author SHA1 Message Date
Ben Schumacher
6e87c94f29
[MM-62770] Validate log levels in AdvancedLoggingJSON (#34414) 2025-12-08 10:54:33 +01:00
Catena cyber
758bdd785f
perf: apply perfpsrint linter (#33967)
* perf: apply perfpsrint linter

* further simplifications

* improved TestParseHashtags coverage

* more simplifications

* simplify renderBlockHTML further

---------

Co-authored-by: Jesse Hallam <jesse@mattermost.com>
2025-11-28 11:23:51 -04:00
Ben Schumacher
a7a8941adb
Add documentation for audit levels (#34457)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-12 14:07:12 +01:00
Jesse Hallam
7dff47ee6d
Sentry context nil session (#34383)
* whitespace changes

* context.WithSession: ignore if nil

* unit test context.WithSession
2025-11-04 10:12:30 -04:00
Ben Schumacher
36b00d9bb6
[MM-64485] Remove separate notification log file (#33473)
- Remove NotificationLogSettings configuration entirely
- Add new notification-specific log levels (NotificationError, NotificationWarn, NotificationInfo, NotificationDebug, NotificationTrace)
- Consolidate all notification logs into standard mattermost.log file
- Update all notification logging code to use new multi-level logging (MlvlNotification*)
- Remove notification logger infrastructure and support packet integration
- Update test configurations and remove deprecated functionality tests
- Add comprehensive tests for new notification log levels

This change simplifies log analysis by unifying all application logging while maintaining flexibility through Advanced Logging configuration for administrators who need separate notification logs.

🤖 Generated with [Claude Code](https://claude.ai/code)
---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 10:17:45 +02:00
Ben Schumacher
53809e4500
Improve HTTP service IP and host validation error messages (#33450)
Co-authored-by: Nick Misasi <nick.misasi@mattermost.com>
2025-07-22 14:49:32 +02:00
Ben Schumacher
9add320011
[MM-64654] Migrate to modern Go features (#31820) 2025-07-18 12:54:51 +02:00
Claudio Costa
611b2a8e79
[MM-62408] Server Code Coverage with Fully Parallel Tests (#30078)
* TestPool

* Store infra

* Store tests updates

* Bump maximum concurrent postgres connections

* More infra

* channels/jobs

* channels/app

* channels/api4

* Protect i18n from concurrent access

* Replace some use of os.Setenv

* Remove debug

* Lint fixes

* Fix more linting

* Fix test

* Remove use of Setenv in drafts tests

* Fix flaky TestWebHubCloseConnOnDBFail

* Fix merge

* [MM-62408] Add CI job to generate test coverage (#30284)

* Add CI job to generate test coverage

* Remove use of Setenv in drafts tests

* Fix flaky TestWebHubCloseConnOnDBFail

* Fix more Setenv usage

* Fix more potential flakyness

* Remove parallelism from flaky test

* Remove conflicting env var

* Fix

* Disable parallelism

* Test atomic covermode

* Disable parallelism

* Enable parallelism

* Add upload coverage step

* Fix codecov.yml

* Add codecov.yml

* Remove redundant workspace field

* Add Parallel() util methods and refactor

* Fix formatting

* More formatting fixes

* Fix reporting
2025-05-30 13:58:26 +02:00
Ben Schumacher
bfb15ab179
[MM-61074] Fix errcheck issues in oauth_test.go and web_test.go (#30707)
Co-authored-by: Claude <noreply@anthropic.com>
2025-05-07 12:41:10 +02:00
Surya Venkata Sainadh Pichika
62753a1481
[MM-63455] Fix Link previews with brackets when making the request (#30507)
* allow opening angle bracket before links

* cut off link before an angle bracket(both open & close)

* add more test cases for ParseURLAutolink func

* add more test cases for ParseWWWAutolink func

* add more test cases for TrimTrailingCharactersFromLink func

* add more test cases for GetFirstLinkAndImages func

* add test cases for isAllowedBeforeWWWLink func

* allow closing angle bracket and opening paranthesis before links

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-07 16:17:46 +05:30
Ben Schumacher
166a676fe5
Enforce use of any instead of interface{} (#30588) 2025-03-31 10:44:34 +02:00
Christopher Poile
aba4434dab
MM-59966 - Compliance Export overhaul - feature branch (#29789)
* [MM-59089] Add a compliance export constant (#27919)

* add a useful constant

* i18n

* another constant

* another i18n

* [MM-60422] Add GetChannelsWithActivityDuring (#28301)

* modify GetUsersInChannelDuring to accept a slice of channelIds

* add GetChannelsWithActivityDuring

* add compliance export progress message; remove unused custom status

* linting

* tests running too fast

* add batch size config settings

* add store tests

* linting

* empty commit

* i18n changes

* fix i18n ordering

* MM-60570 - Server-side changes consolidating the export CLI with server/ent code (#28640)

* add an i18n field; add the CLI's export directory

* int64 -> int

* Add UntilUpdateAt for MessageExport and AnalyticsPostCount

to merge

* remove now-unused i18n strings

* add TranslationsPreInitFromBuffer to allow CLI to use i18n

* use GetBuilder to simplify; rename TranslationsPreInitFromFileBytes

* [MM-59089] Improve compliance export timings (#1733 - Enterprise repo)

* MM-60422 - Performance and logic fixes for Compliance Exports (#1757 - Enterprise repo)

* MM-60570 - Enterprise-side changes consolidating the export CLI with server/ent code (#1769 - Enterprise repo)

* merge conflicts; missed file from ent branch

* MM-61038 - Add an option to sqlstore.New (#28702)

remove useless comment

add test

add an option to sqlstore.New

* MM-60976: Remove RunExport command from Mattermost binary (#28805)

* remove RunExport command from mattermost binary

* remove the code it was calling

* fix i18n

* remove test (was only testing license, not functionality)

* empty commit

* fix flaky GetChannelsWithActivityDuring test

* MM-60063: Dedicated Export Filestore fix, redo of #1772 (enterprise) (#28803)

* redo filestore fix #1772 (enterprise repo) on top of MM-59966 feature

* add new e2e tests for export filestore

* golint

* ok, note to self: shadowing bad, actually (when there's a defer)

* empty commit

* MM-61137 - Message export: Support 7.8.11 era dbs (#28824)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* blank commit

* MM-60974 - Message Export: Add performance metrics (#28836)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* cleanup unneeded named returns

* blank commit

* MM-60975 - Message export: Add startTime and endTime to export folder name (#28840)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* output startTime and endTime in export folder

* empty commit

* merge conflict

* MM-60978 - Message export: Improve xml fields; fix delete semantics (#28873)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* output startTime and endTime in export folder

* empty commit

* add xml fields, omit when empty, tests

* fix delete semantics; test (and test for update semantics)

* clarify comments

* simplify edited post detection, now there's no edge case.

* add some spacing to help fast running tests

* merge conflicts/updates needed for new deleted post semantics

* linting; fixing tests from upstream merge

* use SafeDereference

* linting

* stronger typing; better wrapped errors; better formatting

* blank commit

* goimports formatting

* fix merge mistake

* minor fixes due to changes in master

* MM-61755 - Simplifying and Support reporting to the db from the CLI (#29281)

* finally clean up JobData struct and stringMap; prep for CLI using db

* and now simplify using StringMapToJobDataWithZeroValues

* remove unused fn

* create JobDataExported; clean up errors

* MM-60176 - Message Export: Global relay cleanup (#29168)

* move global relay logic into global_relay_export

* blank commit

* blank commit

* improve errors

* MM-60693 - Refactor CSV to use same codepath as Actiance (#29191)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* MM-61696 - Refactor GlobalRelay to use same codepath as Actiance (#29225)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* remove newly unneeded function and its test. goodbye.

* refactor GetPostAttachments for csv + global relay to share

* refactor global_relay_export and fix tests (no changes to output)

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* remove unneeded nil check

* PR comments

* MM-61715 - Generalize e2e to all export types 🤖  (#29369)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* remove newly unneeded function and its test. goodbye.

* refactor GetPostAttachments for csv + global relay to share

* refactor global_relay_export and fix tests (no changes to output)

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* remove unneeded nil check

* PR comments

* refactor isDeletedMsg for all export types

* fix start and endtime, nasty csv createAt bug; bring closer to Actiance

* align unit tests with new logic (e.g. starttime / endtime)

* refactor a TimestampConvert fn for code + tests

* bug: pass templates to global relay (hurray for e2e tests, otherwise...)

* add global relay zip to allowed list (only for tests)

* test helpers

* new templates for e2e tests

* e2e tests... phew.

* linting

* merge conflicts

* unexport PostToRow; add test helper marker

* cleanup, shortening, thanks to PR comments

* MM-61972 - Generalize export data path - Actiance (#29399)

* extract and generalize the export data generation functions

* finish moving test (bc of previous extraction)

* lift a function from common -> shared (to break an import cycle)

* actiance now takes general export data, processes it into actiance data

* bring tests in line with correct sorting rules (upadateAt, messageId)

* fixups, PR comments

* turn strings.Repeat into a more descriptive const

amended: one letter fix; bad rebase

* MM-62009 - e2e clock heisenbug (#29434)

* consolidate assertions; output debuggable diffs (keeping for future)

* refactor test output generator to generators file

* waitUntilZeroPosts + pass through until to job = fix all clock issues

* simplify messages to model.NewId(); remove unneeded waitUntilZeroPosts

* model.NewId() -> storetest.NewTestID()

* MM-61980 - Generalize export data path - CSV (#29482)

* simple refactoring

* increase sleep times for (very) rare test failures

* add extra information to the generic export for CSV

* adj Actiance to handle new generic export (no difference in its output)

* no longer need mergePosts (yay), move getJoinLeavePosts for everyone

* adjust tests for new csv semantics (detailed in summary)

* and need to add the new exported data to the export_data_tests

* rearrange csv writing to happen after data export (more logical)

* linting

* remove debug statements

* figured out what was wrong with global relay e2e test 3; solid now

* PR comments

* MM-61718 - Generalize export data path - Global Relay (#29508)

* move global relay over to using the generalized export data

* performance pass -- not much can be done

* Update server/enterprise/message_export/global_relay_export/global_relay_export.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

---------

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* MM-62058 - Align CSV with Actiance (#29551)

* refactoring actiance files and var names for clarity

* bug found in exported attachments (we used to miss some start/ends)

* changes needed for actiance due to new generic exports

* bringing CSV up to actiance standards

* fixing global relay b/c of new semantics (adding a note on an edge case)

* aligning e2e tests, adding comments to clarify what is expected/tested

* necessary changes; 1 more test for added functionality (ignoreDeleted)

* comment style

* MM-62059 - Align Global Relay with Actiance/CSV; many fixes (#29665)

* core logic changes to general export_data and the specific export paths

* unit tests and e2e tests, covering all new edge cases and all logic

* linting

* better var naming, const value, and cleaning up functions calls

* MM-62436 - Temporarily skip cypress tests that require download link (#29772)

---------

Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2025-01-10 16:56:02 -05:00
Claudio Costa
8c41ec75db
httpservice: improve validation of proxied URLs (#29600) 2024-12-19 17:55:42 +00:00
Agniva De Sarker
6075b1cd4e
MM-61225: Revert session pooling (#28901)
This originated from https://github.com/mattermost/mattermost/issues/15249.

However, the original idea was discarded https://github.com/mattermost/mattermost/issues/15249#issuecomment-709713065
as being too complicated to implement. Then I had another
idea to implement it just for session objects.

My thinking was that since every single request allocates a new
session struct, it would be good to use a sync.Pool for that.

However, 4 years later, now we know that the primary bottleneck
in app performance comes from websocket event marshalling.
Therefore, while it would be good to do this, it is difficult
to do it correctly (as shown by the numerous racy tests).

Hence, reverting this.

```release-note
NONE
```
2024-10-24 22:50:44 +05:30
Ben Schumacher
7ff22436dc
[MM-60685] Fix racy TestSyncLdap (#28324) 2024-10-14 13:04:48 +02:00
Agniva De Sarker
d89ffe269f
MM-59026: Update reserved IP ranges (#27751)
https://mattermost.atlassian.net/browse/MM-59026
```release-note
NONE
```
2024-07-25 19:38:56 +05:30
Christopher Speller
04181247f8
Move HTTP service to public for plugin use (#27284)
* Move httpservice for use in plugins

* Adapt httpservice for plugin use

* Fix lint
2024-06-05 09:58:04 -07:00
Doug Lauder
2825ab689c
Support for additional audit config via env var (#26785)
* Add support for additional audit config via env var
2024-04-15 12:14:26 -04:00
Ben Schumacher
929ab6b289
Document variable names for request.CTX and mlog.Logger (#26029) 2024-01-26 14:03:35 +01:00
Nick Misasi
f0a336ba07
Feature: Wrangler (#23602)
* Migrate feature/wrangler to mono-repo

* Add wrangler files

* Fix linters, types, etc

* Fix snapshots

* Fix playwright

* Fix pipelines

* Fix more pipeline

* Fixes for pipelines

* More changes for pipeline

* Fix types

* Add support for a feature flag, but leave it defaulted on for spinwick usage for now

* Update snapshot

* fix js error when removing last value of multiselect, support CSV marshaling to string array for textsetting

* Fix linter

* Remove TODO

* Remove another TODO

* fix tests

* Fix i18n

* Add server tests

* Fix linter

* Fix linter

* Use proper icon for dot menu

* Update snapshot

* Add Cypress UI tests for various entrypoints to move thread modal, split SCSS out from forward post into its own thing

* clean up

* fix linter

* More cleanup

* Revert files to master

* Fix linter for e2e tests

* Make ForwardPostChannelSelect channel types configurable with a prop

* Add missing return

* Fixes from PR feedback

* First batch of PR Feedback

* Another batch of PR changes

* Fix linter

* Update snapshots

* Wrangler system messages are translated to each user's locale

* Initially translate Wrangler into system locale rather than initiating user

* More fixes for PR Feedback

* Fix some server tests

* More updates with master. Fixes around pipelines. Enforce Enterprise license on front/back end

* Add tests for dot_menu

* More pipeline fixes

* Fix e2etests prettier

* Update cypress tests, change occurrences of 'Wrangler' with 'Move Thread'

* Fix linter

* Remove enterprise lock

* A couple more occurrences of wrangler strings, and one more enterprise lock

* Fix server tests

* Fix i18n

* Fix e2e linter

* Feature flag shouldn't be on by default

* Enable move threads feature in smoke tests (#25657)

* enable move threads feature
* add @prod tag

* Fix move_thread_from_public_channel e2e test

* Fix e2e style

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: yasserfaraazkhan <attitude3cena.yf@gmail.com>
2023-12-11 15:27:34 -05:00
Ben Schumacher
b2ec1ff8ae
[MM-55595] Use annotated logger in search layer (#25468) 2023-12-04 18:34:57 +01:00
Ben Schumacher
e99ded1635
[MM-55480] Update logr to v2.0.21 (#25431) 2023-11-23 10:30:08 +01:00
Harrison Healey
9397970644
MM-55267 Add ability for server-side Markdown code to understand emojis (#25332)
* MM-55267 Add ability for server-side Markdown code to understand emojis

* Remove unused regex
2023-11-13 14:38:05 -05:00
Ben Schumacher
486e836b83
Fix racy test issues (#24971) 2023-11-06 12:26:17 +01:00
Ben Schumacher
4948aa106b
[MM-54403] Introduce separate AdvanctedLogging levels for LDAP messages (#24873) 2023-11-02 12:31:57 +01:00
Ben Schumacher
671cf6cc3d
[MM-54319] Add AdvancedLoggingJSON setting to system console (#24733)
* Add AdvancedLoggingJSON setting to system console

* Apply suggestions from code review

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>

* Add validation for AdvancedLoggingJSON

---------

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-10-25 16:40:07 +02:00
Tom De Moor
1de790a4fe
Adding Vietnamese as a new language (beta) (#24074)
* adding vietnamese

* ran npm run test:updatesnapshot

* ran npm run test:updatesnapshot

* Update i18n.jsx

* updatesed snapshot

* fix whitespace in snapshots

* add vi as supported server locale

* update docs to point at server-side list of supported locales

* removed incorrect translation

---------

Co-authored-by: Tom De Moor <tom@controlaltdieliet.be>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
2023-10-17 10:08:13 -03:00
Ben Schumacher
4408ece955
Fix data race in bulk import tests (#24897) 2023-10-16 22:40:19 +02:00
Agniva De Sarker
3dd9e3715c
Goodbye, GraphQL (#24827)
```release-note
NONE
```
2023-10-12 09:47:35 +05:30
Ben Schumacher
13c05a571f
Migrate store methods to use request.Context instead of context.Context (#24836) 2023-10-11 13:08:55 +02:00
Ben Schumacher
0d5a8b8841
Add whitespace linter (#24855) 2023-10-11 10:13:36 +02:00
Jesse Hallam
14f784a0cc
MM-54556: Integrate wip i18n languages (#24308)
* merge languages from https://github.com/mattermost/i18n-wip

* allow only supported server locales

* Revert "merge languages from https://github.com/mattermost/i18n-wip"

This reverts commit 36de545102. We'll let
weblate populate these on start instead.

* copy fileutils to public/utils

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-09-26 13:49:27 -03:00
Felipe Martin
f65dad83bb
Migrate emojiStore to use request.CTX instead of context.Context (#24514)
* migrate emojistore to request.ctx

* use mlog.CreateConsoleTestLogger

* Add comment to WithMaster and RequestContextWithMaster
2023-09-11 17:07:29 +02:00
Ben Schumacher
c656ca3fe3
Remove unused err field from request.Context (#24471) 2023-09-07 12:12:25 +02:00
Ben Schumacher
5f868e2469
Cleanup mlog levels (#24459) 2023-09-07 12:11:57 +02:00
Ben Schumacher
30b12f199b
[MM-54132] Use annotated logger for log messages from jobs (#24275) 2023-09-07 08:50:22 +02:00
Felipe Martin
9f7521d003
Move request package into public/shared folder (#24420)
* move request package to public/shared

* updated app-layers

* update app layer

* remove original package location
2023-09-05 09:47:30 +02:00
Ben Schumacher
3ee5432664
[MM-53968] Includes mattermost-plugin-api into the mono repo (#24235)
Include https://github.com/mattermost/mattermost-plugin-api into the mono repo

Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Michael Kochell <mjkochell@gmail.com>
Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Alex Dovenmuehle <alex.dovenmuehle@mattermost.com>
Co-authored-by: Michael Kochell <6913320+mickmister@users.noreply.github.com>
Co-authored-by: Christopher Poile <cpoile@gmail.com>
Co-authored-by: İlker Göktuğ Öztürk <ilkergoktugozturk@gmail.com>
Co-authored-by: Shota Gvinepadze <wineson@gmail.com>
Co-authored-by: Ali Farooq <ali.farooq0@pm.me>
Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
Co-authored-by: Daniel Espino García <larkox@gmail.com>
Co-authored-by: Christopher Speller <crspeller@gmail.com>
Co-authored-by: Alex Dovenmuehle <adovenmuehle@gmail.com>
Co-authored-by: Szymon Gibała <szymongib@gmail.com>
Co-authored-by: Lev <1187448+levb@users.noreply.github.com>
Co-authored-by: Jason Frerich <jason.frerich@mattermost.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Artur M. Wolff <artur.m.wolff@gmail.com>
Co-authored-by: Madhav Hugar <16546715+madhavhugar@users.noreply.github.com>
Co-authored-by: Joe <security.joe@pm.me>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: José Peso <trilopin@users.noreply.github.com>
2023-08-21 09:50:30 +02:00
Agniva De Sarker
c249ba4a66
MM-52532: Fix golangci warnings (#23709)
https://mattermost.atlassian.net/browse/MM-52532

- Replace golint with revive
- Add makezero linter
- Fix all the required linter failures

Some issues in enterprise and public modules
are yet to be fixed. We send this to expediate things.
2023-06-13 14:08:36 +05:30
Agniva De Sarker
efaa6264cc
MM-53032: Fix module path after repo rename (#23689)
It was a good decision in hindsight to keep the public module as 0.x
because this would have been a breaking change again.

https://mattermost.atlassian.net/browse/MM-53032
```release-note
Changed the Go module path from github.com/mattermost/mattermost-server/server/v8 to github.com/mattermost/mattermost/server/v8.

For the public facing module, it's path is also changed from github.com/mattermost/mattermost-server/server/public to github.com/mattermost/mattermost/server/public
```
2023-06-11 10:54:35 +05:30
Vishal
fa1ba4717b
[MM-52527] (#23519)
* limit markdown size
2023-05-29 19:02:22 +05:30
Jesse Hallam
bb02b35048
Expose public/ API as submodule (#23345)
* model -> public/model

* plugin -> public/plugin

* public/model/utils -> public/utils

* platform/shared/mlog -> public/shared/mlog

* platform/shared/i18n -> public/shared/i18n

* platform/shared/markdown -> public/shared/markdown

* platform/services/timezones -> public/shared/timezones

* channels/einterfaces -> einterfaces

* expose public/ submodule

* go mod tidy

* .github: cache-dependency-path, setup-go-work

* modules-tidy for public/ too

* remove old gomodtidy
2023-05-10 13:07:02 -03:00