Commit graph

57 commits

Author SHA1 Message Date
Jesse Hallam
d4fc0ecb1c
MM-68150: Upgrade golangci-lint to v2.12.2 (#36554)
Some checks are pending
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres (shard 0) (push) Blocked by required conditions
Server CI / Postgres (shard 1) (push) Blocked by required conditions
Server CI / Postgres (shard 2) (push) Blocked by required conditions
Server CI / Postgres (shard 3) (push) Blocked by required conditions
Server CI / Merge Postgres Test Results (push) Blocked by required conditions
Server CI / Elasticsearch v8 Compatibility (push) Blocked by required conditions
Server CI / Postgres FIPS (shard 0) (push) Blocked by required conditions
Server CI / Postgres FIPS (shard 1) (push) Blocked by required conditions
Server CI / Postgres FIPS (shard 2) (push) Blocked by required conditions
Server CI / Postgres FIPS (shard 3) (push) Blocked by required conditions
Server CI / Merge Postgres FIPS Test Results (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Tools CI / check-style (mattermost-govet) (push) Waiting to run
Tools CI / Test (mattermost-govet) (push) Waiting to run
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Blocked by required conditions
Web App CI / check-external-links (push) Blocked by required conditions
Web App CI / check-types (push) Blocked by required conditions
Web App CI / test (platform) (push) Blocked by required conditions
Web App CI / test (mattermost-redux) (push) Blocked by required conditions
Web App CI / test (channels shard 1/4) (push) Blocked by required conditions
Web App CI / test (channels shard 2/4) (push) Blocked by required conditions
Web App CI / test (channels shard 3/4) (push) Blocked by required conditions
Web App CI / test (channels shard 4/4) (push) Blocked by required conditions
Web App CI / upload-coverage (push) Blocked by required conditions
Web App CI / build (push) Blocked by required conditions
YAML Lint / yamllint (push) Waiting to run
* Simplify invite_people email parsing

Replace backwards in-place mutation loop with a straightforward forward
filter into a new slice. Extract into parseEmailList so the logic can be
unit tested directly.

* MM-68150: Upgrade golangci-lint to v2.12.2

Remove //go:fix inline from NewPointer, which is a generic function not
yet supported by the inline analyzer, and fix 11 slicesbackward
modernize issues flagged by the new version.

* MM-68150: Enable all linters by default; disable those with >20 existing issues

Switch from opt-in (default: none) to opt-out (default: all) so new
linters added to golangci-lint are evaluated automatically. Explicitly
disable every linter that has more than 20 pre-existing violations,
deferring those for later cleanup. Also disable a handful of linters
whose violations are intentional patterns in this codebase (nilerr,
dogsled, sqlclosecheck, iotamixing, predeclared, containedctx, iface,
gocheckcompilerdirectives, promlinter, goprintffuncname, gomoddirectives).

* MM-68150: Fix mirror linter issues

Replace Write([]byte(s)) with WriteString(s), and FindIndex([]byte(s))
with FindStringIndex(s), to avoid unnecessary allocations.

* MM-68150: Fix nosprintfhostport linter issue

Use net.JoinHostPort to construct host:port strings instead of
fmt.Sprintf with a manually formatted pattern.

* MM-68150: Fix rowserrcheck and sqlclosecheck linter issues

Check rows.Err() after iteration loops in schema_dump.go. In the
sqlx_wrapper test, defer rows.Close() rather than closing inline.

* MM-68150: Fix nilnesserr linter issues — wrong variable in error handlers

In 11 places, a stale variable (often the outer err from a prior
assignment) was used instead of the freshly-checked error variable
(appErr, rowErr, jsonErr, writeErr, esErr). Each produces a typed-nil
wrapped in a non-nil interface, silently discarding the real error.

* MM-68150: Add i18n string for app.compile_csv_chunks.write_error

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2026-05-14 17:29:37 -04:00
Alejandro García Montoro
f604ec7a5c
MM-68662: Add Azure Blob Storage filestore backend (#36498)
* Generalize file backend error types

Replace S3FileBackendAuthError and S3FileBackendNoBucketError with
backend-agnostic FileBackendAuthError and FileBackendNoBucketError so
non-S3 drivers can return them and the admin "Test Connection" flow
keeps surfacing useful messages.

The old S3-prefixed names are kept as type aliases of the generic
types so external code (plugins, historical consumers) continues to
compile, and so existing S3 construction sites stay untouched.

The type switch in connectionTestErrorToAppError now matches the
generic types, with new i18n keys (test_connection_auth.app_error
and test_connection_no_bucket.app_error) whose wording does not name
S3. The old S3-specific i18n keys are dropped via `make i18n-extract`
since they are no longer referenced from code; the api4 test that
asserted on those keys is updated, and the Cypress
`MM-T996 Amazon S3 connection error messaging` spec that asserted
on the old user-facing string is updated to the new wording.

------
AI assisted commit

* Pull in Azure SDK and uuid dependencies

Bring in github.com/Azure/azure-sdk-for-go/sdk/azcore and
.../sdk/storage/azblob (with .../sdk/internal as their indirect
dependency). The two are needed by the upcoming Azure Blob Storage
filestore backend and its lazy-Range-backed reader. The bump of
golang.org/x/{crypto,net,sys,term,text} comes transitively from
azblob's minimum versions.

Also promotes github.com/google/uuid from indirect to direct,
since the Azure backend uses it to generate block IDs that share
the same wire format the SDK itself produces in UploadStream.

------
AI assisted commit

* Add azureRangeReader, a seekable Range-backed blob reader

A small standalone type that satisfies the FileBackend interface's
ReadCloseSeeker + the broader io.ReaderAt contract on top of Azure
Blob Storage HTTP Range requests. Lands as its own commit because
the upcoming Azure FileBackend driver builds on it, and the reader
itself is independently useful — and independently testable against
a fake downloader without standing up an Azure client.

Design notes:

* Read opens an HTTP Range stream lazily at the current offset and
  reuses it for sequential reads. Seek to a different offset closes
  the open stream; the next Read re-opens it.
* Seek to the same offset is a no-op and does not close the open
  stream, so callers like zip.NewReader that probe with redundant
  seeks don't kick off a fresh download.
* ReadAt issues a dedicated ranged DownloadStream per call and does
  not touch the streaming cursor — matches the io.ReaderAt contract
  the bulk-import worker's zip.NewReader path relies on.
* Close cancels the context (which any in-flight Azure call will
  observe and abort), stops the deadline timer, and closes the
  current body if any. It is safe to call when no body was ever
  opened.
* CancelTimeout lets long-running consumers like the import worker
  opt out of the per-operation deadline that would otherwise kill
  multi-minute downloads partway through.

The implementation talks to a small blobDownloader interface rather
than *blob.Client directly so the unit tests can substitute a fake
downloader that records every requested Range and tracks Close
calls on the bodies it hands out.

------
AI assisted commit

* Add Azure Blob Storage filestore driver

Implements the FileBackend interface against Azure Blob Storage in
a new azurestore.go (~520 LOC). The driver is not yet selectable
via NewFileBackend's switch — that wiring lands in the next commit
together with the admin config surface — but the driver itself is
complete and self-contained behind the FileBackendSettings struct.

Filesstore.go grows three pieces of supporting infrastructure that
the driver consumes:

* a `driverAzure = "azureblob"` constant alongside the existing
  driverS3 and driverLocal,
* an Azure-specific block on FileBackendSettings (storage account,
  access key, container, path prefix, endpoint, SSL flag, request
  timeout),
* a CheckMandatoryAzureFields validator that mirrors
  CheckMandatoryS3Fields.

Behavioural notes that warrant calling out:

* Reader returns the previously-added azureRangeReader, so reads
  stream lazily over HTTP Range and ReadAt is available for the
  bulk-import worker's zip.NewReader path. The deadline timer is
  armed before the initial GetProperties call so the HEAD itself
  is bounded.
* WriteFile and AppendFile both go through StageBlock +
  CommitBlockList via a shared stageBlocks helper, never the SDK's
  UploadStream. UploadStream's small-payload fast path falls back
  to single-shot PutBlob, which leaves the resulting blob with no
  committed block list; a subsequent AppendFile that calls
  CommitBlockList on that blob would then clobber its content.
  Routing every write through the block-list mechanism keeps
  AppendFile correct regardless of payload size.
* AppendFile stages the new chunk as one or more blocks and commits
  the existing committed block list plus the newly staged IDs.
  The new bytes go up exactly once — no re-download, no
  re-concatenate, no re-upload of the prior contents.
* WriteFileContext does not wrap the caller-supplied context with
  its own timeout — that timeout is applied in WriteFile only,
  matching the S3 driver, so long-running TryWriteFileContext
  callers (like message-export bulk writes) opt out of the
  per-operation timeout the way the abstraction documents.

Authentication is shared-key only for this drop; Microsoft Entra
ID / managed identity is deferred to a follow-up. The endpoint is
configurable so the same code targets the production Azure host
(vhost style — {account}.blob.core.windows.net) or Azurite /
Azure Government / sovereign clouds (path style —
host[:port]/{account}).

------
AI assisted commit

* Wire Azure backend into config, validation, and driver selection

This commit registers the previously-added AzureFileBackend driver
with the rest of the system. Until now the driver was usable only
via direct construction; after this commit, `DriverName: "azureblob"`
in config.json is a fully-supported deployment configuration.

Five integration sites are touched:

* `newFileBackend` in filesstore.go now dispatches `driverAzure` to
  NewAzureFileBackend, alongside the existing s3 and local cases.
  NewFileBackendSettingsFromConfig (and its export counterpart) gain
  an Azure branch that maps the model.FileSettings fields onto the
  Azure-specific FileBackendSettings fields.
* `model.FileSettings` grows the user-facing Azure config schema:
  storage account, access key, container, path prefix, endpoint,
  SSL flag, request timeout, plus matching Export* fields for the
  dedicated export store. SetDefaults populates them so deployments
  that never opted into Azure don't carry nil pointers. `isValid`
  accepts the new ImageDriverAzure constant.
* `Config.Sanitize()` masks AzureAccessKey and ExportAzureAccessKey
  the same way it masks AmazonS3SecretAccessKey, so the shared key
  never reaches an API consumer in plain text.
* `desanitize()` restores the masked keys on a config write so a
  PATCH that doesn't touch the key doesn't clobber it with the
  FakeSetting placeholder.
* `configSensitivePaths` covers both Azure key paths so audit
  diffs don't include them either.
* `ConfigToFileBackendSettings` in the `mattermost db` CLI helper
  gets the Azure branch its production counterpart already has —
  without it, `mattermost db migrate` / `db downgrade` would fail
  on Azure-configured deployments with "missing azure storage
  account setting".

Finally, the shared FileBackendTestSuite is now wired against
Azurite via TestAzureFileBackendTestSuite, which skips when
CI_AZURITE_HOST is unreachable. The test-infra wiring (the docker
service, the env vars, the start_dependencies entry) landed in a
previous PR; this commit is what makes the suite actually exercise
the Azure driver end to end.

------
AI assisted commit

* Validate Azure timeout and path prefix in Config.IsValid

Parity with the S3-side checks that already cover
AmazonS3RequestTimeoutMilliseconds and AmazonS3PathPrefix. Without
these, a zero/negative AzureRequestTimeoutMilliseconds passes
validation and later creates immediately-expired request contexts,
and leading/trailing whitespace in AzurePathPrefix produces blob
keys that don't match what the admin configured.

Same checks added for the Export* counterparts. The
file_driver.app_error translation is updated to mention the new
'azureblob' option alongside 'local' and 'amazons3'.

------
AI assisted commit

* Stream zip entries from the Azure backend

writeZipEntry was calling ReadFile, which loads the entire blob
into memory before writing it to the archive. For large blobs or
deep directories this spikes RSS or OOMs the goroutine. Switch to
Reader (the streaming azureRangeReader) and io.Copy into the zip
entry so memory stays bounded regardless of blob size.

------
AI assisted commit

* Use a backend-agnostic fallback for FileBackendNoBucketError

The fallback Error() message was "no such bucket", which leaks S3
terminology when an Azure caller returns the type with no wrapped
Err. Use "no such bucket or container" so logs and external error
handling stay neutral across backends.

------
AI assisted commit

* Defend Azure path prefix against directory traversal

Reject ".." in AzurePathPrefix and ExportAzurePathPrefix at config
validation time, since path.Join collapses traversal segments and a
prefix like "../other-tenant" would otherwise escape the configured
isolation boundary.

Harden the prefix helper as a second line of defense: if the joined
path no longer sits inside pathPrefix, fall back to joining the prefix
with the base name of the caller-supplied path. That preserves the
prefix invariant for plugin and import paths that the upload code does
not sanitize uniformly.

------
AI assisted commit

* Honor SkipVerify when constructing the Azure client

FileBackendSettings.SkipVerify is plumbed through from the System Console
the same way it is for S3, so admins toggling the flag for self-signed
endpoints (Azurite, sovereign clouds) get the behavior they expect
without having to drop SSL entirely and send the shared key in clear
text.

------
AI assisted commit

* Warn when the Azure request timeout falls back to its default

Config.IsValid already rejects non-positive AzureRequestTimeoutMilliseconds
for any path that goes through config validation, so this warn only fires
for direct callers that bypass validation (tests, helpers). Logging the
substitution turns a silent coercion into something an operator can
correlate against unexpected request behavior.

------
AI assisted commit

* Cap Azure request timeout at 10 minutes

Reject AzureRequestTimeoutMilliseconds values above the ceiling so an
operator (or someone who has admin access) cannot effectively disable
timeouts by setting the value to math.MaxInt64. A hung Azure call then
holds a goroutine open until the OS gives up.

Applies the same bound to ExportAzureRequestTimeoutMilliseconds. S3 has
the same gap; treating it is out of scope here but worth a follow-up.

------
AI assisted commit

* Refuse AppendFile on blobs without a committed block list

A blob written by another tool (Azure portal, azcopy, a migration script,
a plugin using Put Blob) has its content in the blob but an empty
committed-block list. Committing a new block list against such a blob
silently replaces the existing content with only the appended bytes.

Check the blob's properties before staging when the committed-block list
is empty, and refuse with a clear error if the blob has content. Same
hazard for an admin pointing the backend at an existing container with
pre-existing files.

Adds an integration test against Azurite to lock the behavior in.

------
AI assisted commit

* Surface truncated reads from azureRangeReader

Read closed the body cleanly and returned io.EOF even when the remote
stream terminated before the blob's content length. Callers (and any
retry layer above) then accepted a partial blob as complete.

ReadAt unconditionally rewrote io.ErrUnexpectedEOF to io.EOF, which made
truncated downloads indistinguishable from clean reads. That is exactly
what zip.NewReader consumes for archive readers, so the bulk-import
worker would silently import partial archives.

Read now closes the body, nils it, and returns io.ErrUnexpectedEOF when
EOF arrives before offset reaches size. ReadAt only collapses
ErrUnexpectedEOF to EOF when the full count was delivered and the stream
was consumed to the end of the blob. Otherwise the truncation
propagates with context.

Both code paths are exercised by new fakeDownloader-backed tests.

------
AI assisted commit

* Move container provisioning out of Azure TestConnection

Auto-creating the container inside TestConnection meant a typo in the
System Console (mattermosst instead of mattermost) silently provisioned
an unwanted container in the admin's Azure subscription, with no audit
log and no warning. They'd discover it later when uploads landed
somewhere unexpected.

TestConnection now returns FileBackendNoBucketError when the container
is missing, mirroring the S3 contract. A new MakeContainer method
mirrors S3FileBackend.MakeBucket, and Server.Start dispatches via two
capability interfaces (bucketMaker / containerMaker) instead of a hard
S3 type assertion — so the NoBucket error is no longer silently
swallowed for backends Server.Start has not been taught about.

------
AI assisted commit

* Carry file backend auth detail through to AppError

The Test Connection button collapsed every typed backend failure into
the same generic i18n message. Operators trying to debug bad credentials
or a missing bucket only saw "Unable to authenticate against the file
storage backend" with no SDK code to grep for in their logs.

Use errors.As so the typed checks survive future wrapping, and pass the
underlying error string through the NewAppError details argument. The
AppError serializer surfaces that detail to the admin console alongside
the translated message, so a bad S3 InvalidAccessKeyId or an Azure
AuthenticationFailed shows up in the toast without an i18n schema
change.

------
AI assisted commit

* Remove non-ascii characters from comments

------
AI assisted commit

* Make linter happy

------
AI assisted commit

* Harden Azure prefix boundary check

strings.HasPrefix on the joined path is a string-level check, not a
path-level one, so a configured prefix of "mattermost" accepts a joined
result of "mattermost-evil/...". A crafted caller path like
"../mattermost-evil/secrets" would collapse via path.Join to that exact
sibling and slip through the boundary check, escaping the configured
prefix scope.

Require the joined path to be the cleaned prefix itself or to start with
the prefix followed by a path separator. The fallback path.Join uses the
same cleaned prefix for consistency.

------
AI assisted commit

* Provision Azurite container in standalone test setup

The shared FileBackendTestSuite's SetupTest already handles a missing
container by detecting FileBackendNoBucketError from TestConnection and
calling MakeContainer, but TestAzureFileBackendAppendRefusesNonBlockBlob
bypasses SetupTest and calls TestConnection directly. On a fresh Azurite
instance the test would fail before exercising the append-refusal logic.

Extract a newAzuriteBackend(t) helper alongside azuriteSettings(t) that
builds the backend and ensures the container exists, mirroring the
suite's setup. Use errors.As for forward compatibility with future
wrapping.

------
AI assisted commit

* Fix grammar in email-settings i18n string

"Email settings has unset values." -> "Email settings have unset values."

------
AI assisted commit

* Make Azure MakeContainer idempotent

Treat a ContainerAlreadyExists response as success so that two nodes
racing through TestConnection plus MakeContainer at boot both converge
instead of having the loser fail. Mirrors how the S3 backend handles
the equivalent BucketAlreadyOwnedByYou case.

------
AI assisted commit

* Narrow AzureEndpoint comment to path-style only

The setting only builds path-style URLs, so it cannot reach sovereign
clouds like Azure Government or Azure China, which require vhost-style
endpoints. Update the comment to reflect what the code actually does
and document that sovereign-cloud support is out of scope.

------
AI assisted commit
2026-05-14 16:59:18 +00:00
Jesse Hallam
e3fbf8711f
MM-68149: Upgrade to Go 1.26.2 (#36418)
* MM-68149: upgrade to Go 1.26.2

Update go directive in go.mod and .go-version.

* MM-68149: replace pointer helpers with Go 1.26 new()

Go 1.26 extends the built-in new() to accept an initial value expression,
making typed-pointer helpers like model.NewPointer(x), bToP(x), and boolPtr(x)
redundant. Replace every call site with new(x) and remove the now-unused
helper functions and their //go:fix inline directives.

* MM-68149: apply go fix for reflect API and format-string changes

- reflect.Ptr → reflect.Pointer (renamed in Go 1.18, deprecated alias removed in 1.26)
- reflect range-over-struct: for i := 0; i < t.NumField(); i++ → for field := range t.Fields()
  and the equivalent for Methods() and interface types
- Fix format-string concatenation and variadic-arg mismatches flagged by go vet

* MM-68149: update JPEG fixtures and test infrastructure for Go 1.26 encoder

Go 1.26 ships a new image/jpeg encoder that produces slightly different output.
Regenerate all JPEG fixture files and switch the comparison helpers from
byte-equality to pixel-level comparison with a small per-channel tolerance,
so minor encoder drift across patch versions is handled automatically.

Add -update-fixtures flag to make it easy to regenerate fixtures after future
major Go upgrades. Document the update procedure in tests/README.md.

* MM-68149: CI check that go fix ./... produces no changes

* Fix real bugs flagged by CodeRabbit review

- group.go: set newGroup.MemberCount not group.MemberCount (member count
  was populated on the wrong variable and lost before publish/return)
- file_test.go: guard compareImage(GetFilePreview) on the preview slice
  length, not the thumbnail slice length (copy-paste error)
- config_test.go: remove duplicate MinimumLength assignment

* fixup! Fix real bugs flagged by CodeRabbit review
2026-05-12 15:59:12 +00:00
Carlos Garcia
a0326c91ac
[MM-66203] removes direct jaytaylor/html2text dependency (#34539)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Waiting to run
Web App CI / check-types (push) Waiting to run
Web App CI / test (push) Waiting to run
Web App CI / build (push) Waiting to run
* removes direct jaytaylor/html2text dependency

there is still some indirect dependency on the library preventing
to use latest tablewriter with a PR made to the outdated library
that should be monitored as stated in go.mod comments.

* makes variable not shadow outer one

* fixes typo and makes test fail on error

* uses current docconv dependency to generate plain text email content
2025-12-04 17:15:01 +01:00
Jesse Hallam
61db53dd1a
[MM-66718] Remove unneeded HTML templates watcher (#34557)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Waiting to run
Web App CI / check-types (push) Waiting to run
Web App CI / test (push) Waiting to run
Web App CI / build (push) Waiting to run
* [MM-66718] Remove unneeded HTML templates watcher

The templates package currently supports filesystem watching to
automatically reload templates when files change. This feature is
unnecessary in production and adds complexity.

Changes:
- Removed NewWithWatcher() function from templates package
- Removed Close() method from Container
- Removed watch-related fields (watch, stop, stopped) from Container
- Removed fsnotify dependency usage
- Updated server.go to use New() instead of NewWithWatcher()
- Updated email/helper_test.go to use New()
- Removed watcher-related tests from templates_test.go

Template updates now require a server restart, which provides clearer
behavior and reduces code complexity.

* Remove unused fsnotify dependency
2025-11-27 16:24:59 -04:00
Carlos Garcia
4ba7f7e16e
MM-66202: Migrate to aws-sdk-go-v2 (#34496)
* updated aws-sdk dependency to aws-sdk-go-v2

* simplify error handling in case of timeout errors

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-20 11:26:09 +01:00
Jesse Hallam
06b1bf3a51
MM-64878: FIPS Build (#33809)
* pin to ubuntu-24.04

* always use FIPS compatible Postgres settings

* use sha256 for remote cluster IDs

* use sha256 for client config hash

* rework S3 backend to be FIPS compatible

* skip setup-node during build, since already in container

* support FIPS builds

* Dockerfile for FIPS image, using glibc-openssl-fips

* workaround entrypoint inconsistencies

* authenticate to DockerHub

* fix FIPS_ENABLED, add test-mmctl-fips

* decouple check-mattermost-vet from test/build steps

* fixup! decouple check-mattermost-vet from test/build steps

* only build-linux-amd64 for fips

* rm entrypoint workaround

* tweak comment grammar

* rm unused Dockerfile.fips (for now)

* ignore gpg import errors, since would fail later anyway

* for fips, only make package-linux-amd64

* set FIPS_ENABLED for build step

* Add a FIPS-specific list of prepackaged plugins

Note that the names are still temporary, since they are not uploaded to
S3 yet. We may need to tweak them when that happens.

* s/golangci-lint/check-style/

This ensures we run all the `check-style` checks: previously,
`modernize` was missing.

* pin go-vet to @v2, remove annoying comment

* add -fips to linux-amd64.tz.gz package

* rm unused setup-chainctl

* use BUILD_TYPE_NAME instead

* mv fips build to enterprise-only

* fixup! use BUILD_TYPE_NAME instead

* temporarily pre-package no plugins for FIPS

* split package-cleanup

* undo package-cleanup, just skip ARM, also test

* skip arm for FIPS in second target too

* fmt Makefile

* Revert "rm unused Dockerfile.fips (for now)"

This reverts commit 601e37e0ff.

* reintroduce Dockerfile.fips and align with existing Dockerfile

* s/IMAGE/BUILD_IMAGE/

* bump the glibc-openssl-fips version

* rm redundant comment

* fix FIPS checks

* set PLUGIN_PACKAGES empty until prepackaged plugins ready

* upgrade glibc-openssl-fips, use non-dev version for final stage

* another BUILD_IMAGE case

* Prepackage the FIPS versions of plugins

* relocate FIPS_ENABLED initialization before use

* s/Config File MD5/Config File Hash/

* Update the FIPS plugin names and encode the + sign

* add /var/tmp for local socket manipulation

---------

Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-09-15 10:53:28 -03:00
Ben Schumacher
9add320011
[MM-64654] Migrate to modern Go features (#31820) 2025-07-18 12:54:51 +02:00
Jesse Hallam
e6d8bf5835
Upgrade Go to 1.24.3 (#31220)
* Upgrade Go to 1.24.3

Updates the following files:
- server/.go-version: 1.23.9 → 1.24.3
- server/build/Dockerfile.buildenv: golang:1.23.9-bullseye → golang:1.24.3-bullseye
- server/go.mod: go 1.23.0 → go 1.24.3, toolchain go1.23.9 → go1.24.3
- server/public/go.mod: go 1.23.0 → go 1.24.3, toolchain go1.23.9 → go1.24.3

Also fixes non-constant format string errors introduced by Go 1.24.3's stricter format string checking:
- Added response() helper function in slashcommands/util.go for simple string responses
- Removed unused responsef() function from slashcommands/util.go
- Replaced responsef() with response() for translated strings that don't need formatting
- Fixed fmt.Errorf and fmt.Fprintf calls to use proper format verbs instead of string concatenation
- Updated marketplace buildURL to handle format strings conditionally

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Update generated mocks for Go 1.24.3

Regenerated mocks using mockery v2.53.4 to ensure compatibility with Go 1.24.3.
This addresses mock generation failures that occurred with the Go upgrade.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Update to bookworm and fix non-existent sha

Signed-off-by: Stavros Foteinopoulos <stafot@gmail.com>

* fix non-constant format string

---------

Signed-off-by: Stavros Foteinopoulos <stafot@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Stavros Foteinopoulos <stafot@gmail.com>
2025-06-10 15:04:57 -03:00
Ben Schumacher
0cf6361139
[MM-63578] Fix support packet caching issue (#31133)
Fix support packet caching issue by adding no-cache headers

Added Cache-Control headers to prevent browser caching when downloading support packets.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-10 11:39:06 +02:00
kasyap dharanikota
596053b9af
S3store/image type (#30451)
* add video type

* add video mime-type

* combine MIME type maps and update tests

* correct MIME type detection for video files

* format code

* lint fix

* simplified mimeType

* change avi mime type

* add avi type
2025-06-09 09:09:34 +05:30
Claudio Costa
d66fbd1425
Implement BenchmarkFileStore (#30524) 2025-03-25 11:42:13 -06:00
Alejandro García Montoro
350714f390
Bump Go to v1.23.7 (#30455)
* Update Go version to v1.23.7

* Bump golangci-lint to a version supporting Go 1.23

* Fix golangci-lint warnings

Several rules from gosimple, revive and staticcheck linters were
failing:
- Redefinition of built-in identifiers (max, min, new, recover...)
- Use of printf-like functions with simple strings
- Check for nil slices, when len already takes it into account

* Trigger Build

* Trigger Build

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-03-17 14:22:07 +01:00
Agniva De Sarker
ac10bb12a5
Revert "Bump Go version to 1.23.6 (#30242)" (#30359)
This reverts commit acbbd4c58d.
2025-02-27 14:58:51 +05:30
Alejandro García Montoro
acbbd4c58d
Bump Go version to 1.23.6 (#30242)
* Bump Go version to 1.23.6

* Update CodeQL Github action as well

* Use server's Go version for CodeQL action

Co-authored-by: Antonis Stamatiou <stamatiou.antonis@gmail.com>

* Empty commit to trigger CI

* Bump golangci-lint to a version supporting Go 1.23

* Fix golangci-lint warnings

Several rules from gosimple, revive and staticcheck linters were
failing:
- Redefinition of built-in identifiers (max, min, new, recover...)
- Use of printf-like functions with simple strings
- Check for nil slices, when len already takes it into account

---------

Co-authored-by: Antonis Stamatiou <stamatiou.antonis@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-02-26 16:43:04 +01:00
Ibrahim Serdar Acikgoz
a7905541b8
[MM-62660] Makefile: add ability to run local file store tests as standalone (#30260)
* server/Makefile: add ability to test&benchmark local file store only

* remove benchmark
2025-02-19 10:18:07 +01:00
Agniva De Sarker
c4718e4542
MM-60211: Bump dependencies (#30005)
https://mattermost.atlassian.net/browse/MM-60211
```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-02-06 21:48:08 +05:30
Christopher Poile
737bed311c
MM-60115, MM-62436, MM-62493: Compliance export downloads from local and s3; e2e tests (#29806)
* add ZipReader method to filestore and file/s3 backends

to merge

to merge

* add WriteStreamResponse as an alternative to WriteFileResponse

* add generated layers

* enable download link; download job endpoint now streams export dir zips

* fix MM-62493

* re-enable e2e tests--we have download links, folks

* Add tests for ZipReader in filestore and s3store

* remove unnecessary error return on ZipReader

* little cleanup

* improve tests; some refactoring: s.Nil(err) -> s.NoError(err)

* blank commit

* backwards compatability for pre-10.5 job downloads

* compress file response; better errors; better comments; PR comments

* update generated app layers

* improve/widen tests; improve comments; simplify localstore ZipReader

* regenerate layers

* follow GoDoc conventions

* update generated layers

* remove unnecessary comment

* in jobs/job-id/download, clean exportDir before sending to ZipReader

* better comments; add an error return on ZipReader

* improve file permissions

* adjust tests for new error returns

* linting

* i18n
2025-01-27 09:28:07 +05:30
Agniva De Sarker
53d6f59ff5
shared/filestore: Add detailed error context for S3.TestConnection (#29838)
We were just returning a string without embedding the
actual error returned. This was making things difficult
to debug.

```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-17 20:11:22 +05:30
Nadav Tasher
5e47c97db4
Added support for S3 storage classes (#28319)
* Added support for S3 storage classes

* Added missing translations for S3 storage class

* Changed default storage class values to preserve original behaviour

* Changed storage class description and example

* Fix translations ordering

* Change configuration defaults to empty strings

* Remove redundant empty string check

* Validate storage class variable against constants from S3 API docs

* Validate export storage class against constants

* Use slices for config validation

* Applied patch for translation ordering
2024-11-13 12:29:44 +05:30
Nicolas Le Cam
c90e562528
Migrate mockery to packages feature (#29013) 2024-11-07 12:48:11 +01:00
enzowritescode
991debeb0b
Change log level inside sendMail() (#28669)
* Change log level inside sendMail()
2024-10-09 14:01:01 -06:00
Julien Tant
d1cad8c692
add reuse check on update MFA (#28304) 2024-09-27 17:35:36 +02:00
Julien Tant
1909206e16
[MM-59069] Make sure OTP are actual One Time Password (#28074)
Automatic Merge
2024-09-17 00:44:32 +02:00
Agniva De Sarker
c3ed07e679
OSF: Used model.NewPointer everywhere (#27838)
```release-note
NONE
```
2024-08-06 09:15:00 +05:30
Agniva De Sarker
d20b55bde8
MM-57375: Update to latest minio image (#27475)
* test with latest minio image
* Update the KMS key
- Also use latest config settings. The older ones were deprecated.
2024-06-28 20:34:18 +05:30
Aditya Pratap Singh Hada
075e0df5df
[MM-57034]: Check for s3 object is File or Directory (#26837)
* check for path when number of object is one

* add test and updated condition to check for path

* updated test and removed string trim

* using exported method

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-11 17:32:57 +05:30
Jesse Hallam
630bd40141
Eliminate MM_SERVER_PATH (#24968) 2024-05-10 18:13:05 -03:00
Robert Obermeier
322a500092
Add WebP to MediaFormats #26628 (#26805)
Changes the default content disposition for WebP attachments
from *download* to *inline*.

Add WebP codec support side-effect to decode.go. To prevent future
issues caused by possible changes in emoji.go.
2024-04-26 17:06:07 +02:00
Agniva De Sarker
870cee30c6
MM-57759: Added mockery support for 1.22 (#26774)
* Revert "Revert "MM-57759: Bump mockery to version 2.42.2 to support go  1.22^" (#26772)"

This reverts commit cd3b5b46e1.

* Added the hooks.go file changes as well

```release-note
NONE
```
2024-04-13 08:15:59 +05:30
Agniva De Sarker
cd3b5b46e1
Revert "MM-57759: Bump mockery to version 2.42.2 to support go 1.22^" (#26772) 2024-04-12 15:31:24 +02:00
Ezekiel
98712737e6
MM-57759: Bump mockery to version 2.42.2 to support go 1.22^ (#26761) 2024-04-12 10:38:34 +02:00
Claudio Costa
bc6182229b
Make S3 upload part size configurable (#26533) 2024-04-05 07:20:48 -06:00
Dahlton S
fd713ae9bb
[MM-56651] fix: set cache duration (#26338)
* fix: set cache duration

* fix: extra 0
2024-03-05 22:49:26 +05:30
Matthew Straughn
1e794136c8
MM 19654 s3 debugging logs overlap, Wrapped mlog in an io.Writer func (#25788)
* updated go mod and sum files

* Revert "updated go mod and sum files"

This reverts commit 088dd00a84.

* wrapped mlog in and io.Writer func to pass to TraceOn

* fixed var and func receiver names

* removed the underscore in the func receiver.

* passed a pointer of s3Trace to TraceOn

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-02-20 07:59:26 +05:30
Ibrahim Serdar Acikgoz
0f3553c8ab
shared/filestore/localstore.go: remove unnecessary warning (#25801) 2023-12-22 19:26:57 +01:00
Ben Schumacher
612e4458ef
[MM-55409] Enable revive linter on enterprise code (#25391) 2023-11-10 15:24:12 +01:00
Ben Schumacher
0d5a8b8841
Add whitespace linter (#24855) 2023-10-11 10:13:36 +02:00
Ben Schumacher
30b12f199b
[MM-54132] Use annotated logger for log messages from jobs (#24275) 2023-09-07 08:50:22 +02:00
Ben Schumacher
b2f36c7cdf
[MM-54318] Add file storage information to support package (#24474)
* Reorder stats in support package struct

* Add file storage information to support package

* Add clusterID

* Change method name to DriverName

* Ordering

* Fix test
2023-09-06 09:21:25 +02:00
Ben Schumacher
3ee5432664
[MM-53968] Includes mattermost-plugin-api into the mono repo (#24235)
Include https://github.com/mattermost/mattermost-plugin-api into the mono repo

Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Michael Kochell <mjkochell@gmail.com>
Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Alex Dovenmuehle <alex.dovenmuehle@mattermost.com>
Co-authored-by: Michael Kochell <6913320+mickmister@users.noreply.github.com>
Co-authored-by: Christopher Poile <cpoile@gmail.com>
Co-authored-by: İlker Göktuğ Öztürk <ilkergoktugozturk@gmail.com>
Co-authored-by: Shota Gvinepadze <wineson@gmail.com>
Co-authored-by: Ali Farooq <ali.farooq0@pm.me>
Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
Co-authored-by: Daniel Espino García <larkox@gmail.com>
Co-authored-by: Christopher Speller <crspeller@gmail.com>
Co-authored-by: Alex Dovenmuehle <adovenmuehle@gmail.com>
Co-authored-by: Szymon Gibała <szymongib@gmail.com>
Co-authored-by: Lev <1187448+levb@users.noreply.github.com>
Co-authored-by: Jason Frerich <jason.frerich@mattermost.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Artur M. Wolff <artur.m.wolff@gmail.com>
Co-authored-by: Madhav Hugar <16546715+madhavhugar@users.noreply.github.com>
Co-authored-by: Joe <security.joe@pm.me>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: José Peso <trilopin@users.noreply.github.com>
2023-08-21 09:50:30 +02:00
Julien Tant
d1691833ad
[MM-53881] Make sure export filestore won't use bifrost (#24169) 2023-08-02 09:44:20 -07:00
Jesse Hallam
933b0eb89c
MM-53877: fix export config settings (#24167)
* wip test

* test and use Export* config correctly
2023-08-02 10:24:52 -03:00
Agniva De Sarker
6d6e589c11
MM-53747: Create job to encode older image paths (#24073)
Bifrost now encodes all image paths. Due to this
one-way translation, we need to encode all the older
image paths as well.

After this is done, we can remove the double-lookup.

https://mattermost.atlassian.net/browse/MM-53747

```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-07-25 08:38:35 +05:30
Agniva De Sarker
feeb0cd2ef
MM-53747: Bifrost: write original paths (#24079)
Now that we have identified the true fix
from Bifrost side, we will slowly revert
the changes that have happened.

Starting with this first fix, we will revert
to writing out non-encoded paths while still
doing the double read. This will break uploading
files with + for a short while, and then
the new Bifrost fix should fix that as well.

https://mattermost.atlassian.net/browse/MM-53747

```release-note
NONE
```
2023-07-21 08:59:59 +05:30
Julien Tant
077c16ef61
[MM-53454] Add export file settings + slash command for public link (#23915)
Co-authored-by: Mattermost Build <build@mattermost.com>
2023-07-19 13:01:39 -07:00
Agniva De Sarker
c1dfdb867a
MM-53709: Handle older non-encoded paths in bifrost (#24058)
We do a check to see if a non-encoded path is present
or not, and in that case, choose not to encode it.

This accrues an additional StatFile call. But after
we have encoded all paths to the new style, we will
get rid of this.

https://mattermost.atlassian.net/browse/MM-53709

```release-note
NONE
```
2023-07-19 22:32:04 +05:30
Agniva De Sarker
71c5a8512c
MM-51753: URL encode the file path in S3 uploads (#23849)
If the request is routed via bifrost, then we need to encode the path
to avoid signature validation errors.
This happens because in bifrost, we are signing the URL outside the SDK
and therefore the signature sent from the bifrost client
will contain the encoded path, whereas the original path is sent un-encoded.

More info at: a57c4d9278/aws/signer/v4/v4.go (L8)

https://mattermost.atlassian.net/browse/MM-51753
```release-note
NONE
```
2023-06-27 09:54:53 +05:30
Agniva De Sarker
c249ba4a66
MM-52532: Fix golangci warnings (#23709)
https://mattermost.atlassian.net/browse/MM-52532

- Replace golint with revive
- Add makezero linter
- Fix all the required linter failures

Some issues in enterprise and public modules
are yet to be fixed. We send this to expediate things.
2023-06-13 14:08:36 +05:30
Agniva De Sarker
efaa6264cc
MM-53032: Fix module path after repo rename (#23689)
It was a good decision in hindsight to keep the public module as 0.x
because this would have been a breaking change again.

https://mattermost.atlassian.net/browse/MM-53032
```release-note
Changed the Go module path from github.com/mattermost/mattermost-server/server/v8 to github.com/mattermost/mattermost/server/v8.

For the public facing module, it's path is also changed from github.com/mattermost/mattermost-server/server/public to github.com/mattermost/mattermost/server/public
```
2023-06-11 10:54:35 +05:30