mattermost/server/config/utils.go
Jesse Hallam 41e5c7286b
Remove vestigial MySQL support (#34865)
* Remove legacy quoteColumnName() utility

Since Mattermost only supports PostgreSQL, the quoteColumnName() helper
that was designed to handle database-specific column quoting is no longer
needed. The function was a no-op that simply returned the column name
unchanged.

Remove the function from utils.go and update status_store.go to use
the "Manual" column name directly.

* Remove legacy driver checks from store.go

Since Mattermost only supports PostgreSQL, remove conditional checks
for different database drivers:

- Simplify specialSearchChars() to always return PostgreSQL-compatible chars
- Remove driver check from computeBinaryParam()
- Remove driver check from computeDefaultTextSearchConfig()
- Simplify GetDbVersion() to use PostgreSQL syntax directly
- Remove switch statement from ensureMinimumDBVersion()
- Remove unused driver parameter from versionString()

* Remove MySQL alternatives for batch delete operations

Since Mattermost only supports PostgreSQL, remove the MySQL-specific
DELETE...LIMIT syntax and keep only the PostgreSQL array-based approach:

- reaction_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch
- file_info_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch
- preference_store.go: Use PostgreSQL tuple IN subquery for DeleteInvalidVisibleDmsGms

* Remove MySQL alternatives for UPDATE...FROM syntax

Since Mattermost only supports PostgreSQL, remove the MySQL-specific
UPDATE syntax that joins tables differently:

- thread_store.go: Use PostgreSQL UPDATE...FROM syntax in
  MarkAllAsReadByChannels and MarkAllAsReadByTeam
- post_store.go: Use PostgreSQL UPDATE...FROM syntax in deleteThreadFiles

* Remove MySQL alternatives for JSON and subquery operations

Since Mattermost only supports PostgreSQL, remove the MySQL-specific
JSON and subquery syntax:

- thread_store.go: Use PostgreSQL JSONB operators for updating participants
- access_control_policy_store.go: Use PostgreSQL JSONB @> operator for
  querying JSON imports
- session_store.go: Use PostgreSQL subquery syntax for Cleanup
- job_store.go: Use PostgreSQL subquery syntax for Cleanup

* Remove MySQL alternatives for CTE queries

Since Mattermost only supports PostgreSQL, simplify code that
uses CTEs (Common Table Expressions):

- channel_store.go: Remove MySQL CASE-based fallback in
  UpdateLastViewedAt and use PostgreSQL CTE exclusively
- draft_store.go: Remove driver checks in DeleteEmptyDraftsByCreateAtAndUserId,
  DeleteOrphanDraftsByCreateAtAndUserId, and determineMaxDraftSize

* Remove driver checks in migrate.go and schema_dump.go

Simplify migration code to use PostgreSQL driver directly since
PostgreSQL is the only supported database.

* Remove driver checks in sqlx_wrapper.go

Always apply lowercase named parameter transformation since PostgreSQL
is the only supported database.

* Remove driver checks in user_store.go

Simplify user store functions to use PostgreSQL-only code paths:
- Remove isPostgreSQL parameter from helper functions
- Use LEFT JOIN pattern instead of subqueries for bot filtering
- Always use case-insensitive LIKE with lower() for search
- Remove MySQL-specific role filtering alternatives

* Remove driver checks in post_store.go

Simplify post_store.go to use PostgreSQL-only code paths:
- Inline getParentsPostsPostgreSQL into getParentsPosts
- Use PostgreSQL TO_CHAR/TO_TIMESTAMP for date formatting in analytics
- Use PostgreSQL array syntax for batch deletes
- Simplify determineMaxPostSize to always use information_schema
- Use PostgreSQL jsonb subtraction for thread participants
- Always execute RefreshPostStats (PostgreSQL materialized views)
- Use materialized views for AnalyticsPostCountsByDay
- Simplify AnalyticsPostCountByTeam to always use countByTeam

* Remove driver checks in channel_store.go

Simplify channel_store.go to use PostgreSQL-only code paths:
- Always use sq.Dollar.ReplacePlaceholders for UNION queries
- Use PostgreSQL LEFT JOIN for retention policy exclusion
- Use PostgreSQL jsonb @> operator for access control policy imports
- Simplify buildLIKEClause to always use LOWER() for case-insensitive search
- Simplify buildFulltextClauseX to always use PostgreSQL to_tsvector/to_tsquery
- Simplify searchGroupChannelsQuery to use ARRAY_TO_STRING/ARRAY_AGG

* Remove driver checks in file_info_store.go

Simplify file_info_store.go to use PostgreSQL-only code paths:
- Always use PostgreSQL to_tsvector/to_tsquery for file search
- Use file_stats materialized view for CountAll()
- Use file_stats materialized view for GetStorageUsage() when not including deleted
- Always execute RefreshFileStats() for materialized view refresh

* Remove driver checks in attributes_store.go

Simplify attributes_store.go to use PostgreSQL-only code paths:
- Always execute RefreshAttributes() for materialized view refresh
- Remove isPostgreSQL parameter from generateSearchQueryForExpression
- Always use PostgreSQL LOWER() LIKE LOWER() syntax for case-insensitive search

* Remove driver checks in retention_policy_store.go

Simplify retention_policy_store.go to use PostgreSQL-only code paths:
- Remove isPostgres parameter from scanRetentionIdsForDeletion
- Always use pq.Array for scanning retention IDs
- Always use pq.Array for inserting retention IDs
- Remove unused json import

* Remove driver checks in property stores

Simplify property_field_store.go and property_value_store.go to use
PostgreSQL-only code paths:
- Always use PostgreSQL type casts (::text, ::jsonb, ::bigint, etc.)
- Remove isPostgres variable and conditionals

* Remove driver checks in channel_member_history_store.go

Simplify PermanentDeleteBatch to use PostgreSQL-only code path:
- Always use ctid-based subquery for DELETE with LIMIT

* Remove remaining driver checks in user_store.go

Simplify user_store.go to use PostgreSQL-only code paths:
- Use LEFT JOIN for bot exclusion in AnalyticsActiveCountForPeriod
- Use LEFT JOIN for bot exclusion in IsEmpty

* Simplify fulltext search by consolidating buildFulltextClause functions

Remove convertMySQLFullTextColumnsToPostgres and consolidate
buildFulltextClause and buildFulltextClauseX into a single function
that takes variadic column arguments and returns sq.Sqlizer.

* Simplify SQL stores leveraging PostgreSQL-only support

- Simplify UpdateMembersRole in channel_store.go and team_store.go
  to use UPDATE...RETURNING instead of SELECT + UPDATE
- Simplify GetPostReminders in post_store.go to use DELETE...RETURNING
- Simplify DeleteOrphanedRows queries by removing MySQL workarounds
  for subquery locking issues
- Simplify UpdateUserLastSyncAt to use UPDATE...FROM...RETURNING
  instead of fetching user first then updating
- Remove MySQL index hint workarounds in ORDER BY clauses
- Update outdated comments referencing MySQL
- Consolidate buildFulltextClause and remove convertMySQLFullTextColumnsToPostgres

* Remove MySQL-specific test artifacts

- Delete unused MySQLStopWords variable and stop_word.go file
- Remove redundant testSearchEmailAddressesWithQuotes test
  (already covered by testSearchEmailAddresses)
- Update comment that referenced MySQL query planning

* Remove MySQL references from server code outside sqlstore

- Update config example and DSN parsing docs to reflect PostgreSQL-only support
- Remove mysql:// scheme check from IsDatabaseDSN
- Simplify SanitizeDataSource to only handle PostgreSQL
- Remove outdated MySQL comments from model and plugin code

* Remove MySQL references from test files

- Update test DSNs to use PostgreSQL format
- Remove dead mysql-replica flag and replicaFlag variable
- Simplify tests that had MySQL/PostgreSQL branches

* Update docs and test config to use PostgreSQL

- Update mmctl config set example to use postgres driver
- Update test-config.json to use PostgreSQL DSN format

* Remove MySQL migration scripts, test data, and docker image

Delete MySQL-related files that are no longer needed:
- ESR upgrade scripts (esr.*.mysql.*.sql)
- MySQL schema dumps (mattermost-mysql-*.sql)
- MySQL replication test scripts (replica-*.sh, mysql-migration-test.sh)
- MySQL test warmup data (mysql_migration_warmup.sql)
- MySQL docker image reference from mirror-docker-images.json

* Remove MySQL references from webapp

- Simplify minimumHashtagLength description to remove MySQL-specific configuration note
- Remove unused HIDE_MYSQL_STATS_NOTIFICATION preference constant
- Update en.json i18n source file

* clean up e2e-tests

* rm server/tests/template.load

* Use teamMemberSliceColumns() in UpdateMembersRole RETURNING clause

Refactor to use the existing helper function instead of hardcoding
the column names, ensuring consistency if the columns are updated.

* u.id -> u.Id

* address code review feedback

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2026-01-20 21:01:59 +00:00

237 lines
8 KiB
Go

// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
// See LICENSE.txt for license information.
package config
import (
"bytes"
"encoding/json"
"fmt"
"reflect"
"strings"
"github.com/mattermost/mattermost/server/public/model"
"github.com/mattermost/mattermost/server/public/shared/i18n"
"github.com/mattermost/mattermost/server/public/shared/mlog"
"github.com/mattermost/mattermost/server/v8/channels/utils"
)
// marshalConfig converts the given configuration into JSON bytes for persistence.
func marshalConfig(cfg *model.Config) ([]byte, error) {
return json.MarshalIndent(cfg, "", " ")
}
// desanitize replaces fake settings with their actual values.
func desanitize(actual, target *model.Config) {
if target.LdapSettings.BindPassword != nil && *target.LdapSettings.BindPassword == model.FakeSetting {
*target.LdapSettings.BindPassword = *actual.LdapSettings.BindPassword
}
if *target.FileSettings.PublicLinkSalt == model.FakeSetting {
*target.FileSettings.PublicLinkSalt = *actual.FileSettings.PublicLinkSalt
}
if *target.FileSettings.AmazonS3SecretAccessKey == model.FakeSetting {
target.FileSettings.AmazonS3SecretAccessKey = actual.FileSettings.AmazonS3SecretAccessKey
}
if *target.EmailSettings.SMTPPassword == model.FakeSetting {
target.EmailSettings.SMTPPassword = actual.EmailSettings.SMTPPassword
}
if *target.GitLabSettings.Secret == model.FakeSetting {
target.GitLabSettings.Secret = actual.GitLabSettings.Secret
}
if target.GoogleSettings.Secret != nil && *target.GoogleSettings.Secret == model.FakeSetting {
target.GoogleSettings.Secret = actual.GoogleSettings.Secret
}
if target.Office365Settings.Secret != nil && *target.Office365Settings.Secret == model.FakeSetting {
target.Office365Settings.Secret = actual.Office365Settings.Secret
}
if target.OpenIdSettings.Secret != nil && *target.OpenIdSettings.Secret == model.FakeSetting {
target.OpenIdSettings.Secret = actual.OpenIdSettings.Secret
}
if *target.SqlSettings.DataSource == model.FakeSetting {
*target.SqlSettings.DataSource = *actual.SqlSettings.DataSource
}
if *target.SqlSettings.AtRestEncryptKey == model.FakeSetting {
target.SqlSettings.AtRestEncryptKey = actual.SqlSettings.AtRestEncryptKey
}
if *target.ElasticsearchSettings.Password == model.FakeSetting {
*target.ElasticsearchSettings.Password = *actual.ElasticsearchSettings.Password
}
if len(target.SqlSettings.DataSourceReplicas) == len(actual.SqlSettings.DataSourceReplicas) {
for i, value := range target.SqlSettings.DataSourceReplicas {
if value == model.FakeSetting {
target.SqlSettings.DataSourceReplicas[i] = actual.SqlSettings.DataSourceReplicas[i]
}
}
}
if len(target.SqlSettings.DataSourceSearchReplicas) == len(actual.SqlSettings.DataSourceSearchReplicas) {
for i, value := range target.SqlSettings.DataSourceSearchReplicas {
if value == model.FakeSetting {
target.SqlSettings.DataSourceSearchReplicas[i] = actual.SqlSettings.DataSourceSearchReplicas[i]
}
}
}
if *target.MessageExportSettings.GlobalRelaySettings.SMTPPassword == model.FakeSetting {
*target.MessageExportSettings.GlobalRelaySettings.SMTPPassword = *actual.MessageExportSettings.GlobalRelaySettings.SMTPPassword
}
if *target.ServiceSettings.SplitKey == model.FakeSetting {
*target.ServiceSettings.SplitKey = *actual.ServiceSettings.SplitKey
}
for id, settings := range target.PluginSettings.Plugins {
for k, v := range settings {
if v == model.FakeSetting {
settings[k] = actual.PluginSettings.Plugins[id][k]
}
}
}
}
// fixConfig patches invalid or missing data in the configuration.
func fixConfig(cfg *model.Config) {
// Ensure SiteURL has no trailing slash.
if strings.HasSuffix(*cfg.ServiceSettings.SiteURL, "/") {
*cfg.ServiceSettings.SiteURL = strings.TrimRight(*cfg.ServiceSettings.SiteURL, "/")
}
// Ensure the directory for a local file store has a trailing slash.
if *cfg.FileSettings.DriverName == model.ImageDriverLocal {
if *cfg.FileSettings.Directory != "" && !strings.HasSuffix(*cfg.FileSettings.Directory, "/") {
*cfg.FileSettings.Directory += "/"
}
}
fixInvalidLocales(cfg)
}
// fixInvalidLocales checks and corrects the given config for invalid locale-related settings.
func fixInvalidLocales(cfg *model.Config) bool {
var changed bool
locales := i18n.GetSupportedLocales()
if _, ok := locales[*cfg.LocalizationSettings.DefaultServerLocale]; !ok {
mlog.Warn("DefaultServerLocale must be one of the supported locales. Setting DefaultServerLocale to en as default value.", mlog.String("locale", *cfg.LocalizationSettings.DefaultServerLocale))
*cfg.LocalizationSettings.DefaultServerLocale = model.DefaultLocale
changed = true
}
if _, ok := locales[*cfg.LocalizationSettings.DefaultClientLocale]; !ok {
mlog.Warn("DefaultClientLocale must be one of the supported locales. Setting DefaultClientLocale to en as default value.", mlog.String("locale", *cfg.LocalizationSettings.DefaultClientLocale))
*cfg.LocalizationSettings.DefaultClientLocale = model.DefaultLocale
changed = true
}
if *cfg.LocalizationSettings.AvailableLocales != "" {
isDefaultClientLocaleInAvailableLocales := false
for word := range strings.SplitSeq(*cfg.LocalizationSettings.AvailableLocales, ",") {
if _, ok := locales[word]; !ok {
*cfg.LocalizationSettings.AvailableLocales = ""
isDefaultClientLocaleInAvailableLocales = true
mlog.Warn("AvailableLocales must include DefaultClientLocale. Setting AvailableLocales to all locales as default value.")
changed = true
break
}
if word == *cfg.LocalizationSettings.DefaultClientLocale {
isDefaultClientLocaleInAvailableLocales = true
}
}
availableLocales := *cfg.LocalizationSettings.AvailableLocales
if !isDefaultClientLocaleInAvailableLocales {
availableLocales += "," + *cfg.LocalizationSettings.DefaultClientLocale
mlog.Warn("Adding DefaultClientLocale to AvailableLocales.")
changed = true
}
*cfg.LocalizationSettings.AvailableLocales = strings.Join(utils.RemoveDuplicatesFromStringArray(strings.Split(availableLocales, ",")), ",")
}
return changed
}
// Merge merges two configs together. The receiver's values are overwritten with the patch's
// values except when the patch's values are nil.
func Merge(cfg *model.Config, patch *model.Config, mergeConfig *utils.MergeConfig) (*model.Config, error) {
return utils.Merge(cfg, patch, mergeConfig)
}
func IsDatabaseDSN(dsn string) bool {
return strings.HasPrefix(dsn, "postgres://") ||
strings.HasPrefix(dsn, "postgresql://")
}
func isJSONMap(data []byte) bool {
var m map[string]any
err := json.Unmarshal(data, &m)
return err == nil
}
func GetValueByPath(path []string, obj any) (any, bool) {
r := reflect.ValueOf(obj)
var val reflect.Value
if r.Kind() == reflect.Map {
val = r.MapIndex(reflect.ValueOf(path[0]))
if val.IsValid() {
val = val.Elem()
}
} else {
val = r.FieldByName(path[0])
}
if !val.IsValid() {
return nil, false
}
switch {
case len(path) == 1:
return val.Interface(), true
case val.Kind() == reflect.Struct:
return GetValueByPath(path[1:], val.Interface())
case val.Kind() == reflect.Map:
remainingPath := strings.Join(path[1:], ".")
mapIter := val.MapRange()
for mapIter.Next() {
key := mapIter.Key().String()
if strings.HasPrefix(remainingPath, key) {
i := strings.Count(key, ".") + 2 // number of dots + a dot on each side
mapVal := mapIter.Value()
// if no sub field path specified, return the object
if len(path[i:]) == 0 {
return mapVal.Interface(), true
}
data := mapVal.Interface()
if mapVal.Kind() == reflect.Ptr {
data = mapVal.Elem().Interface() // if value is a pointer, dereference it
}
// pass subpath
return GetValueByPath(path[i:], data)
}
}
}
return nil, false
}
func equal(oldCfg, newCfg *model.Config) (bool, error) {
oldCfgBytes, err := json.Marshal(oldCfg)
if err != nil {
return false, fmt.Errorf("failed to marshal old config: %w", err)
}
newCfgBytes, err := json.Marshal(newCfg)
if err != nil {
return false, fmt.Errorf("failed to marshal new config: %w", err)
}
return !bytes.Equal(oldCfgBytes, newCfgBytes), nil
}