mattermost/server/config/diff.go

164 lines
4.4 KiB
Go
Raw Permalink Normal View History

[MM-28692] Include config diffs in audit record for config changing API calls (#17623) * Replace config generator * Cleanup * Some renaming and docs additions to add clarity * Cleanup logging related methods * Cleanup emitter * Fix TestDefaultsGenerator * Move feature flags synchronization logic out of config package * Remove unnecessary util functions * Simplify load/set logic * Refine semantics and add some test to cover them * Remove unnecessary deep copies * Improve logic further * Fix license header * Review file store tests * Fix test * Fix test * Avoid additional write during initialization * More consistent naming * Update app/feature_flags.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> * Make ConfigStore.Set() return both old and new configs * Implement config diff function * Make app.SaveConfig return previous and current configs * Add config diff to audit record * Fix returned configs * Include high level test * Move FF synchronizer to its own package * Remove unidiomatic use of sync.Once * Add some comments * Rename function * More comment * Save config diff in audit record for local endpoints * Enable audit for config set/reset commands * Improve tests output Co-authored-by: Christopher Speller <crspeller@gmail.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-21 03:04:39 -04:00
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
// See LICENSE.txt for license information.
package config
import (
"fmt"
"reflect"
"github.com/mattermost/mattermost/server/public/model"
[MM-28692] Include config diffs in audit record for config changing API calls (#17623) * Replace config generator * Cleanup * Some renaming and docs additions to add clarity * Cleanup logging related methods * Cleanup emitter * Fix TestDefaultsGenerator * Move feature flags synchronization logic out of config package * Remove unnecessary util functions * Simplify load/set logic * Refine semantics and add some test to cover them * Remove unnecessary deep copies * Improve logic further * Fix license header * Review file store tests * Fix test * Fix test * Avoid additional write during initialization * More consistent naming * Update app/feature_flags.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> * Make ConfigStore.Set() return both old and new configs * Implement config diff function * Make app.SaveConfig return previous and current configs * Add config diff to audit record * Fix returned configs * Include high level test * Move FF synchronizer to its own package * Remove unidiomatic use of sync.Once * Add some comments * Rename function * More comment * Save config diff in audit record for local endpoints * Enable audit for config set/reset commands * Improve tests output Co-authored-by: Christopher Speller <crspeller@gmail.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-21 03:04:39 -04:00
)
type ConfigDiffs []ConfigDiff
type ConfigDiff struct {
Path string `json:"path"`
BaseVal any `json:"base_val"`
ActualVal any `json:"actual_val"`
[MM-28692] Include config diffs in audit record for config changing API calls (#17623) * Replace config generator * Cleanup * Some renaming and docs additions to add clarity * Cleanup logging related methods * Cleanup emitter * Fix TestDefaultsGenerator * Move feature flags synchronization logic out of config package * Remove unnecessary util functions * Simplify load/set logic * Refine semantics and add some test to cover them * Remove unnecessary deep copies * Improve logic further * Fix license header * Review file store tests * Fix test * Fix test * Avoid additional write during initialization * More consistent naming * Update app/feature_flags.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> * Make ConfigStore.Set() return both old and new configs * Implement config diff function * Make app.SaveConfig return previous and current configs * Add config diff to audit record * Fix returned configs * Include high level test * Move FF synchronizer to its own package * Remove unidiomatic use of sync.Once * Add some comments * Rename function * More comment * Save config diff in audit record for local endpoints * Enable audit for config set/reset commands * Improve tests output Co-authored-by: Christopher Speller <crspeller@gmail.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-21 03:04:39 -04:00
}
func (c *ConfigDiff) Auditable() map[string]any {
return map[string]any{
Audit logging -- convert audit logs to use the new schema (#20526) * Audit logging - new schema added, old schema removed. * fix linter error by running goimports * Address review comments * Address review comments * Example usage of new audit logging API for the updateUserAuth call * fixed unit test on auditing updating user record * Changed the `TestUpdateConfigDiffInAuditRecord` testcase---it failed, because this PR changes how the `meta` field is serialized into the audit log records. * fix linter error * use string constants for record keys * new audit api calls for api4/bot * `Auditable` interface implementations for model classes * New audit calls for channel api * New audit calls for channel_local * renamed receivers for required style reasons * New audit calls for api4/command * renamed receiver * New audit calls for api4/command_local * renamed receiver * fix unit test to reflect changes in the Auditable implementation of the user class * new audit calls for compliance * new audit calls for configs * remove auditRec.addMeta from updateConfig and patchConfig * new audit calls for config_local * new audit calls * new audit calls for ldap, license apis * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * fix linter error * fixed linter error * fixed "user update" test * Don't include all of config when audit logging config changes. Also fix unit test on TestUpdateConfigDiffInAuditRecord * address review comments * Added Auditable() method for UserPatch * Fix duplicative method declaration from merge * Fix styling and API changes issues introduced with merge * Fix broken test Co-authored-by: Daniel Schalla <daniel@schalla.me>
2022-07-14 07:52:46 -04:00
"path": c.Path,
"base_val": c.BaseVal,
"actual_val": c.ActualVal,
}
}
func (cd *ConfigDiffs) Auditable() map[string]any {
var s []any
Audit logging -- convert audit logs to use the new schema (#20526) * Audit logging - new schema added, old schema removed. * fix linter error by running goimports * Address review comments * Address review comments * Example usage of new audit logging API for the updateUserAuth call * fixed unit test on auditing updating user record * Changed the `TestUpdateConfigDiffInAuditRecord` testcase---it failed, because this PR changes how the `meta` field is serialized into the audit log records. * fix linter error * use string constants for record keys * new audit api calls for api4/bot * `Auditable` interface implementations for model classes * New audit calls for channel api * New audit calls for channel_local * renamed receivers for required style reasons * New audit calls for api4/command * renamed receiver * New audit calls for api4/command_local * renamed receiver * fix unit test to reflect changes in the Auditable implementation of the user class * new audit calls for compliance * new audit calls for configs * remove auditRec.addMeta from updateConfig and patchConfig * new audit calls for config_local * new audit calls * new audit calls for ldap, license apis * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * fix linter error * fixed linter error * fixed "user update" test * Don't include all of config when audit logging config changes. Also fix unit test on TestUpdateConfigDiffInAuditRecord * address review comments * Added Auditable() method for UserPatch * Fix duplicative method declaration from merge * Fix styling and API changes issues introduced with merge * Fix broken test Co-authored-by: Daniel Schalla <daniel@schalla.me>
2022-07-14 07:52:46 -04:00
for _, d := range cd.Sanitize() {
s = append(s, d.Auditable())
}
return map[string]any{
Audit logging -- convert audit logs to use the new schema (#20526) * Audit logging - new schema added, old schema removed. * fix linter error by running goimports * Address review comments * Address review comments * Example usage of new audit logging API for the updateUserAuth call * fixed unit test on auditing updating user record * Changed the `TestUpdateConfigDiffInAuditRecord` testcase---it failed, because this PR changes how the `meta` field is serialized into the audit log records. * fix linter error * use string constants for record keys * new audit api calls for api4/bot * `Auditable` interface implementations for model classes * New audit calls for channel api * New audit calls for channel_local * renamed receivers for required style reasons * New audit calls for api4/command * renamed receiver * New audit calls for api4/command_local * renamed receiver * fix unit test to reflect changes in the Auditable implementation of the user class * new audit calls for compliance * new audit calls for configs * remove auditRec.addMeta from updateConfig and patchConfig * new audit calls for config_local * new audit calls * new audit calls for ldap, license apis * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * fix linter error * fixed linter error * fixed "user update" test * Don't include all of config when audit logging config changes. Also fix unit test on TestUpdateConfigDiffInAuditRecord * address review comments * Added Auditable() method for UserPatch * Fix duplicative method declaration from merge * Fix styling and API changes issues introduced with merge * Fix broken test Co-authored-by: Daniel Schalla <daniel@schalla.me>
2022-07-14 07:52:46 -04:00
"config_diffs": s,
}
}
var configSensitivePaths = map[string]bool{
"LdapSettings.BindPassword": true,
"FileSettings.PublicLinkSalt": true,
"FileSettings.AmazonS3SecretAccessKey": true,
"SqlSettings.DataSource": true,
"SqlSettings.AtRestEncryptKey": true,
"SqlSettings.DataSourceReplicas": true,
"SqlSettings.DataSourceSearchReplicas": true,
"EmailSettings.SMTPPassword": true,
"GitLabSettings.Secret": true,
"GoogleSettings.Secret": true,
"Office365Settings.Secret": true,
"OpenIdSettings.Secret": true,
"ElasticsearchSettings.Password": true,
"MessageExportSettings.GlobalRelaySettings.SMTPUsername": true,
"MessageExportSettings.GlobalRelaySettings.SMTPPassword": true,
"MessageExportSettings.GlobalRelaySettings.EmailAddress": true,
"ServiceSettings.SplitKey": true,
"PluginSettings.Plugins": true,
}
// Sanitize replaces sensitive config values in the diff with asterisks filled strings.
func (cd ConfigDiffs) Sanitize() ConfigDiffs {
if len(cd) == 1 {
// PluginSettings.Plugins gets sanitized anyway, so there is no need to use the plugin manifests here.
var pluginManifests []*model.Manifest
cfgPtr, ok := cd[0].BaseVal.(*model.Config)
if ok {
cfgPtr.Sanitize(pluginManifests, nil)
}
cfgPtr, ok = cd[0].ActualVal.(*model.Config)
if ok {
cfgPtr.Sanitize(pluginManifests, nil)
}
cfgVal, ok := cd[0].BaseVal.(model.Config)
if ok {
cfgVal.Sanitize(pluginManifests, nil)
}
cfgVal, ok = cd[0].ActualVal.(model.Config)
if ok {
cfgVal.Sanitize(pluginManifests, nil)
}
}
for i := range cd {
if configSensitivePaths[cd[i].Path] {
cd[i].BaseVal = model.FakeSetting
cd[i].ActualVal = model.FakeSetting
}
}
return cd
}
func diff(base, actual reflect.Value, label string) ([]ConfigDiff, error) {
[MM-28692] Include config diffs in audit record for config changing API calls (#17623) * Replace config generator * Cleanup * Some renaming and docs additions to add clarity * Cleanup logging related methods * Cleanup emitter * Fix TestDefaultsGenerator * Move feature flags synchronization logic out of config package * Remove unnecessary util functions * Simplify load/set logic * Refine semantics and add some test to cover them * Remove unnecessary deep copies * Improve logic further * Fix license header * Review file store tests * Fix test * Fix test * Avoid additional write during initialization * More consistent naming * Update app/feature_flags.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> * Make ConfigStore.Set() return both old and new configs * Implement config diff function * Make app.SaveConfig return previous and current configs * Add config diff to audit record * Fix returned configs * Include high level test * Move FF synchronizer to its own package * Remove unidiomatic use of sync.Once * Add some comments * Rename function * More comment * Save config diff in audit record for local endpoints * Enable audit for config set/reset commands * Improve tests output Co-authored-by: Christopher Speller <crspeller@gmail.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-21 03:04:39 -04:00
var diffs []ConfigDiff
if base.IsZero() && actual.IsZero() {
return diffs, nil
}
if base.IsZero() || actual.IsZero() {
return append(diffs, ConfigDiff{
Path: label,
BaseVal: base.Interface(),
ActualVal: actual.Interface(),
}), nil
}
baseType := base.Type()
actualType := actual.Type()
if baseType.Kind() == reflect.Ptr {
base = reflect.Indirect(base)
actual = reflect.Indirect(actual)
baseType = base.Type()
actualType = actual.Type()
}
if baseType != actualType {
return nil, fmt.Errorf("not same type %s %s", baseType, actualType)
}
switch baseType.Kind() {
case reflect.Struct:
if base.NumField() != actual.NumField() {
return nil, fmt.Errorf("not same number of fields in struct")
}
for i := 0; i < base.NumField(); i++ {
fieldLabel := baseType.Field(i).Name
if label != "" {
fieldLabel = label + "." + fieldLabel
}
d, err := diff(base.Field(i), actual.Field(i), fieldLabel)
[MM-28692] Include config diffs in audit record for config changing API calls (#17623) * Replace config generator * Cleanup * Some renaming and docs additions to add clarity * Cleanup logging related methods * Cleanup emitter * Fix TestDefaultsGenerator * Move feature flags synchronization logic out of config package * Remove unnecessary util functions * Simplify load/set logic * Refine semantics and add some test to cover them * Remove unnecessary deep copies * Improve logic further * Fix license header * Review file store tests * Fix test * Fix test * Avoid additional write during initialization * More consistent naming * Update app/feature_flags.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> * Make ConfigStore.Set() return both old and new configs * Implement config diff function * Make app.SaveConfig return previous and current configs * Add config diff to audit record * Fix returned configs * Include high level test * Move FF synchronizer to its own package * Remove unidiomatic use of sync.Once * Add some comments * Rename function * More comment * Save config diff in audit record for local endpoints * Enable audit for config set/reset commands * Improve tests output Co-authored-by: Christopher Speller <crspeller@gmail.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-21 03:04:39 -04:00
if err != nil {
return nil, err
}
diffs = append(diffs, d...)
}
default:
if !reflect.DeepEqual(base.Interface(), actual.Interface()) {
diffs = append(diffs, ConfigDiff{
Path: label,
BaseVal: base.Interface(),
ActualVal: actual.Interface(),
})
}
}
return diffs, nil
}
func Diff(base, actual *model.Config) (ConfigDiffs, error) {
if base == nil || actual == nil {
return nil, fmt.Errorf("input configs should not be nil")
}
baseVal := reflect.Indirect(reflect.ValueOf(base))
actualVal := reflect.Indirect(reflect.ValueOf(actual))
return diff(baseVal, actualVal, "")
[MM-28692] Include config diffs in audit record for config changing API calls (#17623) * Replace config generator * Cleanup * Some renaming and docs additions to add clarity * Cleanup logging related methods * Cleanup emitter * Fix TestDefaultsGenerator * Move feature flags synchronization logic out of config package * Remove unnecessary util functions * Simplify load/set logic * Refine semantics and add some test to cover them * Remove unnecessary deep copies * Improve logic further * Fix license header * Review file store tests * Fix test * Fix test * Avoid additional write during initialization * More consistent naming * Update app/feature_flags.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Christopher Speller <crspeller@gmail.com> * Update config/store.go Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> * Make ConfigStore.Set() return both old and new configs * Implement config diff function * Make app.SaveConfig return previous and current configs * Add config diff to audit record * Fix returned configs * Include high level test * Move FF synchronizer to its own package * Remove unidiomatic use of sync.Once * Add some comments * Rename function * More comment * Save config diff in audit record for local endpoints * Enable audit for config set/reset commands * Improve tests output Co-authored-by: Christopher Speller <crspeller@gmail.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-21 03:04:39 -04:00
}
func (cd ConfigDiffs) String() string {
return fmt.Sprintf("%+v", []ConfigDiff(cd))
}