upstream/redis
1
0
Fork 0
mirror of https://github.com/redis/redis.git synced 2026-02-03 20:39:54 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions 8
redis/src
History
Ozan Tezcan 09f8a2f374
Start AOFRW before streaming repl buffer during fullsync (#13758) 
During fullsync, before loading RDB on the replica, we stop aof child to
prevent copy-on-write disaster.
Once rdb is loaded, aof is started again and it will trigger aof
rewrite. With https://github.com/redis/redis/pull/13732 , for rdbchannel
replication, this behavior was changed. Currently, we start aof after
replication buffer is streamed to db. This PR changes it back to start
aof just after rdb is loaded (before repl buffer is streamed)

Both approaches may have pros and cons. If we start aof before streaming
repl buffers, we may still face with copy-on-write issues as repl
buffers potentially include large amount of changes. If we wait until
replication buffer drained, it means we are delaying starting aof
persistence.

Additional changes are introduced as part of this PR:

- Interface change:
Added `mem_replica_full_sync_buffer` field to the `INFO MEMORY` command
reply. During full sync, it shows total memory consumed by accumulated
replication stream buffer on replica. Added same metric to `MEMORY
STATS` command reply as `replica.fullsync.buffer` field.
  
  
- Fixes: 
- Count repl stream buffer size of replica as part of 'memory overhead'
calculation for fields in "INFO MEMORY" and "MEMORY STATS" outputs.
Before this PR, repl buffer was not counted as part of memory overhead
calculation, causing misreports for fields like `used_memory_overhead`
and `used_memory_dataset` in "INFO STATS" and for `overhead.total` field
in "MEMORY STATS" command reply.
- Dismiss replication stream buffers memory of replica in the fork to
reduce COW impact during a fork.
- Fixed a few time sensitive flaky tests, deleted a noop statement,
fixed some comments and fail messages in rdbchannel tests.
2025-02-04 21:40:18 +03:00
..
commands Start AOFRW before streaming repl buffer during fullsync (#13758) 2025-02-04 21:40:18 +03:00
modules Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
.gitignore Ignore gcov/lcov artifacts 2012-04-13 17:52:33 -07:00
acl.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
adlist.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
adlist.h Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
ae.c Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
ae.h Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
ae_epoll.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
ae_evport.c Fix cluster bus extensions backwards compatibility (#10206) 2022-01-30 19:43:37 +02:00
ae_kqueue.c Fix the timing of read and write events under kqueue (#9416) 2021-09-02 11:07:51 +03:00
ae_select.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
anet.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
anet.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
aof.c Reply LOADING on replica while flushing the db (#13495) 2024-09-03 09:48:44 +03:00
asciilogo.h Align the offset in ASCII logo (#13557) 2024-09-18 14:42:32 +08:00
atomicvar.h Fix typos in multiple Redis source files (#13716) 2025-01-07 15:35:47 +08:00
bio.c Add new SFLUSH command to cluster for slot-based FLUSH (#13564) 2024-09-29 09:13:21 +03:00
bio.h Add new SFLUSH command to cluster for slot-based FLUSH (#13564) 2024-09-29 09:13:21 +03:00
bitops.c Add KEYSIZES section to INFO (#13592) 2024-10-29 13:07:26 +02:00
blocked.c Module unblock on keys: updateStatsOnUnblock is called twice (#13405) 2024-07-11 16:13:38 +08:00
call_reply.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
call_reply.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
childinfo.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
cli_commands.c Reimplement cli hints based on command arg docs (#10515) 2023-03-30 19:03:56 +03:00
cli_commands.h Reimplement cli hints based on command arg docs (#10515) 2023-03-30 19:03:56 +03:00
cli_common.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
cli_common.h Adds connection timeout option to redis-cli (#10609) 2024-01-30 13:43:39 +02:00
cluster.c Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
cluster.h Added a shared secret over Redis cluster. (#13763) 2025-02-03 09:54:37 +02:00
cluster_legacy.c Added a shared secret over Redis cluster. (#13763) 2025-02-03 09:54:37 +02:00
cluster_legacy.h Added a shared secret over Redis cluster. (#13763) 2025-02-03 09:54:37 +02:00
commands.c Reimplement cli hints based on command arg docs (#10515) 2023-03-30 19:03:56 +03:00
commands.def Update history for ban-list propagation (#13749) 2025-01-27 21:05:37 +08:00
commands.h Replaced comment with excessive warning. 2023-07-16 17:04:15 -05:00
config.c Rdb channel replication (#13732) 2025-01-13 15:09:52 +03:00
config.h Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
connection.c Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
connection.h Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
connhelpers.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
crc16.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
crc16_slottable.h Change crc16 slot table to be fixed size character array instead of pointer to strings (#13112) 2024-03-08 15:50:36 -08:00
crc64.c CRC64 perf improvements (#13638) 2024-11-12 09:21:22 +02:00
crc64.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
crccombine.c CRC64 perf improvements (#13638) 2024-11-12 09:21:22 +02:00
crccombine.h CRC64 perf improvements (#13638) 2024-11-12 09:21:22 +02:00
crcspeed.c CRC64 perf improvements (#13638) 2024-11-12 09:21:22 +02:00
crcspeed.h CRC64 perf improvements (#13638) 2024-11-12 09:21:22 +02:00
db.c Fix memleak of SFLUSH experimental command (#13766) 2025-01-30 13:35:02 +02:00
debug.c Added a shared secret over Redis cluster. (#13763) 2025-02-03 09:54:37 +02:00
debugmacro.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
defrag.c Gradually reduce defrag CPU usage when defragmentation is ineffective (#13752) 2025-01-24 11:35:32 +08:00
dict.c Offload lookupCommand into IO threads when threaded IO is enabled (#13696) 2024-12-25 16:03:22 +08:00
dict.h Offload lookupCommand into IO threads when threaded IO is enabled (#13696) 2024-12-25 16:03:22 +08:00
ebuckets.c Fix missing initialization of EbucketsIterator->isRax (#13545) 2024-09-13 17:12:27 +08:00
ebuckets.h Add iterator capability to ebuckets DS (#13519) 2024-09-12 15:02:32 +03:00
endianconv.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
endianconv.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
eval.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
eventnotifier.c Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
eventnotifier.h Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
evict.c Cleanups related to expiry/eviction (#13591) 2024-10-10 16:58:52 +08:00
expire.c Cleanups related to expiry/eviction (#13591) 2024-10-10 16:58:52 +08:00
fmacros.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
fmtargs.h Rewrite huge printf calls to smaller ones for readability (#12257) 2023-09-28 09:21:23 +03:00
function_lua.c Fix LUA garbage collector (CVE-2024-46981) 2025-01-13 21:20:19 +02:00
functions.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
functions.h Add Lua VM memory to memory overhead, now that it's part of zmalloc (#13660) 2024-11-21 08:22:17 +02:00
geo.c changed addReplyHumanLongDouble to addReplyDouble in georadiusGeneric and geoposCommand (#13494) 2024-09-03 20:54:20 +08:00
geo.h RDMF (Redis/Disque merge friendlyness) refactoring WIP 1. 2015-07-26 15:17:18 +02:00
geohash.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
geohash.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
geohash_helper.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
geohash_helper.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
hyperloglog.c Avoid importing memory aligned malloc (#13693) 2024-12-20 09:39:14 +08:00
intset.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
intset.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
iothread.c Offload lookupCommand into IO threads when threaded IO is enabled (#13696) 2024-12-25 16:03:22 +08:00
kvstore.c Avoid unnecessary hfield Creation/Deletion on updates in hashTypeSet. HSET updates improvement of ~10% (#13655) 2024-12-12 19:41:08 +02:00
kvstore.h Avoid unnecessary hfield Creation/Deletion on updates in hashTypeSet. HSET updates improvement of ~10% (#13655) 2024-12-12 19:41:08 +02:00
latency.c Update old links for modules-api-ref.md (#13479) 2024-11-04 18:18:22 +02:00
latency.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
lazyfree.c Add KEYSIZES section to INFO (#13592) 2024-10-29 13:07:26 +02:00
listpack.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
listpack.h Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
listpack_malloc.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
localtime.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
logreqres.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
lolwut.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
lolwut.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
lolwut5.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
lolwut6.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
lzf.h Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
lzf_c.c Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
lzf_d.c Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
lzfP.h Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
Makefile Add codecov for automated code coverage (#13393) 2025-01-27 21:04:11 +08:00
memtest.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
mkreleasehdr.sh fix the wrong path in mkreleasehdr.sh (#12993) 2024-01-26 15:01:54 -08:00
module.c Add flag for ability of a module context to execute debug commands (#13774) 2025-02-03 09:52:41 +02:00
monotonic.c Better standardize around assertions (#12539) 2023-10-02 18:58:44 -07:00
monotonic.h Remove prototypes with empty declarations (#12020) 2023-05-02 17:31:32 -07:00
mstr.c Hash Field Expiration - Basic support 2024-04-18 16:06:30 +03:00
mstr.h HFE to support AOF and replicas (#13285) 2024-05-29 19:47:48 +08:00
mt19937-64.c Fix random element selection for large hash tables. (#8133) 2020-12-23 15:52:07 +02:00
mt19937-64.h Fix random element selection for large hash tables. (#8133) 2020-12-23 15:52:07 +02:00
multi.c Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
networking.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
notify.c Update old links for modules-api-ref.md (#13479) 2024-11-04 18:18:22 +02:00
object.c Start AOFRW before streaming repl buffer during fullsync (#13758) 2025-02-04 21:40:18 +03:00
pqsort.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
pqsort.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
pubsub.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
quicklist.c Fix typos in multiple Redis source files (#13716) 2025-01-07 15:35:47 +08:00
quicklist.h Determine the large limit of the quicklist node based on fill (#12659) 2024-02-22 10:02:38 +02:00
rand.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
rand.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
rax.c Hash Field Expiration - Basic support 2024-04-18 16:06:30 +03:00
rax.h Hash Field Expiration - Basic support 2024-04-18 16:06:30 +03:00
rax_malloc.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
rdb.c Rdb channel replication (#13732) 2025-01-13 15:09:52 +03:00
rdb.h Reply LOADING on replica while flushing the db (#13495) 2024-09-03 09:48:44 +03:00
redis-benchmark.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
redis-check-aof.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
redis-check-rdb.c HFE - RDB serialize also hash min expiration (For ROF flow) (#13391) 2024-07-07 17:00:19 +03:00
redis-cli.c Replace wrongly free with zfree in redis-cli (#13560) 2024-09-23 09:40:47 +08:00
redis-trib.rb Redis-trib deprecated: it no longer works and it 2018-07-13 10:51:58 +02:00
redisassert.c Fixed variable parameter formatting issues in serverPanic function (#13504) 2024-09-03 15:51:46 +08:00
redisassert.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
redismodule.h Add flag for ability of a module context to execute debug commands (#13774) 2025-02-03 09:52:41 +02:00
release.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
replication.c Start AOFRW before streaming repl buffer during fullsync (#13758) 2025-02-04 21:40:18 +03:00
resp_parser.c Using fast_float library for faster parsing of 64 decimal strings. (#11884) 2024-09-15 21:37:29 +08:00
resp_parser.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
rio.c Rdb channel replication (#13732) 2025-01-13 15:09:52 +03:00
rio.h Rdb channel replication (#13732) 2025-01-13 15:09:52 +03:00
script.c Fix memory leak of jemalloc tcache on function flush command (#13661) 2024-11-21 14:12:58 +03:00
script.h Add help for DEBUG SCRIPT command (#13385) 2024-07-04 14:44:42 +08:00
script_lua.c Trigger Lua GC after script loading (#13407) 2024-07-16 09:28:47 +08:00
script_lua.h Trigger Lua GC after script loading (#13407) 2024-07-16 09:28:47 +08:00
sds.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
sds.h Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
sdsalloc.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
sentinel.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
server.c Start AOFRW before streaming repl buffer during fullsync (#13758) 2025-02-04 21:40:18 +03:00
server.h Start AOFRW before streaming repl buffer during fullsync (#13758) 2025-02-04 21:40:18 +03:00
setcpuaffinity.c cpu affinity: DragonFlyBSD support (#7956) 2020-10-25 14:14:05 +02:00
setproctitle.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
sha1.c Ignore -Wstringop-overread warning for SHA1Transform() on GCC 12 (#11538) 2022-11-24 15:27:16 +02:00
sha1.h Fix some compile warnings and errors when building with gcc-12 or clang (#12035) 2023-04-18 09:53:51 +03:00
sha256.c Add sanitizer support and clean up sanitizer findings (#9601) 2021-11-11 13:51:33 +02:00
sha256.h fix explanation of sha256 (#9220) 2021-07-10 10:04:54 -05:00
siphash.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
slowlog.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
slowlog.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
socket.c Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
solarisfixes.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
sort.c Modules API: new HashFieldMinExpire(). Add flag REDISMODULE_HASH_EXPIRE_TIME to HashGet(). (#13676) 2024-12-05 11:14:52 +02:00
sparkline.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
sparkline.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
stream.h XREADGROUP from PEL should not affect server.dirty (#13251) 2024-05-06 16:55:42 +08:00
strl.c Avoid using unsafe C functions (#10932) 2022-07-18 10:56:26 +03:00
syncio.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
syscheck.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
syscheck.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
t_hash.c Avoid unnecessary hfield Creation/Deletion on updates in hashTypeSet. HSET updates improvement of ~10% (#13655) 2024-12-12 19:41:08 +02:00
t_list.c Fix typos in multiple Redis source files (#13716) 2025-01-07 15:35:47 +08:00
t_set.c Add KEYSIZES section to INFO (#13592) 2024-10-29 13:07:26 +02:00
t_stream.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
t_string.c Fix typos in multiple Redis source files (#13716) 2025-01-07 15:35:47 +08:00
t_zset.c Fix typos in multiple Redis source files (#13716) 2025-01-07 15:35:47 +08:00
testhelp.h Hash Field Expiration - Basic support 2024-04-18 16:06:30 +03:00
threads_mngr.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
threads_mngr.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
timeout.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
tls.c Fix compile error with TLS_DEBUGGING (#13772) 2025-01-26 10:14:43 +08:00
tracking.c Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
unix.c Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
util.c Check length before reading in stringmatchlen (#13690) 2024-12-26 12:37:23 +08:00
util.h Modules API: Add RedisModule_ACLCheckKeyPrefixPermissions (#13666) 2024-11-28 18:33:58 +02:00
valgrind.sup Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed 2020-12-06 14:54:34 +02:00
version.h Add Module API for version and compatibility checks (#7865) 2020-10-11 17:21:58 +03:00
ziplist.c Fix incorrect parameter type reports (#13744) 2025-01-14 15:51:05 +08:00
ziplist.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
zipmap.c Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
zipmap.h Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157) 2024-03-20 22:38:24 +00:00
zmalloc.c Avoid used_memory contention when update from multiple threads. (#13431) 2024-08-19 13:22:16 +03:00
zmalloc.h Allocate Lua VM code with jemalloc instead of libc, and count it used memory (#13133) 2024-04-16 12:43:33 +03:00