mirror of
https://github.com/redis/redis.git
synced 2026-03-22 10:50:16 -04:00
70a079db5e
This PR is based on: https://github.com/valkey-io/valkey/pull/861 > ### Memory Access Amortization > (Designed and implemented by [dan touitou](https://github.com/touitou-dan)) > > Memory Access Amortization (MAA) is a technique designed to optimize the performance of dynamic data structures by reducing the impact of memory access latency. It is applicable when multiple operations need to be executed concurrently. The principle behind it is that for certain dynamic data structures, executing operations in a batch is more efficient than executing each one separately. > > Rather than executing operations sequentially, this approach interleaves the execution of all operations. This is done in such a way that whenever a memory access is required during an operation, the program prefetches the necessary memory and transitions to another operation. This ensures that when one operation is blocked awaiting memory access, other memory accesses are executed in parallel, thereby reducing the average access latency. > > We applied this method in the development of dictPrefetch, which takes as parameters a vector of keys and dictionaries. It ensures that all memory addresses required to execute dictionary operations for these keys are loaded into the L1-L3 caches when executing commands. Essentially, dictPrefetch is an interleaved execution of dictFind for all the keys. ### Implementation of Redis When the main thread processes clients with ready-to-execute commands (i.e., clients for which the IO thread has parsed the commands), a batch of up to 16 commands is created. Initially, the command's argv, which were allocated by the IO thread, is prefetched to the main thread's L1 cache. Subsequently, all the dict entries and values required for the commands are prefetched from the dictionary before the command execution. #### Memory prefetching for main hash table As shown in the picture, after https://github.com/redis/redis/pull/13806 , we unify key value and the dict uses no_value optimization, so the memory prefetching has 4 steps: 1. prefetch the bucket of the hash table 2. prefetch the entry associated with the given key's hash 3. prefetch the kv object of the entry 4. prefetch the value data of the kv object we also need to handle the case that the dict entry is the pointer of kv object, just skip step 3. MAA can improves single-threaded memory access efficiency by interleaving the execution of multiple independent operations, allowing memory-level parallelism and better CPU utilization. Its key point is batch-wise interleaved execution. Split a batch of independent operations (such as multiple key lookups) into multiple state machines, and interleave their progress within a single thread to hide the memory access latency of individual requests. The difference between serial execution and interleaved execution: **naive serial execution** ``` key1: step1 → wait → step2 → wait → done key2: step1 → wait → step2 → wait → done ``` **interleaved execution** ``` key1: step1 → step2 → done key2: step1 → step2 → done key3: step1 → step2 → done ↑ While waiting for key1’s memory, progress key2/key3 ``` #### New configuration This PR involves a new configuration `prefetch-batch-max-size`, but we think it is a low level optimization, so we hide this config: When multiple commands are parsed by the I/O threads and ready for execution, we take advantage of knowing the next set of commands and prefetch their required dictionary entries in a batch. This reduces memory access costs. The optimal batch size depends on the specific workflow of the user. The default batch size is 16, which can be modified using the 'prefetch-batch-max-size' config. When the config is set to 0, prefetching is disabled. --------- Co-authored-by: Uri Yagelnik <uriy@amazon.com> Co-authored-by: Ozan Tezcan <ozantezcan@gmail.com> |
||
|---|---|---|
| .. | ||
| cluster | ||
| moduleapi | ||
| type | ||
| acl-v2.tcl | ||
| acl.tcl | ||
| aofrw.tcl | ||
| auth.tcl | ||
| bitfield.tcl | ||
| bitops.tcl | ||
| client-eviction.tcl | ||
| dump.tcl | ||
| expire.tcl | ||
| functions.tcl | ||
| geo.tcl | ||
| hyperloglog.tcl | ||
| info-command.tcl | ||
| info-keysizes.tcl | ||
| info.tcl | ||
| introspection-2.tcl | ||
| introspection.tcl | ||
| keyspace.tcl | ||
| latency-monitor.tcl | ||
| lazyfree.tcl | ||
| limits.tcl | ||
| maxmemory.tcl | ||
| memefficiency.tcl | ||
| multi.tcl | ||
| networking.tcl | ||
| obuf-limits.tcl | ||
| oom-score-adj.tcl | ||
| other.tcl | ||
| pause.tcl | ||
| printver.tcl | ||
| protocol.tcl | ||
| pubsub.tcl | ||
| pubsubshard.tcl | ||
| querybuf.tcl | ||
| quit.tcl | ||
| replybufsize.tcl | ||
| scan.tcl | ||
| scripting.tcl | ||
| shutdown.tcl | ||
| slowlog.tcl | ||
| sort.tcl | ||
| tls.tcl | ||
| tracking.tcl | ||
| violations.tcl | ||
| wait.tcl | ||