upstream/vault
1
0
Fork 0
mirror of https://github.com/hashicorp/vault.git synced 2026-02-03 20:40:45 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

Speed up dynamodb List() by only getting keys (#21159)

Browse source 
* Speed up ddb list by only getting keys

* Add same optimization to has_children

* Add changeling

---------

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
This commit is contained in:
Thomas Hargrove 2024-06-07 12:11:00 -07:00 committed by GitHub
parent 8fd63b0a60
commit 27563039db
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 14 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
changelog/21159.txt Normal file
View file

@ -0,0 +1,4 @@
```release-note:improvement
storage/dynamodb: Speed up list and delete of large directories by only requesting keys from DynamoDB
```

10
physical/dynamodb/dynamodb.go
View file

@ -410,6 +410,11 @@ func (d *DynamoDBBackend) List(ctx context.Context, prefix string) ([]string, er
}},
},
},
ProjectionExpression: aws.String("#key, #path"),
ExpressionAttributeNames: map[string]*string{
"#key": aws.String("Key"),
"#path": aws.String("Path"),
},
}
d.permitPool.Acquire()
@ -453,6 +458,11 @@ func (d *DynamoDBBackend) hasChildren(prefix string, exclude []string) (bool, er
}},
},
},
ProjectionExpression: aws.String("#key, #path"),
ExpressionAttributeNames: map[string]*string{
"#key": aws.String("Key"),
"#path": aws.String("Path"),
},
// Avoid fetching too many items from DynamoDB for performance reasons.
// We want to know if there are any children we don't expect to see.
// Answering that question requires fetching a minimum of one more item