* test(tsdb): add OOO error coverage for ST zero sample appends
Add unit tests exercising the out-of-order error paths in
AppendSTZeroSample, AppendHistogramSTZeroSample (AppenderV1), and
the best-effort ST injection in AppenderV2.Append.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
* make format
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
* test(tsdb): add TestHeadAppenderV2_BestEffortSTZeroSample_OOO
The three OOO cases added to TestHeadAppenderV2_Append_EnableSTAsZeroSample
use a single appender so headChunks is nil at append time; the zero sample
enters the batch and is rejected silently in commitFloats, never reaching
the error-handling branch at line 374 of bestEffortAppendSTZeroSample.
Add a dedicated test that commits the first sample before appending the
second. This makes headChunks non-nil, so appendFloat/appendHistogram/
appendFloatHistogram returns ErrOutOfOrderSample at append time and the
branch at line 374 is actually executed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
---------
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a CI job to run govulncheck.
* Run daily.
* Run when touching go.{mod,sum}.on PRs and merge to the default branch.
* Cleanup unused / broken govulncheck in Makefile.common.
* Add status badge.
* Add to repo sync for all Go projects.
* Reduce green lines in sync script to make it easier to read the log.
Signed-off-by: SuperQ <superq@gmail.com>
* test(cmd/prometheus): add TestFeatureFlagsDocumented and fix help text
Add TestFeatureFlagsDocumented to ensure the --enable-feature help text
in and docs/feature_flags.md list the same set of flags.
The help text was out of sync with the documentation:
- Flags present in docs but missing from help text: `auto-reload-config`,
`metadata-wal-records`, `otlp-native-delta-ingestion`,
`promql-delayed-name-removal`, `type-and-unit-labels`. Added them.
- Flags present in help text but missing from docs: `auto-gomaxprocs`,
`expand-external-labels`. Removed them.
The help text is now sorted for better readability and kept in sync
with the documentation.
Also, the parsing of an empty `--enable-feature` was changed to
print `msg="Unknown option for --enable-feature" option=""` instead of nothing.
Signed-off-by: Ayoub Mrini <ayoubmrini424@gmail.com>
* main.go remove default for --enable-feature to avoid unwanted
Signed-off-by: Ayoub Mrini <ayoubmrini424@gmail.com>
---------
Signed-off-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Upstream distroless now supports all architectures, so there is no
longer a need to exclude specific dockerfile/arch combinations from
the build.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
This mechanism was introduced to skip pushing riscv64 images to
registries that were misconfigured to not accept that architecture.
The misconfiguration has now been fixed across all repositories, so
this workaround is no longer needed.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
tsdb: cache collected head chunks on ChunkReader for O(1) lookup
The query path calls s.chunk() once per chunk meta via
ChunkOrIterableWithCopy. Each call walks the head chunks linked list
from the head to the target position. For a series with N head chunks
iterated oldest-first, total work is O(N²).
Cache the collected []*memChunk slice on headChunkReader, keyed by
series ref, head pointer, and mmapped chunks length. Collected once
per series under lock; reused on subsequent chunk lookups for the same
series. The backing array is reused across series (zero alloc after
first use).
Series with 0 or 1 head chunks skip the cache entirely to avoid
per-series overhead that dominates for typical workloads where most
series have a single head chunk.
The cache is gated behind an enableCache flag, toggled via an optional
chunkCacheToggler interface only when hints.Step > 0 (range queries).
Instant queries only need one chunk per series, so the cache overhead
is not recouped.
Also replace O(N²) linked-list traversals in appendSeriesChunks with
O(N) collectHeadChunks + slice iteration, and thread reusable
headChunksBuf through the index reader paths to avoid per-series
allocations.
---------
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Without the return, a successful respond() call would always follow
the error response, writing a second body to an already-written
ResponseWriter and producing a malformed HTTP response.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
smoothSeries() was stamping output points at offset-adjusted timestamps
instead of evaluator timestamps. When the @ modifier is used, this
causes gatherVector() to miss the points because it matches by exact
timestamp equality against evaluator step timestamps.
Fix by iterating over evaluator timestamps and deriving data timestamps
by subtracting the offset, so output points align with what
gatherVector() expects.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Because quay.io requires manual creation of new repos, we missed
creating one for riscv64. This has been created so publish should now
work.
Signed-off-by: SuperQ <superq@gmail.com>
For smoothed rate/increase, a result should only be returned
when there is data available to interpolate across the range. If the
range has a single data point only on one side, the result is
meaningless and should be empty.
The "data only before" case was already handled: if the last fetched
sample is at or before rangeStart, extendedRate returns nothing.
Add the symmetric guard for the "data only after" case: if the first
fetched sample is strictly after rangeEnd, return nothing as well.
This mirrors the behaviour described in prometheus/prometheus#18295,
where a smoothed rate that has no data before the range should not
return zero.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Remove issue reference and trim the comment down to the assertion's
intent, per @roidelapluie review.
Signed-off-by: alliasgher <alliasgher123@gmail.com>
tsdb: Skip clean series during periodic head chunk mmap
The periodic mmapHeadChunks cycle previously acquired a per-series
lock on every series, even though typically >99% have nothing to
mmap. This was identified as a CPU bottleneck in Grafana Mimir.
Add a headChunkCount field (sync/atomic.Uint32) to memSeries that
tracks the number of head chunks. It is incremented in
cutNewHeadChunk and the histogram new-chunk paths, and reset by
mmapChunks and truncateChunksBefore. mmapHeadChunks uses a lock-free
Load to skip series with fewer than 2 head chunks, avoiding the
per-series lock for clean series.
sync/atomic.Uint32 (4 bytes) is used instead of go.uber.org/atomic
(8 bytes) to fit in existing struct padding without growing
memSeries. Chunk counts are bounded by the 3-byte field in
HeadChunkRef, so cannot overflow uint32.
Also fix pre-existing comment inaccuracies in the touched code:
headChunks.next -> headChunks.prev, mmapHeadChunks() -> mmapChunks()
in the doc comment, and a grammar error.
---------
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
* util/strutil: add Jaro-Winkler similarity implementation
This is part of the implementation of prometheus/proposals#74
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* util/strutil: optimise JaroWinkler with string-native ASCII path
Replace the generic jaroWinkler[T byte|rune] with two specialised
functions: jaroWinklerString (ASCII path) operates directly on the
string values and avoids the []byte conversion that previously caused
two heap allocations per call; jaroWinklerRunes (Unicode path) is
unchanged in algorithm but split out from the generic.
Both paths replace the repeated float64 divisions in the Jaro formula
with precomputed reciprocals (invL1, invL2).
Result: short ASCII strings drop from 2 allocs/op to 0 allocs/op;
long ASCII drops from 4 allocs/op to 2 allocs/op (bool match arrays
only).
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* util/strutil: replace JaroWinkler with JaroWinklerMatcher
Remove the free JaroWinkler function and replace it with a
JaroWinklerMatcher struct. NewJaroWinklerMatcher pre-computes the
ASCII check and rune conversion for the search term once; Score then
runs the comparison against each candidate without repeating that work.
This is the expected usage pattern in Prometheus: one fixed term scored
against many label names or values.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* Update util/strutil/jarowinkler.go and util/strutil/jarowinkler_test.go
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Signed-off-by: Julien <291750+roidelapluie@users.noreply.github.com>
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---------
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Signed-off-by: Julien <291750+roidelapluie@users.noreply.github.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Rather than widening the assertion to accept raw hex codes, skip the
strict _MAGIC format check with t.Skipf when the filesystem is not in
the known map. The test still exercises the error paths and will run
fully on standard Linux/macOS filesystems.
Fixesprometheus/prometheus#18471
Signed-off-by: Ali <ali@kscope.ai>
