Commit graph

1724 commits

Author SHA1 Message Date
Bartlomiej Plotka
44d772b4e7
refactor: use Appender mock for tests (#17992)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-02-02 15:56:11 +00:00
Bartlomiej Plotka
076369fad0
refactor: move OTLP handler to separate file (#17990)
Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-02-02 14:36:00 +00:00
Bartlomiej Plotka
9657c23c37
storage: optimized fanoutAppenderV2 (#17976)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
* storage: add BenchmarkFanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fix: optimized fanoutAppenderV2

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimized more

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-02-02 07:04:30 +00:00
bwplotka
6c18bba5c6 Revert "storage: add BenchmarkFanoutAppenderV2"
Some checks failed
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (push) Has been cancelled
CI / Build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
This reverts commit 91b0f353b0.
2026-01-30 15:54:52 +00:00
bwplotka
91b0f353b0 storage: add BenchmarkFanoutAppenderV2
Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-01-30 15:53:59 +00:00
Arve Knudsen
dc34b90f93
otlptranslator: fix silently swallowed error in addSumNumberDataPoints (#17954)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
The createAttributes error was incorrectly returning nil instead of err,
causing errors to be silently discarded. This could lead to silent data
loss for sum metrics during OTLP ingestion.

Fixes #17953

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2026-01-28 13:58:50 +01:00
Callum Styan
97e7ef802c
remote write: simplify readability of timeseries filtering by using the slices package (#14318)
* simplify readability of timeseries filtering by using the slices package

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* ensure that BenchmarkBuildTimeSeries doesn't account for the building of
the actual proto in the benchmark results, we only care about the
buildTimeSeries call

Signed-off-by: Callum Styan <callumstyan@gmail.com>

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2026-01-28 11:47:34 +00:00
Ganesh Vernekar
9eb78735cf
Merge pull request #16929 from prometheus/codesome/stale-series-compaction
Some checks failed
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (push) Has been cancelled
CI / Build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
tsdb: Early compaction of stale series

Closes #13616

Based on https://github.com/prometheus/proposals/pull/55

Stale series tracking was added in #16925. This PR compacts the stale series into its own block before the normal compaction hits. Here is how the settings:

stale_series_compaction_threshold: As soon as the ratio of stale series in the head block crosses StaleSeriesImmediateCompactionThreshold, TSDB performs a stale series compaction and puts all the stale series into a block and removed it from the head, but it does not remove it from the WAL. (technically this condition is checked every minute and not exactly immediate)
Additional details

WAL replay: after a stale series compaction, tombstones are added with (MinInt64, MaxInt64) for all these stale series. During WAL replay we add a special condition where when we find such tombstone, it immediately removes the series from the memory instead of storing the tombstone. This is required so that we don't spike up memory during WAL replay and also don't keep the compacted stale series in the memory.
Head block truncation ignores this block via the added metadata, similar to out-of-order blocks.
2026-01-24 15:18:17 -08:00
Arve Knudsen
2332962c4b
otlptranslator: filter __name__ from OTLP attributes to prevent duplicates (#17917)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
* otlptranslator: filter __name__ from OTLP attributes to prevent duplicates

OTLP metrics can have a __name__ attribute which, when combined with the
metric name passed via extras, creates duplicate __name__ labels.
This commit implements filtering out of any __name__ metric attribute from OTLP.

Also rename TestCreateAttributes to TestPrometheusConverter_createAttributes
for consistency, and add test cases for __name__, __type__, and __unit__ OTLP metric attributes.

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2026-01-24 16:44:08 +01:00
Ganesh Vernekar
4f3de8da29 tsdb: Add unit tests for stale series compaction
Signed-off-by: Ganesh Vernekar <ganesh.vernekar@reddit.com>
2026-01-23 18:07:34 -08:00
Bartlomiej Plotka
0d116b0994
tests(teststorage): Close Storage in the helper (#17902)
Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-01-23 08:41:35 +00:00
Ben Kochie
1a0c56993c
Use standard library errors in otlptranslator (#17894)
Remove direct use of `go.uber.org/multierr` in favor of standard
library `errors` package.

Signed-off-by: SuperQ <superq@gmail.com>
2026-01-20 10:13:40 +01:00
Bartlomiej Plotka
c4b0da94db
Merge pull request #17835 from prometheus/bwplotka/a2-storage-support
Some checks failed
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (push) Has been cancelled
CI / Build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
feat(storage)[PART4b]: add AppenderV2 to the rest of storage.Storage implementations + mock exemplar fix
2026-01-16 13:04:14 +00:00
bwplotka
f61a83bcd6 fix: appenderV2 mock exemplar appendErr injection before appending it
Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-01-16 10:47:00 +00:00
Arve Knudsen
4afa76d083
OTLP: label caching for OTLP-to-Prometheus conversion to reduce allocations and improve latency (#17860)
* otlptranslator: add label caching for OTLP-to-Prometheus conversion

Add per-request caching to reduce redundant computation and allocations
during OTLP metric conversion:

1. Per-request label sanitization cache: Cache sanitized label names
   within a request to avoid repeated string allocations for commonly
   repeated labels like __name__, job, instance.

2. Resource-level label caching: Precompute and cache job, instance,
   promoted resource attributes, and external labels once per
   ResourceMetrics boundary instead of for each datapoint.

3. Scope-level label caching: Precompute and cache scope metadata labels
   (otel_scope_name, otel_scope_version, etc.) once per ScopeMetrics
   boundary.

4. LabelNamer instance caching: Reuse the LabelNamer struct across
   datapoints within the same resource context.

These optimizations significantly reduce allocations and improve latency
for OTLP ingestion workloads with many datapoints per resource/scope.

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2026-01-16 10:30:05 +00:00
George Krajcsovits
ef350b2b54
Merge pull request #17840 from prometheus/krajo/st-in-chunks
feat(tsdb): new Append parameter and AtST interface for chunks
2026-01-16 10:32:53 +01:00
bwplotka
1d3c6210f3 add extra fanout test
Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-01-15 14:13:58 +00:00
Arve Knudsen
a85a8998c9
otlptranslator: fix BenchmarkPrometheusConverter_FromMetrics (#17866)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
The benchmark was passing appendMetadata=false to NewCombinedAppender,
which caused UpdateMetadata to never be called on the underlying
noOpAppender. This resulted in app.metadata always being 0, failing
the assertion that metadata count should be positive.

Fix by enabling metadata appending in the benchmark.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2026-01-15 09:33:16 +01:00
bwplotka
8a2921e385 addressed feedback
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-01-14 13:57:48 +00:00
bwplotka
49c3aea56d feat(storage)[PART4b]: add AppenderV2 to the rest of storage.Storage impl
Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-01-14 13:54:27 +00:00
György Krajcsovits
6137de085e
test ListSeriesIterator with ST
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2026-01-14 13:15:19 +01:00
György Krajcsovits
5ecc0e7062
test that ChainSampleIterator passes on the AtST call
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2026-01-14 13:15:18 +01:00
György Krajcsovits
8067b3d60a
add test coverage for buffer.go
I've checked that commenting out any one of the new lines produces an error.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2026-01-14 13:15:17 +01:00
György Krajcsovits
6647e512ad
update ExpandSamples
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2026-01-14 13:15:14 +01:00
György Krajcsovits
a00c0d6a66
auto update f/h/fh sample init with positional fields
find . -name "*.go" -type f \
-exec sed -E -i 's/((f|h|fh)Sample\{)([^,{:]+,[^,]+\})/\10, \3/g' {} +

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2026-01-14 13:15:11 +01:00
György Krajcsovits
1e77d9ded8
storage/buffer.go: add ST to sample types and iterators
Also fix the single multi line fSample definition to be one liner.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2026-01-14 13:15:10 +01:00
György Krajcsovits
70bc06718d
feat(tsdb): new AppenderV2 and AtST interface for chunks
No implementation yet. Just to test the shape of the interface.
AtST is implemented for trivial cases, anything else is hard coded
to return 0.

Ref: https://github.com/prometheus/prometheus/issues/17791

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2026-01-14 13:15:09 +01:00
Ben Kochie
836caf7b16
Refactor storage package errors (#17846)
Replace use of `tsdb/errors` with standard library `errors`.

Signed-off-by: SuperQ <superq@gmail.com>
2026-01-13 10:21:33 +01:00
Ben Kochie
03a1d7a350
Refactor storage package errors (#17844)
Replace use of `tsdb/errors` with standard library `errors`.

Signed-off-by: SuperQ <superq@gmail.com>
2026-01-13 09:53:49 +01:00
George Krajcsovits
66bdc88013
fix(remote_read): NHCB not returned over remote read samples (#17794)
NHCB is native histograms with custom buckets.

prompb is used for both remote write 1.0 and remote read. We do not
support NHCB over remote write 1.0 , however we should absolutely
support it for remote read.

Prometheus remote write 1.0 client already refuses to send NHCB.
Prometheus remote write 1.0 server accepts NHCB, but doesn't store
custom values, corrupting the result. I'm now handling NHCB correctly,
instead of refusing or corrupting.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2026-01-07 08:44:57 +01:00
Ben Kochie
e14795bbf4
Remove copyright date from headers (#17785)
Remove copyright dates from various files as part of [PROM-50].

[PROM-50]: https://github.com/prometheus/proposals/blob/main/proposals/0050-remove-copyright-dates.md

Signed-off-by: SuperQ <superq@gmail.com>
2026-01-05 13:46:21 +01:00
dependabot[bot]
87401302f6
chore(deps): bump google.golang.org/api from 0.257.0 to 0.258.0 (#17761)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.257.0 to 0.258.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.257.0...v0.258.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-version: 0.258.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-04 15:47:58 +00:00
Bryan Boreham
950f756e9e Merge remote-tracking branch 'origin/release-3.8' into merge-3.8.1-to-main
Tweak RW2 change for consistency with older lines.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2025-12-19 09:33:41 +00:00
bwplotka
da253bddf5 fix: ensure remote PRWv1 write handler does not send false 0 response headers
Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-12-15 12:27:35 +00:00
bwplotka
ad367b504b refactor(tsdb/agent)[PART3]: add AppenderV2 support to agent
Some checks failed
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (push) Has been cancelled
CI / Build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-12-13 05:17:39 +00:00
bwplotka
cd98ded6ea test: add regression test against remote write handler bad response stats
Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-12-12 22:43:51 +00:00
bwplotka
0b70a07572 refactor(appenderV2): add TSDB AppenderV2 implementation
Signed-off-by: bwplotka <bwplotka@gmail.com>

tmp

Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-12-09 10:39:43 +00:00
bwplotka
129650df9d refactor(appenderV2): 1:1 copy of head_append.go -> head_append_v2.go (starting point)
Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-12-09 09:53:39 +00:00
bwplotka
04696703fe refactor(appenderV2): add AppenderV2 interface
Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-12-09 09:53:34 +00:00
Julien
93edf912a7
Merge pull request #17630 from zjumathcode/main
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
chore: Fix function name typo in createBatchSpan comment
2025-12-02 16:23:48 +01:00
Ben Edmunds
0e682a70a6
RW2: Allow custom scope in azuread (#17483)
Signed-off-by: Ben Edmunds <sammybenblue2@gmail.com>
2025-12-02 11:45:23 +00:00
zjumathcode
0ac2221a20 chore: Fix function name typo in createBatchSpan comment
Signed-off-by: zjumathcode <pai314159@2980.com>
2025-12-01 19:14:19 +08:00
George Krajcsovits
a66c696530
chore(storage): update docstring (#17609)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
The original implementation in #9705 for native histograms included a
technical dept #15177 where samples were committed ordered by type
not by their append order. This was fixed in #17071, but this docstring
was not updated.

I've also took the liberty to mention that we do not order by timestamp
either, thus it is possible to append out of order samples.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-11-25 15:33:35 +01:00
Björn Rabenstein
b8d19543b8
Add histogram validation in remote-read and during reducing resolution (#17561)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
ReduceResolution is currently called before validation during
ingestion. This will cause a panic if there are not enough buckets in
the histogram. If there are too many buckets, the spurious buckets are
ignored, and therefore the error in the input histogram is masked.

Furthermore, invalid negative offsets might cause problems, too.

Therefore, we need to do some minimal validation in reduceResolution.
Fortunately, it is easy and shouldn't slow things down. Sadly, it
requires to return errors, which triggers a bunch of code changes.
Even here is a bright side, we can get rud of a few panics. (Remember:
Don't panic!)

In different news, we haven't done a full validation of histograms
read via remote-read. This is not so much a security concern (as you
can throw off Prometheus easily by feeding it bogus data via
remote-read) but more that remote-read sources might be makeshift and
could accidentally create invalid histograms. We really don't want to
panic in that case. So this commit does not only add a check of the
spans and buckets as needed for resolution reduction but also a full
validation during remote-read.

Signed-off-by: beorn7 <beorn@grafana.com>
2025-11-21 00:22:24 +01:00
Bartlomiej Plotka
3bcc88b053 prw2: Move Remote Write 2.0 CT to be per Sample; Rename to ST (start timestamp) (#17411)
Relates to
https://github.com/prometheus/prometheus/issues/16944#issuecomment-3164760343

Signed-off-by: bwplotka <bwplotka@gmail.com>
(cherry picked from commit cefefc6897)
2025-11-20 08:26:04 +00:00
Minh Nguyen
5087a25848
Remote Write Receive Fix: Remove duplicate labels when type-and-unit-label feature is on (#17546)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
* drop extra label from receiver

Signed-off-by: pipiland2612 <nguyen.t.dang.minh@gmail.com>

* used constant

Signed-off-by: pipiland2612 <nguyen.t.dang.minh@gmail.com>

---------

Signed-off-by: pipiland2612 <nguyen.t.dang.minh@gmail.com>
2025-11-18 09:37:09 +00:00
Bartlomiej Plotka
cefefc6897
prw2: Move Remote Write 2.0 CT to be per Sample; Rename to ST (start timestamp) (#17411)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Relates to
https://github.com/prometheus/prometheus/issues/16944#issuecomment-3164760343

Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-11-17 14:59:40 +00:00
Laurent Dufresne
d99f8dacc4
chore: remove dead code (#17542)
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Signed-off-by: Laurent Dufresne <laurent.dufresne@grafana.com>
2025-11-17 10:37:55 +01:00
Bartlomiej Plotka
f50ff0a40a
feat: rename CreatedTimestamp to StartTimestamp (#17523)
Partially fixes https://github.com/prometheus/prometheus/issues/17416 by
renaming all CT* names to ST* in the whole codebase except RW2 (this is
done in separate
[PR](https://github.com/prometheus/prometheus/pull/17411)) and
PrometheusProto exposition proto.

```
CreatedTimestamp -> StartTimestamp
CreatedTimeStamp -> StartTimestamp
created_timestamp -> start_timestamp
CT -> ST
ct -> st

```

Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-11-13 14:17:51 +00:00
Bartlomiej Plotka
675bafe2fb
Merge pull request #17441 from pipiland2612/refactor_queue_manger
Refactor part of queue_manger.go by creating struct to reuse some common function
2025-11-13 15:07:11 +01:00