541 commits

Author	SHA1	Message	Date
Bartlomiej Plotka	848b16d686	test: Add benchmark without storage + fix skipRecording mock feature (#17987) ... * test: Add benchmark without storage Signed-off-by: bwplotka <bwplotka@gmail.com> make bench fair Signed-off-by: bwplotka <bwplotka@gmail.com> tmp Signed-off-by: bwplotka <bwplotka@gmail.com> * Apply suggestions from code review Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2026-02-02 12:44:11 +00:00
Bartlomiej Plotka	88f6ee4c8e	tests(scrape): add TestScrapeLoopAppend_WithStorage (#17937) ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-01-30 11:44:07 +00:00
Bartlomiej Plotka	36ea75d203	scrape: fix flaky appender test (#17962) ... Fixes https://github.com/prometheus/prometheus/issues/17941 Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-01-29 10:50:17 +00:00
Aditya Prakash	5e66c9305f	scrape: clarify test channel name in manager_test (#17929) ... Signed-off-by: Nova <adityaprakash1357908@gmail.com>	2026-01-27 08:57:40 +00:00
Bartlomiej Plotka	bec70227f1	feat(scrape)[PART5b]: Add AppenderV2 support to scrape.NewManager constructor (#17872) ... * feat(scrape)[PART5b]: Add AppenderV2 support to scrape.NewManager optionally to V1 Signed-off-by: bwplotka <bwplotka@gmail.com> * Update scrape/manager.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * fixes after rebase Signed-off-by: bwplotka <bwplotka@gmail.com> * Apply suggestions from code review Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2026-01-23 09:04:05 +00:00
Bartlomiej Plotka	0d116b0994	tests(teststorage): Close Storage in the helper (#17902) ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-01-23 08:41:35 +00:00
Bartlomiej Plotka	664b255699	Merge pull request #17867 from prometheus/bwplotka/a2-scrape-1 ... refactor(appenderV2)[PART5a]: add AppendableV2 support to scrape loop + tests	2026-01-21 08:21:56 +00:00
Bryan Boreham	2dec6da3d1	[TESTS] Scraping: Reset appender in BenchmarkScrapeLoopAppend ... Otherwise performance is dominated by adding to a slice that gets longer and longer as the benchmark progresses. I chose to Rollback rather than Commit because that should do less work. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2026-01-17 16:37:21 +00:00
Devarsh	c7bc56cf6c	Add scrape commit and total duration metrics (#17665) ... * Add scrape commit and total duration metrics Signed-off-by: Devarsh <devarshshah2608@gmail.com> * update metric based on the review Signed-off-by: Devarsh <devarshshah2608@gmail.com> * conditionally record scrape duration Signed-off-by: Devarsh <devarshshah2608@gmail.com> * Fix formatting in scrape.go Signed-off-by: Devarsh <devarshshah2608@gmail.com> --------- Signed-off-by: Devarsh <devarshshah2608@gmail.com>	2026-01-13 14:07:27 -03:00
Marvin Rösch	fff29d330d	[BUGFIX] Scraping: drop sample if relabeling config says so ... Signed-off-by: Marvin Rösch <marvinroesch99@gmail.com>	2026-01-07 16:11:22 +01:00
Arthur Silva Sens	1e317d0098	Add configuration option to control extra-scrape-metrics (#17606)	2026-01-06 09:00:49 -03:00
Ben Kochie	e14795bbf4	Remove copyright date from headers (#17785) ... Remove copyright dates from various files as part of [PROM-50]. [PROM-50]: https://github.com/prometheus/proposals/blob/main/proposals/0050-remove-copyright-dates.md Signed-off-by: SuperQ <superq@gmail.com>	2026-01-05 13:46:21 +01:00
Arve Knudsen	f0dfb9f802	fix(scrape): use HonorLabels instead of HonorTimestamps in newScrapeLoop (#17731) ... * fix(scrape): use HonorLabels instead of HonorTimestamps in newScrapeLoop The sampleMutator closure in newScrapeLoop was incorrectly passing HonorTimestamps to mutateSampleLabels instead of HonorLabels. This caused honor_labels configuration to be ignored, with the behavior incorrectly controlled by honor_timestamps instead. Adding TestNewScrapeLoopHonorLabelsWiring integration test that exercises the real newScrapeLoop constructor with HonorLabels and HonorTimestamps set to opposite values to catch this class of wiring bug. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update scrape/scrape_test.go Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add honor_labels=false test case Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2025-12-22 16:28:08 +01:00
Bartlomiej Plotka	17e06dbab5	refactor(scrape)[PART2]: simplified scrapeLoop constructors & tests; add teststorage.Appendable mock (#17631) ... * refactor(scrape): simplified scrapeLoop constructors & tests; add teststorage.Appender mock Signed-off-by: bwplotka <bwplotka@gmail.com> debug * refactor(scrape): simplified newLoop even more Signed-off-by: bwplotka <bwplotka@gmail.com> * refactor(scrape): rename sl -> app, slApp -> app Signed-off-by: bwplotka <bwplotka@gmail.com> * fix TestScrapeLoopRun flakiness Signed-off-by: bwplotka <bwplotka@gmail.com> * fix lint Signed-off-by: bwplotka <bwplotka@gmail.com> * kill unused listSeriesSet code Signed-off-by: bwplotka <bwplotka@gmail.com> * fix closing to not panic Signed-off-by: bwplotka <bwplotka@gmail.com> * added extra benchmark for scrapeAndReport Signed-off-by: bwplotka <bwplotka@gmail.com> * added extra benchmark for restartLoops Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed last comments Signed-off-by: bwplotka <bwplotka@gmail.com> * fix TestConcurrentAppender_ReturnsErrAppender naming Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed small comments Signed-off-by: bwplotka <bwplotka@gmail.com> * refactor(scrape): ensure scrape config is reloaded; added test Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-12-22 09:38:48 +00:00
Bryan Boreham	0711e89092	Merge pull request #17530 from bboreham/faster-scrape-relabel ... [PERF] Scraping: skip an unnecessary step when there are relabel rules	2025-12-16 10:40:16 +00:00
Rushabh Mehta	c2b86775b6	scrape: Fix potential goroutine leak in scrapeAndReport (#17554) ... * [scrape] Fix potential goroutine leak in scrape loop Signed-off-by: Rushabh Mehta <mehtarushabh2005@gmail.com> * Use correct error var Signed-off-by: Rushabh Mehta <mehtarushabh2005@gmail.com> * Add regression test Signed-off-by: Rushabh Mehta <mehtarushabh2005@gmail.com> --------- Signed-off-by: Rushabh Mehta <mehtarushabh2005@gmail.com>	2025-12-12 14:01:57 +01:00
Julien Pivotto	a5671a002f	API: Add a /api/v1/features endpoint ... Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2025-12-09 16:13:14 +01:00
Bryan Boreham	77ba5c5fbd	[PERF] Scraping: skip an unnecessary step when there are relabel rules ... Before it would do Builder->Labels->Builder, now we skip the conversions. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-11-28 15:07:40 +00:00
harsh kumar	30be1483d1	instrumentation: add native histograms to complement high-traffic summaries (#17374) ... This adds the following native histograms (with a few classic buckets for backwards compatibility), while keeping the corresponding summaries (same name, just without `_histogram`): - `prometheus_sd_refresh_duration_histogram_seconds` - `prometheus_rule_evaluation_duration_histogram_seconds` - `prometheus_rule_group_duration_histogram_seconds` - `prometheus_target_sync_length_histogram_seconds` - `prometheus_target_interval_length_histogram_seconds` - `prometheus_engine_query_duration_histogram_seconds` Signed-off-by: Harsh <harshmastic@gmail.com> Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2025-11-27 18:45:35 +01:00
Björn Rabenstein	b8d19543b8	Add histogram validation in remote-read and during reducing resolution (#17561) ... ReduceResolution is currently called before validation during ingestion. This will cause a panic if there are not enough buckets in the histogram. If there are too many buckets, the spurious buckets are ignored, and therefore the error in the input histogram is masked. Furthermore, invalid negative offsets might cause problems, too. Therefore, we need to do some minimal validation in reduceResolution. Fortunately, it is easy and shouldn't slow things down. Sadly, it requires to return errors, which triggers a bunch of code changes. Even here is a bright side, we can get rud of a few panics. (Remember: Don't panic!) In different news, we haven't done a full validation of histograms read via remote-read. This is not so much a security concern (as you can throw off Prometheus easily by feeding it bogus data via remote-read) but more that remote-read sources might be makeshift and could accidentally create invalid histograms. We really don't want to panic in that case. So this commit does not only add a check of the spans and buckets as needed for resolution reduction but also a full validation during remote-read. Signed-off-by: beorn7 <beorn@grafana.com>	2025-11-21 00:22:24 +01:00
Bryan Boreham	b7aae06181	Merge pull request #17114 from bboreham/scrape-stale-by-ref ... Scraping: detect staleness via unique reference	2025-11-14 18:32:26 +01:00
Bartlomiej Plotka	f50ff0a40a	feat: rename CreatedTimestamp to StartTimestamp (#17523) ... Partially fixes https://github.com/prometheus/prometheus/issues/17416 by renaming all CT* names to ST* in the whole codebase except RW2 (this is done in separate [PR](https://github.com/prometheus/prometheus/pull/17411)) and PrometheusProto exposition proto. ``` CreatedTimestamp -> StartTimestamp CreatedTimeStamp -> StartTimestamp created_timestamp -> start_timestamp CT -> ST ct -> st ``` Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-11-13 14:17:51 +00:00
Ben Kochie	204249fcb5	Update golangci-lint (#17478) ... * Update golangci-lint to v2.6.0 * Fixup various linting issues. * Fixup deprecations. * Add exception for `labels.MetricName` deprecation. Signed-off-by: SuperQ <superq@gmail.com>	2025-11-05 13:47:34 +01:00
George Krajcsovits	d7bfc89f7a	Merge pull request #17431 from grafana/thampiotr/upstream-staleness-disabling ... scrape: Allow disabling end-of-run staleness markers for targets	2025-11-04 13:24:05 +01:00
Piotr	d6848c9f40	scrape: Allow disabling end-of-run staleness markers for targets ... Signed-off-by: Piotr <17101802+thampiotr@users.noreply.github.com>	2025-11-04 11:59:23 +00:00
Ben Kochie	48956f60d7	Update modernize (#17471) ... Apply additional Go modernize tool improvements. Signed-off-by: SuperQ <superq@gmail.com>	2025-11-04 05:13:49 +00:00
Julius Volz	0093e2159e	Merge pull request #17337 from prometheus/ui/visualize-relabel-steps ... ui: Allow viewing detailed relabeling steps for each discovered target	2025-11-02 13:51:55 +01:00
Lukasz Mierzwa	aac472df5b	Fix TestScrapeLoop_HistogramBucketLimit ... TestScrapeLoop_HistogramBucketLimit tests the bucket limiter but it also sets sample_limit to the same value, which seems incorrect. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-10-24 10:04:18 +01:00
Lukasz Mierzwa	ce520b9256	Include histograms in sample_limit logic ... Currently histograms bypass sample_limit logic as the limitAppender only implements the Append() method, while histograms are appended using AppendHistogram. This means that they are effectively ignored during sample_limit checks and a scrape with sample_limit=100 and 500 histograms will accept all samples. Add AppendHistogram method to the limitAppender so histograms are also counted towards sample_limit. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-10-24 10:04:18 +01:00
Julius Volz	8b1bd7d6c3	ui: Allow viewing detailed relabeling steps for each discovered target ... This adds: * A `ScrapePoolConfig()` method to the scrape manager that allows getting the scrape config for a given pool. * An API endpoint at `/api/v1/targets/relabel_steps` that takes a pool name and a label set of a target and returns a detailed list of applied relabeling rules and their output for each step. * A "show relabeling" link/button for each target on the discovery page that shows the detailed flow of all relabeling rules (based on the API response) for that target. Note that this changes the JSON encoding of the relabeling rule config struct to output the original snake_case (instead of camelCase) field names, and before merging, we need to be sure that's ok :) See my comment about that at https://github.com/prometheus/prometheus/pull/15383#issuecomment-3405591487 Fixes https://github.com/prometheus/prometheus/issues/17283 Signed-off-by: Julius Volz <julius.volz@gmail.com>	2025-10-15 15:33:27 +02:00
beorn7	ad7d1aed99	Phase out native histogram feature flag ... The detailed plan for this is laid out in https://github.com/prometheus/prometheus/issues/16572 . This commit adds a global and local scrape config option `scrape_native_histograms`, which has to be set to true to ingest native histograms. To ease the transition, the feature flag is changed to simply set the default of `scrape_native_histograms` to true. Further implications: - The default scrape protocols now depend on the `scrape_native_histograms` setting. - Everywhere else, histograms are now "on by default". Documentation beyond the one for the feature flag and the scrape config are deliberately left out. See https://github.com/prometheus/prometheus/pull/17232 for that. Signed-off-by: beorn7 <beorn@grafana.com>	2025-10-15 14:50:52 +02:00
beorn7	72960c076d	scrape: "Modernize" test files ... Applied the analyzer "modernize" to the test files. $ go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./... Signed-off-by: beorn7 <beorn@grafana.com>	2025-10-14 19:30:22 +02:00
Harsh	4d7d8ebcfa	lint fixes ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-13 13:59:10 +05:30
harsh kumar	a612cd8954	Update scrape/manager_test.go ... Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com>	2025-10-13 12:41:46 +05:30
harsh kumar	5244b0e3f7	Update scrape/manager_test.go ... Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com>	2025-10-13 00:50:58 +05:30
harsh kumar	a1e163a462	Update scrape/manager_test.go ... Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com>	2025-10-13 00:50:42 +05:30
Harsh	a63414b8e8	suggested changes added ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-12 21:29:18 +05:30
Harsh	3a7a8d7547	promql: Enhance TestNHCBAndCTZeroIngestion to validate exemplar parsing with created timestamps ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-11 17:27:47 +05:30
harsh kumar	2b2cbec8cd	Merge branch 'prometheus:main' into fix-17224-remove-nhcb-check	2025-10-11 17:13:22 +05:30
Ali Nazari	f64ee61312	Use rlock in read methods of scrape target ... Signed-off-by: Ali Nazari <ali4nazari4@gmail.com>	2025-10-08 09:06:09 +03:30
Harsh	febd734137	test: Add TestNHCBAndCTZeroIngestion to verify simultaneous feature usage ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-08 09:07:24 +05:30
Harsh	c6793e717a	Remove obsolete check ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-08 08:41:43 +05:30
György Krajcsovits	bdf547ae9c	fix(nativehistograms): validation should fail on unsupported schemas ... Histogram.Validate and FloatHistogram.Validate now return error on unsupported schemas. Scrape and remote-write handler reduces the schema to the maximum allowed if it is above the maximum, but below theoretical maximum of 52. For scrape the maximum is a configuration option, for remote-write it is 8. Note: OTLP endpont already does the reduction, without checking that it is below 52 as the spec does not specify a maximum. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-09-13 16:54:44 +02:00
NamanParlecha	594f9d63a5	refactor(textparse): Introduce Variadic options in textParse.New (#17155) ... * refactor(textparse): introduce ParserOptions struct for cleaner parser initialization Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(fuzz): update fuzzParseMetricWithContentType to use ParserOptions Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): simplify ParserOptions usage in tests and implementations Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parse): using variadic options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): add fallbackType & SymbolTable to variadic options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): private fields Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(scrape): compose parser options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): add comments Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): update to use ParserOptions struct for configuration Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(scrape): remove unused parserOptions field from scrapeLoop Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): update ParserOptions field names and add comments for clarity Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> --------- Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>	2025-09-11 10:49:42 +01:00
Arve Knudsen	913cc8f72b	Replace gopkg.in/yaml.v2 with go.yaml.in/yaml/v2 (#17151) ... * Replace gopkg.in/yaml.v2 with go.yaml.in/yaml/v2 * Upgrade to client_golang@v1.23.2 --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2025-09-06 13:04:24 +02:00
George Krajcsovits	31e4d84edd	refactor(textparse): allow for parsers with direct NHCB support (#17153) ... Hide adding NHCB parser on top another parser in New() function so we can easily add direct NHCB capable parsers. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-09-06 11:45:44 +02:00
Owen Williams	6ee965c255	common: Update to prom/common v0.66.0, fix TextParser creation (#17139) ... TextParser as of prom/common v0.66.0 requires an explicit validation scheme. Signed-off-by: Owen Williams <owen.williams@grafana.com>	2025-09-03 11:20:04 -04:00
Bryan Boreham	8563ed03e0	Scraping: use clear builtin function ... This was added in Go 1.21, and is neater than a loop deleting all elements. Also move the comment noting why we do this, because it could be read as saying this is the only reason we have two maps. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-09-02 15:18:14 +01:00
Bryan Boreham	5915a013b7	Scraping: detect staleness via unique reference ... Instead of the labels hash, which could collide between two different series, use the SeriesRef which is unique. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-09-02 15:17:24 +01:00
Bryan Boreham	8dea1f04a5	Scrape tests: Better series references ... `collectResultAppender` on its own will now use the labels hash instead of a random number. This avoids the situation where a series could be added twice under different references. When there is an underlying appender, use the reference it generates. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-09-02 14:51:19 +01:00