From c92957871816fc46222ca585ace0e99ca5e5b3db Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Fri, 24 Oct 2025 12:29:18 +0200 Subject: [PATCH] feat(nh): mark native histograms as stable in docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes: #16572 Mark as stable means that breaking changes are only allowed together with major version release of Prometheus. Co-authored-by: Björn Rabenstein Signed-off-by: George Krajcsovits Signed-off-by: György Krajcsovits --- docs/federation.md | 19 +++++------ docs/migration.md | 4 +++ docs/querying/api.md | 9 ++---- docs/querying/basics.md | 70 +++++++++++++++++++++++++++-------------- 4 files changed, 64 insertions(+), 38 deletions(-) diff --git a/docs/federation.md b/docs/federation.md index 580fa30874..cb0fad7f69 100644 --- a/docs/federation.md +++ b/docs/federation.md @@ -6,15 +6,16 @@ sort_rank: 6 Federation allows a Prometheus server to scrape selected time series from another Prometheus server. -_Note about native histograms (experimental feature): To scrape native histograms -via federation, the scraping Prometheus server needs to run with native histograms -enabled (via the command line flag `--enable-feature=native-histograms`), implying -that the protobuf format is used for scraping. Should the federated metrics contain -a mix of different sample types (float64, counter histogram, gauge histogram) for -the same metric name, the federation payload will contain multiple metric families -with the same name (but different types). Technically, this violates the rules of -the protobuf exposition format, but Prometheus is nevertheless able to ingest all -metrics correctly._ +_Note about native histograms: To scrape native histograms via federation, the +scraping Prometheus server needs to set `scrape_native_histograms: true` in its +scrape config, implying that the protobuf format is used for scraping. Should +the federated metrics contain a mix of different sample types (floats of any +flavor, counter histograms, or gauge histograms) for the same metric name, the +federation payload will contain multiple metric families with the same name +(but different types – for technical reasons, float samples are always federated +as `untyped`, while histogram samples are federated with their full type +information). Technically, this violates the rules of the protobuf exposition +format, but Prometheus is nevertheless able to ingest all metrics correctly._ ## Use cases diff --git a/docs/migration.md b/docs/migration.md index 55e366456d..78db5b7d0f 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -43,6 +43,10 @@ This document offers guidance on migrating from Prometheus 2.x to Prometheus 3.0 Prometheus v3 will log a warning if you continue to pass these to `--enable-feature`. +- Starting from Prometheus version v3.8, the feature flag `native-histograms` is + deprecated. Use the new `scrape_native_histograms` global and per-scrape + configuration option instead. + ## Configuration - The scrape job level configuration option `scrape_classic_histograms` has been diff --git a/docs/querying/api.md b/docs/querying/api.md index f077e0c5b9..b9f77a3a9d 100644 --- a/docs/querying/api.md +++ b/docs/querying/api.md @@ -580,9 +580,9 @@ sample values. JSON does not support special float values such as `NaN`, `Inf`, and `-Inf`, so sample values are transferred as quoted JSON strings rather than raw numbers. -The keys `"histogram"` and `"histograms"` only show up if the experimental -native histograms are present in the response. Their placeholder `` -is explained in detail in its own section below. +The keys `"histogram"` and `"histograms"` only show up if native histograms +are present in the response. Their placeholder `` is explained +in detail in its own section below. ### Range vectors @@ -650,9 +650,6 @@ String results are returned as result type `string`. The corresponding The `` placeholder used above is formatted as follows. -_Note that native histograms are an experimental feature, and the format below -might still change._ - ```json { "count": "", diff --git a/docs/querying/basics.md b/docs/querying/basics.md index 4a4f9ada37..e7f1173af4 100644 --- a/docs/querying/basics.md +++ b/docs/querying/basics.md @@ -21,6 +21,34 @@ Other programs can fetch the result of a PromQL expression via the [HTTP API](ap This document is a Prometheus basic language reference. For learning, it may be easier to start with a couple of [examples](examples.md). +## Samples + +The value of a sample at a given timestamp returned by PromQL may be a float or +a [native histogram](https://prometheus.io/docs/specs/native_histograms). A +float sample is a simple floating point number, whereas a native histograms +sample contains a full histogram including count, sum, and buckets. + +Note that the term “histogram sample” in the PromQL documentation always refers +to a native histogram. The term "classic histogram" refers to a set of time +series containing float samples with the `_bucket`, `_count`, and `_sum` +suffixes that together describe a histogram. From the perspective of PromQL, +these contain just float samples, there are no “classic histogram samples”. + +Both float samples and histogram samples can have a counter or a gauge “flavor”. +Float samples with a counter or gauge flavor are generally simply called +“counters” or “gauges”, respectively, while their histogram counterparts are +called “counter histograms” or “gauge histograms”. Float samples do not store +their flavor, leaving it to the user to take their flavor into account when +writing PromQL queries. (By convention, time series containing float counters +have a name ending on `_total` to help with the distinction.) + +Since histogram samples “know” their counter or gauge flavor, this allows +reliable warnings about mismatched operations. For example, applying the `rate` +function to gauge floats will most likely produce a +nonsensical result, but the query will be processed without complains. However, +if applied to gauge histograms, the result of the query will be +annotated with a warning. + ## Expression language data types In Prometheus's expression language, an expression or sub-expression can @@ -37,33 +65,29 @@ user-specified expression. For [instant queries](api.md#instant-queries), any of the above data types are allowed as the root of the expression. [Range queries](api.md#range-queries) only support scalar-typed and instant-vector-typed expressions. -_Notes about the experimental native histograms:_ +Both vectors and time series may contain a mix of float samples and histogram +samples. -* Ingesting native histograms has to be enabled via a [feature - flag](../feature_flags.md#native-histograms). -* Once native histograms have been ingested into the TSDB (and even after - disabling the feature flag again), both instant vectors and range vectors may - now contain samples that aren't simple floating point numbers (float samples) - but complete histograms (histogram samples). A vector may contain a mix of - float samples and histogram samples. Note that the term “histogram sample” in - the PromQL documentation always refers to a native histogram. Classic - histograms are broken up into a number of series of float samples. From the - perspective of PromQL, there are no “classic histogram samples”. -* Like float samples, histogram samples can have a counter or a gauge “flavor”, - marking them as counter histograms or gauge histograms, respectively. In - contrast to float samples, histogram samples “know” their flavor, allowing - reliable warnings about mismatched operations (e.g. applying the `rate` - function to a range vector of gauge histograms). -* Native histograms can have different bucket layouts, but they are generally - convertible to compatible versions to apply binary and aggregation operations - to them. This is not true for all bucketing schemas. If incompatible - histograms are encountered in an operation, the corresponding output vector - element is removed from the result, flagged with a warn-level annotation. - More details can be found in the [native histogram - specification](https://prometheus.io/docs/specs/native_histograms/#compatibility-between-histograms). +## Reconciliation of histogram bucket layouts + +Native histograms can have different bucket layouts, but they are generally +convertible to compatible versions to apply binary and aggregation operations +to them. Functions acting on range vectors that are applicable to native +histograms also perform such reconciliation. In binary operations this +reconciliation is performed pairwise, in aggregation operations and functions +all histogram samples are reconciled to one compatible bucket layout. + +Not all bucket layouts can be reconciled, if incompatible histograms are +encountered in an operation, the corresponding output vector element is removed +from the result, flagged with a warn-level annotation. +More details can be found in the +[native histogram specification](https://prometheus.io/docs/specs/native_histograms/#compatibility-between-histograms). ## Literals +The following section describes literal values of various kinds. +Note that there is no “histogram literal”. + ### String literals String literals are designated by single quotes, double quotes or backticks.