mirror of
https://github.com/prometheus/prometheus.git
synced 2026-04-29 10:12:14 -04:00
8fd73b1d28
* Write exemplars to the WAL and send them over remote write. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update example for exemplars, print data in a more obvious format. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add metrics for remote write of exemplars. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix incorrect slices passed to send in remote write. Signed-off-by: Callum Styan <callumstyan@gmail.com> * We need to unregister the new metrics. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address review comments Signed-off-by: Callum Styan <callumstyan@gmail.com> * Order of exemplar append vs write exemplar to WAL needs to change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Several fixes to prevent sending uninitialized or incorrect samples with an exemplar. Fix dropping exemplar for missing series. Add tests for queue_manager sending exemplars Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Store both samples and exemplars in the same timeseries buffer to remove the alloc when building final request, keep sub-slices in separate buffers for re-use Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Condense sample/exemplar delivery tests to parameterized sub-tests Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Rename test methods for clarity now that they also handle exemplars Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Rename counter variable. Fix instances where metrics were not updated correctly Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Add exemplars to LoadWAL benchmark Signed-off-by: Callum Styan <callumstyan@gmail.com> * last exemplars timestamp metric needs to convert value to seconds with ms precision Signed-off-by: Callum Styan <callumstyan@gmail.com> * Process exemplar records in a separate go routine when loading the WAL. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address review comments related to clarifying comments and variable names. Also refactor sample/exemplar to enqueue prompb types. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Regenerate types proto with comments, update protoc version again. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Put remote write of exemplars behind a feature flag. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address some of Ganesh's review comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Move exemplar remote write feature flag to a config file field. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address Bartek's review comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Don't allocate exemplar buffers in queue_manager if we're not going to send exemplars over remote write. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add ValidateExemplar function, validate exemplars when appending to head and log them all to WAL before adding them to exemplar storage. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address more reivew comments from Ganesh. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add exemplar total label length check. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address a few last review comments Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Martin Disibio <mdisibio@gmail.com>
55 lines
No EOL
2.7 KiB
Markdown
55 lines
No EOL
2.7 KiB
Markdown
---
|
||
title: Disabled Features
|
||
sort_rank: 10
|
||
---
|
||
|
||
# Disabled Features
|
||
|
||
Here is a list of features that are disabled by default since they are breaking changes or are considered experimental.
|
||
Their behaviour can change in future releases which will be communicated via the [release changelog](https://github.com/prometheus/prometheus/blob/main/CHANGELOG.md).
|
||
|
||
You can enable them using the `--enable-feature` flag with a comma separated list of features.
|
||
They may be enabled by default in future versions.
|
||
|
||
## `@` Modifier in PromQL
|
||
|
||
`--enable-feature=promql-at-modifier`
|
||
|
||
The `@` modifier lets you specify the evaluation time for instant vector selectors,
|
||
range vector selectors, and subqueries. More details can be found [here](querying/basics.md#modifier).
|
||
|
||
## Expand environment variables in external labels
|
||
|
||
`--enable-feature=expand-external-labels`
|
||
|
||
Replace `${var}` or `$var` in the [`external_labels`](configuration/configuration.md#configuration-file)
|
||
values according to the values of the current environment variables. References
|
||
to undefined variables are replaced by the empty string.
|
||
|
||
## Negative offset in PromQL
|
||
|
||
This negative offset is disabled by default since it breaks the invariant
|
||
that PromQL does not look ahead of the evaluation time for samples.
|
||
|
||
`--enable-feature=promql-negative-offset`
|
||
|
||
In contrast to the positive offset modifier, the negative offset modifier lets
|
||
one shift a vector selector into the future. An example in which one may want
|
||
to use a negative offset is reviewing past data and making temporal comparisons
|
||
with more recent data.
|
||
|
||
More details can be found [here](querying/basics.md#offset-modifier).
|
||
|
||
## Remote Write Receiver
|
||
|
||
`--enable-feature=remote-write-receiver`
|
||
|
||
The remote write receiver allows Prometheus to accept remote write requests from other Prometheus servers. More details can be found [here](storage.md#overview).
|
||
|
||
## Exemplars Storage
|
||
|
||
`--enable-feature=exemplar-storage`
|
||
|
||
[OpenMetrics](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#exemplars) introduces the ability for scrape targets to add exemplars to certain metrics. Exemplars are references to data outside of the MetricSet. A common use case are IDs of program traces.
|
||
|
||
Exemplar storage is implemented as a fixed size circular buffer that stores exemplars in memory for all series. Enabling this feature will enable the storage of exemplars scraped by Prometheus. The flag `storage.exemplars.exemplars-limit` can be used to control the size of circular buffer by # of exemplars. An exemplar with just a `traceID=<jaeger-trace-id>` uses roughly 100 bytes of memory via the in-memory exemplar storage. If the exemplar storage is enabled, we will also append the exemplars to WAL for local persistence (for WAL duration). |