Extended Kubernetes SD to support the following pod-based labels:
* `__meta_kubernetes_pod_deployment_name`
* `__meta_kubernetes_pod_cronjob_name`
* `__meta_kubernetes_pod_job_name`
Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>
This adds documentation for the external_id parameter when assuming
AWS roles in the sigv4 configuration.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* AWS SD: Elasticache
This change adds Elasticache to the AWS SD.
Co-authored-by: Ben Kochie <superq@gmail.com>
Signed-off-by: Matt <small_minority@hotmail.com>
---------
Signed-off-by: Matt <small_minority@hotmail.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
Namely promql-duration-expr and promql-extended-range-selectors. This
allows promtool to e.g. check rules files using syntax gated by these
features.
Signed-off-by: Ian Kerins <git@isk.haus>
* Add OpenAPI 3.2 specification generation for Prometheus HTTP API
This commit introduces an OpenAPI specification for the Prometheus API.
After testing multiple code-generation servers with built-in APIs, this
implementation uses an independent spec file outside of the critical path.
This spec file is tested with a framework present in this pull request.
The specification helps clients know which parameters they can use and is
served at /api/v1/openapi.yaml. The spec file will evolve with the
Prometheus API and has the same version number.
Downstream projects can tune the APIs presented in the spec file with
configuration options using the IncludePaths setting for path filtering.
In the future, there is room to generate a server from this spec file
(e.g. with interfaces), but this is out of scope for this pull request.
Architecture:
- Core OpenAPI infrastructure (openapi.go): Dynamic spec building,
caching, and thread-safe spec generation
- Schema definitions (openapi_schemas.go): Complete type definitions
for all API request and response types
- Path specifications (openapi_paths.go): Endpoint definitions with
parameters, request bodies, and response schemas
- Examples (openapi_examples.go): Realistic request/response examples
- Helper functions (openapi_helpers.go): Reusable builders for common
OpenAPI structures
Testing:
- Comprehensive test suite with golden file validation
- Test helpers package for API testing infrastructure
- OpenAPI compliance validation utilities
The golden file captures the complete specification for snapshot testing.
Update with: go test -run TestOpenAPIGolden -update-openapi-spec
REVIEWERS: The most important thing to check would be the OpenAPI golden
file (web/api/v1/testdata/openapi_golden.yaml). Test scenarios are important
as they test the actual OpenAPI spec validity.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* Add OpenAPI 3.1 support with version selection
Add support for both OpenAPI 3.1 and 3.2 specifications with version
selection via openapi_version query parameter. Defaults to 3.1 for
broader compatibility
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* Enhance OpenAPI examples and add helper functions
- Add timestampExamples helper for consistent time formatting
- Add exampleMap helper to simplify example creation
- Improve example summaries with query details
- Add matrix result example for range vector queries
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* web/api: Add AtST method to test helper iterators
Implement the AtST() method required by chunkenc.Iterator interface
for FakeSeriesIterator and FakeHistogramSeriesIterator test helpers.
The method returns 0 as these test helpers don't use start timestamps
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* OpenAPI: Add minimum coverage test
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* OpenAPI: Improve examples handling
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---------
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Previously the AWS SD ECS Role only discovered instances that used
`awsvpc` network mode, which attaches a dedicated Elastic Network
Interface (ENI). This change adds in additional logic so that we
discover instances that are using `host` and `bridge` networking modes,
where the IP address is that of the EC2 instance that is hosting the
container. Also this change exposes a number of additional labels that
relate to the EC2 instance when the launch type is `EC2`.
Signed-off-by: matt-gp <small_minority@hotmail.com>
* promtool: allow cardinality with metrics linting and add --lint to check metrics
Signed-off-by: ADITYA TIWARI <adityatiwari342005@gmail.com>
* fix/ci: Simplify test case variable declaration
Remove unnecessary variable declaration in test cases.
Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com>
* promtool: avoid Tee for --lint=none
Signed-off-by: ADITYA TIWARI <adityatiwari342005@gmail.com>
* promtool: validate at least one feature enabled in check metrics
addresses feedback to ensure the command does something useful
now fails with clear error when both --lint=none and no --extended flag.
Signed-off-by: ADITYA TIWARI <adityatiwari342005@gmail.com>
---------
Signed-off-by: ADITYA TIWARI <adityatiwari342005@gmail.com>
Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com>
Add a maximum limit of 10,000 to the TSDB status endpoint to prevent
resource exhaustion from excessively large limit values, as we preallocate
[]Stat for up to the limit: `make([]Stat, 0, length)`.
Note that the endpoint acquires a cardinality mutex during
stats calculation, so this can not be run in parallel.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
This commit adds support for configuring a custom start timestamp
for Prometheus unit tests, allowing tests to use realistic timestamps
instead of starting at Unix epoch 0.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
The return value of functions relating to the current time, e.g. time(),
is set by promtool to start at timestamp 0 at the start of a test's
evaluation.
This has the very nice consequence that tests can run reliably without
depending on when they are run.
It does, however, mean that tests will give out results that can be
unexpected by users.
If this behaviour is documented, then users will be empowered to write
tests for their rules that use time-dependent functions.
(Closes: prometheus/docs#1464)
Signed-off-by: Gabriel Filion <lelutin@torproject.org>
* Delay compactions until Thanos uploads all blocks
Using Thanos sidecar with Prometheus requires us to disable TSDB compactions on Prometheus side by setting --storage.tsdb.min-block-duration and --storage.tsdb.max-block-duration to the same value. See https://thanos.io/tip/components/sidecar.md. The main problem this avoids is that Prometheus might compact given block before Thanos uploads it, creating a gap in Thanos metrics. Thanos does not upload compacted blocks because that would upload the same sample multiple times. You can tell Thanos to upload compacted blocks but that is aimed at one time migrations. This patch creates a bridge between Thanos and Prometheus by allowing Prometheus to read the shipper file Thanos creates, where it tracks which blocks were already uploaded, and using that data delays compaction of blocks until they are marked as uploaded by Thanos. Thanks to this both services can coordinate with each other (in a way) and we can stop disabling compaction on Prometheus side when Thanos uploads are enabled.
The reason to have this is that disabling compactions have very dramatic performance cost. Since most time series exist for longer than a single block duration (2h by default) large chunks of block index will reference the same series, so 10 * 2h blocks will each have an index that is usually fairly big and is almost the same for all 10 blocks. Compaction de-duplicates the index so merging 10 blocks together would leave us with a single index that is around the same size as each of these 10 2h blocks would have (plus some extra for series that only exists in some blocks, but not all). Every range query that iterates over all 10 blocks would then have to read each index and so we're doing 10x more work then if we had a single compacted block.
Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>
* Rename structs and functions to make this more generic
Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>
* Address review comments
Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>
* Cache UploadMeta for 1 minute
Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>
---------
Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>
