* actions: pin to latest actions
- actions/checkout@9c091bb21b => v7.0.0
Adds a guardrail to prevent accidentally checking out fork pull
request code in privileged GitHub Actions contexts
(pull_request_target and PR-triggered workflow_run), with an
explicit opt-in escape hatch for advanced workflows.
- pnpm/action-setup@0ebf47130e => v6.0.9
Update pnpm to v11.7.0
- Add .github/actions/build-ui to ui changed files group
- Add .github/actions/build-ui to ui/frontend CODEOWNERS
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Add separate Saturday morning sweeps (3 AM & 5 AM ET) for IAM resources
that lack age metadata or tag support. Conditionally exclude `IAMUserPolicy`,
`IAMRolePolicy`, and policy attachments from weekday sweeps to prevent
interference with active CI runs.
Additional changes:
- Add explicit `America/New_York` timezone to workflow schedules
- Update nightly tests to run at 9 AM ET (was 1 PM UTC)
- Grant `iam:TagInstanceProfile` permission to CI service user to tag instance
profiles to build date based nuke filters.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Since moving to the standard runner labels we've seen a significant
decrease in job wait time but the smaller runners that we can get for
various jobs are too unreliable and we're seeing many disconnects.
Instead, increase the baseline runner size for build and test
workflows to large. This will likely require waiting longer but that
is almost certainly better than a runner disconnect and retry.
Signed-off-by: Ryan Cragun <me@ryan.ec>
* updates build-ui action to run setup-node before pnpm/action-setup
* fixes issue with empty pnpm store path
* executes pnpm from directory since it isn't on the path
* updates to hardcoded pnpm store path
* one more attempt at getting pnpm store path
* reverts to hardcoded path
* fixes bad revert
* updates pnpm store cache key and adds cache hit guard
* removes restore-key
Co-authored-by: Jordan Reimer <zofskeez@gmail.com>
* Use a large runner. When we get small ones the runner can get OOMed.
* Don't run the action in the context of the container so we can use
the runners Node 24 to assume the role before executing the quota
check.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* Complete vault_verify_replication migration to blackbox tests
Migrate all remaining enos scenarios to use vault_run_blackbox_test:
- enos-scenario-proxy.hcl
- enos-scenario-seal-ha.hcl
- enos-scenario-upgrade.hcl
- enos-scenario-agent.hcl
- enos-scenario-autopilot.hcl
Remove vault_verify_replication module from enos-modules.hcl
All scenarios now use the blackbox test framework for replication verification.
* Update setup-enos action to v1.53
Co-authored-by: hashigator <280075563+hashigator@users.noreply.github.com>
Add the vault-frontend team to each CODEOWNERS entry that
already lists vault-ui.
Keep frontend reviewers on general UI changes and the
OIDC/JWT/SAML exceptions alongside the existing ecosystem
ownership.
Co-authored-by: Angelo Cordon <angelo.cordon@hashicorp.com>
Co-authored-by: OpenCode (gpt-5.4) <opencode.noreply@hashicorp.com>
* feat(enos): migrate vault_verify_replication to blackbox tests
Convert vault_verify_replication from bash scripts to blackbox SDK tests.
Changes:
- Created vault/external_tests/blackbox/verify/replication_test.go with TestReplicationAvailability
- Updated enos-scenario-smoke.hcl to use vault_run_blackbox_test module
- Removed enos/modules/vault_verify_replication module and bash script
The new test verifies:
- CE: replication mode is 'disabled'
- ENT: DR and performance replication are available
Fixes: Converts bash-based verification to Go-based blackbox tests for better maintainability
* Add detailed error messages to replication test for debugging
* Add debug logging to replication test
* Exclude TestReplicationAvailability from race detection
The TestReplicationAvailability test requires a live Vault instance with
VAULT_ADDR and VAULT_TOKEN environment variables set. This test is not
compatible with race detection runs in CI which don't have these
prerequisites configured.
Add //go:build !race tag to exclude this test from race detection runs.
* Revert "Exclude TestReplicationAvailability from race detection"
This reverts commit 5afc7c1bf243e7e833864288cdd5bd16c9ed3018.
* Fix replication test to read from root namespace
The test was failing because it tried to read sys/replication/status
from within the test's isolated namespace. Replication status is only
available at the root namespace level.
Changes:
- Use WithRootNamespace() to read replication status from root
- Add proper error handling for the namespace operation
- Add api import for WithRootNamespace return type
* Add testonly build tag and update CI workflow pattern for verify tests
* Add missing ip_version parameter to vault_run_blackbox_test calls
Co-authored-by: hashigator <280075563+hashigator@users.noreply.github.com>
* actions: expressions in composite action defaults don't work 🫢 (#15023)
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* go: remove SKIP_SETCAP env vars and add IPC_LOCK when using vault containers
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Backport community files that changed as part the enterprise only zap scenarios. This mostly includes fixes to scenario execution, retries, and blackbox SDK tests that were broken.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* copies v2 form components from POC branch
* fixes issue in form-config-generator when path parameters are not defined
* adds api code-generator for snippet creation
* expands cli and terraform code generators
* updates form-config-generator to return api path from spec
* fixes issue setting field value in v2-form class
* updates form-config types
* updates v2 form and renderer components to conditional render fields
* adds v2 form apply component
* updates v2 form wizard component to support apply step
* add support for field types (text input variants, text area, checkbox, radio, masked input) and add test coverage
* Dynamic field visibility and Select field support
* [POC] Public PKI (mocked) Wizard - revert this before merging
* Revert "[POC] Public PKI (mocked) Wizard - revert this before merging"
This reverts commit 66646f1d7a71d0e67028ebcabcfe33925197ffc9.
* cleanup & address copilot pr comments
* address PR comments
---------
Co-authored-by: Shannon Roberts (Beagin) <beagins@users.noreply.github.com>
Co-authored-by: Jordan Reimer <jordan.reimer@hashicorp.com>
We've been seeing Github throw 500s a lot today when downloading nfpm.
My assumption is that this is due to throttling while Github works to
resolve their platform reliability. Instead of relying on the default
workflow token which has a lower priority than users, use the service
user token when calling the action that downloads nfpm to package Vault.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
I've only seen a single instance where this can fail but even if it does
it should not prevent merges in an otherwise successful run.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Use standard runner labels for all workflows. This will allow us to pull
from the hot pools for most jobs and on-demand when more are needed.
This does elimate our cost optimization but latest on-demand runners
have taken so long to provision as to be unbearable.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* updating matrix workflow format for easier visualization
* adding test to create and delete Vault AWS Roles
* refactoring functions
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* finishing up role deletion test
* finishing up role deletion test
Co-authored-by: Tin Vo <tintvo08@gmail.com>
* [VAULT-42245] Add IBM license update to enos upgrade scenario (#12661)
* initial changes
* more changes
* test
* test changes
* Fix test
* try ignoring customer id
* clean up
* more clean up
* lint
* PR comments
* make edition a variable
* lint
* PR comments
* add default for customer id
* fix script and lint
* specify license file
* Apply suggestion from @ryancragun
Co-authored-by: Ryan Cragun <me@ryan.ec>
* always configure ibm license
* Update enos/modules/verify_log_secrets/main.tf
Co-authored-by: Ryan Cragun <me@ryan.ec>
* lint
---------
Co-authored-by: Ryan Cragun <me@ryan.ec>
* lint
---------
Co-authored-by: Jenny Deng <jenny.deng@hashicorp.com>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* [VAULT-43364] pipeline: add template generation support
Add a new `template` to the `pipeline generate` command tree. It allows
rendering Go text templates with pipeline requests available via context
functions. The new system is now product agnostic and can be used to
generate any template we wish. This will supersede the enos specific
configuration command.
We also add support for multiple cadences when fetching the list of
release versions. Previously it was assumed that we followed a minor
version bump cadence when fetching versions with an n-minus style lower
bound. Now we can specify the major or minor cadence. To support a
migration from one cadence to another you can also specify an prior
cadence and the version at which the transition happened. This allows
the n-3 reverse traversal to drop into the prior cadence if/when
necessary.
**Template Rendering System**
- New `pipeline generate template` command renders Go templates with
pipeline data access
- Supports stdin/stdout or file-based input/output
- Templates access version data via function calls rather than
pre-populated context
**Version Cadence Support**
- Added `VersionCadence` type with `minor` and `major` release cadence
tracking
- Supports cadence transitions (e.g., minor→major) with
`TransitionVersion` and `PriorCadence` fields
- Calculates version ranges respecting different release cadences
**Template Functions**
- `VersionsNMinus` / `VersionsBounded` - List versions with explicit
cadence parameter
- `VersionsNMinusTransition` / `VersionsBoundedTransition` - Handle
cadence transitions
- `ParseVersion`, `CompareVersions`, `FilterVersions` - Version
utilities
- All functions require cadence to be explicitly specified
**CLI Integration**
- `--version` and `--edition` flags expose current version/edition to
templates
- Templates reference these via `.Version` and `.Edition` context fields
**Enos Migration**
- Converted `enos-dynamic-config.hcl` to template-based generation
- Uses `VersionsNMinusTransition` to handle Vault's minor→major cadence
shift at 1.21.5
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* Add LDAP secrets engine blackbox tests
* Format
* format
* cleanup environment
* Install ldap-utils in CI for LDAP domain provisioning
* wrap in eventually
* debugging
* fix ip issues
Co-authored-by: Luis (LT) Carbonell <lt.carbonell@hashicorp.com>
- actions/cache => v5.0.4
Dep updates
- actions/download-artifact => v8.0.1
Support for CJK characters
- dorny/paths-filter => v4.0.1
Node 24, support for merge queues
- hashicorp/action-setup-enos => v1.52
Security release for downstream vuln
- pnpm/action-setup => v5.0.0
Node 24, support for native caching
- slackapi/slack-github-action => v3.0.1
Node 24, lots of internal dep updates, ability to run Slack commands
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* Fix GitHub Actions expression evaluation error in build workflow
- Add hcp-setup job with explicit step-by-step parameter validation
- Replace problematic inline expressions with debuggable logic steps
- Use proper fallback values (0 instead of '') for number type inputs
- Resolve 'Unexpected value' error on scheduled runs
- Maintain existing workflow logic and conditional behavior
- Add clear logging for troubleshooting parameter resolution
* Fix type conversion for pull-request number in build workflow
- Use fromJSON() to convert string output to number type
- Resolves type mismatch error in reusable workflow input
Co-authored-by: Luis (LT) Carbonell <lt.carbonell@hashicorp.com>
* actions: pull in gotestsum when executing the cloud scenario
* cloud: add 'hcp' changed-file group and trigger cloud scenario when the files change
* slightly simplify expression
---------
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
- docker/setup-buildx-action v3.12.0 => v4.0.0
Node 24 upgrade, switch to ESM, some deprecated inputs have been
removed.
- docker/build-push-action v6.19.2 => v7.0.0
Node 24 upgrade, switch to ESM, some deprecated envs have been
removed.
- actions/setup-node v6.2.0 => v6.3.0
Bug fixes, internal dep updates, support for parsing `devEngines`.
- action-setup-enos v1.50 => v1.51
Use enos 0.0.36
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* rework UI CI workflow to partition JS tests (#11967)
* add setup-pnpm action
* remove reading vault keys from vault server output
* update ci workflow to build app and go binary first, then run tests in partitions
* fix errant tests
* address PR feedback
* Apply suggestions from code review
Co-authored-by: Ryan Cragun <me@ryan.ec>
* more feedback changes
* restore test-helper.js
* restore auth test helpers
* check in ui/tests/helpers/vault-keys.js
* use v7 of download-artifact action
* make test-ui reusable workflow
* add status job
---------
Co-authored-by: Ryan Cragun <me@ryan.ec>
* update new UI tests to run CE tests on the CE branch (#12537)
---------
Co-authored-by: Matthew Irish <39469+meirish@users.noreply.github.com>
Co-authored-by: Ryan Cragun <me@ryan.ec>
The `pipeline` utility started as collection of small CLI utilities that we found useful for the Vault CI/CD pipeline. Rather than engineering complex bash scripts in YAML blocks, instead, we could build small, reusable, testable actions and integrate the into a single binary. No more copying and pasting loads of bash from YAML, instead we can copy a single command and run the same thing locally that we can in CI.
As we've continued to invest in the utilities capability, it's become clear that other CI pipelines would benefit from the same functionality that we've been building. This change represents the first significant work to make the utility truly generic in a HashiCorp repo that utilizes CRT sense. Once all the Vault specifics have been extracted we hope to move the utility out of the repo and make it available everywhere.
The primary change here is to move our changed file grouping configuration out of the `changed` package entirely. Instead of checkers that are written as Go code, we have created a new configuration file for the `pipeline` utility called `pipeline.hcl` While there are certainly other things that will eventually be configurable here, the only thing we've added support for is `changed_files`, which allows configuring how to match a given changed files path to a group name.
The DSL is fairly simple:
```hcl
changed_files {
// One or more groups can be defined
group "group_name_label" {
// Zero or more ignore blocks can be defined
ignore {
base_dir = []
base_name = []
base_name_prefix = []
contains = []
extension = []
file = []
}
// One or more match blocks can be defined
match {
base_dir = []
base_name = []
base_name_prefix = []
contains = []
extension = []
file = []
}
}
}
```
For example,
```hcl
// Create a changed_files block where we can define our changed files groups
changed_files {
// Group blocks take one label which is the name of the group
group "app" {
// Groups can ignore based on some criteria.
ignore {
// In this instance, we'll ignore any file that begins with
// tools/pipeline. All paths will be relative to the git repository
// root directory. The joinpath() function is here to support paths
// that are agnostic to the operating systems path separator. While
// it's unlikely that you'll need them, several cty stdlib functions
// are available.
base_dir = [joinpath("tools", "pipeline")]
}
// Groups must define at least one match block.
match {
// This will match any file with the .go extension (except for
// those that will be excluded with our ignore directive aboe
extension = [".go"]
}
// Groups can contain more than one match block. If any of the match
// blocks meet their criteria the group will be associated with the
// changed file
match {
base_name = ["go.mod", "go.sum"]
}
// If groups have more than one attribute set, each attribute group
// must match in order for the match.
match {
// Here we only match files that contain "raft_autopilot" in the
// path with the .go extension
extension = [".go"]
contains = ["raft_autopilot"]
}
}
group "autopilot" {
// Ignore blocks have the same attributes as match blocks
match {
// The base directory.
base_dir = [
"changelog",
joinpath("tools", "codechecker"),
]
// The base of the file
base_name = ["README.md"]
// A prefix string match on a files name.
base_name_prefix = ["buf."]
// Any string match in the files full path
contains = [
"-ce",
"_ce",
"-oss",
"_oss",
]
// The file's extension
extension = [
".hcl",
".md",
".sh",
".yaml",
".yml",
]
// An exact file match
file = [
# These exist on CE branches to please Github Actions.
joinpath(".github", "workflows", "build-artifacts-ent.yml"),
joinpath(".github", "workflows", "backport-automation-ent.yml"),
]
}
}
}
```
The default location of the config is `.release/pipeline.hcl`. All of our prior checks have been migrated to the DSL file present in this change.
- We had several commands that used the changed files groups that were built into the library. This change requires us to instead load the configuration from the file and use the user defined groupings.
- Several commands now take some part of that configuration in the request type. When possible we use the version parsed by the root command and verify in the request body rather than attempt to load the configuration.
- We also refactor the loading and parsing of `.release/versions.hcl` in the same manner. Now we automatically parse the file in the default locations relative to the git repo root.
- Our root command now has two new flags `--pipeline-config` and `--versions-config` which allow specifying a default location for each file. Commands which previously accepted flags or args to configure the versions file have been updated to use the global root flags instead. We've also removed the previous implementation that would recursively search backwards from the working directory to find the `versions.hcl` file. Instead we only support loading the file from the default location relative to the Git repo root.
- All instances of changed `pipeline` command invocations have been update to support the new auto-loading of configuration.
- A new configuration sub-command with validation exists to quickly validate a configuration file. `pipeline config validate`
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* adding ibm tests for ent files
* adding debug commands
* adding code changes
* adding reload tests
* remove settings.json
* remove ryboe q
* changing isHashicorpLicense to isIBMLicense and moving DiagnoseCheckLicenseGeneration to core_util_common.go
* fix test
* reverting non-license related tests
* reverting non-license related tests
* removing hashicorp license test
* modify reload server_ent_test.go
* change ibm-license paths
* adding census reload server test
* moving LicensingEntitlementSelectionConfig to core_util_common.go
* add EntReloadLicenseAndConfig to stubs
* fix operator diagnose bug
* move bug fix into ce and ent files
* add more ibm test cases
* Update command/command_testonly/server_testonly_ent_test.go
* address comments
* make fmt
---------
Co-authored-by: akshya96 <87045294+akshya96@users.noreply.github.com>
Co-authored-by: Jenny Deng <jenny.deng@hashicorp.com>
* rough draft
* add some stuff for dynamic secrets
* add some more helpers and sample tests
* new helpers, new tests, refactoring
* Add Basic Smoke SDK Scenario (#11678)
* Add simple test for stepdown election
* Add a smoke_sdk scenario
* add script to run tests locally
* fix up a few things
* VAULT-39746 - Add Tests to Smoke SDK and Cloud Scenarios (#11795)
* Add some go verification steps in enos sdk test run script
* formatting
* Add a smoke_sdk scenario userpass secret engine create test (#11808)
* Add a smoke_sdk scenario userpass secret engine create test
* Add the some additional tests
* Add Smoke tests to Cloud Scenario (#11876)
* Add a smoke_sdk scenario userpass secret engine create test
* Add the some additional tests
* Add smoke testing to cloud
* Add test results to output and test filtering
* comment
* fix test
* fix the smoke scenario
* Address some various feedback
* missed cleanup
* remove node count dependency in the tests
* Fix test perms
* Adjust the testing and clean them up a bit
* formatting
* fmt
* fmt2
* more fmt
* formatting
* tryagain
* remove the docker/hcp divide
* use the SHA as ID
* adjust perms
* Add transit test
* skip blackbox testing in test-go
* copywrite
* Apply suggestion from @brewgator
* Add godoc
* grep cleanup
---------
Co-authored-by: Josh Black <raskchanky@gmail.com>
Co-authored-by: Luis (LT) Carbonell <lt.carbonell@hashicorp.com>
