Datasources use their attribute's expressions to determine whether or
not they depend on another datasource, in order to get the list of
dependencies and execute them before executing a datasource.
This code may be useful later on for figuring out the dependencies for
any block, so we move this code to the utils.go file, and use this for
datasources.
When a datasource fails to be evaluated because a cycle has been
detected, we point out one of the links of the chain now so that users
have a better idea of what to look at.
Since datasources are recursively evaluated depending on their
dependencies, we don't need to pre-execute those that depend on nothing,
as the recursive traversal of the datasources will take care of that for
us.
Not sure why this was defined and returned, but the value was set, but
never used, as such this is not useful to keep in the code, so let's
simplify this now.
* Updating the license from MPL to Business Source License
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl.
* Update copyright file headers to BUSL-1.1
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Packer's telemetry package captures the field names from a configuration
on legacy JSON templates, but did not contain any code for handling HCL2
template configurations.
This commit adds a routine to extract the field names from a HCL2
configuration once flattened, and adds some glue code to propagate the
configurations to the telemetry structures.
* hcl2template: use && for chaining conditions
For deciding if we need to run an error provisioner block, we checked
using a nested if that we didn't need to avoid running it.
This is unnecessary, and adds a superfluous level of inbrication to the
code, so we chain both conditions with &&.
* hcl2: don't warn on excluded post-processor
When a post-processor is excluded through the -except command-line
argument for packer build, we used to print a warning on HCL2 templates
if this except statement did not match a build.
However, since we can use the except option to avoid running
post-processors, we should account for this as well when printing out a
warning that there were no matches.
This commit adds this counter to the except checks for the
post-processors as well.
This commit irons out one of the pain points of the HCP rework by
introducing a HCPPublisher interface, implemented both by the JSON Core,
and the HCL2 PackerConfig, which keeps a map of the build names used by
Packer to the build names pushed on HCP.
This in turn lets us go back to the old implementation of the GetBuilds
function, which returns a list of (filtered) builds, and eventually an
error if something went wrong while processing.
* Update validation options for undeclared variables
In an effort to help users move from JSON to HCL2 templates the support for
variable definitions files are being updated to ignore undeclared
variable warnings on build execution. For legacy JSON templates builds no
warnings are displayed when var-files contain undeclared variables.
Since preferred mode HCL2 templates is to be explicit with variable
declarations - they must be declared to be used - validation for
undeclared variables still warns when running `packer validate`. A new
flag has been added to the validate command that can be used to disable
undeclared variable warnings.
* Update validation test for unused variables
Example Run
```
~> go run . validate -no-warn-undeclared-var -var-file
command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl
command/test-fixtures/validate/var-file-tests/basic.pkr.hcl
The configuration is valid.
~> go run . validate -var-file
command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl
command/test-fixtures/validate/var-file-tests/basic.pkr.hcl
Warning: Undefined variable
The variable "unused" was set but was not declared as an input variable.
To declare variable "unused" place this block in one of your .pkr.hcl
files,
such as variables.pkr.hcl
variable "unused" {
type = string
default = null
}
The configuration is valid.
~> go run . build -var-file
command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl
command/test-fixtures/validate/var-file-tests/basic.pkr.hcl
file.chocolate: output will be in this color.
Build 'file.chocolate' finished after 744 microseconds.
==> Wait completed after 798 microseconds
==> Builds finished. The artifacts of successful builds are:
--> file.chocolate: Stored file: chocolate.txt
```
* Rename Strict field to WarnOnUndeclaredVar
The field name Strict is a bit vague since it is only used for
checking against undeclared variables within a var-file definition.
To mitigate against potential overloading of this field it is
being renamed to be more explicit on its usage.
* command/build: Add warn-on-undeclared-var flag
Now that the default behaviour is to not display warnings for undeclared variables
an optional flag has been added to toggle the old behaviour.
```
~> go run . build -warn-on-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl
Warning: Undefined variable
The variable "unused" was set but was not declared as an input variable.
To declare variable "unused" place this block in one of your .pkr.hcl files,
such as variables.pkr.hcl
variable "unused" {
type = string
default = null
}
file.chocolate: output will be in this color.
Build 'file.chocolate' finished after 762 microseconds.
==> Wait completed after 799 microseconds
==> Builds finished. The artifacts of successful builds are:
--> file.chocolate: Stored file: chocolate.txt
```
As part of the work to displace everything related to HCP from scattered
places around the Packer code, we move it all to an hcp package.
This in turn reduces the amount of code that the commands have to
integrate, and leaves the HCP details to its own enclave.
To be able to track HCP builds during a Packer build, we need to
propagate the name used for registering the build to HCP.
This works out-of-the-box for JSON templates, as the build's name is
always the builder's, so there's no confusion on that.
For HCP templates however, there's a possibility for a template to
define a name for the build, which is returned by a CoreBuild's Name()
function, in addition to the source's Type/Name.
The builds are registered to HCP with the source.String() function
however, which does not contain the build's name. This prevents any
build containing a name to work, as their name/source.String does not
match.
The name we registered the build with is stored in a CoreBuild as the
Type attribute, but cannot be safely accessed, as builds are types
within the command as packersdk.Build, which only exposes Name() for
this purpose.
In order to circumvent this problem, and as a way to present a prototype
of solution that will likely have to be discussed before we merge it,
this commit hotfixes the issue.
When a `packer build' is run on a config that does not define anything
to be built, the process would run normally, and only state that no
artifacts were produced after builds were run successfully.
This is misleading as we reach the end of the command without having a
proper warning that nothing was even attempted to be built.
This commit adds a warning when that happens, and leaves immediately
after.
Before this commit, some HCP-related logic was embedded in the parser,
making all of it common to all commands.
This complexifies the logic as some parts rely on the template being in
a HCP-valid environment, and the datasources need to be evaluated in
order for some operations to be executed.
To simplify this and avoid those pitfalls, we move this logic to its own
set of functions that will be performed after the parsing is done, on
any valid HCL2 or JSON template.
* evaluateLocalVariables: modify code for readability and some (not benchmarked) perfs
* Make default input variable type the DynamicPseudoType
This should be the default, and avoids a panic. This type can represent situations where a type is not yet known. Its meaning is undefined in cty.
* do not take Empty types from default value
* Update types.variables.go
Co-authored-by: Wilken Rivera <wilken@hashicorp.com>
* Update HCL2 PackerConfig to call SetDebug mode for CoreBuild when --debug flag is passed
```
~> packer build --debug /tmp/source.pkr.hcl
Debug mode enabled. Builds will not be parallelized.
null.example: output will be in this color.
==> null.example: Pausing after run of step 'StepConnect'. Press enter to continue.
==> null.example: Pausing before the next provisioner . Press enter to continue.
==> null.example: Running local shell script: /var/folders/vz/rv7bk6v15211jxg8q801f7kw0000gq/T/packer-shell4149319610
null.example: hi
==> null.example: Pausing before the next provisioner . Press enter to continue.
==> null.example: Running local shell script: /var/folders/vz/rv7bk6v15211jxg8q801f7kw0000gq/T/packer-shell3210691290
null.example: hi 2
==> null.example: Pausing after run of step 'StepProvision'. Press enter to continue.
==> null.example: Pausing before cleanup of step 'StepProvision'. Press enter to continue.
==> null.example: Pausing before cleanup of step 'StepConnect'. Press enter to continue.
Build 'null.example' finished after 8 seconds 284 milliseconds.
==> Wait completed after 8 seconds 284 milliseconds
==> Builds finished. The artifacts of successful builds are:
--> null.example: Did not export anything. This is the null builder
```
* Add call to SetOnError, and SetForce
* prevent duplicate local block creation
* remove duplicate locals block bug
* local variables: first get block, then decode it + simplify retry loop
* Update types.packer_config.go
* revert go get of hcl lib
* Update handling of registry artifacts
This change uses the github.com/hashicorp/packer-plgin-sdk/packer/registryimage for querying
Artifact State for HCP Registry Image metadata. To handle the conversion
of the RPC response, mapstructure was introduced to conversion state
data into an registryimage.Image before publishing to a image bucket.
* Update to use registry image from packersdk
* Rename internal registry service pkg
* Update vendored plugins to latest version
* The latest release of Amazon, GoogleCompute, and Azure have support
for publishing images to the HCP Packer registry.
create a null data source for testing. We can choose to document if we want to, but it's a convenience for us
add a test to catch cyclic datasource dependency, update tests to include out of order data sources, and update the code to clean up the returned diagnostics generated from the recursive evaluation
PR review comments
* Add working registry pkg
* Add custom error for handling the loading of PAR environment variables
* Working Publish to Build, with proper error handling for bucket names
* Update hcp-sdk-go to use branch instead of mod replace directive
* Update Packer build status configuration
* Add support for HCP_PACKER_BUILD_FINGERPRINT env
* Add support for publishing one or more PARtifacts from a single build
* add git shas to this branch
* Add ability to set provider name if available
* Add working RegistryBuilder type
* Add RegistryPostProcessor as wrapper post-processor
* When in PAR mode a empty RegistryPostProcessor is added to the end of
the post-processor list to publish all final image data.
* Add support for updating a build from PAR that is not in a DONE state
* Fix a small issue with creation the initial builds for an empty
iteration.
* Add PAR URL to post-processor display
* Implement hcp_packer_registry block (#11168)
* Update vendored Amazon plugin to v1.0.1-dev
* Fix panic when running a Packer registry build in a clean directory
* Remove the publishing of post-processor metadata from the registry
post-processor.
* Remove metadata add from registry_builder
* Update registry builder to skip a build that was found to be DONE
Co-authored-by: Megan Marsh <megan@hashicorp.com>
Co-authored-by: Sylvia Moss <moss@hashicorp.com>
* HCL2 variables: split validation from getting value, to only
This way we do this only once and log this only once. The errors were being ignored anyways.
* Update types.variables_test.go
This adds the new `required_plugins` block to be nested under the packer block.
Example:
```hcl
packer {
required_plugins {
aws = {
version = ">= 2.7.0"
source = "azr/aws"
}
azure = ">= 2.7.0"
}
}
```
For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"
+ docs
+ tests
* Allow locals to be delcared as individual blocks, and give them the Sensitive flag
* add docs for new local block
* linting
* add tests
* modified parsing to use schema, check for dupes properly
* update comment
fix wording a liiitle
* add tests for duplicate variables definition in two different files
* remove unnecessary slice initialisation
* fix crash by returning when decode error is hit
* parseLocalVariables: only treat a local vars if its not nil
also return in case of error
return locals in case of error too
* fix duplicate_locals test for windows
Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>
* add the possibility to set the packer.required_version field; to make sure the template file works with that version of Packer
* add tests
* add documentation on packer.required_version
Example:
packer {
required_version = ">= 1.2.0, < 2.0.0"
}
added `post-processors` block to run chained post-processors after a build.
Before this, defining multiple `post-processor` blocks after
provisioning steps would run them sequentially, now doing this makes them start
from the build's artifact. To queue post-processors you now have to define them
in a `post-processors` block.
This is a breaking change.
The initialization of packer core in JSON also validates that `null` variables were set, except in the case of `packer validate --syntax-only` , but after the refactor to allow to have all commands work with HCL2 and JSON this subtlety was lost.
This refactors the initialisation of the core in order to allow to have `packer validate --syntax-only` not error in case a variable is not set. Since these calls are refactored this works for HCL2 too.
fix#9478
