When running a build with HCP Packer enabled, Packer attempts to push
the build status to HCP.
If the build fails, we update the status to BUILD_FAILED, and that's the
end of it.
If however the build succeeds, Packer attempts to get the HCP artifact
from the builder, which will only succeed if the builder supports it.
Otherwise, we'll get either nil, or an artifact type that is not
compatible with what is expected for HCP support.
When either of those happens, we warn that the builder may not support
HCP Packer at all, so users are aware of the problem.
However, when the error was introduced, it only looked at the fact that
an error was produced, independently of the type of error. This caused
legitimate errors while building to be reported as potential
incompatibility between the builder and HCP, which was confusing to
users.
This commit changes this by introducing a new error type, only produced
when the artifact either is nil, or failed to be deserialised into a HCP
artifact, which lets us produce the incompatibility warning with more
accuracy.
Single-component plugins are a relic from the past that has been
deprecated from version 1.7.0 and onwards.
Since we're revisiting how plugins are installed/loaded, and the changes
will be incompatible with those, we remove them in preparation of this
work.
Before Change
```
Options:
- path <path>: install the plugin from a locally-sourced plugin binary. This
installs the plugin where a normal invocation would, but will
not try to download it from a remote location, and instead
install the binary in the Packer plugins path.
This option cannot be specified with a version constraint.
- force: forces reinstallation of plugins, even if already installed.
```
After Change
```
Options:
-path <path> Install the plugin from a locally-sourced plugin binary.
This installs the plugin where a normal invocation would, but will
not try to download it from a remote location, and instead
install the binary in the Packer plugins path. This option cannot
be specified with a version constraint.
-force Forces reinstallation of plugins, even if already installed.
```
When installing a plugin with packer plugins install --path, we only
accept release versions of a plugin, as otherwise the loading can be
inconsistent if for example a user specifies a required_plugins block in
their template, in which case the plugins will be ignored.
Until we have a simpler loading scheme then, we will reject non-release
versions of plugins to avoid confusion.
To avoid plugins being installed with a specific version when a path is
used for installing a plugin from a locally sourced plugin binary, we
explicitly reject the combination of both a path and a version for
plugins install.
The --force option for packer init and packer plugins install enforces
installation of a plugin, even if it is already locally installed.
This will become useful if for some reason a pre-existing plugin
binary/version is already installed, and we want to overwrite it.
This new flag allows the `packer plugins install' command to install a
plugin from a local binary rather than from Github.
This command will only call `describe' on the plugin, and won't do any
further checks for functionality. The SHA256SUM will be directly
computed from the binary, so as with anything manual and potentially
sourced by the community, extra care should be applied when invoking
this.
If a user attempts to remove a plugin through the `packer plugins
remove' subcommand, and the specified plugin is not installed, the
command succeeds, but does nothing, and exits silently.
This is not clear what is happening, and arguably, calling a command
that does nothing, not even explain what went wrong, is not intuitive.
Because of that, this commit changes how the command behaves in this
case, stating what went wrong, and exiting with a non-zero status.
When a user invokes `packer plugins remove', the plugin binary gets
removed, but not the corresponding SHA256SUM file.
This patch changes this so that when a binary is removed, so is its
SHA256SUM file.
This change updates hcl2_upgrade to not flag known plugin components, those used for generating the required plugins
block, when upgrading a legacy JSON template to HCL2. Any unknown plugins will be installed after running packer init
on the generated template so we don't error. We may want to suggest running packer init to install any missing plugins.
* Move knownPluginPrefixes into the hcl2_upgrade command
With the removal of vendored_plugins we are no longer interested in tacking bundled plugins usage.
For plugins such as file, null, or anything bultin into Packer we don't track because there is no
way to install them outside of Packer, for now.
The enumeration for FlagSetFlags, which presumably was added when the
Meta structure was introduced, aims to pre-populate the flagset for a
subcommand with a series of arguments.
However, despite it being documented, it is actually not used, and
therefore can safely be removed from the codebase.
When users call `packer init' on a template that does not specify a
`required_plugin' block, the command succeeds but does nothing, which is
not helpful for users that may expect their plugins to install.
To remedy that problem, we now output a message like what `packer
plugins required' does on templates without such a block, so that users
have an idea of what to change in order to get the command to work.
Since this feature is no longer something we plan to activate later, as
it contradicts with our efforts to remove bundled plugins, and
encouraging users to move to either manually installing plugins, or
managing them through `packer init', we clean-up the code for this
feature.
* Updating the license from MPL to Business Source License
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl.
* Update copyright file headers to BUSL-1.1
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
`packer init' is meant to be invoked on a template file or a directory.
This was described in the command-line help message as
`[template.pkr.hcl|folder]'.
Because this is encapsulated between square brackets, this could be
taken as an option instead of a positional, mandatory argument.
This commit changes the wording to adopt a similar convention as `packer
build', so the two are consistent in how they are invoked, and to avoid
mistaking this argument as optional.
Since bundled plugins will be removed in an upcoming version of Packer,
this commit adds a new warning message whenever a template uses one such
plugin.
This warning has been implemented on build, validate, console and the
inspect subcommands.
In addition to warning about the upcoming change and potential issue
this will cause, this warning message proposes solutions to the user so
they know what they'll have to do in order not to rely on those bundled
plugins later.
* Update external plugin documentation source
The Parallels plugin for Packer is now maintained by the Parallels team, under
their respective GitHub org. This changes updates the source address for
the external plugin that should be used for pulling new plugin
documentation.
* Removed packer-plugin-parallels for list of vendored plugins
This step removes community plugins from the Packer binary release. These plugins are being maintained and released independently of Packer and have become out of date. To ensure users are using the latest version of the plugins they are being removed in favor of the external plugin installation processes.
This change removes a set of plugins that have not been updated in a while. These
plugins have been archived for some time now. Users wishing to continue using these plugins
should use the `packer plugins install` or `packer init` commands to install the external plugin.
* Remove profitbricks plugin
* Remove oneandone plugin
The following plugins have been unmaintained for some time now, and their
upstream cloud provider has consolidate the services. These plugins
will continue to be available to Packer via direct installation using
packer init or the packer plugins install command. But they will no longer
be bundled with Packer.
The Digital Ocean plugin is now maintained and released by the Digital Ocean team.
To ensure users are downloading the latest available version the plugin will nolonger
be bundled with Packer.
The template type for a builder, provisioner, post-processor or
datasource was not tracked with telemetry data.
This commit adds the infrastructure to track this, bumping the schema
for non-crashes to `beta/packer/6' at the same time.
It is not clear that the console command runs in legacy JSON mode by
default and that users can specify the -config-type flag to toggle HCL2
mode. This change adds the config-type flag to the command's help text
to alert users of the gotcha.
In the next minor Packer release the console command will be updated to
default to HCL2 mode.
Relates to: #12359
Relates to: #10603
* Update external plugin documentation source
The UCloud plugin for Packer is now maintained by the UCloud team, under
their respective GitHub org. This changes updates the source address for
the external plugin that should be used for pulling new plugin
documentation.
* Remove packer-plugin-ucloud from vendored plugins.
* Update external plugin documentation source
The Linode plugin for Packer is now maintained by the Linode team, under
their respective GitHub org. This changes updates the source address for
the external plugin that should be used for pulling new plugin
documentation.
* Removed packer-plugin-linode for list of vendored plugins
The Linode plugin for Packer is now maintained by the Linode team, under
their respective GitHub org. This changes removes Linode from the vendored plugins
lists.
This change fixes a crash that occurs when trying to add undeclared variables into an empty
variables map. The variables block when defined will always be non-nil but in the case
where variables are declared and initialized from a legacy JSON variables file the map
must be initialized first.
Fingerprints are how we link a packer build to an iteration on HCP.
These are computed automatically from the Git SHA in the current state,
and are unique to the bucket/iteration.
The main problem with this approach is that while sound in theory, it
quickly falls apart when users want to run the same build configuration
twice, but expect a new image to be created.
With the current model, this fails, as the iteration with the current
SHA already exists.
While this is solvable through environment variables, or by committing a
change to the repository, we think this is not clear enough, and causes
an extra step to what should otherwise be a simple process.
Therefore, to lower the barrier of entry into HCP, we change this
behaviour with this commit.
Now, fingerprints are randomly generated ULIDs instead of a git SHA, and
a new one is always generated, unless one is already specified in the
environment.
This makes continuation of an existing iteration a conscious choice
rather than something automatic, and virtually eliminates conflicts such
as the ones described above.
* hcl2template: use && for chaining conditions
For deciding if we need to run an error provisioner block, we checked
using a nested if that we didn't need to avoid running it.
This is unnecessary, and adds a superfluous level of inbrication to the
code, so we chain both conditions with &&.
* hcl2: don't warn on excluded post-processor
When a post-processor is excluded through the -except command-line
argument for packer build, we used to print a warning on HCL2 templates
if this except statement did not match a build.
However, since we can use the except option to avoid running
post-processors, we should account for this as well when printing out a
warning that there were no matches.
This commit adds this counter to the except checks for the
post-processors as well.
This commit irons out one of the pain points of the HCP rework by
introducing a HCPPublisher interface, implemented both by the JSON Core,
and the HCL2 PackerConfig, which keeps a map of the build names used by
Packer to the build names pushed on HCP.
This in turn lets us go back to the old implementation of the GetBuilds
function, which returns a list of (filtered) builds, and eventually an
error if something went wrong while processing.
While looking into #11932 I found that Packer will throw an error when
an unknown source is referenced from within the sources attribute for
a build block. The hcl.Diagnostics error includes a Subject, which
should highlight where in the HCL2 configuration file the unknown source resides.
But when creating the BuildBlock no HCL2Ref data is copied over, thus the
Subject is displayed with with zero value for an hcl.Range type.
This change updates the build creation logic to copy the HCL2Ref data
from the build block at decode time so that it can be properly
referenced downstream.
Closes#11932
Failure due to change in output
```
--- FAIL: TestValidateCommand_ShowLineNumForMissing (0.00s)
--- FAIL: TestValidateCommand_ShowLineNumForMissing/test-fixtures/validate-invalid/missing_build_block.pkr.hcl (0.00s)
validate_test.go:377: Unexpected output: (
"""
Error: Unknown source file.cho
- on line 0:
+ on test-fixtures/validate-invalid/missing_build_block.pkr.hcl line 6:
(source code not available)
... // 4 identical lines
"""
)
validate_test.go:379:
FAIL
FAIL github.com/hashicorp/packer/command 1.002s
```
```
Usage: packer validate [options] TEMPLATE
Checks the template is valid by parsing the template and also
checking the configuration with the various builders, provisioners, etc.
If it is not valid, the errors will be shown and the command will exit
with a non-zero exit status. If it is valid, it will exit with a zero
exit status.
Options:
-syntax-only Only check syntax. Do not verify config of the template.
-except=foo,bar,baz Validate all builds other than these.
-only=foo,bar,baz Validate only these builds.
-machine-readable Produce machine-readable output.
-var 'key=value' Variable for templates, can be used multiple times.
-var-file=path JSON or HCL2 file containing user variables, can be used multiple times.
-no-warn-undeclared-var Disable warnings for user variable files containing undeclared variables.
-evaluate-datasources Evaluate data sources during validation (HCL2 only, may incur costs); Defaults to false.
```
Currently the HCL2 upgrade command generates a valid variable block for all variables
within the JSON variables property. However JSON templates in Packer support variable
interpolation for input variables defined within a variable definition file without
it being declared within the variables property. When upgrading a JSON template to HCL2
the user variable reference gets converted to var.<var_name>. The upgraded template ultimately
fails on validation or build execution with undefined variable errors, with this change the upgrade
command will now create a variable block for all input variables referenced within the build template
to ensure all required variables been defined.
When packer validate is invoked, it does not try to evaluate the
datasources before attempting to decide if the template is valid.
In many cases, this works, but sometimes it will fail as the value is
unknown by the validation code.
Since the validation code for all the elements of a Packer template is
left to be implemented by plugins, we cannot rely on checking for
unknown values everywhere, especially since the unknown references are
replaced automatically by a value of the right type for the
configuration expected.
So, in order for such configurations to be validable, we add an extra
option to packer validate, that will let users evaluate the datasources
from a template.
