Commit graph

620 commits

Author SHA1 Message Date
Sylvia Moss
37d05c85ff
upgrade pause_before (#10654) 2021-02-18 15:05:12 +01:00
Megan Marsh
00e503388e
change template parsing error to include warning about file extensions (#10652) 2021-02-18 09:59:30 +01:00
Megan Marsh
1710590418
Merge pull request #10633 from hashicorp/azr_init_no_magic_host
Packer init: remove host and namespace guessing
2021-02-16 14:32:20 -08:00
Wilken Rivera
728c5a217d Add test case for non-github hostname
Tests results on current branch; install succeeded which was not expected
```
2021/02/16 14:02:24 ui: Installed plugin example.com/sylviamoss/comment v0.2.19 in "/tmp/pkr-test-cfg-dir-6_pkr_config458005728/example.com/sylviamoss/comment/packer-plugin-comment_v0.2.19_x5.0_linux_amd64"
    init_test.go:361: InitCommand.Run() = 0, want 1
    init_test.go:381: unexpected dir hash after init:   string(
        -       "h1:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
        +       "h1:iVtzkl/nVm2KiLvlz8rH56ME8QEqRxq8+XT2Lo6bzGU=",
          )
--- FAIL: TestInitCommand_Run (6.39s)
    --- PASS: TestInitCommand_Run/already-installed-no-op (0.01s)
        --- PASS: TestInitCommand_Run/already-installed-no-op/-subtest-0 (0.00s)
    --- PASS: TestInitCommand_Run/already-installed-upgrade (2.30s)
        --- PASS: TestInitCommand_Run/already-installed-upgrade/-subtest-0 (0.06s)
    --- PASS: TestInitCommand_Run/release-with-no-binary (0.17s)
    --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-works (1.32s)
        --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-works/-subtest-0 (0.01s)
    --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-old-api-fails (1.42s)
        --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-old-api-fails/-subtest-0 (0.01s)
    --- FAIL: TestInitCommand_Run/unsupported-non-github-source-address (1.18s)
```

Tests results after change with change in this branch
```
2021/02/16 14:03:14 [TRACE] getting available versions for the example.com/sylviamoss/comment plugin
2021/02/16 14:03:14 [TRACE] &{%!q(*github.Client=<nil>) "packer-getter-github-1.7.0-dev"} getter could not get release: example.com/sylviamoss/comment doesn't appear to be a valid github.com source address; check source and try again.
2021/02/16 14:03:14 [DEBUG] will try to install: []
2021/02/16 14:03:14 ui error: no release version found for the example.com/sylviamoss/comment plugin matching the constraint(s): "v0.2.19"
--- PASS: TestInitCommand_Run (5.38s)
    --- PASS: TestInitCommand_Run/already-installed-no-op (0.01s)
        --- PASS: TestInitCommand_Run/already-installed-no-op/-subtest-0 (0.00s)
    --- PASS: TestInitCommand_Run/already-installed-upgrade (2.08s)
        --- PASS: TestInitCommand_Run/already-installed-upgrade/-subtest-0 (0.07s)
    --- PASS: TestInitCommand_Run/release-with-no-binary (0.21s)
    --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-works (1.20s)
        --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-works/-subtest-0 (0.01s)
    --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-old-api-fails (1.88s)
        --- PASS: TestInitCommand_Run/manually-installed-single-component-plugin-old-api-fails/-subtest-0 (0.01s)
    --- PASS: TestInitCommand_Run/unsupported-non-github-source-address (0.00s)
```
2021-02-16 14:03:29 -05:00
Sylvia Moss
e0557f84e9
Add access config to generated amazon ami data source (#10634) 2021-02-15 17:35:03 +01:00
Adrien Delorme
8208f425c8 addrs: remove Plugin.ForDisplay func, the String one does the job
to make things less confusing
2021-02-15 15:32:42 +01:00
Megan Marsh
cac6b04546
update help text for hcl2_upgrade (#10624) 2021-02-15 11:03:44 +01:00
Megan Marsh
80ed988ffe
Hcl2 upgrade fix (#10625)
* fix random nil pointer dereference I found while debugging hcl2_upgrade issues

* fix hcl2_upgrade command by creating passthroughs for all text template fields
2021-02-15 10:40:21 +01:00
Megan Marsh
b20dea6aec
Merge pull request #10614 from hashicorp/azr_acc_test_old_single_component_plugins
Acc test "old style" single component plugins
2021-02-12 14:22:29 -08:00
sylviamoss
e69410b633 fix lint 2021-02-12 15:38:44 +01:00
sylviamoss
855ba9775a add docs 2021-02-12 15:31:13 +01:00
sylviamoss
76e3d57cae add with-annotation flag to hcl2_upgrade 2021-02-12 15:18:53 +01:00
teddylear
0637601eda Fixing recursive formatting tests to work on all platforms 2021-02-11 22:08:36 -05:00
Adrien Delorme
7089e0854a rename setup func to avoid name conflict 2021-02-11 18:00:18 +01:00
Adrien Delorme
40c2b2a153 add manually-installed-single-component-plugin-old-api-fails 2021-02-11 16:55:30 +01:00
Adrien Delorme
3677069010 test: manually-installed-single-component-plugin-works 2021-02-11 16:35:08 +01:00
Adrien Delorme
b321c64c1e Update init_test.go 2021-02-11 15:26:33 +01:00
Adrien Delorme
ad0ce49cec fix tmpdir creation 2021-02-11 15:17:05 +01:00
Adrien Delorme
021044e963 put name first 2021-02-11 14:56:42 +01:00
Adrien Delorme
ba8484ee6b tests: pre make dir in case nothing happens 2021-02-11 14:49:52 +01:00
Adrien Delorme
3052e3c5d5 tests 2021-02-11 14:46:53 +01:00
Sylvia Moss
774c5903f6
Add error-cleanup-provisioner to HCL2 (#10604) 2021-02-11 10:23:15 +01:00
teddylear
d85286e228 Merging in master 2021-02-10 17:44:11 -05:00
Sylvia Moss
d28e6fe009
write timestamp local only when necessary (#10602) 2021-02-10 14:54:19 +01:00
Adrien Delorme
ffbb110167
Test running plugins with fake and fresh plugins (#10595)
* show error as to why plugin discovery failed
* allow to run more manual tests after a plugin installation
* test that a freshly installed external plugin can run
2021-02-09 17:07:59 +01:00
Sylvia Moss
f5006d0842
fix empty locals and source name with blank space (#10593) 2021-02-09 14:57:42 +01:00
Sylvia Moss
d53488db68
Add aws_secretsmanager transformation to hcl2_upgrade (#10553) 2021-02-08 11:28:26 +01:00
teddylear
d3754e3021 Updating recursive formatter tests to be cleaner and table driven 2021-02-06 18:02:26 -05:00
teddylear
40a97e29db Clean up recursive format tests to be more accurate 2021-02-06 18:02:26 -05:00
teddylear
93df53a275 Refactor recursive formatting test cases to be table driven 2021-02-06 18:02:26 -05:00
teddylear
6adf1f6659 Fixing recursive fmt tests syntax and adding test case when recursive
option is off
2021-02-06 18:02:26 -05:00
teddylear
261abe0cae Setting recursive fmt to false, updatting recursive fmt test to validate
formatted files
2021-02-06 18:02:26 -05:00
teddylear
ab4b3a8465 Adding recursive flag to formatter to format subdirectories 2021-02-06 18:02:23 -05:00
Adrien Delorme
692433721d
Add some acceptance testing for Packer init commands (#10566) 2021-02-05 14:06:28 +01:00
Sylvia Moss
ef4afafde9
Fix force flag for hcl2 provisioners and post-processors (#10571) 2021-02-05 10:57:14 +01:00
Adrien Delorme
92ccd5fa1d
init: show successful installs in cyan (#10557) 2021-02-04 11:11:42 +01:00
Adrien Delorme
ed091163be
HCL2 Parse packer.required_plugins block + packer init (#10304)
This adds the new `required_plugins` block to be nested under the packer block.

Example:
```hcl
packer {
  required_plugins {
    aws = {
      version = ">= 2.7.0"
      source = "azr/aws"
    }
    azure = ">= 2.7.0"
  }
}
```

For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"

+ docs
+ tests
2021-02-02 18:05:04 +01:00
Sylvia Moss
d1ada744e1
Aws Secrets Manager data sources (#10505) 2021-01-22 14:49:45 +01:00
Megan Marsh
93008045cd
Merge pull request #10500 from hashicorp/format_stdin
enable Packer fmt to read from stdin
2021-01-20 11:47:55 -08:00
Sylvia Moss
3c7944624a
(3) Add amazon-ami data source (#10467) 2021-01-20 11:05:03 +01:00
Sylvia Moss
291121dd55
(2) Implement datasources (#10440) 2021-01-20 10:37:16 +01:00
Megan Marsh
4323b49130 enable Packer fmt to read from stdin 2021-01-19 15:30:34 -08:00
Sylvia Moss
e44cb9d7e5
Add Amazon AMI data source to hcl_upgrade command (#10491) 2021-01-19 10:21:39 +01:00
Megan Marsh
4cb6e07900
Hcl beta tag (#10493)
* change beta tag

* remove beta tag from hcl2_upgrade comment

* add a partial to describe the legacy json limitations

* exclude legacy category from templates until it is deployed to the website
2021-01-18 15:08:04 +01:00
Megan Marsh
54a75421ea regenerate 2021-01-15 11:59:36 -08:00
Megan Marsh
1cb9aea752 change markdown references to new pathing 2021-01-15 11:59:36 -08:00
Adrien Delorme
52d2d7fe5f
Copy MapOf... plugin types back into Packer core (#10466)
* move maps of plugins back in core

* go mod vendor

* more fixes

* fix imports

* Update core_test.go

* fix build

* more fixes

* more fixes

* up vendors after fixing sdk

* Update post_processor_mock.hcl2spec.go

* Leave implementatino of MapOf in the sdk for plugi tests

Other wise use the interface

* go mod tidy

* add MapOfDatasource type too
2021-01-13 12:14:06 +01:00
Tristan Watson
17c806fbf5
Removing obsolete packer HCL warnings from CLI for validate and command (#10461) 2021-01-11 11:50:23 +01:00
Megan Marsh
88b7b0e14a remove sdk 2020-12-17 13:29:25 -08:00
Wilken Rivera
4e58987026
command/fmt: Ensure all variable files ending in .pkrvars.hcl get formatted (#10377)
Before change
```
⇶  packer fmt -check /tmp/unformatted.pkrvars.hcl
Error: Cannot tell whether /tmp/unformatted.pkrvars.hcl contains HCL2 configuration data

⇶  echo $?
1
```

After fix
```
⇶  packer fmt -check /tmp/unformatted.pkrvars.hcl
/tmp/unformatted.pkrvars.hcl

⇶  echo $?
3

⇶  packer fmt -check command/test-fixtures/fmt
command/test-fixtures/fmt/unformatted.pkr.hcl
command/test-fixtures/fmt/unformatted.auto.pkrvars.hcl
command/test-fixtures/fmt/unformatted.pkrvars.hcl

```
2020-12-14 10:29:58 -05:00
Megan Marsh
39ab646236
move plugin and rpc code into sdk; other minor tweaks (#10359) 2020-12-09 12:39:54 +01:00
Megan Marsh
5576bb5caa move packer config constants next to the packer config 2020-12-03 10:17:35 -08:00
Megan Marsh
b69d69095e move postprocessor to sdk, fix generation code 2020-12-01 14:48:55 -08:00
Megan Marsh
be5763ec41 move provisioner interface into sdk 2020-12-01 14:25:14 -08:00
Megan Marsh
ada91b24e9 extract builder interface to sdk 2020-12-01 13:42:42 -08:00
Megan Marsh
d5971ca25e fix tests by moving mock hooks and communicators, and BasicUi definition, into SDK 2020-11-30 14:30:30 -08:00
Megan Marsh
56a45b04ab Move hook and communicator definitions to packer-plugin-sdk 2020-11-30 14:30:30 -08:00
Megan Marsh
14bdb9516c move Artifact and artifact mock to the sdk 2020-11-30 14:29:52 -08:00
Megan Marsh
001886670d move Ui definition into the packer plugin sdk. 2020-11-30 14:26:54 -08:00
Adrien Delorme
73b7499811
HCL2: version block: test validation & document version/availability (#10298)
* HCL2: Test that the packer block passes in packer validate

* HCL2: Test invalid packer blocks are invalid

* docs: state from which version the packer block is available
2020-11-23 10:27:26 -05:00
Megan Marsh
6a1a22151c move the various flag packages from helper to command, since command is the only package that uses them. 2020-11-16 14:23:35 -08:00
Megan Marsh
efe11da4a3
Merge pull request #10254 from hashicorp/plugin-sdk-2
Plugin sdk 2
2020-11-16 11:29:20 -08:00
Megan Marsh
3e54e9ea80
Merge pull request #10235 from hashicorp/refactor_version_code
Refactor version code
2020-11-16 11:12:59 -08:00
Megan Marsh
6402362018 create packer-plugin-sdk directory and begin moving the relevant folders into it. 2020-11-13 10:47:36 -08:00
Adrien Delorme
f44e912072
Update hcl2upgrade command to update env calls + tests (#10244) 2020-11-11 14:54:22 -05:00
Wilken Rivera
acabc1c1aa
Add packer fmt command (#10225)
* Add packer fmt command

This change adds a new command that allows users to format one or more
HCL2 Packer configuration template files.

Related to: #9174

* command/fmt: Add check flag

Packer's fmt command now supports a check flag that will output the name
of any file that would be changed by the HCL2 formatting engine. The
check flag is mutually exclusive with the write flag and will only check
if formatting is needed.

The update write flag will now overwrite the source files with the newly
formatted HCL2 source unless the `-write=false` or `-check` is passed at
the command line.

* Returns a diagnostic error if Format is unable to show a diff - equivalent to `terraform fmt`
* Updates testing to run against #Format and not the private methods of the HCL2Formatter; fixes ShowDiff test failure on Windows
* Updates comments for exported functions

* Add docs for fmt command
2020-11-11 11:49:39 -05:00
Adrien Delorme
deba1484ff
HCL2: allow calling env as input var default value (#10240)
* HCL2: allow to use env in default value of input variables
2020-11-11 11:27:32 +01:00
Adrien Delorme
17ec88246f
hcl2_upgrade update (#10221)
* handle template_dir, pwd funcs, handle min_packer_version
* gotpl packer_version becomes hcl2 packer.version
* uuid becomes hcl2 uuidv4
* test code with emojis
* handle all cases were we cannot guess what the argument was
* handle clean_resource_name cases
* up docs
2020-11-10 10:46:20 +01:00
Megan Marsh
bc85854a53 refactor packer version out of hcltemplate code. 2020-11-09 12:29:53 -08:00
Adrien Delorme
6dd06fad14 add command/ tests 2020-11-02 15:50:38 +01:00
Adrien Delorme
4bc16455b4
HCL2: add a packer block with a required_version input setting (#10149)
* add the possibility to set the packer.required_version field; to make sure the template file works with that version of Packer
* add tests
* add documentation on packer.required_version

Example:

packer {
  required_version = ">= 1.2.0, < 2.0.0"
}
2020-10-27 10:03:36 +01:00
Megan Marsh
fc619dc977
Merge pull request #9626 from featheredtoast/add-proxmox-vm-clone
builder/proxmox FEATURE: split Proxmox into proxmox-iso and proxmox-clone
2020-10-23 13:36:45 -07:00
Megan Marsh
66b81d9bee
add build.name to hcl code (#10114) 2020-10-16 10:57:42 +02:00
Megan Marsh
2cdc052d7c
Merge pull request #10097 from hashicorp/more_hcl_docs
Update HCL docs with a local resource-free example to empower users t…
2020-10-14 13:34:54 -07:00
Megan Marsh
00cc425b84 docs tweaks 2020-10-14 12:58:04 -07:00
Peter Willis
f1b57b1e66
Fix typos 2020-10-14 15:04:18 -04:00
Peter Willis
a81811663a
Fix typos 2020-10-14 14:52:17 -04:00
Jeff Wong
0f12414126
Merge branch 'master' into add-proxmox-vm-clone 2020-10-12 10:48:15 -07:00
James Cradock
153d4111fa
Fix typo in build argument description (#10066) 2020-10-08 14:58:22 +02:00
Jeff Wong
a140c13943
[proxmox] add proxmox builder alias
Adds an alias for `proxmox` that points to proxmox-iso builder for
backwards compatibility
2020-10-07 23:43:12 -07:00
Adrien Delorme
3e0633fc20 scripts/generate-plugins.go: ignore "common" packages 2020-10-07 11:43:15 +02:00
Adrien Delorme
b2be255057 Merge remote-tracking branch 'origin/master' into pr/featheredtoast/9626-1 2020-10-06 11:11:04 +02:00
Adrien Delorme
fcf16315a3 add tests 2020-10-02 10:49:21 +02:00
Adrien Delorme
3d7ab9ea43 add an hcl2-json test 2020-09-18 16:38:04 +02:00
Calle Pettersson
905869308d Split proxmox builder into a common part and iso/clone builders
Clone builder is still just a stub. Proof-of-concept for #9626

Signed-off-by: Calle Pettersson <calle@cape.nu>
2020-09-04 23:53:09 +02:00
Adrien Delorme
efd69aea2a
console: allow to use console when there are untyped vars (#9864)
* the var will simply be "unknown"
* add a test
2020-09-02 17:19:56 +02:00
Adrien Delorme
a70164f489
HCL2: when the type of a variable is not known evaluate setting as a litteral. (#9863)
* tests
* docs
2020-09-02 16:26:50 +02:00
Adrien Delorme
a25f057984 add tests for unknown values 2020-09-01 15:28:16 +02:00
Adrien Delorme
161879b98a test unknown variables and locals 2020-09-01 11:59:07 +02:00
Adrien Delorme
652878059c inspect command: ignore init errors as some value can be unset 2020-09-01 11:59:07 +02:00
Adrien Delorme
91c5a4613c HCL2 inspect remove debug values 2020-09-01 11:59:07 +02:00
Wilken Rivera
67cd123d1c
Merge pull request #9834 from hashicorp/azr-fix-hcl2_upgrade_random_generate
hcl2_upgrade: fix a case where the generated type is wrong
2020-08-27 13:03:34 -04:00
Wilken Rivera
0e2a3e1058 command/hcl2_upgrade: Update description text for command
Before change
```
Usage: packer [--version] [--help] <command> [<args>]

Available commands are:
    build           build image(s) from template
    console         creates a console for testing variable interpolation
    fix             fixes templates from old versions of packer
    hcl2_upgrade    build image(s) from template
    inspect         see components of a template
    validate        check that a template is valid
    version         Prints the Packer version
```

After change
```
Usage: packer [--version] [--help] <command> [<args>]

Available commands are:
    build           build image(s) from template
    console         creates a console for testing variable interpolation
    fix             fixes templates from old versions of packer
    hcl2_upgrade    transform a JSON template into a HCL2 configuration
    inspect         see components of a template
    validate        check that a template is valid
    version         Prints the Packer version

```
2020-08-27 11:52:09 -04:00
Adrien Delorme
0f00709fb6 hcl2_upgrade: fix a case where the generated type is wrong
when it encounters map[string]interface{} or []interface{} types,  hcl2_upgrade now takes the 'most complex' entry from those in order to tell wether this is going to be a body `body {}` or an attribute `attribute = {}`. Before that the hcl2_upgrade command could be a bit random there.

A way better ( but may be somewhat hard ) way to do this would be to use the actual plugins structs in order to generate the HCL2.
2020-08-27 16:47:14 +02:00
Adrien Delorme
5ba134ac5b
JSON to HCL2 (minimal best-effort) transpiler (#9659)
hcl2_upgrade transforms a JSON build-file in a HCL2 build-file.
This starts a validated Packer core and from that core we generate an HCL 'block' per plugin/configuration. So for a builder, a provisioner, a post-processor or a variable. The contents of each block is just transformed as is and basically all fields are HCL2-ified.
A generated field can be valid in JSON but invalid on HCL2; for example JSON templating (in mapstructure) allows to set arrays of strings - like `x = ["a", "b"]` - with single strings - like `x="a"` -, HCL does not allow this.
Since JSON does not make the distinction between variables and locals, everything will be a variable. So variables that use other variables will not work.
hcl2_upgrade tries to transform go templating interpolation calls to HCL2 calls when possible, leaving the go templating calls like they are in case it cannot.

Work:
* transpiler
* tests
* update hcl v2 library so that output looks great.
* update docs
2020-08-25 10:51:43 +02:00
sylviamoss
39a8dee4ea add tests and update override example 2020-08-14 11:22:51 +02:00
Chris Gilmer
6e918c4bd3
Update test for builder output 2020-08-11 15:25:33 -07:00
Chris Gilmer
04f952a2f4
Pretty print build durations 2020-08-11 15:04:42 -07:00
Adrien Delorme
c7b35dd6bc
HCL2: add post-processors block to run multiple post-processor after a build (#9638)
added `post-processors` block to run chained post-processors after a build.
Before this, defining multiple `post-processor` blocks after
provisioning steps would run them sequentially, now doing this makes them start
from the build's artifact. To queue post-processors you now have to define them
in a `post-processors` block.

This is a breaking change.
2020-07-28 10:02:37 +02:00
Adrien Delorme
44616d3bff
refactor initialization out from packer configs + tests (#9627)
The initialization of packer core in JSON also validates that `null` variables were set, except in the case of `packer validate --syntax-only` , but after the refactor to allow to have all commands work with HCL2 and JSON this subtlety was lost.

This refactors the initialisation of the core in order to allow to have `packer validate --syntax-only` not error in case a variable is not set. Since these calls are refactored this works for HCL2 too.

fix #9478
2020-07-24 10:58:03 +02:00