5 commits

Author	SHA1	Message	Date
Mathieu Fenniak	c198082975	fix: empty dynamic matrix can leave action run hanging incomplete (#11063) ... Fixes #11030. When a `strategy.matrix` needs to be evaluated on the output of another job, it can become evaluated into an empty set of jobs. In this case, and assuming no other jobs in the run are active, the run should reach a settled state. The logic to check the other jobs in the run and determine if this state has been hit needs to be explicitly added to the job emitter. To accomplish this change, this action run state logic was extracted out of `UpdateRunJobWithoutNotification` where it could be reused. ## Checklist The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org). ### Tests - I added test coverage for Go changes... - [x] in their respective `*_test.go` for unit tests. - [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server. - I added test coverage for JavaScript changes... - [ ] in `web_src/js/*.test.js` if it can be unit tested. - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)). ### Documentation - [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change. - [x] I did not document these changes and I do not expect someone else to do it. ### Release notes - [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change. - [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change. *The decision if the pull request will be shown in the release notes is up to the mergers / release team.* The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead. Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11063 Reviewed-by: Gusted <gusted@noreply.codeberg.org> Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net> Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>	2026-01-27 17:10:59 +01:00
Mathieu Fenniak	75cb38faa6	feat: support reusable workflow expansion when with or strategy.matrix contains ${{ needs... }} (#10647) ... This change allows the `with:` field of a reusable workflow to reference a previous job, such as `with: { some-input: "${{ needs.other-job.outputs.other-output }}" }`. `strategy.matrix` can also reference `${{ needs... }}`. When a job is parsed and encounters this situation, the outer job of the workflow is marked with a field `incomplete_with` (or `incomplete_matrix`), indicating to Forgejo that it can't be executed as-is and the other jobs in its `needs` list need to be completed first. And then in `job_emitter.go` when one job is completed, it checks if other jobs had a `needs` reference to it and unblocks those jobs -- but if they're marked with `incomplete_with` then they can be sent back through the job parser, with the now-available job outputs, to be expanded into the correct definition of the job. The core functionality for this already exists to allow `runs-on` and `strategy.matrix` to reference the outputs of other jobs, but it is expanded upon here to include `with` for reusable workflows. There is one known defect in this implementation, but it has a limited scope -- if this code path is used to expand a nested reusable workflow, then the `${{ input.... }}` context will be incorrect. This will require an update to the jobparser in runner version 12.4.0, and so it is out-of-scope of this PR. ## Checklist The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org). ### Tests - I added test coverage for Go changes... - [x] in their respective `*_test.go` for unit tests. - [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server. - I added test coverage for JavaScript changes... - [ ] in `web_src/js/*.test.js` if it can be unit tested. - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)). - **end-to-end test:** will require the noted "known defect" to be resolved, but tests are authored at https://code.forgejo.org/forgejo/end-to-end/compare/main...mfenniak:expand-reusable-workflows-needs ### Documentation - [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change. - [x] I did not document these changes and I do not expect someone else to do it. ### Release notes - [ ] I do not want this change to show in the release notes. - [x] I want the title to show in the release notes with a link to this pull request. - [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title. Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10647 Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org> Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net> Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>	2025-12-31 19:04:35 +01:00
Mathieu Fenniak	ffbd500600	feat(actions): support referencing ${{ needs... }} variables in runs-on (#10308) ... Allows referencing the outputs of previously executed jobs in the `runs-on` field directly by a `${{ needs.some-job.outputs.some-output }}`, and also *indirectly* through the job's `strategy.matrix`. At its most complicated, supports a workflow with dynamic matrices like this: ```yaml jobs: define-matrix: runs-on: docker outputs: array-value: ${{ steps.define.outputs.array }} steps: - id: define run: | echo 'array=["debian-bookworm", "debian-trixie"]' >> "$FORGEJO_OUTPUT" runs-on-dynamic-matrix: needs: define-matrix strategy: matrix: my-runners: ${{ fromJSON(needs.define-matrix.outputs.array-value) }} runs-on: ${{ matrix.my-runners }} steps: - run: uname -a ``` ## Checklist The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org). ### Tests - I added test coverage for Go changes... - [x] in their respective `*_test.go` for unit tests. - [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server. - I added test coverage for JavaScript changes... - [ ] in `web_src/js/*.test.js` if it can be unit tested. - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)). ### Documentation - [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change. - [x] I did not document these changes and I do not expect someone else to do it. - Documentation already (incorrectly) states that `jobs.<job-id>.runs-on` can access the `needs` context. 😛 https://forgejo.org/docs/latest/user/actions/reference/#availability ### Release notes - [ ] I do not want this change to show in the release notes. - [x] I want the title to show in the release notes with a link to this pull request. - [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title. <!--start release-notes-assistant--> ## Release notes <!--URL:https://codeberg.org/forgejo/forgejo--> - Features - [PR](https://codeberg.org/forgejo/forgejo/pulls/10308): <!--number 10308 --><!--line 0 --><!--description ZmVhdChhY3Rpb25zKTogc3VwcG9ydCByZWZlcmVuY2luZyBgJHt7IG5lZWRzLi4uIH19YCB2YXJpYWJsZXMgaW4gYHJ1bnMtb25g-->feat(actions): support referencing `${{ needs... }}` variables in `runs-on`<!--description--> <!--end release-notes-assistant--> Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10308 Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org> Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net> Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>	2025-12-05 18:14:43 +01:00
Mathieu Fenniak	0ecc6ef632	fix(actions): improve errors when ${{ needs... }} is used in strategy.matrix incorrectly (#10298) ... Three fixes are presented together in this PR: - When a `strategy.matrix` entry in an Action job contains `${{ needs.some-job.outputs.some-output }}`, if that output *never* becomes available, different error messages will be presented if `some-job` isn't found or if `some-output` isn't found. This clarifies an error message that was previously "it could be this, or it could be this". - In the error case described in the previous point, other jobs in the workflow could continue running or could be left "blocked" forever. A centralized `FailRunPreExecutionError` function ensures that all incomplete jobs in the run are failed in this case. - In a rare error case when a job referenced another job in `strategy.matrix` but no other jobs were defined in the workflow, the job would be marked as blocked forever because the `job_emitter` code would never be invoked to detect this case. A new `consistencyCheckRun` function for a newly created `ActionRun` adds a location to perform a pre-execution check for this case so that the run can be failed. These fixes are all interconnected around the refactor for the `FailRunPreExecutionError`, causing them to be bundled rather than individual PRs. ## Checklist The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org). ### Tests - I added test coverage for Go changes... - [x] in their respective `*_test.go` for unit tests. - [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server. - I added test coverage for JavaScript changes... - [ ] in `web_src/js/*.test.js` if it can be unit tested. - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)). ### Documentation - [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change. - [x] I did not document these changes and I do not expect someone else to do it. ### Release notes - [x] I do not want this change to show in the release notes. - These are fixes to an unreleased feature and don't require release notes. - [ ] I want the title to show in the release notes with a link to this pull request. - [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title. Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10298 Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org> Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net> Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>	2025-12-05 17:17:37 +01:00
Mathieu Fenniak	482ba3a4e5	feat(actions): support referencing ${{ needs... }} variables in strategy.matrix (#10244) ... https://code.forgejo.org/forgejo/forgejo-actions-feature-requests/issues/71 requires partial implementation in runner, and partial in Forgejo; this is the Forgejo implementation. Allows for the definition of dynamic job matrixes in Forgejo Actions, where an earlier job provides and output that is used in `strategy.matrix` for a later job that requires it. For example, adapted from the GitHub Actions example for this feature: ```yaml name: shared matrix on: push: workflow_dispatch: jobs: define-matrix: runs-on: docker outputs: colors: ${{ steps.colors.outputs.colors }} steps: - name: Define Colors id: colors run: | echo 'colors=["red", "green", "blue"]' >> "$GITHUB_OUTPUT" produce-artifacts: runs-on: docker needs: define-matrix strategy: matrix: color: ${{ fromJSON(needs.define-matrix.outputs.colors) }} steps: - name: Define Color env: color: ${{ matrix.color }} run: | echo "$color" > color - name: Produce Artifact uses: https://data.forgejo.org/forgejo/upload-artifact@v4 with: name: ${{ matrix.color }} path: color consume-artifacts: runs-on: docker needs: - define-matrix - produce-artifacts strategy: matrix: color: ${{ fromJSON(needs.define-matrix.outputs.colors) }} steps: - name: Retrieve Artifact uses: https://data.forgejo.org/forgejo/download-artifact@v4 with: name: ${{ matrix.color }} - name: Report Color run: | cat color ``` ## Checklist The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org). ### Tests - I added test coverage for Go changes... - [x] in their respective `*_test.go` for unit tests. - [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server. - I added test coverage for JavaScript changes... - [ ] in `web_src/js/*.test.js` if it can be unit tested. - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)). ### Documentation - [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change. - https://codeberg.org/forgejo/docs/pulls/1607 - [ ] I did not document these changes and I do not expect someone else to do it. ### Release notes - [ ] I do not want this change to show in the release notes. - [x] I want the title to show in the release notes with a link to this pull request. - [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title. <!--start release-notes-assistant--> ## Release notes <!--URL:https://codeberg.org/forgejo/forgejo--> - Features - [PR](https://codeberg.org/forgejo/forgejo/pulls/10244): <!--number 10244 --><!--line 0 --><!--description ZmVhdChhY3Rpb25zKTogc3VwcG9ydCByZWZlcmVuY2luZyAke3sgbmVlZHMuLi4gfX0gdmFyaWFibGVzIGluIGBzdHJhdGVneS5tYXRyaXhg-->feat(actions): support referencing ${{ needs... }} variables in `strategy.matrix`<!--description--> <!--end release-notes-assistant--> Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10244 Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org> Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net> Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>	2025-11-29 17:49:04 +01:00