forgejo/services/actions/reusable_workflows_test.go
Mathieu Fenniak 71623b1ab1
Some checks are pending
/ release (push) Waiting to run
testing-integration / test-unit (push) Waiting to run
testing-integration / test-sqlite (push) Waiting to run
testing-integration / test-mariadb (v10.6) (push) Waiting to run
testing-integration / test-mariadb (v11.8) (push) Waiting to run
testing / backend-checks (push) Waiting to run
testing / frontend-checks (push) Waiting to run
testing / test-unit (push) Blocked by required conditions
testing / test-e2e (push) Blocked by required conditions
testing / test-remote-cacher (redis) (push) Blocked by required conditions
testing / test-remote-cacher (valkey) (push) Blocked by required conditions
testing / test-remote-cacher (garnet) (push) Blocked by required conditions
testing / test-remote-cacher (redict) (push) Blocked by required conditions
testing / test-mysql (push) Blocked by required conditions
testing / test-pgsql (push) Blocked by required conditions
testing / test-sqlite (push) Blocked by required conditions
testing / security-check (push) Blocked by required conditions
feat: expand reusable workflow calls into their inner jobs (#10525)
Previously, Forgejo's behaviour for an Actions reusable workflow was to send the entire job to one specific Forgejo Runner based upon its required `runs-on` label, and that single Runner would then read the workflow file and perform all the jobs inside simultaneously, merging their log output into one output (#9768).

This PR begins an implementation of expanding reusable workflows into their internal jobs.

In this PR, the most basic support is implemented for expanding reusable workflows:
- If a `runs-on` field is provided on the workflow, then the legacy behaviour of sending the reusable workflow to a runner is maintained.
- If the `runs-on` field is omitted, then the job may be expanded, if:
    - If the `uses:` is a local path within the repo -- expanded
    - If the `uses:` is a path to another repo that is on the same Forgejo server -- expanded
    - If the `uses:` is a fully-qualified URL -- not expanded

Because this is an "opt-in" implementation by omitting `runs-on`, and all existing capability is retained, I've **omitted some features** from this PR to make the scope small and manageable for review and testing.  These features will be implemented after the initial support is landed:
- Workflow input variables
- Workflow secrets
- Workflow output variables
- "Incomplete" workflows which require multiple passes to evaluate -- any job within a reusable workflow where the `with`, `runs-on`, or `strategy.matrix` fields contain an output from another job with  `${{ needs... }}`

Although this implementation has restrictions with missing features, it is intended to fix #9768.

Replaces PR #10448.

## Checklist

The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).

### Tests

- I added test coverage for Go changes...
  - [x] in their respective `*_test.go` for unit tests.
  - [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
  - [ ] in `web_src/js/*.test.js` if it can be unit tested.
  - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
- end-to-end testing: https://code.forgejo.org/forgejo/end-to-end/pulls/1316

### Documentation

- [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
    - https://codeberg.org/forgejo/docs/pulls/1648
- [ ] I did not document these changes and I do not expect someone else to do it.

### Release notes

- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10525
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
2025-12-24 20:47:21 +01:00

203 lines
5.8 KiB
Go

// Copyright 2025 The Forgejo Authors. All rights reserved.
// SPDX-License-Identifier: GPL-3.0-or-later
package actions
import (
"os"
"path/filepath"
"strings"
"testing"
"forgejo.org/models/unittest"
"forgejo.org/modules/git"
"forgejo.org/modules/setting"
"code.forgejo.org/forgejo/runner/v12/act/jobparser"
"code.forgejo.org/forgejo/runner/v12/act/model"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.yaml.in/yaml/v3"
)
const testWorkflow string = `on:
workflow_call:
inputs:
example-string-required:
required: true
type: string
name: test
jobs:
job1:
name: "job1 (local)"
runs-on: ubuntu-slim
steps:
- name: Echo inputs
run: |
echo example-string-required="${{ inputs.example-string-required }}"
`
func TestExpandForJob(t *testing.T) {
job := jobparser.Job{}
err := yaml.Unmarshal([]byte("{ name: job1 }"), &job)
require.NoError(t, err)
assert.True(t, expandForJob(&job))
err = yaml.Unmarshal([]byte("{ name: job1, runs-on: ubuntu-latest }"), &job)
require.NoError(t, err)
assert.False(t, expandForJob(&job))
err = yaml.Unmarshal([]byte("{ name: job1, runs-on: [x64, ubuntu-latest] }"), &job)
require.NoError(t, err)
assert.False(t, expandForJob(&job))
}
func TestExpandLocalReusableWorkflows(t *testing.T) {
gitRepo, err := git.OpenRepository(git.DefaultContext, "./TestExpandLocalReusableWorkflows")
require.NoError(t, err)
defer gitRepo.Close()
commit, err := gitRepo.GetCommit("e3868ecb4f8b483fc0bdd422561bf0062a7df907")
require.NoError(t, err)
fetcher := expandLocalReusableWorkflows(commit)
require.NotNil(t, fetcher)
t.Run("successful fetch", func(t *testing.T) {
content, err := fetcher(&jobparser.Job{}, "./.forgejo/workflows/reusable-1.yml")
require.NoError(t, err)
assert.Equal(t, testWorkflow, string(content))
})
t.Run("file not exist", func(t *testing.T) {
_, err = fetcher(&jobparser.Job{}, "./forgejo/workflows/reusable-2.yml")
require.ErrorContains(t, err, "expanding reusable workflow failed to access path ./forgejo/workflows/reusable-2.yml: object does not exist")
})
t.Run("do not expand due to runs-on", func(t *testing.T) {
jobWithRunsOn := jobparser.Job{}
err = yaml.Unmarshal([]byte("{ name: job1, runs-on: ubuntu-latest }"), &jobWithRunsOn)
require.NoError(t, err)
assert.False(t, expandForJob(&jobWithRunsOn))
_, err = fetcher(&jobWithRunsOn, "./.forgejo/workflows/reusable-1.yml")
require.ErrorIs(t, jobparser.ErrUnsupportedReusableWorkflowFetch, err)
})
}
func replaceTestRepo(t *testing.T, owner, repo, replacement string) {
t.Helper()
// Copy the repository into the target path that `gitrepo.OpenRepository` will look for it.
repoPath := filepath.Join(setting.RepoRootPath, strings.ToLower(owner), strings.ToLower(repo)+".git")
err := os.RemoveAll(repoPath) // there's a default repo copied here by the fixture setup that we want to replace
require.NoError(t, err)
err = os.CopyFS(repoPath, os.DirFS(replacement))
require.NoError(t, err)
}
func TestLazyRepoExpandLocalReusableWorkflows(t *testing.T) {
require.NoError(t, unittest.PrepareTestDatabase())
// Shouldn't need valid content if we never call the lazy evaluator
lazy1, cleanup := lazyRepoExpandLocalReusableWorkflow(t.Context(), -123456, "this is not a valid commit SHA")
assert.NotNil(t, lazy1)
assert.NotNil(t, cleanup)
cleanup()
replaceTestRepo(t, "user2", "repo1", "./TestExpandLocalReusableWorkflows")
lazy2, cleanup := lazyRepoExpandLocalReusableWorkflow(t.Context(), 1, "e3868ecb4f8b483fc0bdd422561bf0062a7df907")
assert.NotNil(t, lazy2)
assert.NotNil(t, cleanup)
content, err := lazy2(&jobparser.Job{}, "./.forgejo/workflows/reusable-1.yml")
require.NoError(t, err)
assert.Equal(t, testWorkflow, string(content))
cleanup()
}
func TestExpandInstanceReusableWorkflows(t *testing.T) {
require.NoError(t, unittest.PrepareTestDatabase())
tests := []struct {
name string
ref *model.NonLocalReusableWorkflowReference
errIs error
errorContains string
repo string
hasRunsOn bool
}{
{
name: "hasRunsOn",
hasRunsOn: true,
ref: &model.NonLocalReusableWorkflowReference{},
errIs: jobparser.ErrUnsupportedReusableWorkflowFetch,
},
{
name: "non-existent owner",
ref: &model.NonLocalReusableWorkflowReference{
Org: "owner-does-not-exist",
},
errorContains: "owner-does-not-exist: user does not exist",
},
{
name: "non-public owner",
ref: &model.NonLocalReusableWorkflowReference{
Org: "user33",
},
errorContains: "user33: user does not exist",
},
{
name: "non-existent repo",
ref: &model.NonLocalReusableWorkflowReference{
Org: "user2",
Repo: "repo10000",
},
errorContains: "repo10000: repo does not exist",
},
{
name: "non-public repo",
ref: &model.NonLocalReusableWorkflowReference{
Org: "user2",
Repo: "repo2",
},
errorContains: "repo2: repo does not exist",
},
{
name: "public repo",
ref: &model.NonLocalReusableWorkflowReference{
Org: "user2",
Repo: "repo1",
GitPlatform: "forgejo",
Filename: "reusable-1.yml",
Ref: "main",
},
repo: "./TestExpandLocalReusableWorkflows",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.repo != "" {
replaceTestRepo(t, tt.ref.Org, tt.ref.Repo, tt.repo)
}
job := jobparser.Job{}
if tt.hasRunsOn {
err := yaml.Unmarshal([]byte("{ name: job1, runs-on: ubuntu-latest }"), &job)
require.NoError(t, err)
}
fetcher := expandInstanceReusableWorkflows(t.Context())
content, err := fetcher(&job, tt.ref)
if tt.errIs != nil {
require.ErrorIs(t, err, tt.errIs)
} else if tt.errorContains != "" {
require.ErrorContains(t, err, tt.errorContains)
} else {
require.NoError(t, err)
assert.Equal(t, testWorkflow, string(content))
}
})
}
}