- Add support to set `gpg.format` in the Git config, via the new `[repository.signing].FORMAT` option. This is to tell Git that the instance would like to use SSH instead of OpenPGP to sign its commits. This is guarded behind a Git version check for v2.34.0 and a check that a `ssh-keygen` binary is present.
- Add support to recognize the public SSH key that is given to `[repository.signing].SIGNING_KEY` as the signing key by the instance.
- Thus this allows the instance to use SSH commit signing for commits that the instance creates (e.g. initial and squash commits) instead of using PGP.
- Technically (although I have no clue how as this is not documented) you can have a different PGP signing key for different repositories; this is not implemented for SSH signing.
- Add unit and integration testing.
- `TestInstanceSigning` was reworked from `TestGPGGit`, now also includes testing for SHA256 repositories. Is the main integration test that actually signs commits and checks that they are marked as verified by Forgejo.
- `TestParseCommitWithSSHSignature` is a unit test that makes sure that if a SSH instnace signing key is set, that it is used to possibly verify instance SSH signed commits.
- `TestSyncConfigGPGFormat` is a unit test that makes sure the correct git config is set according to the signing format setting. Also checks that the guarded git version check and ssh-keygen binary presence check is done correctly.
- `TestSSHInstanceKey` is a unit test that makes sure the parsing of a SSH signing key is done correctly.
- `TestAPISSHSigningKey` is a integration test that makes sure the newly added API route `/api/v1/signing-key.ssh` responds correctly.
Documentation PR: forgejo/docs#1122
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6897
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
This allows syncing a branch of a fork with a branch of the base repo. It looks like this:
This is only possible, if the fork don't have commits that are not in the main repo.
The feature is already working, but it is missing Finetuning, a better API, translations and tests, so this is currently WIP. It is also not tested with go-git.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/2364): <!--number 2364 --><!--line 0 --><!--description c3luYyBmb3Jrcw==-->sync forks<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/2364
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-committed-by: JakobDev <jakobdev@gmx.de>
Fuzzy searching for code has been known to be problematic #5264 and in my personal opinion isn't very useful.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6947
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Shiny Nematoda <snematoda.751k2@aleeas.com>
Co-committed-by: Shiny Nematoda <snematoda.751k2@aleeas.com>
## Dropping SSPI auth support
SSPI authentication relied on Microsoft Windows support, removal started in https://codeberg.org/forgejo/forgejo/pulls/5353, because it was broken anyway. We have no knowledge of any users using SSPI authentication. However, if you somehow managed to run Forgejo on Windows, or want to upgrade from a Gitea version which does, please ensure that you do not use SSPI as an authentication mechanism for user accounts. Feel free to reach out if you need assistance.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7148
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Otto Richter <otto@codeberg.org>
Co-committed-by: Otto Richter <otto@codeberg.org>
- adds the `toml` plugin to the `eslint` linting → expect to have `options/setting/config.toml` by #6862
- fixes `make lint-codespell` commands
- related concerning `codespell`: #3270
- info: codespell check is and was not activated in the workflows (could maybe, runs only few seconds on my system)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7007
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Robert Wolff <mahlzahn@posteo.de>
Co-committed-by: Robert Wolff <mahlzahn@posteo.de>
improve language stats rounding:
- Add tests (I had to omit some edge cases as the current method is
non-determistic in some cases, due to random order of map access).
- Document the algorithm used.
- Lower the amount of calculations that need to be done.
- Because of the aforementioned non-determistic don't use stable sort
and instead regular sort.
better sorting for `RepositoryList`:
- Add testing
- Use `slices.Sortfunc` instead of `sort.Sort`.
- Remove the methods needed for `sort.Sort`.
better git tag sorter:
- Use `slices.SortFunc` instead of `sort.Sort`.
- Remove `tagSorter` and its related methods.
- Added testing.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6689
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
The latest commit cache is currently used only for listing tree files.
However, a cold start may take longer than directly invoking the Git
command. This PR addresses the issue of slow response times when
accessing raw files, improving performance in such scenarios.
```log
gitea.log:105521:2024/12/23 08:22:18 ...eb/routing/logger.go:68:func1() [W] router: slow GET /xxxx/xxxxxx/raw/commit/xxxxxxxxxxxxxxxxxxxxxxxxxxx/.editorconfig for 172.18.0.5:53252, elapsed 3526.8ms @ repo/download.go:117(repo.SingleDownload)
```
(cherry picked from commit f4ccbd38dca77b1515a08ddf927f4f20cf644d30)
- For every process that is spawned (every new non-trivial goroutine
such as http requests, queues or tasks) start a [execution
tracer](https://pkg.go.dev/runtime/trace). This allows very precise
diagnosis of how each individual process over a time period.
- It's safe and [fast](https://go.dev/blog/execution-traces-2024#low-overhead-tracing) to
be run in production, hence no setting to disable this. There's only
noticable overhead when tracing is actually performed and not continuous.
- Proper tracing support would mean the codebase would be full of
`trace.WithRegion` and `trace.Log`, which feels premature for this patch
as there's no real-world usage yet to indicate which places would need
this the most. So far only Git commands and SQL queries receive somewhat
proper tracing support given that these are used throughout the codebase.
- Make git commands a new process type.
- Add tracing to diagnosis zip file.
Added support for searching content in a specific directory or file.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6143
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Shiny Nematoda <snematoda.751k2@aleeas.com>
Co-committed-by: Shiny Nematoda <snematoda.751k2@aleeas.com>
- `GetSubModules` already solely stores the URL of the submodule and not
a `*SubModule` entry, so don't try to type assert it to be a struct.
- I am not able to pinpoint when this was regressed but if I had to
guess it might be #4941.
- Added integration test.
(cherry picked from commit 33850a83fe4ebd23a762a7aac81614c42e303bfa)
This really is just the cherry pick of 407b6e6dfc7ee9ebb8a16c7f1a786e4c24d0516e
which is the first commit of the pull request, the one with the
change. The rest of the changes is a refactor that is unrelated to the
bug fix.
Conflicts:
modules/git/commit_test.go
trivial context conflict
Git has a cool feature called git notes. It allows adding a text to a commit without changing the commit itself. Forgejo already displays git notes. With this PR you can also now change git notes.
<details>
<summary>Screenshots</summary>
</details>
## Checklist
The [developer guide](https://forgejo.org/docs/next/developer/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/4753): <!--number 4753 --><!--line 0 --><!--description QWxsb3cgY2hhbmdpbmcgZ2l0IG5vdGVz-->Allow changing git notes<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4753
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-committed-by: JakobDev <jakobdev@gmx.de>
---
Conflict resolution: Trivial, for `repo_attributes.go` move where the
`IsErrCanceledOrKilled` needs to happen because of other changes that
happened in this file.
To add some words to this change: It seems to be mostly simplifying the
error handling of git operations.
(cherry picked from commit e524f63d58900557d7d57fc3bcd19d9facc8b8ee)
- If `GetAffectedFiles` is called for a push with an empty oldCommitID,
then set the oldCommitID to the empty tree. This will effictively diff
all the changes included in the push, which is the expected behavior for
branches.
- Integration test added.
- Resolves#5683
- Port of gitea#31778 but implemented differently.
- When an admin wants syncronize tags in the Git data to the database
via the admin dashboard all annoted tags loses their title. This was
caused because the code didn't correctly handle annoted tags. Annoted
tags have their own objectID to store the annoted message, unlike
'normal' tags which point to the commitID. While the function was being
run for annoted tags, the code thought it found a mismatch in the
objectIDs, because the stored version was actually correct which pointed
to the commitID but the code found the objectID of the annoted tag.
- Make `SyncReleasesWithTags` corectly handle annoted tags.
- Added unit and integration tests.
- Resolves#5628
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Co-authored-by: Ehsan Shirvanian <ehsan@duck.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5503
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: ehshi <ehshi@noreply.codeberg.org>
Co-committed-by: ehshi <ehshi@noreply.codeberg.org>
- Follow up of #4819
- When no `ssh` executable is present, disable the UI and backend bits
that allow the creation of push mirrors that use SSH authentication. As
this feature requires the usage of the `ssh` binary.
- Integration test added.
When opening a repository, it will call `ensureValidRepository` and also
`CatFileBatch`. But sometimes these will not be used until repository
closed. So it's a waste of CPU to invoke 3 times git command for every
open repository.
This PR removed all of these from `OpenRepository` but only kept
checking whether the folder exists. When a batch is necessary, the
necessary functions will be invoked.
---
Conflict resolution: Because of the removal of go-git in (#4941)
`_nogogit.go` files were either renamed or merged into the 'common'
file. Git does handle the renames correctly, but for those that were
merged has to be manually copied pasted over. The patch looks the same,
201 additions 90 deletions as the original patch.
(cherry picked from commit c03baab678ba5b2e9d974aea147e660417f5d3f7)
- Continuation of https://github.com/go-gitea/gitea/pull/18835 (by
@Gusted, so it's fine to change copyright holder to Forgejo).
- Add the option to use SSH for push mirrors, this would allow for the
deploy keys feature to be used and not require tokens to be used which
cannot be limited to a specific repository. The private key is stored
encrypted (via the `keying` module) on the database and NEVER given to
the user, to avoid accidental exposure and misuse.
- CAVEAT: This does require the `ssh` binary to be present, which may
not be available in containerized environments, this could be solved by
adding a SSH client into forgejo itself and use the forgejo binary as
SSH command, but should be done in another PR.
- CAVEAT: Mirroring of LFS content is not supported, this would require
the previous stated problem to be solved due to LFS authentication (an
attempt was made at forgejo/forgejo#2544).
- Integration test added.
- Resolves#4416
One method to set them all... or something like that.
The defaults for git-grep options were scattered over the run
function body. This change refactors them into a separate method.
The application of defaults is checked implicitly by existing
tests and linters, and the new approach makes it very easy
to inspect the desired defaults are set.
We need to shorten the timeout to bound effectively for
computation size. This protects against "too big" repos.
This also protects to some extent against too long lines
if kept to very low values (basically so that grep cannot run out
of memory beforehand).
Docs-PR: forgejo/docs#812
Fix#31271.
When gogit is enabled, `IsObjectExist` calls
`repo.gogitRepo.ResolveRevision`, which is not correct. It's for
checking references not objects, it could work with commit hash since
it's both a valid reference and a commit object, but it doesn't work
with blob objects.
So it causes #31271 because it reports that all blob objects do not
exist.
(cherry picked from commit f4d3120f9d1de6a260a5e625b3ffa6b35a069e9b)
Conflicts:
trivial resolution because go-git support was dropped https://codeberg.org/forgejo/forgejo/pulls/4941
There is no reason to reject initial dashes in git-grep
expressions... other than the code not supporting it previously.
A new method is introduced to relax the security checks.
See https://codeberg.org/forgejo/discussions/issues/164 for the
rationale and discussion of this change.
Everything related to the `go-git` dependency is dropped (Only a single
instance is left in a test file to test for an XSS, it requires crafting
an commit that Git itself refuses to craft). `_gogit` files have
been removed entirely, `go:build: !gogit` is removed, `XXX_nogogit.go` files
either have been renamed or had their code being merged into the
`XXX.go` file.
It is a waste of resources to scan them looking for matches
because they are never returned back - they appear as empty
lines in the current format.
Notably, even if they were returned, it is unlikely that matching
in binary files makes sense when the goal is "code search".
Analogously to how it happens for MaxResultLimit.
The default of 20 is inspired by a well-known, commercial code
hosting platform.
Unbounded limits are risky because they expose Forgejo to a class
of DoS attacks where queries are crafted to take advantage of
missing bounds.
Running git update-index for every individual file is slow, so add and
remove everything with a single git command.
When such a big commit lands in the default branch, it could cause PR
creation and patch checking for all open PRs to be slow, or time out
entirely. For example, a commit that removes 1383 files was measured to
take more than 60 seconds and timed out. With this change checking took
about a second.
This is related to #27967, though this will not help with commits that
change many lines in few files.
(cherry picked from commit b88e5fc72d99e9d4a0aa9c13f70e0a9e967fe057)
If a repository has
git config --add push.pushOption submit=".sourcehut/*.yml"
it failed when pushed because of the unknown submit push
option. It will be ignored instead.
Filtering out the push options is done in an earlier stage, when the
hook command runs, before it submits the options map to the private
endpoint.
* move all the push options logic to modules/git/pushoptions
* add 100% test coverage for modules/git/pushoptions
Test coverage for the code paths from which code was moved to the
modules/git/pushoptions package:
* cmd/hook.go:runHookPreReceive
* routers/private/hook_pre_receive.go:validatePushOptions
tests/integration/git_push_test.go:TestOptionsGitPush runs through
both. The test verifying the option is rejected was removed and, if
added again, will fail because the option is now ignored instead of
being rejected.
* cmd/hook.go:runHookProcReceive
* services/agit/agit.go:ProcReceive
tests/integration/git_test.go: doCreateAgitFlowPull runs through
both. It uses variations of AGit related push options.
* cmd/hook.go:runHookPostReceive
* routers/private/hook_post_receive.go:HookPostReceive
tests/integration/git_test.go:doPushCreate called by TestGit/HTTP/sha1/PushCreate
runs through both.
Note that although it provides coverage for this code path it does not use push options.
Fixes: https://codeberg.org/forgejo/forgejo/issues/3651
Enable [unparam](https://github.com/mvdan/unparam) linter.
Often I could not tell the intention why param is unused, so I put
`//nolint` for those cases like webhook request creation functions never
using `ctx`.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit fc2d75f86d77b022ece848acf2581c14ef21d43b)
Conflicts:
modules/setting/config_env.go
modules/storage/azureblob.go
services/webhook/dingtalk.go
services/webhook/discord.go
services/webhook/feishu.go
services/webhook/matrix.go
services/webhook/msteams.go
services/webhook/packagist.go
services/webhook/slack.go
services/webhook/telegram.go
services/webhook/wechatwork.go
run make lint-go and fix Forgejo specific warnings
Remove "EncodeSha1", it shouldn't be used as a general purpose hasher
(just like we have removed "EncodeMD5" in #28622)
Rewrite the "time-limited code" related code and write better tests, the
old code doesn't seem quite right.
(cherry picked from commit fb1ad920b769799aa1287441289d15477d9878c5)
Conflicts:
modules/git/utils_test.go
trivial context conflict because sha256 testing in Forgejo has diverged
More about codespell: https://github.com/codespell-project/codespell .
I personally introduced it to dozens if not hundreds of projects already and so far only positive feedback.
```
❯ grep lint-spell Makefile
@echo " - lint-spell lint spelling"
@echo " - lint-spell-fix lint spelling and fix issues"
lint: lint-frontend lint-backend lint-spell
lint-fix: lint-frontend-fix lint-backend-fix lint-spell-fix
.PHONY: lint-spell
lint-spell: lint-codespell
.PHONY: lint-spell-fix
lint-spell-fix: lint-codespell-fix
❯ git grep lint- -- .forgejo/
.forgejo/workflows/testing.yml: - run: make --always-make -j$(nproc) lint-backend checks-backend # ensure the "go-licenses" make target runs
.forgejo/workflows/testing.yml: - run: make lint-frontend
```
so how would you like me to invoke `lint-codespell` on CI? (without that would be IMHO very suboptimal and let typos sneak in)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3270
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
Resolve all cases for `unused parameter` and `unnecessary type
arguments`
Related: #30729
---------
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit e80466f7349164ce4cf3c07bdac30d736d20f035)
Conflicts:
modules/markup/markdown/transform_codespan.go
modules/setting/incoming_email.go
routers/api/v1/admin/user_badge.go
routers/private/hook_pre_receive.go
tests/integration/repo_search_test.go
resolved by discarding the change, this is linting only and
for the sake of avoiding future conflicts
Noteable additions:
- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches
Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
(cherry picked from commit 74f0c84fa4245a20ce6fb87dac1faf2aeeded2a2)
Conflicts:
.golangci.yml
apply the linter recommendations to Forgejo code as well
- The parser of `git grep`'s output uses `bufio.Scanner`, which is a good
choice overall, however it does have a limit that's usually not noticed,
it will not read more than `64 * 1024` bytes at once which can be hit in
practical scenarios.
- Use `bufio.Reader` instead which doesn't have this limitation, but is
a bit harder to work with as it's a more lower level primitive.
- Adds unit test.
- Resolves https://codeberg.org/forgejo/forgejo/issues/3149
- `%w` is to wrap errors, but can only be used by `fmt.Errorf`. Instead
use `%v` to display the error.
- Regression of #2763
Before:
[E] failed to run attr-check. Error: %!w(*exec.ExitError=&{0xc006568e28 []})
Stderr: fatal: this operation must be run in a work tree
After:
[E] failed to run attr-check. Error: exit status 128
Stderr: fatal: this operation must be run in a work tree
`CommitGPGSignature` was originally made to store information about a
commit's GPG signature. Nowadays, it is used to store information about
SSH signatures too, and not just commit signatures, but tag signatures
too.
As such, rename it to `ObjectSignature`, because that describes what it
does a whole lot better.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Just like commits, tags can be signed with either an OpenPGP, or with an
SSH key. While the latter is supported already, SSH-signed tags have not
been. This patch teaches the git module to recognize and handle
SSH-signed tags too.
This will stop the signatures appearing in release notes, but are
currently unused otherwise.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Most time, when invoking `git.OpenRepository`, `objectFormat` will not
be used, so it's a waste to invoke commandline to get the object format.
This PR make it a lazy operation, only invoke that when necessary.
(cherry picked from commit e84e5db6de0306d514b1f1a9657931fb7197a188)
(cherry picked from commit 25b842df261452a29570ba89ffc3a4842d73f68c)
Conflicts:
routers/web/repo/wiki.go
services/repository/branch.go
services/repository/migrate.go
services/wiki/wiki.go
also apply to Forgejo specific usage of the refactored functions
Close#29509
Windows, unlike Linux, does not have signal-specified exit codes.
Therefore, we should add a Windows-specific check for Windows. If we
don't do this, the logs will always show a failed status, even though
the command actually works correctly.
If you check the Go source code in exec_windows.go, you will see that it
always returns exit code 1.
The exit code 1 does not exclusively signify a SIGNAL KILL; it can
indicate any issue that occurs when a program fails.
(cherry picked from commit 423372d84ab3d885e47d4a00cd69d6040b61cc4c)
- When a user goes opens a symlink file in Forgejo, the file would be
rendered with the path of the symlink as content.
- Add a button that is shown when the user opens a *valid* symlink file,
which means that the symlink must have an valid path to an existent
file and after 999 follows isn't a symlink anymore.
- Return the relative path from the `FollowLink` functions, because Git
really doesn't want to tell where an file is located based on the blob ID.
- Adds integration tests.
