(cherry picked from commit 33850a83fe4ebd23a762a7aac81614c42e303bfa)
This really is just the cherry pick of 407b6e6dfc7ee9ebb8a16c7f1a786e4c24d0516e
which is the first commit of the pull request, the one with the
change. The rest of the changes is a refactor that is unrelated to the
bug fix.
Conflicts:
modules/git/commit_test.go
trivial context conflict
Git has a cool feature called git notes. It allows adding a text to a commit without changing the commit itself. Forgejo already displays git notes. With this PR you can also now change git notes.
<details>
<summary>Screenshots</summary>
</details>
## Checklist
The [developer guide](https://forgejo.org/docs/next/developer/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/4753): <!--number 4753 --><!--line 0 --><!--description QWxsb3cgY2hhbmdpbmcgZ2l0IG5vdGVz-->Allow changing git notes<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4753
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-committed-by: JakobDev <jakobdev@gmx.de>
---
Conflict resolution: Trivial, for `repo_attributes.go` move where the
`IsErrCanceledOrKilled` needs to happen because of other changes that
happened in this file.
To add some words to this change: It seems to be mostly simplifying the
error handling of git operations.
(cherry picked from commit e524f63d58900557d7d57fc3bcd19d9facc8b8ee)
- If `GetAffectedFiles` is called for a push with an empty oldCommitID,
then set the oldCommitID to the empty tree. This will effictively diff
all the changes included in the push, which is the expected behavior for
branches.
- Integration test added.
- Resolves#5683
- Port of gitea#31778 but implemented differently.
- When an admin wants syncronize tags in the Git data to the database
via the admin dashboard all annoted tags loses their title. This was
caused because the code didn't correctly handle annoted tags. Annoted
tags have their own objectID to store the annoted message, unlike
'normal' tags which point to the commitID. While the function was being
run for annoted tags, the code thought it found a mismatch in the
objectIDs, because the stored version was actually correct which pointed
to the commitID but the code found the objectID of the annoted tag.
- Make `SyncReleasesWithTags` corectly handle annoted tags.
- Added unit and integration tests.
- Resolves#5628
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Co-authored-by: Ehsan Shirvanian <ehsan@duck.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5503
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: ehshi <ehshi@noreply.codeberg.org>
Co-committed-by: ehshi <ehshi@noreply.codeberg.org>
- Follow up of #4819
- When no `ssh` executable is present, disable the UI and backend bits
that allow the creation of push mirrors that use SSH authentication. As
this feature requires the usage of the `ssh` binary.
- Integration test added.
When opening a repository, it will call `ensureValidRepository` and also
`CatFileBatch`. But sometimes these will not be used until repository
closed. So it's a waste of CPU to invoke 3 times git command for every
open repository.
This PR removed all of these from `OpenRepository` but only kept
checking whether the folder exists. When a batch is necessary, the
necessary functions will be invoked.
---
Conflict resolution: Because of the removal of go-git in (#4941)
`_nogogit.go` files were either renamed or merged into the 'common'
file. Git does handle the renames correctly, but for those that were
merged has to be manually copied pasted over. The patch looks the same,
201 additions 90 deletions as the original patch.
(cherry picked from commit c03baab678ba5b2e9d974aea147e660417f5d3f7)
- Continuation of https://github.com/go-gitea/gitea/pull/18835 (by
@Gusted, so it's fine to change copyright holder to Forgejo).
- Add the option to use SSH for push mirrors, this would allow for the
deploy keys feature to be used and not require tokens to be used which
cannot be limited to a specific repository. The private key is stored
encrypted (via the `keying` module) on the database and NEVER given to
the user, to avoid accidental exposure and misuse.
- CAVEAT: This does require the `ssh` binary to be present, which may
not be available in containerized environments, this could be solved by
adding a SSH client into forgejo itself and use the forgejo binary as
SSH command, but should be done in another PR.
- CAVEAT: Mirroring of LFS content is not supported, this would require
the previous stated problem to be solved due to LFS authentication (an
attempt was made at forgejo/forgejo#2544).
- Integration test added.
- Resolves#4416
One method to set them all... or something like that.
The defaults for git-grep options were scattered over the run
function body. This change refactors them into a separate method.
The application of defaults is checked implicitly by existing
tests and linters, and the new approach makes it very easy
to inspect the desired defaults are set.
We need to shorten the timeout to bound effectively for
computation size. This protects against "too big" repos.
This also protects to some extent against too long lines
if kept to very low values (basically so that grep cannot run out
of memory beforehand).
Docs-PR: forgejo/docs#812
Fix#31271.
When gogit is enabled, `IsObjectExist` calls
`repo.gogitRepo.ResolveRevision`, which is not correct. It's for
checking references not objects, it could work with commit hash since
it's both a valid reference and a commit object, but it doesn't work
with blob objects.
So it causes #31271 because it reports that all blob objects do not
exist.
(cherry picked from commit f4d3120f9d1de6a260a5e625b3ffa6b35a069e9b)
Conflicts:
trivial resolution because go-git support was dropped https://codeberg.org/forgejo/forgejo/pulls/4941
There is no reason to reject initial dashes in git-grep
expressions... other than the code not supporting it previously.
A new method is introduced to relax the security checks.
See https://codeberg.org/forgejo/discussions/issues/164 for the
rationale and discussion of this change.
Everything related to the `go-git` dependency is dropped (Only a single
instance is left in a test file to test for an XSS, it requires crafting
an commit that Git itself refuses to craft). `_gogit` files have
been removed entirely, `go:build: !gogit` is removed, `XXX_nogogit.go` files
either have been renamed or had their code being merged into the
`XXX.go` file.
It is a waste of resources to scan them looking for matches
because they are never returned back - they appear as empty
lines in the current format.
Notably, even if they were returned, it is unlikely that matching
in binary files makes sense when the goal is "code search".
Analogously to how it happens for MaxResultLimit.
The default of 20 is inspired by a well-known, commercial code
hosting platform.
Unbounded limits are risky because they expose Forgejo to a class
of DoS attacks where queries are crafted to take advantage of
missing bounds.
Running git update-index for every individual file is slow, so add and
remove everything with a single git command.
When such a big commit lands in the default branch, it could cause PR
creation and patch checking for all open PRs to be slow, or time out
entirely. For example, a commit that removes 1383 files was measured to
take more than 60 seconds and timed out. With this change checking took
about a second.
This is related to #27967, though this will not help with commits that
change many lines in few files.
(cherry picked from commit b88e5fc72d99e9d4a0aa9c13f70e0a9e967fe057)
If a repository has
git config --add push.pushOption submit=".sourcehut/*.yml"
it failed when pushed because of the unknown submit push
option. It will be ignored instead.
Filtering out the push options is done in an earlier stage, when the
hook command runs, before it submits the options map to the private
endpoint.
* move all the push options logic to modules/git/pushoptions
* add 100% test coverage for modules/git/pushoptions
Test coverage for the code paths from which code was moved to the
modules/git/pushoptions package:
* cmd/hook.go:runHookPreReceive
* routers/private/hook_pre_receive.go:validatePushOptions
tests/integration/git_push_test.go:TestOptionsGitPush runs through
both. The test verifying the option is rejected was removed and, if
added again, will fail because the option is now ignored instead of
being rejected.
* cmd/hook.go:runHookProcReceive
* services/agit/agit.go:ProcReceive
tests/integration/git_test.go: doCreateAgitFlowPull runs through
both. It uses variations of AGit related push options.
* cmd/hook.go:runHookPostReceive
* routers/private/hook_post_receive.go:HookPostReceive
tests/integration/git_test.go:doPushCreate called by TestGit/HTTP/sha1/PushCreate
runs through both.
Note that although it provides coverage for this code path it does not use push options.
Fixes: https://codeberg.org/forgejo/forgejo/issues/3651
Enable [unparam](https://github.com/mvdan/unparam) linter.
Often I could not tell the intention why param is unused, so I put
`//nolint` for those cases like webhook request creation functions never
using `ctx`.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit fc2d75f86d77b022ece848acf2581c14ef21d43b)
Conflicts:
modules/setting/config_env.go
modules/storage/azureblob.go
services/webhook/dingtalk.go
services/webhook/discord.go
services/webhook/feishu.go
services/webhook/matrix.go
services/webhook/msteams.go
services/webhook/packagist.go
services/webhook/slack.go
services/webhook/telegram.go
services/webhook/wechatwork.go
run make lint-go and fix Forgejo specific warnings
Remove "EncodeSha1", it shouldn't be used as a general purpose hasher
(just like we have removed "EncodeMD5" in #28622)
Rewrite the "time-limited code" related code and write better tests, the
old code doesn't seem quite right.
(cherry picked from commit fb1ad920b769799aa1287441289d15477d9878c5)
Conflicts:
modules/git/utils_test.go
trivial context conflict because sha256 testing in Forgejo has diverged
More about codespell: https://github.com/codespell-project/codespell .
I personally introduced it to dozens if not hundreds of projects already and so far only positive feedback.
```
❯ grep lint-spell Makefile
@echo " - lint-spell lint spelling"
@echo " - lint-spell-fix lint spelling and fix issues"
lint: lint-frontend lint-backend lint-spell
lint-fix: lint-frontend-fix lint-backend-fix lint-spell-fix
.PHONY: lint-spell
lint-spell: lint-codespell
.PHONY: lint-spell-fix
lint-spell-fix: lint-codespell-fix
❯ git grep lint- -- .forgejo/
.forgejo/workflows/testing.yml: - run: make --always-make -j$(nproc) lint-backend checks-backend # ensure the "go-licenses" make target runs
.forgejo/workflows/testing.yml: - run: make lint-frontend
```
so how would you like me to invoke `lint-codespell` on CI? (without that would be IMHO very suboptimal and let typos sneak in)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3270
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
Resolve all cases for `unused parameter` and `unnecessary type
arguments`
Related: #30729
---------
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit e80466f7349164ce4cf3c07bdac30d736d20f035)
Conflicts:
modules/markup/markdown/transform_codespan.go
modules/setting/incoming_email.go
routers/api/v1/admin/user_badge.go
routers/private/hook_pre_receive.go
tests/integration/repo_search_test.go
resolved by discarding the change, this is linting only and
for the sake of avoiding future conflicts
Noteable additions:
- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches
Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
(cherry picked from commit 74f0c84fa4245a20ce6fb87dac1faf2aeeded2a2)
Conflicts:
.golangci.yml
apply the linter recommendations to Forgejo code as well
- The parser of `git grep`'s output uses `bufio.Scanner`, which is a good
choice overall, however it does have a limit that's usually not noticed,
it will not read more than `64 * 1024` bytes at once which can be hit in
practical scenarios.
- Use `bufio.Reader` instead which doesn't have this limitation, but is
a bit harder to work with as it's a more lower level primitive.
- Adds unit test.
- Resolves https://codeberg.org/forgejo/forgejo/issues/3149
- `%w` is to wrap errors, but can only be used by `fmt.Errorf`. Instead
use `%v` to display the error.
- Regression of #2763
Before:
[E] failed to run attr-check. Error: %!w(*exec.ExitError=&{0xc006568e28 []})
Stderr: fatal: this operation must be run in a work tree
After:
[E] failed to run attr-check. Error: exit status 128
Stderr: fatal: this operation must be run in a work tree
`CommitGPGSignature` was originally made to store information about a
commit's GPG signature. Nowadays, it is used to store information about
SSH signatures too, and not just commit signatures, but tag signatures
too.
As such, rename it to `ObjectSignature`, because that describes what it
does a whole lot better.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Just like commits, tags can be signed with either an OpenPGP, or with an
SSH key. While the latter is supported already, SSH-signed tags have not
been. This patch teaches the git module to recognize and handle
SSH-signed tags too.
This will stop the signatures appearing in release notes, but are
currently unused otherwise.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Most time, when invoking `git.OpenRepository`, `objectFormat` will not
be used, so it's a waste to invoke commandline to get the object format.
This PR make it a lazy operation, only invoke that when necessary.
(cherry picked from commit e84e5db6de0306d514b1f1a9657931fb7197a188)
(cherry picked from commit 25b842df261452a29570ba89ffc3a4842d73f68c)
Conflicts:
routers/web/repo/wiki.go
services/repository/branch.go
services/repository/migrate.go
services/wiki/wiki.go
also apply to Forgejo specific usage of the refactored functions
Close#29509
Windows, unlike Linux, does not have signal-specified exit codes.
Therefore, we should add a Windows-specific check for Windows. If we
don't do this, the logs will always show a failed status, even though
the command actually works correctly.
If you check the Go source code in exec_windows.go, you will see that it
always returns exit code 1.
The exit code 1 does not exclusively signify a SIGNAL KILL; it can
indicate any issue that occurs when a program fails.
(cherry picked from commit 423372d84ab3d885e47d4a00cd69d6040b61cc4c)
- When a user goes opens a symlink file in Forgejo, the file would be
rendered with the path of the symlink as content.
- Add a button that is shown when the user opens a *valid* symlink file,
which means that the symlink must have an valid path to an existent
file and after 999 follows isn't a symlink anymore.
- Return the relative path from the `FollowLink` functions, because Git
really doesn't want to tell where an file is located based on the blob ID.
- Adds integration tests.
Fixes#29101
Related #29298
Discard all read data to prevent misinterpreting existing data. Some
discard calls were missing in error cases.
---------
Co-authored-by: yp05327 <576951401@qq.com>
(cherry picked from commit d6811baf88ca6d58b92d4dc12b1f2a292198751f)
Fixes the reason why #29101 is hard to replicate.
Related #29297
Create a repo with a file with minimum size 4097 bytes (I use 10000) and
execute the following code:
```go
gitRepo, err := gitrepo.OpenRepository(db.DefaultContext, <repo>)
assert.NoError(t, err)
commit, err := gitRepo.GetCommit(<sha>)
assert.NoError(t, err)
entry, err := commit.GetTreeEntryByPath(<file>)
assert.NoError(t, err)
b := entry.Blob()
// Create a reader
r, err := b.DataAsync()
assert.NoError(t, err)
defer r.Close()
// Create a second reader
r2, err := b.DataAsync()
assert.NoError(t, err) // Should be no error but is ErrNotExist
defer r2.Close()
```
The problem is the check in `CatFileBatch`:
79217ea63c/modules/git/repo_base_nogogit.go (L81-L87)
`Buffered() > 0` is used to check if there is a "operation" in progress
at the moment. This is a problem because we can't control the internal
buffer in the `bufio.Reader`. The code above demonstrates a sequence
which initiates an operation for which the code thinks there is no
active processing. The second call to `DataAsync()` therefore reuses the
existing instances instead of creating a new batch reader.
(cherry picked from commit f74c869221624092999097af38b6f7fae4701420)
If a documentation file is marked with a `linguist-documentation=false`
attribute, include it in language stats.
However, make sure that we do *not* include documentation languages as
fallback.
Added a new test case to exercise the formerly buggy behaviour.
Problem discovered while reviewing @KN4CK3R's tests from gitea#29267.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Based on @KN4CK3R's work in gitea#29267. This drops the custom
`LinguistBoolAttrib` type, and uses `optional.Option` instead. I added
the `isTrue()` and `isFalse()` (function-local) helpers to make the code
easier to follow, because these names convey their goal better than
`v.ValueorDefault(false)` or `!v.ValueOrDefault(true)`.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
The test suite was broken e.g. on Debian 12 due to requiring a very
recent version of Git installed on the system. This commit skips SHA256
tests in the git module, if a Git version older than 2.42 or gogit is used.
With this option, it is possible to require a linear commit history with
the following benefits over the next best option `Rebase+fast-forward`:
The original commits continue existing, with the original signatures
continuing to stay valid instead of being rewritten, there is no merge
commit, and reverting commits becomes easier.
Closes#24906
- In Git version v2.43.1, the behavior of `GIT_FLUSH` was accidentially
flipped. This causes Forgejo to hang on the `check-attr` command,
because no output was being flushed.
- Workaround this by detecting if Git v2.43.1 is used and set
`GIT_FLUSH=0` thus getting the correct behavior.
- Ref: https://lore.kernel.org/git/CABn0oJvg3M_kBW-u=j3QhKnO=6QOzk-YFTgonYw_UvFS1NTX4g@mail.gmail.com/
- Resolves#2333.
Replace #28849. Thanks to @yp05327 for the looking into the problem.
Fix#28840
The old behavior of newSignatureFromCommitline is not right. The new
parseSignatureFromCommitLine:
1. never fails
2. only accept one format (if there is any other, it could be easily added)
And add some tests.
(cherry picked from commit a24e1da7e9e38fc5f5c84c083d122c0cc3da4b74)
Recognise the `linguist-documentation` and `linguist-detectable`
attributes in `.gitattributes` files, and use them in
`GetLanguageStats()` to make a decision whether to include a particular
file in the stats or not.
This allows one more control over which files in their repositories
contribute toward the language statistics, so that for a project that is
mostly documentation, the language stats can reflect that.
Fixes#1672.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
(cherry picked from commit 6d4e02fe5f)
(cherry picked from commit ee1ead8189)
(cherry picked from commit 2dbec730e8)
When trying to find a `README.md` in a `.profile` repo, do so case
insensitively. This change does not make it possible to render readmes
in formats other than Markdown, it just removes the hard-coded
"README.md".
Also adds a few tests to make sure the change works.
Fixes#1494.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
(cherry picked from commit edd219d8e9)
(cherry picked from commit 2c0105ef17)
(cherry picked from commit 3975a9f3aa)
(cherry picked from commit dee4a18423)
(cherry picked from commit 60aee6370f)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.
## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:
Firstly, there are temporary repositories which do not change:
```go
git.OpenRepository(ctx, diskPath)
```
Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:
```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```
Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.
## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.
---------
Co-authored-by: delvh <dev.lh@web.de>
This should fix https://github.com/go-gitea/gitea/issues/28927
Technically older versions of Git would support this flag as well, but
per https://github.com/go-gitea/gitea/pull/28466 that's the version
where using it (object-format=sha256) left "experimental" state.
`sha1` is (currently) the default, so older clients should be unaffected
in either case.
Signed-off-by: jolheiser <john.olheiser@gmail.com>
When LFS hooks are present in gitea-repositories, operations like git
push for creating a pull request fail. These repositories are not meant
to include LFS files or git push them, that is handled separately. And
so they should not have LFS hooks.
Installing git-lfs on some systems (like Debian Linux) will
automatically set up /etc/gitconfig to create LFS hooks in repositories.
For most git commands in Gitea this is not a problem, either because
they run on a temporary clone or the git command does not create LFS
hooks.
But one case where this happens is git archive for creating repository
archives. To fix that, add a GIT_CONFIG_NOSYSTEM=1 to disable using the
system configuration for that command.
According to a comment, GIT_CONFIG_NOSYSTEM is not used for all git
commands because the system configuration can be intentionally set up
for Gitea to use.
Resolves#19810, #21148
