With this pull request, diffs for Git are loaded in chunks. This means, that for diffs with a lot of files only a part of them are loaded. In the UI a button will be displayed to load more. In the REST API, the number of files can be specified. This only works for diffs, that are delivered as "parsed" diffs. Currently, this is only available for Git.
Co-authored-by: Sebastian Sdorra <sebastian.sdorra@cloudogu.com>
We will fire an RepositoryImportHookEvent instead of PostReceiveRepositoryHookEvent for repository imports with metadata. The event is only fired if all parts of the repository could be successfully imported. The extra event is required to avoid heavy recalculations which can be triggered by the PostReceiveRepositoryHookEvent for example the scm-statistic-plugin uses the PostReceiveRepositoryHookEvent to calculate its statistics.
Co-authored-by: Eduard Heimbuch <eduard.heimbuch@cloudogu.com>
Fire post receive repository hook event after pull from remote
and after unbundle (Git, HG and SVN)
Co-authored-by: René Pfeuffer <rene.pfeuffer@cloudogu.com>
The change tackles a sporadic error in integration tests, where during or after a change in a git repository a pack file could not be read correctly due to a "short compressed stream". The exception could look like this:
ERROR org.eclipse.jgit.internal.storage.file.ObjectDirectory - Exception caught while accessing pack file scm-home/repositories/8gSNr4ogc5X/data/objects/pack/pack-51a3500283ece83ab8efa7edfb9370e6f97311b5.pack, the pack file might be corrupt. Caught 1 consecutive errors while trying to read this pack.
java.io.EOFException: Short compressed stream at 197
at org.eclipse.jgit.internal.storage.file.PackFile.decompress(PackFile.java:381)
at org.eclipse.jgit.internal.storage.file.PackFile.load(PackFile.java:815)
at org.eclipse.jgit.internal.storage.file.PackFile.get(PackFile.java:284)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedObject(ObjectDirectory.java:455)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedFromSelfOrAlternate(ObjectDirectory.java:413)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:404)
at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:132)
at org.eclipse.jgit.treewalk.CanonicalTreeParser.reset(CanonicalTreeParser.java:191)
at org.eclipse.jgit.treewalk.TreeWalk.parserFor(TreeWalk.java:1344)
at org.eclipse.jgit.treewalk.TreeWalk.addTree(TreeWalk.java:732)
at sonia.scm.repository.spi.Differ.create(Differ.java:102)
at sonia.scm.repository.spi.Differ.diff(Differ.java:63)
We found out that this seems to be linked to the asynchronous execution of post commit hooks, that originally are triggered in the midst of the processing of a receive pack.
With this change the fireing of these triggers is delayed until the end of the internal git processing. The central class for this is the GitHookEventFacade, where hook events are either
- put into a ThreadLocal, so that they can be triggered later, or
- triggered in a thread that is joined with an internal JGit thread for internal pushes (eg. for modify command or merge command)
With this change, work dirs are created in the
directory of the repository and no longer in the
global scm work dir directory. This is relevant due
to two facts:
1. Repositories may contain confidential data and therefore
reside in special directories (that may be mounted on
special drives). It may be considered a breach when these
directories are cloned or otherwise copied to global
temporary drives.
2. Big repositories may overload global temp spaces. It may be
easier to create special drives with more space for such
big repositories.
* Add store exporter to collect the repository metadata
* Add EnvironmentInformationXmlGenerator
* Collect export data and put into compressed tar archive output stream
* Create full repository export endpoint.
* Add full repository export to ui
* Ignore irrelevant files from config store directory
* write metadata stores to file since a baos could teardown the server memory
* Migrate store name for git lfs files (#1504)
Changes the directory name for the git LFS blob store by
removing the repository id from the store name.
This is necessary for im- and exports of lfs blob stores,
because the original name had the repository id as a part
of it and therefore the old store would not be found when
the repository is imported with another id.
Existing blob files will be moved to the new store location
by an update step.
Co-authored-by: Eduard Heimbuch <eduard.heimbuch@cloudogu.com>
* Introduce util for migrations (#1505)
With this util it is more simple to rename
or delete stores.
* Rename files in export
Co-authored-by: René Pfeuffer <rene.pfeuffer@cloudogu.com>
The former code only resolved the object id of the tags, when those were
annotated tags (for "simple" tags the code worked fine). Now we even can
handle nested tags (tags that reference other tags).
