Nemcio/SCM-Manager
1
0
Fork 0
mirror of https://github.com/scm-manager/scm-manager.git synced 2026-02-26 08:20:52 +01:00
Code Issues Packages Projects Releases Activity

Git import with lfs support (#2133)

Browse Source 
This adds the possibility to load files managed by lfs to the repository import of git repositories.

Co-authored-by: Konstantin Schaper <konstantin.schaper@cloudogu.com>
This commit is contained in:
René Pfeuffer
2022-10-25 09:14:40 +02:00
committed by GitHub
parent 96ce4cb8e6
commit 54081ccdc6
33 changed files with 673 additions and 117 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/de/user/repo/assets/import-repository.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 60 KiB

After

Width:  |  Height:  |  Size: 69 KiB

26
docs/de/user/repo/index.md
View File

@@ -37,17 +37,29 @@ neu erstellt werden.
![Repository erstellen](assets/create-repository.png)
### Repository importieren
Neben dem Erstellen von neuen Repository können auch bestehende Repository in SCM-Manager importiert werden.
Neben dem Erstellen von neuen Repository können auch bestehende Repository in den SCM-Manager importiert werden.
Wechseln Sie über den Schalter oben rechts auf die Importseite und füllen Sie die benötigten Informationen aus.
Das gewählte Repository wird zum SCM-Manager hinzugefügt und sämtliche Repository Daten inklusive aller Branches und Tags werden importiert.
Zusätzlich zum normalen Repository Import gibt es die Möglichkeit ein Repository Archiv mit Metadaten zu importieren.
Dieses Repository Archiv muss von einem anderen SCM-Manager exportiert worden sein und wird vor dem Import auf
Kompatibilität der Daten überprüft (der SCM-Manager und alle installierten Plugins müssen mindestens die Version des
exportierenden Systems haben).
Ist die zu importierende Datei verschlüsselt, muss das korrekte Passwort zum Entschlüsseln mitgeliefert werden.
In Abhängigkeit vom Typen des zu importierenden Repositories gibt es verschiedene Möglichkeiten:
- **Import via URL** (nur Git und Mercurial): Hier wird ein Repository von einem anderen Server über die gegebene URL
importiert. Zusätzlich kann ein Benutzername und ein Passwort für die Authentifizierung angegeben werden. Für Git
Repositories kann darüber hinaus der Import von LFS Dateien (falls vorhanden) ausgeschlossen werden.
- **Import aus Dump ohne Metadaten**: Hier kann eine Datei hochgeladen werden. Dieses kann entweder ein einfacher Export
aus einer anderen SCM-Manager Instanz sein oder ein Dump aus einem anderen Repository:
- Für Git und Mercurial muss es das per Tar gepackte "interne" Verzeichnis sein (das `.git` bzw. das `.hg` Verzeichnis).
- Für SVN kann ein per `svnamdin` erstellter Dump genutzt werden.
Wenn diese Dateien per Zip komprimiert sind, muss die Option "Komprimiert" gewählt werden.
- **Import aus SCM-Manager-Dump mit Metadaten**: Mit dieser Option können Exporte mit Metadaten aus anderen SCM-Manager
Instanzen importiert werden. Dieses Repository Archiv wird vor dem Import auf
Kompatibilität der Daten überprüft (der SCM-Manager und alle installierten Plugins müssen mindestens die Version des
exportierenden Systems haben).
Ist die zu importierende Datei verschlüsselt, muss das korrekte Passwort zum Entschlüsseln mitgeliefert werden.
Wird kein Passwort gesetzt, geht der SCM-Manager davon aus, dass die Datei unverschlüsselt ist.
Das gewählte Repository wird zum SCM-Manager hinzugefügt und sämtliche Repository Daten inklusive aller Branches und Tags werden importiert.
![Repository importieren](assets/import-repository.png)
### Repository Informationen

BIN
docs/en/user/repo/assets/import-repository.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 55 KiB

After

Width:  |  Height:  |  Size: 65 KiB

26
docs/en/user/repo/index.md
View File

@@ -33,17 +33,29 @@ be chosen. To create a new namespace, this has to be chosen from the drop down e
![Create Repository](assets/create-repository.png)
### Import a Repository
Beneath creating new repositories you also may import existing repositories to SCM-Manager.
Besided creating new repositories you also may import existing repositories to SCM-Manager.
Just use the Switcher on top right to navigate to the import page and fill the import wizard with the required information.
Your repository will be added to SCM-Manager and all repository data including all branches and tags will be imported.
In addition to the normal repository import, there is the possibility to import a repository archive with metadata.
This repository archive must have been exported from another SCM-Manager and is checked for data compatibility before
import (the SCM-Manager and all its installed plugins have to have at least the versions of the system the export has
been created on).
If the file to be imported is encrypted, the correct password must be supplied for decryption.
Depending on the type of the new repository, there are different "ways" to import repositories:
- **Import via URL** (Git and Mercurial only): Here the repository is imported from a remote server specified by the
remote repository's URL. In addition, a username and a password can be provided. Furthermore, for Git repositories
LFS files can be excluded (if LFS is used in the remote repository).
- **Import from dump without metadata**: Here a file can be uploaded. This may be a simple export from another SCN-Manager
instance, or a dump file from another repository:
- For Git and Mercurial, this has to be a tar packed file of the "internal" repository (the `.git` or the
`.hg` directory).
- For SVN a standard dump file created with `svnadmin` can be used.
If the import file is zip compressed, check the "compressed" option.
- **Import from SCM-Manager dump with metadata**: This option is to be used with exports of repositories with metadata
from other SCM-Manager instances. It will be checked for data compatibility before
import (the SCM-Manager and all its installed plugins have to have at least the versions of the system the export has
been created on).
File dumps can be encrypted if they were exported from an SCM-Manager. In this case, the password has to be specified.
If no password is set, the SCM Manager assumes that the file is unencrypted.
Your repository will be added to SCM-Manager and all repository data including all branches and tags will be imported.
![Import Repository](assets/import-repository.png)
### Repository Information

2
gradle/changelog/lfs_import.yaml Normal file
View File

@@ -0,0 +1,2 @@
- type: added
description: Git import with lfs support ([#2133](https://github.com/scm-manager/scm-manager/pull/2133))

11
scm-core/src/main/java/sonia/scm/notifications/Notification.java
View File

@@ -27,6 +27,7 @@ package sonia.scm.notifications;
import lombok.Value;
import java.time.Instant;
import java.util.Map;
/**
* Notifications can be used to send a message to specific user.
@@ -39,16 +40,22 @@ public class Notification {
Type type;
String link;
String message;
Map<String, String> parameters;
Instant createdAt;
public Notification(Type type, String link, String message) {
this(type, link, message, Instant.now());
this(type, link, message, null);
}
Notification(Type type, String link, String message, Instant createdAt) {
public Notification(Type type, String link, String message, Map<String, String> parameters) {
this(type, link, message, parameters, Instant.now());
}
Notification(Type type, String link, String message, Map<String, String> parameters, Instant createdAt) {
this.type = type;
this.link = link;
this.message = message;
this.parameters = parameters;
this.createdAt = createdAt;
}

13
scm-core/src/main/java/sonia/scm/repository/api/PullCommandBuilder.java
View File

@@ -86,6 +86,19 @@ public final class PullCommandBuilder {
return this;
}
/**
* Set whether to fetch LFS files (<code>true</code>) or not (<code>false</code>).
* This may not work for all repository types.
*
* @param fetchLfs Whether to fetch LFS files or not
* @return this builder instance.
* @since 2.40.0
*/
public PullCommandBuilder doFetchLfs(boolean fetchLfs) {
request.setFetchLfs(fetchLfs);
return this;
}
/**
* Pull all changes from the given remote url.
*

49
scm-core/src/main/java/sonia/scm/repository/api/PullResponse.java
View File

@@ -21,11 +21,13 @@
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.repository.api;
import lombok.Value;
/**
* The {@link PullResponse} is the result of the
* The {@link PullResponse} is the result of the
* {@link PullCommandBuilder#pull(sonia.scm.repository.Repository)} method and
* contains informations over the executed pull command.
*
@@ -35,20 +37,53 @@ package sonia.scm.repository.api;
public final class PullResponse extends AbstractPushOrPullResponse
{
/**
* Constructs a new PullResponse.
*
*/
public PullResponse() {}
private final LfsCount lfsCount;
/**
* Constructs a new PullResponse.
*
*/
public PullResponse() {
this.lfsCount = new LfsCount(0, 0);
}
/**
* Constructs a new PullResponse.
*
* @param changesetCount count of pulled changesets
*/
public PullResponse(long changesetCount)
{
this(changesetCount, new LfsCount(0, 0));
}
/**
* Constructs a new PullResponse.
*
* @param changesetCount count of pulled changesets
* @param lfsCount Object for the count of potentially loaded lfs files
*/
public PullResponse(long changesetCount, LfsCount lfsCount) {
super(changesetCount);
this.lfsCount = lfsCount;
}
/**
* Object for the count of potentially loaded lfs files.
*/
public LfsCount getLfsCount() {
return lfsCount;
}
@Value
public static class LfsCount {
/**
* Count of successfully loaded lfs files.
*/
int successCount;
/**
* Count of failed lfs files.
*/
int failureCount;
}
}

20
scm-core/src/main/java/sonia/scm/repository/spi/PullCommandRequest.java
View File

@@ -21,13 +21,27 @@
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.repository.spi;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.Setter;
import lombok.ToString;
/**
* Request object for {@link PullCommand}.
*
*
* @author Sebastian Sdorra
* @since 1.31
*/
public final class PullCommandRequest extends RemoteCommandRequest {}
@Getter
@Setter
@EqualsAndHashCode(callSuper = true)
@ToString
public final class PullCommandRequest extends RemoteCommandRequest {
/**
* @since 2.40.0
*/
private boolean fetchLfs;
}

28
scm-plugins/scm-git-plugin/src/main/java/sonia/scm/repository/spi/GitMirrorCommand.java
View File

@@ -60,8 +60,8 @@ import sonia.scm.repository.api.MirrorCommandResult.ResultType;
import sonia.scm.repository.api.MirrorFilter;
import sonia.scm.repository.api.MirrorFilter.Result;
import sonia.scm.repository.api.UsernamePasswordCredential;
import sonia.scm.repository.spi.LfsLoader.LfsLoaderLogger;
import sonia.scm.store.ConfigurationStore;
import sonia.scm.web.lfs.LfsBlobStoreFactory;
import javax.inject.Inject;
import java.io.IOException;
@@ -367,7 +367,16 @@ public class GitMirrorCommand extends AbstractGitCommand implements MirrorComman
logger.logChange(ref, referenceName, getUpdateType(ref));
if (!mirrorCommandRequest.isIgnoreLfs()) {
lfsLoader.inspectTree(ref.getNewObjectId(), mirrorCommandRequest, git.getRepository(), mirrorLog, lfsUpdateResult, repository);
LfsLoaderLogger lfsLoaderLogger = new MirrorLfsLoaderLogger();
lfsLoader.inspectTree(
ref.getNewObjectId(),
git.getRepository(),
lfsLoaderLogger,
lfsUpdateResult,
repository,
mirrorHttpConnectionProvider.createHttpConnectionFactory(mirrorCommandRequest, mirrorLog),
mirrorCommandRequest.getSourceUrl()
);
}
}
}
@@ -406,6 +415,19 @@ public class GitMirrorCommand extends AbstractGitCommand implements MirrorComman
private String getUpdateType(TrackingRefUpdate trackingRefUpdate) {
return trackingRefUpdate.getResult().name().toLowerCase(Locale.ENGLISH);
}
private class MirrorLfsLoaderLogger implements LfsLoaderLogger {
@Override
public void failed(Exception e) {
mirrorLog.add("Failed to load lfs file:");
mirrorLog.add(e.getMessage());
}
@Override
public void loading(String name) {
mirrorLog.add(String.format("Loading lfs file with id '%s'", name));
}
}
}
private class LoggerWithHeader {
@@ -653,7 +675,7 @@ public class GitMirrorCommand extends AbstractGitCommand implements MirrorComman
}
private boolean isOfTypeOrEmpty(Optional<MirrorFilter.UpdateType> updateType, MirrorFilter.UpdateType type) {
return !updateType.isPresent() || updateType.get() == type;
return updateType.isEmpty() || updateType.get() == type;
}
private Optional<MirrorFilter.UpdateType> getUpdateTypeFor(ReceiveCommand receiveCommand) {

58
scm-plugins/scm-git-plugin/src/main/java/sonia/scm/repository/spi/GitPullCommand.java
View File

@@ -41,6 +41,7 @@ import sonia.scm.repository.GitRepositoryHandler;
import sonia.scm.repository.GitUtil;
import sonia.scm.repository.Repository;
import sonia.scm.repository.api.ImportFailedException;
import sonia.scm.repository.api.MirrorCommandResult;
import sonia.scm.repository.api.PullResponse;
import javax.inject.Inject;
@@ -54,14 +55,21 @@ public class GitPullCommand extends AbstractGitPushOrPullCommand
implements PullCommand {
private static final Logger LOG = LoggerFactory.getLogger(GitPullCommand.class);
private final PostReceiveRepositoryHookEventFactory postReceiveRepositoryHookEventFactory;
private final LfsLoader lfsLoader;
private final PullHttpConnectionProvider pullHttpConnectionProvider;
@Inject
public GitPullCommand(GitRepositoryHandler handler,
GitContext context,
PostReceiveRepositoryHookEventFactory postReceiveRepositoryHookEventFactory) {
PostReceiveRepositoryHookEventFactory postReceiveRepositoryHookEventFactory,
LfsLoader lfsLoader,
PullHttpConnectionProvider pullHttpConnectionProvider) {
super(handler, context);
this.postReceiveRepositoryHookEventFactory = postReceiveRepositoryHookEventFactory;
this.lfsLoader = lfsLoader;
this.pullHttpConnectionProvider = pullHttpConnectionProvider;
}
@Override
@@ -81,7 +89,7 @@ public class GitPullCommand extends AbstractGitPushOrPullCommand
return response;
}
private PullResponse convert(Git git, FetchResult fetch) {
private PullResponse convert(Git git, FetchResult fetch, CountingLfsLoaderLogger lfsLoaderLogger) {
long counter = 0;
for (TrackingRefUpdate tru : fetch.getTrackingRefUpdates()) {
@@ -90,7 +98,7 @@ public class GitPullCommand extends AbstractGitPushOrPullCommand
LOG.debug("received {} changesets by pull", counter);
return new PullResponse(counter);
return new PullResponse(counter, new PullResponse.LfsCount(lfsLoaderLogger.getSuccessCount(), lfsLoaderLogger.getFailureCount()));
}
private long count(Git git, TrackingRefUpdate tru) {
@@ -178,7 +186,11 @@ public class GitPullCommand extends AbstractGitPushOrPullCommand
.call();
//J+
response = convert(git, result);
CountingLfsLoaderLogger lfsLoaderLogger = new CountingLfsLoaderLogger();
if (request.isFetchLfs()) {
fetchLfs(request, git, lfsLoaderLogger);
}
response = convert(git, result, lfsLoaderLogger);
} catch
(GitAPIException ex) {
throw new ImportFailedException(
@@ -193,7 +205,45 @@ public class GitPullCommand extends AbstractGitPushOrPullCommand
return response;
}
private void fetchLfs(PullCommandRequest request, Git git, LfsLoader.LfsLoaderLogger lfsLoaderLogger) throws IOException {
open().getRefDatabase().getRefs().forEach(
ref -> lfsLoader.inspectTree(
ref.getObjectId(),
git.getRepository(),
lfsLoaderLogger,
new MirrorCommandResult.LfsUpdateResult(),
repository,
pullHttpConnectionProvider.createHttpConnectionFactory(request),
request.getRemoteUrl().toString()
)
);
}
private void firePostReceiveRepositoryHookEvent(Git git, FetchResult result) {
postReceiveRepositoryHookEventFactory.fireForFetch(git, result);
}
private static class CountingLfsLoaderLogger implements LfsLoader.LfsLoaderLogger {
private int successCount = 0;
private int failureCount = 0;
@Override
public void failed(Exception e) {
++failureCount;
}
@Override
public void loading(String name) {
++successCount;
}
public int getSuccessCount() {
return successCount;
}
public int getFailureCount() {
return failureCount;
}
}
}

58
scm-plugins/scm-git-plugin/src/main/java/sonia/scm/repository/spi/LfsLoader.java
View File

@@ -50,34 +50,39 @@ import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Collection;
import java.util.List;
class LfsLoader {
private static final Logger LOG = LoggerFactory.getLogger(LfsLoader.class);
private final LfsBlobStoreFactory lfsBlobStoreFactory;
private final MirrorHttpConnectionProvider mirrorHttpConnectionProvider;
@Inject
LfsLoader(LfsBlobStoreFactory lfsBlobStoreFactory, MirrorHttpConnectionProvider mirrorHttpConnectionProvider) {
LfsLoader(LfsBlobStoreFactory lfsBlobStoreFactory) {
this.lfsBlobStoreFactory = lfsBlobStoreFactory;
this.mirrorHttpConnectionProvider = mirrorHttpConnectionProvider;
}
void inspectTree(ObjectId newObjectId,
MirrorCommandRequest mirrorCommandRequest,
Repository gitRepository,
List<String> mirrorLog,
LfsLoaderLogger mirrorLog,
LfsUpdateResult lfsUpdateResult,
sonia.scm.repository.Repository repository) {
String mirrorUrl = mirrorCommandRequest.getSourceUrl();
EntryHandler entryHandler = new EntryHandler(repository, gitRepository, mirrorCommandRequest, mirrorLog, lfsUpdateResult);
sonia.scm.repository.Repository repository,
HttpConnectionFactory httpConnectionFactory,
String url) {
EntryHandler entryHandler = new EntryHandler(repository, gitRepository, mirrorLog, lfsUpdateResult, httpConnectionFactory);
inspectTree(newObjectId, entryHandler, gitRepository, mirrorLog, lfsUpdateResult, url);
}
private void inspectTree(ObjectId newObjectId,
EntryHandler entryHandler,
Repository gitRepository,
LfsLoaderLogger mirrorLog,
LfsUpdateResult lfsUpdateResult,
String sourceUrl) {
try {
gitRepository
.getConfig()
.setString(ConfigConstants.CONFIG_SECTION_LFS, null, ConfigConstants.CONFIG_KEY_URL, computeLfsUrl(mirrorUrl));
.setString(ConfigConstants.CONFIG_SECTION_LFS, null, ConfigConstants.CONFIG_KEY_URL, computeLfsUrl(sourceUrl));
TreeWalk treeWalk = new TreeWalk(gitRepository);
treeWalk.setFilter(new LfsPointerFilter());
@@ -94,17 +99,16 @@ class LfsLoader {
}
} catch (Exception e) {
LOG.warn("failed to load lfs files", e);
mirrorLog.add("Failed to load lfs files:");
mirrorLog.add(e.getMessage());
mirrorLog.failed(e);
lfsUpdateResult.increaseFailureCount();
}
}
private String computeLfsUrl(String mirrorUrl) {
if (mirrorUrl.endsWith(".git")) {
return mirrorUrl + Protocol.INFO_LFS_ENDPOINT;
private String computeLfsUrl(String sourceUrl) {
if (sourceUrl.endsWith(".git")) {
return sourceUrl + Protocol.INFO_LFS_ENDPOINT;
} else {
return mirrorUrl + ".git" + Protocol.INFO_LFS_ENDPOINT;
return sourceUrl + ".git" + Protocol.INFO_LFS_ENDPOINT;
}
}
@@ -112,22 +116,22 @@ class LfsLoader {
private final BlobStore lfsBlobStore;
private final Repository gitRepository;
private final List<String> mirrorLog;
private final LfsLoaderLogger mirrorLog;
private final LfsUpdateResult lfsUpdateResult;
private final sonia.scm.repository.Repository repository;
private final HttpConnectionFactory httpConnectionFactory;
private EntryHandler(sonia.scm.repository.Repository repository,
Repository gitRepository,
MirrorCommandRequest mirrorCommandRequest,
List<String> mirrorLog,
LfsUpdateResult lfsUpdateResult) {
LfsLoaderLogger mirrorLog,
LfsUpdateResult lfsUpdateResult,
HttpConnectionFactory httpConnectionFactory) {
this.lfsBlobStore = lfsBlobStoreFactory.getLfsBlobStore(repository);
this.repository = repository;
this.gitRepository = gitRepository;
this.mirrorLog = mirrorLog;
this.lfsUpdateResult = lfsUpdateResult;
this.httpConnectionFactory = mirrorHttpConnectionProvider.createHttpConnectionFactory(mirrorCommandRequest, mirrorLog);
this.httpConnectionFactory = httpConnectionFactory;
}
private void handleTreeEntry(TreeWalk treeWalk) {
@@ -142,8 +146,7 @@ class LfsLoader {
}
} catch (Exception e) {
LOG.warn("failed to load lfs file", e);
mirrorLog.add("Failed to load lfs file:");
mirrorLog.add(e.getMessage());
mirrorLog.failed(e);
lfsUpdateResult.increaseFailureCount();
}
}
@@ -151,7 +154,7 @@ class LfsLoader {
private Path loadLfsFile(LfsPointer lfsPointer) throws IOException {
lfsUpdateResult.increaseOverallCount();
LOG.trace("trying to load lfs file '{}' for repository {}", lfsPointer.getOid(), repository);
mirrorLog.add(String.format("Loading lfs file with id '%s'", lfsPointer.getOid().name()));
mirrorLog.loading(lfsPointer.getOid().name());
Lfs lfs = new Lfs(gitRepository);
lfs.getMediaFile(lfsPointer.getOid());
@@ -174,4 +177,11 @@ class LfsLoader {
);
}
}
interface LfsLoaderLogger {
void failed(Exception e);
void loading(String name);
}
}

1
scm-plugins/scm-git-plugin/src/main/java/sonia/scm/repository/spi/MirrorHttpConnectionProvider.java
View File

@@ -24,7 +24,6 @@
package sonia.scm.repository.spi;
import org.eclipse.jgit.transport.UserAgent;
import org.eclipse.jgit.transport.http.HttpConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

55
scm-plugins/scm-git-plugin/src/main/java/sonia/scm/repository/spi/PullHttpConnectionProvider.java Normal file
View File

@@ -0,0 +1,55 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.repository.spi;
import com.google.common.base.Strings;
import org.eclipse.jgit.transport.http.HttpConnectionFactory;
import sonia.scm.net.HttpConnectionOptions;
import sonia.scm.net.HttpURLConnectionFactory;
import sonia.scm.web.ScmHttpConnectionFactory;
import javax.inject.Inject;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
class PullHttpConnectionProvider {
private final HttpURLConnectionFactory httpURLConnectionFactory;
@Inject
PullHttpConnectionProvider(HttpURLConnectionFactory httpURLConnectionFactory) {
this.httpURLConnectionFactory = httpURLConnectionFactory;
}
HttpConnectionFactory createHttpConnectionFactory(PullCommandRequest request) {
HttpConnectionOptions options = new HttpConnectionOptions();
if (!Strings.isNullOrEmpty(request.getUsername()) && !Strings.isNullOrEmpty(request.getPassword())) {
String encodedAuth = Base64.getEncoder().encodeToString((request.getUsername() + ":" + request.getPassword()).getBytes(StandardCharsets.UTF_8));
String authHeaderValue = "Basic " + encodedAuth;
options.addRequestProperty("Authorization", authHeaderValue);
}
return new ScmHttpConnectionFactory(httpURLConnectionFactory, options);
}
}

8
scm-plugins/scm-git-plugin/src/test/java/sonia/scm/repository/spi/GitIncomingCommandTest.java
View File

@@ -40,6 +40,7 @@ import java.io.IOException;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.mockito.Mockito.mock;
//~--- JDK imports ------------------------------------------------------------
@@ -49,6 +50,9 @@ import static org.junit.Assert.assertNotNull;
public class GitIncomingCommandTest
extends AbstractRemoteCommandTestBase {
private final LfsLoader lfsLoader = mock(LfsLoader.class);
private final PullHttpConnectionProvider pullHttpConnectionProvider = mock(PullHttpConnectionProvider.class);
/**
* Method description
*
@@ -99,7 +103,9 @@ public class GitIncomingCommandTest
GitPullCommand pull = new GitPullCommand(
handler,
context,
postReceiveRepositoryHookEventFactory);
postReceiveRepositoryHookEventFactory,
lfsLoader,
pullHttpConnectionProvider);
PullCommandRequest req = new PullCommandRequest();
req.setRemoteRepository(outgoingRepository);
pull.pull(req);

11
scm-plugins/scm-git-plugin/src/test/java/sonia/scm/repository/spi/GitMirrorCommandTest.java
View File

@@ -858,18 +858,17 @@ public class GitMirrorCommandTest extends AbstractGitCommandTestBase {
// one revision is missing here ("fcd0ef1831e4002ac43ea539f4094334c79ea9ec"), because this is iterated twice, what is hard to test
}).forEach(expectedRevision ->
verify(lfsLoader)
.inspectTree(eq(ObjectId.fromString(expectedRevision)), any(), any(), any(), any(), eq(repository)));
.inspectTree(eq(ObjectId.fromString(expectedRevision)), any(), any(), any(), eq(repository), any(), any()));
}
@Test
public void shouldMarkMirrorAsFailedIfLfsFileFailes() {
public void shouldMarkMirrorAsFailedIfLfsFileFails() {
doAnswer(invocation -> {
invocation.getArgument(4, MirrorCommandResult.LfsUpdateResult.class).increaseFailureCount();
invocation.getArgument(3, MirrorCommandResult.LfsUpdateResult.class).increaseFailureCount();
return null;
})
.when(lfsLoader)
.inspectTree(eq(ObjectId.fromString("a8495c0335a13e6e432df90b3727fa91943189a7")), any(), any(), any(), any(), eq(repository));
.inspectTree(eq(ObjectId.fromString("a8495c0335a13e6e432df90b3727fa91943189a7")), any(), any(), any(), eq(repository), any(), any());
MirrorCommandResult mirrorCommandResult = callMirrorCommand();
@@ -881,7 +880,7 @@ public class GitMirrorCommandTest extends AbstractGitCommandTestBase {
callMirrorCommand(repositoryDirectory.getAbsolutePath(), c -> c.setIgnoreLfs(true));
verify(lfsLoader, never())
.inspectTree(any(), any(), any(), any(), any(), any());
.inspectTree(any(), any(), any(), any(), any(), any(), any());
}
public static class DefaultBranchSelectorTest {

10
scm-plugins/scm-git-plugin/src/test/java/sonia/scm/repository/spi/GitModificationsCommandTest.java
View File

@@ -30,6 +30,7 @@ import org.eclipse.jgit.revwalk.RevCommit;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.MockitoJUnitRunner;
import sonia.scm.repository.GitConfig;
@@ -49,6 +50,11 @@ public class GitModificationsCommandTest extends AbstractRemoteCommandTestBase {
private GitModificationsCommand incomingModificationsCommand;
private GitModificationsCommand outgoingModificationsCommand;
@Mock
private LfsLoader lfsLoader;
@Mock
private PullHttpConnectionProvider pullHttpConnectionProvider;
@Before
public void init() {
incomingModificationsCommand = new GitModificationsCommand(Mockito.spy(new GitContext(incomingDirectory, incomingRepository, null, new GitConfig())));
@@ -171,7 +177,9 @@ public class GitModificationsCommandTest extends AbstractRemoteCommandTestBase {
GitPullCommand pullCommand = new GitPullCommand(
handler,
context,
postReceiveRepositoryHookEventFactory);
postReceiveRepositoryHookEventFactory,
lfsLoader,
pullHttpConnectionProvider);
PullCommandRequest pullRequest = new PullCommandRequest();
pullRequest.setRemoteRepository(incomingRepository);
pullCommand.pull(pullRequest);

1
scm-ui/ui-types/src/Notifications.ts
View File

@@ -29,6 +29,7 @@ export type Notification = HalRepresentation & {
type: "INFO" | "SUCCESS" | "WARNING" | "ERROR";
link: string;
message: string;
parameters?: Record<string, string>;
};
type EmbeddedNotifications = {

1
scm-ui/ui-types/src/Repositories.ts
View File

@@ -60,6 +60,7 @@ export type RepositoryUrlImport = RepositoryCreation & {
importUrl: string;
username?: string;
password?: string;
skipLfs?: boolean;
};
export type ExportInfo = HalRepresentation & {

6
scm-ui/ui-webapp/public/locales/de/repos.json
View File

@@ -30,7 +30,8 @@
"initializeRepository": "Erstellt einen ersten Branch und committet eine README.md.",
"importUrlHelpText": "Importiert das gesamte Repository inkl. aller Branches und Tags über die Remote URL.",
"usernameHelpText": "Benutzername könnte für den Import benötigt werden. Wird ignoriert, falls nicht gesetzt.",
"passwordHelpText": "Password könnte für den Import benötigt werden. Wird ignoriert, falls nicht gesetzt."
"passwordHelpText": "Passwort könnte für den Import benötigt werden. Wird ignoriert, falls nicht gesetzt.",
"skipLfsHelpText": "Ist diese Option aktiviert, werden beim Import keine (potentiell vorhandenen) LFS-Dateien in den SCM-Manager geladen. Diese Option ist nur für Git Repositories relevant."
},
"repositoryRoot": {
"errorTitle": "Fehler",
@@ -115,7 +116,8 @@
"helpText": "Der Dump muss von einer SCM-Manager-Instanz mit Metadaten erstellt worden sein."
}
},
"navigationWarning": "Es wird dringend empfohlen auf dieser Seite zu warten bis der Import abgeschlossen ist. Beim Verlassen der Seite könnte der Import abbrechen. Seite trotzdem verlassen?"
"navigationWarning": "Es wird dringend empfohlen auf dieser Seite zu warten bis der Import abgeschlossen ist. Beim Verlassen der Seite könnte der Import abbrechen. Seite trotzdem verlassen?",
"skipLfs": "Keine LFS Dateien laden (nur für Git Repositories)"
},
"branches": {
"overview": {

6
scm-ui/ui-webapp/public/locales/en/repos.json
View File

@@ -30,7 +30,8 @@
"initializeRepository": "Creates an initial branch and commits a basic README.md.",
"importUrlHelpText": "Import the whole repository including all branches and tags via remote url",
"usernameHelpText": "Username may be required to import the remote repository. Will be ignored if not provided.",
"passwordHelpText": "Password may be required to import the remote repository. Will be ignored if not provided."
"passwordHelpText": "Password may be required to import the remote repository. Will be ignored if not provided.",
"skipLfsHelpText": "Check this if (potentially available) LFS files shall not be loaded into SCM-Manager during the import. This option is relevant only for git repositories."
},
"repositoryRoot": {
"errorTitle": "Error",
@@ -115,7 +116,8 @@
"helpText": "The dump containing the data must be generated by a SCM-Manager instance and include specific metadata."
}
},
"navigationWarning": "It is strongly recommend that you stay on this page until the import is finished. Leaving this page could abort your import. Leave page anyway?"
"navigationWarning": "It is strongly recommend that you stay on this page until the import is finished. Leaving this page could abort your import. Leave page anyway?",
"skipLfs": "Do not load LFS files (git only)"
},
"branches": {
"overview": {

26
scm-ui/ui-webapp/src/containers/Notifications.tsx
View File

@@ -71,8 +71,8 @@ const NotificationEntry: FC<EntryProps> = ({ notification, removeToast }) => {
}
return (
<tr className={`is-${color(notification)}`}>
<Column onClick={() => history.push(notification.link)} className="is-clickable">
<NotificationMessage message={notification.message} />
<Column onClick={notification.link ? () => history.push(notification.link) : undefined} className="is-clickable">
<NotificationMessage message={notification.message} parameters={notification.parameters} />
</Column>
<OnlyMobileWrappingColumn className="has-text-right">
<DateFromNow date={notification.createdAt} />
@@ -175,9 +175,13 @@ const color = (notification: Notification) => {
return c;
};
const NotificationMessage: FC<{ message: string }> = ({ message }) => {
const NotificationMessage: FC<{ message: string; parameters?: Record<string, string> }> = ({ message, parameters }) => {
const [t] = useTranslation("plugins");
return t("notifications." + message, message);
if (parameters) {
return t("notifications." + message, message, parameters);
} else {
return t("notifications." + message, message);
}
};
type SubscriptionProps = {
@@ -185,6 +189,14 @@ type SubscriptionProps = {
remove: (notification: Notification) => void;
};
const PotentialLink: FC<{ link?: string }> = ({ link, children }) => {
if (link) {
return <Link to={link}>{children}</Link>;
} else {
return <>{children}</>;
}
};
const NotificationSubscription: FC<SubscriptionProps> = ({ notifications, remove }) => {
const [t] = useTranslation("commons");
@@ -200,9 +212,9 @@ const NotificationSubscription: FC<SubscriptionProps> = ({ notifications, remove
close={() => remove(notification)}
>
<p>
<Link to={notification.link}>
<NotificationMessage message={notification.message} />
</Link>
<PotentialLink link={notification.link}>
<NotificationMessage message={notification.message} parameters={notification.parameters} />
</PotentialLink>
</p>
</ToastNotification>
))}

11
scm-ui/ui-webapp/src/repos/components/ImportFromUrlForm.tsx
View File

@@ -24,7 +24,7 @@
import React, { FC, useState } from "react";
import { useTranslation } from "react-i18next";
import { RepositoryUrlImport } from "@scm-manager/ui-types";
import { InputField, validation } from "@scm-manager/ui-components";
import { Checkbox, InputField, validation } from "@scm-manager/ui-components";
type Props = {
repository: RepositoryUrlImport;
@@ -87,6 +87,15 @@ const ImportFromUrlForm: FC<Props> = ({ repository, onChange, setValid, disabled
disabled={disabled}
/>
</div>
<div className="column is-full px-3">
<Checkbox
label={t("import.skipLfs")}
onChange={skipLfs => onChange({ ...repository, skipLfs })}
checked={repository.skipLfs}
helpText={t("help.skipLfsHelpText")}
disabled={disabled}
/>
</div>
</div>
);
};

3
scm-webapp/src/main/java/sonia/scm/api/v2/resources/NotificationDto.java
View File

@@ -31,6 +31,7 @@ import sonia.scm.notifications.StoredNotification;
import sonia.scm.notifications.Type;
import java.time.Instant;
import java.util.Map;
@Data
public class NotificationDto extends HalRepresentation {
@@ -39,12 +40,14 @@ public class NotificationDto extends HalRepresentation {
private Type type;
private String link;
private String message;
private Map<String, String> parameters;
public NotificationDto(StoredNotification notification, Links links) {
super(links);
this.type = notification.getType();
this.link = notification.getLink();
this.message = notification.getMessage();
this.parameters = notification.getParameters();
this.createdAt = notification.getCreatedAt();
}
}

3
scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryImportResource.java
View File

@@ -383,6 +383,7 @@ public class RepositoryImportResource {
private String importUrl;
private String username;
private String password;
private boolean skipLfs;
}
@Getter
@@ -400,6 +401,8 @@ public class RepositoryImportResource {
String getUsername();
String getPassword();
boolean isSkipLfs();
}
interface ImportRepositoryFromFileDto extends CreateRepositoryDto {

21
scm-webapp/src/main/java/sonia/scm/importexport/FromUrlImporter.java
View File

@@ -42,6 +42,7 @@ import sonia.scm.repository.RepositoryType;
import sonia.scm.repository.api.Command;
import sonia.scm.repository.api.ImportFailedException;
import sonia.scm.repository.api.PullCommandBuilder;
import sonia.scm.repository.api.PullResponse;
import sonia.scm.repository.api.RepositoryService;
import sonia.scm.repository.api.RepositoryServiceFactory;
@@ -63,13 +64,15 @@ public class FromUrlImporter {
private final RepositoryServiceFactory serviceFactory;
private final ScmEventBus eventBus;
private final RepositoryImportLoggerFactory loggerFactory;
private final ImportNotificationHandler notificationHandler;
@Inject
public FromUrlImporter(RepositoryManager manager, RepositoryServiceFactory serviceFactory, ScmEventBus eventBus, RepositoryImportLoggerFactory loggerFactory) {
public FromUrlImporter(RepositoryManager manager, RepositoryServiceFactory serviceFactory, ScmEventBus eventBus, RepositoryImportLoggerFactory loggerFactory, ImportNotificationHandler notificationHandler) {
this.manager = manager;
this.serviceFactory = serviceFactory;
this.eventBus = eventBus;
this.loggerFactory = loggerFactory;
this.notificationHandler = notificationHandler;
}
public Repository importFromUrl(RepositoryImportParameters parameters, Repository repository) {
@@ -112,21 +115,35 @@ public class FromUrlImporter {
.withUsername(parameters.getUsername())
.withPassword(parameters.getPassword());
}
pullCommand.doFetchLfs(!parameters.isSkipLfs());
logger.step("pulling repository from " + parameters.getImportUrl());
pullCommand.pull(parameters.getImportUrl());
PullResponse pullResponse = pullCommand.pull(parameters.getImportUrl());
logger.finished();
handle(pullResponse, repository);
} catch (IOException e) {
throw new InternalRepositoryException(repository, "Failed to import from remote url: " + e.getMessage(), e);
} catch (ImportFailedException e) {
notificationHandler.handleFailedImport();
throw e;
}
};
}
private void handle(PullResponse pullResponse, Repository repository) {
if (pullResponse.getLfsCount().getFailureCount() == 0) {
notificationHandler.handleSuccessfulImport(repository, pullResponse.getLfsCount());
} else {
notificationHandler.handleSuccessfulImportWithLfsFailures(repository, pullResponse.getLfsCount());
}
}
@Getter
@Setter
public static class RepositoryImportParameters {
private String importUrl;
private String username;
private String password;
private boolean skipLfs;
}
}

88
scm-webapp/src/main/java/sonia/scm/importexport/ImportNotificationHandler.java Normal file
View File

@@ -0,0 +1,88 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import sonia.scm.notifications.Notification;
import sonia.scm.notifications.NotificationSender;
import sonia.scm.notifications.Type;
import sonia.scm.repository.Repository;
import sonia.scm.repository.api.PullResponse;
import javax.inject.Inject;
import java.util.Map;
class ImportNotificationHandler {
private final NotificationSender notificationSender;
@Inject
ImportNotificationHandler(NotificationSender notificationSender) {
this.notificationSender = notificationSender;
}
void handleSuccessfulImport(Repository repository) {
handleSuccessfulImport(repository, new PullResponse.LfsCount(0, 0));
}
void handleSuccessfulImport(Repository repository, PullResponse.LfsCount lfsCount) {
notificationSender.send(getImportSuccessfulNotification(repository, lfsCount));
}
void handleSuccessfulImportWithLfsFailures(Repository repository, PullResponse.LfsCount lfsCount) {
notificationSender.send(getImportLfsFailedNotification(repository, lfsCount));
}
void handleFailedImport() {
notificationSender.send(getImportFailedNotification());
}
private Notification getImportSuccessfulNotification(Repository repository, PullResponse.LfsCount lfsCount) {
if (lfsCount.getSuccessCount() > 0 || lfsCount.getFailureCount() > 0) {
return new Notification(Type.SUCCESS, createLink(repository), "importWithLfsFinished", createParameters(lfsCount));
} else {
return new Notification(Type.SUCCESS, createLink(repository), "importFinished");
}
}
private Notification getImportLfsFailedNotification(Repository repository, PullResponse.LfsCount lfsCount) {
return new Notification(Type.ERROR, createLink(repository), "importLfsFailed", createParameters(lfsCount));
}
private static Map<String, String> createParameters(PullResponse.LfsCount lfsCount) {
return Map.of(
"successCount", Integer.toString(lfsCount.getSuccessCount()),
"failureCount", Integer.toString(lfsCount.getFailureCount()),
"overallCount", Integer.toString(lfsCount.getSuccessCount() + lfsCount.getFailureCount())
);
}
private Notification getImportFailedNotification() {
return new Notification(Type.ERROR, null, "importFailed");
}
private static String createLink(Repository repository) {
return "/repo/" + repository.getNamespaceAndName();
}
}

3
scm-webapp/src/main/java/sonia/scm/notifications/StoredNotification.java
View File

@@ -33,6 +33,7 @@ import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import java.time.Instant;
import java.util.Map;
@Data
@NoArgsConstructor
@@ -44,6 +45,7 @@ public class StoredNotification {
Type type;
String link;
String message;
Map<String, String> parameters;
@XmlJavaTypeAdapter(XmlInstantAdapter.class)
Instant createdAt;
@@ -53,5 +55,6 @@ public class StoredNotification {
this.type = notification.getType();
this.link = notification.getLink();
this.message = notification.getMessage();
this.parameters = notification.getParameters();
}
}

4
scm-webapp/src/main/resources/locales/de/plugins.json
View File

@@ -480,6 +480,10 @@
"notifications": {
"exportFinished": "Der Repository Export wurde abgeschlossen.",
"exportFailed": "Der Repository Export ist fehlgeschlagen. Versuchen Sie es erneut oder wenden Sie sich an einen Administrator.",
"importFinished": "Der Repository Import wurde abgeschlossen.",
"importWithLfsFinished": "Der Repository Import wurde abgeschlossen. Dabei wurden {{successCount}} LFS Dateien importiert.",
"importLfsFailed": "Der Repository Import wurde abgeschlossen. Jedoch konnten {{failureCount}} von {{overallCount}} LFS Dateien nicht geladen werden.",
"importFailed": "Der Repository Import ist fehlgeschlagen. Versuchen Sie es erneut oder wenden Sie sich an einen Administrator.",
"healthCheckFailed": "Der Repository Health Check ist fehlgeschlagen.",
"healthCheckSuccess": "Der Repository Health Check war erfolgreich."
},

4
scm-webapp/src/main/resources/locales/en/plugins.json
View File

@@ -424,6 +424,10 @@
"notifications": {
"exportFinished": "The repository export has been finished.",
"exportFailed": "The repository export has failed. Try it again or contact your administrator.",
"importFinished": "The repository import has been finished.",
"importWithLfsFinished": "The repository import has been finished, including {{successCount}} LFS files.",
"importLfsFailed": "The repository import has been finished. There were {{failureCount}} errors during the import of {{overallCount}} LFS files.",
"importFailed": "The repository import has failed. Try it again or contact your administrator.",
"healthCheckFailed": "The repository health check has failed.",
"healthCheckSuccess": "The repository health check was successful."
},

4
scm-webapp/src/test/java/sonia/scm/api/v2/resources/NotificationResourceTest.java
View File

@@ -33,7 +33,6 @@ import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import sonia.scm.notifications.Notification;
import sonia.scm.notifications.NotificationStore;
import sonia.scm.notifications.StoredNotification;
import sonia.scm.notifications.Type;
@@ -50,6 +49,7 @@ import java.util.List;
import java.util.UUID;
import java.util.stream.Collectors;
import static java.util.Collections.emptyMap;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@@ -162,7 +162,7 @@ class NotificationResourceTest {
}
private StoredNotification notification(String m) {
return new StoredNotification(UUID.randomUUID().toString(), Type.INFO, "/notify", m, Instant.now());
return new StoredNotification(UUID.randomUUID().toString(), Type.INFO, "/notify", m, emptyMap(), Instant.now());
}
@Path("/api/v2")

123
scm-webapp/src/test/java/sonia/scm/importexport/FromUrlImporterTest.java
View File

@@ -28,8 +28,10 @@ import org.apache.shiro.subject.Subject;
import org.apache.shiro.util.ThreadContext;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Answers;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
@@ -42,6 +44,7 @@ import sonia.scm.repository.RepositoryTestData;
import sonia.scm.repository.RepositoryType;
import sonia.scm.repository.api.ImportFailedException;
import sonia.scm.repository.api.PullCommandBuilder;
import sonia.scm.repository.api.PullResponse;
import sonia.scm.repository.api.RepositoryService;
import sonia.scm.repository.api.RepositoryServiceFactory;
@@ -54,7 +57,7 @@ import static org.junit.Assert.assertThrows;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.argThat;
import static org.mockito.Mockito.RETURNS_SELF;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
@@ -78,11 +81,18 @@ class FromUrlImporterTest {
private RepositoryImportLogger logger;
@Mock
private Subject subject;
@Mock(answer = Answers.RETURNS_SELF)
private PullCommandBuilder pullCommandBuilder;
@Mock
private ImportNotificationHandler notificationHandler;
@InjectMocks
private FromUrlImporter importer;
private final Repository repository = RepositoryTestData.createHeartOfGold("git");
private Repository createdRepository;
private PullResponse mockedResponse = new PullResponse();
@BeforeEach
void setUpMocks() {
@@ -91,12 +101,13 @@ class FromUrlImporterTest {
when(manager.create(any(), any())).thenAnswer(
invocation -> {
Repository repository = invocation.getArgument(0, Repository.class);
Repository createdRepository = repository.clone();
createdRepository = repository.clone();
createdRepository.setNamespace("created");
invocation.getArgument(1, Consumer.class).accept(createdRepository);
return createdRepository;
}
);
when(service.getPullCommand()).thenReturn(pullCommandBuilder);
}
@BeforeEach
@@ -119,50 +130,92 @@ class FromUrlImporterTest {
ThreadContext.unbindSubject();
}
@Test
void shouldPullChangesFromRemoteUrl() throws IOException {
PullCommandBuilder pullCommandBuilder = mock(PullCommandBuilder.class, RETURNS_SELF);
when(service.getPullCommand()).thenReturn(pullCommandBuilder);
@Nested
class ForSuccessfulImports {
FromUrlImporter.RepositoryImportParameters parameters = new FromUrlImporter.RepositoryImportParameters();
parameters.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
@BeforeEach
void mockImportResult() throws IOException {
when(pullCommandBuilder.pull(anyString())).thenAnswer(invocation -> mockedResponse);
}
Repository createdRepository = importer.importFromUrl(parameters, repository);
@Test
void shouldPullChangesFromRemoteUrl() throws IOException {
FromUrlImporter.RepositoryImportParameters parameters = new FromUrlImporter.RepositoryImportParameters();
parameters.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
assertThat(createdRepository.getNamespace()).isEqualTo("created");
verify(pullCommandBuilder).pull("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
verify(logger).finished();
verify(eventBus).post(argThat(
event -> {
assertThat(event).isInstanceOf(RepositoryImportEvent.class);
RepositoryImportEvent repositoryImportEvent = (RepositoryImportEvent) event;
assertThat(repositoryImportEvent.getItem().getNamespace()).isEqualTo("created");
assertThat(repositoryImportEvent.isFailed()).isFalse();
return true;
}
));
}
Repository createdRepository = importer.importFromUrl(parameters, repository);
@Test
void shouldPullChangesFromRemoteUrlWithCredentials() {
PullCommandBuilder pullCommandBuilder = mock(PullCommandBuilder.class, RETURNS_SELF);
when(service.getPullCommand()).thenReturn(pullCommandBuilder);
assertThat(createdRepository.getNamespace()).isEqualTo("created");
verify(pullCommandBuilder).pull("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
verify(logger).finished();
verify(eventBus).post(argThat(
event -> {
assertThat(event).isInstanceOf(RepositoryImportEvent.class);
RepositoryImportEvent repositoryImportEvent = (RepositoryImportEvent) event;
assertThat(repositoryImportEvent.getItem().getNamespace()).isEqualTo("created");
assertThat(repositoryImportEvent.isFailed()).isFalse();
return true;
}
));
verify(notificationHandler)
.handleSuccessfulImport(
eq(createdRepository),
argThat(argument -> argument.getSuccessCount() == 0));
}
FromUrlImporter.RepositoryImportParameters parameters = new FromUrlImporter.RepositoryImportParameters();
parameters.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
parameters.setUsername("trillian");
parameters.setPassword("secret");
@Test
void shouldPullChangesFromRemoteUrlWithCredentials() {
FromUrlImporter.RepositoryImportParameters parameters = new FromUrlImporter.RepositoryImportParameters();
parameters.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
parameters.setUsername("trillian");
parameters.setPassword("secret");
importer.importFromUrl(parameters, repository);
importer.importFromUrl(parameters, repository);
verify(pullCommandBuilder).withUsername("trillian");
verify(pullCommandBuilder).withPassword("secret");
verify(pullCommandBuilder).withUsername("trillian");
verify(pullCommandBuilder).withPassword("secret");
}
@Test
void shouldPullChangesWithLfsIfNotDisabled() {
FromUrlImporter.RepositoryImportParameters parameters = new FromUrlImporter.RepositoryImportParameters();
parameters.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
parameters.setSkipLfs(false);
importer.importFromUrl(parameters, repository);
verify(pullCommandBuilder).doFetchLfs(true);
}
@Test
void shouldPullChangesWithoutLfsIfSelected() {
FromUrlImporter.RepositoryImportParameters parameters = new FromUrlImporter.RepositoryImportParameters();
parameters.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
parameters.setSkipLfs(true);
importer.importFromUrl(parameters, repository);
verify(pullCommandBuilder).doFetchLfs(false);
}
@Test
void shouldHandleFailedLfsFiles() {
FromUrlImporter.RepositoryImportParameters parameters = new FromUrlImporter.RepositoryImportParameters();
parameters.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
parameters.setSkipLfs(false);
mockedResponse = new PullResponse(42, new PullResponse.LfsCount(0, 1));
importer.importFromUrl(parameters, repository);
verify(notificationHandler).
handleSuccessfulImportWithLfsFailures(
eq(createdRepository),
argThat(argument -> argument.getFailureCount() == 1 && argument.getSuccessCount() == 0));
}
}
@Test
void shouldThrowImportFailedEvent() throws IOException {
PullCommandBuilder pullCommandBuilder = mock(PullCommandBuilder.class, RETURNS_SELF);
when(service.getPullCommand()).thenReturn(pullCommandBuilder);
doThrow(TestException.class).when(pullCommandBuilder).pull(anyString());
when(logger.started()).thenReturn(true);

110
scm-webapp/src/test/java/sonia/scm/importexport/ImportNotificationHandlerTest.java Normal file
View File

@@ -0,0 +1,110 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import sonia.scm.notifications.NotificationSender;
import sonia.scm.notifications.Type;
import sonia.scm.repository.Repository;
import sonia.scm.repository.RepositoryTestData;
import sonia.scm.repository.api.PullResponse;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.argThat;
import static org.mockito.Mockito.verify;
@ExtendWith(MockitoExtension.class)
class ImportNotificationHandlerTest {
@Mock
private NotificationSender notificationSender;
@InjectMocks
private ImportNotificationHandler handler;
private final Repository repository = RepositoryTestData.createHeartOfGold();
@Test
void shouldCreateSuccessNotification() {
handler.handleSuccessfulImport(repository);
verify(notificationSender).send(argThat(
notification -> {
assertThat(notification.getType()).isEqualTo(Type.SUCCESS);
assertThat(notification.getMessage()).isEqualTo("importFinished");
assertThat(notification.getParameters()).isNull();
return true;
}
));
}
@Test
void shouldCreateSuccessNotificationWithLfs() {
handler.handleSuccessfulImport(repository, new PullResponse.LfsCount(42, 0));
verify(notificationSender).send(argThat(
notification -> {
assertThat(notification.getType()).isEqualTo(Type.SUCCESS);
assertThat(notification.getMessage()).isEqualTo("importWithLfsFinished");
assertThat(notification.getParameters()).containsEntry("successCount", "42");
return true;
}
));
}
@Test
void shouldCreateFailureNotification() {
handler.handleFailedImport();
verify(notificationSender).send(argThat(
notification -> {
assertThat(notification.getType()).isEqualTo(Type.ERROR);
assertThat(notification.getMessage()).isEqualTo("importFailed");
assertThat(notification.getParameters()).isNull();
return true;
}
));
}
@Test
void shouldCreateLfsFailureNotification() {
handler.handleSuccessfulImportWithLfsFailures(repository, new PullResponse.LfsCount(42, 7));
verify(notificationSender).send(argThat(
notification -> {
assertThat(notification.getType()).isEqualTo(Type.ERROR);
assertThat(notification.getMessage()).isEqualTo("importLfsFailed");
assertThat(notification.getParameters())
.containsEntry("successCount", "42")
.containsEntry("failureCount", "7")
.containsEntry("overallCount", "49");
return true;
}
));
}
}