diff --git a/CHANGELOG.md b/CHANGELOG.md index c997b37726..4c97dd0025 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Add markdown codeblock renderer extension point ([#1492](https://github.com/scm-manager/scm-manager/pull/1492)) - Add Java version to plugin center url ([#1494](https://github.com/scm-manager/scm-manager/pull/1494)) - Add Font ttf-dejavu to oci image ([#1498](https://github.com/scm-manager/scm-manager/issues/1498)) +- Add repository import and export with metadata for Subversion ([#1501](https://github.com/scm-manager/scm-manager/pull/1501)) +- API for store rename/delete in update steps ([#1505](https://github.com/scm-manager/scm-manager/pull/1505)) + +### Changed +- Directory name for git LFS files ([#1504](https://github.com/scm-manager/scm-manager/pull/1504)) ### Changed - Migrate integration tests to bdd ([#1497](https://github.com/scm-manager/scm-manager/pull/1497)) diff --git a/docs/de/user/repo/assets/import-repository-with-metadata.png b/docs/de/user/repo/assets/import-repository-with-metadata.png new file mode 100644 index 0000000000..43b60b92c8 Binary files /dev/null and b/docs/de/user/repo/assets/import-repository-with-metadata.png differ diff --git a/docs/de/user/repo/assets/repository-settings-general-svn.png b/docs/de/user/repo/assets/repository-settings-general-svn.png index 8428eeaa25..d1a7184bc9 100644 Binary files a/docs/de/user/repo/assets/repository-settings-general-svn.png and b/docs/de/user/repo/assets/repository-settings-general-svn.png differ diff --git a/docs/de/user/repo/index.md b/docs/de/user/repo/index.md index aea842f5a7..0f6ae32107 100644 --- a/docs/de/user/repo/index.md +++ b/docs/de/user/repo/index.md @@ -47,8 +47,15 @@ Das gewählte Repository wird zum SCM-Manager hinzugefügt und sämtliche Reposi ![Repository importieren](assets/import-repository.png) +Für Subversion Repositories besteht die Möglichkeit, ein Repository inkl. Metadaten zu importieren. +Dabei muss als Quelle ein Repository Archiv ausgewählt werden, welches vorher von einem SCM-Manager exportiert wurde. +Der Import mit Metadaten unterstützt noch keine Migration der Plugin Daten, +deshalb müssen die Versionen des SCM-Managers und die Versionen sämtlicher Plugins zwischen der exportierenden Instanz und der importierenden Instanz exakt übereinstimmen. +Wenn sich die installierten Plugins zwischen diesen beiden Instanzen unterscheiden, sollte dies kein Problem verursachen. +![Repository mit Metadaten importieren](assets/import-repository-with-metadata.png) ### Repository Informationen -Die Informationsseite eines Repository zeigt die Metadaten zum Repository an. Darunter befinden sich Beschreibungen zu den unterschiedlichen Möglichkeiten wie man mit diesem Repository arbeiten kann. In der Überschrift kann der Namespace angeklickt werden, um alle Repositories aus diesem Namespace anzuzeigen. +Die Informationsseite eines Repository zeigt die Metadaten zum Repository an. Darunter befinden sich Beschreibungen zu den unterschiedlichen Möglichkeiten wie man mit diesem Repository arbeiten kann. +In der Überschrift kann der Namespace angeklickt werden, um alle Repositories aus diesem Namespace anzuzeigen. ![Repository-Information](assets/repository-information.png) diff --git a/docs/de/user/repo/settings.md b/docs/de/user/repo/settings.md index 7ecd807097..6771f39743 100644 --- a/docs/de/user/repo/settings.md +++ b/docs/de/user/repo/settings.md @@ -19,8 +19,9 @@ Ein archiviertes Repository kann nicht mehr verändert werden. ![Repository-Settings-General-Git](assets/repository-settings-general-git.png) -In dem Bereich "Repository exportieren" kann das Repository als Dump exportiert werden. -Für den Download kann zwischen einem komprimierten Dump oder dem einfachen Dump-Format gewählt werden. +In dem Bereich "Repository exportieren" kann das Repository exportiert werden. +Für den Download kann zwischen einem einfachen Dump des reinen Repositories und einem Repository Archiv inkl. der SCM-Manager Metadaten wie Plugin-Konfigurationen oder anderen Daten gewählt werden. +Der Dump kann optional komprimiert werden. Das Repository Archiv mit Metadaten wird immer komprimiert ausgeliefert. Diese Export-Funktion wird derzeit nur von Subversion Repositories unterstützt. ![Repository-Settings-General-Svn](assets/repository-settings-general-svn.png) diff --git a/docs/en/user/repo/assets/import-repository-with-metadata.png b/docs/en/user/repo/assets/import-repository-with-metadata.png new file mode 100644 index 0000000000..2c859aca9a Binary files /dev/null and b/docs/en/user/repo/assets/import-repository-with-metadata.png differ diff --git a/docs/en/user/repo/assets/repository-settings-general-svn.png b/docs/en/user/repo/assets/repository-settings-general-svn.png index cb9b59ba70..e6e24a091e 100644 Binary files a/docs/en/user/repo/assets/repository-settings-general-svn.png and b/docs/en/user/repo/assets/repository-settings-general-svn.png differ diff --git a/docs/en/user/repo/index.md b/docs/en/user/repo/index.md index c078777634..6dce67aa9b 100644 --- a/docs/en/user/repo/index.md +++ b/docs/en/user/repo/index.md @@ -45,7 +45,18 @@ Your repository will be added to SCM-Manager and all repository data including a ![Import Repository](assets/import-repository.png) + +Subversion also supports the import of a repository archive with metadata. +This repository archive must be exported from an SCM-Manager. +This import mode doesn't support data migration yet. +So the repository archive can only be imported if the version between the exporting SCM-Manager and the importing SCM-Manager and **also all plugin versions** are equal. +If the installed plugins differ between those two instances it shouldn't create an issue. +![Import Repository with Metadata](assets/import-repository-with-metadata.png) + + ### Repository Information -The information screen of repositories shows meta data about the repository. Amongst that are descriptions for the different options on how the repository can be used. In the heading you can click the namespace to get the list of all repositories for this namespace. +The information screen of repositories shows meta data about the repository. +Amongst that are descriptions for the different options on how the repository can be used. +In the heading you can click the namespace to get the list of all repositories for this namespace. ![Repository Information](assets/repository-information.png) diff --git a/docs/en/user/repo/settings.md b/docs/en/user/repo/settings.md index 7c1596e7a9..fc9d8eaed4 100644 --- a/docs/en/user/repo/settings.md +++ b/docs/en/user/repo/settings.md @@ -17,8 +17,9 @@ repository is marked as archived, it can no longer be modified. ![Repository-Settings-General-Git](assets/repository-settings-general-git.png) -In the area "Repository Export" you may export this repository as dump file. -You can choose between compressed and uncompressed download format. +In the area "Repository Export" you may export this repository. +You can choose to export a simple dump of the repository or a repository archive including all SCM-Manager metadata like plugin configuration or other data. +For a simple repository dump you can choose between compressed or uncompressed file format. The repository archive is always compressed. This export function is currently only supported by Subversion repositories. ![Repository-Settings-General-Svn](assets/repository-settings-general-svn.png) diff --git a/scm-core/src/main/java/sonia/scm/migration/UpdateStep.java b/scm-core/src/main/java/sonia/scm/migration/UpdateStep.java index a5c784cd64..79ad1c6d19 100644 --- a/scm-core/src/main/java/sonia/scm/migration/UpdateStep.java +++ b/scm-core/src/main/java/sonia/scm/migration/UpdateStep.java @@ -21,7 +21,7 @@ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ - + package sonia.scm.migration; import sonia.scm.plugin.ExtensionPoint; @@ -50,6 +50,7 @@ import sonia.scm.version.Version; * *

+ *

Mind that an implementation of this class has to be annotated with {@link sonia.scm.plugin.Extension}, so that the + * step will be found.

*/ @ExtensionPoint public interface UpdateStep { diff --git a/scm-core/src/main/java/sonia/scm/repository/Repository.java b/scm-core/src/main/java/sonia/scm/repository/Repository.java index c0e47713bd..d829d7f4fd 100644 --- a/scm-core/src/main/java/sonia/scm/repository/Repository.java +++ b/scm-core/src/main/java/sonia/scm/repository/Repository.java @@ -56,7 +56,7 @@ import java.util.Set; @XmlRootElement(name = "repositories") @StaticPermissions( value = "repository", - permissions = {"read", "modify", "delete", "rename", "healthCheck", "pull", "push", "permissionRead", "permissionWrite", "archive"}, + permissions = {"read", "modify", "delete", "rename", "healthCheck", "pull", "push", "permissionRead", "permissionWrite", "archive", "export"}, custom = true, customGlobal = true, guards = { @Guard(guard = RepositoryPermissionGuard.class) diff --git a/scm-core/src/main/java/sonia/scm/repository/api/ExportFailedException.java b/scm-core/src/main/java/sonia/scm/repository/api/ExportFailedException.java new file mode 100644 index 0000000000..48db71f220 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/repository/api/ExportFailedException.java @@ -0,0 +1,48 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.repository.api; + +import sonia.scm.ContextEntry; +import sonia.scm.ExceptionWithContext; + +import java.util.List; + +public class ExportFailedException extends ExceptionWithContext { + + private static final String CODE = "67SM3DANZ1"; + + public ExportFailedException(List context, String message, Exception cause) { + super(context, message, cause); + } + + public ExportFailedException(List context, String message) { + super(context, message); + } + + @Override + public String getCode() { + return CODE; + } +} diff --git a/scm-core/src/main/java/sonia/scm/repository/api/ImportFailedException.java b/scm-core/src/main/java/sonia/scm/repository/api/ImportFailedException.java index 1d73104944..c79ed181ac 100644 --- a/scm-core/src/main/java/sonia/scm/repository/api/ImportFailedException.java +++ b/scm-core/src/main/java/sonia/scm/repository/api/ImportFailedException.java @@ -42,6 +42,10 @@ public class ImportFailedException extends ExceptionWithContext { super(context, message, cause); } + public ImportFailedException(List context, String message) { + super(context, message); + } + @Override public String getCode() { return CODE; diff --git a/scm-core/src/main/java/sonia/scm/store/ExportableStore.java b/scm-core/src/main/java/sonia/scm/store/ExportableStore.java new file mode 100644 index 0000000000..322377d7c8 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/ExportableStore.java @@ -0,0 +1,49 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import com.google.common.annotations.Beta; + +import java.io.IOException; + +/** + * The {@link ExportableStore} is used to export the stored data inside the store. + *

This interface is not yet finalized and might change in the upcoming versions.

+ * + * @since 2.13.0 + */ +@Beta +public interface ExportableStore { + + /** + * Contains the information about this store. + */ + StoreEntryMetaData getMetaData(); + + /** + * Exports the data of this store to the given {@param exporter}. + */ + void export(Exporter exporter) throws IOException; +} diff --git a/scm-core/src/main/java/sonia/scm/store/Exporter.java b/scm-core/src/main/java/sonia/scm/store/Exporter.java new file mode 100644 index 0000000000..86a7355809 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/Exporter.java @@ -0,0 +1,48 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import com.google.common.annotations.Beta; + +import java.io.IOException; +import java.io.OutputStream; + +/** + * The {@link Exporter} is used to export a single store entry to an {@link OutputStream}. + *

This interface is not yet finalized and might change in the upcoming versions.

+ * + * @since 2.13.0 + */ +@Beta +public interface Exporter { + /** + * Returns the {@link OutputStream} that should be used to export a single store entry with the given name. + * + * @param name The name of the exported store entry. + * @param size The size of the exported store entry (the size of the bytes that will be written to the output stream). + * @return The output stream the raw data of the store entry must be written to. + */ + OutputStream put(String name, long size) throws IOException; +} diff --git a/scm-core/src/main/java/sonia/scm/store/StoreEntryImporter.java b/scm-core/src/main/java/sonia/scm/store/StoreEntryImporter.java new file mode 100644 index 0000000000..1f7cbbf029 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/StoreEntryImporter.java @@ -0,0 +1,46 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import com.google.common.annotations.Beta; + +import java.io.InputStream; + +/** + * The {@link StoreEntryImporter} is used to import a store entry from an {@link InputStream}. + *

This interface is not yet finalized and might change in the upcoming versions.

+ * + * @since 2.13.0 + */ +@Beta +public interface StoreEntryImporter { + + /** + * Will be called for each entry of the store. + * @param name The name of the store entry. + * @param stream An input stream with the raw data of the store entry. + */ + void importEntry(String name, InputStream stream); +} diff --git a/scm-core/src/main/java/sonia/scm/store/StoreEntryImporterFactory.java b/scm-core/src/main/java/sonia/scm/store/StoreEntryImporterFactory.java new file mode 100644 index 0000000000..2c4525f750 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/StoreEntryImporterFactory.java @@ -0,0 +1,43 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import com.google.common.annotations.Beta; + +/** + * Create a {@link StoreEntryImporter} for the store type and store name. + *

This interface is not yet finalized and might change in the upcoming versions.

+ * + * @since 2.13.0 + */ +@Beta +public interface StoreEntryImporterFactory { + /** + * Returns a {@link StoreEntryImporter}. + * + * @param metaData The metaData about this store. For example store type and store name. + */ + StoreEntryImporter importStore(StoreEntryMetaData metaData); +} diff --git a/scm-core/src/main/java/sonia/scm/store/StoreEntryMetaData.java b/scm-core/src/main/java/sonia/scm/store/StoreEntryMetaData.java new file mode 100644 index 0000000000..54f59ea291 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/StoreEntryMetaData.java @@ -0,0 +1,33 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import lombok.Value; + +@Value +public class StoreEntryMetaData { + StoreType type; + String name; +} diff --git a/scm-core/src/main/java/sonia/scm/store/StoreExporter.java b/scm-core/src/main/java/sonia/scm/store/StoreExporter.java new file mode 100644 index 0000000000..200abae0c0 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/StoreExporter.java @@ -0,0 +1,42 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import com.google.common.annotations.Beta; +import sonia.scm.repository.Repository; + +import java.util.List; + +/** + * The {@link StoreExporter} is used to collect all {@link ExportableStore}s for a given repository. + * An {@link ExportableStore} can be used to export all data which is stored inside. + *

This interface is not yet finalized and might change in the upcoming versions.

+ * + * @since 2.13.0 + */ +@Beta +public interface StoreExporter { + List listExportableStores(Repository repository); +} diff --git a/scm-core/src/main/java/sonia/scm/store/StoreImporter.java b/scm-core/src/main/java/sonia/scm/store/StoreImporter.java new file mode 100644 index 0000000000..dbcc6bda39 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/StoreImporter.java @@ -0,0 +1,44 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import com.google.common.annotations.Beta; +import sonia.scm.repository.Repository; + +/** + * The {@link StoreImporter} is used to create a {@link StoreEntryImporterFactory} for a {@link Repository}. + *

This interface is not yet finalized and might change in the upcoming versions.

+ * + * @since 2.13.0 + */ +@Beta +public interface StoreImporter { + /** + * Returns a {@link StoreEntryImporterFactory} for the {@link Repository} + * + * @param repository + */ + StoreEntryImporterFactory doImport(Repository repository); +} diff --git a/scm-core/src/main/java/sonia/scm/store/StoreType.java b/scm-core/src/main/java/sonia/scm/store/StoreType.java new file mode 100644 index 0000000000..982eb71f4b --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/StoreType.java @@ -0,0 +1,43 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +public enum StoreType { + + DATA("data"), + CONFIG("config"), + BLOB("blob"), + CONFIG_ENTRY("configEntry"); + + StoreType(String value) { + this.value = value; + } + + private final String value; + + public String getValue() { + return value; + } +} diff --git a/scm-core/src/main/java/sonia/scm/update/RepositoryUpdateIterator.java b/scm-core/src/main/java/sonia/scm/update/RepositoryUpdateIterator.java new file mode 100644 index 0000000000..7ac218bcd7 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/update/RepositoryUpdateIterator.java @@ -0,0 +1,42 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.update; + +import java.util.function.Consumer; + +/** + * Implementations of this interface can be used to iterate all repositories in update steps. + * + * @since 2.13.0 + */ +public interface RepositoryUpdateIterator { + + /** + * Calls the given consumer with each repository id. + * + * @since 2.13.0 + */ + void forEachRepository(Consumer repositoryIdConsumer); +} diff --git a/scm-core/src/main/java/sonia/scm/update/StoreUpdateStepUtilFactory.java b/scm-core/src/main/java/sonia/scm/update/StoreUpdateStepUtilFactory.java new file mode 100644 index 0000000000..f42e0ed3c3 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/update/StoreUpdateStepUtilFactory.java @@ -0,0 +1,93 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.update; + +import sonia.scm.store.StoreParameters; +import sonia.scm.store.StoreType; + +public interface StoreUpdateStepUtilFactory { + + default UtilForTypeBuilder forType(StoreType type) { + return new UtilForTypeBuilder(this, type); + } + + final class UtilForTypeBuilder { + private final StoreUpdateStepUtilFactory factory; + private final StoreType type; + + public UtilForTypeBuilder(StoreUpdateStepUtilFactory factory, StoreType type) { + this.factory = factory; + this.type = type; + } + + public UtilForNameBuilder forName(String name) { + return new UtilForNameBuilder(factory, type, name); + } + } + + final class UtilForNameBuilder { + + private final StoreUpdateStepUtilFactory factory; + private final StoreType type; + private final String name; + private String repositoryId; + + public UtilForNameBuilder(StoreUpdateStepUtilFactory factory, StoreType type, String name) { + this.factory = factory; + this.type = type; + this.name = name; + } + + public UtilForNameBuilder forRepository(String repositoryId) { + this.repositoryId = repositoryId; + return this; + } + + public StoreUpdateStepUtil build() { + return factory.build( + type, + new StoreParameters() { + @Override + public String getName() { + return name; + } + + @Override + public String getRepositoryId() { + return repositoryId; + } + } + ); + } + } + + StoreUpdateStepUtil build(StoreType type, StoreParameters parameters); + + interface StoreUpdateStepUtil { + void renameStore(String newName); + + void deleteStore(); + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportCopier.java b/scm-dao-xml/src/main/java/sonia/scm/store/ExportCopier.java new file mode 100644 index 0000000000..460c7fff0f --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/ExportCopier.java @@ -0,0 +1,52 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import sonia.scm.repository.api.ExportFailedException; + +import java.io.IOException; +import java.io.OutputStream; +import java.nio.file.Files; +import java.nio.file.Path; + +import static sonia.scm.ContextEntry.ContextBuilder.noContext; + +final class ExportCopier { + + private ExportCopier() { + } + + static void putFileContentIntoStream(Exporter exporter, Path file) { + try (OutputStream stream = exporter.put(file.getFileName().toString(), Files.size(file))) { + Files.copy(file, stream); + } catch (IOException e) { + throw new ExportFailedException( + noContext(), + "Could not copy file to export stream: " + file, + e + ); + } + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableBlobFileStore.java b/scm-dao-xml/src/main/java/sonia/scm/store/ExportableBlobFileStore.java new file mode 100644 index 0000000000..d2b45837f3 --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/ExportableBlobFileStore.java @@ -0,0 +1,51 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import java.nio.file.Path; +import java.util.Optional; +import java.util.function.Function; + +import static java.util.Optional.empty; +import static java.util.Optional.of; + +class ExportableBlobFileStore extends ExportableDirectoryBasedFileStore { + + static Function>> BLOB_FACTORY = + storeType -> storeType == StoreType.BLOB ? of(ExportableBlobFileStore::new) : empty(); + + ExportableBlobFileStore(Path directory) { + super(directory); + } + + @Override + StoreType getStoreType() { + return StoreType.BLOB; + } + + boolean shouldIncludeFile(Path file) { + return file.getFileName().toString().endsWith(".blob"); + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigFileStore.java b/scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigFileStore.java new file mode 100644 index 0000000000..b2decc4cf0 --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigFileStore.java @@ -0,0 +1,58 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import java.io.IOException; +import java.nio.file.Path; +import java.util.Optional; +import java.util.function.Function; + +import static java.util.Optional.empty; +import static java.util.Optional.of; +import static sonia.scm.store.ExportCopier.putFileContentIntoStream; + +class ExportableConfigFileStore implements ExportableStore { + + private final Path file; + + static Function>> CONFIG_FACTORY = + storeType -> storeType == StoreType.CONFIG ? of(ExportableConfigFileStore::new) : empty(); + + ExportableConfigFileStore(Path file) { + this.file = file; + } + + @Override + public StoreEntryMetaData getMetaData() { + return new StoreEntryMetaData(StoreType.CONFIG, file.getFileName().toString()); + } + + @Override + public void export(Exporter exporter) throws IOException { + if (file.getFileName().toString().endsWith(".xml")) { + putFileContentIntoStream(exporter, file); + } + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDataFileStore.java b/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDataFileStore.java new file mode 100644 index 0000000000..4841775e49 --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDataFileStore.java @@ -0,0 +1,51 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import java.nio.file.Path; +import java.util.Optional; +import java.util.function.Function; + +import static java.util.Optional.empty; +import static java.util.Optional.of; + +class ExportableDataFileStore extends ExportableDirectoryBasedFileStore { + + static Function>> DATA_FACTORY = + storeType -> storeType == StoreType.DATA ? of(ExportableDataFileStore::new) : empty(); + + ExportableDataFileStore(Path directory) { + super(directory); + } + + @Override + StoreType getStoreType() { + return StoreType.DATA; + } + + boolean shouldIncludeFile(Path file) { + return file.getFileName().toString().endsWith(".xml"); + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDirectoryBasedFileStore.java b/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDirectoryBasedFileStore.java new file mode 100644 index 0000000000..47dfd32830 --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDirectoryBasedFileStore.java @@ -0,0 +1,76 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import sonia.scm.repository.api.ExportFailedException; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.stream.Stream; + +import static sonia.scm.ContextEntry.ContextBuilder.noContext; +import static sonia.scm.store.ExportCopier.putFileContentIntoStream; + +abstract class ExportableDirectoryBasedFileStore implements ExportableStore { + + private final Path directory; + + ExportableDirectoryBasedFileStore(Path directory) { + this.directory = directory; + } + + @Override + public StoreEntryMetaData getMetaData() { + return new StoreEntryMetaData(getStoreType(), directory.getFileName().toString()); + } + + abstract StoreType getStoreType(); + + abstract boolean shouldIncludeFile(Path file); + + @Override + public void export(Exporter exporter) throws IOException { + exportDirectoryEntries(exporter, directory); + } + + private void exportDirectoryEntries(Exporter exporter, Path directory) { + try (Stream fileList = Files.list(directory)) { + fileList.forEach(fileOrDir -> exportIfRelevant(exporter, fileOrDir)); + } catch (IOException e) { + throw new ExportFailedException( + noContext(), + "Could not read directory " + directory, + e + ); + } + } + + private void exportIfRelevant(Exporter exporter, Path fileOrDir) { + if (!Files.isDirectory(fileOrDir) && shouldIncludeFile(fileOrDir)) { + putFileContentIntoStream(exporter, fileOrDir); + } + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporter.java b/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporter.java new file mode 100644 index 0000000000..040892abb5 --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporter.java @@ -0,0 +1,62 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import com.google.common.annotations.VisibleForTesting; +import sonia.scm.ContextEntry; +import sonia.scm.repository.api.ImportFailedException; + +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Path; + +class FileBasedStoreEntryImporter implements StoreEntryImporter { + + private final Path directory; + + FileBasedStoreEntryImporter(Path directory) { + this.directory = directory; + } + + @VisibleForTesting + Path getDirectory() { + return this.directory; + } + + @Override + public void importEntry(String name, InputStream stream) { + Path filePath = directory.resolve(name); + try { + Files.copy(stream, filePath); + } catch (IOException e) { + throw new ImportFailedException( + ContextEntry.ContextBuilder.noContext(), + String.format("Could not import file %s for store %s", name, directory.toString()), + e + ); + } + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporterFactory.java b/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporterFactory.java new file mode 100644 index 0000000000..470055ef0b --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporterFactory.java @@ -0,0 +1,74 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import sonia.scm.ContextEntry; +import sonia.scm.repository.api.ImportFailedException; + +import java.io.File; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; + +class FileBasedStoreEntryImporterFactory implements StoreEntryImporterFactory { + + private final Path directory; + + FileBasedStoreEntryImporterFactory(Path directory) { + this.directory = directory; + } + + @Override + public StoreEntryImporter importStore(StoreEntryMetaData metaData) { + StoreType storeType = metaData.getType(); + String storeName = metaData.getName(); + Path storeDirectory = directory.resolve(Store.STORE_DIRECTORY); + try { + storeDirectory = storeDirectory.resolve(resolveFilePath(storeType.getValue(), storeName)); + Files.createDirectories(storeDirectory); + if (!Files.exists(storeDirectory)) { + throw new ImportFailedException( + ContextEntry.ContextBuilder.noContext(), + String.format("Could not create store for type %s and name %s", storeType, storeName) + ); + } + return new FileBasedStoreEntryImporter(storeDirectory); + + } catch (IOException e) { + throw new ImportFailedException( + ContextEntry.ContextBuilder.noContext(), + String.format("Could not create store directory %s for type %s and name %s", storeDirectory, storeType, storeName) + ); + } + } + + private Path resolveFilePath(String type, String name) { + if (name == null || name.isEmpty()) { + return Paths.get(type); + } + return Paths.get(type, name); + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileRepositoryUpdateIterator.java b/scm-dao-xml/src/main/java/sonia/scm/store/FileRepositoryUpdateIterator.java new file mode 100644 index 0000000000..11278b4729 --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/FileRepositoryUpdateIterator.java @@ -0,0 +1,49 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.update.RepositoryUpdateIterator; + +import javax.inject.Inject; +import java.nio.file.Path; +import java.util.function.Consumer; + +public class FileRepositoryUpdateIterator implements RepositoryUpdateIterator { + + private final RepositoryLocationResolver locationResolver; + + @Inject + public FileRepositoryUpdateIterator(RepositoryLocationResolver locationResolver) { + this.locationResolver = locationResolver; + } + + @Override + public void forEachRepository(Consumer repositoryIdConsumer) { + locationResolver + .forClass(Path.class) + .forAllLocations((repositoryId, path) -> repositoryIdConsumer.accept(repositoryId)); + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreExporter.java b/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreExporter.java new file mode 100644 index 0000000000..c81e76c46d --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreExporter.java @@ -0,0 +1,121 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.repository.api.ExportFailedException; + +import javax.inject.Inject; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; +import java.util.Optional; +import java.util.function.Function; +import java.util.stream.Stream; + +import static java.util.Arrays.asList; +import static java.util.Collections.emptyList; +import static sonia.scm.ContextEntry.ContextBuilder.noContext; +import static sonia.scm.store.ExportableBlobFileStore.BLOB_FACTORY; +import static sonia.scm.store.ExportableConfigFileStore.CONFIG_FACTORY; +import static sonia.scm.store.ExportableDataFileStore.DATA_FACTORY; + +public class FileStoreExporter implements StoreExporter { + + private final RepositoryLocationResolver locationResolver; + + private static final Collection>>> STORE_FACTORIES = + asList(DATA_FACTORY, BLOB_FACTORY, CONFIG_FACTORY); + + @Inject + public FileStoreExporter(RepositoryLocationResolver locationResolver) { + this.locationResolver = locationResolver; + } + + @Override + public List listExportableStores(Repository repository) { + List exportableStores = new ArrayList<>(); + Path storeDirectory = resolveStoreDirectory(repository); + if (!Files.exists(storeDirectory)) { + return emptyList(); + } + try (Stream storeTypeDirectories = Files.list(storeDirectory)) { + storeTypeDirectories.forEach(storeTypeDirectory -> + exportStoreTypeDirectories(exportableStores, storeTypeDirectory) + ); + } catch (IOException e) { + throw new ExportFailedException( + noContext(), + "Could not list content of directory " + storeDirectory, + e + ); + } + return exportableStores; + } + + private Path resolveStoreDirectory(Repository repository) { + return locationResolver + .forClass(Path.class) + .getLocation(repository.getId()) + .resolve(Store.STORE_DIRECTORY); + } + + private void exportStoreTypeDirectories(List exportableStores, Path storeTypeDirectory) { + try (Stream storeDirectories = Files.list(storeTypeDirectory)) { + storeDirectories.forEach(storeDirectory -> + getStoreFor(storeDirectory).ifPresent(exportableStores::add) + ); + } catch (IOException e) { + throw new ExportFailedException( + noContext(), + "Could not list content of directory " + storeTypeDirectory, + e + ); + } + } + + private Optional getStoreFor(Path storePath) { + return STORE_FACTORIES + .stream() + .map(factory -> factory.apply(getEnumForValue(storePath.getParent()))) + .filter(Optional::isPresent) + .map(Optional::get) + .findFirst() + .map(f -> f.apply(storePath)); + } + + private StoreType getEnumForValue(Path storeTypeDirectory) { + for (StoreType type : StoreType.values()) { + if (type.getValue().equals(storeTypeDirectory.getFileName().toString())) { + return type; + } + } + return null; + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtil.java b/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtil.java new file mode 100644 index 0000000000..ee8ce5df61 --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtil.java @@ -0,0 +1,86 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import sonia.scm.SCMContextProvider; +import sonia.scm.migration.UpdateException; +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.update.StoreUpdateStepUtilFactory; +import sonia.scm.util.IOUtil; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; + +class FileStoreUpdateStepUtil implements StoreUpdateStepUtilFactory.StoreUpdateStepUtil { + + private final RepositoryLocationResolver locationResolver; + private final SCMContextProvider contextProvider; + + private final StoreParameters parameters; + private final StoreType type; + + FileStoreUpdateStepUtil(RepositoryLocationResolver locationResolver, SCMContextProvider contextProvider, StoreParameters parameters, StoreType type) { + this.locationResolver = locationResolver; + this.contextProvider = contextProvider; + this.parameters = parameters; + this.type = type; + } + + @Override + public void renameStore(String newName) { + Path oldStorePath = resolveBasePath().resolve(parameters.getName()); + if (Files.exists(oldStorePath)) { + Path newStorePath = resolveBasePath().resolve(newName); + try { + Files.move(oldStorePath, newStorePath); + } catch (IOException e) { + throw new UpdateException(String.format("Could not move store path %s to %s", oldStorePath, newStorePath), e); + } + } + } + + @Override + public void deleteStore() { + Path oldStorePath = resolveBasePath().resolve(parameters.getName()); + IOUtil.deleteSilently(oldStorePath.toFile()); + } + + private Path resolveBasePath() { + Path basePath; + if (parameters.getRepositoryId() != null) { + basePath = locationResolver.forClass(Path.class).getLocation(parameters.getRepositoryId()); + } else { + basePath = contextProvider.getBaseDirectory().toPath(); + } + Path storeBasePath; + if (parameters.getRepositoryId() == null) { + storeBasePath = basePath.resolve(Store.forStoreType(type).getGlobalStoreDirectory()); + } else { + storeBasePath = basePath.resolve(Store.forStoreType(type).getRepositoryStoreDirectory()); + } + return storeBasePath; + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java b/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java new file mode 100644 index 0000000000..bae718cd8f --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java @@ -0,0 +1,49 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import sonia.scm.SCMContextProvider; +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.update.StoreUpdateStepUtilFactory; + +import javax.inject.Inject; + +public class FileStoreUpdateStepUtilFactory implements StoreUpdateStepUtilFactory { + + private final RepositoryLocationResolver locationResolver; + private final SCMContextProvider contextProvider; + + @Inject + public FileStoreUpdateStepUtilFactory(RepositoryLocationResolver locationResolver, SCMContextProvider contextProvider) { + this.locationResolver = locationResolver; + this.contextProvider = contextProvider; + } + + @Override + public StoreUpdateStepUtil build(StoreType type, StoreParameters parameters) { + return new FileStoreUpdateStepUtil(locationResolver, contextProvider, parameters, type); + } + +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/RepositoryStoreImporter.java b/scm-dao-xml/src/main/java/sonia/scm/store/RepositoryStoreImporter.java new file mode 100644 index 0000000000..927510175b --- /dev/null +++ b/scm-dao-xml/src/main/java/sonia/scm/store/RepositoryStoreImporter.java @@ -0,0 +1,49 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryLocationResolver; + +import javax.inject.Inject; +import java.nio.file.Path; + +public class RepositoryStoreImporter implements StoreImporter { + + private final RepositoryLocationResolver locationResolver; + + @Inject + public RepositoryStoreImporter(RepositoryLocationResolver locationResolver) { + this.locationResolver = locationResolver; + } + + @Override + public StoreEntryImporterFactory doImport(Repository repository) { + Path storeLocation = locationResolver + .forClass(Path.class) + .getLocation(repository.getId()); + return new FileBasedStoreEntryImporterFactory(storeLocation); + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/Store.java b/scm-dao-xml/src/main/java/sonia/scm/store/Store.java index cc259c7e55..5c134dcfee 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/Store.java +++ b/scm-dao-xml/src/main/java/sonia/scm/store/Store.java @@ -21,18 +21,32 @@ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ - + package sonia.scm.store; import java.io.File; -public enum Store { +enum Store { CONFIG("config"), DATA("data"), BLOB("blob"); private static final String GLOBAL_STORE_BASE_DIRECTORY = "var"; - private static final String STORE_DIRECTORY = "store"; + static final String STORE_DIRECTORY = "store"; + + static Store forStoreType(StoreType storeType) { + switch (storeType) { + case BLOB: + return BLOB; + case DATA: + return DATA; + case CONFIG: + case CONFIG_ENTRY: + return CONFIG; + default: + throw new IllegalArgumentException("unknown store type: " + storeType); + } + } private String directory; diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/ExportableFileStoreTest.java b/scm-dao-xml/src/test/java/sonia/scm/store/ExportableFileStoreTest.java new file mode 100644 index 0000000000..25437031b8 --- /dev/null +++ b/scm-dao-xml/src/test/java/sonia/scm/store/ExportableFileStoreTest.java @@ -0,0 +1,144 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; + +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.FileWriter; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class ExportableFileStoreTest { + + @Mock + Exporter exporter; + + @Test + void shouldNotPutContentIfNoFilesExists(@TempDir Path temp) throws IOException { + Path dataStoreDir = temp.resolve("some-store"); + Files.createDirectories(dataStoreDir); + + ExportableStore exportableFileStore = new ExportableDataFileStore(dataStoreDir); + exportableFileStore.export(exporter); + + verify(exporter, never()).put(anyString(), anyLong()); + } + + @Test + void shouldPutContentIntoExporterForDataStore(@TempDir Path temp) throws IOException { + createFile(temp, "data", "trace", "first.xml"); + createFile(temp, "data", "trace", "second.xml"); + ByteArrayOutputStream os = new ByteArrayOutputStream(); + ExportableStore exportableFileStore = new ExportableDataFileStore(temp.resolve("data").resolve("trace")); + when(exporter.put(anyString(), anyLong())).thenReturn(os); + + exportableFileStore.export(exporter); + + verify(exporter).put(eq("first.xml"), anyLong()); + verify(exporter).put(eq("second.xml"), anyLong()); + assertThat(os.toString()).isNotBlank(); + } + + @Test + void shouldPutContentIntoExporterForConfigStore(@TempDir Path temp) throws IOException { + createFile(temp, "config", "", "first.xml"); + ByteArrayOutputStream os = new ByteArrayOutputStream(); + ExportableStore exportableConfigFileStore = new ExportableConfigFileStore(temp.resolve("config").resolve("first.xml")); + when(exporter.put(anyString(), anyLong())).thenReturn(os); + + exportableConfigFileStore.export(exporter); + + verify(exporter).put(eq("first.xml"), anyLong()); + assertThat(os.toString()).isNotBlank(); + } + + @Test + void shouldFilterNoneConfigFiles(@TempDir Path temp) throws IOException { + createFile(temp, "config", "", "first.bck"); + ExportableStore exportableConfigFileStore = new ExportableConfigFileStore(temp.resolve("config").resolve("first.bck")); + + exportableConfigFileStore.export(exporter); + + verify(exporter, never()).put(anyString(), anyLong()); + } + + @Test + void shouldPutContentIntoExporterForBlobStore(@TempDir Path temp) throws IOException { + createFile(temp, "blob", "assets", "first.blob"); + ByteArrayOutputStream os = new ByteArrayOutputStream(); + Exporter exporter = mock(Exporter.class); + ExportableStore exportableBlobFileStore = new ExportableBlobFileStore(temp.resolve("blob").resolve("assets")); + when(exporter.put(anyString(), anyLong())).thenReturn(os); + + exportableBlobFileStore.export(exporter); + + verify(exporter).put(eq("first.blob"), anyLong()); + assertThat(os.toString()).isNotBlank(); + } + + @Test + void shouldSkipFilteredBlobFiles(@TempDir Path temp) throws IOException { + createFile(temp, "blob", "security", "second.xml"); + ByteArrayOutputStream os = new ByteArrayOutputStream(); + Exporter exporter = mock(Exporter.class); + ExportableStore exportableBlobFileStore = new ExportableBlobFileStore(temp.resolve("blob").resolve("security")); + + exportableBlobFileStore.export(exporter); + + verify(exporter, never()).put(anyString(), anyLong()); + assertThat(os.toString()).isBlank(); + } + + private File createFile(Path temp, String type, String name, String fileName) throws IOException { + Path path = name != null ? temp.resolve(type).resolve(name) : temp.resolve(type); + + new File(path.toUri()).mkdirs(); + File file = new File(path.toFile(), fileName); + if (!file.exists()) { + file.createNewFile(); + } + FileWriter source = new FileWriter(file); + source.write("something"); + source.close(); + return file; + } +} diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterFactoryTest.java b/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterFactoryTest.java new file mode 100644 index 0000000000..c532f087fa --- /dev/null +++ b/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterFactoryTest.java @@ -0,0 +1,50 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +package sonia.scm.store; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.nio.file.Path; + +import static org.assertj.core.api.Assertions.assertThat; + +class FileBasedStoreEntryImporterFactoryTest { + + @Test + void shouldCreateStoreEntryImporterForDataStore(@TempDir Path temp) { + FileBasedStoreEntryImporterFactory factory = new FileBasedStoreEntryImporterFactory(temp); + + FileBasedStoreEntryImporter dataImporter = (FileBasedStoreEntryImporter) factory.importStore(new StoreEntryMetaData(StoreType.DATA, "hitchhiker")); + assertThat(dataImporter.getDirectory()).isEqualTo(temp.resolve("store").resolve("data").resolve("hitchhiker")); + } + + @Test + void shouldCreateStoreEntryImporterForConfigStore(@TempDir Path temp) { + FileBasedStoreEntryImporterFactory factory = new FileBasedStoreEntryImporterFactory(temp); + + FileBasedStoreEntryImporter configImporter = (FileBasedStoreEntryImporter) factory.importStore(new StoreEntryMetaData(StoreType.CONFIG, "")); + assertThat(configImporter.getDirectory()).isEqualTo(temp.resolve("store").resolve("config")); + } +} diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterTest.java b/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterTest.java new file mode 100644 index 0000000000..00099e0150 --- /dev/null +++ b/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterTest.java @@ -0,0 +1,50 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; + +import static org.assertj.core.api.Assertions.assertThat; + +class FileBasedStoreEntryImporterTest { + + @Test + void shouldCreateFileFromInputStream(@TempDir Path temp) { + FileBasedStoreEntryImporter importer = new FileBasedStoreEntryImporter(temp); + String fileName = "testStore.xml"; + + importer.importEntry(fileName, new ByteArrayInputStream("testdata".getBytes())); + + assertThat(Files.exists(temp.resolve(fileName))).isTrue(); + assertThat(temp.resolve(fileName)).hasContent("testdata"); + } +} diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreExporterTest.java b/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreExporterTest.java new file mode 100644 index 0000000000..d7677e94a4 --- /dev/null +++ b/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreExporterTest.java @@ -0,0 +1,90 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.Answers; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.repository.RepositoryTestData; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Collections; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class FileStoreExporterTest { + + private static final Repository REPOSITORY = RepositoryTestData.create42Puzzle(); + + @Mock(answer = Answers.RETURNS_DEEP_STUBS) + private RepositoryLocationResolver resolver; + + @InjectMocks + private FileStoreExporter fileStoreExporter; + + @Test + void shouldReturnEmptyList(@TempDir Path temp) { + when(resolver.forClass(Path.class).getLocation(REPOSITORY.getId())).thenReturn(temp); + + List exportableStores = fileStoreExporter.listExportableStores(REPOSITORY); + assertThat(exportableStores).isEmpty(); + } + + @Test + void shouldReturnListOfExportableStores(@TempDir Path temp) throws IOException { + Path storePath = temp.resolve("store"); + createFile(storePath, StoreType.CONFIG.getValue(), null, "first.xml"); + createFile(storePath, StoreType.DATA.getValue(), "ci", "second.xml"); + createFile(storePath, StoreType.DATA.getValue(), "jenkins", "third.xml"); + when(resolver.forClass(Path.class).getLocation(REPOSITORY.getId())).thenReturn(temp); + + List exportableStores = fileStoreExporter.listExportableStores(REPOSITORY); + + assertThat(exportableStores).hasSize(3); + assertThat(exportableStores.stream().filter(e -> e.getMetaData().getType().equals(StoreType.CONFIG))).hasSize(1); + assertThat(exportableStores.stream().filter(e -> e.getMetaData().getType().equals(StoreType.DATA))).hasSize(2); + } + + private void createFile(Path storePath, String type, String name, String fileName) throws IOException { + Path path = name != null ? storePath.resolve(type).resolve(name) : storePath.resolve(type); + Files.createDirectories(path); + Path file = path.resolve(fileName); + if (!Files.exists(file)) { + Files.createFile(file); + } + Files.write(file, Collections.singletonList("something")); + } +} diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreUpdateStepUtilFactoryTest.java b/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreUpdateStepUtilFactoryTest.java new file mode 100644 index 0000000000..570141239f --- /dev/null +++ b/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreUpdateStepUtilFactoryTest.java @@ -0,0 +1,121 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.store; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.SCMContextProvider; +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.update.StoreUpdateStepUtilFactory; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.lenient; + +@ExtendWith(MockitoExtension.class) +class FileStoreUpdateStepUtilFactoryTest { + + @Mock + private RepositoryLocationResolver locationResolver; + @Mock + private RepositoryLocationResolver.RepositoryLocationResolverInstance locationResolverInstance; + @Mock + private SCMContextProvider contextProvider; + + @InjectMocks + private FileStoreUpdateStepUtilFactory factory; + + private Path globalPath; + private Path repositoryPath; + + @BeforeEach + void initPaths(@TempDir Path temp) throws IOException { + globalPath = temp.resolve("global"); + Files.createDirectories(globalPath); + lenient().doReturn(globalPath.toFile()).when(contextProvider).getBaseDirectory(); + + repositoryPath = temp.resolve("repo"); + Files.createDirectories(repositoryPath); + lenient().doReturn(true).when(locationResolver).supportsLocationType(Path.class); + lenient().doReturn(locationResolverInstance).when(locationResolver).forClass(Path.class); + lenient().doReturn(repositoryPath).when(locationResolverInstance).getLocation("repo-id"); + } + + @Test + void shouldMoveGlobalDataDirectory() throws IOException { + Path dataPath = globalPath.resolve("var").resolve("data"); + Files.createDirectories(dataPath.resolve("something")); + Files.createFile(dataPath.resolve("something").resolve("some.file")); + StoreUpdateStepUtilFactory.StoreUpdateStepUtil util = + factory + .forType(StoreType.DATA) + .forName("something") + .build(); + + util.renameStore("new-name"); + + assertThat(dataPath.resolve("new-name").resolve("some.file")).exists(); + assertThat(dataPath.resolve("something")).doesNotExist(); + } + + @Test + void shouldMoveRepositoryDataDirectory() throws IOException { + Path dataPath = repositoryPath.resolve("store").resolve("data"); + Files.createDirectories(dataPath.resolve("something")); + Files.createFile(dataPath.resolve("something").resolve("some.file")); + StoreUpdateStepUtilFactory.StoreUpdateStepUtil util = + factory + .forType(StoreType.DATA) + .forName("something") + .forRepository("repo-id") + .build(); + + util.renameStore("new-name"); + + assertThat(dataPath.resolve("new-name").resolve("some.file")).exists(); + assertThat(dataPath.resolve("something")).doesNotExist(); + } + + @Test + void shouldHandleMissingMoveGlobalDataDirectory() throws IOException { + StoreUpdateStepUtilFactory.StoreUpdateStepUtil util = + factory + .forType(StoreType.DATA) + .forName("something") + .build(); + + util.renameStore("new-name"); + + assertThat(globalPath.resolve("new-name")).doesNotExist(); + } +} diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/RepositoryStoreImporterTest.java b/scm-dao-xml/src/test/java/sonia/scm/store/RepositoryStoreImporterTest.java new file mode 100644 index 0000000000..bc1e18e696 --- /dev/null +++ b/scm-dao-xml/src/test/java/sonia/scm/store/RepositoryStoreImporterTest.java @@ -0,0 +1,53 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +package sonia.scm.store; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Answers; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.repository.RepositoryTestData; + +import static org.assertj.core.api.Assertions.assertThat; + +@ExtendWith(MockitoExtension.class) +class RepositoryStoreImporterTest { + private static final Repository REPOSITORY = RepositoryTestData.createHeartOfGold(); + + @Mock(answer = Answers.RETURNS_DEEP_STUBS) + private RepositoryLocationResolver locationResolver; + + @InjectMocks + private RepositoryStoreImporter repositoryStoreImporter; + + @Test + void shouldImportStore() { + StoreEntryImporterFactory storeEntryImporterFactory = repositoryStoreImporter.doImport(REPOSITORY); + assertThat(storeEntryImporterFactory).isInstanceOf(StoreEntryImporterFactory.class); + } +} diff --git a/scm-dao-xml/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker b/scm-dao-xml/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker new file mode 100644 index 0000000000..1f0955d450 --- /dev/null +++ b/scm-dao-xml/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker @@ -0,0 +1 @@ +mock-maker-inline diff --git a/scm-plugins/scm-git-plugin/src/main/java/sonia/scm/web/lfs/LfsBlobStoreFactory.java b/scm-plugins/scm-git-plugin/src/main/java/sonia/scm/web/lfs/LfsBlobStoreFactory.java index b37b23208a..4d87ea188a 100644 --- a/scm-plugins/scm-git-plugin/src/main/java/sonia/scm/web/lfs/LfsBlobStoreFactory.java +++ b/scm-plugins/scm-git-plugin/src/main/java/sonia/scm/web/lfs/LfsBlobStoreFactory.java @@ -21,7 +21,7 @@ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. */ - + package sonia.scm.web.lfs; import com.google.inject.Inject; @@ -32,44 +32,37 @@ import sonia.scm.store.BlobStoreFactory; /** * Creates {@link BlobStore} objects to store lfs objects. - * + * * @author Sebastian Sdorra * @since 1.54 */ @Singleton public class LfsBlobStoreFactory { - - private static final String GIT_LFS_REPOSITORY_POSTFIX = "-git-lfs"; - + + private static final String GIT_LFS_STORE_NAME = "git-lfs"; + private final BlobStoreFactory blobStoreFactory; /** * Create a new instance. - * + * * @param blobStoreFactory blob store factory */ @Inject public LfsBlobStoreFactory(BlobStoreFactory blobStoreFactory) { this.blobStoreFactory = blobStoreFactory; } - + /** * Provides a {@link BlobStore} corresponding to the SCM Repository. - *

- * git-lfs repositories should generally carry the same name as their regular SCM repository counterparts. However, - * we have decided to store them under their IDs instead of their names, since the names might change and provide - * other drawbacks, as well. - *

- * These repositories will have {@linkplain #GIT_LFS_REPOSITORY_POSTFIX} appended to their IDs. * * @param repository The SCM Repository to provide a LFS {@link BlobStore} for. - * + * * @return blob store for the corresponding scm repository */ - @SuppressWarnings("unchecked") public BlobStore getLfsBlobStore(Repository repository) { return blobStoreFactory - .withName(repository.getId() + GIT_LFS_REPOSITORY_POSTFIX) + .withName(GIT_LFS_STORE_NAME) .forRepository(repository) .build(); } diff --git a/scm-plugins/scm-git-plugin/src/main/java/sonia/scm/web/lfs/RemoveRepositoryIdFromBlobStoreUpdateStep.java b/scm-plugins/scm-git-plugin/src/main/java/sonia/scm/web/lfs/RemoveRepositoryIdFromBlobStoreUpdateStep.java new file mode 100644 index 0000000000..919a656b16 --- /dev/null +++ b/scm-plugins/scm-git-plugin/src/main/java/sonia/scm/web/lfs/RemoveRepositoryIdFromBlobStoreUpdateStep.java @@ -0,0 +1,73 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.web.lfs; + +import sonia.scm.migration.UpdateStep; +import sonia.scm.plugin.Extension; +import sonia.scm.store.StoreType; +import sonia.scm.update.RepositoryUpdateIterator; +import sonia.scm.update.StoreUpdateStepUtilFactory; +import sonia.scm.version.Version; + +import javax.inject.Inject; + +import static sonia.scm.version.Version.parse; + +@Extension +public class RemoveRepositoryIdFromBlobStoreUpdateStep implements UpdateStep { + + private final RepositoryUpdateIterator repositoryUpdateIterator; + private final StoreUpdateStepUtilFactory utilFactory; + + @Inject + public RemoveRepositoryIdFromBlobStoreUpdateStep(RepositoryUpdateIterator repositoryUpdateIterator, StoreUpdateStepUtilFactory utilFactory) { + this.repositoryUpdateIterator = repositoryUpdateIterator; + this.utilFactory = utilFactory; + } + + @Override + public void doUpdate() { + repositoryUpdateIterator.forEachRepository(this::doUpdate); + } + + private void doUpdate(String repositoryId) { + utilFactory + .forType(StoreType.BLOB) + .forName(repositoryId + "-git-lfs") + .forRepository(repositoryId) + .build() + .renameStore("git-lfs"); + } + + @Override + public Version getTargetVersion() { + return parse("2.0.1"); + } + + @Override + public String getAffectedDataType() { + return "sonia.scm.git.lfs"; + } +} diff --git a/scm-plugins/scm-git-plugin/src/test/java/sonia/scm/web/lfs/LfsBlobStoreFactoryTest.java b/scm-plugins/scm-git-plugin/src/test/java/sonia/scm/web/lfs/LfsBlobStoreFactoryTest.java index 180612a207..143f60caa6 100644 --- a/scm-plugins/scm-git-plugin/src/test/java/sonia/scm/web/lfs/LfsBlobStoreFactoryTest.java +++ b/scm-plugins/scm-git-plugin/src/test/java/sonia/scm/web/lfs/LfsBlobStoreFactoryTest.java @@ -62,7 +62,7 @@ public class LfsBlobStoreFactoryTest { // just make sure the right parameter is passed, as properly validating the return value is nearly impossible with // the return value (and should not be part of this test) verify(blobStoreFactory).getStore(argThat(blobStoreParameters -> { - assertThat(blobStoreParameters.getName()).isEqualTo("the-id-git-lfs"); + assertThat(blobStoreParameters.getName()).isEqualTo("git-lfs"); assertThat(blobStoreParameters.getRepositoryId()).isEqualTo("the-id"); return true; })); diff --git a/scm-plugins/scm-git-plugin/src/test/java/sonia/scm/web/lfs/RemoveRepositoryIdFromBlobStoreUpdateStepTest.java b/scm-plugins/scm-git-plugin/src/test/java/sonia/scm/web/lfs/RemoveRepositoryIdFromBlobStoreUpdateStepTest.java new file mode 100644 index 0000000000..bef6cb6a68 --- /dev/null +++ b/scm-plugins/scm-git-plugin/src/test/java/sonia/scm/web/lfs/RemoveRepositoryIdFromBlobStoreUpdateStepTest.java @@ -0,0 +1,74 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.web.lfs; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Answers; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.store.StoreType; +import sonia.scm.update.RepositoryUpdateIterator; +import sonia.scm.update.StoreUpdateStepUtilFactory; + +import java.io.IOException; +import java.util.function.Consumer; + +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.argThat; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.doReturn; +import static org.mockito.Mockito.verify; + +@ExtendWith(MockitoExtension.class) +class RemoveRepositoryIdFromBlobStoreUpdateStepTest { + + @Mock + private RepositoryUpdateIterator repositoryUpdateIterator; + @Mock(answer = Answers.CALLS_REAL_METHODS) + private StoreUpdateStepUtilFactory utilFactory; + @Mock + private StoreUpdateStepUtilFactory.StoreUpdateStepUtil util; + + @InjectMocks + private RemoveRepositoryIdFromBlobStoreUpdateStep updateStep; + + @Test + void migrateBlobsFromOldStoreToNewStore() throws IOException { + Mockito.doAnswer(invocation -> { + invocation.getArgument(0, Consumer.class).accept("repo-id"); + return null; + }).when(repositoryUpdateIterator).forEachRepository(any()); + + doReturn(util) + .when(utilFactory).build(eq(StoreType.BLOB), argThat(argument -> argument.getName().equals("repo-id-git-lfs"))); + + updateStep.doUpdate(); + + verify(util).renameStore("git-lfs"); + } +} diff --git a/scm-ui/ui-webapp/public/locales/de/repos.json b/scm-ui/ui-webapp/public/locales/de/repos.json index 289ca66d30..a5f60986f7 100644 --- a/scm-ui/ui-webapp/public/locales/de/repos.json +++ b/scm-ui/ui-webapp/public/locales/de/repos.json @@ -75,6 +75,10 @@ "title": "Wählen Sie Ihre Datei aus", "helpText": "Wählen Sie die Datei aus der das Repository importiert werden soll." }, + "fullImport": { + "title": "SCM-Manager Repository Archiv", + "helpText": "Wählen Sie das Repository Archiv aus. Das Archiv muss von einer SCM-Manager Instanz exportiert worden sein." + }, "pending": { "subtitle": "Repository wird importiert...", "infoText": "Ihr Repository wird gerade importiert. Dies kann einen Moment dauern. Sie werden weitergeleitet, sobald der Import abgeschlossen ist. Wenn Sie diese Seite verlassen, können Sie nicht zurückkehren, um den Import-Status zu erfahren." @@ -87,7 +91,11 @@ }, "bundle": { "label": "Import aus Dump", - "helpText": "Das Repository wird aus einen Datei Dump importiert." + "helpText": "Das Repository wird aus einem Datei Dump importiert." + }, + "fullImport": { + "label": "Import aus Archiv mit Metadaten (Experimentell)", + "helpText": "Das Repository inkl. der Metadaten wird aus einem archivierten Dump importiert. Das Import Archiv muss von einem anderen SCM-Manager generiert worden sein." } } }, @@ -248,6 +256,10 @@ "label": "Komprimieren", "helpText": "Export Datei vor dem Download komprimieren. Reduziert die Downloadgröße." }, + "fullExport": { + "label": "Mit Metadaten (Experimentell)", + "helpText": "Zusätzlich zum Repository Dump werden Metadaten zum Repository und zur SCM-Instanz exportiert. Gespeicherte Passwörter funktionieren nicht bei einem Import in andere SCM-Manager Instanzen. Dieses Feature ist noch experimentell. Es sollte (noch) nicht für Backups genutzt werden!" + }, "exportButton": "Repository exportieren", "exportStarted": "Der Repository Export wurde gestartet. Abhängig von der Größe des Repository kann dies einige Momente dauern." }, diff --git a/scm-ui/ui-webapp/public/locales/en/repos.json b/scm-ui/ui-webapp/public/locales/en/repos.json index b7b6f1b797..1566237066 100644 --- a/scm-ui/ui-webapp/public/locales/en/repos.json +++ b/scm-ui/ui-webapp/public/locales/en/repos.json @@ -75,6 +75,10 @@ "title": "Dump File", "helpText": "Select your dump file from which the repository should be imported." }, + "fullImport": { + "title": "SCM-Manager Repository Archive", + "helpText": "Select the archive file which should be imported. The archive must have been exported from an SCM-Manager instance." + }, "pending": { "subtitle": "Importing Repository...", "infoText": "Your repository is currently being imported. This may take a moment. You will be forwarded as soon as the import is finished. If you leave this page you cannot return to find out the import status." @@ -88,6 +92,10 @@ "bundle": { "label": "Import from dump", "helpText": "The repository will be imported from a dump file." + }, + "fullImport": { + "label": "Import from archive with metadata (experimental)", + "helpText": "The repository will be imported with metadata. The archive containing the data must be generated by an SCM-Manager instance." } } }, @@ -248,6 +256,10 @@ "label": "Compress", "helpText": "Compress the export dump size to reduce the download size." }, + "fullExport": { + "label": "With metadata (Experimental)", + "helpText": "In addition to the repository dump, metadata about the repository and SCM instance is exported. Stored passwords will not work if this is imported in other instances of SCM-Manager. This feature is still experimental. Do not use this as a backup mechanism (yet)!" + }, "exportButton": "Export Repository", "exportStarted": "The repository export was started. Depending on the repository size this may take a while." }, diff --git a/scm-ui/ui-webapp/src/repos/components/ImportFullRepository.tsx b/scm-ui/ui-webapp/src/repos/components/ImportFullRepository.tsx new file mode 100644 index 0000000000..c75914379b --- /dev/null +++ b/scm-ui/ui-webapp/src/repos/components/ImportFullRepository.tsx @@ -0,0 +1,114 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +import React, { FC, FormEvent, useState } from "react"; +import NamespaceAndNameFields from "./NamespaceAndNameFields"; +import { File, Repository } from "@scm-manager/ui-types"; +import RepositoryInformationForm from "./RepositoryInformationForm"; +import { apiClient, ErrorNotification, Level, SubmitButton } from "@scm-manager/ui-components"; +import { useTranslation } from "react-i18next"; +import { useHistory } from "react-router-dom"; +import ImportFromBundleForm from "./ImportFromBundleForm"; +import ImportFullRepositoryForm from "./ImportFullRepositoryForm"; + +type Props = { + url: string; + repositoryType: string; + setImportPending: (pending: boolean) => void; +}; + +const ImportFullRepository: FC = ({ url, repositoryType, setImportPending }) => { + const [repo, setRepo] = useState({ + name: "", + namespace: "", + type: repositoryType, + contact: "", + description: "", + _links: {}, + }); + + const [valid, setValid] = useState({ namespaceAndName: false, contact: true, file: false }); + const [loading, setLoading] = useState(false); + const [error, setError] = useState(); + const [file, setFile] = useState(null); + const history = useHistory(); + const [t] = useTranslation("repos"); + + const handleImportLoading = (loading: boolean) => { + setImportPending(loading); + setLoading(loading); + }; + + const isValid = () => Object.values(valid).every((v) => v); + + const submit = (event: FormEvent) => { + event.preventDefault(); + const currentPath = history.location.pathname; + setError(undefined); + handleImportLoading(true); + apiClient + .postBinary(url, (formData) => { + formData.append("bundle", file, file?.name); + formData.append("repository", JSON.stringify(repo)); + }) + .then((response) => { + const location = response.headers.get("Location"); + return apiClient.get(location!); + }) + .then((response) => response.json()) + .then((repo) => { + if (history.location.pathname === currentPath) { + history.push(`/repo/${repo.namespace}/${repo.name}/code/sources`); + } + }) + .catch((error) => { + setError(error); + handleImportLoading(false); + }); + }; + + return ( +

+ + setValid({ ...valid, file })}/> +
+ >} + setValid={(namespaceAndName: boolean) => setValid({ ...valid, namespaceAndName })} + disabled={loading} + /> + >} + disabled={loading} + setValid={(contact: boolean) => setValid({ ...valid, contact })} + /> + } + /> + + ); +}; + +export default ImportFullRepository; diff --git a/scm-ui/ui-webapp/src/repos/components/ImportFullRepositoryForm.tsx b/scm-ui/ui-webapp/src/repos/components/ImportFullRepositoryForm.tsx new file mode 100644 index 0000000000..5a3911e1db --- /dev/null +++ b/scm-ui/ui-webapp/src/repos/components/ImportFullRepositoryForm.tsx @@ -0,0 +1,53 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +import React, { FC } from "react"; +import { FileUpload, LabelWithHelpIcon, Checkbox } from "@scm-manager/ui-components"; +import { File } from "@scm-manager/ui-types"; +import { useTranslation } from "react-i18next"; + +type Props = { + setFile: (file: File) => void; + setValid: (valid: boolean) => void; +}; + +const ImportFullRepositoryForm: FC = ({ setFile, setValid}) => { + const [t] = useTranslation("repos"); + + return ( +
+
+ + { + setFile(file); + setValid(!!file); + }} + /> +
+
+ ); +}; + +export default ImportFullRepositoryForm; diff --git a/scm-ui/ui-webapp/src/repos/containers/ExportRepository.tsx b/scm-ui/ui-webapp/src/repos/containers/ExportRepository.tsx index 9ec42e3748..c2e897b505 100644 --- a/scm-ui/ui-webapp/src/repos/containers/ExportRepository.tsx +++ b/scm-ui/ui-webapp/src/repos/containers/ExportRepository.tsx @@ -33,14 +33,19 @@ type Props = { const ExportRepository: FC = ({ repository }) => { const [t] = useTranslation("repos"); const [compressed, setCompressed] = useState(true); + const [fullExport, setFullExport] = useState(false); const [loading, setLoading] = useState(false); const createExportLink = () => { - let exportLink = (repository?._links.export as Link).href; - if (compressed) { - exportLink += "?compressed=true"; + if (fullExport) { + return (repository?._links?.fullExport as Link).href; + } else { + let exportLink = (repository?._links.export as Link).href; + if (compressed) { + exportLink += "?compressed=true"; + } + return exportLink; } - return exportLink; }; if (!repository?._links?.export) { @@ -53,11 +58,20 @@ const ExportRepository: FC = ({ repository }) => { <> + {repository?._links?.fullExport && ( + + )} setLoading(true)}> diff --git a/scm-ui/ui-webapp/src/repos/containers/ImportRepository.tsx b/scm-ui/ui-webapp/src/repos/containers/ImportRepository.tsx index 730f207ec3..8c72e75726 100644 --- a/scm-ui/ui-webapp/src/repos/containers/ImportRepository.tsx +++ b/scm-ui/ui-webapp/src/repos/containers/ImportRepository.tsx @@ -40,6 +40,7 @@ import { import { connect } from "react-redux"; import { fetchNamespaceStrategiesIfNeeded } from "../../admin/modules/namespaceStrategies"; import ImportRepositoryFromBundle from "../components/ImportRepositoryFromBundle"; +import ImportFullRepository from "../components/ImportFullRepository"; type Props = { repositoryTypes: RepositoryType[]; @@ -106,6 +107,16 @@ const ImportRepository: FC = ({ ); } + if (importType === "fullImport") { + return ( + link.name === "fullImport") as Link).href} + repositoryType={repositoryType!.name} + setImportPending={setImportPending} + /> + ); + } + throw new Error("Unknown import type"); }; diff --git a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryExportResource.java b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryExportResource.java index d684a42f5e..c1e77b1553 100644 --- a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryExportResource.java +++ b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryExportResource.java @@ -30,9 +30,9 @@ import io.swagger.v3.oas.annotations.media.Content; import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.responses.ApiResponse; import org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; +import sonia.scm.BadRequestException; import sonia.scm.Type; +import sonia.scm.importexport.FullScmRepositoryExporter; import sonia.scm.repository.InternalRepositoryException; import sonia.scm.repository.NamespaceAndName; import sonia.scm.repository.Repository; @@ -43,6 +43,7 @@ import sonia.scm.repository.api.RepositoryService; import sonia.scm.repository.api.RepositoryServiceFactory; import sonia.scm.web.VndMediaType; +import javax.validation.constraints.Pattern; import javax.ws.rs.Consumes; import javax.ws.rs.DefaultValue; import javax.ws.rs.GET; @@ -54,25 +55,25 @@ import javax.ws.rs.core.MediaType; import javax.ws.rs.core.Response; import javax.ws.rs.core.StreamingOutput; import javax.ws.rs.core.UriInfo; -import java.io.FileOutputStream; import java.io.IOException; import java.time.Instant; +import static sonia.scm.ContextEntry.ContextBuilder.entity; import static sonia.scm.api.v2.resources.RepositoryTypeSupportChecker.checkSupport; import static sonia.scm.api.v2.resources.RepositoryTypeSupportChecker.type; public class RepositoryExportResource { - private static final Logger logger = LoggerFactory.getLogger(RepositoryExportResource.class); - private final RepositoryManager manager; private final RepositoryServiceFactory serviceFactory; + private final FullScmRepositoryExporter fullScmRepositoryExporter; @Inject public RepositoryExportResource(RepositoryManager manager, - RepositoryServiceFactory serviceFactory) { + RepositoryServiceFactory serviceFactory, FullScmRepositoryExporter fullScmRepositoryExporter) { this.manager = manager; this.serviceFactory = serviceFactory; + this.fullScmRepositoryExporter = fullScmRepositoryExporter; } /** @@ -80,9 +81,9 @@ public class RepositoryExportResource { * only be used, if the repository type supports the {@link Command#BUNDLE}. * * @param uriInfo uri info - * @param namespace of the repository - * @param name of the repository - * @param type of the repository + * @param namespace namespace of the repository + * @param name name of the repository + * @param type type of the repository * @return response with readable stream of repository dump * @since 2.13.0 */ @@ -113,16 +114,72 @@ public class RepositoryExportResource { public Response exportRepository(@Context UriInfo uriInfo, @PathParam("namespace") String namespace, @PathParam("name") String name, - @PathParam("type") String type, + @Pattern(regexp = "\\w{1,10}") @PathParam("type") String type, @DefaultValue("false") @QueryParam("compressed") boolean compressed ) { + Repository repository = getVerifiedRepository(namespace, name, type); + return exportRepository(repository, compressed); + } + + /** + * Exports an existing repository with all additional metadata and environment information. The method can + * only be used, if the repository type supports the {@link Command#BUNDLE}. + * + * @param uriInfo uri info + * @param namespace namespace of the repository + * @param name name of the repository + * @param type type of the repository + * @return response with readable stream of repository dump + * @since 2.13.0 + */ + @GET + @Path("{type}/full") + @Consumes(VndMediaType.REPOSITORY) + @Operation(summary = "Exports the repository", description = "Exports the repository with metadata and environment information.", tags = "Repository") + @ApiResponse( + responseCode = "200", + description = "Repository export was successful" + ) + @ApiResponse( + responseCode = "401", + description = "not authenticated / invalid credentials" + ) + @ApiResponse( + responseCode = "403", + description = "not authorized, the current user has no privileges to read the repository" + ) + @ApiResponse( + responseCode = "500", + description = "internal server error", + content = @Content( + mediaType = VndMediaType.ERROR_TYPE, + schema = @Schema(implementation = ErrorDto.class) + ) + ) + public Response exportFullRepository(@Context UriInfo uriInfo, + @PathParam("namespace") String namespace, + @PathParam("name") String name, + @Pattern(regexp = "\\w{1,10}") @PathParam("type") String type + ) { + Repository repository = getVerifiedRepository(namespace, name, type); + StreamingOutput output = os -> fullScmRepositoryExporter.export(repository, os); + + return Response + .ok(output, "application/x-gzip") + .header("content-disposition", createContentDispositionHeaderValue(repository, "tar.gz")) + .build(); + } + + private Repository getVerifiedRepository(@PathParam("namespace") String namespace, @PathParam("name") String name, @PathParam("type") @Pattern(regexp = "\\w{1,10}") String type) { Repository repository = manager.get(new NamespaceAndName(namespace, name)); RepositoryPermissions.read().check(repository); + if (!type.equals(repository.getType())) { + throw new WrongTypeException(repository); + } Type repositoryType = type(manager, type); checkSupport(repositoryType, Command.BUNDLE); - - return exportRepository(repository, compressed); + return repository; } private Response exportRepository(Repository repository, boolean compressed) { @@ -140,24 +197,42 @@ public class RepositoryExportResource { } }; + return createResponse(repository, compressed, output); + } + + private Response createResponse(Repository repository, boolean compressed, StreamingOutput output) { return Response .ok(output, compressed ? "application/x-gzip" : MediaType.APPLICATION_OCTET_STREAM) - .header("content-disposition", createContentDispositionHeaderValue(repository, compressed)) + .header("content-disposition", createContentDispositionHeaderValue(repository, compressed ? "dump.gz" : "dump")) .build(); } - private String createContentDispositionHeaderValue(Repository repository, boolean compressed) { + private String createContentDispositionHeaderValue(Repository repository, String filetype) { String timestamp = createFormattedTimestamp(); return String.format( "attachment; filename = %s-%s-%s.%s", repository.getNamespace(), repository.getName(), timestamp, - compressed ? "dump.gz" : "dump" + filetype ); } private String createFormattedTimestamp() { return Instant.now().toString().replace(":", "-").split("\\.")[0]; } + + private static class WrongTypeException extends BadRequestException { + + private static final String CODE = "4hSNNTBiu1"; + + public WrongTypeException(Repository repository) { + super(entity(repository).build(), "illegal type for repository"); + } + + @Override + public String getCode() { + return CODE; + } + } } diff --git a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryImportResource.java b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryImportResource.java index 2a0de41d1a..577dd0727e 100644 --- a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryImportResource.java +++ b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryImportResource.java @@ -50,6 +50,7 @@ import org.slf4j.LoggerFactory; import sonia.scm.HandlerEventType; import sonia.scm.Type; import sonia.scm.event.ScmEventBus; +import sonia.scm.importexport.FullScmRepositoryImporter; import sonia.scm.repository.InternalRepositoryException; import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryImportEvent; @@ -63,6 +64,7 @@ import sonia.scm.repository.api.RepositoryServiceFactory; import sonia.scm.util.IOUtil; import sonia.scm.util.ValidationUtil; import sonia.scm.web.VndMediaType; +import sonia.scm.web.api.DtoValidator; import javax.validation.Valid; import javax.validation.constraints.Email; @@ -87,8 +89,6 @@ import java.util.List; import java.util.Map; import java.util.function.Consumer; -import static com.google.common.base.Preconditions.checkArgument; -import static com.google.common.base.Preconditions.checkNotNull; import static java.nio.charset.StandardCharsets.UTF_8; import static java.util.Collections.singletonList; import static sonia.scm.api.v2.resources.RepositoryTypeSupportChecker.checkSupport; @@ -103,18 +103,21 @@ public class RepositoryImportResource { private final RepositoryServiceFactory serviceFactory; private final ResourceLinks resourceLinks; private final ScmEventBus eventBus; + private final FullScmRepositoryImporter fullScmRepositoryImporter; @Inject public RepositoryImportResource(RepositoryManager manager, RepositoryDtoToRepositoryMapper mapper, RepositoryServiceFactory serviceFactory, ResourceLinks resourceLinks, - ScmEventBus eventBus) { + ScmEventBus eventBus, + FullScmRepositoryImporter fullScmRepositoryImporter) { this.manager = manager; this.mapper = mapper; this.serviceFactory = serviceFactory; this.resourceLinks = resourceLinks; this.eventBus = eventBus; + this.fullScmRepositoryImporter = fullScmRepositoryImporter; } /** @@ -254,6 +257,62 @@ public class RepositoryImportResource { return Response.created(URI.create(resourceLinks.repository().self(repository.getNamespace(), repository.getName()))).build(); } + /** + * Imports a repository as SCM-Manager provided import archive. The method can + * only be used, if the repository type supports the {@link Command#UNBUNDLE}. The + * method will return a location header with the url to the imported + * repository. + * + * @param uriInfo uri info + * @param type repository type + * @param input multi part form data which should contain a valid repository dto and the input stream of the bundle + * @return empty response with location header which points to the imported + * repository + * @since 2.13.0 + */ + @POST + @Path("{type}/full") + @Consumes(MediaType.MULTIPART_FORM_DATA) + @Operation( + summary = "Import repository from SCM-Manager repository archive", + description = "Imports the repository with metadata from the provided bundle.", + tags = "Repository" + ) + @ApiResponse( + responseCode = "201", + description = "Repository import was successful" + ) + @ApiResponse( + responseCode = "401", + description = "not authenticated / invalid credentials" + ) + @ApiResponse( + responseCode = "403", + description = "not authorized, the current user has no privileges to import repositories" + ) + @ApiResponse( + responseCode = "500", + description = "internal server error", + content = @Content( + mediaType = VndMediaType.ERROR_TYPE, + schema = @Schema(implementation = ErrorDto.class) + ) + ) + public Response importFullRepository(@Context UriInfo uriInfo, + @Pattern(regexp = "\\w{1,10}") @PathParam("type") String type, + MultipartFormDataInput input) { + RepositoryPermissions.create().check(); + Repository createdRepository = importFullRepositoryFromInput(input); + return Response.created(URI.create(resourceLinks.repository().self(createdRepository.getNamespace(), createdRepository.getName()))).build(); + } + + private Repository importFullRepositoryFromInput(MultipartFormDataInput input) { + Map> formParts = input.getFormDataMap(); + InputStream inputStream = extractInputStream(formParts); + RepositoryDto repositoryDto = extractRepositoryDto(formParts); + return fullScmRepositoryImporter.importFromStream(mapper.map(repositoryDto), inputStream); + } + /** * Start bundle import. * @@ -264,12 +323,8 @@ public class RepositoryImportResource { */ private Repository doImportFromBundle(String type, MultipartFormDataInput input, boolean compressed) { Map> formParts = input.getFormDataMap(); - RepositoryDto repositoryDto = extractFromInputPart(formParts.get("repository"), RepositoryDto.class); - InputStream inputStream = extractFromInputPart(formParts.get("bundle"), InputStream.class); - - checkNotNull(repositoryDto, "repository data is required"); - checkNotNull(inputStream, "bundle inputStream is required"); - checkArgument(!Strings.isNullOrEmpty(repositoryDto.getName()), "request does not contain name of the repository"); + InputStream inputStream = extractInputStream(formParts); + RepositoryDto repositoryDto = extractRepositoryDto(formParts); Type t = type(manager, type); checkSupport(t, Command.UNBUNDLE); @@ -315,6 +370,25 @@ public class RepositoryImportResource { }; } + private RepositoryDto extractRepositoryDto(Map> formParts) { + RepositoryDto repositoryDto = extractFromInputPart(formParts.get("repository"), RepositoryDto.class); + checkNotNull(repositoryDto, "repository data is required"); + DtoValidator.validate(repositoryDto); + return repositoryDto; + } + + private void checkNotNull(Object object, String errorMessage) { + if (object == null) { + throw new WebApplicationException(errorMessage, 400); + } + } + + private InputStream extractInputStream(Map> formParts) { + InputStream inputStream = extractFromInputPart(formParts.get("bundle"), InputStream.class); + checkNotNull(inputStream, "bundle inputStream is required"); + return inputStream; + } + private T extractFromInputPart(List input, Class type) { try { if (input != null && !input.isEmpty()) { @@ -343,6 +417,7 @@ public class RepositoryImportResource { @NoArgsConstructor @SuppressWarnings("java:S2160") public static class RepositoryImportDto extends RepositoryDto implements ImportRepositoryDto { + @NotEmpty private String importUrl; private String username; diff --git a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryToRepositoryDtoMapper.java b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryToRepositoryDtoMapper.java index 1b5eeed5f9..8f9a617ae1 100644 --- a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryToRepositoryDtoMapper.java +++ b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/RepositoryToRepositoryDtoMapper.java @@ -104,8 +104,9 @@ public abstract class RepositoryToRepositoryDtoMapper extends BaseMapper plugins = new ArrayList<>(); + for (InstalledPlugin plugin : pluginManager.getInstalled()) { + PluginInformation pluginInformation = plugin.getDescriptor().getInformation(); + plugins.add(new EnvironmentPluginDescriptor(pluginInformation.getName(), pluginInformation.getVersion())); + } + scmEnvironment.setPlugins(new EnvironmentPluginsDescriptor(plugins)); + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/EnvironmentPluginDescriptor.java b/scm-webapp/src/main/java/sonia/scm/importexport/EnvironmentPluginDescriptor.java new file mode 100644 index 0000000000..df900c9cbc --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/EnvironmentPluginDescriptor.java @@ -0,0 +1,42 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import lombok.AllArgsConstructor; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import javax.xml.bind.annotation.XmlRootElement; + +@XmlRootElement(name = "plugin") +@AllArgsConstructor +@NoArgsConstructor +@Setter +@Getter +class EnvironmentPluginDescriptor { + private String name; + private String version; +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/EnvironmentPluginsDescriptor.java b/scm-webapp/src/main/java/sonia/scm/importexport/EnvironmentPluginsDescriptor.java new file mode 100644 index 0000000000..e908fc2114 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/EnvironmentPluginsDescriptor.java @@ -0,0 +1,42 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import lombok.AllArgsConstructor; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import javax.xml.bind.annotation.XmlRootElement; +import java.util.List; + +@XmlRootElement(name = "plugins") +@AllArgsConstructor +@NoArgsConstructor +@Setter +@Getter +class EnvironmentPluginsDescriptor { + private List plugin; +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryExporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryExporter.java new file mode 100644 index 0000000000..b0db3d3084 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryExporter.java @@ -0,0 +1,142 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import org.apache.commons.compress.archivers.tar.TarArchiveEntry; +import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream; +import org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream; +import sonia.scm.ContextEntry; +import sonia.scm.repository.Repository; +import sonia.scm.repository.api.ExportFailedException; +import sonia.scm.repository.api.RepositoryService; +import sonia.scm.repository.api.RepositoryServiceFactory; +import sonia.scm.repository.work.WorkdirProvider; +import sonia.scm.util.IOUtil; + +import javax.inject.Inject; +import java.io.BufferedOutputStream; +import java.io.File; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.OutputStream; +import java.nio.file.Files; +import java.nio.file.Paths; + +public class FullScmRepositoryExporter { + + static final String SCM_ENVIRONMENT_FILE_NAME = "scm-environment.xml"; + static final String METADATA_FILE_NAME = "metadata.xml"; + static final String STORE_DATA_FILE_NAME = "store-data.tar"; + private final EnvironmentInformationXmlGenerator environmentGenerator; + private final RepositoryMetadataXmlGenerator metadataGenerator; + private final RepositoryServiceFactory serviceFactory; + private final TarArchiveRepositoryStoreExporter storeExporter; + private final WorkdirProvider workdirProvider; + + @Inject + public FullScmRepositoryExporter(EnvironmentInformationXmlGenerator environmentGenerator, + RepositoryMetadataXmlGenerator metadataGenerator, + RepositoryServiceFactory serviceFactory, + TarArchiveRepositoryStoreExporter storeExporter, WorkdirProvider workdirProvider) { + this.environmentGenerator = environmentGenerator; + this.metadataGenerator = metadataGenerator; + this.serviceFactory = serviceFactory; + this.storeExporter = storeExporter; + this.workdirProvider = workdirProvider; + } + + public void export(Repository repository, OutputStream outputStream) { + try ( + RepositoryService service = serviceFactory.create(repository); + BufferedOutputStream bos = new BufferedOutputStream(outputStream); + GzipCompressorOutputStream gzos = new GzipCompressorOutputStream(bos); + TarArchiveOutputStream taos = new TarArchiveOutputStream(gzos); + ) { + writeEnvironmentData(taos); + writeMetadata(repository, taos); + writeRepository(service, taos); + writeStoreData(repository, taos); + taos.finish(); + } catch (IOException e) { + throw new ExportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Could not export repository with metadata", + e + ); + } + } + + private void writeEnvironmentData(TarArchiveOutputStream taos) throws IOException { + byte[] envBytes = environmentGenerator.generate(); + TarArchiveEntry entry = new TarArchiveEntry(SCM_ENVIRONMENT_FILE_NAME); + entry.setSize(envBytes.length); + taos.putArchiveEntry(entry); + taos.write(envBytes); + taos.closeArchiveEntry(); + } + + private void writeMetadata(Repository repository, TarArchiveOutputStream taos) throws IOException { + byte[] metadataBytes = metadataGenerator.generate(repository); + TarArchiveEntry entry = new TarArchiveEntry(METADATA_FILE_NAME); + entry.setSize(metadataBytes.length); + taos.putArchiveEntry(entry); + taos.write(metadataBytes); + taos.closeArchiveEntry(); + } + + private void writeRepository(RepositoryService service, TarArchiveOutputStream taos) throws IOException { + File newWorkdir = workdirProvider.createNewWorkdir(); + try { + File repositoryFile = Files.createFile(Paths.get(newWorkdir.getPath(), "repository")).toFile(); + try (FileOutputStream repositoryFos = new FileOutputStream(repositoryFile)) { + service.getBundleCommand().bundle(repositoryFos); + } + TarArchiveEntry entry = new TarArchiveEntry(service.getRepository().getName() + ".dump"); + entry.setSize(repositoryFile.length()); + taos.putArchiveEntry(entry); + Files.copy(repositoryFile.toPath(), taos); + taos.closeArchiveEntry(); + } finally { + IOUtil.deleteSilently(newWorkdir); + } + } + + private void writeStoreData(Repository repository, TarArchiveOutputStream taos) throws IOException { + File newWorkdir = workdirProvider.createNewWorkdir(); + try { + File metadata = Files.createFile(Paths.get(newWorkdir.getPath(), "metadata")).toFile(); + try (FileOutputStream metadataFos = new FileOutputStream(metadata)) { + storeExporter.export(repository, metadataFos); + } + TarArchiveEntry entry = new TarArchiveEntry(STORE_DATA_FILE_NAME); + entry.setSize(metadata.length()); + taos.putArchiveEntry(entry); + Files.copy(metadata.toPath(), taos); + taos.closeArchiveEntry(); + } finally { + IOUtil.deleteSilently(newWorkdir); + } + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryImporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryImporter.java new file mode 100644 index 0000000000..743521c143 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryImporter.java @@ -0,0 +1,166 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import org.apache.commons.compress.archivers.ArchiveEntry; +import org.apache.commons.compress.archivers.tar.TarArchiveInputStream; +import org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream; +import sonia.scm.ContextEntry; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryManager; +import sonia.scm.repository.api.ImportFailedException; +import sonia.scm.repository.api.RepositoryService; +import sonia.scm.repository.api.RepositoryServiceFactory; + +import javax.inject.Inject; +import javax.xml.bind.JAXB; +import java.io.BufferedInputStream; +import java.io.FilterInputStream; +import java.io.IOException; +import java.io.InputStream; + +import static sonia.scm.importexport.FullScmRepositoryExporter.SCM_ENVIRONMENT_FILE_NAME; +import static sonia.scm.importexport.FullScmRepositoryExporter.STORE_DATA_FILE_NAME; + +public class FullScmRepositoryImporter { + + private static final int _1_MB = 1000000; + + private final RepositoryServiceFactory serviceFactory; + private final RepositoryManager repositoryManager; + private final ScmEnvironmentCompatibilityChecker compatibilityChecker; + private final TarArchiveRepositoryStoreImporter storeImporter; + + @Inject + public FullScmRepositoryImporter(RepositoryServiceFactory serviceFactory, + RepositoryManager repositoryManager, + ScmEnvironmentCompatibilityChecker compatibilityChecker, + TarArchiveRepositoryStoreImporter storeImporter) { + this.serviceFactory = serviceFactory; + this.repositoryManager = repositoryManager; + this.compatibilityChecker = compatibilityChecker; + this.storeImporter = storeImporter; + } + + public Repository importFromStream(Repository repository, InputStream inputStream) { + try { + if (inputStream.available() > 0) { + try ( + BufferedInputStream bif = new BufferedInputStream(inputStream); + GzipCompressorInputStream gcis = new GzipCompressorInputStream(bif); + TarArchiveInputStream tais = new TarArchiveInputStream(gcis) + ) { + checkScmEnvironment(repository, tais); + skipRepositoryMetadata(tais); + Repository createdRepository = importRepositoryFromFile(repository, tais); + importStoresForCreatedRepository(createdRepository, tais); + return createdRepository; + } + } else { + throw new ImportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Stream to import from is empty." + ); + } + } catch (IOException e) { + throw new ImportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Could not import repository data from stream; got io exception while reading", + e + ); + } + } + + private void importStoresForCreatedRepository(Repository repository, TarArchiveInputStream tais) throws IOException { + ArchiveEntry metadataEntry = tais.getNextEntry(); + if (metadataEntry.getName().equals(STORE_DATA_FILE_NAME) && !metadataEntry.isDirectory()) { + // Inside the repository tar archive stream is another tar archive. + // The nested tar archive is wrapped in another TarArchiveInputStream inside the storeImporter + storeImporter.importFromTarArchive(repository, tais); + } else { + throw new ImportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Invalid import format. Missing metadata file 'scm-metadata.tar' in tar." + ); + } + } + + private Repository importRepositoryFromFile(Repository repository, TarArchiveInputStream tais) throws IOException { + ArchiveEntry repositoryEntry = tais.getNextEntry(); + if (repositoryEntry.getName().endsWith(".dump") && !repositoryEntry.isDirectory()) { + return repositoryManager.create(repository, repo -> { + try (RepositoryService service = serviceFactory.create(repo)) { + service.getUnbundleCommand().unbundle(new NoneClosingInputStream(tais)); + } catch (IOException e) { + throw new ImportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Repository import failed. Could not import repository from file.", + e + ); + } + }); + } else { + throw new ImportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Invalid import format. Missing repository dump file." + ); + } + } + + private void checkScmEnvironment(Repository repository, TarArchiveInputStream tais) throws IOException { + ArchiveEntry environmentEntry = tais.getNextEntry(); + if (environmentEntry.getName().equals(SCM_ENVIRONMENT_FILE_NAME) && !environmentEntry.isDirectory() && environmentEntry.getSize() < _1_MB) { + boolean validEnvironment = compatibilityChecker.check(JAXB.unmarshal(new NoneClosingInputStream(tais), ScmEnvironment.class)); + if (!validEnvironment) { + throw new ImportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Incompatible SCM-Manager environment. Could not import file." + ); + } + } else { + throw new ImportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Invalid import format. Missing SCM-Manager environment description file 'scm-environment.xml' or file too big." + ); + } + } + + private void skipRepositoryMetadata(TarArchiveInputStream tais) throws IOException { + tais.getNextEntry(); + } + + @SuppressWarnings("java:S4929") // we only want to override close here + static class NoneClosingInputStream extends FilterInputStream { + + NoneClosingInputStream(InputStream delegate) { + super(delegate); + } + + @Override + public void close() { + // Avoid closing stream because JAXB tries to close the stream + } + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/RepositoryMetadataXmlGenerator.java b/scm-webapp/src/main/java/sonia/scm/importexport/RepositoryMetadataXmlGenerator.java new file mode 100644 index 0000000000..5596218820 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/RepositoryMetadataXmlGenerator.java @@ -0,0 +1,81 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import lombok.AllArgsConstructor; +import lombok.NoArgsConstructor; +import sonia.scm.ContextEntry; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryPermission; +import sonia.scm.repository.api.ExportFailedException; + +import javax.xml.bind.JAXB; +import javax.xml.bind.annotation.XmlAccessType; +import javax.xml.bind.annotation.XmlAccessorType; +import javax.xml.bind.annotation.XmlRootElement; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.util.Collection; + +class RepositoryMetadataXmlGenerator { + + byte[] generate(Repository repository) { + try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) { + RepositoryMetadata metadata = new RepositoryMetadata(repository); + JAXB.marshal(metadata, baos); + return baos.toByteArray(); + } catch (IOException e) { + throw new ExportFailedException( + ContextEntry.ContextBuilder.noContext(), + "Could not generate SCM-Manager environment description.", + e + ); + } + } + + @AllArgsConstructor + @NoArgsConstructor + @XmlRootElement(name = "metadata") + @XmlAccessorType(XmlAccessType.FIELD) + private static class RepositoryMetadata { + + private String namespace; + private String name; + private String type; + private String contact; + private String description; + private Collection permissions; + + public RepositoryMetadata(Repository repository) { + this( + repository.getNamespace(), + repository.getName(), + repository.getType(), + repository.getContact(), + repository.getDescription(), + repository.getPermissions()); + } + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/ScmEnvironment.java b/scm-webapp/src/main/java/sonia/scm/importexport/ScmEnvironment.java new file mode 100644 index 0000000000..307daedc5c --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/ScmEnvironment.java @@ -0,0 +1,43 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import javax.xml.bind.annotation.XmlRootElement; + +@XmlRootElement(name = "scm-environment") +@Getter +@Setter +@NoArgsConstructor +class ScmEnvironment { + private EnvironmentPluginsDescriptor plugins; + private String coreVersion; + private String os; + private String arch; +} + diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/ScmEnvironmentCompatibilityChecker.java b/scm-webapp/src/main/java/sonia/scm/importexport/ScmEnvironmentCompatibilityChecker.java new file mode 100644 index 0000000000..b9bf237fe4 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/ScmEnvironmentCompatibilityChecker.java @@ -0,0 +1,103 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import sonia.scm.SCMContextProvider; +import sonia.scm.plugin.PluginInformation; +import sonia.scm.plugin.PluginManager; + +import javax.inject.Inject; +import java.util.ArrayList; +import java.util.List; +import java.util.Optional; +import java.util.stream.Collectors; + +public class ScmEnvironmentCompatibilityChecker { + + private static final Logger LOG = LoggerFactory.getLogger(ScmEnvironmentCompatibilityChecker.class); + private final PluginManager pluginManager; + private final SCMContextProvider scmContextProvider; + + @Inject + public ScmEnvironmentCompatibilityChecker(PluginManager pluginManager, SCMContextProvider scmContextProvider) { + this.pluginManager = pluginManager; + this.scmContextProvider = scmContextProvider; + } + + boolean check(ScmEnvironment environment) { + return isCoreVersionCompatible(scmContextProvider.getVersion(), environment.getCoreVersion()) + && arePluginsCompatible(environment); + } + + private boolean isCoreVersionCompatible(String currentCoreVersion, String coreVersionFromImport) { + boolean compatible = currentCoreVersion.equals(coreVersionFromImport); + if (!compatible) { + LOG.info( + "SCM-Manager version is not compatible with dump. Dump can only be imported with SCM-Manager version: {}; you are running version {}", + coreVersionFromImport, + currentCoreVersion + ); + } + return compatible; + } + + private boolean arePluginsCompatible(ScmEnvironment environment) { + List currentlyInstalledPlugins = pluginManager.getInstalled() + .stream() + .map(p -> p.getDescriptor().getInformation()) + .collect(Collectors.toList()); + + for (EnvironmentPluginDescriptor plugin : environment.getPlugins().getPlugin()) { + Optional matchingInstalledPlugin = findMatchingInstalledPlugin(currentlyInstalledPlugins, plugin); + if (isPluginIncompatible(plugin, matchingInstalledPlugin)) { + LOG.info( + "The installed plugin \"{}\" with version \"{}\" doesn't match the plugin data version \"{}\" from the SCM-Manager environment the dump was created with.", + matchingInstalledPlugin.get().getName(), + matchingInstalledPlugin.get().getVersion(), + plugin.getVersion() + ); + return false; + } + } + return true; + } + + private boolean isPluginIncompatible(EnvironmentPluginDescriptor plugin, Optional matchingInstalledPlugin) { + return matchingInstalledPlugin.isPresent() && isPluginVersionIncompatible(plugin.getVersion(), matchingInstalledPlugin.get().getVersion()); + } + + private Optional findMatchingInstalledPlugin(List currentlyInstalledPlugins, EnvironmentPluginDescriptor plugin) { + return currentlyInstalledPlugins + .stream() + .filter(p -> p.getName().equalsIgnoreCase(plugin.getName())) + .findFirst(); + } + + private boolean isPluginVersionIncompatible(String previousPluginVersion, String installedPluginVersion) { + return !installedPluginVersion.equals(previousPluginVersion); + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreExporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreExporter.java new file mode 100644 index 0000000000..69708152fa --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreExporter.java @@ -0,0 +1,126 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import org.apache.commons.compress.archivers.tar.TarArchiveEntry; +import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import sonia.scm.ContextEntry; +import sonia.scm.repository.Repository; +import sonia.scm.repository.api.ExportFailedException; +import sonia.scm.store.ExportableStore; +import sonia.scm.store.StoreEntryMetaData; +import sonia.scm.store.StoreExporter; +import sonia.scm.store.StoreType; + +import javax.inject.Inject; +import java.io.BufferedOutputStream; +import java.io.FilterOutputStream; +import java.io.IOException; +import java.io.OutputStream; +import java.util.List; + +import static java.util.Arrays.asList; + +public class TarArchiveRepositoryStoreExporter { + + private static final Logger LOG = LoggerFactory.getLogger(TarArchiveRepositoryStoreExporter.class); + + private final StoreExporter storeExporter; + + @Inject + public TarArchiveRepositoryStoreExporter(StoreExporter storeExporter) { + this.storeExporter = storeExporter; + } + + public void export(Repository repository, OutputStream output) { + try ( + BufferedOutputStream bos = new BufferedOutputStream(output); + final TarArchiveOutputStream taos = new TarArchiveOutputStream(bos) + ) { + List exportableStores = storeExporter.listExportableStores(repository); + for (ExportableStore store : exportableStores) { + store.export((name, filesize) -> { + StoreEntryMetaData storeMetaData = store.getMetaData(); + if (isOneOfStoreTypes(store, StoreType.DATA, StoreType.BLOB)) { + String storePath = createStorePath(storeMetaData.getType().getValue(), storeMetaData.getName(), name); + addEntryToArchive(taos, storePath, filesize); + } else if (isOneOfStoreTypes(store, StoreType.CONFIG, StoreType.CONFIG_ENTRY)) { + String storePath = createStorePath(storeMetaData.getType().getValue(), name); + addEntryToArchive(taos, storePath, filesize); + } else { + LOG.debug("Skip file {} on export", name); + } + return createOutputStream(taos); + }); + } + + } catch (IOException e) { + throw new ExportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Could not export repository metadata stores.", + e + ); + } + } + + private boolean isOneOfStoreTypes(ExportableStore store, StoreType... types) { + return asList(types).contains(store.getMetaData().getType()); + } + + private void addEntryToArchive(TarArchiveOutputStream taos, String storePath, long filesize) throws IOException { + TarArchiveEntry entry = new TarArchiveEntry(storePath); + entry.setSize(filesize); + taos.putArchiveEntry(entry); + } + + private String createStorePath(String... pathParts) { + StringBuilder storePath = new StringBuilder("stores"); + for (String part : pathParts) { + storePath.append('/').append(part); + } + return storePath.toString(); + } + + private OutputStream createOutputStream(TarArchiveOutputStream taos) { + return new CloseArchiveOutputStream(taos); + } + + static class CloseArchiveOutputStream extends FilterOutputStream { + + private final TarArchiveOutputStream delegate; + + CloseArchiveOutputStream(TarArchiveOutputStream delegate) { + super(delegate); + this.delegate = delegate; + } + + @Override + public void close() throws IOException { + delegate.closeArchiveEntry(); + } + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporter.java new file mode 100644 index 0000000000..86910418f5 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporter.java @@ -0,0 +1,107 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import org.apache.commons.compress.archivers.ArchiveEntry; +import org.apache.commons.compress.archivers.tar.TarArchiveInputStream; +import sonia.scm.ContextEntry; +import sonia.scm.repository.Repository; +import sonia.scm.repository.api.ImportFailedException; +import sonia.scm.store.RepositoryStoreImporter; +import sonia.scm.store.StoreEntryMetaData; +import sonia.scm.store.StoreType; + +import javax.inject.Inject; +import java.io.File; +import java.io.IOException; +import java.io.InputStream; + +public class TarArchiveRepositoryStoreImporter { + + private final RepositoryStoreImporter repositoryStoreImporter; + + @Inject + public TarArchiveRepositoryStoreImporter(RepositoryStoreImporter repositoryStoreImporter) { + this.repositoryStoreImporter = repositoryStoreImporter; + } + + public void importFromTarArchive(Repository repository, InputStream inputStream) { + try (TarArchiveInputStream tais = new TarArchiveInputStream(inputStream)) { + ArchiveEntry entry = tais.getNextEntry(); + while (entry != null) { + String[] entryPathParts = entry.getName().split(File.separator); + validateStorePath(repository, entryPathParts); + importStoreByType(repository, tais, entryPathParts); + entry = tais.getNextEntry(); + } + } catch (IOException e) { + throw new ImportFailedException(ContextEntry.ContextBuilder.entity(repository).build(), "Could not import stores from metadata file.", e); + } + } + + private void importStoreByType(Repository repository, TarArchiveInputStream tais, String[] entryPathParts) { + String storeType = entryPathParts[1]; + if (storeType.equals(StoreType.DATA.getValue())) { + repositoryStoreImporter + .doImport(repository) + .importStore(new StoreEntryMetaData(StoreType.DATA, entryPathParts[2])) + .importEntry(entryPathParts[3], tais); + } else if (storeType.equals(StoreType.CONFIG.getValue())){ + repositoryStoreImporter + .doImport(repository) + .importStore(new StoreEntryMetaData(StoreType.CONFIG, "")) + .importEntry(entryPathParts[2], tais); + } else if(storeType.equals(StoreType.BLOB.getValue())) { + repositoryStoreImporter + .doImport(repository) + .importStore(new StoreEntryMetaData(StoreType.BLOB, entryPathParts[2])) + .importEntry(entryPathParts[3], tais); + } + } + + private void validateStorePath(Repository repository, String[] entryPathParts) { + if (!isValidStorePath(entryPathParts)) { + throw new ImportFailedException( + ContextEntry.ContextBuilder.entity(repository).build(), + "Invalid store path in metadata file" + ); + } + } + + private boolean isValidStorePath(String[] entryPathParts) { + //This prevents array out of bound exceptions + if (entryPathParts.length > 1) { + String storeType = entryPathParts[1]; + if (storeType.equals(StoreType.DATA.getValue()) || storeType.equals(StoreType.BLOB.getValue())) { + return entryPathParts.length == 4; + } + if (storeType.equals(StoreType.CONFIG.getValue())) { + return entryPathParts.length == 3; + } + } + // We only support config and data stores yet + return false; + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/BootstrapModule.java b/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/BootstrapModule.java index baa22fa801..83c95fa4ff 100644 --- a/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/BootstrapModule.java +++ b/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/BootstrapModule.java @@ -27,7 +27,6 @@ package sonia.scm.lifecycle.modules; import com.google.inject.AbstractModule; import com.google.inject.TypeLiteral; import com.google.inject.throwingproviders.ThrowingProviderBinder; -import lombok.RequiredArgsConstructor; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import sonia.scm.SCMContext; @@ -52,12 +51,16 @@ import sonia.scm.store.ConfigurationStoreFactory; import sonia.scm.store.DataStoreFactory; import sonia.scm.store.DefaultBlobDirectoryAccess; import sonia.scm.store.FileBlobStoreFactory; +import sonia.scm.store.FileRepositoryUpdateIterator; +import sonia.scm.store.FileStoreUpdateStepUtilFactory; import sonia.scm.store.JAXBConfigurationEntryStoreFactory; import sonia.scm.store.JAXBConfigurationStoreFactory; import sonia.scm.store.JAXBDataStoreFactory; import sonia.scm.store.JAXBPropertyFileAccess; import sonia.scm.update.BlobDirectoryAccess; import sonia.scm.update.PropertyFileAccess; +import sonia.scm.update.RepositoryUpdateIterator; +import sonia.scm.update.StoreUpdateStepUtilFactory; import sonia.scm.update.UpdateStepRepositoryMetadataAccess; import sonia.scm.update.V1PropertyDAO; import sonia.scm.update.xml.XmlV1PropertyDAO; @@ -105,6 +108,8 @@ public class BootstrapModule extends AbstractModule { bind(V1PropertyDAO.class, XmlV1PropertyDAO.class); bind(PropertyFileAccess.class, JAXBPropertyFileAccess.class); bind(BlobDirectoryAccess.class, DefaultBlobDirectoryAccess.class); + bind(RepositoryUpdateIterator.class, FileRepositoryUpdateIterator.class); + bind(StoreUpdateStepUtilFactory.class, FileStoreUpdateStepUtilFactory.class); bind(new TypeLiteral>() {}).to(new TypeLiteral() {}); } diff --git a/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/ScmServletModule.java b/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/ScmServletModule.java index 3293089196..4cd480cda4 100644 --- a/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/ScmServletModule.java +++ b/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/ScmServletModule.java @@ -95,6 +95,8 @@ import sonia.scm.security.DefaultSecuritySystem; import sonia.scm.security.LoginAttemptHandler; import sonia.scm.security.RepositoryPermissionProvider; import sonia.scm.security.SecuritySystem; +import sonia.scm.store.FileStoreExporter; +import sonia.scm.store.StoreExporter; import sonia.scm.template.MustacheTemplateEngine; import sonia.scm.template.TemplateEngine; import sonia.scm.template.TemplateEngineFactory; @@ -198,6 +200,7 @@ class ScmServletModule extends ServletModule { bind(NamespaceManager.class, DefaultNamespaceManager.class); bind(GroupCollector.class, DefaultGroupCollector.class); bind(CGIExecutorFactory.class, DefaultCGIExecutorFactory.class); + bind(StoreExporter.class, FileStoreExporter.class); // bind sslcontext provider bind(SSLContext.class).toProvider(SSLContextProvider.class); diff --git a/scm-webapp/src/main/resources/META-INF/scm/permissions.xml b/scm-webapp/src/main/resources/META-INF/scm/permissions.xml index df56e818be..db5ac5c201 100644 --- a/scm-webapp/src/main/resources/META-INF/scm/permissions.xml +++ b/scm-webapp/src/main/resources/META-INF/scm/permissions.xml @@ -47,6 +47,9 @@ repository:archive:* + + repository:read,export:* + namespace:permissionRead diff --git a/scm-webapp/src/main/resources/META-INF/scm/repository-permissions.xml b/scm-webapp/src/main/resources/META-INF/scm/repository-permissions.xml index 84aa849463..2f5c48b8d2 100644 --- a/scm-webapp/src/main/resources/META-INF/scm/repository-permissions.xml +++ b/scm-webapp/src/main/resources/META-INF/scm/repository-permissions.xml @@ -34,6 +34,7 @@ permissionRead permissionWrite archive + export * diff --git a/scm-webapp/src/main/resources/locales/de/plugins.json b/scm-webapp/src/main/resources/locales/de/plugins.json index 9e41cb0ad2..ee588f05f2 100644 --- a/scm-webapp/src/main/resources/locales/de/plugins.json +++ b/scm-webapp/src/main/resources/locales/de/plugins.json @@ -47,6 +47,12 @@ "displayName": "Repositories archivieren", "description": "Darf Repositories als \"archiviert\" und somit als schreibgeschützt markieren." } + }, + "read,export": { + "*": { + "displayName": "Repositories exportieren", + "description": "Darf alle Repositories exportieren." + } } }, "user": { @@ -149,6 +155,10 @@ "displayName": "Repository archivieren", "description": "Darf das Repository als \"archiviert\" und somit als schreibgeschützt markieren." }, + "export": { + "displayName": "Repository exportieren", + "description": "Darf das Repository exportieren." + }, "*": { "displayName": "Alle Repository Rechte", "description": "Darf im Repository Kontext alles ausführen. Dies beinhaltet alle Repository Berechtigungen." @@ -308,6 +318,14 @@ "3hSIlptme1": { "displayName": "Repository ist archiviert", "description": "Das Repository ist als \"archiviert\" markiert und darf nicht modifiziert werden." + }, + "67SM3DANZ1": { + "displayName": "Repository Export fehlgeschlagen", + "description": "Das Repository konnte nicht exportiert werden. Für weitere Informationen wenden Sie sich an Ihren Administrator oder schauen Sie im Log." + }, + "4hSNNTBiu1": { + "displayName": "Falscher Repository Typ", + "description": "Der gegebene Typ entspricht nicht dem Typen des Repositories." } }, "namespaceStrategies": { diff --git a/scm-webapp/src/main/resources/locales/en/plugins.json b/scm-webapp/src/main/resources/locales/en/plugins.json index 270560dade..5eee2e48a2 100644 --- a/scm-webapp/src/main/resources/locales/en/plugins.json +++ b/scm-webapp/src/main/resources/locales/en/plugins.json @@ -47,6 +47,12 @@ "displayName": "Archive repositories", "description": "May mark repositories as \"archived\" and therefore unmodifiable" } + }, + "read,export": { + "*": { + "displayName": "Export repositories", + "description": "May export all repositories" + } } }, "user": { @@ -149,6 +155,10 @@ "displayName": "archive repository", "description": "May mark the repository as \"archived\" and therefore unmodifiable" }, + "export": { + "displayName": "export repository", + "description": "May export the repository" + }, "*": { "displayName": "own repository", "description": "May change everything for the repository (includes all other permissions)" @@ -308,6 +318,14 @@ "3hSIlptme1": { "displayName": "Repository is archived", "description": "The repository is marked as \"archived\" and therefore must noch be modified." + }, + "67SM3DANZ1": { + "displayName": "Repository export failed", + "description": "The repository could not be exported. Contact your administrator or see the logs for more information." + }, + "4hSNNTBiu1": { + "displayName": "Wrong repository type", + "description": "The given type does not match the type of the repository." } }, "namespaceStrategies": { diff --git a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryRootResourceTest.java b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryRootResourceTest.java index 04f4c71e15..070e852e5d 100644 --- a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryRootResourceTest.java +++ b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryRootResourceTest.java @@ -44,6 +44,8 @@ import org.mockito.Mock; import sonia.scm.PageResult; import sonia.scm.config.ScmConfiguration; import sonia.scm.event.ScmEventBus; +import sonia.scm.importexport.FullScmRepositoryExporter; +import sonia.scm.importexport.FullScmRepositoryImporter; import sonia.scm.repository.CustomNamespaceStrategy; import sonia.scm.repository.NamespaceAndName; import sonia.scm.repository.NamespaceStrategy; @@ -73,6 +75,7 @@ import java.io.ByteArrayOutputStream; import java.io.File; import java.io.IOException; import java.io.InputStream; +import java.io.OutputStream; import java.io.OutputStreamWriter; import java.io.StringWriter; import java.io.UnsupportedEncodingException; @@ -147,6 +150,10 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { private Set strategies; @Mock private ScmEventBus eventBus; + @Mock + private FullScmRepositoryExporter fullScmRepositoryExporter; + @Mock + private FullScmRepositoryImporter fullScmRepositoryImporter; @Captor private ArgumentCaptor> filterCaptor; @@ -167,8 +174,8 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { super.manager = repositoryManager; RepositoryCollectionToDtoMapper repositoryCollectionToDtoMapper = new RepositoryCollectionToDtoMapper(repositoryToDtoMapper, resourceLinks); super.repositoryCollectionResource = new RepositoryCollectionResource(repositoryManager, repositoryCollectionToDtoMapper, dtoToRepositoryMapper, resourceLinks, repositoryInitializer); - super.repositoryImportResource = new RepositoryImportResource(repositoryManager, dtoToRepositoryMapper, serviceFactory, resourceLinks, eventBus); - super.repositoryExportResource = new RepositoryExportResource(repositoryManager, serviceFactory); + super.repositoryImportResource = new RepositoryImportResource(repositoryManager, dtoToRepositoryMapper, serviceFactory, resourceLinks, eventBus, fullScmRepositoryImporter); + super.repositoryExportResource = new RepositoryExportResource(repositoryManager, serviceFactory, fullScmRepositoryExporter); dispatcher.addSingletonResource(getRepositoryRootResource()); when(serviceFactory.create(any(Repository.class))).thenReturn(service); when(scmPathInfoStore.get()).thenReturn(uriInfo); @@ -186,7 +193,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldFailForNotExistingRepository() throws URISyntaxException { when(repositoryManager.get(any(NamespaceAndName.class))).thenReturn(null); - mockRepository("space", "repo"); + createRepository("space", "repo"); MockHttpRequest request = MockHttpRequest.get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/other"); MockHttpResponse response = new MockHttpResponse(); @@ -198,7 +205,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldFindExistingRepository() throws URISyntaxException, UnsupportedEncodingException { - mockRepository("space", "repo"); + createRepository("space", "repo"); when(configuration.getNamespaceStrategy()).thenReturn("CustomNamespaceStrategy"); MockHttpRequest request = MockHttpRequest.get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo"); @@ -212,7 +219,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldGetAll() throws URISyntaxException, UnsupportedEncodingException { - PageResult singletonPageResult = createSingletonPageResult(mockRepository("space", "repo")); + PageResult singletonPageResult = createSingletonPageResult(createRepository("space", "repo")); when(repositoryManager.getPage(any(), any(), eq(0), eq(10))).thenReturn(singletonPageResult); when(configuration.getNamespaceStrategy()).thenReturn("CustomNamespaceStrategy"); @@ -227,7 +234,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldCreateFilterForSearch() throws URISyntaxException { - PageResult singletonPageResult = createSingletonPageResult(mockRepository("space", "repo")); + PageResult singletonPageResult = createSingletonPageResult(createRepository("space", "repo")); when(repositoryManager.getPage(filterCaptor.capture(), any(), eq(0), eq(10))).thenReturn(singletonPageResult); when(configuration.getNamespaceStrategy()).thenReturn("CustomNamespaceStrategy"); @@ -244,7 +251,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldCreateFilterForNamespace() throws URISyntaxException { - PageResult singletonPageResult = createSingletonPageResult(mockRepository("space", "repo")); + PageResult singletonPageResult = createSingletonPageResult(createRepository("space", "repo")); when(repositoryManager.getPage(filterCaptor.capture(), any(), eq(0), eq(10))).thenReturn(singletonPageResult); when(configuration.getNamespaceStrategy()).thenReturn("CustomNamespaceStrategy"); @@ -261,7 +268,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldCreateFilterForNamespaceWithQuery() throws URISyntaxException { - PageResult singletonPageResult = createSingletonPageResult(mockRepository("space", "repo")); + PageResult singletonPageResult = createSingletonPageResult(createRepository("space", "repo")); when(repositoryManager.getPage(filterCaptor.capture(), any(), eq(0), eq(10))).thenReturn(singletonPageResult); when(configuration.getNamespaceStrategy()).thenReturn("CustomNamespaceStrategy"); @@ -295,7 +302,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldHandleUpdateForExistingRepository() throws Exception { - mockRepository("space", "repo"); + createRepository("space", "repo"); URL url = Resources.getResource("sonia/scm/api/v2/repository-test-update.json"); byte[] repository = Resources.toByteArray(url); @@ -314,7 +321,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldHandleUpdateForConcurrentlyChangedRepository() throws Exception { - mockRepository("space", "repo", 1337); + createRepository("space", "repo", 1337); URL url = Resources.getResource("sonia/scm/api/v2/repository-test-update.json"); byte[] repository = Resources.toByteArray(url); @@ -334,7 +341,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldHandleUpdateForExistingRepositoryForChangedNamespace() throws Exception { - mockRepository("wrong", "repo"); + createRepository("wrong", "repo"); URL url = Resources.getResource("sonia/scm/api/v2/repository-test-update.json"); byte[] repository = Resources.toByteArray(url); @@ -353,7 +360,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldHandleDeleteForExistingRepository() throws Exception { - mockRepository("space", "repo"); + createRepository("space", "repo"); MockHttpRequest request = MockHttpRequest.delete("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo"); MockHttpResponse response = new MockHttpResponse(); @@ -444,7 +451,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { @Test public void shouldCreateArrayOfProtocolUrls() throws Exception { - mockRepository("space", "repo"); + createRepository("space", "repo"); when(service.getSupportedProtocols()).thenReturn(of(new MockScmProtocol("http", "http://"), new MockScmProtocol("ssh", "ssh://"))); when(configuration.getNamespaceStrategy()).thenReturn("CustomNamespaceStrategy"); @@ -461,7 +468,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { public void shouldRenameRepository() throws Exception { String namespace = "space"; String name = "repo"; - Repository repository1 = mockRepository(namespace, name); + Repository repository1 = createRepository(namespace, name); when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository1); URL url = Resources.getResource("sonia/scm/api/v2/rename-repo.json"); @@ -679,7 +686,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { public void shouldMarkRepositoryAsArchived() throws Exception { String namespace = "space"; String name = "repo"; - Repository repository = mockRepository(namespace, name); + Repository repository = createRepository(namespace, name); when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository); MockHttpRequest request = MockHttpRequest @@ -697,7 +704,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { public void shouldRemoveArchiveMarkFromRepository() throws Exception { String namespace = "space"; String name = "repo"; - Repository repository = mockRepository(namespace, name); + Repository repository = createRepository(namespace, name); repository.setArchived(true); when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository); @@ -716,7 +723,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { public void shouldExportRepository() throws URISyntaxException { String namespace = "space"; String name = "repo"; - Repository repository = mockRepository(namespace, name); + Repository repository = createRepository(namespace, name, "svn"); when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository); mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE)); @@ -738,7 +745,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { public void shouldExportRepositoryCompressed() throws URISyntaxException { String namespace = "space"; String name = "repo"; - Repository repository = mockRepository(namespace, name); + Repository repository = createRepository(namespace, name, "svn"); when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository); mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE)); @@ -756,6 +763,28 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { verify(service).getBundleCommand(); } + @Test + public void shouldExportFullRepository() throws URISyntaxException { + String namespace = "space"; + String name = "repo"; + Repository repository = createRepository(namespace, name, "svn"); + when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository); + mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE)); + + BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class); + when(service.getBundleCommand()).thenReturn(bundleCommandBuilder); + + MockHttpRequest request = MockHttpRequest + .get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/svn/full"); + MockHttpResponse response = new MockHttpResponse(); + + dispatcher.invoke(request, response); + + assertEquals(SC_OK, response.getStatus()); + assertEquals("application/x-gzip", response.getOutputHeaders().get("Content-Type").get(0).toString()); + verify(fullScmRepositoryExporter).export(eq(repository), any(OutputStream.class)); + } + private void mockRepositoryHandler(Set cmds) { RepositoryHandler repositoryHandler = mock(RepositoryHandler.class); RepositoryType repositoryType = mock(RepositoryType.class); @@ -769,11 +798,17 @@ public class RepositoryRootResourceTest extends RepositoryTestBase { return new PageResult<>(singletonList(repository), 0); } - private Repository mockRepository(String namespace, String name) { - return mockRepository(namespace, name, 0); + private Repository createRepository(String namespace, String name, String type) { + Repository repository = createRepository(namespace, name); + repository.setType(type); + return repository; } - private Repository mockRepository(String namespace, String name, long lastModified) { + private Repository createRepository(String namespace, String name) { + return createRepository(namespace, name, 0); + } + + private Repository createRepository(String namespace, String name, long lastModified) { Repository repository = new Repository(); repository.setNamespace(namespace); repository.setName(name); diff --git a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryToRepositoryDtoMapperTest.java b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryToRepositoryDtoMapperTest.java index ac2eed2f56..ec8fadd10e 100644 --- a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryToRepositoryDtoMapperTest.java +++ b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryToRepositoryDtoMapperTest.java @@ -294,6 +294,16 @@ public class RepositoryToRepositoryDtoMapperTest { dto.getLinks().getLinkBy("export").get().getHref()); } + @Test + public void shouldCreateFullExportLink() { + Repository repository = createTestRepository(); + repository.setType("svn"); + RepositoryDto dto = mapper.map(repository); + assertEquals( + "http://example.com/base/v2/repositories/testspace/test/export/svn/full", + dto.getLinks().getLinkBy("fullExport").get().getHref()); + } + private ScmProtocol mockProtocol(String type, String protocol) { return new MockScmProtocol(type, protocol); } diff --git a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryTypeToRepositoryTypeDtoMapperTest.java b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryTypeToRepositoryTypeDtoMapperTest.java index 0041ef1332..fd528a2e8a 100644 --- a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryTypeToRepositoryTypeDtoMapperTest.java +++ b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/RepositoryTypeToRepositoryTypeDtoMapperTest.java @@ -26,6 +26,7 @@ package sonia.scm.api.v2.resources; import com.google.common.collect.ImmutableSet; import com.google.common.collect.Sets; +import de.otto.edison.hal.Link; import org.apache.shiro.subject.Subject; import org.apache.shiro.util.ThreadContext; import org.junit.After; @@ -39,6 +40,7 @@ import sonia.scm.repository.RepositoryType; import sonia.scm.repository.api.Command; import java.net.URI; +import java.util.List; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; @@ -114,19 +116,25 @@ public class RepositoryTypeToRepositoryTypeDtoMapperTest { } @Test - public void shouldAppendImportFromBundleLink() { + public void shouldAppendImportFromBundleLinkAndFullImportLink() { RepositoryType type = new RepositoryType("hk", "Hitchhiker", ImmutableSet.of(Command.UNBUNDLE)); when(subject.isPermitted("repository:create")).thenReturn(true); RepositoryTypeDto dto = mapper.map(type); + List links = dto.getLinks().getLinksBy("import"); + assertEquals(2, links.size()); assertEquals( "https://scm-manager.org/scm/v2/repositories/import/hk/bundle", - dto.getLinks().getLinkBy("import").get().getHref() + links.get(0).getHref() + ); + assertEquals( + "https://scm-manager.org/scm/v2/repositories/import/hk/full", + links.get(1).getHref() ); } @Test - public void shouldNotAppendImportFromBundleLinkIfCommandNotSupported() { + public void shouldNotAppendImportFromBundleLinkOrFullImportLinkIfCommandNotSupported() { when(subject.isPermitted("repository:create")).thenReturn(true); RepositoryTypeDto dto = mapper.map(type); assertFalse(dto.getLinks().getLinkBy("import").isPresent()); diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/EnvironmentInformationXmlGeneratorTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/EnvironmentInformationXmlGeneratorTest.java new file mode 100644 index 0000000000..d26d5e9c8e --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/EnvironmentInformationXmlGeneratorTest.java @@ -0,0 +1,80 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import com.google.common.collect.ImmutableList; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Answers; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.SCMContextProvider; +import sonia.scm.plugin.InstalledPlugin; +import sonia.scm.plugin.InstalledPluginDescriptor; +import sonia.scm.plugin.PluginManager; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class EnvironmentInformationXmlGeneratorTest { + + @Mock + SCMContextProvider contextProvider; + + @Mock + PluginManager pluginManager; + + @InjectMocks + EnvironmentInformationXmlGenerator generator; + + @Test + void shouldGenerateXmlContent() { + InstalledPluginDescriptor descriptor = mock(InstalledPluginDescriptor.class, Answers.RETURNS_DEEP_STUBS); + when(descriptor.getInformation().getName()).thenReturn("scm-exporter-test-plugin"); + when(descriptor.getInformation().getVersion()).thenReturn("42.0"); + when(contextProvider.getVersion()).thenReturn("2.13.0"); + InstalledPlugin installedPlugin = new InstalledPlugin(descriptor, null, null, null, false); + when(pluginManager.getInstalled()).thenReturn(ImmutableList.of(installedPlugin)); + + byte[] content = generator.generate(); + + String xmlContent = new String(content); + assertThat(xmlContent).contains( + "", + " \n" + + " \n" + + " scm-exporter-test-plugin\n" + + " 42.0\n" + + " \n" + + " ", + "2.13.0", + "", + ""); + } + +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryExporterTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryExporterTest.java new file mode 100644 index 0000000000..fd3adabb97 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryExporterTest.java @@ -0,0 +1,107 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.Answers; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryTestData; +import sonia.scm.repository.api.BundleCommandBuilder; +import sonia.scm.repository.api.RepositoryService; +import sonia.scm.repository.api.RepositoryServiceFactory; +import sonia.scm.repository.work.WorkdirProvider; + +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.IOException; +import java.io.OutputStream; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Collection; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.*; + +@ExtendWith(MockitoExtension.class) +class FullScmRepositoryExporterTest { + + private static final Repository REPOSITORY = RepositoryTestData.createHeartOfGold(); + + @Mock + private RepositoryServiceFactory serviceFactory; + @Mock(answer = Answers.RETURNS_DEEP_STUBS) + private RepositoryService repositoryService; + @Mock + private EnvironmentInformationXmlGenerator environmentGenerator; + @Mock + private RepositoryMetadataXmlGenerator metadataGenerator; + @Mock + private TarArchiveRepositoryStoreExporter storeExporter; + @Mock + private WorkdirProvider workdirProvider; + + @InjectMocks + private FullScmRepositoryExporter exporter; + + private Collection workDirsCreated = new ArrayList<>(); + + @BeforeEach + void initRepoService() { + when(serviceFactory.create(REPOSITORY)).thenReturn(repositoryService); + when(environmentGenerator.generate()).thenReturn(new byte[0]); + when(metadataGenerator.generate(REPOSITORY)).thenReturn(new byte[0]); + } + + @Test + void shouldExportEverythingAsTarArchive(@TempDir Path temp) throws IOException { + BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class); + when(repositoryService.getBundleCommand()).thenReturn(bundleCommandBuilder); + when(workdirProvider.createNewWorkdir()).thenAnswer(invocation -> createWorkDir(temp)); + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + exporter.export(REPOSITORY, baos); + + verify(storeExporter, times(1)).export(eq(REPOSITORY), any(OutputStream.class)); + verify(environmentGenerator, times(1)).generate(); + verify(metadataGenerator, times(1)).generate(REPOSITORY); + verify(bundleCommandBuilder, times(1)).bundle(any(OutputStream.class)); + workDirsCreated.forEach(wd -> assertThat(wd).doesNotExist()); + } + + private File createWorkDir(Path temp) throws IOException { + Path newWorkDir = temp.resolve("workDir-" + workDirsCreated.size()); + workDirsCreated.add(newWorkDir); + Files.createDirectories(newWorkDir); + return newWorkDir.toFile(); + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryImporterTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryImporterTest.java new file mode 100644 index 0000000000..97f9dd7ce1 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryImporterTest.java @@ -0,0 +1,113 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import com.google.common.io.Files; +import com.google.common.io.Resources; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.Answers; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryManager; +import sonia.scm.repository.RepositoryTestData; +import sonia.scm.repository.api.ImportFailedException; +import sonia.scm.repository.api.RepositoryService; +import sonia.scm.repository.api.RepositoryServiceFactory; +import sonia.scm.repository.api.UnbundleCommandBuilder; + +import java.io.File; +import java.io.FileInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Path; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.lenient; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class FullScmRepositoryImporterTest { + + private static final Repository REPOSITORY = RepositoryTestData.createHeartOfGold(); + + @Mock + private RepositoryServiceFactory serviceFactory; + @Mock(answer = Answers.RETURNS_DEEP_STUBS) + private RepositoryService service; + @Mock + private UnbundleCommandBuilder unbundleCommandBuilder; + @Mock + private RepositoryManager repositoryManager; + @Mock + private ScmEnvironmentCompatibilityChecker compatibilityChecker; + @Mock + private TarArchiveRepositoryStoreImporter storeImporter; + + @InjectMocks + private FullScmRepositoryImporter fullImporter; + + @BeforeEach + void initRepositoryService() { + lenient().when(serviceFactory.create(REPOSITORY)).thenReturn(service); + lenient().when(service.getUnbundleCommand()).thenReturn(unbundleCommandBuilder); + } + + @Test + void shouldNotImportRepositoryIfFileNotExists(@TempDir Path temp) throws IOException { + File emptyFile = new File(temp.resolve("empty").toString()); + Files.touch(emptyFile); + assertThrows(ImportFailedException.class, () -> fullImporter.importFromStream(REPOSITORY, new FileInputStream(emptyFile))); + } + + @Test + void shouldFailIfScmEnvironmentIsIncompatible() { + when(compatibilityChecker.check(any())).thenReturn(false); + + assertThrows( + ImportFailedException.class, + () -> fullImporter.importFromStream(REPOSITORY, Resources.getResource("sonia/scm/repository/import/scm-import.tar.gz").openStream()) + ); + } + + @Test + void shouldImportScmRepositoryArchive() throws IOException { + when(compatibilityChecker.check(any())).thenReturn(true); + when(repositoryManager.create(eq(REPOSITORY), any())).thenReturn(REPOSITORY); + + Repository repository = fullImporter.importFromStream(REPOSITORY, Resources.getResource("sonia/scm/repository/import/scm-import.tar.gz").openStream()); + assertThat(repository).isEqualTo(REPOSITORY); + verify(storeImporter).importFromTarArchive(eq(REPOSITORY), any(InputStream.class)); + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/RepositoryMetadataXmlGeneratorTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/RepositoryMetadataXmlGeneratorTest.java new file mode 100644 index 0000000000..8f97bd3d06 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/RepositoryMetadataXmlGeneratorTest.java @@ -0,0 +1,72 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import org.junit.jupiter.api.Test; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryPermission; +import sonia.scm.repository.RepositoryTestData; + +import static org.assertj.core.api.Assertions.assertThat; + +class RepositoryMetadataXmlGeneratorTest { + + private final static Repository REPOSITORY = RepositoryTestData.createHeartOfGold("git"); + private RepositoryMetadataXmlGenerator generator = new RepositoryMetadataXmlGenerator(); + + @Test + void shouldCreateMetadataWithRepositoryType() { + byte[] metadata = generator.generate(REPOSITORY); + + assertThat(new String(metadata)).contains("git"); + } + + @Test + void shouldCreateMetadataWithRepositoryNamespaceAndName() { + byte[] metadata = generator.generate(REPOSITORY); + + assertThat(new String(metadata)).contains("hitchhiker"); + assertThat(new String(metadata)).contains("HeartOfGold"); + } + + @Test + void shouldCreateMetadataWithRepositoryContactAndDescription() { + byte[] metadata = generator.generate(REPOSITORY); + + assertThat(new String(metadata)).contains("zaphod.beeblebrox@hitchhiker.com"); + assertThat(new String(metadata)).contains("Heart of Gold is the first prototype ship to successfully utilise the revolutionary Infinite Improbability Drive"); + } + + @Test + void shouldCreateMetadataWithRepositoryPermissions() { + REPOSITORY.addPermission(new RepositoryPermission("arthur", "READ", false)); + + byte[] metadata = generator.generate(REPOSITORY); + + assertThat(new String(metadata)).contains(""); + assertThat(new String(metadata)).contains("arthur"); + assertThat(new String(metadata)).contains("READ"); + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/ScmEnvironmentCompatibilityCheckerTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/ScmEnvironmentCompatibilityCheckerTest.java new file mode 100644 index 0000000000..8afaa3f6b2 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/ScmEnvironmentCompatibilityCheckerTest.java @@ -0,0 +1,128 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import com.google.common.collect.ImmutableList; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Answers; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.SCMContextProvider; +import sonia.scm.plugin.InstalledPlugin; +import sonia.scm.plugin.PluginManager; + +import java.util.Collections; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.*; + +@ExtendWith(MockitoExtension.class) +class ScmEnvironmentCompatibilityCheckerTest { + + @Mock + private PluginManager pluginManager; + @Mock + private SCMContextProvider scmContextProvider; + + @InjectMocks + private ScmEnvironmentCompatibilityChecker checker; + + @BeforeEach + void preparePluginManager() { + InstalledPlugin first = mockPlugin("scm-first-plugin", "1.0.0"); + InstalledPlugin second = mockPlugin("scm-second-plugin", "1.1.0"); + lenient().when(pluginManager.getInstalled()).thenReturn(ImmutableList.of(first, second)); + } + + @Test + void shouldReturnTrueIfEnvironmentIsCompatible() { + when(scmContextProvider.getVersion()).thenReturn("2.0.0"); + ImmutableList plugins = ImmutableList.of( + new EnvironmentPluginDescriptor("scm-first-plugin", "1.0.0"), + new EnvironmentPluginDescriptor("scm-second-plugin", "1.1.0") + ); + ScmEnvironment env = createScmEnvironment("2.0.0", "linux", "64", plugins); + + boolean compatible = checker.check(env); + + assertThat(compatible).isTrue(); + } + + @Test + void shouldReturnFalseIfCoreVersionIncompatible() { + when(scmContextProvider.getVersion()).thenReturn("2.0.0"); + ScmEnvironment env = createScmEnvironment("2.13.0", "linux", "64", Collections.emptyList()); + + boolean compatible = checker.check(env); + + assertThat(compatible).isFalse(); + } + + @Test + void shouldReturnFalseIfPluginIsIncompatible() { + when(scmContextProvider.getVersion()).thenReturn("2.13.0"); + ImmutableList plugins = ImmutableList.of(new EnvironmentPluginDescriptor("scm-second-plugin", "1.2.0")); + ScmEnvironment env = createScmEnvironment("2.13.0", "linux", "64", plugins); + + boolean compatible = checker.check(env); + + assertThat(compatible).isFalse(); + } + + @Test + void shouldReturnTrueIfPluginDoNotMatch() { + when(scmContextProvider.getVersion()).thenReturn("2.13.0"); + ImmutableList plugins = ImmutableList.of(new EnvironmentPluginDescriptor("scm-third-plugin", "42.0.0")); + ScmEnvironment env = createScmEnvironment("2.13.0", "linux", "64", plugins); + + boolean compatible = checker.check(env); + + assertThat(compatible).isTrue(); + } + + private InstalledPlugin mockPlugin(String name, String version) { + InstalledPlugin plugin = mock(InstalledPlugin.class, Answers.RETURNS_DEEP_STUBS); + lenient().when(plugin.getDescriptor().getInformation().getName()).thenReturn(name); + lenient().when(plugin.getDescriptor().getInformation().getVersion()).thenReturn(version); + return plugin; + } + + private ScmEnvironment createScmEnvironment(String coreVersion, String os, String arch, List pluginList) { + ScmEnvironment scmEnvironment = new ScmEnvironment(); + scmEnvironment.setCoreVersion(coreVersion); + scmEnvironment.setOs(os); + scmEnvironment.setArch(arch); + + EnvironmentPluginsDescriptor environmentPluginsDescriptor = new EnvironmentPluginsDescriptor(); + environmentPluginsDescriptor.setPlugin(pluginList); + scmEnvironment.setPlugins(environmentPluginsDescriptor); + return scmEnvironment; + } + +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/TarArchiveRepositoryStoreExporterTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/TarArchiveRepositoryStoreExporterTest.java new file mode 100644 index 0000000000..7efe1abfe6 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/TarArchiveRepositoryStoreExporterTest.java @@ -0,0 +1,107 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import com.google.common.collect.ImmutableList; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryTestData; +import sonia.scm.store.ExportableStore; +import sonia.scm.store.Exporter; +import sonia.scm.store.StoreEntryMetaData; +import sonia.scm.store.StoreExporter; +import sonia.scm.store.StoreType; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.OutputStream; +import java.util.Collections; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class TarArchiveRepositoryStoreExporterTest { + + private static final Repository REPOSITORY = RepositoryTestData.create42Puzzle(); + + @Mock + private StoreExporter storeExporter; + + @InjectMocks + private TarArchiveRepositoryStoreExporter tarArchiveRepositoryStoreExporter; + + @Test + void shouldExportNothingIfNoStoresFound() throws IOException { + when(storeExporter.listExportableStores(REPOSITORY)).thenReturn(Collections.emptyList()); + OutputStream outputStream = mock(OutputStream.class); + tarArchiveRepositoryStoreExporter.export(REPOSITORY, outputStream); + + verify(outputStream, never()).write(any()); + } + + @Test + void shouldWriteDataIfRepoStoreFound() { + when(storeExporter.listExportableStores(REPOSITORY)).thenReturn(ImmutableList.of(new TestExportableStore())); + ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); + tarArchiveRepositoryStoreExporter.export(REPOSITORY, outputStream); + + String content = outputStream.toString(); + assertThat(content).isNotBlank(); + } + + @Test + void shouldExportFromFoundRepoStore() throws IOException { + ExportableStore exportableStore = mock(ExportableStore.class); + when(storeExporter.listExportableStores(REPOSITORY)).thenReturn(ImmutableList.of(exportableStore)); + OutputStream outputStream = mock(OutputStream.class); + tarArchiveRepositoryStoreExporter.export(REPOSITORY, outputStream); + + verify(exportableStore).export(any(Exporter.class)); + } + + static class TestExportableStore implements ExportableStore { + + @Override + public StoreEntryMetaData getMetaData() { + return new StoreEntryMetaData(StoreType.CONFIG, "puzzle42"); + } + + @Override + public void export(Exporter exporter) throws IOException { + try (OutputStream stream = exporter.put("testStore", 0)) { + stream.flush(); + } + } + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporterTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporterTest.java new file mode 100644 index 0000000000..9f07bd19d1 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporterTest.java @@ -0,0 +1,79 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.importexport; + +import com.google.common.io.Resources; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Answers; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryTestData; +import sonia.scm.repository.api.ImportFailedException; +import sonia.scm.store.RepositoryStoreImporter; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; + +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; + +@ExtendWith(MockitoExtension.class) +class TarArchiveRepositoryStoreImporterTest { + + private final Repository repository = RepositoryTestData.createHeartOfGold(); + + @Mock(answer = Answers.RETURNS_DEEP_STUBS) + private RepositoryStoreImporter repositoryStoreImporter; + + @InjectMocks + private TarArchiveRepositoryStoreImporter tarArchiveRepositoryStoreImporter; + + @Test + void shouldDoNothingIfNoEntries() { + ByteArrayInputStream bais = new ByteArrayInputStream("".getBytes()); + tarArchiveRepositoryStoreImporter.importFromTarArchive(repository, bais); + verify(repositoryStoreImporter, never()).doImport(any(Repository.class)); + } + + @Test + void shouldImportEachEntry() throws IOException { + InputStream inputStream = Resources.getResource("sonia/scm/repository/import/scm-metadata.tar").openStream(); + tarArchiveRepositoryStoreImporter.importFromTarArchive(repository, inputStream); + verify(repositoryStoreImporter, times(2)).doImport(repository); + } + + @Test + void shouldThrowImportFailedExceptionIfInvalidStorePath() throws IOException { + InputStream inputStream = Resources.getResource("sonia/scm/repository/import/scm-metadata_invalid.tar").openStream(); + assertThrows(ImportFailedException.class, () -> tarArchiveRepositoryStoreImporter.importFromTarArchive(repository, inputStream)); + } +} diff --git a/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-import.tar.gz b/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-import.tar.gz new file mode 100644 index 0000000000..bcd6e85167 Binary files /dev/null and b/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-import.tar.gz differ diff --git a/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-metadata.tar b/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-metadata.tar new file mode 100644 index 0000000000..06f486ff99 Binary files /dev/null and b/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-metadata.tar differ diff --git a/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-metadata_invalid.tar b/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-metadata_invalid.tar new file mode 100644 index 0000000000..134adbbc83 Binary files /dev/null and b/scm-webapp/src/test/resources/sonia/scm/repository/import/scm-metadata_invalid.tar differ