diff --git a/docs/en/development/storage.md b/docs/en/development/storage.md new file mode 100644 index 0000000000..ee296a9e8c --- /dev/null +++ b/docs/en/development/storage.md @@ -0,0 +1,452 @@ +--- +title: Architecture of the Persistence Layer with Embedded SQLite +--- + +# Introduction + +In SCM-Manager, data outside the actual repositories has been stored in XML files since version 2.0. For this purpose, a +persistence layer was developed that allows various types of data to be stored. It is also possible to choose whether +data should be stored globally or associated with a repository. + +This type of storage has generally proven effective and offers several advantages (data is stored with the repository, +easy troubleshooting, simple backup, etc.). However, with large amounts of data or frequently changing data, this +architecture reaches its limits. Several optimizations have been made (e.g., through caches), but the fundamental +limitations remain. In particular, searches are difficult because many files have to be read and processed. + +It was therefore necessary to look for new possibilities. The fundamental advantages of SCM-Manager should remain, +especially the easy installation, simple operation, the ability to easily transfer repositories between different +instances through export and import, and not least the easy use of the persistence layer by plugins. + +It quickly became clear that using a database system would be a sensible alternative. However, it was uncertain whether it +should be a "classic" database or a NoSQL database. XML storage had proven to be very helpful, as the only prerequisite for +a persistent data type was the use of JaxB annotations. However, a widely recognized technology should also be used. + +The choice finally fell on SQLite. This system is available for almost every platform and databases can be used " +embedded", so no separate server process is needed. The deciding factor was the performance, which is also present with +embedded JSON data. + +The next point to clarify was how the abstraction should look. It was clear from the beginning that plugins should not +directly access databases via SQL. Rather, the API for persistence should be oriented towards the XML-based solution. + +The following sections introduce the most important concepts. + +# Important Components of the Architecture + +## Objectives + +The following aspects were decisive in introducing the new persistence layer: + +- Primarily, an alternative should be developed for the existing Data Store, as most data is stored there. + Configuration stores are unlikely to pose a performance problem. +- The specific choice of database should not be noticeable in the API, so that a change of the specific technology + remains possible in principle. +- The API for using the new persistence layer should be as similar as possible to the existing API. In particular, it + should not be necessary to create a mapping from the entities to be stored to a database schema (like an OR mapping). +- As an extension to the XML layer, it should be possible to store data not only globally or related to a repository or + namespace but also to allow other hierarchies (even those that may only arise through plugins and cannot be + anticipated in the API). +- Unlike XML persistence, queries should be possible that span multiple entries. Additionally, for entities assigned to + individual repositories, queries should also be able to cross repository boundaries. +- The API should, especially for queries, provide the best possible options, such as which fields can be searched and + which operators are possible for these fields. +- The previous functions such as export and import with metadata, update steps, and automatic data cleanup, + for example when deleting a repository, should remain available. +- A switch from the old to the new persistence layer should be as simple as possible. +- The principle "All data belonging to a repository is in a single directory" can be relaxed for performance reasons. + +## Annotations and API Generation + +To achieve the best possible "Developer Experience", code generation is used. This is triggered by using a new +annotation for persistent entities. + +### The "Queryable Type" + +The `@QueryableType` annotation is the central element of the persistence architecture. It allows classes to be marked +for use in SQL-based database queries. In the annotation, so-called parent classes can be listed to which the entities +should later belong. For a repository-related type, the `Repository` class must be entered here. Multiple classes can +also be specified here in the sense of a hierarchy (e.g., a comment can belong to a pull request, which in turn belongs +to a repository). + +For such marked classes, additional classes are automatically generated: a Store Factory and a class with constants for +the individual fields that can be used in queries (the "Query Fields"). + +### Store Factories and Stores + +The generated Store Factories are similar to the known `DataStoreFactory`. Unlike the generic `DataStoreFactory`, +however, specific methods are created here based on the parent classes mentioned in the annotation. + +To create and change data, specific IDs must be specified to access the store if parent classes have been defined. This +store implements the known Store API (the `DataStore` interface), so no adjustments are needed in the application. + +### Queryable Store + +For more advanced queries that also extend beyond the boundaries of the parent classes, there is a new store with a new +API, the `QueryableStore`. This offers a `query` function in which conditions can be specified and a query can be +started. The conditions are based on the generated Queryable Fields described below. + +### Queryable Mutable Store + +To store, delete, and change data, a new store with the `QueryableMutableStore` API is used. This API +extends `QueryableStore` and `DataStore` to allow both queries and changes to stored objects. In contrast to the +pure Queryable Store, it is mandatory to specify all parents to create a mutable store. This is needed so that new +entities can be assigned to the correct parent(s). + +### Queryable Maintenance Store + +The `QueryableMaintenanceStore` is responsible for maintenance tasks, +such as deleting all data of a specific type or updating stored JSON data. + +- One use case is deleting a parent ID (e.g., repository ID): + + For example, if a repository is deleted, all entries with this ID as the parent ID must also be removed. This automatic + cleanup is ensured by the `QueryableMaintenanceStore`. + With the `clear()` function, all entries of a specific type can be specifically removed. + +- Another use case is "update steps". Here, all entries of a store can be iterated and potentially updated or deleted using + the `QueryableMaintenanceStore`. + +### Queryable Fields + +The individually generated Queryable Fields for a "Queryable Type" are a collection of constants that can be used to +define conditions for queries over Queryable Stores. For all attributes of the Queryable Type with supported data types, +a corresponding constant is generated. These offer functions for operators such as equality, greater and less for scalar +values, or "contains" for collections, depending on the data type. + +The generated store factories described in the previous section restrict the usable queryable fields per generic to +prevent incorrect queries from being created. + +### Queryable Type Annotation Processor + +The `QueryableTypeAnnotationProcessor` is an annotation processor that automatically generates SQL-related classes +during compilation. It identifies classes annotated with `@QueryableType` and creates corresponding `QueryField` classes +and Store Factories. + +Functions: + +- Identification of classes annotated with `@QueryableType` +- Generation of Query Field classes and Store Factories + +## Implementation in the Database + +When the SCM-Manager starts, an embedded SQLite database is set up. This is stored in the `scm.db` file in the SCM home +directory. Additionally, during startup, a table is created in the database for each queryable type if it does not already exist. +Each table includes the following columns: + +- A column for the ID of each parent level +- A column for the ID of the actual entity +- A column containing the entity converted to JSON + +### Rules for the Database Structure + +- The existing table structure must not be changed. +- No new parent classes (parents) may be added to or removed from an existing entity. +- The JSON data within the existing column may be updated to make changes to the stored entities. + +These restrictions ensure that the integrity of the database structure is maintained and migrations can be performed +without manual adjustments to the schema definition. + +### Table Creation with the TableCreator + +The `TableCreator` class is responsible for creating and validating the table structure. It checks whether a table +exists and whether the required columns (ID, JSON, and specific columns for the parents) are present. + +The implementation ensures that only consistent table structures are created and used. + +### Implementation of StoreFactory and Stores + +#### SQLiteQueryableStoreFactory + +The `SQLiteQueryableStoreFactory` class is the concrete implementation of `QueryableStoreFactory` for SQLite databases. + +Functions: + +- Management of SQLite database connections: + - Connects the application to the SQLite database (`scm.db`). + - Ensures that the connection is correctly opened and closed. +- Table initialization: + - Tables are automatically created based on the metadata of `@QueryableType`. +- Creation of stores: + - Supports both reading (`QueryableStore`) and writing (`QueryableMutableStore`) stores as well as stores for + maintenance (`QueryableMaintenanceStore`). + +#### SQLiteQueryableStore + +`SQLiteQueryableStore` is a generic implementation of `QueryableStore` that abstracts SQL logic and seamlessly +integrates into the persistence architecture. + +Purpose and scope: + +- Abstraction of SQL logic: + - Developers define queries in an object-oriented manner without having to write SQL directly. +- Integration with SQLite: + - Uses a JDBC connection to perform database operations. +- Data management: + - Supports reading queries on persisted data defined by annotations such as `@QueryableType`. +- Architecture and operation: + - Metadata integration: + + Uses `QueryableTypeDescriptor` to interpret table structure and relationships. + + *Note:* The parents of an already existing `QueryableType` must not be changed (new ones added or old ones removed) + as this would differ from the existing database structure and could lead to errors. + Declarative queries: Queries are created and internally translated into SQL. Results are mapped to objects of type + T. + +#### SQLiteStoreMetadataProvider + +The `SQLiteStoreMetaDataProvider` class serves as a provider of metadata for stored types within the SQLite database. It +manages the mapping of stored entity types to their respective parent types and provides mechanisms for querying this +information. The use case is to be able to recognize which tables are repository-related, i.e., which tables have a +repository as a parent. + +Functions: + +- Loading metadata: + + When initializing, all types annotated with `@QueryableType` are loaded and registered. + The information comes from the `PluginLoader` and is organized based on the specified parent classes. +- Management of the type hierarchy: + + Stores the mapping between parent types and their subordinate types in a map. +- Retrieval of types based on parent classes: + + Provides a method for querying all entity types associated with a specific parent class. + Uses a mapping list (`Map, Collection>>`) to enable efficient searching for stored types. + +This class is essential for the correct management of stored data types in the SQLite database and ensures that the data +hierarchy can be correctly built and queried. + +#### StoreDeletionNotifier + +The `StoreDeletionNotifier` interface serves as an extension point (`@ExtensionPoint`) to notify components about the +deletion of persisted objects. + +Functions: + +- Registration of deletion handlers: + + Allows the registration of `DeletionHandler` instances that should be notified when a stored object is deleted. +- Notification of deleted entities: + + `DeletionHandler` can receive deletion events and react to them. + + Supports both single and multiple objects to be deleted. + +Inner components: + +- `DeletionHandler` + Is notified when an object is removed from the store. + +This interface is essential to ensure consistent management of deleted entities and can be used, for example, to remove +dependent data or perform actions after an object is deleted from the store. + +## Testability + +To support unit tests, there is an extension for JUnit Jupiter, the `QueryableStoreExtension`. In a unit test, this must +be specified in a JUnit extension annotation. Additionally, the test class must be annotated +with `QueryableStoreExtension#QueryableTypes` to specify which types are needed in the test. Subsequently, it is +possible to obtain store factories via parameters to test methods (or also to methods annotated with `@BeforeEach`). + +# Examples + +## Using the New Queryable Store API + +First, a data type must be marked as a "Queryable Type": + +```java +import lombok.Data; +import sonia.scm.store.QueryableType; + +@Data +@QueryableType +public class MyEntity { + private String id; + private String name; + private String alias; + private int age; + private List tags; +} +``` + +In this example, the entity has no relation to parent elements. The `@QueryableType` annotation is sufficient to store +the entity in the database. During compilation, the following classes are automatically generated: + +- `MyEntityQueryFields`: Constants for the fields that can be used in queries +- `MyEntityStoreFactory`: Factory for accessing the store + +Using these classes, data can then be stored and queried as shown in the following example: + +```java +public class Demo { + + private final MyEntityStoreFactory storeFactory; + + @Inject + public Demo(MyEntityStoreFactory storeFactory) { + this.storeFactory = storeFactory; + } + + public String create(String name, int age, List tags) { + MyEntity entity = new MyEntity(); + entity.setName(name); + entity.setAge(age); + entity.setTags(tags); + + QueryableMutableStore store = storeFactory.getMutable(); + return store.put(entity); + } + + public MyEntity readById(String id) { + QueryableMutableStore store = storeFactory.getMutable(); + return store.get(id); + } + + public Collection findByAge(int age) { + QueryableStore store = storeFactory.get(); + return store.query(MyEntityQueryFields.AGE.eq(age)).findAll(); + } + + public Collection findByName(String name) { + QueryableStore store = storeFactory.get(); + return store.query( + Conditions.or( + MyEntityQueryFields.NAME.eq(name), + MyEntityQueryFields.ALIAS.eq(name) + ) + ).findAll(); + } + + public Collection findByTag(String tag) { + QueryableStore store = storeFactory.get(); + return store.query(MyEntityQueryFields.TAGS.contains(tag)).findAll(); + } +} +``` + +## Using the Queryable Store API with Parent Element + +Consider the following example with a parent element where we want to store multiple contacts for a user: + +```java +@Data +@QueryableType(User.class) +public class Contact { + private String mail; +} +``` + +For entities with parent elements, queries can be made both for specific parents and across all parents. + +```java +public class Demo { + + private final ContactStoreFactory storeFactory; + + @Inject + public Demo(ContactStoreFactory storeFactory) { + this.storeFactory = storeFactory; + } + + public void addContact(User user, String mail) { + QueryableMutableStore store = storeFactory.getMutable(user); + Contact contact = new Contact(); + contact.setMail(mail); + store.put(contact); + } + + /** Get contact for a single user. */ + public Collection getContacts(User user) { + QueryableMutableStore store = storeFactory.getMutable(user); + return store.getAll().values(); + } + + /** Get all contacts for all users. */ + public Collection getAllContacts() { + QueryableStore store = storeFactory.getOverall(); + return store.query().findAll(); + } +} +``` + +In this example, all `Contact` entries will be deleted, when the related `User` is deleted. This works out-of-the-box +for all entities whose top level parent is a `User`, a `Group`, or a `Repository`. You can build this behavior for your +own parent types by implementing a `StoreDeletionNotifier` as an extension. Best take a look at the `GroupDeletionNotifier` +for an example: + +```java +@Extension +public class GroupDeletionNotifier implements StoreDeletionNotifier { + private DeletionHandler handler; + + @Override + public void registerHandler(DeletionHandler handler) { + this.handler = handler; + } + + @Subscribe(referenceType = ReferenceType.STRONG) + public void onDelete(GroupEvent event) { + if (handler != null && event.getEventType() == HandlerEventType.DELETE) { + handler.notifyDeleted(Group.class, event.getItem().getId()); + } + } +} +``` + +## Update Steps + +Update steps can be used to update data in the database. The following example shows how to update all entities of a +specific type. For this let's assume, that we want to add a `type` field to the `Contact` entity from the previous +example: + +```java +@Data +@QueryableType(User.class) +public class Contact { + private String mail; + private String type; +} +``` + +The following update step can be used to add the `type` field to all `Contact` entities: + +```java +@Extension +public class AddTypeToContactsUpdateStep implements UpdateStep { + + private final StoreUpdateStepUtilFactory updateStepUtilFactory; + + @Inject + public AddTypeToContactsUpdateStep(StoreUpdateStepUtilFactory updateStepUtilFactory) { + this.updateStepUtilFactory = updateStepUtilFactory; + } + + @Override + public void doUpdate() { + try (MaintenanceIterator iter = updateStepUtilFactory.forQueryableType(Contact.class).iterateAll()) { + while(iter.hasNext()) { + MaintenanceStoreEntry entry = iter.next(); + Contact contact = entry.get(); + contact.setType("personal"); + entry.update(contact); + } + } + } + + @Override + public Version getTargetVersion() { + return Version.parse("2.0.0"); + } + + @Override + public String getAffectedDataType() { + return "userContacts"; + } +} +``` + +Please note that the iterator from the `StoreUpdateStepUtilFactory` has to be closed after usage. This is done best with +a try-with-resources block like in the example above. + +If the new entity differs in a significant way so that the old stored data can no longer be read from the store using +the new entity, you can use the method `entry#getAs(Class)` with a class that matches the old structure of the entity +and use this to create a new entity that can be stored with the new structure. diff --git a/gradle/changelog/queryable.yaml b/gradle/changelog/queryable.yaml new file mode 100644 index 0000000000..8f6fd04bb3 --- /dev/null +++ b/gradle/changelog/queryable.yaml @@ -0,0 +1,2 @@ +- type: added + description: New store API with enhanced query options backed by SQLite diff --git a/gradle/dependencies.gradle b/gradle/dependencies.gradle index 775935b374..65bc5c3c1f 100644 --- a/gradle/dependencies.gradle +++ b/gradle/dependencies.gradle @@ -15,6 +15,7 @@ ext { bouncycastleVersion = '2.73.6' jettyVersion = '11.0.24' luceneVersion = '8.11.4' + sqliteVersion = '3.49.1.0' junitJupiterVersion = '5.10.3' hamcrestVersion = '3.0' @@ -193,6 +194,9 @@ ext { // metrics micrometerCore: "io.micrometer:micrometer-core:${micrometerVersion}", - micrometerExtra: "io.github.mweirauch:micrometer-jvm-extras:0.2.2" + micrometerExtra: "io.github.mweirauch:micrometer-jvm-extras:0.2.2", + + // SQLite + sqlite: "org.xerial:sqlite-jdbc:${sqliteVersion}" ] } diff --git a/scm-annotation-processor/build.gradle b/scm-annotation-processor/build.gradle index a9fb6b51ca..3aa51e8be5 100644 --- a/scm-annotation-processor/build.gradle +++ b/scm-annotation-processor/build.gradle @@ -41,7 +41,24 @@ dependencies { // utils implementation libraries.guava + implementation "com.google.auto:auto-common:1.2.2" + + implementation 'com.squareup:javapoet:1.13.0' + + testImplementation "com.google.testing.compile:compile-testing:0.21.0" + testImplementation libraries.junitJupiterApi + testImplementation libraries.junitJupiterEngine + testImplementation libraries.assertj + // service registration compileOnly libraries.metainfServices annotationProcessor libraries.metainfServices + +} + +test { + // See: https://github.com/google/compile-testing/issues/222 + jvmArgs("--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED") + jvmArgs("--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED") + jvmArgs("--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED") } diff --git a/scm-annotations/src/main/java/sonia/scm/store/QueryableType.java b/scm-annotations/src/main/java/sonia/scm/store/QueryableType.java new file mode 100644 index 0000000000..10248530b5 --- /dev/null +++ b/scm-annotations/src/main/java/sonia/scm/store/QueryableType.java @@ -0,0 +1,56 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import sonia.scm.plugin.PluginAnnotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * This annotation is used to mark a class as queryable type. Classes annotated with this annotation can be stored in + * a store and later be queried more efficiently and with more flexibility than with a simple key-value store. + *
+ * If the annotation is used without any parameters, the class name is used as the name of the queryable type and + * the objects of this class will be stored with ids (either given or generated by the store) independently of any + * other (parent) objects. If the objects are related to other objects, the parent objects can be specified with the + * {@link #value()} parameter. The parent objects are used to create a hierarchy of objects (formerly it only was + * possible to store objects related to repositories; with this annotation it is possible to use other objects as + * parents, too, like for instance users). + * + * @since 3.7.0 + */ +@Documented +@Target(ElementType.TYPE) +@PluginAnnotation("queryable-type") +@Retention(RetentionPolicy.RUNTIME) +public @interface QueryableType { + /** + * The parent types of the queryable type. The parent objects are used to create a hierarchy of objects. If no parent + * types are specified, the type is stored independently of any other objects. + */ + Class[] value() default {}; + + /** + * This can be used to specify a name for the queryable type. If no name is specified, the class name is used as the + * name of the queryable type. + */ + String name() default ""; +} diff --git a/scm-core-annotation-processor/build.gradle b/scm-core-annotation-processor/build.gradle new file mode 100644 index 0000000000..d96ee00eef --- /dev/null +++ b/scm-core-annotation-processor/build.gradle @@ -0,0 +1,50 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +plugins { + id 'java-library' + id 'org.scm-manager.java' +} + +dependencies { + implementation platform(project(':')) + implementation project(':scm-annotations') + implementation project(':scm-core') + + implementation "com.google.auto:auto-common:1.2.2" + + implementation 'com.squareup:javapoet:1.13.0' + + compileOnly libraries.lombok; + annotationProcessor libraries.lombok; + + testImplementation "com.google.testing.compile:compile-testing:0.21.0" + testImplementation libraries.junitJupiterApi + testImplementation libraries.junitJupiterParams + testImplementation libraries.junitJupiterEngine + testImplementation libraries.assertj + + // service registration + compileOnly libraries.metainfServices + annotationProcessor libraries.metainfServices +} + +test { + // See: https://github.com/google/compile-testing/issues/222 + jvmArgs("--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED") + jvmArgs("--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED") + jvmArgs("--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED") +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/AnnotationHelper.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/AnnotationHelper.java new file mode 100644 index 0000000000..aa99b79b7e --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/AnnotationHelper.java @@ -0,0 +1,34 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import javax.lang.model.element.AnnotationMirror; +import javax.lang.model.element.AnnotationValue; +import java.util.Map; +import java.util.Optional; + +class AnnotationHelper { + + public Optional findAnnotationValue(AnnotationMirror annotationMirror, String name) { + return annotationMirror.getElementValues() + .entrySet() + .stream() + .filter(entry -> entry.getKey().getSimpleName().toString().equals(name)) + .map(Map.Entry::getValue) + .findFirst(); + } +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/AnnotationProcessor.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/AnnotationProcessor.java new file mode 100644 index 0000000000..400342bf9a --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/AnnotationProcessor.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import javax.lang.model.element.AnnotationMirror; +import javax.lang.model.element.Element; +import java.util.Optional; + +class AnnotationProcessor { + Optional findAnnotation(Element element, Class annotationClass) { + return element.getAnnotationMirrors() + .stream() + .filter(annotationMirror -> annotationMirror.getAnnotationType().toString().equals(annotationClass.getCanonicalName())) + .findFirst(); + } +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/FactoryClassCreator.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/FactoryClassCreator.java new file mode 100644 index 0000000000..209632a6a5 --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/FactoryClassCreator.java @@ -0,0 +1,280 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import com.squareup.javapoet.ClassName; +import com.squareup.javapoet.JavaFile; +import com.squareup.javapoet.MethodSpec; +import com.squareup.javapoet.ParameterizedTypeName; +import com.squareup.javapoet.TypeName; +import com.squareup.javapoet.TypeSpec; +import jakarta.inject.Inject; +import sonia.scm.ModelObject; +import sonia.scm.store.QueryableStoreFactory; + +import javax.annotation.processing.ProcessingEnvironment; +import javax.lang.model.element.Element; +import javax.lang.model.element.Modifier; +import javax.lang.model.element.TypeElement; +import javax.tools.Diagnostic; +import java.io.IOException; +import java.util.Arrays; +import java.util.List; +import java.util.Locale; +import java.util.function.BiConsumer; +import java.util.function.Function; + +class FactoryClassCreator { + + private static final String STORE_PACKAGE_NAME = "sonia.scm.store"; + private static final String QUERYABLE_MUTABLE_STORE_CLASS_NAME = "QueryableMutableStore"; + private static final String QUERYABLE_STORE_CLASS_NAME = "QueryableStore"; + + private final ProcessingEnvironment processingEnv; + + FactoryClassCreator(ProcessingEnvironment processingEnv) { + this.processingEnv = processingEnv; + } + + void createFactoryClass(Element element, String packageName, TypeElement dataClassTypeElement) throws IOException { + TypeName typeNameOfDataClass = TypeName.get(dataClassTypeElement.asType()); + TypeSpec.Builder builder = + TypeSpec + .classBuilder(element.getSimpleName() + "StoreFactory") + .addModifiers(Modifier.PUBLIC) + .addJavadoc("Generated queryable store factory for type {@link $T}.\nTo create conditions in queries, use the static fields in the class {@link $TQueryFields}.\n", typeNameOfDataClass, typeNameOfDataClass); + + createStoreFactoryField(builder); + createConstructor(builder); + + List parents = determineParentSpecs(dataClassTypeElement); + if (parents.isEmpty()) { + createGetterForDataTypeWithoutParent(builder, typeNameOfDataClass); + } else { + createOverallGetterForDataTypeWithParents(builder, typeNameOfDataClass); + createMutableGetterForDataTypeWithParents(typeNameOfDataClass, parents, builder); + createPartialGetterForDataTypeWithParents(builder, typeNameOfDataClass, parents); + } + + JavaFile.builder(packageName, builder.build()) + .build() + .writeTo(processingEnv.getFiler()); + } + + private void createMutableGetterForDataTypeWithParents(TypeName typeNameOfDataClass, List parents, TypeSpec.Builder builder) { + builder.addMethod( + createGetMutableMethodSpec( + typeNameOfDataClass, + "Returns a store to modify elements of the type {@link $T}.\nTo do so, an id has to be specified for each parent type.\n", + parents, + ParentSpec::buildParameterNameWithIdSuffix, + ParentSpec::appendParentAsIdStringArgument + ) + ); + + if (isEveryParentModelObject(parents)) { + builder.addMethod( + createGetMutableMethodSpec( + typeNameOfDataClass, + "Returns a store to modify elements of the type {@link $T}.\nTo do so, an instance of each parent type has to be specified.\n", + parents, + ParentSpec::buildIdGetterWithParameterName, + ParentSpec::appendParentAsObjectArgument + ) + ); + } + } + + private MethodSpec createGetMutableMethodSpec(TypeName typeNameOfDataClass, + String javaDoc, + List parents, + Function parentIdProcessor, + BiConsumer parentArgumentProcessor) { + MethodSpec.Builder getMutableBuilder = MethodSpec + .methodBuilder("getMutable") + .addModifiers(Modifier.PUBLIC) + .addJavadoc(javaDoc, typeNameOfDataClass) + .returns(ParameterizedTypeName.get(ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_MUTABLE_STORE_CLASS_NAME), typeNameOfDataClass)) + .addStatement( + "return storeFactory.getMutable($T.class, $L)", + typeNameOfDataClass, + parents.stream().map(parentIdProcessor).reduce((s1, s2) -> s1 + ", " + s2).orElseThrow() + ); + + parents.forEach( + parent -> parentArgumentProcessor.accept(getMutableBuilder, parent) + ); + + return getMutableBuilder.build(); + } + + private boolean isEveryParentModelObject(List parents) { + return parents.stream().allMatch(ParentSpec::isModelObject); + } + + private void createPartialGetterForDataTypeWithParents(TypeSpec.Builder builder, TypeName typeNameOfDataClass, List parents) { + for (int i = 0; i < parents.size(); i++) { + String javaDocParentDescriptor; + if (i == 0) { + javaDocParentDescriptor = "only to the first parent"; + } else if (i < parents.size() - 1) { + javaDocParentDescriptor = "only to the first " + (i + 1) + "parents"; + } else { + javaDocParentDescriptor = "to all parents"; + } + + int currentParentLimit = i + 1; + builder.addMethod( + createPartialGetterMethodSpec( + typeNameOfDataClass, + "Returns a store to query elements of the type {@link $T} limited " + javaDocParentDescriptor + " specified by their ids.\n", + parents, + currentParentLimit, + ParentSpec::buildParameterNameWithIdSuffix, + ParentSpec::appendParentAsIdStringArgument + ) + ); + + if (isEveryParentModelObject(parents)) { + builder.addMethod( + createPartialGetterMethodSpec( + typeNameOfDataClass, + "Returns a store to query elements of the type {@link $T} limited " + javaDocParentDescriptor + " specified as instances of the parent type.\n", + parents, + currentParentLimit, + ParentSpec::buildIdGetterWithParameterName, + ParentSpec::appendParentAsObjectArgument + ) + ); + } + } + } + + private MethodSpec createPartialGetterMethodSpec(TypeName typeNameOfDataClass, + String javaDoc, + List parents, + int parentLimit, + Function parentIdProcessor, + BiConsumer parentArgumentProcessor) { + MethodSpec.Builder getBuilder = MethodSpec + .methodBuilder(parentLimit == parents.size() ? "get" : "getOverlapping") + .addModifiers(Modifier.PUBLIC) + .returns(ParameterizedTypeName.get(ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME), typeNameOfDataClass)) + .addJavadoc(javaDoc, typeNameOfDataClass) + .addStatement( + "return storeFactory.getReadOnly($T.class, $L)", + typeNameOfDataClass, + parents.stream() + .limit(parentLimit) + .map(parentIdProcessor) + .reduce((s1, s2) -> s1 + ", " + s2) + .orElseThrow() + ); + + parents.stream() + .limit(parentLimit) + .forEach(parent -> parentArgumentProcessor.accept(getBuilder, parent)); + + return getBuilder.build(); + } + + private void createOverallGetterForDataTypeWithParents(TypeSpec.Builder builder, TypeName typeNameOfDataClass) { + builder.addMethod( + MethodSpec.methodBuilder("getOverall") + .addModifiers(Modifier.PUBLIC) + .returns(ParameterizedTypeName.get(ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME), typeNameOfDataClass)) + .addStatement("return storeFactory.getReadOnly($T.class)", typeNameOfDataClass) + .addJavadoc("Returns a store to overall query elements of the type {@link $T} independent of any parent.\n", typeNameOfDataClass) + .build()); + } + + private void createGetterForDataTypeWithoutParent(TypeSpec.Builder builder, TypeName typeNameOfDataClass) { + builder.addMethod( + MethodSpec.methodBuilder("get") + .addModifiers(Modifier.PUBLIC) + .returns(ParameterizedTypeName.get(ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME), typeNameOfDataClass)) + .addStatement("return storeFactory.getReadOnly($T.class)", typeNameOfDataClass) + .addJavadoc("Returns a store to query elements of the type {@link $T}.\n", typeNameOfDataClass) + .build()); + builder.addMethod( + MethodSpec.methodBuilder("getMutable") + .addModifiers(Modifier.PUBLIC) + .returns(ParameterizedTypeName.get(ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_MUTABLE_STORE_CLASS_NAME), typeNameOfDataClass)) + .addStatement("return storeFactory.getMutable($T.class)", typeNameOfDataClass) + .addJavadoc("Returns a store to modify elements of the type {@link $T}.\n", typeNameOfDataClass) + .build()); + } + + private List determineParentSpecs(TypeElement typeElement) { + return new QueryableTypeParentProcessor().getQueryableTypeValues(typeElement) + .stream() + .map(queryableType -> { + String parentClassPackage = queryableType.substring(0, queryableType.lastIndexOf(".")); + String parentClassName = queryableType.substring(queryableType.lastIndexOf(".") + 1); + String parameterName = lowercaseFirstLetter(parentClassName); + return new ParentSpec(parentClassPackage, parentClassName, parameterName, isParentModelObject(queryableType)); + }) + .toList(); + } + + private boolean isParentModelObject(String parentType) { + try { + Class parentClass = Class.forName(parentType); + return Arrays.stream(parentClass.getInterfaces()).anyMatch(parentInterface -> parentInterface.getName().equals(ModelObject.class.getName())); + } catch (ClassNotFoundException e) { + processingEnv.getMessager().printMessage(Diagnostic.Kind.WARNING, String.format("Failed to find class of parent '%s'. Unable to determine whether this is a ModelObject or not. Will not generate factory methods for parent objects, only for ids.", parentType)); + return false; + } + } + + private String lowercaseFirstLetter(String parentClassName) { + return parentClassName.substring(0, 1).toLowerCase(Locale.ENGLISH) + parentClassName.substring(1); + } + + private void createConstructor(TypeSpec.Builder builder) { + builder.addMethod( + MethodSpec + .constructorBuilder() + .addParameter(QueryableStoreFactory.class, "storeFactory") + .addStatement("this.storeFactory = storeFactory") + .addAnnotation(Inject.class) + .addJavadoc("Instances should not be created manually, but injected by dependency injection using {@link $T}.\n", Inject.class) + .build()); + } + + private void createStoreFactoryField(TypeSpec.Builder builder) { + builder.addField(QueryableStoreFactory.class, "storeFactory", Modifier.PRIVATE, Modifier.FINAL); + } + + private record ParentSpec(String classPackage, String className, String parameterName, boolean isModelObject) { + String buildParameterNameWithIdSuffix() { + return parameterName + "Id"; + } + + String buildIdGetterWithParameterName() { + return parameterName + ".getId()"; + } + + static void appendParentAsIdStringArgument(MethodSpec.Builder builder, ParentSpec parent) { + builder.addParameter(String.class, parent.buildParameterNameWithIdSuffix()); + } + + static void appendParentAsObjectArgument(MethodSpec.Builder builder, ParentSpec parent) { + builder.addParameter(ClassName.get(parent.classPackage(), parent.className()), parent.parameterName()); + } + } +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/FieldInitializer.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/FieldInitializer.java new file mode 100644 index 0000000000..0ff6154e3b --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/FieldInitializer.java @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import com.squareup.javapoet.FieldSpec; + +import javax.lang.model.element.TypeElement; + +interface FieldInitializer { + void initialize(FieldSpec.Builder fieldBuilder, TypeElement element, String fieldClass, String fieldName); +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/NumberQueryFieldHandler.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/NumberQueryFieldHandler.java new file mode 100644 index 0000000000..a6d5d11acc --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/NumberQueryFieldHandler.java @@ -0,0 +1,40 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import com.squareup.javapoet.ClassName; +import com.squareup.javapoet.TypeName; + +class NumberQueryFieldHandler extends QueryFieldHandler { + public NumberQueryFieldHandler(String packageName, String className) { + this(packageName, className, null); + } + + public NumberQueryFieldHandler(String packageName, String className, String suffix) { + super( + "NumberQueryField", + new TypeName[]{ClassName.get(packageName, className)}, + (fieldBuilder, element, fieldClass, fieldName) -> fieldBuilder + .initializer( + "new $T<>($S)", + ClassName.get("sonia.scm.store", "QueryableStore").nestedClass(fieldClass), + fieldName + ), + suffix + ); + } +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryFieldClassCreator.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryFieldClassCreator.java new file mode 100644 index 0000000000..53cdf88caf --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryFieldClassCreator.java @@ -0,0 +1,295 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import com.google.auto.common.MoreElements; +import com.google.auto.common.MoreTypes; +import com.squareup.javapoet.ClassName; +import com.squareup.javapoet.FieldSpec; +import com.squareup.javapoet.JavaFile; +import com.squareup.javapoet.MethodSpec; +import com.squareup.javapoet.ParameterizedTypeName; +import com.squareup.javapoet.TypeName; +import com.squareup.javapoet.TypeSpec; +import jakarta.xml.bind.annotation.adapters.XmlAdapter; +import jakarta.xml.bind.annotation.adapters.XmlJavaTypeAdapter; + +import javax.annotation.processing.ProcessingEnvironment; +import javax.lang.model.element.AnnotationMirror; +import javax.lang.model.element.AnnotationValue; +import javax.lang.model.element.Element; +import javax.lang.model.element.ElementKind; +import javax.lang.model.element.Modifier; +import javax.lang.model.element.TypeElement; +import javax.lang.model.element.VariableElement; +import javax.lang.model.type.DeclaredType; +import javax.lang.model.type.TypeMirror; +import java.io.IOException; +import java.util.Collection; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.stream.Stream; + +class QueryFieldClassCreator { + + private static final String SIZE_SUFFIX = "SIZE"; + private static final String STORE_PACKAGE_NAME = "sonia.scm.store"; + private static final String QUERYABLE_STORE_CLASS_NAME = "QueryableStore"; + private static final String ID_QUERY_FIELD_CLASS_NAME = "IdQueryField"; + + private static final FieldInitializer SIMPLE_INITIALIZER = (fieldBuilder, element, fieldClass, fieldName) -> fieldBuilder + .initializer( + "new $T<>($S)", + ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME).nestedClass(fieldClass), + fieldName + ); + + private final ProcessingEnvironment processingEnv; + + QueryFieldClassCreator(ProcessingEnvironment processingEnv) { + this.processingEnv = processingEnv; + } + + void createQueryFieldClass(Element element, String packageName, TypeElement typeElement) throws IOException { + TypeSpec.Builder builder = + TypeSpec + .classBuilder(element.getSimpleName() + "QueryFields") + .addModifiers(Modifier.PUBLIC, Modifier.FINAL) + .addJavadoc("Generated query fields for type {@link $T}.\nTo create a queryable store for this, use an injected instance of the {@link $TStoreFactory}.\n", TypeName.get(typeElement.asType()), TypeName.get(typeElement.asType())); + + createPrivateConstructor(builder); + processParents(typeElement, builder); + processId(typeElement, builder); + processFields(typeElement, builder); + + JavaFile.builder(packageName, builder.build()) + .build() + .writeTo(processingEnv.getFiler()); + } + + private void createPrivateConstructor(TypeSpec.Builder builder) { + builder.addMethod( + MethodSpec + .constructorBuilder() + .addModifiers(Modifier.PRIVATE) + .build()); + } + + private void processParents(TypeElement typeElement, TypeSpec.Builder builder) { + new QueryableTypeParentProcessor().getQueryableTypeValues(typeElement) + .forEach(queryableType -> { + String parentClassPackage = queryableType.substring(0, queryableType.lastIndexOf(".")); + String parentClassName = queryableType.substring(queryableType.lastIndexOf(".") + 1); + builder.addField( + FieldSpec + .builder( + ParameterizedTypeName.get( + ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME) + .nestedClass(ID_QUERY_FIELD_CLASS_NAME), + TypeName.get(typeElement.asType())), + parentClassName.toUpperCase(Locale.ENGLISH) + "_ID" + ) + .addModifiers(Modifier.PUBLIC, Modifier.STATIC, Modifier.FINAL) + .initializer( + "new $T<>($T.class)", + ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME).nestedClass(ID_QUERY_FIELD_CLASS_NAME), + ClassName.get(parentClassPackage, parentClassName)) + .build()); + }); + } + + private void processId(TypeElement typeElement, TypeSpec.Builder builder) { + builder.addField( + FieldSpec + .builder( + ParameterizedTypeName.get( + ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME) + .nestedClass(ID_QUERY_FIELD_CLASS_NAME), + TypeName.get(typeElement.asType())), + "INTERNAL_ID" + ) + .addModifiers(Modifier.PUBLIC, Modifier.STATIC, Modifier.FINAL) + .initializer( + "new $T<>()", + ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME).nestedClass(ID_QUERY_FIELD_CLASS_NAME)) + .build()); + } + + private void processFields(TypeElement typeElement, TypeSpec.Builder builder) { + processFields(typeElement, typeElement, builder); + } + + @SuppressWarnings("UnstableApiUsage") + private void processFields(TypeElement typeElement, TypeElement superTypeElement, TypeSpec.Builder builder) { + processingEnv + .getElementUtils() + .getAllMembers(typeElement) + .stream() + .filter(member -> member.getKind() == ElementKind.FIELD) + .filter(member -> !member.getModifiers().contains(Modifier.STATIC)) + .filter(member -> !member.getModifiers().contains(Modifier.TRANSIENT)) + .flatMap(field -> + createFieldSpec( + superTypeElement, + MoreElements.asVariable(field)) + ) + .forEach(builder::addField); + TypeElement superclass = (TypeElement) processingEnv.getTypeUtils().asElement(typeElement.getSuperclass()); + if (superclass != null && !superclass.getQualifiedName().toString().equals(Object.class.getCanonicalName())) { + processFields(superclass, typeElement, builder); + } + } + + private Stream createFieldSpec(TypeElement element, VariableElement field) { + TypeMirror effectiveFieldType = determineFieldType(field); + return createFieldHandler(effectiveFieldType).stream() + .map(queryFieldHandler -> { + String fieldName = field.getSimpleName().toString(); + String fieldClass = queryFieldHandler.getClazz(); + TypeName[] furtherGenerics = queryFieldHandler.getGenerics(); + TypeName[] generics = new TypeName[furtherGenerics.length + 1]; + generics[0] = TypeName.get(element.asType()); + System.arraycopy(furtherGenerics, 0, generics, 1, furtherGenerics.length); + FieldSpec.Builder fieldBuilder = FieldSpec + .builder( + ParameterizedTypeName.get( + ClassName + .get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME) + .nestedClass(fieldClass), + generics), + determineFieldNameWithSuffix(fieldName, queryFieldHandler).toUpperCase(Locale.ENGLISH) + ) + .addJavadoc("Generated query field to create conditions for field {@link $L#$L} of type {@link $L}.\n", TypeName.get(element.asType()), fieldName, effectiveFieldType) + .addModifiers(Modifier.PUBLIC, Modifier.STATIC, Modifier.FINAL); + queryFieldHandler.getInitializer().initialize(fieldBuilder, element, fieldClass, fieldName); + return fieldBuilder.build(); + }); + } + + private TypeMirror determineFieldType(VariableElement field) { + return new AnnotationProcessor().findAnnotation(field, XmlJavaTypeAdapter.class) + .map(this::determineTypeFromAdapter) + .orElseGet(field::asType); + } + + private TypeMirror determineTypeFromAdapter(AnnotationMirror annotationMirror) { + AnnotationValue value = new AnnotationHelper().findAnnotationValue(annotationMirror, "value").orElseThrow(); + TypeMirror adapterType = (TypeMirror) value.getValue(); + TypeMirror xmlAdapterType = processingEnv.getTypeUtils() + .directSupertypes(adapterType) + .stream() + .filter(typeMirror -> processingEnv.getTypeUtils() + .isAssignable( + processingEnv.getTypeUtils().erasure(typeMirror), + processingEnv.getElementUtils().getTypeElement(XmlAdapter.class.getCanonicalName()).asType() + )) + .findFirst() + .orElseThrow(RuntimeException::new); + DeclaredType declaredType = MoreTypes.asDeclared(xmlAdapterType); + return declaredType.getTypeArguments().get(0); + } + + private Collection createFieldHandler(TypeMirror fieldType) { + TypeMirror collectionType = processingEnv.getElementUtils().getTypeElement(Collection.class.getCanonicalName()).asType(); + TypeMirror erasure = processingEnv.getTypeUtils().erasure(fieldType); + if (processingEnv.getTypeUtils().isAssignable(erasure, collectionType)) { + return List.of( + new QueryFieldHandler( + "CollectionQueryField", + new TypeName[]{}, + SIMPLE_INITIALIZER + ), + new QueryFieldHandler( + "CollectionSizeQueryField", + new TypeName[]{}, + SIMPLE_INITIALIZER, + SIZE_SUFFIX + ) + ); + } + TypeMirror mapType = processingEnv.getElementUtils().getTypeElement(Map.class.getCanonicalName()).asType(); + if (processingEnv.getTypeUtils().isAssignable(erasure, mapType)) { + return List.of( + new QueryFieldHandler( + "MapQueryField", + new TypeName[]{}, + SIMPLE_INITIALIZER + ), + new QueryFieldHandler( + "MapSizeQueryField", + new TypeName[]{}, + SIMPLE_INITIALIZER, + SIZE_SUFFIX + ) + ); + } + Element fieldAsElement = processingEnv.getTypeUtils().asElement(fieldType); + if (fieldAsElement != null && fieldAsElement.getKind() == ElementKind.ENUM) { + return List.of(new QueryFieldHandler( + "EnumQueryField", + new TypeName[]{TypeName.get(fieldAsElement.asType())}, + (fieldBuilder, element, fieldClass, fieldName) -> fieldBuilder + .initializer( + "new $T<>($S)", + ClassName.get(STORE_PACKAGE_NAME, QUERYABLE_STORE_CLASS_NAME).nestedClass(fieldClass), + fieldName + ) + )); + } + return switch (fieldType.toString()) { + case "java.lang.String" -> List.of( + new QueryFieldHandler( + "StringQueryField", + new TypeName[]{}, + SIMPLE_INITIALIZER)); + case "boolean", "java.lang.Boolean" -> List.of( + new QueryFieldHandler( + "BooleanQueryField", + new TypeName[]{}, + SIMPLE_INITIALIZER)); + case "int", "java.lang.Integer" -> List.of( + new NumberQueryFieldHandler( + "java.lang", + "Integer")); + case "long", "java.lang.Long" -> List.of( + new NumberQueryFieldHandler( + "java.lang", + "Long")); + case "float", "java.lang.Float" -> List.of( + new NumberQueryFieldHandler( + "java.lang", + "Float")); + case "double", "java.lang.Double" -> List.of( + new NumberQueryFieldHandler( + "java.lang", + "Double")); + case "java.util.Date", "java.time.Instant" -> List.of( + new QueryFieldHandler( + "InstantQueryField", + new TypeName[]{}, + SIMPLE_INITIALIZER)); + default -> List.of(); + }; + } + + private String determineFieldNameWithSuffix(String fieldName, QueryFieldHandler fieldHandler) { + return fieldHandler.getSuffix() + .map(suffix -> String.format("%s_%s", fieldName, suffix)) + .orElse(fieldName); + } +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryFieldHandler.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryFieldHandler.java new file mode 100644 index 0000000000..e48c28be2e --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryFieldHandler.java @@ -0,0 +1,40 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import com.squareup.javapoet.TypeName; +import lombok.AllArgsConstructor; +import lombok.Getter; + +import java.util.Optional; + +@Getter +@AllArgsConstructor +class QueryFieldHandler { + private final String clazz; + private final TypeName[] generics; + private final FieldInitializer initializer; + private final String suffix; + + public QueryFieldHandler(String clazz, TypeName[] generics, FieldInitializer initializer) { + this(clazz, generics, initializer, null); + } + + public Optional getSuffix() { + return Optional.ofNullable(suffix); + } +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryableTypeAnnotationProcessor.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryableTypeAnnotationProcessor.java new file mode 100644 index 0000000000..5d4827eb15 --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryableTypeAnnotationProcessor.java @@ -0,0 +1,100 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import com.google.auto.common.MoreElements; +import org.kohsuke.MetaInfServices; + +import javax.annotation.processing.AbstractProcessor; +import javax.annotation.processing.Processor; +import javax.annotation.processing.RoundEnvironment; +import javax.annotation.processing.SupportedAnnotationTypes; +import javax.annotation.processing.SupportedSourceVersion; +import javax.lang.model.SourceVersion; +import javax.lang.model.element.Element; +import javax.lang.model.element.TypeElement; +import javax.tools.Diagnostic; +import java.io.IOException; +import java.util.Optional; +import java.util.Set; + +import static java.util.Optional.empty; +import static java.util.Optional.of; + +@SupportedAnnotationTypes("sonia.scm.store.QueryableType") +@MetaInfServices(Processor.class) +@SupportedSourceVersion(SourceVersion.RELEASE_17) +public class QueryableTypeAnnotationProcessor extends AbstractProcessor { + + @Override + public boolean process(Set annotations, RoundEnvironment roundEnvironment) { + for (TypeElement annotation : annotations) { + log("Found annotation: " + annotation.getQualifiedName()); + roundEnvironment.getElementsAnnotatedWith(annotation).forEach(element -> { + log("Found annotated element: " + element.getSimpleName()); + tryToCreateQueryFieldClass(element); + tryToCreateFactoryClass(element); + }); + } + return true; + } + + @SuppressWarnings("UnstableApiUsage") + private void tryToCreateQueryFieldClass(Element element) { + TypeElement typeElement = MoreElements.asType(element); + getPackageName(typeElement) + .ifPresent(packageName -> { + try { + new QueryFieldClassCreator(processingEnv).createQueryFieldClass(element, packageName, typeElement); + } catch (IOException e) { + error("Failed to create query field class for type " + typeElement + ": " + e.getMessage()); + } + }); + } + + @SuppressWarnings("UnstableApiUsage") + private void tryToCreateFactoryClass(Element element) { + TypeElement typeElement = MoreElements.asType(element); + getPackageName(typeElement) + .ifPresent(packageName -> { + try { + new FactoryClassCreator(processingEnv).createFactoryClass(element, packageName, typeElement); + } catch (IOException e) { + error("Failed to create factory class for type " + typeElement + ": " + e.getMessage()); + } + }); + } + + @SuppressWarnings("UnstableApiUsage") + private Optional getPackageName(TypeElement typeElement) { + Element enclosingElement = typeElement.getEnclosingElement(); + try { + return of(MoreElements.asPackage(enclosingElement).getQualifiedName().toString()); + } catch (IllegalArgumentException e) { + error("Could not determine package name for " + typeElement + ". QueryableType annotation does not support inner classes. Exception: " + e.getMessage()); + return empty(); + } + } + + private void log(String message) { + processingEnv.getMessager().printMessage(Diagnostic.Kind.NOTE, message); + } + + private void error(String message) { + processingEnv.getMessager().printMessage(Diagnostic.Kind.ERROR, message); + } +} diff --git a/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryableTypeParentProcessor.java b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryableTypeParentProcessor.java new file mode 100644 index 0000000000..93b6ef8171 --- /dev/null +++ b/scm-core-annotation-processor/src/main/java/sonia/scm/annotation/QueryableTypeParentProcessor.java @@ -0,0 +1,44 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import sonia.scm.store.QueryableType; + +import javax.lang.model.element.AnnotationValue; +import javax.lang.model.element.TypeElement; +import java.util.ArrayList; +import java.util.List; +import java.util.Optional; + +class QueryableTypeParentProcessor { + @SuppressWarnings("unchecked") + public List getQueryableTypeValues(TypeElement typeElement) { + return new AnnotationProcessor().findAnnotation(typeElement, QueryableType.class) + .map(annotationMirror -> { + Optional value = new AnnotationHelper().findAnnotationValue(annotationMirror, "value"); + if (value.isEmpty()) { + return new ArrayList(); + } + List parentClassTypes = (List) value.orElseThrow().getValue(); + return parentClassTypes.stream() + .map(AnnotationValue::getValue) + .map(Object::toString) + .toList(); + }) + .orElseGet(List::of); + } +} diff --git a/scm-core-annotation-processor/src/main/resources/META-INF/gradle/incremental.annotation.processors b/scm-core-annotation-processor/src/main/resources/META-INF/gradle/incremental.annotation.processors new file mode 100644 index 0000000000..e3d48a1ec7 --- /dev/null +++ b/scm-core-annotation-processor/src/main/resources/META-INF/gradle/incremental.annotation.processors @@ -0,0 +1 @@ +sonia.scm.annotation.QueryableTypeAnnotationProcessor,isolating diff --git a/scm-core-annotation-processor/src/test/java/sonia/scm/annotation/QueryableTypeAnnotationProcessorTest.java b/scm-core-annotation-processor/src/test/java/sonia/scm/annotation/QueryableTypeAnnotationProcessorTest.java new file mode 100644 index 0000000000..6119ef373e --- /dev/null +++ b/scm-core-annotation-processor/src/test/java/sonia/scm/annotation/QueryableTypeAnnotationProcessorTest.java @@ -0,0 +1,115 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.annotation; + +import com.google.common.truth.Truth; +import com.google.testing.compile.JavaFileObjects; +import com.google.testing.compile.JavaSourcesSubjectFactory; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.EnumSource; + +import javax.tools.JavaFileObject; +import java.util.List; + +@SuppressWarnings("java:S115") // we do not heed enum naming conventions for better readability in the test +class QueryableTypeAnnotationProcessorTest { + + enum FieldScenario { + A("empty class"), + B("string query field"), + C("boolean query fields"), + D("number query fields"), + E("enum query field"), + F("collection query field"), + G("map query field"), + H("unknown field"), + I("instant field mapped to string"), + K("parent id field"), + L("unmapped instant field"), + M("unmapped java util date field"), + N("static field"), + O("transient field"), + BSub("fields from super class"); + + private final String description; + + FieldScenario(String description) { + this.description = description; + } + + @Override + public String toString() { + return description; + } + } + + @ParameterizedTest(name = "should test field scenario for {0}") + @EnumSource(FieldScenario.class) + void shouldTest(FieldScenario scenario) { + JavaFileObject someObject = JavaFileObjects.forResource(String.format("sonia/scm/testing/%s.java", scenario.name())); + Truth.assert_() + .about(JavaSourcesSubjectFactory.javaSources()) + .that(List.of(someObject)) + .processedWith(new QueryableTypeAnnotationProcessor()) + .compilesWithoutError() + .and() + .generatesSources(JavaFileObjects.forResource(String.format("sonia/scm/testing/%sQueryFields.java", scenario.name()))); + } + + enum FactoryScenario { + A("class without parent"), + OneParent("class with one parent"), + TwoParents("class with two parents"), + ThreeParents("class with three parents"), + OneNonModelObjectParent("class with one model object parent and one non model object parent"); + + private final String description; + + FactoryScenario(String description) { + this.description = description; + } + + @Override + public String toString() { + return description; + } + } + + @ParameterizedTest(name = "should test factory scenario for {0}") + @EnumSource(FactoryScenario.class) + void shouldTest(FactoryScenario scenario) { + JavaFileObject someObject = JavaFileObjects.forResource(String.format("sonia/scm/testing/%s.java", scenario.name())); + Truth.assert_() + .about(JavaSourcesSubjectFactory.javaSources()) + .that(List.of(someObject)) + .processedWith(new QueryableTypeAnnotationProcessor()) + .compilesWithoutError() + .and() + .generatesSources(JavaFileObjects.forResource(String.format("sonia/scm/testing/%sStoreFactory.java", scenario.name()))); + } + + @Test + void shouldHandleInnerClasses() { + JavaFileObject someObject = JavaFileObjects.forResource("sonia/scm/testing/InnerA.java"); + Truth.assert_() + .about(JavaSourcesSubjectFactory.javaSources()) + .that(List.of(someObject)) + .processedWith(new QueryableTypeAnnotationProcessor()) + .failsToCompile(); + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/A.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/A.java new file mode 100644 index 0000000000..a9fff518e7 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/A.java @@ -0,0 +1,23 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +@QueryableType +public class A { +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/AQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/AQueryFields.java new file mode 100644 index 0000000000..a7a6b23446 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/AQueryFields.java @@ -0,0 +1,27 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class AQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + private AQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/AStoreFactory.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/AStoreFactory.java new file mode 100644 index 0000000000..dde123472d --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/AStoreFactory.java @@ -0,0 +1,40 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import jakarta.inject.Inject; +import sonia.scm.store.QueryableMutableStore; +import sonia.scm.store.QueryableStore; +import sonia.scm.store.QueryableStoreFactory; + +public class AStoreFactory { + + private final QueryableStoreFactory storeFactory; + + @Inject + AStoreFactory(QueryableStoreFactory storeFactory) { + this.storeFactory = storeFactory; + } + + public QueryableStore get() { + return storeFactory.getReadOnly(A.class); + } + + public QueryableMutableStore getMutable() { + return storeFactory.getMutable(A.class); + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/B.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/B.java new file mode 100644 index 0000000000..1db525afd6 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/B.java @@ -0,0 +1,24 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +@QueryableType +public class B { + private String name; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BQueryFields.java new file mode 100644 index 0000000000..416c4694c3 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BQueryFields.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class BQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.StringQueryField NAME = + new QueryableStore.StringQueryField<>("name"); + + private BQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BSub.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BSub.java new file mode 100644 index 0000000000..5d615ec0da --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BSub.java @@ -0,0 +1,23 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +@QueryableType +public class BSub extends B { +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BSubQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BSubQueryFields.java new file mode 100644 index 0000000000..bded15d0ce --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/BSubQueryFields.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class BSubQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.StringQueryField NAME = + new QueryableStore.StringQueryField<>("name"); + + private BSubQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/C.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/C.java new file mode 100644 index 0000000000..9659d7ab59 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/C.java @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +@QueryableType +public class C { + private boolean active; + private Boolean enabled; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/CQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/CQueryFields.java new file mode 100644 index 0000000000..46fb9f587e --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/CQueryFields.java @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class CQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.BooleanQueryField ACTIVE = + new QueryableStore.BooleanQueryField<>("active"); + + public static final QueryableStore.BooleanQueryField ENABLED = + new QueryableStore.BooleanQueryField<>("enabled"); + + private CQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/D.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/D.java new file mode 100644 index 0000000000..78bdce9d0f --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/D.java @@ -0,0 +1,34 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +@QueryableType +public class D { + private int age; + private Integer weight; + + private long creationTime; + private Long lastModified; + + private float height; + private Float width; + + private double price; + private Double margin; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/DQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/DQueryFields.java new file mode 100644 index 0000000000..808476c484 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/DQueryFields.java @@ -0,0 +1,51 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import java.lang.Double; +import java.lang.Float; +import java.lang.Integer; +import java.lang.Long; +import sonia.scm.store.QueryableStore; + +public final class DQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.NumberQueryField AGE = + new QueryableStore.NumberQueryField<>("age"); + public static final QueryableStore.NumberQueryField WEIGHT = + new QueryableStore.NumberQueryField<>("weight"); + + public static final QueryableStore.NumberQueryField CREATIONTIME = + new QueryableStore.NumberQueryField<>("creationTime"); + public static final QueryableStore.NumberQueryField LASTMODIFIED = + new QueryableStore.NumberQueryField<>("lastModified"); + + public static final QueryableStore.NumberQueryField HEIGHT = + new QueryableStore.NumberQueryField<>("height"); + public static final QueryableStore.NumberQueryField WIDTH = + new QueryableStore.NumberQueryField<>("width"); + + public static final QueryableStore.NumberQueryField PRICE = + new QueryableStore.NumberQueryField<>("price"); + public static final QueryableStore.NumberQueryField MARGIN = + new QueryableStore.NumberQueryField<>("margin"); + + private DQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/E.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/E.java new file mode 100644 index 0000000000..909ac0a9f1 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/E.java @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.Stage; +import sonia.scm.store.QueryableType; + +@QueryableType +public class E { + private Stage stage; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/EQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/EQueryFields.java new file mode 100644 index 0000000000..7c47110145 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/EQueryFields.java @@ -0,0 +1,31 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.Stage; +import sonia.scm.store.QueryableStore; + +public final class EQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.EnumQueryField STAGE = + new QueryableStore.EnumQueryField<>("stage"); + + private EQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/F.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/F.java new file mode 100644 index 0000000000..de67bc3785 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/F.java @@ -0,0 +1,27 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.Stage; +import sonia.scm.store.QueryableType; + +import java.util.List; + +@QueryableType +public class F { + private List names; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/FQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/FQueryFields.java new file mode 100644 index 0000000000..ebbc1396b7 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/FQueryFields.java @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class FQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.CollectionQueryField NAMES = + new QueryableStore.CollectionQueryField<>("names"); + + public static final QueryableStore.CollectionSizeQueryField NAMES_SIZE = + new QueryableStore.CollectionSizeQueryField<>("names"); + + private FQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/G.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/G.java new file mode 100644 index 0000000000..1780e448a4 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/G.java @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; +import java.util.Map; + +@QueryableType +public class G { + private Map dictionary; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/GQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/GQueryFields.java new file mode 100644 index 0000000000..9d777a6164 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/GQueryFields.java @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class GQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.MapQueryField DICTIONARY = + new QueryableStore.MapQueryField<>("dictionary"); + + public static final QueryableStore.MapSizeQueryField DICTIONARY_SIZE = + new QueryableStore.MapSizeQueryField<>("dictionary"); + + private GQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/H.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/H.java new file mode 100644 index 0000000000..e63b4d7371 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/H.java @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; +import java.util.Map; + +@QueryableType +public class H { + private Object somethingStrange; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/HQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/HQueryFields.java new file mode 100644 index 0000000000..4390231937 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/HQueryFields.java @@ -0,0 +1,27 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class HQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + private HQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/I.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/I.java new file mode 100644 index 0000000000..caddd90655 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/I.java @@ -0,0 +1,29 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import jakarta.xml.bind.annotation.adapters.XmlJavaTypeAdapter; +import sonia.scm.store.QueryableType; +import sonia.scm.xml.XmlInstantAdapter; + +import java.time.Instant; + +@QueryableType +public class I { + @XmlJavaTypeAdapter(XmlInstantAdapter.class) + private Instant birthday; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/IQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/IQueryFields.java new file mode 100644 index 0000000000..8a287ccbc5 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/IQueryFields.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class IQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.StringQueryField BIRTHDAY = + new QueryableStore.StringQueryField<>("birthday"); + + private IQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/InnerA.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/InnerA.java new file mode 100644 index 0000000000..c863039bfa --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/InnerA.java @@ -0,0 +1,29 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +public class InnerA { + + + + @QueryableType + public static class A { + } + +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/K.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/K.java new file mode 100644 index 0000000000..4270f46559 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/K.java @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.Stage; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableType; + +@QueryableType(Repository.class) +public class K { +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/KQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/KQueryFields.java new file mode 100644 index 0000000000..c0b2675494 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/KQueryFields.java @@ -0,0 +1,31 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableStore; + +public final class KQueryFields { + public static final QueryableStore.IdQueryField REPOSITORY_ID = + new QueryableStore.IdQueryField<>(Repository.class); + + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + private KQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/L.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/L.java new file mode 100644 index 0000000000..7182120212 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/L.java @@ -0,0 +1,28 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import jakarta.xml.bind.annotation.adapters.XmlJavaTypeAdapter; +import sonia.scm.store.QueryableType; +import sonia.scm.xml.XmlInstantAdapter; + +import java.time.Instant; + +@QueryableType +public class L { + private Instant birthday; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/LQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/LQueryFields.java new file mode 100644 index 0000000000..32a5f41e63 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/LQueryFields.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class LQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.InstantQueryField BIRTHDAY = + new QueryableStore.InstantQueryField<>("birthday"); + + private LQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/M.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/M.java new file mode 100644 index 0000000000..e4d738847e --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/M.java @@ -0,0 +1,26 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +import java.util.Date; + +@QueryableType +public class M { + private Date birthday; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/MQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/MQueryFields.java new file mode 100644 index 0000000000..bc02997b56 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/MQueryFields.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class MQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + public static final QueryableStore.InstantQueryField BIRTHDAY = + new QueryableStore.InstantQueryField<>("birthday"); + + private MQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/N.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/N.java new file mode 100644 index 0000000000..426f26926b --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/N.java @@ -0,0 +1,24 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +@QueryableType +public class N { + private static String someField; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/NQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/NQueryFields.java new file mode 100644 index 0000000000..7600dcae84 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/NQueryFields.java @@ -0,0 +1,27 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class NQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + private NQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/O.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/O.java new file mode 100644 index 0000000000..49e13dbff8 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/O.java @@ -0,0 +1,24 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableType; + +@QueryableType +public class O { + private transient String transientField; +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OQueryFields.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OQueryFields.java new file mode 100644 index 0000000000..4f575b470b --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OQueryFields.java @@ -0,0 +1,27 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.store.QueryableStore; + +public final class OQueryFields { + public static final QueryableStore.IdQueryField INTERNAL_ID = + new QueryableStore.IdQueryField<>(); + + private OQueryFields() { + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneNonModelObjectParent.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneNonModelObjectParent.java new file mode 100644 index 0000000000..f557f4985a --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneNonModelObjectParent.java @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.repository.NamespaceAndName; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableType; + +@QueryableType({NamespaceAndName.class, Repository.class}) +public class OneNonModelObjectParent { +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneNonModelObjectParentStoreFactory.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneNonModelObjectParentStoreFactory.java new file mode 100644 index 0000000000..da6b4d7d41 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneNonModelObjectParentStoreFactory.java @@ -0,0 +1,49 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import jakarta.inject.Inject; +import java.lang.String; +import sonia.scm.store.QueryableMutableStore; +import sonia.scm.store.QueryableStore; +import sonia.scm.store.QueryableStoreFactory; + +public class OneNonModelObjectParentStoreFactory { + + private final QueryableStoreFactory storeFactory; + + @Inject + OneNonModelObjectParentStoreFactory(QueryableStoreFactory storeFactory) { + this.storeFactory = storeFactory; + } + + public QueryableStore getOverall() { + return storeFactory.getReadOnly(OneNonModelObjectParent.class); + } + + public QueryableMutableStore getMutable(String namespaceAndNameId, String repositoryId) { + return storeFactory.getMutable(OneNonModelObjectParent.class, namespaceAndNameId, repositoryId); + } + + public QueryableStore getOverlapping(String namespaceAndNameId) { + return storeFactory.getReadOnly(OneNonModelObjectParent.class, namespaceAndNameId); + } + + public QueryableStore get(String namespaceAndNameId, String repositoryId) { + return storeFactory.getReadOnly(OneNonModelObjectParent.class, namespaceAndNameId, repositoryId); + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneParent.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneParent.java new file mode 100644 index 0000000000..3467ab6a58 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneParent.java @@ -0,0 +1,24 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableType; + +@QueryableType(Repository.class) +public class OneParent { +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneParentStoreFactory.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneParentStoreFactory.java new file mode 100644 index 0000000000..0fb795e641 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/OneParentStoreFactory.java @@ -0,0 +1,54 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import jakarta.inject.Inject; +import java.lang.String; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableMutableStore; +import sonia.scm.store.QueryableStore; +import sonia.scm.store.QueryableStoreFactory; + +public class OneParentStoreFactory { + + private final QueryableStoreFactory storeFactory; + + @Inject + OneParentStoreFactory(QueryableStoreFactory storeFactory) { + this.storeFactory = storeFactory; + } + + public QueryableStore getOverall() { + return storeFactory.getReadOnly(OneParent.class); + } + + public QueryableMutableStore getMutable(String repositoryId) { + return storeFactory.getMutable(OneParent.class, repositoryId); + } + + public QueryableMutableStore getMutable(Repository repository) { + return storeFactory.getMutable(OneParent.class, repository.getId()); + } + + public QueryableStore get(String repositoryId) { + return storeFactory.getReadOnly(OneParent.class, repositoryId); + } + + public QueryableStore get(Repository repository) { + return storeFactory.getReadOnly(OneParent.class, repository.getId()); + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/ThreeParents.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/ThreeParents.java new file mode 100644 index 0000000000..1b20461a75 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/ThreeParents.java @@ -0,0 +1,26 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.group.Group; +import sonia.scm.repository.Repository; +import sonia.scm.user.User; +import sonia.scm.store.QueryableType; + +@QueryableType({Repository.class, User.class, Group.class}) +public class ThreeParents { +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/ThreeParentsStoreFactory.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/ThreeParentsStoreFactory.java new file mode 100644 index 0000000000..d4a77cd787 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/ThreeParentsStoreFactory.java @@ -0,0 +1,73 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import jakarta.inject.Inject; +import java.lang.String; + +import sonia.scm.group.Group; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableMutableStore; +import sonia.scm.store.QueryableStore; +import sonia.scm.store.QueryableStoreFactory; +import sonia.scm.user.User; + +public class ThreeParentsStoreFactory { + + private final QueryableStoreFactory storeFactory; + + @Inject + ThreeParentsStoreFactory(QueryableStoreFactory storeFactory) { + this.storeFactory = storeFactory; + } + + public QueryableStore getOverall() { + return storeFactory.getReadOnly(ThreeParents.class); + } + + public QueryableMutableStore getMutable(String repositoryId, String userId, String groupId) { + return storeFactory.getMutable(ThreeParents.class, repositoryId, userId, groupId); + } + + public QueryableMutableStore getMutable(Repository repository, User user, Group group) { + return storeFactory.getMutable(ThreeParents.class, repository.getId(), user.getId(), group.getId()); + } + + public QueryableStore getOverlapping(String repositoryId) { + return storeFactory.getReadOnly(ThreeParents.class, repositoryId); + } + + public QueryableStore getOverlapping(Repository repository) { + return storeFactory.getReadOnly(ThreeParents.class, repository.getId()); + } + + public QueryableStore getOverlapping(String repositoryId, String userId) { + return storeFactory.getReadOnly(ThreeParents.class, repositoryId, userId); + } + + public QueryableStore getOverlapping(Repository repository, User user) { + return storeFactory.getReadOnly(ThreeParents.class, repository.getId(), user.getId()); + } + + public QueryableStore get(String repositoryId, String userId, String groupId) { + return storeFactory.getReadOnly(ThreeParents.class, repositoryId, userId, groupId); + } + + public QueryableStore get(Repository repository, User user, Group group) { + return storeFactory.getReadOnly(ThreeParents.class, repository.getId(), user.getId(), group.getId()); + } +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/TwoParents.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/TwoParents.java new file mode 100644 index 0000000000..f4bf33d14a --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/TwoParents.java @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import sonia.scm.repository.Repository; +import sonia.scm.user.User; +import sonia.scm.store.QueryableType; + +@QueryableType({Repository.class, User.class}) +public class TwoParents { +} diff --git a/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/TwoParentsStoreFactory.java b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/TwoParentsStoreFactory.java new file mode 100644 index 0000000000..76c74a8c41 --- /dev/null +++ b/scm-core-annotation-processor/src/test/resources/sonia/scm/testing/TwoParentsStoreFactory.java @@ -0,0 +1,63 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.testing; + +import jakarta.inject.Inject; +import java.lang.String; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableMutableStore; +import sonia.scm.store.QueryableStore; +import sonia.scm.store.QueryableStoreFactory; +import sonia.scm.user.User; + +public class TwoParentsStoreFactory { + + private final QueryableStoreFactory storeFactory; + + @Inject + TwoParentsStoreFactory(QueryableStoreFactory storeFactory) { + this.storeFactory = storeFactory; + } + + public QueryableStore getOverall() { + return storeFactory.getReadOnly(TwoParents.class); + } + + public QueryableMutableStore getMutable(String repositoryId, String userId) { + return storeFactory.getMutable(TwoParents.class, repositoryId, userId); + } + + public QueryableMutableStore getMutable(Repository repository, User user) { + return storeFactory.getMutable(TwoParents.class, repository.getId(), user.getId()); + } + + public QueryableStore getOverlapping(String repositoryId) { + return storeFactory.getReadOnly(TwoParents.class, repositoryId); + } + + public QueryableStore getOverlapping(Repository repository) { + return storeFactory.getReadOnly(TwoParents.class, repository.getId()); + } + + public QueryableStore get(String repositoryId, String userId) { + return storeFactory.getReadOnly(TwoParents.class, repositoryId, userId); + } + + public QueryableStore get(Repository repository, User user) { + return storeFactory.getReadOnly(TwoParents.class, repository.getId(), user.getId()); + } +} diff --git a/scm-core/src/main/java/sonia/scm/plugin/ExtensionProcessor.java b/scm-core/src/main/java/sonia/scm/plugin/ExtensionProcessor.java index dc5254717e..f94f7f2796 100644 --- a/scm-core/src/main/java/sonia/scm/plugin/ExtensionProcessor.java +++ b/scm-core/src/main/java/sonia/scm/plugin/ExtensionProcessor.java @@ -77,4 +77,12 @@ public interface ExtensionProcessor default Iterable> getIndexedTypes() { return emptySet(); } + + /** + * Returns all queryable types. + * @since 3.7.0 + */ + default Iterable getQueryableTypes() { + return emptySet(); + } } diff --git a/scm-core/src/main/java/sonia/scm/plugin/NamedClassElement.java b/scm-core/src/main/java/sonia/scm/plugin/NamedClassElement.java index 05010f2629..0bc272b77f 100644 --- a/scm-core/src/main/java/sonia/scm/plugin/NamedClassElement.java +++ b/scm-core/src/main/java/sonia/scm/plugin/NamedClassElement.java @@ -27,7 +27,7 @@ import lombok.ToString; import java.util.HashSet; @Getter -@ToString +@ToString(callSuper = true) @EqualsAndHashCode(callSuper = true) @XmlAccessorType(XmlAccessType.FIELD) @NoArgsConstructor(access = AccessLevel.PACKAGE) diff --git a/scm-core/src/main/java/sonia/scm/plugin/QueryableTypeDescriptor.java b/scm-core/src/main/java/sonia/scm/plugin/QueryableTypeDescriptor.java new file mode 100644 index 0000000000..de440b0aba --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/plugin/QueryableTypeDescriptor.java @@ -0,0 +1,39 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.plugin; + +import jakarta.xml.bind.annotation.XmlAccessType; +import jakarta.xml.bind.annotation.XmlAccessorType; +import jakarta.xml.bind.annotation.XmlElement; +import jakarta.xml.bind.annotation.adapters.XmlJavaTypeAdapter; +import lombok.*; +import sonia.scm.xml.XmlArrayStringAdapter; + +@ToString(callSuper = true) +@EqualsAndHashCode(callSuper = true) +@XmlAccessorType(XmlAccessType.FIELD) +@NoArgsConstructor(access = AccessLevel.PACKAGE) +public class QueryableTypeDescriptor extends NamedClassElement { + + @XmlElement(name = "value") + @XmlJavaTypeAdapter(XmlArrayStringAdapter.class) + private String[] types; + + public String[] getTypes() { + return types == null ? new String[0] : types; + } +} diff --git a/scm-core/src/main/java/sonia/scm/plugin/ScmModule.java b/scm-core/src/main/java/sonia/scm/plugin/ScmModule.java index 2b48581742..0109779257 100644 --- a/scm-core/src/main/java/sonia/scm/plugin/ScmModule.java +++ b/scm-core/src/main/java/sonia/scm/plugin/ScmModule.java @@ -65,6 +65,9 @@ public class ScmModule { @XmlElement(name = "web-element") private Set webElements; + @XmlElement(name = "queryable-type") + private Set queryableTypes; + public Iterable getEvents() { return nonNull(events); } @@ -107,12 +110,18 @@ public class ScmModule { /** * @since 3.0.0 - */ public Iterable getConfigElements() { return nonNull(configElements); } + /** + * @since 3.7.0 + */ + public Iterable getQueryableTypes() { + return nonNull(queryableTypes); + } + private Iterable nonNull(Iterable iterable) { if (iterable == null) { iterable = ImmutableSet.of(); diff --git a/scm-core/src/main/java/sonia/scm/repository/RepositoryHookEvent.java b/scm-core/src/main/java/sonia/scm/repository/RepositoryHookEvent.java index c3a29cec12..58f2d9ece1 100644 --- a/scm-core/src/main/java/sonia/scm/repository/RepositoryHookEvent.java +++ b/scm-core/src/main/java/sonia/scm/repository/RepositoryHookEvent.java @@ -17,13 +17,17 @@ package sonia.scm.repository; +import lombok.Getter; import sonia.scm.repository.api.HookContext; +import java.time.Instant; + /** * Repository hook event represents an change event of a repository. * * @since 1.6 */ +@Getter public class RepositoryHookEvent { @@ -36,6 +40,13 @@ public class RepositoryHookEvent /** hook type */ private final RepositoryHookType type; + /** + * creation date of the event + * + * @since 3.8.0 + */ + private final Instant creationDate = Instant.now(); + public RepositoryHookEvent(HookContext context, Repository repository, RepositoryHookType type) { @@ -44,24 +55,6 @@ public class RepositoryHookEvent this.type = type; } - - public HookContext getContext() - { - return context; - } - - - public Repository getRepository() - { - return repository; - } - - - public RepositoryHookType getType() - { - return type; - } - @Override public String toString() { return "RepositoryHookEvent{" + diff --git a/scm-core/src/main/java/sonia/scm/store/Condition.java b/scm-core/src/main/java/sonia/scm/store/Condition.java new file mode 100644 index 0000000000..aba919aa2c --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/Condition.java @@ -0,0 +1,21 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +@SuppressWarnings("unused") // We need the type 'T' here to keep type safety +public interface Condition { +} diff --git a/scm-core/src/main/java/sonia/scm/store/Conditions.java b/scm-core/src/main/java/sonia/scm/store/Conditions.java new file mode 100644 index 0000000000..1bff68a0da --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/Conditions.java @@ -0,0 +1,38 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +public final class Conditions { + private Conditions() { + } + + @SafeVarargs + public static Condition and(Condition... conditions) { + return new LogicalCondition<>(LogicalOperator.AND, conditions); + } + + @SafeVarargs + public static Condition or(Condition... conditions) { + return new LogicalCondition<>(LogicalOperator.OR, conditions); + } + + @SafeVarargs + @SuppressWarnings({"unchecked", "rawtypes"}) + public static Condition not(Condition... conditions) { + return new LogicalCondition<>(LogicalOperator.NOT, new Condition[]{and(conditions)}); + } +} diff --git a/scm-core/src/main/java/sonia/scm/store/LeafCondition.java b/scm-core/src/main/java/sonia/scm/store/LeafCondition.java new file mode 100644 index 0000000000..963cf18be8 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/LeafCondition.java @@ -0,0 +1,50 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import lombok.Getter; +import lombok.Value; + +/** + * A LeafCondition is a condition builder on a {@link QueryableStore.QueryField} as part of a store statement. + * + * @param type of the object held by the {@link QueryableStore.QueryField} + * @param value type (only required for binary operators) + */ +@Value +@Getter +public class LeafCondition implements Condition { + + /** + * Argument for the operator to check against.
+ * Example: fruit EQ apple + */ + QueryableStore.QueryField field; + + /** + * A binary (e.g. EQ, CONTAINS) or unary (e.g. NULL) operator. Binary operators require a non-null value field.
+ * Example: fruit EQ apple, fruit NULL + */ + Operator operator; + + + /** + * Value for binary operators.
+ * Example: fruit EQ apple + */ + C value; +} diff --git a/scm-core/src/main/java/sonia/scm/store/LogicalCondition.java b/scm-core/src/main/java/sonia/scm/store/LogicalCondition.java new file mode 100644 index 0000000000..7e5ae1a83b --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/LogicalCondition.java @@ -0,0 +1,32 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import lombok.Getter; +import lombok.Value; + +@Value +@Getter +public class LogicalCondition implements Condition { + LogicalOperator operator; + Condition[] conditions; + + LogicalCondition(LogicalOperator operator, Condition[] conditions) { + this.operator = operator; + this.conditions = conditions; + } +} diff --git a/scm-core/src/main/java/sonia/scm/store/LogicalOperator.java b/scm-core/src/main/java/sonia/scm/store/LogicalOperator.java new file mode 100644 index 0000000000..d9a3383286 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/LogicalOperator.java @@ -0,0 +1,21 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +public enum LogicalOperator { + AND, OR, NOT; +} diff --git a/scm-core/src/main/java/sonia/scm/store/Operator.java b/scm-core/src/main/java/sonia/scm/store/Operator.java new file mode 100644 index 0000000000..d54c252ac1 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/Operator.java @@ -0,0 +1,32 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +public enum Operator { + + EQ, + LESS, + GREATER, + LESS_OR_EQUAL, + GREATER_OR_EQUAL, + CONTAINS, + IN, + NULL, + KEY, + VALUE + +} diff --git a/scm-core/src/main/java/sonia/scm/store/QueryableMaintenanceStore.java b/scm-core/src/main/java/sonia/scm/store/QueryableMaintenanceStore.java new file mode 100644 index 0000000000..88ec1075a1 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/QueryableMaintenanceStore.java @@ -0,0 +1,146 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import com.google.common.collect.Streams; +import jakarta.xml.bind.annotation.XmlAccessType; +import jakarta.xml.bind.annotation.XmlAccessorType; +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.NoArgsConstructor; + +import java.util.Collection; +import java.util.Iterator; +import java.util.Optional; +import java.util.stream.Stream; + +/** + * This store should be used only in update steps or other maintenance tasks like deleting all entries for a deleted + * parent entity. + * + * @param The entity type of the store. + */ +public interface QueryableMaintenanceStore { + + Collection> readAll() throws SerializationException; + + Collection> readAllAs(Class type) throws SerializationException; + + Collection readRaw(); + + @SuppressWarnings("rawtypes") + default void writeAll(Iterable rows) throws SerializationException { + writeAll(Streams.stream(rows)); + } + + @SuppressWarnings("rawtypes") + void writeAll(Stream rows) throws SerializationException; + + default void writeRaw(Iterable rows) { + writeRaw(Streams.stream(rows)); + } + + void writeRaw(Stream rows); + + @Data + @XmlAccessorType(XmlAccessType.FIELD) + @NoArgsConstructor + @AllArgsConstructor + class Row { + private String[] parentIds; + private String id; + private U value; + } + + @Data + @XmlAccessorType(XmlAccessType.FIELD) + @NoArgsConstructor + @AllArgsConstructor + class RawRow { + private String[] parentIds; + private String id; + private String value; + } + + /** + * Deletes all entries from the store. If the store has been created limited to a concrete parent + * or a subset of parents, only the entries for this parent(s) will be deleted. + */ + void clear(); + + /** + * Returns an iterator to iterate over all entries in the store. If the store has been created limited to a concrete parent + * or a subset of parents, only the entries for this parent(s) will be returned. + * The iterated values offer additional methods to update or delete entries. + *
+ * The iterator must be closed after usage. Otherwise, updates may not be persisted. + */ + MaintenanceIterator iterateAll(); + + /** + * Iterator for existing entries in the store. + */ + interface MaintenanceIterator extends Iterator>, AutoCloseable { + } + + /** + * Maintenance helper for a concrete entry in the store. + */ + interface MaintenanceStoreEntry { + + /** + * The id of the entry. + */ + String getId(); + + /** + * Returns the id of the parent for the given class. + */ + Optional getParentId(Class clazz); + + /** + * Returns the entity as the specified type of the store. + * + * @throws SerializationException if the entry cannot be deserialized to the type of the store. + */ + T get(); + + /** + * Returns the entry as the given type, not as the type that has been specified for the store. + * This can be used whenever the type of the store has been changed in a way that no longer is compatible with the + * stored data. In this case, the entry can be deserialized to a different type that is only used during the + * migration. + * + * @param The type of the entry. + * @throws SerializationException if the entry cannot be deserialized to the given type. + */ + U getAs(Class type); + + /** + * Update the store entry with the given object. + * + * @throws SerializationException if the object cannot be serialized. + */ + void update(Object object); + } + + class SerializationException extends RuntimeException { + public SerializationException(String message, Throwable cause) { + super(message, cause); + } + } +} diff --git a/scm-core/src/main/java/sonia/scm/store/QueryableMutableStore.java b/scm-core/src/main/java/sonia/scm/store/QueryableMutableStore.java new file mode 100644 index 0000000000..e80aa6ccba --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/QueryableMutableStore.java @@ -0,0 +1,39 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import java.util.function.BooleanSupplier; + +/** + * This interface is used to store objects annotated with {@link QueryableType}. + * It combines the functionality of a {@link DataStore} and a {@link QueryableStore}. + * In contrast to the {@link QueryableStore}, instances are always scoped to a specific parent (if the type this store + * is created for as parent types specified in its annotation). + * It will be created by the {@link QueryableStoreFactory}. + *
+ * It is not meant to be instantiated by users of the API. Instead, use the query factory created by the annotation + * processor for the annotated type. + * + * @param The type of the objects to query. + * @since 3.7.0 + */ +public interface QueryableMutableStore extends DataStore, QueryableStore, AutoCloseable { + void transactional(BooleanSupplier callback); + + @Override + void close(); +} diff --git a/scm-core/src/main/java/sonia/scm/store/QueryableStore.java b/scm-core/src/main/java/sonia/scm/store/QueryableStore.java new file mode 100644 index 0000000000..01799dad26 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/QueryableStore.java @@ -0,0 +1,668 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.Arrays; +import java.util.Collection; +import java.util.Date; +import java.util.List; +import java.util.Optional; + +/** + * This interface is used to query objects annotated with {@link QueryableType}. It will be created by the + * {@link QueryableStoreFactory}. + *
+ * It is not meant to be instantiated by users of the API. Instead, use the query factory created by the annotation + * processor for the annotated type. + * + * @param The type of the objects to query. + * @since 3.7.0 + */ +public interface QueryableStore extends AutoCloseable { + + /** + * Creates a query for the objects of the type {@code T} with the given conditions. Conditions should be created by + * either using the static methods of the {@link Conditions} class or by using the query fields of the type {@code T} + * that will be created by the annotation processor in a separate class. If your annotated type is named + * {@code MyType}, the query fields class will be named {@code MyTypeQueryFields}. + *
+ * If no conditions are given, all objects of the type {@code T} will be returned (limited by the ids of the + * parent objects that had been specified when this instance of the store had been created by the factory). If more + * than one condition is given, the conditions will be combined with a logical AND. + * + * @param conditions The conditions to filter the objects. + * @return The query object to retrieve the result. + */ + Query query(Condition... conditions); + + /** + * Used to specify the order of the result of a query. + */ + enum Order { + /** + * Ascending order. + */ + ASC, + /** + * Descending order. + */ + DESC + } + + /** + * The terminal interface for a query build by {@link #query(Condition[])}. It provides methods to retrieve the + * result of the query in different forms. + * + * @param The type of the objects to query. + * @param The type of the result objects (if a projection had been made, for example using + * {@link #withIds()}). + */ + interface Query { + + /** + * Returns the first found object, if the query returns at least one result. + * If the query returns no result, an empty optional will be returned. + */ + Optional findFirst(); + + /** + * Returns the found object, if the query returns one exactly one result. When the query returns more than one + * result, a {@link TooManyResultsException} will be thrown. If the query returns no result, an empty optional will be returned. + */ + Optional findOne() throws TooManyResultsException; + + /** + * Returns all objects that match the query. If the query returns no result, an empty list will be returned. + */ + List findAll(); + + /** + * Returns a subset of all objects that match the query. If the query returns no result or the {@code offset} and + * {@code limit} are set in a way, that the result is exceeded, an empty list will be returned. + * + * @param offset The offset to start the result list. + * @param limit The maximum number of results to return. + */ + List findAll(long offset, long limit); + + /** + * Returns the found objects in combination with the parent ids they belong to. This is useful if you are using a + * queryable store that is not scoped to specific parent objects, and you therefore want to know to which parent + * objects each of the found objects belong to. + * + * @return The query object to continue building the query. + */ + Query> withIds(); + + /** + * Orders the result by the given field in the given order. If the order is not set, the order of the result is not + * specified. Orders can be chained, so you can call this method multiple times to order by multiple fields. + * + * @param field The field to order by. + * @param order The order to use (either ascending or descending). + * @return The query object to continue building the query. + */ + Query orderBy(QueryField field, Order order); + + /** + * Returns the count of all objects that match the query. + */ + long count(); + } + + /** + * The result of a query that was built by {@link QueryableStore.Query#withIds()}. It contains the parent ids of the + * found objects in addition to the objects and their ids themselves. + * + * @param The type of the queried objects. + */ + interface Result { + /** + * Returns the parent ids of the found objects. The parent ids are ordered in the same way as their types are + * specified in the @{@link QueryableType} annotation for the queried type. + */ + Optional getParentId(Class clazz); + + /** + * Returns the id of the found object. + */ + String getId(); + + /** + * Returns the found object itself. + */ + T getEntity(); + } + + /** + * Instances of this class will be created by the annotation processor for each class annotated with + * {@link QueryableType}. It provides query fields for the annotated class to build queries with. + *
+ * This is not meant to be extended or instantiated by users of the API! + * + * @param The type of the objects this field is used for. + * @param The type of the field. + */ + @SuppressWarnings("unused") + class QueryField { + final String name; + + public QueryField(String name) { + this.name = name; + } + + public String getName() { + return name; + } + + public boolean isIdField() { + return false; + } + + /** + * Creates a condition that checks if the field is null. + * + * @return The condition to use in a query. + */ + public Condition isNull() { + return new LeafCondition<>(this, Operator.NULL, null); + } + } + + /** + * This class is used to create conditions for queries. Instances of this class will be created by the annotation + * processor for each {@link String} field of a class annotated with {@link QueryableType}. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + */ + class StringQueryField extends QueryField { + + public StringQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the field is equal to the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public LeafCondition eq(String value) { + return new LeafCondition<>(this, Operator.EQ, value); + } + + /** + * Creates a condition that checks if the field contains the given value as a substring. + * + * @param value The value to check for. + * @return The condition to use in a query. + */ + public Condition contains(String value) { + return new LeafCondition<>(this, Operator.CONTAINS, value); + } + + + /** + * Creates a condition that checks if the field is equal to any of the given values. + * + * @param values The values to compare the field with. + * @return The condition to use in a query. + */ + public Condition in(String... values) { + return new LeafCondition<>(this, Operator.IN, values); + } + + /** + * Creates a condition that checks if the field is equal to any of the given values. + * + * @param values The values to compare the field with. + * @return The condition to use in a query. + */ + public Condition in(Collection values) { + return in(values.toArray(new String[0])); + } + } + + /** + * This class is used to create conditions for queries. Instances of this class will be created by the annotation + * processor for a class annotated with {@link QueryableType}. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + */ + class IdQueryField extends StringQueryField { + public IdQueryField(Class clazz) { + super(clazz.getName()); + } + + public IdQueryField() { + super(null); + } + + @Override + public boolean isIdField() { + return true; + } + } + + /** + * This class is used to create conditions for queries. Instances of this class will be created by the annotation + * processor for each number field (either {@link Integer}, {@link Long}, {@code int}, or {@code long}) of a class + * annotated with {@link QueryableType}. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + * @param The type of the number field. + */ + class NumberQueryField extends QueryField { + + public NumberQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the field is equal to the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition eq(N value) { + return new LeafCondition<>(this, Operator.EQ, value); + } + + /** + * Creates a condition that checks if the field is greater than the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition greater(N value) { + return new LeafCondition<>(this, Operator.GREATER, value); + } + + /** + * Creates a condition that checks if the field is less than the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition less(N value) { + return new LeafCondition<>(this, Operator.LESS, value); + } + + /** + * Creates a condition that checks if the field is greater than or equal to the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition greaterOrEquals(N value) { + return new LeafCondition<>(this, Operator.GREATER_OR_EQUAL, value); + } + + /** + * Creates a condition that checks if the field is less than or equal to the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition lessOrEquals(N value) { + return new LeafCondition<>(this, Operator.LESS_OR_EQUAL, value); + } + + /** + * Creates a condition that checks if the fields is inclusively between the from and to values. + * + * @param from The lower limit to compare the value with. + * @param to The upper limit to compare the value with. + * @return The condition to use in a query. + */ + public Condition between(N from, N to) { + return Conditions.and(lessOrEquals(to), greaterOrEquals(from)); + } + + /** + * Creates a condition that checks if the field is equal to any of the given values. + * + * @param values The values to compare the field with. + * @return The condition to use in a query. + */ + public Condition in(N... values) { + return new LeafCondition<>(this, Operator.IN, values); + } + + /** + * Creates a condition that checks if the field is equal to any of the given values. + * + * @param values The values to compare the field with. + * @return The condition to use in a query. + */ + public Condition in(Collection values) { + return new LeafCondition<>(this, Operator.IN, values.toArray(new Object[0])); + } + } + + /** + * This class is used to create conditions for queries. Instances of this class will be created by the annotation + * processor for each date field of a class annotated with {@link QueryableType}. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + */ + class InstantQueryField extends QueryField { + public InstantQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the field is equal to the given value. The given instant will be truncated to + * milliseconds. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition eq(Instant value) { + return new LeafCondition<>(this, Operator.EQ, value.truncatedTo(ChronoUnit.MILLIS)); + } + + /** + * Creates a condition that checks if the field is after the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition after(Instant value) { + return new LeafCondition<>(this, Operator.GREATER, value); + } + + /** + * Creates a condition that checks if the field is before the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition before(Instant value) { + return new LeafCondition<>(this, Operator.LESS, value); + } + + /** + * Creates a condition that checks if the field is between the given values. + * + * @param from The lower bound of the range to compare the field with. + * @param to The upper bound of the range to compare the field with. + * @return The condition to use in a query. + */ + public Condition between(Instant from, Instant to) { + return Conditions.and(after(from), before(to)); + } + + /** + * Creates a condition that checks if the field is equal to the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition eq(Date value) { + return eq(value.toInstant()); + } + + /** + * Creates a condition that checks if the field is after the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition after(Date value) { + return after(value.toInstant()); + } + + /** + * Creates a condition that checks if the field is before the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition before(Date value) { + return before(value.toInstant()); + } + + /** + * Creates a condition that checks if the field is between the given values. + * + * @param from The lower bound of the range to compare the field with. + * @param to The upper bound of the range to compare the field with. + * @return The condition to use in a query. + */ + public Condition between(Date from, Date to) { + return between(from.toInstant(), to.toInstant()); + } + } + + /** + * This class is used to create conditions for queries. Instances of this class will be created by the annotation + * processor for each boolean field of a class annotated with {@link QueryableType}. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + */ + class BooleanQueryField extends QueryField { + + public BooleanQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the field is equal to the given value. + * + * @param b The value to compare the field with. + * @return The condition to use in a query. + */ + public Condition eq(Boolean b) { + return new LeafCondition<>(this, Operator.EQ, b); + } + + /** + * Creates a condition that checks if the field is true. + * + * @return The condition to use in a query. + */ + public Condition isTrue() { + return eq(Boolean.TRUE); + } + + /** + * Creates a condition that checks if the field is false. + * + * @return The condition to use in a query. + */ + public Condition isFalse() { + return eq(Boolean.FALSE); + } + } + + /** + * This class is used to create conditions for queries. Instances of this class will be created by the annotation + * processor for each enum field of a class annotated with {@link QueryableType}. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + * @param The type of the enum field. + */ + class EnumQueryField> extends QueryField> { + public EnumQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the field is equal to the given value. + * + * @param value The value to compare the field with. + * @return The condition to use in a query. + */ + public LeafCondition eq(E value) { + return new LeafCondition<>(this, Operator.EQ, value.name()); + } + + /** + * Creates a condition that checks if the field is equal to any of the given values. + * + * @param values The values to compare the field with. + * @return The condition to use in a query. + */ + public Condition in(E... values) { + return new LeafCondition<>(this, Operator.IN, Arrays.stream(values).map(Enum::name).toArray()); + } + + /** + * Creates a condition that checks if the field is equal to any of the given values. + * + * @param values The values to compare the field with. + * @return The condition to use in a query. + */ + public Condition in(Collection values) { + return new LeafCondition<>(this, Operator.IN, values.stream().map(Enum::name).toArray()); + } + } + + /** + * This class is used to create conditions for queries. Instances of this class will be created by the annotation + * processor for each collection field of a class annotated with {@link QueryableType}. Note that this can only be + * used for collections of base types like {@link String}, number types, enums or booleans. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + */ + class CollectionQueryField extends QueryField { + public CollectionQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the field contains the given value. + * + * @param value The value to check for. + * @return The condition to use in a query. + */ + public Condition contains(Object value) { + return new LeafCondition<>(this, Operator.EQ, value); + } + } + + /** + * This class is used to create conditions for queries, based on the size of a collection. + * Instances of this class will be created by the annotation processor for each collection + * field of a class annotated with {@link QueryableType}. Note that this can only be used + * for collections of base types like {@link String}, number types, enums or booleans. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + */ + class CollectionSizeQueryField extends NumberQueryField { + public CollectionSizeQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the collection field is empty. + * + * @return The condition to use in a query. + */ + public Condition isEmpty() { + return eq(0L); + } + } + + /** + * This class is used to create conditions for queries. Instances of this class will be created by the annotation + * processor for each map field of a class annotated with {@link QueryableType}. Note that this can only be used for + * maps with base types like {@link String}, number types, enums or booleans as keys. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + */ + class MapQueryField extends QueryField { + public MapQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the field contains the given key. + * + * @param key The key to check for. + * @return The condition to use in a query. + */ + public Condition containsKey(Object key) { + return new LeafCondition<>(this, Operator.KEY, key); + } + + /** + * Creates a condition that checks if the field contains the given value. + * + * @param value The value to check for. + * @return The condition to use in a query. + */ + public Condition containsValue(Object value) { + return new LeafCondition<>(this, Operator.VALUE, value); + } + } + + /** + * This class is used to create conditions for queries, based on the size of a map. + * Instances of this class will be created by the annotation processor for each map + * field of a class annotated with {@link QueryableType}. Note that this can only be used + * for collections of base types like {@link String}, number types, enums or booleans. + *
+ * This is not meant to be instantiated by users of the API! + * + * @param The type of the objects this condition is used for. + */ + class MapSizeQueryField extends NumberQueryField { + public MapSizeQueryField(String name) { + super(name); + } + + /** + * Creates a condition that checks if the map field is empty. + * + * @return The condition to use in a query. + */ + public Condition isEmpty() { + return eq(0L); + } + } + + /** + * An exception occurring, if the client queried for one result with {@link Query#findOne()}, but the query returned multiple results. + */ + class TooManyResultsException extends RuntimeException { + public TooManyResultsException() { + super("Found more than one result"); + } + } +} diff --git a/scm-core/src/main/java/sonia/scm/store/QueryableStoreFactory.java b/scm-core/src/main/java/sonia/scm/store/QueryableStoreFactory.java new file mode 100644 index 0000000000..8efe749b9d --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/QueryableStoreFactory.java @@ -0,0 +1,66 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +/** + * Factory to create {@link QueryableStore} and {@link QueryableMutableStore} instances. + * In comparison to the {@link DataStoreFactory}, this factory is used to create stores which can execute + * queries on the stored data. Queryable stores can be used for types which are annotated with {@link QueryableType}. + *
+ * Normally, there should be no need to use this factory directly. Instead, for each type annotated with + * {@link QueryableType} a dedicated store factory is generated which can be injected into other components. + * For instance, if your data class is named {@code MyData} and annotated with {@link QueryableType}, a factory + * you should find a {@code MyDataStoreFactory} for your needs which is backed by this class. + *
+ * Implementations probably are backed by a database or a similar storage system instead of the familiar + * file based storage using XML. + * + * @since 3.7.0 + */ +public interface QueryableStoreFactory { + + /** + * Creates a read-only store for the given class and optional parent ids. If parent ids are omitted, queries + * will not be restricted to a specific parent (for example a repository) but will run on all data of the given type. + * + * @param clazz The class of the data type (must be annotated with {@link QueryableType}). + * @param parentIds Optional parent ids to restrict the query to a specific parent. + * @param The type of the data. + * @return A read-only store for the given class and optional parent ids. + */ + QueryableStore getReadOnly(Class clazz, String... parentIds); + + /** + * Creates a mutable store for the given class and parent ids. In contrast to the read-only store, for a mutable store + * the parent ids are mandatory. For each parent class given in the {@link QueryableType} annotation of the type, a + * concrete id has to be specified. This is because mutable stores are used to store data, which is done for the + * concrete parents only. So if data should be stored for different parents, separate mutable stores have to be + * created. + *
+ * The mutable store provides methods to store, update and delete data but also all query methods of the read-only + * store. + * + * @param clazz The class of the data type (must be annotated with {@link QueryableType}). + * @param parentIds Ids for all parent classes named in the {@link QueryableType} annotation. + * @param The type of the data. + * @return A mutable store for the given class scoped to the given parents. + */ + QueryableMutableStore getMutable(Class clazz, String... parentIds); + + QueryableMaintenanceStore getForMaintenance(Class clazz, String... parentIds); + +} diff --git a/scm-core/src/main/java/sonia/scm/store/StoreDeletionNotifier.java b/scm-core/src/main/java/sonia/scm/store/StoreDeletionNotifier.java new file mode 100644 index 0000000000..efa10dc6e8 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/StoreDeletionNotifier.java @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import sonia.scm.plugin.ExtensionPoint; + +@ExtensionPoint +public interface StoreDeletionNotifier { + void registerHandler(DeletionHandler handler); + + interface DeletionHandler { + default void notifyDeleted(Class clazz, String id) { + notifyDeleted(new ClassWithId(clazz, id)); + } + void notifyDeleted(ClassWithId... classWithIds); + } + + record ClassWithId(Class clazz, String id) {} +} diff --git a/scm-core/src/main/java/sonia/scm/store/StoreMetaDataProvider.java b/scm-core/src/main/java/sonia/scm/store/StoreMetaDataProvider.java new file mode 100644 index 0000000000..f3452e2dd6 --- /dev/null +++ b/scm-core/src/main/java/sonia/scm/store/StoreMetaDataProvider.java @@ -0,0 +1,23 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import java.util.Collection; + +public interface StoreMetaDataProvider { + Collection> getTypesWithParent(Class... classes); +} diff --git a/scm-core/src/main/java/sonia/scm/update/StoreUpdateStepUtilFactory.java b/scm-core/src/main/java/sonia/scm/update/StoreUpdateStepUtilFactory.java index cf13b7a486..0e9ca2bc44 100644 --- a/scm-core/src/main/java/sonia/scm/update/StoreUpdateStepUtilFactory.java +++ b/scm-core/src/main/java/sonia/scm/update/StoreUpdateStepUtilFactory.java @@ -16,6 +16,7 @@ package sonia.scm.update; +import sonia.scm.store.QueryableMaintenanceStore; import sonia.scm.store.StoreParameters; import sonia.scm.store.StoreType; @@ -25,6 +26,8 @@ public interface StoreUpdateStepUtilFactory { return new UtilForTypeBuilder(this, type); } + QueryableMaintenanceStore forQueryableType(Class clazz, String... parents); + final class UtilForTypeBuilder { private final StoreUpdateStepUtilFactory factory; private final StoreType type; diff --git a/scm-it/src/test/java/sonia/scm/it/ExportITCase.java b/scm-it/src/test/java/sonia/scm/it/ExportITCase.java new file mode 100644 index 0000000000..089b30fe5a --- /dev/null +++ b/scm-it/src/test/java/sonia/scm/it/ExportITCase.java @@ -0,0 +1,68 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.it; + +import org.junit.Before; +import org.junit.Test; +import sonia.scm.it.utils.ScmRequests; +import sonia.scm.it.utils.TestData; + +import java.nio.file.Path; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; +import static sonia.scm.it.utils.RestUtil.ADMIN_PASSWORD; +import static sonia.scm.it.utils.RestUtil.ADMIN_USERNAME; + +public class ExportITCase { + + @Before + public void init() { + TestData.cleanup(); + } + + @Test + public void shouldExportAndImportRepository() { + String namespace = ADMIN_USERNAME; + TestData.createDefault(); + String repo = TestData.getDefaultRepoName("git"); + + ScmRequests.start() + .requestIndexResource(ADMIN_USERNAME, ADMIN_PASSWORD) + .requestRepository(namespace, repo) + .writeTestData("value"); + + Path exportFile = ScmRequests.start() + .requestIndexResource(ADMIN_USERNAME, ADMIN_PASSWORD) + .requestRepository(namespace, repo) + .requestFullExport() + .exportFile(); + + ScmRequests.start() + .requestIndexResource(ADMIN_USERNAME, ADMIN_PASSWORD) + .requestRepositoryType("git") + .requestImport("fullImport", exportFile, "imported"); + + List importedTestData = ScmRequests.start() + .requestIndexResource(ADMIN_USERNAME, ADMIN_PASSWORD) + .requestRepository(namespace, "imported") + .requestTestData() + .getTestData(); + + assertThat(importedTestData).containsExactly("value"); + } +} diff --git a/scm-it/src/test/java/sonia/scm/it/utils/ScmRequests.java b/scm-it/src/test/java/sonia/scm/it/utils/ScmRequests.java index e432fd60b8..6bd32a8c8b 100644 --- a/scm-it/src/test/java/sonia/scm/it/utils/ScmRequests.java +++ b/scm-it/src/test/java/sonia/scm/it/utils/ScmRequests.java @@ -18,15 +18,23 @@ package sonia.scm.it.utils; import io.restassured.RestAssured; import io.restassured.response.Response; +import io.restassured.specification.RequestSpecification; import org.junit.Assert; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import sonia.scm.api.v2.resources.RepositoryDto; import sonia.scm.web.VndMediaType; +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Path; import java.util.List; import java.util.Map; import java.util.function.Consumer; +import static jakarta.ws.rs.core.MediaType.APPLICATION_JSON; +import static java.nio.file.StandardCopyOption.REPLACE_EXISTING; import static org.hamcrest.Matchers.is; import static sonia.scm.it.utils.TestData.createPasswordChangeJson; @@ -41,6 +49,7 @@ import static sonia.scm.it.utils.TestData.createPasswordChangeJson; * that return the *Response class containing specific operations related to the specific response * the *Response class contains also the request*() method to apply the next GET request from a link in the response. */ +@SuppressWarnings("rawtypes") public class ScmRequests { private static final Logger LOG = LoggerFactory.getLogger(ScmRequests.class); @@ -65,13 +74,13 @@ public class ScmRequests { public UserResponse requestUser(String username, String password, String pathParam) { setUsername(username); setPassword(password); - return new UserResponse<>(applyGETRequest(RestUtil.REST_BASE_URL.resolve("users/"+pathParam).toString()), null); + return new UserResponse<>(applyGETRequest(RestUtil.REST_BASE_URL.resolve("users/" + pathParam).toString()), null); } public ChangePasswordResponse requestUserChangePassword(String username, String password, String userPathParam, String newPassword) { setUsername(username); setPassword(password); - return new ChangePasswordResponse<>(applyPUTRequest(RestUtil.REST_BASE_URL.resolve("users/"+userPathParam+"/password").toString(), VndMediaType.PASSWORD_OVERWRITE, TestData.createPasswordChangeJson(password,newPassword)), null); + return new ChangePasswordResponse<>(applyPUTRequest(RestUtil.REST_BASE_URL.resolve("users/" + userPathParam + "/password").toString(), VndMediaType.PASSWORD_OVERWRITE, TestData.createPasswordChangeJson(password, newPassword)), null); } @SuppressWarnings("unchecked") @@ -80,6 +89,11 @@ public class ScmRequests { return new ModelResponse(response, null); } + public RequestSpecification withBasicAuth() { + return RestAssured.given() + .auth().preemptive().basic(username, password); + } + /** * Apply a GET Request to the extracted url from the given link * @@ -117,13 +131,12 @@ public class ScmRequests { */ private Response applyGETRequestWithQueryParams(String url, String params) { LOG.info("GET {}", url); - if (username == null || password == null){ + if (username == null || password == null) { return RestAssured.given() .when() .get(url + params); } - return RestAssured.given() - .auth().preemptive().basic(username, password) + return withBasicAuth() .when() .get(url + params); } @@ -138,13 +151,11 @@ public class ScmRequests { return applyGETRequestWithQueryParams(url, ""); } - /** * Apply a PUT Request to the extracted url from the given link * * @param response the response containing the link * @param linkPropertyName the property name of link - * @param body * @return the response of the PUT request using the given link */ private Response applyPUTRequestFromLink(Response response, String linkPropertyName, String content, String body) { @@ -158,15 +169,12 @@ public class ScmRequests { /** * Apply a PUT Request to the given url and return the response. * - * @param url the url of the PUT request - * @param mediaType - * @param body + * @param url the url of the PUT request * @return the response of the PUT request using the given url */ private Response applyPUTRequest(String url, String mediaType, String body) { LOG.info("PUT {}", url); - return RestAssured.given() - .auth().preemptive().basic(username, password) + return withBasicAuth() .when() .contentType(mediaType) .accept(mediaType) @@ -174,6 +182,37 @@ public class ScmRequests { .put(url); } + /** + * Apply a PUT Request to the extracted url from the given link + * + * @param response the response containing the link + * @param linkPropertyName the property name of link + * @return the response of the PUT request using the given link + */ + private Response applyPOSTRequestFromLink(Response response, String linkPropertyName, String content, String body) { + return applyPOSTRequest(response + .then() + .extract() + .path(linkPropertyName), content, body); + } + + + /** + * Apply a POST Request to the given url and return the response. + * + * @param url the url of the PUT request + * @return the response of the PUT request using the given url + */ + private Response applyPOSTRequest(String url, String mediaType, String body) { + LOG.info("POST {}", url); + return withBasicAuth() + .when() + .contentType(mediaType) + .accept(mediaType) + .body(body) + .post(url); + } + private void setUsername(String username) { this.username = username; } @@ -186,6 +225,7 @@ public class ScmRequests { public static final String LINK_AUTOCOMPLETE_USERS = "_links.autocomplete.find{it.name=='users'}.href"; public static final String LINK_AUTOCOMPLETE_GROUPS = "_links.autocomplete.find{it.name=='groups'}.href"; public static final String LINK_REPOSITORIES = "_links.repositories.href"; + public static final String LINK_REPOSITORY_TYPES = "_links.repositoryTypes.href"; private static final String LINK_ME = "_links.me.href"; private static final String LINK_USERS = "_links.users.href"; @@ -201,6 +241,10 @@ public class ScmRequests { return new AutoCompleteResponse<>(applyGETRequestFromLinkWithParams(response, LINK_AUTOCOMPLETE_GROUPS, "?q=" + q), this); } + public RepositoryTypeResponse requestRepositoryType(String type) { + return new RepositoryTypeResponse<>(applyGETRequestFromLinkWithParams(response, LINK_REPOSITORY_TYPES, type), this); + } + public RepositoryResponse requestRepository(String namespace, String name) { return new RepositoryResponse<>(applyGETRequestFromLinkWithParams(response, LINK_REPOSITORIES, namespace + "/" + name), this); } @@ -221,13 +265,12 @@ public class ScmRequests { return response .then() .extract() - .path("_links." + linkName + ".href"); + .path("_links." + linkName + ".href"); } } public class RepositoryResponse extends ModelResponse, PREV> { - public static final String LINKS_SOURCES = "_links.sources.href"; public static final String LINKS_CHANGESETS = "_links.changesets.href"; @@ -243,6 +286,45 @@ public class ScmRequests { return new ChangesetsResponse<>(applyGETRequestFromLink(response, LINKS_CHANGESETS), this); } + public FullExportResponse requestFullExport() { + return new FullExportResponse<>(applyGETRequestFromLinkWithParams(response, "_links.fullExport.href", "?compressed=true"), this); + } + + public void writeTestData(String value) { + applyPOSTRequestFromLink(response, "_links.test-data.href", APPLICATION_JSON, "{\"value\":\"" + value + "\"}"); + } + + public TestDataResponse requestTestData() { + return new TestDataResponse<>(applyGETRequestFromLink(response, "_links.test-data.href"), this); + } + } + + public class RepositoryTypeResponse extends ModelResponse, PREV> { + + public RepositoryTypeResponse(Response response, PREV previousResponse) { + super(response, previousResponse); + } + + public void requestImport(String type, Path file, String repositoryName) { + String url = response + .then() + .extract() + .path("_links.import.find{it.name=='" + type + "'}.href"); + Assert.assertNotNull("no url found for link " + "_links.import.find{it.name=='" + type + "'}.href", url); + + LOG.info("POST for import to {}", url); + RepositoryDto repository = new RepositoryDto(); + repository.setType("git"); + repository.setName(repositoryName); + withBasicAuth() + .multiPart("bundle", file.toFile()) + .multiPart("repository", repository) + .when() + .post(url + "?compressed=true") + .then() + .statusCode(201); + System.out.println("done"); + } } public class ChangesetsResponse extends ModelResponse, PREV> { @@ -294,6 +376,35 @@ public class ScmRequests { } } + public class FullExportResponse extends ModelResponse, PREV> { + + public FullExportResponse(Response response, PREV previousResponse) { + super(response, previousResponse); + } + + public Path exportFile() { + InputStream exportStream = response.asInputStream(); + try { + Path tempFile = Files.createTempFile("scm-export", ".tgz"); + Files.copy(exportStream, tempFile, REPLACE_EXISTING); + return tempFile; + } catch (IOException e) { + throw new RuntimeException(e); + } + } + } + + public class TestDataResponse extends ModelResponse, PREV> { + + public TestDataResponse(Response response, PREV previousResponse) { + super(response, previousResponse); + } + + public List getTestData() { + return response.then().contentType(APPLICATION_JSON).extract().path("value"); + } + } + public class ModificationsResponse extends ModelResponse, PREV> { public ModificationsResponse(Response response, PREV previousResponse) { @@ -382,7 +493,7 @@ public class ScmRequests { this.previousResponse = previousResponse; } - public Response getResponse(){ + public Response getResponse() { return response; } diff --git a/scm-dao-xml/build.gradle b/scm-persistence/build.gradle similarity index 90% rename from scm-dao-xml/build.gradle rename to scm-persistence/build.gradle index 261b18fd8c..f747c33ccd 100644 --- a/scm-dao-xml/build.gradle +++ b/scm-persistence/build.gradle @@ -22,6 +22,7 @@ plugins { dependencies { implementation libraries.commonsIo implementation libraries.commonsLang3 + implementation libraries.sqlite api platform(project(':')) @@ -31,5 +32,7 @@ dependencies { // lombok compileOnly libraries.lombok + testCompileOnly libraries.lombok annotationProcessor libraries.lombok + testAnnotationProcessor libraries.lombok } diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/CopyOnWrite.java b/scm-persistence/src/main/java/sonia/scm/CopyOnWrite.java similarity index 99% rename from scm-dao-xml/src/main/java/sonia/scm/store/CopyOnWrite.java rename to scm-persistence/src/main/java/sonia/scm/CopyOnWrite.java index fed7eba6c0..3d83b0f4f7 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/CopyOnWrite.java +++ b/scm-persistence/src/main/java/sonia/scm/CopyOnWrite.java @@ -14,11 +14,12 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm; import com.google.common.util.concurrent.Striped; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import sonia.scm.store.StoreException; import java.io.File; import java.io.IOException; diff --git a/scm-dao-xml/src/main/java/sonia/scm/group/xml/XmlGroupDAO.java b/scm-persistence/src/main/java/sonia/scm/group/xml/XmlGroupDAO.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/group/xml/XmlGroupDAO.java rename to scm-persistence/src/main/java/sonia/scm/group/xml/XmlGroupDAO.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/group/xml/XmlGroupDatabase.java b/scm-persistence/src/main/java/sonia/scm/group/xml/XmlGroupDatabase.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/group/xml/XmlGroupDatabase.java rename to scm-persistence/src/main/java/sonia/scm/group/xml/XmlGroupDatabase.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/group/xml/XmlGroupList.java b/scm-persistence/src/main/java/sonia/scm/group/xml/XmlGroupList.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/group/xml/XmlGroupList.java rename to scm-persistence/src/main/java/sonia/scm/group/xml/XmlGroupList.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/group/xml/XmlGroupMapAdapter.java b/scm-persistence/src/main/java/sonia/scm/group/xml/XmlGroupMapAdapter.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/group/xml/XmlGroupMapAdapter.java rename to scm-persistence/src/main/java/sonia/scm/group/xml/XmlGroupMapAdapter.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/MetadataStore.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/MetadataStore.java similarity index 95% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/MetadataStore.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/MetadataStore.java index 820edd9b99..1720618d99 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/MetadataStore.java +++ b/scm-persistence/src/main/java/sonia/scm/repository/xml/MetadataStore.java @@ -24,13 +24,13 @@ import org.slf4j.LoggerFactory; import sonia.scm.ContextEntry; import sonia.scm.repository.InternalRepositoryException; import sonia.scm.repository.Repository; -import sonia.scm.store.CopyOnWrite; -import sonia.scm.store.StoreConstants; +import sonia.scm.CopyOnWrite; +import sonia.scm.store.file.StoreConstants; import sonia.scm.update.UpdateStepRepositoryMetadataAccess; import java.nio.file.Path; -import static sonia.scm.store.CopyOnWrite.compute; +import static sonia.scm.CopyOnWrite.compute; public class MetadataStore implements UpdateStepRepositoryMetadataAccess { diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolver.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolver.java similarity index 98% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolver.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolver.java index 4f6668f735..a6e39be6eb 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolver.java +++ b/scm-persistence/src/main/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolver.java @@ -27,7 +27,7 @@ import sonia.scm.io.FileSystem; import sonia.scm.repository.BasicRepositoryLocationResolver; import sonia.scm.repository.InitialRepositoryLocationResolver; import sonia.scm.repository.Repository; -import sonia.scm.store.StoreConstants; +import sonia.scm.store.file.StoreConstants; import java.io.IOException; import java.nio.file.Files; @@ -89,7 +89,7 @@ public class PathBasedRepositoryLocationResolver extends BasicRepositoryLocation @Override @SuppressWarnings("unchecked") - protected RepositoryLocationResolverInstance create(Class type) { + public RepositoryLocationResolverInstance create(Class type) { if (type.isAssignableFrom(Path.class)) { return (RepositoryLocationResolverInstance) new RepositoryLocationResolverInstance() { @Override diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/PathDatabase.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/PathDatabase.java similarity index 98% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/PathDatabase.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/PathDatabase.java index 45bc13ae21..96a6a66a0e 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/PathDatabase.java +++ b/scm-persistence/src/main/java/sonia/scm/repository/xml/PathDatabase.java @@ -20,7 +20,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; import sonia.scm.ContextEntry; import sonia.scm.repository.InternalRepositoryException; -import sonia.scm.store.CopyOnWrite; +import sonia.scm.CopyOnWrite; import sonia.scm.xml.XmlStreams; import sonia.scm.xml.XmlStreams.AutoCloseableXMLReader; import sonia.scm.xml.XmlStreams.AutoCloseableXMLWriter; @@ -35,7 +35,7 @@ import java.nio.file.Path; import java.nio.file.Paths; import java.util.Map; -import static sonia.scm.store.CopyOnWrite.execute; +import static sonia.scm.CopyOnWrite.execute; class PathDatabase { diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/SingleRepositoryUpdateProcessor.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/SingleRepositoryUpdateProcessor.java similarity index 84% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/SingleRepositoryUpdateProcessor.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/SingleRepositoryUpdateProcessor.java index 48a9d1b6fd..363516e5db 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/SingleRepositoryUpdateProcessor.java +++ b/scm-persistence/src/main/java/sonia/scm/repository/xml/SingleRepositoryUpdateProcessor.java @@ -24,8 +24,12 @@ import java.util.function.BiConsumer; public class SingleRepositoryUpdateProcessor { + private final RepositoryLocationResolver locationResolver; + @Inject - private RepositoryLocationResolver locationResolver; + public SingleRepositoryUpdateProcessor(RepositoryLocationResolver locationResolver) { + this.locationResolver = locationResolver; + } public void doUpdate(BiConsumer forEachRepository) { locationResolver.forClass(Path.class).forAllLocations(forEachRepository); diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryDAO.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryDAO.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryDAO.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryDAO.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleDAO.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleDAO.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleDAO.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleDAO.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleDatabase.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleDatabase.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleDatabase.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleDatabase.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleList.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleList.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleList.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleList.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleMapAdapter.java b/scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleMapAdapter.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleMapAdapter.java rename to scm-persistence/src/main/java/sonia/scm/repository/xml/XmlRepositoryRoleMapAdapter.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java b/scm-persistence/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java similarity index 75% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java rename to scm-persistence/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java index fed7bf5f34..ea2d792218 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java +++ b/scm-persistence/src/main/java/sonia/scm/store/FileStoreUpdateStepUtilFactory.java @@ -19,17 +19,20 @@ package sonia.scm.store; import jakarta.inject.Inject; import sonia.scm.SCMContextProvider; import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.store.file.FileStoreUpdateStepUtil; import sonia.scm.update.StoreUpdateStepUtilFactory; public class FileStoreUpdateStepUtilFactory implements StoreUpdateStepUtilFactory { private final RepositoryLocationResolver locationResolver; private final SCMContextProvider contextProvider; + private final QueryableStoreFactory queryableStoreFactory; @Inject - public FileStoreUpdateStepUtilFactory(RepositoryLocationResolver locationResolver, SCMContextProvider contextProvider) { + public FileStoreUpdateStepUtilFactory(RepositoryLocationResolver locationResolver, SCMContextProvider contextProvider, QueryableStoreFactory queryableStoreFactory) { this.locationResolver = locationResolver; this.contextProvider = contextProvider; + this.queryableStoreFactory = queryableStoreFactory; } @Override @@ -37,4 +40,8 @@ public class FileStoreUpdateStepUtilFactory implements StoreUpdateStepUtilFactor return new FileStoreUpdateStepUtil(locationResolver, contextProvider, parameters, type); } + @Override + public QueryableMaintenanceStore forQueryableType(Class clazz, String... parents) { + return queryableStoreFactory.getForMaintenance(clazz, parents); + } } diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/RepositoryStoreImporter.java b/scm-persistence/src/main/java/sonia/scm/store/RepositoryStoreImporter.java similarity index 95% rename from scm-dao-xml/src/main/java/sonia/scm/store/RepositoryStoreImporter.java rename to scm-persistence/src/main/java/sonia/scm/store/RepositoryStoreImporter.java index b0ffca085e..bef0be98a8 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/RepositoryStoreImporter.java +++ b/scm-persistence/src/main/java/sonia/scm/store/RepositoryStoreImporter.java @@ -19,6 +19,7 @@ package sonia.scm.store; import jakarta.inject.Inject; import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.store.file.FileBasedStoreEntryImporterFactory; import java.nio.file.Path; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/DataFileCache.java b/scm-persistence/src/main/java/sonia/scm/store/file/DataFileCache.java similarity index 98% rename from scm-dao-xml/src/main/java/sonia/scm/store/DataFileCache.java rename to scm-persistence/src/main/java/sonia/scm/store/file/DataFileCache.java index bffe0591a9..9bde9e25ca 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/DataFileCache.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/DataFileCache.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.common.annotations.VisibleForTesting; import jakarta.inject.Inject; @@ -29,7 +29,7 @@ import java.io.File; import java.util.function.Supplier; @Singleton -public class DataFileCache { +class DataFileCache { private static final String CACHE_NAME = "sonia.cache.dataFileCache"; private static final Logger LOG = LoggerFactory.getLogger(DataFileCache.class); diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/DefaultBlobDirectoryAccess.java b/scm-persistence/src/main/java/sonia/scm/store/file/DefaultBlobDirectoryAccess.java similarity index 99% rename from scm-dao-xml/src/main/java/sonia/scm/store/DefaultBlobDirectoryAccess.java rename to scm-persistence/src/main/java/sonia/scm/store/file/DefaultBlobDirectoryAccess.java index c700495b98..8f293dd491 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/DefaultBlobDirectoryAccess.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/DefaultBlobDirectoryAccess.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import jakarta.inject.Inject; import org.slf4j.Logger; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportCopier.java b/scm-persistence/src/main/java/sonia/scm/store/file/ExportCopier.java similarity index 95% rename from scm-dao-xml/src/main/java/sonia/scm/store/ExportCopier.java rename to scm-persistence/src/main/java/sonia/scm/store/file/ExportCopier.java index 1431e59b34..2eda7b2614 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/ExportCopier.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/ExportCopier.java @@ -14,9 +14,10 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import sonia.scm.repository.api.ExportFailedException; +import sonia.scm.store.Exporter; import java.io.IOException; import java.io.OutputStream; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableBlobFileStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableBlobFileStore.java similarity index 93% rename from scm-dao-xml/src/main/java/sonia/scm/store/ExportableBlobFileStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/ExportableBlobFileStore.java index 363f2df6df..f0a999368a 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableBlobFileStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableBlobFileStore.java @@ -14,7 +14,10 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; + +import sonia.scm.store.ExportableStore; +import sonia.scm.store.StoreType; import java.nio.file.Path; import java.util.Optional; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigEntryFileStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableConfigEntryFileStore.java similarity index 85% rename from scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigEntryFileStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/ExportableConfigEntryFileStore.java index f393cb86e4..362235f93c 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigEntryFileStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableConfigEntryFileStore.java @@ -14,7 +14,12 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; + +import sonia.scm.store.ExportableStore; +import sonia.scm.store.Exporter; +import sonia.scm.store.StoreEntryMetaData; +import sonia.scm.store.StoreType; import java.io.IOException; import java.nio.file.Path; @@ -23,7 +28,7 @@ import java.util.function.Function; import static java.util.Optional.empty; import static java.util.Optional.of; -import static sonia.scm.store.ExportCopier.putFileContentIntoStream; +import static sonia.scm.store.file.ExportCopier.putFileContentIntoStream; class ExportableConfigEntryFileStore implements ExportableStore { diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigFileStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableConfigFileStore.java similarity index 85% rename from scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigFileStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/ExportableConfigFileStore.java index 8ed830e4c9..39d87e44e2 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableConfigFileStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableConfigFileStore.java @@ -14,7 +14,12 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; + +import sonia.scm.store.ExportableStore; +import sonia.scm.store.Exporter; +import sonia.scm.store.StoreEntryMetaData; +import sonia.scm.store.StoreType; import java.io.IOException; import java.nio.file.Path; @@ -23,7 +28,7 @@ import java.util.function.Function; import static java.util.Optional.empty; import static java.util.Optional.of; -import static sonia.scm.store.ExportCopier.putFileContentIntoStream; +import static sonia.scm.store.file.ExportCopier.putFileContentIntoStream; class ExportableConfigFileStore implements ExportableStore { diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDataFileStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableDataFileStore.java similarity index 92% rename from scm-dao-xml/src/main/java/sonia/scm/store/ExportableDataFileStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/ExportableDataFileStore.java index 4525310f67..cb3b6c5b84 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDataFileStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableDataFileStore.java @@ -14,7 +14,10 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; + +import sonia.scm.store.ExportableStore; +import sonia.scm.store.StoreType; import java.nio.file.Path; import java.util.Optional; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDirectoryBasedFileStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableDirectoryBasedFileStore.java similarity index 89% rename from scm-dao-xml/src/main/java/sonia/scm/store/ExportableDirectoryBasedFileStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/ExportableDirectoryBasedFileStore.java index 492b0a1bcc..8a75639568 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/ExportableDirectoryBasedFileStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/ExportableDirectoryBasedFileStore.java @@ -14,9 +14,13 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import sonia.scm.repository.api.ExportFailedException; +import sonia.scm.store.ExportableStore; +import sonia.scm.store.Exporter; +import sonia.scm.store.StoreEntryMetaData; +import sonia.scm.store.StoreType; import java.io.IOException; import java.nio.file.Files; @@ -24,7 +28,7 @@ import java.nio.file.Path; import java.util.stream.Stream; import static sonia.scm.ContextEntry.ContextBuilder.noContext; -import static sonia.scm.store.ExportCopier.putFileContentIntoStream; +import static sonia.scm.store.file.ExportCopier.putFileContentIntoStream; abstract class ExportableDirectoryBasedFileStore implements ExportableStore { diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStore.java similarity index 92% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStore.java index 42ce984458..e625cd5886 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStore.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.common.base.Preconditions; @@ -22,19 +22,18 @@ import com.google.common.base.Strings; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import sonia.scm.store.MultiEntryStore; +import sonia.scm.store.StoreException; +import sonia.scm.store.StoreReadOnlyException; import java.io.File; -public abstract class FileBasedStore implements MultiEntryStore -{ +abstract class FileBasedStore implements MultiEntryStore { - private static final Logger logger = LoggerFactory.getLogger(FileBasedStore.class); - - public FileBasedStore(File directory, String suffix, boolean readOnly) { this.directory = directory; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporter.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreEntryImporter.java similarity index 95% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporter.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreEntryImporter.java index 80d7c3fc2d..1290289d42 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporter.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreEntryImporter.java @@ -14,11 +14,12 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.common.annotations.VisibleForTesting; import sonia.scm.ContextEntry; import sonia.scm.repository.api.ImportFailedException; +import sonia.scm.store.StoreEntryImporter; import java.io.IOException; import java.io.InputStream; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporterFactory.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreEntryImporterFactory.java similarity index 85% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporterFactory.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreEntryImporterFactory.java index 43a4381c2c..8576f6de3b 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreEntryImporterFactory.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreEntryImporterFactory.java @@ -14,22 +14,25 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import sonia.scm.ContextEntry; import sonia.scm.repository.api.ImportFailedException; +import sonia.scm.store.StoreEntryImporter; +import sonia.scm.store.StoreEntryImporterFactory; +import sonia.scm.store.StoreEntryMetaData; +import sonia.scm.store.StoreType; -import java.io.File; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; -class FileBasedStoreEntryImporterFactory implements StoreEntryImporterFactory { +public class FileBasedStoreEntryImporterFactory implements StoreEntryImporterFactory { private final Path directory; - FileBasedStoreEntryImporterFactory(Path directory) { + public FileBasedStoreEntryImporterFactory(Path directory) { this.directory = directory; } diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreFactory.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreFactory.java similarity index 96% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreFactory.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreFactory.java index df5c373a55..5ce74732bf 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileBasedStoreFactory.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileBasedStoreFactory.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.slf4j.Logger; @@ -22,6 +22,8 @@ import org.slf4j.LoggerFactory; import sonia.scm.SCMContextProvider; import sonia.scm.repository.RepositoryLocationResolver; import sonia.scm.repository.RepositoryReadOnlyChecker; +import sonia.scm.store.StoreParameters; +import sonia.scm.store.TypedStoreParameters; import sonia.scm.util.IOUtil; import java.io.File; @@ -31,7 +33,7 @@ import java.nio.file.Path; * Abstract store factory for file based stores. * */ -public abstract class FileBasedStoreFactory { +abstract class FileBasedStoreFactory { private static final String NAMESPACES_DIR = "namespaces"; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBlob.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileBlob.java similarity index 94% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileBlob.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileBlob.java index 4eaad2cbca..51795c1536 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileBlob.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileBlob.java @@ -14,9 +14,11 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; +import sonia.scm.store.Blob; + import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; @@ -29,7 +31,7 @@ import java.io.OutputStream; * File base implementation of {@link Blob}. * */ -public final class FileBlob implements Blob { +final class FileBlob implements Blob { private final String id; private final File file; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBlobStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileBlobStore.java similarity index 91% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileBlobStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileBlobStore.java index 0ad83de9fb..da33b28017 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileBlobStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileBlobStore.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.common.base.Preconditions; import com.google.common.base.Strings; @@ -25,6 +25,10 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; import sonia.scm.security.KeyGenerator; +import sonia.scm.store.Blob; +import sonia.scm.store.BlobStore; +import sonia.scm.store.EntryAlreadyExistsStoreException; +import sonia.scm.store.StoreException; import java.io.File; @@ -36,7 +40,7 @@ import java.util.List; * File based implementation of {@link BlobStore}. * */ -public class FileBlobStore extends FileBasedStore implements BlobStore { +class FileBlobStore extends FileBasedStore implements BlobStore { private static final Logger LOG diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileBlobStoreFactory.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileBlobStoreFactory.java similarity index 92% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileBlobStoreFactory.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileBlobStoreFactory.java index 06edcbec74..0b612483d5 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileBlobStoreFactory.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileBlobStoreFactory.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.inject.Inject; @@ -23,6 +23,9 @@ import sonia.scm.SCMContextProvider; import sonia.scm.repository.RepositoryLocationResolver; import sonia.scm.repository.RepositoryReadOnlyChecker; import sonia.scm.security.KeyGenerator; +import sonia.scm.store.BlobStore; +import sonia.scm.store.BlobStoreFactory; +import sonia.scm.store.StoreParameters; import sonia.scm.util.IOUtil; import java.io.File; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileNamespaceUpdateIterator.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileNamespaceUpdateIterator.java similarity index 98% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileNamespaceUpdateIterator.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileNamespaceUpdateIterator.java index 0be545599a..3644b3e9ea 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileNamespaceUpdateIterator.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileNamespaceUpdateIterator.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import sonia.scm.migration.UpdateException; import sonia.scm.repository.Repository; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileRepositoryUpdateIterator.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileRepositoryUpdateIterator.java similarity index 97% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileRepositoryUpdateIterator.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileRepositoryUpdateIterator.java index b335df3348..40bed32b30 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileRepositoryUpdateIterator.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileRepositoryUpdateIterator.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import jakarta.inject.Inject; import sonia.scm.repository.RepositoryLocationResolver; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreExporter.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileStoreExporter.java similarity index 91% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileStoreExporter.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileStoreExporter.java index c2505f9c80..c81bf12682 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreExporter.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileStoreExporter.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import jakarta.inject.Inject; import org.slf4j.Logger; @@ -22,6 +22,9 @@ import org.slf4j.LoggerFactory; import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryLocationResolver; import sonia.scm.repository.api.ExportFailedException; +import sonia.scm.store.ExportableStore; +import sonia.scm.store.StoreExporter; +import sonia.scm.store.StoreType; import sonia.scm.xml.XmlStreams; import sonia.scm.xml.XmlStreams.AutoCloseableXMLReader; @@ -39,10 +42,10 @@ import java.util.stream.Stream; import static java.util.Arrays.asList; import static java.util.Collections.emptyList; import static sonia.scm.ContextEntry.ContextBuilder.noContext; -import static sonia.scm.store.ExportableBlobFileStore.BLOB_FACTORY; -import static sonia.scm.store.ExportableConfigEntryFileStore.CONFIG_ENTRY_FACTORY; -import static sonia.scm.store.ExportableConfigFileStore.CONFIG_FACTORY; -import static sonia.scm.store.ExportableDataFileStore.DATA_FACTORY; +import static sonia.scm.store.file.ExportableBlobFileStore.BLOB_FACTORY; +import static sonia.scm.store.file.ExportableConfigEntryFileStore.CONFIG_ENTRY_FACTORY; +import static sonia.scm.store.file.ExportableConfigFileStore.CONFIG_FACTORY; +import static sonia.scm.store.file.ExportableDataFileStore.DATA_FACTORY; public class FileStoreExporter implements StoreExporter { diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtil.java b/scm-persistence/src/main/java/sonia/scm/store/file/FileStoreUpdateStepUtil.java similarity index 87% rename from scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtil.java rename to scm-persistence/src/main/java/sonia/scm/store/file/FileStoreUpdateStepUtil.java index 12809032df..1572075d5d 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/FileStoreUpdateStepUtil.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/FileStoreUpdateStepUtil.java @@ -14,11 +14,13 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import sonia.scm.SCMContextProvider; import sonia.scm.migration.UpdateException; import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.store.StoreParameters; +import sonia.scm.store.StoreType; import sonia.scm.update.StoreUpdateStepUtilFactory; import sonia.scm.util.IOUtil; @@ -26,7 +28,7 @@ import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; -class FileStoreUpdateStepUtil implements StoreUpdateStepUtilFactory.StoreUpdateStepUtil { +public class FileStoreUpdateStepUtil implements StoreUpdateStepUtilFactory.StoreUpdateStepUtil { private final RepositoryLocationResolver locationResolver; private final SCMContextProvider contextProvider; @@ -34,7 +36,7 @@ class FileStoreUpdateStepUtil implements StoreUpdateStepUtilFactory.StoreUpdateS private final StoreParameters parameters; private final StoreType type; - FileStoreUpdateStepUtil(RepositoryLocationResolver locationResolver, SCMContextProvider contextProvider, StoreParameters parameters, StoreType type) { + public FileStoreUpdateStepUtil(RepositoryLocationResolver locationResolver, SCMContextProvider contextProvider, StoreParameters parameters, StoreType type) { this.locationResolver = locationResolver; this.contextProvider = contextProvider; this.parameters = parameters; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationEntryStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationEntryStore.java similarity index 96% rename from scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationEntryStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationEntryStore.java index b740f64708..e031cbdc26 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationEntryStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationEntryStore.java @@ -14,14 +14,16 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.common.collect.Maps; import jakarta.xml.bind.JAXBElement; import jakarta.xml.bind.Marshaller; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import sonia.scm.CopyOnWrite; import sonia.scm.security.KeyGenerator; +import sonia.scm.store.ConfigurationEntryStore; import sonia.scm.xml.XmlStreams; import sonia.scm.xml.XmlStreams.AutoCloseableXMLReader; import sonia.scm.xml.XmlStreams.AutoCloseableXMLWriter; @@ -33,9 +35,9 @@ import java.util.Map; import java.util.Map.Entry; import static javax.xml.stream.XMLStreamConstants.END_ELEMENT; -import static sonia.scm.store.CopyOnWrite.execute; +import static sonia.scm.CopyOnWrite.execute; -public class JAXBConfigurationEntryStore implements ConfigurationEntryStore { +class JAXBConfigurationEntryStore implements ConfigurationEntryStore { private static final String TAG_CONFIGURATION = "configuration"; private static final String TAG_ENTRY = "entry"; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationEntryStoreFactory.java b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationEntryStoreFactory.java similarity index 88% rename from scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationEntryStoreFactory.java rename to scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationEntryStoreFactory.java index 735b70ab34..53b4e53aba 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationEntryStoreFactory.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationEntryStoreFactory.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.inject.Inject; import com.google.inject.Singleton; @@ -22,6 +22,9 @@ import sonia.scm.SCMContextProvider; import sonia.scm.repository.RepositoryLocationResolver; import sonia.scm.repository.RepositoryReadOnlyChecker; import sonia.scm.security.KeyGenerator; +import sonia.scm.store.ConfigurationEntryStore; +import sonia.scm.store.ConfigurationEntryStoreFactory; +import sonia.scm.store.TypedStoreParameters; @Singleton @@ -38,11 +41,11 @@ public class JAXBConfigurationEntryStoreFactory extends FileBasedStoreFactory RepositoryLocationResolver repositoryLocationResolver, KeyGenerator keyGenerator, RepositoryReadOnlyChecker readOnlyChecker, - StoreCacheConfigProvider storeCacheConfigProvider + StoreCacheFactory storeCacheFactory ) { super(contextProvider, repositoryLocationResolver, Store.CONFIG, readOnlyChecker); this.keyGenerator = keyGenerator; - this.storeCache = new StoreCache<>(this::createStore, storeCacheConfigProvider.isStoreCacheEnabled()); + this.storeCache = storeCacheFactory.createStoreCache(this::createStore); } @Override diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationStore.java similarity index 87% rename from scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationStore.java index 3dba2360d8..f0f78ee68f 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationStore.java @@ -14,25 +14,29 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import sonia.scm.CopyOnWrite; +import sonia.scm.store.AbstractStore; +import sonia.scm.store.ConfigurationStore; +import sonia.scm.store.StoreException; import sonia.scm.util.IOUtil; import java.io.File; import java.io.IOException; import java.util.function.BooleanSupplier; -import static sonia.scm.store.CopyOnWrite.compute; -import static sonia.scm.store.CopyOnWrite.execute; +import static sonia.scm.CopyOnWrite.compute; +import static sonia.scm.CopyOnWrite.execute; /** * JAXB implementation of {@link ConfigurationStore}. * * @param */ -public class JAXBConfigurationStore extends AbstractStore { +class JAXBConfigurationStore extends AbstractStore { private static final Logger LOG = LoggerFactory.getLogger(JAXBConfigurationStore.class); diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationStoreFactory.java b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationStoreFactory.java similarity index 86% rename from scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationStoreFactory.java rename to scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationStoreFactory.java index 49afaaf31a..609e2b09f1 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBConfigurationStoreFactory.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBConfigurationStoreFactory.java @@ -14,13 +14,18 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.inject.Inject; import com.google.inject.Singleton; import sonia.scm.SCMContextProvider; import sonia.scm.repository.RepositoryLocationResolver; import sonia.scm.repository.RepositoryReadOnlyChecker; +import sonia.scm.store.ConfigurationStore; +import sonia.scm.store.ConfigurationStoreDecoratorFactory; +import sonia.scm.store.ConfigurationStoreFactory; +import sonia.scm.store.StoreDecoratorFactory; +import sonia.scm.store.TypedStoreParameters; import java.util.Set; @@ -41,11 +46,11 @@ public class JAXBConfigurationStoreFactory extends FileBasedStoreFactory impleme RepositoryLocationResolver repositoryLocationResolver, RepositoryReadOnlyChecker readOnlyChecker, Set decoratorFactories, - StoreCacheConfigProvider storeCacheConfigProvider + StoreCacheFactory storeCacheFactory ) { super(contextProvider, repositoryLocationResolver, Store.CONFIG, readOnlyChecker); this.decoratorFactories = decoratorFactories; - this.storeCache = new StoreCache<>(this::createStore, storeCacheConfigProvider.isStoreCacheEnabled()); + this.storeCache = storeCacheFactory.createStoreCache(this::createStore); } @Override diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBDataStore.java b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBDataStore.java similarity index 92% rename from scm-dao-xml/src/main/java/sonia/scm/store/JAXBDataStore.java rename to scm-persistence/src/main/java/sonia/scm/store/file/JAXBDataStore.java index 8020245efa..2a58261433 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBDataStore.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBDataStore.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.common.collect.ImmutableMap; import com.google.common.collect.ImmutableMap.Builder; @@ -22,21 +22,24 @@ import jakarta.xml.bind.JAXBException; import jakarta.xml.bind.Marshaller; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import sonia.scm.CopyOnWrite; import sonia.scm.security.KeyGenerator; +import sonia.scm.store.DataStore; +import sonia.scm.store.StoreException; import sonia.scm.xml.XmlStreams; import java.io.File; import java.util.Map; import java.util.Objects; -import static sonia.scm.store.CopyOnWrite.compute; +import static sonia.scm.CopyOnWrite.compute; /** * Jaxb implementation of {@link DataStore}. * * @param type of stored data. */ -public class JAXBDataStore extends FileBasedStore implements DataStore { +class JAXBDataStore extends FileBasedStore implements DataStore { private static final Logger LOG = LoggerFactory.getLogger(JAXBDataStore.class); diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBDataStoreFactory.java b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBDataStoreFactory.java similarity index 89% rename from scm-dao-xml/src/main/java/sonia/scm/store/JAXBDataStoreFactory.java rename to scm-persistence/src/main/java/sonia/scm/store/file/JAXBDataStoreFactory.java index 104df5fcc5..908c3af02e 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBDataStoreFactory.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBDataStoreFactory.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.inject.Inject; import com.google.inject.Singleton; @@ -22,6 +22,9 @@ import sonia.scm.SCMContextProvider; import sonia.scm.repository.RepositoryLocationResolver; import sonia.scm.repository.RepositoryReadOnlyChecker; import sonia.scm.security.KeyGenerator; +import sonia.scm.store.DataStore; +import sonia.scm.store.DataStoreFactory; +import sonia.scm.store.TypedStoreParameters; import sonia.scm.util.IOUtil; import java.io.File; @@ -44,12 +47,12 @@ public class JAXBDataStoreFactory extends FileBasedStoreFactory KeyGenerator keyGenerator, RepositoryReadOnlyChecker readOnlyChecker, DataFileCache dataFileCache, - StoreCacheConfigProvider storeCacheConfigProvider + StoreCacheFactory storeCacheFactory ) { super(contextProvider, repositoryLocationResolver, Store.DATA, readOnlyChecker); this.keyGenerator = keyGenerator; this.dataFileCache = dataFileCache; - this.storeCache = new StoreCache<>(this::createStore, storeCacheConfigProvider.isStoreCacheEnabled()); + this.storeCache = storeCacheFactory.createStoreCache(this::createStore); } @Override diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBPropertyFileAccess.java b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBPropertyFileAccess.java similarity index 99% rename from scm-dao-xml/src/main/java/sonia/scm/store/JAXBPropertyFileAccess.java rename to scm-persistence/src/main/java/sonia/scm/store/file/JAXBPropertyFileAccess.java index 2980ea0cea..47e422e0b2 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/JAXBPropertyFileAccess.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/JAXBPropertyFileAccess.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import jakarta.inject.Inject; import org.slf4j.Logger; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/Store.java b/scm-persistence/src/main/java/sonia/scm/store/file/Store.java similarity index 97% rename from scm-dao-xml/src/main/java/sonia/scm/store/Store.java rename to scm-persistence/src/main/java/sonia/scm/store/file/Store.java index dfb8624c13..b6dc42655a 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/Store.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/Store.java @@ -14,7 +14,9 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; + +import sonia.scm.store.StoreType; import java.io.File; diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/StoreCache.java b/scm-persistence/src/main/java/sonia/scm/store/file/StoreCache.java similarity index 78% rename from scm-dao-xml/src/main/java/sonia/scm/store/StoreCache.java rename to scm-persistence/src/main/java/sonia/scm/store/file/StoreCache.java index 14749d8f80..b0b5d0ef3d 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/StoreCache.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/StoreCache.java @@ -14,10 +14,11 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import sonia.scm.store.TypedStoreParameters; import java.util.HashMap; import java.util.Map; @@ -29,20 +30,28 @@ class StoreCache { private static final Logger LOG = LoggerFactory.getLogger(StoreCache.class); - private final Function, S> cachingStoreCreator; + private final Map, S> storeCache; + StoreCache(Function, S> storeCreator, Boolean storeCacheEnabled) { if (storeCacheEnabled) { LOG.info("store cache enabled"); - Map, S> storeCache = synchronizedMap(new HashMap<>()); + storeCache = synchronizedMap(new HashMap<>()); cachingStoreCreator = storeParameters -> storeCache.computeIfAbsent(storeParameters, storeCreator); } else { cachingStoreCreator = storeCreator; + storeCache = null; } } S getStore(TypedStoreParameters storeParameters) { return cachingStoreCreator.apply(storeParameters); } + + void clearCache(String repositoryId) { + if (storeCache != null) { + storeCache.keySet().removeIf(parameters -> repositoryId.equals(parameters.getRepositoryId())); + } + } } diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/StoreCacheConfigProvider.java b/scm-persistence/src/main/java/sonia/scm/store/file/StoreCacheConfigProvider.java similarity index 97% rename from scm-dao-xml/src/main/java/sonia/scm/store/StoreCacheConfigProvider.java rename to scm-persistence/src/main/java/sonia/scm/store/file/StoreCacheConfigProvider.java index 75d27a6ac6..38325becf8 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/StoreCacheConfigProvider.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/StoreCacheConfigProvider.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import jakarta.inject.Inject; import sonia.scm.EagerSingleton; diff --git a/scm-persistence/src/main/java/sonia/scm/store/file/StoreCacheFactory.java b/scm-persistence/src/main/java/sonia/scm/store/file/StoreCacheFactory.java new file mode 100644 index 0000000000..6b29afebe6 --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/file/StoreCacheFactory.java @@ -0,0 +1,50 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.file; + +import com.github.legman.Subscribe; +import jakarta.inject.Inject; +import jakarta.inject.Singleton; +import sonia.scm.repository.ClearRepositoryCacheEvent; +import sonia.scm.store.TypedStoreParameters; + +import java.util.Collection; +import java.util.LinkedList; +import java.util.function.Function; + +@Singleton +public class StoreCacheFactory { + + private final StoreCacheConfigProvider storeCacheConfigProvider; + private final Collection> caches = new LinkedList<>(); + + @Inject + public StoreCacheFactory(StoreCacheConfigProvider storeCacheConfigProvider) { + this.storeCacheConfigProvider = storeCacheConfigProvider; + } + + StoreCache createStoreCache(Function, S> storeCreator) { + StoreCache cache = new StoreCache<>(storeCreator, storeCacheConfigProvider.isStoreCacheEnabled()); + caches.add(cache); + return cache; + } + + @Subscribe(async = false) + public void clearCache(ClearRepositoryCacheEvent event) { + caches.forEach(storeCache -> storeCache.clearCache(event.getRepository().getId())); + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/StoreConstants.java b/scm-persistence/src/main/java/sonia/scm/store/file/StoreConstants.java similarity index 97% rename from scm-dao-xml/src/main/java/sonia/scm/store/StoreConstants.java rename to scm-persistence/src/main/java/sonia/scm/store/file/StoreConstants.java index aa9d0d1aaa..09cbf38a53 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/StoreConstants.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/StoreConstants.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; /** * Store constants for xml implementations. diff --git a/scm-dao-xml/src/main/java/sonia/scm/store/TypedStoreContext.java b/scm-persistence/src/main/java/sonia/scm/store/file/TypedStoreContext.java similarity index 97% rename from scm-dao-xml/src/main/java/sonia/scm/store/TypedStoreContext.java rename to scm-persistence/src/main/java/sonia/scm/store/file/TypedStoreContext.java index 5cb94561a3..5aa8a46491 100644 --- a/scm-dao-xml/src/main/java/sonia/scm/store/TypedStoreContext.java +++ b/scm-persistence/src/main/java/sonia/scm/store/file/TypedStoreContext.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import jakarta.xml.bind.JAXBContext; import jakarta.xml.bind.JAXBException; @@ -22,6 +22,8 @@ import jakarta.xml.bind.Marshaller; import jakarta.xml.bind.Unmarshaller; import jakarta.xml.bind.annotation.adapters.XmlAdapter; import lombok.extern.slf4j.Slf4j; +import sonia.scm.store.StoreException; +import sonia.scm.store.TypedStoreParameters; import sonia.scm.xml.XmlStreams; import java.io.File; diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/BadStoreNameException.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/BadStoreNameException.java new file mode 100644 index 0000000000..cc02ceac5f --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/BadStoreNameException.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import sonia.scm.store.StoreException; + +/** + * This exception is thrown if a name for a store element doesn't meet the internal verification requirements. + * + * @since 3.7.0 + */ +class BadStoreNameException extends StoreException { + BadStoreNameException(String badName) { + super("This name has been rejected: " + badName); + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/ConditionalSQLStatement.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/ConditionalSQLStatement.java new file mode 100644 index 0000000000..a0c4c09c95 --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/ConditionalSQLStatement.java @@ -0,0 +1,49 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import java.sql.PreparedStatement; +import java.sql.SQLException; +import java.util.List; + +abstract class ConditionalSQLStatement implements SQLNodeWithValue { + + private final List whereCondition; + + ConditionalSQLStatement(List whereCondition) { + this.whereCondition = whereCondition; + } + + void appendWhereClause(StringBuilder query) { + if (!whereCondition.isEmpty()) { + query.append(" WHERE ").append(new SQLLogicalCondition("AND", whereCondition).toSQL()); + } + } + + @Override + public int apply(PreparedStatement statement, int index) throws SQLException { + for (SQLNodeWithValue condition : whereCondition) { + index = condition.apply(statement, index); + } + return index; + } + + @Override + public String toString() { + return "SQL statement: " + toSQL(); + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/LoggingReadWriteLock.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/LoggingReadWriteLock.java new file mode 100644 index 0000000000..7c7ccc965a --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/LoggingReadWriteLock.java @@ -0,0 +1,122 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import lombok.extern.slf4j.Slf4j; + +import java.util.concurrent.TimeUnit; +import java.util.concurrent.locks.Condition; +import java.util.concurrent.locks.Lock; +import java.util.concurrent.locks.ReadWriteLock; + +@Slf4j +class LoggingReadWriteLock implements ReadWriteLock { + + private static int rwLockCounter = 0; + private static int lockCounter = 0; + + private final ReadWriteLock delegate; + private final int nr; + + LoggingReadWriteLock(ReadWriteLock delegate) { + this.delegate = delegate; + synchronized (LoggingReadWriteLock.class) { + nr = ++rwLockCounter; + } + } + + @Override + public Lock readLock() { + return new LoggingLock(nr, delegate.readLock(), "read"); + } + + @Override + public Lock writeLock() { + return new LoggingLock(nr, delegate.writeLock(), "write"); + } + + private static class LoggingLock implements Lock { + + private final int nr; + private final int subNr; + private final Lock delegate; + private final String purpose; + private long lockStart; + + private LoggingLock(int nr, Lock delegate, String purpose) { + this.nr = nr; + this.delegate = delegate; + this.purpose = purpose; + synchronized (LoggingReadWriteLock.class) { + subNr = ++lockCounter; + } + } + + @Override + public void lock() { + log.trace("request {} lock for lock nr {}-{}", purpose, nr, subNr); + delegate.lock(); + lockStart = System.nanoTime(); + log.trace("got {} lock for lock nr {}-{}", purpose, nr, subNr); + } + + @Override + public void lockInterruptibly() throws InterruptedException { + log.trace("try interruptibly {} lock for lock nr {}-{}", purpose, nr, subNr); + delegate.lockInterruptibly(); + lockStart = System.nanoTime(); + log.trace("got {} lock for lock nr {}-{}", purpose, nr, subNr); + } + + @Override + public boolean tryLock() { + log.trace("try {} lock for lock nr {}-{}", purpose, nr, subNr); + boolean result = delegate.tryLock(); + if (result) { + lockStart = System.nanoTime(); + } + log.trace("result for {} lock for lock nr {}-{}: {}", purpose, nr, subNr, result); + return result; + } + + @Override + public boolean tryLock(long l, TimeUnit timeUnit) throws InterruptedException { + log.trace("try {} lock for lock nr {}-{}", purpose, nr, subNr); + boolean result = delegate.tryLock(l, timeUnit); + if (result) { + lockStart = System.nanoTime(); + } + log.trace("result for {} lock for lock nr {}-{}: {}", purpose, nr, subNr, result); + return result; + } + + @Override + public void unlock() { + log.trace("release {} lock for lock nr {}-{}", purpose, nr, subNr); + delegate.unlock(); + long duration = System.nanoTime() - lockStart; + log.trace("{} lock released after {}ns for lock nr {}-{}", purpose, duration, nr, subNr); + lockStart = 0; + } + + @Override + public Condition newCondition() { + return delegate.newCondition(); + } + } +} + diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLCondition.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLCondition.java new file mode 100644 index 0000000000..37205f28e0 --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLCondition.java @@ -0,0 +1,159 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import lombok.Getter; +import lombok.Setter; +import sonia.scm.store.LeafCondition; +import sonia.scm.store.Operator; +import sonia.scm.store.QueryableStore; + +import java.sql.PreparedStatement; +import java.sql.SQLException; +import java.time.Instant; + +import static sonia.scm.store.sqlite.SQLiteIdentifiers.computeColumnIdentifier; + +/** + * SQLCondition represents a condition given in an agnostic SQL statement. + * + * @since 3.7.0 + */ +@Getter +@Setter +class SQLCondition implements SQLNodeWithValue { + private String operatorPrefix; + private String operatorPostfix; + private SQLField field; + private SQLValue value; + + public SQLCondition(String operator, SQLField field, SQLValue value) { + this(operator, "", field, value); + } + + public SQLCondition(String operatorPrefix, String operatorPostfix, SQLField field, SQLValue value) { + this.operatorPrefix = operatorPrefix; + this.operatorPostfix = operatorPostfix; + this.field = field; + this.value = value; + } + + public static SQLCondition createConditionWithOperatorAndValue(LeafCondition leafCondition) { + QueryableStore.QueryField queryField = leafCondition.getField(); + String operatorPrefix = mapOperatorPrefix(leafCondition.getOperator()); + String operatorPostfix = mapOperatorPostfix(leafCondition.getOperator()); + SQLField field = new SQLField(computeSQLField(queryField)); + SQLValue value = determineValueBasedOnOperator(leafCondition); + if (queryField instanceof QueryableStore.CollectionQueryField) { + return new ExistsSQLCondition(operatorPrefix, field, value); + } else if (queryField instanceof QueryableStore.MapQueryField) { + return new ExistsSQLCondition(operatorPrefix, field, value); + } else { + return new SQLCondition(operatorPrefix, operatorPostfix, field, value); + } + } + + private static String mapOperatorPrefix(Operator operator) { + return switch (operator) { + case EQ -> "="; + case LESS -> "<"; + case LESS_OR_EQUAL -> "<="; + case GREATER -> ">"; + case GREATER_OR_EQUAL -> ">="; + case CONTAINS -> "LIKE '%' ||"; + case NULL -> "IS NULL"; + case IN -> "IN"; + case KEY -> "key ="; + case VALUE -> "value ="; + }; + } + + private static String mapOperatorPostfix(Operator operator) { + return switch (operator) { + case CONTAINS -> "|| '%'"; + default -> ""; + }; + } + + private static String computeSQLField(QueryableStore.QueryField queryField) { + if (queryField instanceof QueryableStore.CollectionQueryField) { + return "select * from json_each(payload, '$." + queryField.getName() + "') where value "; + } else if (queryField instanceof QueryableStore.MapQueryField) { + return "select * from json_each(payload, '$." + queryField.getName() + "') where "; + } else if (queryField instanceof QueryableStore.InstantQueryField) { + return "json_extract(payload, '$." + queryField.getName() + "')"; + } else if (queryField instanceof QueryableStore.CollectionSizeQueryField) { + return "json_array_length(payload, '$." + queryField.getName() + "')"; + } else if (queryField instanceof QueryableStore.MapSizeQueryField) { + return "(select count(*) from json_each(payload, '$." + queryField.getName() + "')) "; + } else if (queryField.isIdField()) { + return computeColumnIdentifier(queryField.getName()); + } else { + return "json_extract(payload, '$." + queryField.getName() + "')"; + } + } + + private static SQLValue determineValueBasedOnOperator(LeafCondition leafCondition) { + Operator operator = leafCondition.getOperator(); + Object value = leafCondition.getValue(); + + switch (operator) { + case NULL: + return new SQLValue(null); + + case IN: + if (value instanceof Object[] valueArray) { + return new SQLValue(valueArray); + } else { + throw new IllegalArgumentException("Value for IN operator must be an array."); + } + + default: + return new SQLValue(computeParameter(leafCondition)); + } + } + + private static Object computeParameter(LeafCondition leafCondition) { + if (leafCondition.getField() instanceof QueryableStore.InstantQueryField) { + return ((Instant) leafCondition.getValue()).toEpochMilli(); + } else { + return leafCondition.getValue(); + } + } + + @Override + public String toSQL() { + String fieldSQL = (field != null) ? field.toSQL() : ""; + return fieldSQL + " " + operatorPrefix + " " + value.toSQL() + " " + operatorPostfix + " "; + } + + @Override + public int apply(PreparedStatement statement, int index) throws SQLException { + return value.apply(statement, index); + } + + private static class ExistsSQLCondition extends SQLCondition { + public ExistsSQLCondition(String operator, SQLField field, SQLValue value) { + super(operator, field, value); + } + + @Override + public String toSQL() { + return "exists(" + super.toSQL() + ")"; + } + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLConditionMapper.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLConditionMapper.java new file mode 100644 index 0000000000..7902d122eb --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLConditionMapper.java @@ -0,0 +1,60 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import sonia.scm.store.Condition; +import sonia.scm.store.LeafCondition; +import sonia.scm.store.LogicalCondition; + +import java.util.ArrayList; +import java.util.List; + +class SQLConditionMapper { + + /** + * Maps a LogicalCondition to an SQLLogicalCondition and appends its value to the provided parameters list. + * + * @param logicalCondition The condition to map. + * @return A new SQLCondition object representing the mapped condition. + */ + static SQLLogicalCondition mapToSQLLogicalCondition(LogicalCondition logicalCondition) { + + List sqlConditions = new ArrayList<>(); + for (Condition condition : logicalCondition.getConditions()) { + if (condition instanceof LeafCondition) { + sqlConditions.add(SQLConditionMapper.mapToSQLCondition((LeafCondition) condition)); + } else { + sqlConditions.add(SQLConditionMapper.mapToSQLLogicalCondition((LogicalCondition) condition)); + } + } + + return new SQLLogicalCondition( + logicalCondition.getOperator().toString(), + sqlConditions + ); + } + + /** + * Maps a LeafCondition to an SQLCondition and appends its value to the provided parameters list. + * + * @param leafCondition The condition to map. + * @return A new SQLCondition object representing the mapped condition. + */ + static SQLCondition mapToSQLCondition(LeafCondition leafCondition) { + return SQLCondition.createConditionWithOperatorAndValue(leafCondition); + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLDeleteStatement.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLDeleteStatement.java new file mode 100644 index 0000000000..774e6a83fe --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLDeleteStatement.java @@ -0,0 +1,38 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import java.util.List; + +class SQLDeleteStatement extends ConditionalSQLStatement { + + private final SQLTable fromTable; + + SQLDeleteStatement(SQLTable fromTable, List whereCondition) { + super(whereCondition); + this.fromTable = fromTable; + + } + + @Override + public String toSQL() { + StringBuilder query = new StringBuilder(); + query.append("DELETE FROM ").append(fromTable.toSQL()); + appendWhereClause(query); + return query.toString(); + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLField.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLField.java new file mode 100644 index 0000000000..965e92ede0 --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLField.java @@ -0,0 +1,41 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import lombok.Getter; + +/** + * Representation of a value of a row within an {@link SQLTable}. + * + * @since 3.7.0 + */ +@Getter +class SQLField implements SQLNode { + + static final SQLField PAYLOAD = new SQLField("json(payload)"); + + private final String name; + + SQLField(String name) { + this.name = name; + } + + @Override + public String toSQL() { + return name; + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLInsertStatement.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLInsertStatement.java new file mode 100644 index 0000000000..0b745b23bc --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLInsertStatement.java @@ -0,0 +1,46 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import java.sql.PreparedStatement; +import java.sql.SQLException; + +class SQLInsertStatement implements SQLNodeWithValue { + + private final SQLTable fromTable; + private final SQLValue values; + + SQLInsertStatement(SQLTable fromTable, SQLValue values) { + this.fromTable = fromTable; + this.values = values; + } + + @Override + public String toSQL() { + return "REPLACE INTO " + fromTable.toSQL() + " VALUES " + values.toSQL(); + } + + @Override + public int apply(PreparedStatement statement, int index) throws SQLException { + return values.apply(statement, index); + } + + @Override + public String toString() { + return "SQL insert statement: " + toSQL(); + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLLogicalCondition.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLLogicalCondition.java new file mode 100644 index 0000000000..b0a4b3ad81 --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLLogicalCondition.java @@ -0,0 +1,69 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.sql.PreparedStatement; +import java.sql.SQLException; +import java.util.List; + +@Getter +@Setter +@NoArgsConstructor +class SQLLogicalCondition implements SQLNodeWithValue { + private String operator; // AND, OR + private List conditions; + + SQLLogicalCondition(String operator, List conditions) { + this.operator = operator; + this.conditions = conditions; + } + + @Override + public String toSQL() { + if (conditions == null || conditions.isEmpty()) { + return ""; + } + + StringBuilder sql = new StringBuilder(); + + if (operator.equals("NOT")) { + sql.append("NOT "); + } + + for (int i = 0; i < conditions.size(); i++) { + if (i > 0) { + sql.append(" ").append(operator).append(" "); + } + String conditionSQL = conditions.get(i).toSQL(); + sql.append("(").append(conditionSQL).append(")"); + } + return sql.toString(); + } + + @Override + public int apply(PreparedStatement statement, int index) throws SQLException { + int currentIndex = index; + for (SQLNodeWithValue condition : conditions) { + currentIndex = condition.apply(statement, currentIndex); + } + return currentIndex; + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLNode.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLNode.java new file mode 100644 index 0000000000..c7e0f4c475 --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLNode.java @@ -0,0 +1,21 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +interface SQLNode { + String toSQL(); +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLNodeWithValue.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLNodeWithValue.java new file mode 100644 index 0000000000..9b1d728bfa --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLNodeWithValue.java @@ -0,0 +1,24 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import java.sql.PreparedStatement; +import java.sql.SQLException; + +interface SQLNodeWithValue extends SQLNode { + int apply(PreparedStatement statement, int index) throws SQLException; +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLSelectStatement.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLSelectStatement.java new file mode 100644 index 0000000000..f6b9249727 --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLSelectStatement.java @@ -0,0 +1,72 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import java.util.List; +import java.util.stream.Collectors; + +class SQLSelectStatement extends ConditionalSQLStatement { + + private final List columns; + private final SQLTable fromTable; + private final String orderBy; + private final long limit; + private final long offset; + + SQLSelectStatement(List columns, SQLTable fromTable, List whereCondition) { + this(columns, fromTable, whereCondition, null, 0, 0); + } + + SQLSelectStatement(List columns, SQLTable fromTable, List whereCondition, String orderBy, long limit, long offset) { + super(whereCondition); + this.columns = columns; + this.fromTable = fromTable; + this.orderBy = orderBy; + this.limit = limit; + this.offset = offset; + } + + @Override + public String toSQL() { + StringBuilder query = new StringBuilder(); + + query.append("SELECT "); + if (columns != null && !columns.isEmpty()) { + String columnList = columns.stream() + .map(SQLField::toSQL) + .collect(Collectors.joining(", ")); + query.append(columnList); + } + query.append(" FROM ").append(fromTable.toSQL()); + + appendWhereClause(query); + + if (orderBy != null && !orderBy.isEmpty()) { + query.append(" ORDER BY ").append(orderBy); + } + + if (limit > 0) { + query.append(" LIMIT ").append(limit); + } + + if (offset > 0) { + query.append(" OFFSET ").append(offset); + } + + return query.toString(); + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLTable.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLTable.java new file mode 100644 index 0000000000..3c09a1111d --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLTable.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +class SQLTable implements SQLNode { + private final String name; + + SQLTable(String name) { + this.name = name; + } + + @Override + public String toSQL() { + return name; + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLValue.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLValue.java new file mode 100644 index 0000000000..6510378f8d --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLValue.java @@ -0,0 +1,96 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import lombok.extern.slf4j.Slf4j; + +import java.sql.PreparedStatement; +import java.sql.SQLException; +import java.util.List; + +/** + * Representation of a column or a list of columns within an {@link SQLTable}. + * + * @since 3.7.0 + */ +@Slf4j +class SQLValue implements SQLNodeWithValue { + private final Object value; + + SQLValue(Object value) { + this.value = value; + } + + @Override + public String toSQL() { + if (value == null) { + return ""; + } + + if (value instanceof Object[] valueArray) { + return generatePlaceholders(valueArray); + } else if (value instanceof List valueList) { + return generatePlaceholders(valueList); + } + + return "?"; + } + + @Override + public int apply(PreparedStatement statement, int index) throws SQLException { + if (value instanceof Object[] valueArray) { + for (int i = 0; i < valueArray.length; i++) { + set(index + i, valueArray[i], statement); + } + return index + valueArray.length; + } else if (value instanceof List valueList) { + for (int i = 0; i < valueList.size(); i++) { + set(index + i, valueList.get(i), statement); + } + return index + valueList.size(); + } else if (value == null) { + return index; + } else { + set(index, value, statement); + return index + 1; + } + } + + private static void set(int index, Object value, PreparedStatement statement) throws SQLException { + log.trace("set index {} to value '{}'", index, value); + statement.setObject(index, value); + } + + private String generatePlaceholders(Object[] valueArray) { + return generatePlaceholders(valueArray.length); + } + + private String generatePlaceholders(List valueList) { + return generatePlaceholders(valueList.size()); + } + + private String generatePlaceholders(int length) { + StringBuilder placeholdersBuilder = new StringBuilder(); + for (int i = 0; i < length; i++) { + if (i > 0) { + placeholdersBuilder.append(", "); + } + placeholdersBuilder.append("?"); + } + return "(" + placeholdersBuilder + ")"; + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteIdentifiers.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteIdentifiers.java new file mode 100644 index 0000000000..49028ea9bb --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteIdentifiers.java @@ -0,0 +1,67 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import com.google.common.base.Strings; +import sonia.scm.plugin.QueryableTypeDescriptor; + +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +class SQLiteIdentifiers { + + private static final Pattern PATTERN = Pattern.compile("^\\w+$"); + private static final String STORE_TABLE_SUFFIX = "_STORE"; + + static String computeTableName(QueryableTypeDescriptor queryableTypeDescriptor) { + if (Strings.isNullOrEmpty(queryableTypeDescriptor.getName())) { + String className = queryableTypeDescriptor.getClazz(); + return sanitize(computeSqlIdentifier(removeClassSuffix(className)) + STORE_TABLE_SUFFIX); + } else { + return sanitize(queryableTypeDescriptor.getName() + STORE_TABLE_SUFFIX); + } + } + + static String computeColumnIdentifier(String className) { + if (className == null) { + return "ID"; + } + String nameWithoutClassSuffix = removeClassSuffix(className); + String classNameWithoutPackage = nameWithoutClassSuffix.substring(nameWithoutClassSuffix.lastIndexOf('.') + 1); + return computeSqlIdentifier(classNameWithoutPackage) + "_ID"; + } + + private static String computeSqlIdentifier(String className) { + return sanitize(className.replace("_", "__").replace('.', '_')); + } + + private static String removeClassSuffix(String className) { + if (className.endsWith(".class")) { + return className.substring(0, className.length() - 6); + } + return className; + } + + static String sanitize(String name) throws BadStoreNameException { + Matcher matcher = PATTERN.matcher(name); + if (!matcher.matches()) { + throw new BadStoreNameException(name); + } else { + return name; + } + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableMutableStore.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableMutableStore.java new file mode 100644 index 0000000000..8513e2fccd --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableMutableStore.java @@ -0,0 +1,169 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import lombok.extern.slf4j.Slf4j; +import sonia.scm.plugin.QueryableTypeDescriptor; +import sonia.scm.security.KeyGenerator; +import sonia.scm.store.QueryableMutableStore; +import sonia.scm.store.StoreException; + +import java.sql.Connection; +import java.sql.ResultSet; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.locks.ReadWriteLock; + +@Slf4j +class SQLiteQueryableMutableStore extends SQLiteQueryableStore implements QueryableMutableStore { + + private final ObjectMapper objectMapper; + private final KeyGenerator keyGenerator; + + private final Class clazz; + private final String[] parentIds; + + public SQLiteQueryableMutableStore(ObjectMapper objectMapper, + KeyGenerator keyGenerator, + Connection connection, + Class clazz, + QueryableTypeDescriptor queryableTypeDescriptor, + String[] parentIds, + ReadWriteLock lock) { + super(objectMapper, connection, clazz, queryableTypeDescriptor, parentIds, lock); + this.objectMapper = objectMapper; + this.keyGenerator = keyGenerator; + this.clazz = clazz; + this.parentIds = parentIds; + } + + @Override + public String put(T item) { + String id = keyGenerator.createKey(); + put(id, item); + return id; + } + + @Override + public void put(String id, T item) { + List columnsToInsert = new ArrayList<>(Arrays.asList(parentIds)); + columnsToInsert.add(id); + columnsToInsert.add(marshal(item)); + SQLInsertStatement sqlInsertStatement = + new SQLInsertStatement( + computeFromTable(), + new SQLValue(columnsToInsert) + ); + + executeWrite( + sqlInsertStatement, + statement -> { + statement.executeUpdate(); + return null; + } + ); + } + + @Override + public Map getAll() { + List columns = List.of( + SQLField.PAYLOAD, + new SQLField("ID") + ); + + SQLSelectStatement sqlStatementQuery = + new SQLSelectStatement( + columns, + computeFromTable(), + computeConditionsForAllValues() + ); + + return executeRead( + sqlStatementQuery, + statement -> { + HashMap result = new HashMap<>(); + ResultSet resultSet = statement.executeQuery(); + while (resultSet.next()) { + result.put(resultSet.getString(2), objectMapper.readValue(resultSet.getString(1), clazz)); + } + return Collections.unmodifiableMap(result); + } + ); + } + + @Override + public void remove(String id) { + SQLDeleteStatement sqlStatementQuery = + new SQLDeleteStatement( + computeFromTable(), + computeConditionsFor(id) + ); + + executeWrite( + sqlStatementQuery, + statement -> { + statement.executeUpdate(); + return null; + } + ); + } + + @Override + public T get(String id) { + SQLSelectStatement sqlStatementQuery = + new SQLSelectStatement( + List.of(SQLField.PAYLOAD), + computeFromTable(), + computeConditionsFor(id) + ); + + return executeRead( + sqlStatementQuery, + statement -> { + ResultSet resultSet = statement.executeQuery(); + if (!resultSet.next()) { + return null; + } + String json = resultSet.getString(1); + if (json == null) { + return null; + } + return objectMapper.readValue(json, clazz); + } + ); + } + + private String marshal(T item) { + try { + return objectMapper.writeValueAsString(item); + } catch (JsonProcessingException e) { + throw new StoreException("Failed to marshal item as json", e); + } + } + + private List computeConditionsFor(String id) { + List conditions = computeConditionsForAllValues(); + conditions.add(new SQLCondition("=", new SQLField("ID"), new SQLValue(id))); + return conditions; + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableStore.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableStore.java new file mode 100644 index 0000000000..1da7a2cffe --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableStore.java @@ -0,0 +1,705 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import lombok.extern.slf4j.Slf4j; +import sonia.scm.plugin.QueryableTypeDescriptor; +import sonia.scm.store.Condition; +import sonia.scm.store.Conditions; +import sonia.scm.store.LeafCondition; +import sonia.scm.store.LogicalCondition; +import sonia.scm.store.QueryableMaintenanceStore; +import sonia.scm.store.QueryableStore; +import sonia.scm.store.StoreException; + +import java.sql.Connection; +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.sql.Statement; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.Iterator; +import java.util.LinkedHashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.NoSuchElementException; +import java.util.Optional; +import java.util.concurrent.locks.Lock; +import java.util.concurrent.locks.ReadWriteLock; +import java.util.function.BooleanSupplier; +import java.util.stream.Stream; + +import static sonia.scm.store.sqlite.SQLiteIdentifiers.computeColumnIdentifier; +import static sonia.scm.store.sqlite.SQLiteIdentifiers.computeTableName; + +@Slf4j +class SQLiteQueryableStore implements QueryableStore, QueryableMaintenanceStore { + + public static final String TEMPORARY_UPDATE_TABLE_NAME = "update_tmp"; + private final ObjectMapper objectMapper; + private final Connection connection; + + private final Class clazz; + private final QueryableTypeDescriptor queryableTypeDescriptor; + private final String[] parentIds; + + private final ReadWriteLock lock; + + public SQLiteQueryableStore(ObjectMapper objectMapper, + Connection connection, + Class clazz, + QueryableTypeDescriptor queryableTypeDescriptor, + String[] parentIds, + ReadWriteLock lock) { + this.objectMapper = objectMapper; + this.connection = connection; + this.clazz = clazz; + this.parentIds = parentIds; + this.queryableTypeDescriptor = queryableTypeDescriptor; + this.lock = lock; + } + + @Override + public Query query(Condition... conditions) { + return new SQLiteQuery<>(clazz, conditions); + } + + @Override + public void clear() { + List parentConditions = new ArrayList<>(); + evaluateParentConditions(parentConditions); + + SQLDeleteStatement sqlStatementQuery = + new SQLDeleteStatement( + computeFromTable(), + parentConditions + ); + + executeWrite( + sqlStatementQuery, + statement -> { + statement.executeUpdate(); + return null; + } + ); + } + + @Override + public Collection readRaw() { + return readAllAs(RawRow::new); + } + + @Override + public Collection> readAll() { + return readAllAs(clazz); + } + + @Override + public Collection> readAllAs(Class type) { + return readAllAs((parentIds, id, json) -> new Row<>(parentIds, id, objectMapper.readValue(json, type))); + } + + private Collection readAllAs(RowBuilder rowBuilder) { + List parentConditions = new ArrayList<>(); + evaluateParentConditions(parentConditions); + List fields = new ArrayList<>(); + addParentIdSQLFields(fields); + int parentIdsLength = fields.size() - 1; // addParentIdSQLFields has already added the ID field + fields.add(new SQLField("PAYLOAD")); + SQLSelectStatement sqlSelectQuery = + new SQLSelectStatement( + fields, + computeFromTable(), + parentConditions + ); + return executeRead( + sqlSelectQuery, + statement -> { + List result = new ArrayList<>(); + ResultSet resultSet = statement.executeQuery(); + String[] allParentIds = new String[parentIdsLength]; + while (resultSet.next()) { + for (int i = 0; i < parentIdsLength; i++) { + allParentIds[i] = resultSet.getString(i + 1); + } + String id = resultSet.getString(parentIdsLength + 1); + String json = resultSet.getString(parentIdsLength + 2); + result.add(rowBuilder.build(allParentIds, id, json)); + } + return Collections.unmodifiableList(result); + } + ); + } + + @Override + @SuppressWarnings("rawtypes") + public void writeAll(Stream rows) { + writeRaw(rows.map(row -> new RawRow(row.getParentIds(), row.getId(), serialize(row.getValue())))); + } + + @Override + public void writeRaw(Stream rows) { + transactional( + () -> { + rows.forEach(row -> { + List columnsToInsert = new ArrayList<>(Arrays.asList(row.getParentIds())); + // overwrite parentIds from the export with the parentIds of the current store: + for (int i = 0; i < parentIds.length; i++) { + columnsToInsert.set(i, parentIds[i]); + } + columnsToInsert.add(row.getId()); + columnsToInsert.add(row.getValue()); + SQLInsertStatement sqlInsertStatement = + new SQLInsertStatement( + computeFromTable(), + new SQLValue(columnsToInsert) + ); + + executeWrite( + sqlInsertStatement, + statement -> { + statement.executeUpdate(); + return null; + } + ); + }); + return true; + } + ); + } + + @Override + public MaintenanceIterator iterateAll() { + List columns = new LinkedList<>(); + columns.add(new SQLField("payload")); + addParentIdSQLFields(columns); + + return new TemporaryTableMaintenanceIterator(columns); + } + + public void transactional(BooleanSupplier callback) { + log.debug("start transactional operation"); + Lock writeLock = lock.writeLock(); + writeLock.lock(); + try { + getConnection().setAutoCommit(false); + boolean commit = callback.getAsBoolean(); + if (commit) { + log.debug("commit operation"); + getConnection().commit(); + } else { + log.debug("rollback operation"); + getConnection().rollback(); + } + log.debug("operation finished"); + } catch (SQLException e) { + throw new StoreException("failed to disable auto-commit", e); + } finally { + writeLock.unlock(); + } + } + + List computeConditionsForAllValues() { + List conditions = new ArrayList<>(); + evaluateParentConditions(conditions); + return conditions; + } + + SQLTable computeFromTable() { + return new SQLTable(computeTableName(queryableTypeDescriptor)); + } + + R executeRead(SQLNodeWithValue sqlStatement, StatementCallback callback) { + String sql = sqlStatement.toSQL(); + log.debug("executing 'read' SQL: {}", sql); + return executeWithLock(sqlStatement, callback, lock.readLock(), sql); + } + + R executeWrite(SQLNodeWithValue sqlStatement, StatementCallback callback) { + String sql = sqlStatement.toSQL(); + log.debug("executing 'write' SQL: {}", sql); + return executeWithLock(sqlStatement, callback, lock.writeLock(), sql); + } + + private R executeWithLock(SQLNodeWithValue sqlStatement, StatementCallback callback, Lock writeLock, String sql) { + writeLock.lock(); + try (PreparedStatement statement = connection.prepareStatement(sql)) { + sqlStatement.apply(statement, 1); + return callback.apply(statement); + } catch (SQLException | JsonProcessingException e) { + throw new StoreException("An exception occurred while executing a query on SQLite database: " + sql, e); + } finally { + writeLock.unlock(); + } + } + + @Override + public void close() { + try { + log.debug("closing connection"); + connection.close(); + } catch (SQLException e) { + throw new StoreException("failed to close connection", e); + } + } + + Connection getConnection() { + return connection; + } + + private class SQLiteQuery implements Query { + + private final Class resultType; + private final Class entityType; + private final Condition condition; + private final List> orderBy; + + private SQLiteQuery(Class resultType, Condition[] conditions) { + this(resultType, resultType, conjunct(conditions), Collections.emptyList()); + } + + @SuppressWarnings({"rawtypes", "unchecked"}) + private SQLiteQuery(Class resultType, Class entityType, Condition condition, List> orderBy) { + this.resultType = resultType; + this.entityType = entityType; + this.condition = condition; + this.orderBy = orderBy; + } + + @Override + public Optional findFirst() { + return findAll(0, 1).stream().findFirst(); + } + + @Override + public Optional findOne() { + List all = findAll(0, 2); + if (all.size() > 1) { + throw new TooManyResultsException(); + } else if (all.size() == 1) { + return Optional.of(all.get(0)); + } else { + return Optional.empty(); + } + } + + @Override + public List findAll() { + return findAll(0, Integer.MAX_VALUE); + } + + @Override + public List findAll(long offset, long limit) { + StringBuilder orderByBuilder = new StringBuilder(); + if (orderBy != null && !orderBy.isEmpty()) { + toOrderBySQL(orderByBuilder); + } + + SQLSelectStatement sqlSelectQuery = + new SQLSelectStatement( + computeFields(), + computeFromTable(), + computeCondition(), + orderByBuilder.toString(), + limit, + offset + ); + + return executeRead( + sqlSelectQuery, + statement -> { + List result = new ArrayList<>(); + ResultSet resultSet = statement.executeQuery(); + while (resultSet.next()) { + result.add(extractResult(resultSet)); + } + return Collections.unmodifiableList(result); + } + ); + } + + @Override + @SuppressWarnings("unchecked") + public Query> withIds() { + return new SQLiteQuery<>((Class>) (Class) Result.class, resultType, condition, orderBy); + } + + @Override + public long count() { + SQLSelectStatement sqlStatementQuery = + new SQLSelectStatement( + List.of(new SQLField("COUNT(*)")), + computeFromTable(), + computeCondition() + ); + + return executeRead( + sqlStatementQuery, + statement -> { + ResultSet resultSet = statement.executeQuery(); + if (resultSet.next()) { + return resultSet.getLong(1); + } + throw new IllegalStateException("failed to read count for type " + queryableTypeDescriptor); + } + ); + } + + @Override + public Query orderBy(QueryField field, Order order) { + List> extendedOrderBy = new ArrayList<>(this.orderBy); + extendedOrderBy.add(new OrderBy<>(field, order)); + return new SQLiteQuery<>(resultType, entityType, condition, extendedOrderBy); + } + + private List computeFields() { + List fields = new ArrayList<>(); + fields.add(SQLField.PAYLOAD); + if (resultType.isAssignableFrom(Result.class)) { + addParentIdSQLFields(fields); + } + return fields; + } + + private List computeCondition() { + List conditions = new ArrayList<>(); + + evaluateParentConditions(conditions); + + if (condition != null) { + if (condition instanceof LeafCondition leafCondition) { + SQLCondition sqlCondition = SQLConditionMapper.mapToSQLCondition(leafCondition); + conditions.add(sqlCondition); + } + if (condition instanceof LogicalCondition logicalCondition) { + SQLLogicalCondition sqlLogicalCondition = SQLConditionMapper.mapToSQLLogicalCondition(logicalCondition); + conditions.add(sqlLogicalCondition); + } + } + + return conditions; + } + + private void toOrderBySQL(StringBuilder orderByBuilder) { + Iterator> it = orderBy.iterator(); + while (it.hasNext()) { + OrderBy order = it.next(); + orderByBuilder.append("json_extract(payload, '$.").append(order.field.getName()).append("') ").append(order.order.name()); + if (it.hasNext()) { + orderByBuilder.append(", "); + } + } + } + + @SuppressWarnings("unchecked") + private T_RESULT extractResult(ResultSet resultSet) throws JsonProcessingException, SQLException { + T entity = objectMapper.readValue(resultSet.getString(1), entityType); + if (resultType.isAssignableFrom(Result.class)) { + Map parentIds = new HashMap<>(queryableTypeDescriptor.getTypes().length); + for (int i = 0; i < queryableTypeDescriptor.getTypes().length; i++) { + parentIds.put(computeColumnIdentifier(queryableTypeDescriptor.getTypes()[i]), resultSet.getString(i + 2)); + } + String id = resultSet.getString(queryableTypeDescriptor.getTypes().length + 2); + return (T_RESULT) new Result() { + @Override + public Optional getParentId(Class clazz) { + String parentClassName = computeColumnIdentifier(clazz.getName()); + return Optional.ofNullable(parentIds.get(parentClassName)); + } + + @Override + public String getId() { + return id; + } + + @Override + public T getEntity() { + return entity; + } + }; + } else { + return (T_RESULT) entity; + } + } + + private static Condition conjunct(Condition[] conditions) { + if (conditions.length == 0) { + return null; + } else if (conditions.length == 1) { + return conditions[0]; + } else { + return Conditions.and(conditions); + } + } + } + + private void evaluateParentConditions(List conditions) { + for (int i = 0; i < parentIds.length; i++) { + SQLCondition condition = new SQLCondition("=", new SQLField(computeColumnIdentifier(queryableTypeDescriptor.getTypes()[i])), new SQLValue(parentIds[i])); + conditions.add(condition); + } + } + + private void addParentIdSQLFields(List fields) { + for (int i = 0; i < queryableTypeDescriptor.getTypes().length; i++) { + fields.add(new SQLField(computeColumnIdentifier(queryableTypeDescriptor.getTypes()[i]))); + } + fields.add(new SQLField("ID")); + } + + interface StatementCallback { + R apply(PreparedStatement statement) throws SQLException, JsonProcessingException; + } + + record OrderBy(QueryField field, Order order) { + } + + private class TemporaryTableMaintenanceIterator implements MaintenanceIterator { + private final PreparedStatement iterateStatement; + private final List columns; + private final ResultSet resultSet; + private Boolean hasNext; + + public TemporaryTableMaintenanceIterator(List columns) { + this.columns = columns; + this.hasNext = null; + SQLSelectStatement iterateQuery = + new SQLSelectStatement( + columns, + computeFromTable(), + computeConditionsForAllValues() + ); + String sql = iterateQuery.toSQL(); + log.debug("iterating SQL: {}", sql); + try { + iterateStatement = connection.prepareStatement(sql); + iterateQuery.apply(iterateStatement, 1); + resultSet = iterateStatement.executeQuery(); + } catch (SQLException e) { + throw new StoreException("Failed to iterate: " + sql, e); + } + + createTemporaryTable(); + } + + private void createTemporaryTable() { + dropTemporaryTable(); + StringBuilder tmpTableStatement = new StringBuilder("create table if not exists ").append(TEMPORARY_UPDATE_TABLE_NAME).append(" ("); + for (int i = 0; i < queryableTypeDescriptor.getTypes().length; i++) { + tmpTableStatement.append(computeColumnIdentifier(queryableTypeDescriptor.getTypes()[i])).append(" TEXT NOT NULL, "); + } + tmpTableStatement.append("ID TEXT NOT NULL, payload JSONB)"); + try (Statement statement = connection.createStatement()) { + String createTableSql = tmpTableStatement.toString(); + log.debug("creating table: {}", createTableSql); + statement.execute(createTableSql); + } catch (SQLException e) { + throw new StoreException("Failed to create temporary table: " + tmpTableStatement, e); + } + } + + private void dropTemporaryTable() { + String sql = "DROP TABLE IF EXISTS " + TEMPORARY_UPDATE_TABLE_NAME; + try (Statement statement = connection.createStatement()) { + log.trace("dropping table: {}", sql); + statement.executeUpdate(sql); + } catch (SQLException e) { + throw new StoreException("Failed to drop temporary table: " + sql, e); + } + } + + @Override + public boolean hasNext() { + if (hasNext != null) { + return hasNext; + } + try { + hasNext = resultSet.next(); + return hasNext; + } catch (SQLException e) { + throw new StoreException("Failed to get next row from result set", e); + } + } + + @Override + public MaintenanceStoreEntry next() { + if (!hasNext()) { + throw new NoSuchElementException(); + } + hasNext = null; + return new InnerStoreEntry(); + } + + @Override + public void remove() { + List parentConditions = new ArrayList<>(); + try { + for (int i = 0; i < queryableTypeDescriptor.getTypes().length; i++) { + String columnName = computeColumnIdentifier(queryableTypeDescriptor.getTypes()[i]); + parentConditions.add(new SQLCondition("=", new SQLField(columnName), new SQLValue(resultSet.getString(columnName)))); + } + parentConditions.add(new SQLCondition("=", new SQLField("ID"), new SQLValue(resultSet.getString("ID")))); + } catch (SQLException e) { + throw new StoreException("Failed to delete item from table", e); + } + SQLDeleteStatement sqlStatementQuery = + new SQLDeleteStatement( + computeFromTable(), + parentConditions + ); + + executeWrite( + sqlStatementQuery, + statement -> { + statement.executeUpdate(); + return null; + } + ); + } + + @Override + public void close() throws Exception { + iterateStatement.close(); + + SQLSelectStatement tmpIterateQuery = + new SQLSelectStatement( + columns, + new SQLTable(TEMPORARY_UPDATE_TABLE_NAME), + computeConditionsForAllValues() + ); + String sql = tmpIterateQuery.toSQL(); + log.debug("iterating temporary table: {}", sql); + iterateStatement.close(); + try (PreparedStatement tmpIterateStatement = connection.prepareStatement(sql)) { + tmpIterateQuery.apply(tmpIterateStatement, 1); + ResultSet tmpResultSet = tmpIterateStatement.executeQuery(); + while (tmpResultSet.next()) { + Collection allParentIds = computeAllParentIds(tmpResultSet); + writeJsonInTable( + computeFromTable(), + allParentIds, + tmpResultSet.getString(queryableTypeDescriptor.getTypes().length + 2), + tmpResultSet.getString(1) + ); + } + } catch (SQLException e) { + throw new StoreException("Failed to transfer entries from temporary table", e); + } + dropTemporaryTable(); + } + + private List computeAllParentIds(ResultSet tmpResultSet) throws SQLException { + List allParentIds = new ArrayList<>(); + for (int columnNr = 0; columnNr < queryableTypeDescriptor.getTypes().length; ++columnNr) { + allParentIds.add(tmpResultSet.getString(columnNr + 2)); + } + return allParentIds; + } + + private void writeJsonInTable(SQLTable table, Collection allParentIds, String id, String json) { + List columnsToInsert = new ArrayList<>(allParentIds); + columnsToInsert.add(id); + columnsToInsert.add(json); + SQLInsertStatement sqlInsertStatement = + new SQLInsertStatement( + table, + new SQLValue(columnsToInsert) + ); + + executeWrite( + sqlInsertStatement, + statement -> { + statement.executeUpdate(); + return null; + } + ); + } + + private class InnerStoreEntry implements MaintenanceStoreEntry { + + private final Map parentIds = new LinkedHashMap<>(); + private final String id; + private final String json; + + InnerStoreEntry() { + try { + json = resultSet.getString(1); + for (int i = 0; i < queryableTypeDescriptor.getTypes().length; i++) { + parentIds.put(computeColumnIdentifier(queryableTypeDescriptor.getTypes()[i]), resultSet.getString(i + 2)); + } + id = resultSet.getString(queryableTypeDescriptor.getTypes().length + 2); + } catch (SQLException e) { + throw new StoreException("Failed to read next entry for maintenance", e); + } + } + + @Override + public String getId() { + return id; + } + + @Override + public Optional getParentId(Class clazz) { + String parentClassName = computeColumnIdentifier(clazz.getName()); + return Optional.ofNullable(parentIds.get(parentClassName)); + } + + @Override + public T get() { + return getAs(clazz); + } + + @Override + public U getAs(Class type) { + try { + return objectMapper.readValue(json, type); + } catch (JsonProcessingException e) { + throw new SerializationException("failed to read object from json", e); + } + } + + void updateJson(String json) { + SQLTable table = new SQLTable(TEMPORARY_UPDATE_TABLE_NAME); + writeJsonInTable(table, parentIds.values(), id, json); + } + + @Override + public void update(Object object) { + updateJson(serialize(object)); + } + } + } + + private String serialize(Object object) { + try { + return objectMapper.writeValueAsString(object); + } catch (JsonProcessingException e) { + throw new SerializationException("failed to serialize object to json", e); + } + } + + private interface RowBuilder { + R build(String[] parentIds, String id, String json) throws JsonProcessingException; + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableStoreFactory.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableStoreFactory.java new file mode 100644 index 0000000000..77c4820e24 --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteQueryableStoreFactory.java @@ -0,0 +1,136 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.google.common.annotations.VisibleForTesting; +import jakarta.inject.Inject; +import jakarta.inject.Singleton; +import lombok.extern.slf4j.Slf4j; +import org.sqlite.SQLiteConfig; +import org.sqlite.SQLiteDataSource; +import sonia.scm.SCMContextProvider; +import sonia.scm.plugin.PluginLoader; +import sonia.scm.plugin.QueryableTypeDescriptor; +import sonia.scm.security.KeyGenerator; +import sonia.scm.store.QueryableMaintenanceStore; +import sonia.scm.store.QueryableStoreFactory; +import sonia.scm.store.StoreException; + +import javax.sql.DataSource; +import java.nio.file.Path; +import java.sql.Connection; +import java.sql.SQLException; +import java.util.HashMap; +import java.util.Map; +import java.util.concurrent.locks.ReadWriteLock; +import java.util.concurrent.locks.ReentrantReadWriteLock; + +import static com.fasterxml.jackson.databind.DeserializationFeature.READ_DATE_TIMESTAMPS_AS_NANOSECONDS; +import static com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATES_AS_TIMESTAMPS; +import static com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS; + +@Slf4j +@Singleton +public class SQLiteQueryableStoreFactory implements QueryableStoreFactory { + + private final ObjectMapper objectMapper; + private final KeyGenerator keyGenerator; + private final DataSource dataSource; + + private final Map queryableTypes = new HashMap<>(); + + private final ReadWriteLock lock = new LoggingReadWriteLock(new ReentrantReadWriteLock()); + + @Inject + public SQLiteQueryableStoreFactory(SCMContextProvider contextProvider, + PluginLoader pluginLoader, + ObjectMapper objectMapper, + KeyGenerator keyGenerator) { + this( + "jdbc:sqlite:" + contextProvider.resolve(Path.of("scm.db")), + objectMapper + .copy() + .configure(WRITE_DATES_AS_TIMESTAMPS, true) + .configure(WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS, false) + .configure(READ_DATE_TIMESTAMPS_AS_NANOSECONDS, false), + keyGenerator, + pluginLoader.getExtensionProcessor().getQueryableTypes() + ); + } + + @VisibleForTesting + public SQLiteQueryableStoreFactory(String connectionString, + ObjectMapper objectMapper, + KeyGenerator keyGenerator, + Iterable queryableTypeIterable) { + SQLiteConfig config = new SQLiteConfig(); + config.setSharedCache(true); + config.setJournalMode(SQLiteConfig.JournalMode.WAL); + + this.dataSource = new SQLiteDataSource( + config + ); + ((SQLiteDataSource) dataSource).setUrl(connectionString); + this.objectMapper = objectMapper; + this.keyGenerator = keyGenerator; + Connection connection = openDefaultConnection(); + try { + TableCreator tableCreator = new TableCreator(connection); + for (QueryableTypeDescriptor queryableTypeDescriptor : queryableTypeIterable) { + queryableTypes.put(queryableTypeDescriptor.getClazz(), queryableTypeDescriptor); + tableCreator.initializeTable(queryableTypeDescriptor); + } + } finally { + try { + connection.close(); + } catch (SQLException e) { + log.warn("could not close connection", e); + } + } + } + + private Connection openDefaultConnection() { + try { + log.debug("open connection"); + Connection connection = dataSource.getConnection(); + connection.setAutoCommit(true); + return connection; + } catch (SQLException e) { + throw new StoreException("could not connect to database", e); + } + } + + @Override + public SQLiteQueryableStore getReadOnly(Class clazz, String... parentIds) { + return new SQLiteQueryableStore<>(objectMapper, openDefaultConnection(), clazz, getQueryableTypeDescriptor(clazz), parentIds, lock); + } + + @Override + public QueryableMaintenanceStore getForMaintenance(Class clazz, String... parentIds) { + return new SQLiteQueryableStore<>(objectMapper, openDefaultConnection(), clazz, getQueryableTypeDescriptor(clazz), parentIds, lock); + } + + @Override + public SQLiteQueryableMutableStore getMutable(Class clazz, String... parentIds) { + return new SQLiteQueryableMutableStore<>(objectMapper, keyGenerator, openDefaultConnection(), clazz, getQueryableTypeDescriptor(clazz), parentIds, lock); + } + + private QueryableTypeDescriptor getQueryableTypeDescriptor(Class clazz) { + return queryableTypes.get(clazz.getName().replace('$', '.')); + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteStoreMetaDataProvider.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteStoreMetaDataProvider.java new file mode 100644 index 0000000000..a756954cdb --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/SQLiteStoreMetaDataProvider.java @@ -0,0 +1,75 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import jakarta.inject.Inject; +import jakarta.inject.Singleton; +import lombok.extern.slf4j.Slf4j; +import sonia.scm.plugin.PluginLoader; +import sonia.scm.plugin.QueryableTypeDescriptor; +import sonia.scm.store.StoreException; +import sonia.scm.store.StoreMetaDataProvider; + +import java.util.Arrays; +import java.util.Collection; +import java.util.HashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; + +@Slf4j +@Singleton +public class SQLiteStoreMetaDataProvider implements StoreMetaDataProvider { + + private final Map, Collection>> typesForParents = new HashMap<>(); + private final ClassLoader classLoader; + + @Inject + SQLiteStoreMetaDataProvider(PluginLoader pluginLoader) { + classLoader = pluginLoader.getUberClassLoader(); + Iterable queryableTypes = pluginLoader.getExtensionProcessor().getQueryableTypes(); + queryableTypes.forEach(this::initializeType); + } + + private void initializeType(QueryableTypeDescriptor descriptor) { + for (int i = 0; i < descriptor.getTypes().length; i++) { + Collection parentClasses = + Arrays.stream(Arrays.copyOf(descriptor.getTypes(), i + 1)) + .map(SQLiteStoreMetaDataProvider::removeTrailingClass) + .toList(); + Collection> classes = typesForParents.computeIfAbsent(parentClasses, k -> new LinkedList<>()); + try { + classes.add(classLoader.loadClass(descriptor.getClazz())); + } catch (ClassNotFoundException e) { + throw new StoreException("Failed to load class '" + descriptor.getClazz() + "' for queryable type descriptor " + descriptor.getName(), e); + } + } + } + + private static String removeTrailingClass(String parentClass) { + return parentClass.endsWith(".class") ? parentClass.substring(0, parentClass.length() - ".class".length()) : parentClass; + } + + @Override + public Collection> getTypesWithParent(Class... classes) { + Collection classNames = + Arrays.stream(classes) + .map(Class::getName) + .toList(); + return typesForParents.getOrDefault(classNames, List.of()); + } +} diff --git a/scm-persistence/src/main/java/sonia/scm/store/sqlite/TableCreator.java b/scm-persistence/src/main/java/sonia/scm/store/sqlite/TableCreator.java new file mode 100644 index 0000000000..80879755bd --- /dev/null +++ b/scm-persistence/src/main/java/sonia/scm/store/sqlite/TableCreator.java @@ -0,0 +1,109 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import lombok.extern.slf4j.Slf4j; +import sonia.scm.plugin.QueryableTypeDescriptor; +import sonia.scm.store.StoreException; + +import java.sql.Connection; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.util.Collection; +import java.util.LinkedList; + +import static sonia.scm.store.sqlite.SQLiteIdentifiers.computeColumnIdentifier; +import static sonia.scm.store.sqlite.SQLiteIdentifiers.computeTableName; + +@Slf4j +class TableCreator { + + private final Connection connection; + + TableCreator(Connection connection) { + this.connection = connection; + } + + void initializeTable(QueryableTypeDescriptor descriptor) { + log.info("initializing table for {}", descriptor); + + String tableName = computeTableName(descriptor); + + Collection columns = getColumns(tableName); + + if (columns.isEmpty()) { + createTable(descriptor, tableName); + } else if (!columns.contains("ID")) { + log.error("table {} exists but does not contain ID column", tableName); + throw new StoreException("Table " + tableName + " exists but does not contain ID column"); + } else if (!columns.contains("payload")) { + log.error("table {} exists but does not contain payload column", tableName); + throw new StoreException("Table " + tableName + " exists but does not contain payload column"); + } else { + for (String type : descriptor.getTypes()) { + String column = computeColumnIdentifier(type); + if (!columns.contains(column)) { + log.error("table {} exists but does not contain column {}", tableName, column); + throw new StoreException("Table " + tableName + " exists but does not contain column " + column); + } + } + if (descriptor.getTypes().length != columns.size() - 2) { + log.error("table {} exists but has too many columns", tableName); + throw new StoreException("Table " + tableName + " exists but has too many columns"); + } + } + } + + private void createTable(QueryableTypeDescriptor descriptor, String tableName) { + StringBuilder builder = new StringBuilder("CREATE TABLE ") + .append(tableName) + .append(" ("); + for (String type : descriptor.getTypes()) { + builder.append(computeColumnIdentifier(type)).append(" TEXT NOT NULL, "); + } + builder.append("ID TEXT NOT NULL, payload JSONB"); + builder.append(", PRIMARY KEY ("); + for (String type : descriptor.getTypes()) { + builder.append(computeColumnIdentifier(type)).append(", "); + } + builder.append("ID)"); + builder.append(')'); + try { + log.info("creating table {} for {}", tableName, descriptor); + log.trace("sql: {}", builder); + boolean result = connection.createStatement().execute(builder.toString()); + log.trace("created: {}", result); + } catch (SQLException e) { + throw new StoreException("Failed to create table for class " + descriptor.getClazz() + ": " + builder, e); + } + } + + Collection getColumns(String tableName) { + log.debug("checking table {}", tableName); + try { + ResultSet resultSet = connection.createStatement().executeQuery("PRAGMA table_info(" + tableName + ")"); + Collection columns = new LinkedList<>(); + while (resultSet.next()) { + columns.add(resultSet.getString("name")); + } + resultSet.close(); + return columns; + } catch (SQLException e) { + throw new StoreException("Failed to get columns for table " + tableName, e); + } + } +} diff --git a/scm-dao-xml/src/main/java/sonia/scm/update/xml/XmlV1PropertyDAO.java b/scm-persistence/src/main/java/sonia/scm/update/xml/XmlV1PropertyDAO.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/update/xml/XmlV1PropertyDAO.java rename to scm-persistence/src/main/java/sonia/scm/update/xml/XmlV1PropertyDAO.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/user/xml/XmlUserDAO.java b/scm-persistence/src/main/java/sonia/scm/user/xml/XmlUserDAO.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/user/xml/XmlUserDAO.java rename to scm-persistence/src/main/java/sonia/scm/user/xml/XmlUserDAO.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/user/xml/XmlUserDatabase.java b/scm-persistence/src/main/java/sonia/scm/user/xml/XmlUserDatabase.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/user/xml/XmlUserDatabase.java rename to scm-persistence/src/main/java/sonia/scm/user/xml/XmlUserDatabase.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/user/xml/XmlUserList.java b/scm-persistence/src/main/java/sonia/scm/user/xml/XmlUserList.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/user/xml/XmlUserList.java rename to scm-persistence/src/main/java/sonia/scm/user/xml/XmlUserList.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/user/xml/XmlUserMapAdapter.java b/scm-persistence/src/main/java/sonia/scm/user/xml/XmlUserMapAdapter.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/user/xml/XmlUserMapAdapter.java rename to scm-persistence/src/main/java/sonia/scm/user/xml/XmlUserMapAdapter.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/xml/AbstractXmlDAO.java b/scm-persistence/src/main/java/sonia/scm/xml/AbstractXmlDAO.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/xml/AbstractXmlDAO.java rename to scm-persistence/src/main/java/sonia/scm/xml/AbstractXmlDAO.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/xml/XmlDatabase.java b/scm-persistence/src/main/java/sonia/scm/xml/XmlDatabase.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/xml/XmlDatabase.java rename to scm-persistence/src/main/java/sonia/scm/xml/XmlDatabase.java diff --git a/scm-dao-xml/src/main/java/sonia/scm/xml/XmlStreams.java b/scm-persistence/src/main/java/sonia/scm/xml/XmlStreams.java similarity index 100% rename from scm-dao-xml/src/main/java/sonia/scm/xml/XmlStreams.java rename to scm-persistence/src/main/java/sonia/scm/xml/XmlStreams.java diff --git a/scm-dao-xml/src/test/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolverTest.java b/scm-persistence/src/test/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolverTest.java similarity index 100% rename from scm-dao-xml/src/test/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolverTest.java rename to scm-persistence/src/test/java/sonia/scm/repository/xml/PathBasedRepositoryLocationResolverTest.java diff --git a/scm-dao-xml/src/test/java/sonia/scm/repository/xml/XmlRepositoryDAOSynchronizationTest.java b/scm-persistence/src/test/java/sonia/scm/repository/xml/XmlRepositoryDAOSynchronizationTest.java similarity index 100% rename from scm-dao-xml/src/test/java/sonia/scm/repository/xml/XmlRepositoryDAOSynchronizationTest.java rename to scm-persistence/src/test/java/sonia/scm/repository/xml/XmlRepositoryDAOSynchronizationTest.java diff --git a/scm-dao-xml/src/test/java/sonia/scm/repository/xml/XmlRepositoryDAOTest.java b/scm-persistence/src/test/java/sonia/scm/repository/xml/XmlRepositoryDAOTest.java similarity index 100% rename from scm-dao-xml/src/test/java/sonia/scm/repository/xml/XmlRepositoryDAOTest.java rename to scm-persistence/src/test/java/sonia/scm/repository/xml/XmlRepositoryDAOTest.java diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreUpdateStepUtilFactoryTest.java b/scm-persistence/src/test/java/sonia/scm/store/FileStoreUpdateStepUtilFactoryTest.java similarity index 100% rename from scm-dao-xml/src/test/java/sonia/scm/store/FileStoreUpdateStepUtilFactoryTest.java rename to scm-persistence/src/test/java/sonia/scm/store/FileStoreUpdateStepUtilFactoryTest.java diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/RepositoryStoreImporterTest.java b/scm-persistence/src/test/java/sonia/scm/store/RepositoryStoreImporterTest.java similarity index 100% rename from scm-dao-xml/src/test/java/sonia/scm/store/RepositoryStoreImporterTest.java rename to scm-persistence/src/test/java/sonia/scm/store/RepositoryStoreImporterTest.java diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/CopyOnWriteTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/CopyOnWriteTest.java similarity index 97% rename from scm-dao-xml/src/test/java/sonia/scm/store/CopyOnWriteTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/CopyOnWriteTest.java index 20aa8972a9..3eeea2c9ae 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/CopyOnWriteTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/CopyOnWriteTest.java @@ -14,10 +14,11 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.io.TempDir; +import sonia.scm.store.StoreException; import java.io.FileOutputStream; import java.io.IOException; @@ -28,7 +29,7 @@ import java.nio.file.Paths; import static org.assertj.core.api.Assertions.assertThat; import static org.junit.jupiter.api.Assertions.assertThrows; -import static sonia.scm.store.CopyOnWrite.withTemporaryFile; +import static sonia.scm.CopyOnWrite.withTemporaryFile; class CopyOnWriteTest { diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/DataFileCacheTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/DataFileCacheTest.java similarity index 96% rename from scm-dao-xml/src/test/java/sonia/scm/store/DataFileCacheTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/DataFileCacheTest.java index 89530f1c5c..a4410ccff0 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/DataFileCacheTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/DataFileCacheTest.java @@ -14,20 +14,17 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.Nested; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; -import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; import sonia.scm.cache.MapCache; import java.io.File; import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.in; -import static org.mockito.Mockito.when; @ExtendWith(MockitoExtension.class) class DataFileCacheTest { diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/ExportableBlobFileStoreTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/ExportableBlobFileStoreTest.java similarity index 98% rename from scm-dao-xml/src/test/java/sonia/scm/store/ExportableBlobFileStoreTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/ExportableBlobFileStoreTest.java index 7c46b2393d..5a28393aac 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/ExportableBlobFileStoreTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/ExportableBlobFileStoreTest.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.Test; diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/ExportableFileStoreTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/ExportableFileStoreTest.java similarity index 97% rename from scm-dao-xml/src/test/java/sonia/scm/store/ExportableFileStoreTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/ExportableFileStoreTest.java index 7de50c5d3f..e68036ae3d 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/ExportableFileStoreTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/ExportableFileStoreTest.java @@ -14,13 +14,15 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.junit.jupiter.api.io.TempDir; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.store.ExportableStore; +import sonia.scm.store.Exporter; import java.io.ByteArrayOutputStream; import java.io.File; diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterFactoryTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/FileBasedStoreEntryImporterFactoryTest.java similarity index 94% rename from scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterFactoryTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/FileBasedStoreEntryImporterFactoryTest.java index e6cd83463b..0226229abb 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterFactoryTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/FileBasedStoreEntryImporterFactoryTest.java @@ -14,10 +14,12 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.io.TempDir; +import sonia.scm.store.StoreEntryMetaData; +import sonia.scm.store.StoreType; import java.nio.file.Path; diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/FileBasedStoreEntryImporterTest.java similarity index 94% rename from scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/FileBasedStoreEntryImporterTest.java index 025920aa5a..baac75bb98 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/FileBasedStoreEntryImporterTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/FileBasedStoreEntryImporterTest.java @@ -14,16 +14,14 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.io.TempDir; import java.io.ByteArrayInputStream; -import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; -import java.nio.file.Paths; import static org.assertj.core.api.Assertions.assertThat; diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileBlobStoreTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/FileBlobStoreTest.java similarity index 95% rename from scm-dao-xml/src/test/java/sonia/scm/store/FileBlobStoreTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/FileBlobStoreTest.java index d2747230e9..e6c953fdaf 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/FileBlobStoreTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/FileBlobStoreTest.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.common.io.ByteStreams; import org.junit.jupiter.api.BeforeEach; @@ -25,6 +25,11 @@ import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryReadOnlyChecker; import sonia.scm.repository.RepositoryTestData; import sonia.scm.security.UUIDKeyGenerator; +import sonia.scm.store.Blob; +import sonia.scm.store.BlobStore; +import sonia.scm.store.BlobStoreFactory; +import sonia.scm.store.EntryAlreadyExistsStoreException; +import sonia.scm.store.StoreReadOnlyException; import java.io.IOException; import java.io.InputStream; diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileNamespaceUpdateIteratorTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/FileNamespaceUpdateIteratorTest.java similarity index 95% rename from scm-dao-xml/src/test/java/sonia/scm/store/FileNamespaceUpdateIteratorTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/FileNamespaceUpdateIteratorTest.java index 7bbfc7ba7c..c7c19c867e 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/FileNamespaceUpdateIteratorTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/FileNamespaceUpdateIteratorTest.java @@ -14,13 +14,12 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.junit.jupiter.api.io.TempDir; -import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; import sonia.scm.TempDirRepositoryLocationResolver; @@ -32,7 +31,6 @@ import java.util.Collection; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.when; @ExtendWith(MockitoExtension.class) class FileNamespaceUpdateIteratorTest { diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreExporterTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/FileStoreExporterTest.java similarity index 97% rename from scm-dao-xml/src/test/java/sonia/scm/store/FileStoreExporterTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/FileStoreExporterTest.java index f1c874910f..be84db7fe7 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/FileStoreExporterTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/FileStoreExporterTest.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; @@ -27,6 +27,8 @@ import org.mockito.junit.jupiter.MockitoExtension; import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryLocationResolver; import sonia.scm.repository.RepositoryTestData; +import sonia.scm.store.ExportableStore; +import sonia.scm.store.StoreType; import java.io.IOException; import java.nio.file.Files; diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/JAXBConfigurationEntryStoreTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/JAXBConfigurationEntryStoreTest.java similarity index 92% rename from scm-dao-xml/src/test/java/sonia/scm/store/JAXBConfigurationEntryStoreTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/JAXBConfigurationEntryStoreTest.java index 06641b0e1a..1649e2e462 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/JAXBConfigurationEntryStoreTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/JAXBConfigurationEntryStoreTest.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import com.google.common.io.Closeables; @@ -22,6 +22,10 @@ import com.google.common.io.Resources; import org.junit.Test; import sonia.scm.security.AssignedPermission; import sonia.scm.security.UUIDKeyGenerator; +import sonia.scm.store.ConfigurationEntryStore; +import sonia.scm.store.ConfigurationEntryStoreFactory; +import sonia.scm.store.ConfigurationEntryStoreTestBase; +import sonia.scm.store.StoreObject; import java.io.File; import java.io.FileOutputStream; @@ -120,9 +124,9 @@ public class JAXBConfigurationEntryStoreTest @Override - protected ConfigurationEntryStoreFactory createConfigurationStoreFactory() + protected ConfigurationEntryStoreFactory createConfigurationStoreFactory() { - return new JAXBConfigurationEntryStoreFactory(contextProvider, repositoryLocationResolver, new UUIDKeyGenerator(), null, new StoreCacheConfigProvider(false)); + return new JAXBConfigurationEntryStoreFactory(contextProvider, repositoryLocationResolver, new UUIDKeyGenerator(), null, new StoreCacheFactory(new StoreCacheConfigProvider(false))); } diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/JAXBConfigurationStoreTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/JAXBConfigurationStoreTest.java similarity index 93% rename from scm-dao-xml/src/test/java/sonia/scm/store/JAXBConfigurationStoreTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/JAXBConfigurationStoreTest.java index b8a955e4c4..13362509c3 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/JAXBConfigurationStoreTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/JAXBConfigurationStoreTest.java @@ -14,11 +14,15 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.Test; import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryReadOnlyChecker; +import sonia.scm.store.ConfigurationStore; +import sonia.scm.store.StoreObject; +import sonia.scm.store.StoreReadOnlyException; +import sonia.scm.store.StoreTestBase; import static java.util.Collections.emptySet; import static org.assertj.core.api.Assertions.assertThat; @@ -38,7 +42,7 @@ public class JAXBConfigurationStoreTest extends StoreTestBase { @Override protected JAXBConfigurationStoreFactory createStoreFactory() { - return new JAXBConfigurationStoreFactory(contextProvider, repositoryLocationResolver, readOnlyChecker, emptySet(), new StoreCacheConfigProvider(false)); + return new JAXBConfigurationStoreFactory(contextProvider, repositoryLocationResolver, readOnlyChecker, emptySet(), new StoreCacheFactory(new StoreCacheConfigProvider(false))); } diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/JAXBDataStoreTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/JAXBDataStoreTest.java similarity index 85% rename from scm-dao-xml/src/test/java/sonia/scm/store/JAXBDataStoreTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/JAXBDataStoreTest.java index efefbd556d..168ec50fb8 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/JAXBDataStoreTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/JAXBDataStoreTest.java @@ -14,12 +14,17 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.Test; import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryReadOnlyChecker; import sonia.scm.security.UUIDKeyGenerator; +import sonia.scm.store.DataStore; +import sonia.scm.store.DataStoreFactory; +import sonia.scm.store.DataStoreTestBase; +import sonia.scm.store.StoreObject; +import sonia.scm.store.StoreReadOnlyException; import java.io.File; import java.io.IOException; @@ -36,15 +41,14 @@ public class JAXBDataStoreTest extends DataStoreTestBase { private final RepositoryReadOnlyChecker readOnlyChecker = mock(RepositoryReadOnlyChecker.class); @Override - protected DataStoreFactory createDataStoreFactory() - { + protected DataStoreFactory createDataStoreFactory() { return new JAXBDataStoreFactory( contextProvider, repositoryLocationResolver, new UUIDKeyGenerator(), readOnlyChecker, new DataFileCache(null, false), - new StoreCacheConfigProvider(false) + new StoreCacheFactory(new StoreCacheConfigProvider(false)) ); } @@ -66,8 +70,7 @@ public class JAXBDataStoreTest extends DataStoreTestBase { } @Test - public void shouldStoreAndLoadInRepository() - { + public void shouldStoreAndLoadInRepository() { repoStore.put("abc", new StoreObject("abc_value")); StoreObject storeObject = repoStore.get("abc"); @@ -76,8 +79,7 @@ public class JAXBDataStoreTest extends DataStoreTestBase { } @Test(expected = StoreReadOnlyException.class) - public void shouldNotStoreForReadOnlyRepository() - { + public void shouldNotStoreForReadOnlyRepository() { when(readOnlyChecker.isReadOnly(repository.getId())).thenReturn(true); getDataStore(StoreObject.class, repository).put("abc", new StoreObject("abc_value")); } diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/JAXBPropertyFileAccessTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/JAXBPropertyFileAccessTest.java similarity index 99% rename from scm-dao-xml/src/test/java/sonia/scm/store/JAXBPropertyFileAccessTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/JAXBPropertyFileAccessTest.java index da300e62ee..1537339930 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/JAXBPropertyFileAccessTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/JAXBPropertyFileAccessTest.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Nested; diff --git a/scm-dao-xml/src/test/java/sonia/scm/store/TypedStoreContextTest.java b/scm-persistence/src/test/java/sonia/scm/store/file/TypedStoreContextTest.java similarity index 98% rename from scm-dao-xml/src/test/java/sonia/scm/store/TypedStoreContextTest.java rename to scm-persistence/src/test/java/sonia/scm/store/file/TypedStoreContextTest.java index 90751e20fa..1acfe370ce 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/store/TypedStoreContextTest.java +++ b/scm-persistence/src/test/java/sonia/scm/store/file/TypedStoreContextTest.java @@ -14,7 +14,7 @@ * along with this program. If not, see https://www.gnu.org/licenses/. */ -package sonia.scm.store; +package sonia.scm.store.file; import jakarta.xml.bind.annotation.XmlAccessType; import jakarta.xml.bind.annotation.XmlAccessorType; @@ -25,6 +25,7 @@ import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.junit.jupiter.api.io.TempDir; import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.store.TypedStoreParameters; import java.io.File; import java.net.URL; diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/QueryableTypeDescriptorTestData.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/QueryableTypeDescriptorTestData.java new file mode 100644 index 0000000000..8f87526be8 --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/QueryableTypeDescriptorTestData.java @@ -0,0 +1,36 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import sonia.scm.plugin.QueryableTypeDescriptor; + +import static org.mockito.Mockito.lenient; +import static org.mockito.Mockito.mock; + +public class QueryableTypeDescriptorTestData { + static QueryableTypeDescriptor createDescriptor(String[] t) { + return createDescriptor("com.cloudogu.space.to.be.Spaceship", t); + } + + static QueryableTypeDescriptor createDescriptor(String clazz, String[] t) { + QueryableTypeDescriptor descriptor = mock(QueryableTypeDescriptor.class); + lenient().when(descriptor.getTypes()).thenReturn(t); + lenient().when(descriptor.getClazz()).thenReturn(clazz); + lenient().when(descriptor.getName()).thenReturn(""); + return descriptor; + } +} diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteIdentifiersTest.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteIdentifiersTest.java new file mode 100644 index 0000000000..40d43c8eee --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteIdentifiersTest.java @@ -0,0 +1,134 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import lombok.Getter; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.EnumSource; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.plugin.QueryableTypeDescriptor; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.AssertionsForClassTypes.assertThatThrownBy; +import static org.mockito.Mockito.lenient; + +@ExtendWith(MockitoExtension.class) +@SuppressWarnings("java:S115") // we do not heed enum naming conventions for better readability in the test +class SQLiteIdentifiersTest { + + @Nested + class Sanitize { + @Getter + private enum BadName { + OneToOne("examplename or 1=1"), + BatchedSQLStatement("105; DROP TABLE Classes"), + CommentOut("--"), + CommentOutWithContent("spaceship'--"), + BlindIfInjection("iif(count(*)>2,\"True\",\"False\")"), + VersionRequest("splite_version()"), + InnocentNameWithSpace("Traumschiff Enterprise"); + + BadName(String name) { + this.name = name; + } + + private final String name; + } + + @Getter + private enum GoodName { + Alphabetical("spaceship"), + AlphabeticalWithUnderscore("spaceship_STORE"), + Alphanumerical("rollerCoaster2000"), + AlphanumericalWithUnderscore("rollerCoaster2000_STORE"); + + GoodName(String name) { + this.name = name; + } + + private final String name; + } + + @ParameterizedTest + @EnumSource(BadName.class) + void shouldBlockSuspiciousNames(BadName name) { + assertThatThrownBy(() -> SQLiteIdentifiers.sanitize(name.getName())); + } + + @ParameterizedTest + @EnumSource(GoodName.class) + void shouldPassCorrectNames(GoodName name) { + String outputName = SQLiteIdentifiers.sanitize(name.getName()); + assertThat(outputName).isEqualTo(name.getName()); + } + } + + @Nested + class ComputeTableName { + @Mock + QueryableTypeDescriptor typeDescriptor; + + void setUp(String clazzName, String name) { + lenient().when(typeDescriptor.getClazz()).thenReturn(clazzName); + lenient().when(typeDescriptor.getName()).thenReturn(name); + } + + @Test + void shouldReturnCorrectTableNameIncludingPath() { + setUp("sonia.scm.store.sqlite.Spaceship", null); + + String output = SQLiteIdentifiers.computeTableName(typeDescriptor); + + assertThat(output).isEqualTo("sonia_scm_store_sqlite_Spaceship_STORE"); + } + + @Test + void shouldReturnTableNameEscapingUnderscores() { + setUp("sonia.scm.store.sqlite.Spaceship_One", null); + + String output = SQLiteIdentifiers.computeTableName(typeDescriptor); + + assertThat(output).isEqualTo("sonia_scm_store_sqlite_Spaceship__One_STORE"); + } + + @Test + void shouldReturnCorrectNameWithName() { + setUp("sonia.scm.store.sqlite.Spaceship", "TraumschiffEnterprise"); + + String output = SQLiteIdentifiers.computeTableName(typeDescriptor); + + assertThat(output).isEqualTo("TraumschiffEnterprise_STORE"); + } + } + + @Nested + class ComputeColumnIdentifier { + @Test + void shouldReturnIdOnlyWithNullValue() { + assertThat(SQLiteIdentifiers.computeColumnIdentifier(null)).isEqualTo("ID"); + } + + @Test + void shouldReturnCombinedNameWithGivenClassName() { + assertThat(SQLiteIdentifiers.computeColumnIdentifier("sonia.scm.store.sqlite.Spaceship.class")).isEqualTo("Spaceship_ID"); + } + } +} diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteParallelizationTest.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteParallelizationTest.java new file mode 100644 index 0000000000..c93c279ffe --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteParallelizationTest.java @@ -0,0 +1,139 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import lombok.extern.slf4j.Slf4j; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableMaintenanceStore; +import sonia.scm.user.User; + +import java.nio.file.Path; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.Future; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.fail; + +@Slf4j +class SQLiteParallelizationTest { + + private String connectionString; + + @BeforeEach + void init(@TempDir Path path) { + connectionString = "jdbc:sqlite:" + path.toString() + "/test.db"; + } + + @Test + void shouldTestParallelPutOperations() throws InterruptedException, ExecutionException, SQLException { + int numThreads = 100; + ExecutorService executor = Executors.newFixedThreadPool(numThreads); + List> futures = new ArrayList<>(); + + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + + for (int i = 0; i < numThreads; i++) { + final String userId = "user-" + i; + final String userName = "User" + i; + + futures.add(executor.submit(() -> { + try { + store.transactional(() -> { + store.put(userId, new User(userName)); + return true; + }); + } catch (Exception e) { + fail("Error storing user", e); + } + })); + } + + for (Future future : futures) { + future.get(); + } + executor.shutdown(); + + int count = actualCount(); + assertEquals(numThreads, count, "All threads should have been successfully saved"); + } + + @Test + void shouldWriteMultipleRowsConcurrently() throws InterruptedException, ExecutionException, SQLException { + int numThreads = 100; + int rowsPerThread = 50; + ExecutorService executor = Executors.newFixedThreadPool(numThreads); + List> futures = new ArrayList<>(); + + StoreTestBuilder testStoreBuilder = new StoreTestBuilder(connectionString, Repository.class.getName()); + QueryableMaintenanceStore store = testStoreBuilder.forMaintenanceWithSubIds("42"); + + for (int i = 0; i < numThreads; i++) { + final int threadIndex = i; + futures.add(executor.submit(() -> { + List rows = new ArrayList<>(); + try { + for (int j = 1; j <= rowsPerThread; j++) { + QueryableMaintenanceStore.Row row = new QueryableMaintenanceStore.Row<>( + new String[]{String.valueOf(threadIndex)}, + "user-" + threadIndex + "-" + j, + new User("User" + threadIndex + "-" + j, "User " + threadIndex + "-" + j, + "user" + threadIndex + "-" + j + "@example.com") + ); + rows.add(row); + } + + store.writeAll(rows); + } catch (Exception e) { + fail("Error writing rows", e); + } + })); + } + + for (Future future : futures) { + future.get(); + } + executor.shutdown(); + + int expectedCount = numThreads * rowsPerThread; + int count = actualCount(); + assertEquals(expectedCount, count, "Exactly " + expectedCount + " entries should have been saved"); + } + + private int actualCount() throws SQLException { + int count; + try (Connection conn = DriverManager.getConnection(connectionString); + PreparedStatement stmt = conn.prepareStatement("SELECT COUNT(*) FROM sonia_scm_user_User_STORE"); + ResultSet rs = stmt.executeQuery()) { + rs.next(); + count = rs.getInt(1); + } + return count; + } +} + diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteQueryableMutableStoreTest.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteQueryableMutableStoreTest.java new file mode 100644 index 0000000000..bb9d469725 --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteQueryableMutableStoreTest.java @@ -0,0 +1,270 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; +import sonia.scm.user.User; + +import java.nio.file.Path; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.util.Map; + +import static org.assertj.core.api.Assertions.assertThat; + +class SQLiteQueryableMutableStoreTest { + + private Connection connection; + private String connectionString; + + @BeforeEach + void init(@TempDir Path path) throws SQLException { + connectionString = "jdbc:sqlite:" + path.toString() + "/test.db"; + connection = DriverManager.getConnection(connectionString); + } + + @Nested + class Put { + + @Test + void shouldPutObjectWithoutParent() throws SQLException { + new StoreTestBuilder(connectionString).withIds().put("tricia", new User("trillian")); + + ResultSet resultSet = connection + .createStatement() + .executeQuery("SELECT json_extract(u.payload, '$.name') as name FROM sonia_scm_user_User_STORE u WHERE ID = 'tricia'"); + + assertThat(resultSet.next()).isTrue(); + assertThat(resultSet.getString("name")).isEqualTo("trillian"); + } + + @Test + void shouldOverwriteExistingObject() throws SQLException { + new StoreTestBuilder(connectionString).withIds().put("tricia", new User("Trillian")); + new StoreTestBuilder(connectionString).withIds().put("tricia", new User("McMillan")); + + ResultSet resultSet = connection + .createStatement() + .executeQuery("SELECT json_extract(u.payload, '$.name') as name FROM sonia_scm_user_User_STORE u WHERE ID = 'tricia'"); + + assertThat(resultSet.next()).isTrue(); + assertThat(resultSet.getString("name")).isEqualTo("McMillan"); + } + + @Test + void shouldPutObjectWithSingleParent() throws SQLException { + new StoreTestBuilder(connectionString, "sonia.Group").withIds("42") + .put("tricia", new User("trillian")); + + ResultSet resultSet = connection + .createStatement() + .executeQuery("SELECT json_extract(u.payload, '$.name') as name FROM sonia_scm_user_User_STORE u WHERE ID = 'tricia' and GROUP_ID = '42'"); + + assertThat(resultSet.next()).isTrue(); + assertThat(resultSet.getString("name")).isEqualTo("trillian"); + } + + @Test + void shouldPutObjectWithMultipleParents() throws SQLException { + new StoreTestBuilder(connectionString, "sonia.Company", "sonia.Group").withIds("cloudogu", "42") + .put("tricia", new User("trillian")); + + ResultSet resultSet = connection + .createStatement() + .executeQuery(""" + SELECT json_extract(u.payload, '$.name') as name + FROM sonia_scm_user_User_STORE u + WHERE ID = 'tricia' + AND GROUP_ID = '42' + AND COMPANY_ID = 'cloudogu' + """); + + assertThat(resultSet.next()).isTrue(); + assertThat(resultSet.getString("name")).isEqualTo("trillian"); + } + + @Test + void shouldRollback() throws SQLException { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + + store.transactional(() -> { + store.put("tricia", new User("trillian")); + return false; + }); + + ResultSet resultSet = connection + .createStatement() + .executeQuery("SELECT * FROM sonia_scm_user_User_STORE"); + assertThat(resultSet.next()).isFalse(); + } + + @Test + void shouldDisableAutoCommit() throws SQLException { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + + store.transactional(() -> { + store.put("tricia", new User("trillian")); + + try { + ResultSet resultSet = connection + .createStatement() + .executeQuery("SELECT * FROM sonia_scm_user_User_STORE"); + assertThat(resultSet.next()).isFalse(); + } catch (SQLException e) { + throw new RuntimeException(e); + } + + return true; + }); + + ResultSet resultSet = connection + .createStatement() + .executeQuery("SELECT * FROM sonia_scm_user_User_STORE"); + assertThat(resultSet.next()).isTrue(); + } + } + + @Nested + class Get { + + @Test + void shouldGetObjectWithoutParent() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("tricia", new User("trillian")); + + User tricia = store.get("tricia"); + + assertThat(tricia) + .isNotNull() + .extracting("name") + .isEqualTo("trillian"); + } + + @Test + void shouldReturnForNotExistingValue() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + + User earth = store.get("earth"); + + assertThat(earth) + .isNull(); + } + + @Test + void shouldGetObjectWithSingleParent() { + new StoreTestBuilder(connectionString, new String[]{"sonia.Group"}).withIds("1337").put("tricia", new User("McMillan")); + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Group").withIds("42"); + store.put("tricia", new User("trillian")); + + User tricia = store.get("tricia"); + + assertThat(tricia) + .isNotNull() + .extracting("name") + .isEqualTo("trillian"); + } + + @Test + void shouldGetObjectWithMultipleParents() { + new StoreTestBuilder(connectionString, new String[]{"sonia.Company", "sonia.Group"}).withIds("cloudogu", "1337").put("tricia", new User("McMillan")); + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Company", "sonia.Group").withIds("cloudogu", "42"); + store.put("tricia", new User("trillian")); + + User tricia = store.get("tricia"); + + assertThat(tricia) + .isNotNull() + .extracting("name") + .isEqualTo("trillian"); + } + + @Test + void shouldGetAllForSingleEntry() { + new StoreTestBuilder(connectionString, new String[]{"sonia.Company", "sonia.Group"}).withIds("cloudogu", "1337").put("tricia", new User("McMillan")); + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Company", "sonia.Group").withIds("cloudogu", "42"); + store.put("tricia", new User("trillian")); + + Map users = store.getAll(); + + assertThat(users).hasSize(1); + assertThat(users.get("tricia")) + .isNotNull() + .extracting("name") + .isEqualTo("trillian"); + } + + @Test + void shouldGetAllForMultipleEntries() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Company", "sonia.Group").withIds("cloudogu", "42"); + store.put("dent", new User("arthur")); + store.put("tricia", new User("trillian")); + + Map users = store.getAll(); + + assertThat(users).hasSize(2); + assertThat(users.get("tricia")) + .isNotNull() + .extracting("name") + .isEqualTo("trillian"); + assertThat(users.get("dent")) + .isNotNull() + .extracting("name") + .isEqualTo("arthur"); + } + } + + @Nested + class Clear { + @Test + void shouldClear() { + SQLiteQueryableMutableStore uneffectedStore = new StoreTestBuilder(connectionString, "sonia.Company", "sonia.Group").withIds("cloudogu", "1337"); + uneffectedStore.put("tricia", new User("McMillan")); + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Company", "sonia.Group").withIds("cloudogu", "42"); + store.put("tricia", new User("trillian")); + + store.clear(); + + assertThat(store.getAll()).isEmpty(); + assertThat(uneffectedStore.getAll()).hasSize(1); + } + } + + @Nested + class Remove { + @Test + void shouldRemove() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Company", "sonia.Group").withIds("cloudogu", "42"); + store.put("dent", new User("arthur")); + store.put("tricia", new User("trillian")); + + store.remove("dent"); + + assertThat(store.getAll()).containsOnlyKeys("tricia"); + } + } + + @AfterEach + void closeDB() throws SQLException { + connection.close(); + } +} diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteQueryableStoreTest.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteQueryableStoreTest.java new file mode 100644 index 0000000000..0dbf0fa299 --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteQueryableStoreTest.java @@ -0,0 +1,908 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; +import sonia.scm.group.Group; +import sonia.scm.repository.Repository; +import sonia.scm.store.Conditions; +import sonia.scm.store.LeafCondition; +import sonia.scm.store.Operator; +import sonia.scm.store.QueryableMaintenanceStore; +import sonia.scm.store.QueryableMaintenanceStore.MaintenanceIterator; +import sonia.scm.store.QueryableMaintenanceStore.MaintenanceStoreEntry; +import sonia.scm.store.QueryableStore; +import sonia.scm.user.User; + +import java.nio.file.Path; +import java.time.Instant; +import java.util.Collection; +import java.util.List; +import java.util.Map; +import java.util.Optional; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +@SuppressWarnings({"resource", "unchecked"}) +class SQLiteQueryableStoreTest { + + private String connectionString; + + @BeforeEach + void init(@TempDir Path path) { + connectionString = "jdbc:sqlite:" + path.toString() + "/test.db"; + } + + @Nested + class FindAll { + + @Nested + class QueryClassTypes { + + @Test + void shouldWorkWithEnums() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + store.put(new Spaceship("Space Shuttle", Range.SOLAR_SYSTEM)); + store.put(new Spaceship("Heart Of Gold", Range.INTER_GALACTIC)); + + List all = store + .query(SPACESHIP_RANGE_ENUM_QUERY_FIELD.eq(Range.SOLAR_SYSTEM)) + .findAll(); + + assertThat(all).hasSize(1); + } + + + @Test + void shouldWorkWithLongs() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + User trillian = new User("trillian", "McMillan", "tricia@hog.org"); + trillian.setCreationDate(10000000000L); + store.put(trillian); + User arthur = new User("arthur", "Dent", "arthur@hog.org"); + arthur.setCreationDate(9999999999L); + store.put(arthur); + + List all = store.query( + CREATION_DATE_QUERY_FIELD.lessOrEquals(9999999999L) + ) + .findAll(); + + assertThat(all) + .extracting("name") + .containsExactly("arthur"); + } + + @Test + void shouldWorkWithIntegers() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + User trillian = new User("trillian", "McMillan", "tricia@hog.org"); + trillian.setCreationDate(42L); + store.put(trillian); + User arthur = new User("arthur", "Dent", "arthur@hog.org"); + arthur.setCreationDate(23L); + store.put(arthur); + + List all = store.query( + CREATION_DATE_AS_INTEGER_QUERY_FIELD.less(40) + ) + .findAll(); + + assertThat(all) + .extracting("name") + .containsExactly("arthur"); + } + + @Test + void shouldWorkWithNumberCollection() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + User trillian = new User("trillian", "McMillan", "tricia@hog.org"); + trillian.setActive(true); + store.put(trillian); + User arthur = new User("arthur", "Dent", "arthur@hog.org"); + arthur.setActive(false); + store.put(arthur); + + List all = store.query( + ACTIVE_QUERY_FIELD.isTrue() + ) + .findAll(); + + assertThat(all) + .extracting("name") + .containsExactly("trillian"); + } + + @Test + void shouldCountAndWorkWithNumberCollection() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + User trillian = new User("trillian", "McMillan", "tricia@hog.org"); + trillian.setActive(true); + store.put(trillian); + User arthur = new User("arthur", "Dent", "arthur@hog.org"); + arthur.setActive(false); + store.put(arthur); + + long count = store.query( + ACTIVE_QUERY_FIELD.isTrue() + ) + .count(); + + assertThat(count).isEqualTo(1); + + } + } + + @Nested + class QueryFeatures { + + @Test + void shouldHandleCollections() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + store.put(new Spaceship("Spaceshuttle", "Buzz", "Anndre")); + store.put(new Spaceship("Heart Of Gold", "Trillian", "Arthur", "Ford", "Zaphod", "Marvin")); + + List result = store.query( + SPACESHIP_CREW_QUERY_FIELD.contains("Marvin") + ).findAll(); + + assertThat(result).hasSize(1); + } + + @Test + void shouldCountAndHandleCollections() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + store.put(new Spaceship("Spaceshuttle", "Buzz", "Anndre")); + store.put(new Spaceship("Heart Of Gold", "Trillian", "Arthur", "Ford", "Zaphod", "Marvin")); + + long result = store.query( + SPACESHIP_CREW_QUERY_FIELD.contains("Marvin") + ).count(); + + assertThat(result).isEqualTo(1); + } + + @Test + void shouldCountWithoutConditions() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + store.put(new Spaceship("Spaceshuttle", "Buzz", "Anndre")); + store.put(new Spaceship("Heart Of Gold", "Trillian", "Arthur", "Ford", "Zaphod", "Marvin")); + + long result = store.query().count(); + + assertThat(result).isEqualTo(2); + } + + @Test + void shouldHandleCollectionSize() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + store.put(new Spaceship("Spaceshuttle", "Buzz", "Anndre")); + store.put(new Spaceship("Heart of Gold", "Trillian", "Arthur", "Ford", "Zaphod", "Marvin")); + store.put(new Spaceship("MillenniumFalcon")); + + List onlyEmpty = store.query( + SPACESHIP_CREW_SIZE_QUERY_FIELD.isEmpty() + ).findAll(); + assertThat(onlyEmpty).hasSize(1); + assertThat(onlyEmpty.get(0).getName()).isEqualTo("MillenniumFalcon"); + + + List exactlyTwoCrewMates = store.query( + SPACESHIP_CREW_SIZE_QUERY_FIELD.eq(2L) + ).findAll(); + assertThat(exactlyTwoCrewMates).hasSize(1); + assertThat(exactlyTwoCrewMates.get(0).getName()).isEqualTo("Spaceshuttle"); + + List moreThanTwoCrewMates = store.query( + SPACESHIP_CREW_SIZE_QUERY_FIELD.greater(2L) + ).findAll(); + assertThat(moreThanTwoCrewMates).hasSize(1); + assertThat(moreThanTwoCrewMates.get(0).getName()).isEqualTo("Heart of Gold"); + } + + @Test + void shouldHandleMap() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + store.put(new Spaceship("Spaceshuttle", Map.of("moon", true, "earth", true))); + store.put(new Spaceship("Heart of Gold", Map.of("vogon", true, "earth", true))); + store.put(new Spaceship("MillenniumFalcon", Map.of("dagobah", false))); + + List keyResult = store.query( + SPACESHIP_DESTINATIONS_QUERY_FIELD.containsKey("vogon") + ).findAll(); + assertThat(keyResult).hasSize(1); + assertThat(keyResult.get(0).getName()).isEqualTo("Heart of Gold"); + + List valueResult = store.query( + SPACESHIP_DESTINATIONS_QUERY_FIELD.containsValue(false) + ).findAll(); + assertThat(valueResult).hasSize(1); + assertThat(valueResult.get(0).getName()).isEqualTo("MillenniumFalcon"); + } + + @Test + void shouldCountAndHandleMap() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + store.put(new Spaceship("Spaceshuttle", Map.of("moon", true, "earth", true))); + store.put(new Spaceship("Heart of Gold", Map.of("vogon", true, "earth", true))); + store.put(new Spaceship("MillenniumFalcon", Map.of("dagobah", false))); + + long keyResult = store.query( + SPACESHIP_DESTINATIONS_QUERY_FIELD.containsKey("vogon") + ).count(); + + assertThat(keyResult).isEqualTo(1); + + + long valueResult = store.query( + SPACESHIP_DESTINATIONS_QUERY_FIELD.containsValue(false) + ).count(); + assertThat(valueResult).isEqualTo(1); + } + + + @Test + void shouldHandleMapSize() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + store.put(new Spaceship("Spaceshuttle", Map.of("moon", true, "earth", true))); + store.put(new Spaceship("Heart of Gold", Map.of("vogon", true, "earth", true, "dagobah", true))); + store.put(new Spaceship("MillenniumFalcon", Map.of())); + + List onlyEmpty = store.query( + SPACESHIP_DESTINATIONS_SIZE_QUERY_FIELD.isEmpty() + ).findAll(); + assertThat(onlyEmpty).hasSize(1); + assertThat(onlyEmpty.get(0).getName()).isEqualTo("MillenniumFalcon"); + + + List exactlyTwoDestinations = store.query( + SPACESHIP_DESTINATIONS_SIZE_QUERY_FIELD.eq(2L) + ).findAll(); + assertThat(exactlyTwoDestinations).hasSize(1); + assertThat(exactlyTwoDestinations.get(0).getName()).isEqualTo("Spaceshuttle"); + + List moreThanTwoDestinations = store.query( + SPACESHIP_DESTINATIONS_SIZE_QUERY_FIELD.greater(2L) + ).findAll(); + assertThat(moreThanTwoDestinations).hasSize(1); + assertThat(moreThanTwoDestinations.get(0).getName()).isEqualTo("Heart of Gold"); + } + + @Test + void shouldRetrieveTime() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + Spaceship spaceshuttle = new Spaceship("Spaceshuttle", Range.SOLAR_SYSTEM); + spaceshuttle.setInServiceSince(Instant.parse("1981-04-12T10:00:00Z")); + store.put(spaceshuttle); + + Spaceship falcon = new Spaceship("Falcon9", Range.SOLAR_SYSTEM); + falcon.setInServiceSince(Instant.parse("2015-12-21T10:00:00Z")); + store.put(falcon); + + List resultEqOperator = store.query( + SPACESHIP_INSERVICE_QUERY_FIELD.eq(Instant.parse("2015-12-21T10:00:00Z"))).findAll(); + assertThat(resultEqOperator).hasSize(1); + assertThat(resultEqOperator.get(0).getName()).isEqualTo("Falcon9"); + + List resultBeforeOperator = store.query( + SPACESHIP_INSERVICE_QUERY_FIELD.before(Instant.parse("2000-12-21T10:00:00Z"))).findAll(); + assertThat(resultBeforeOperator).hasSize(1); + assertThat(resultBeforeOperator.get(0).getName()).isEqualTo("Spaceshuttle"); + + List resultAfterOperator = store.query( + SPACESHIP_INSERVICE_QUERY_FIELD.after(Instant.parse("2000-01-01T00:00:00Z"))).findAll(); + assertThat(resultAfterOperator).hasSize(1); + assertThat(resultAfterOperator.get(0).getName()).isEqualTo("Falcon9"); + + List resultBetweenOperator = store.query( + SPACESHIP_INSERVICE_QUERY_FIELD.between(Instant.parse("1980-04-12T10:00:00Z"), Instant.parse("2016-12-21T10:00:00Z"))).findAll(); + assertThat(resultBetweenOperator).hasSize(2); + + } + + @Test + void shouldLimitQuery() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put(new User("trillian", "McMillan", "tricia@hog.org")); + store.put(new User("arthur", "Dent", "arthur@hog.org")); + store.put(new User("zaphod", "Beeblebrox", "zaphod@hog.org")); + store.put(new User("marvin", "Marvin", "marvin@hog.org")); + + List all = store.query() + .findAll(1, 2); + + assertThat(all) + .extracting("name") + .containsExactly("arthur", "zaphod"); + } + + @Test + void shouldOrderResults() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put(new User("trillian", "McMillan", "tricia@hog.org")); + store.put(new User("arthur", "Dent", "arthur@hog.org")); + store.put(new User("zaphod", "Beeblebrox Head 1", "zaphod1@hog.org")); + store.put(new User("zaphod", "Beeblebrox Head 2", "zaphod2@hog.org")); + store.put(new User("marvin", "Marvin", "marvin@hog.org")); + + List all = store.query() + .orderBy(USER_NAME_QUERY_FIELD, QueryableStore.Order.ASC) + .orderBy(DISPLAY_NAME_QUERY_FIELD, QueryableStore.Order.DESC) + .findAll(); + + assertThat(all) + .extracting("mail") + .containsExactly("arthur@hog.org", "marvin@hog.org", "tricia@hog.org", "zaphod2@hog.org", "zaphod1@hog.org"); + } + } + + @Nested + class QueryLogicalHandling { + @Test + void shouldQueryForId() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("1", new User("trillian", "Tricia", "tricia@hog.org")); + store.put("2", new User("trillian", "Trillian McMillan", "mcmillan@gmail.com")); + store.put("3", new User("arthur", "Arthur Dent", "arthur@hog.org")); + + List all = store.query( + ID_QUERY_FIELD.eq("1") + ) + .findAll(); + + assertThat(all) + .extracting("displayName") + .containsExactly("Tricia"); + } + + @Test + void shouldQueryForParents() { + new StoreTestBuilder(connectionString, Group.class.getName()) + .withIds("42") + .put("tricia", new User("trillian", "Tricia", "tricia@hog.org")); + new StoreTestBuilder(connectionString, Group.class.getName()) + .withIds("1337") + .put("tricia", new User("trillian", "Trillian McMillan", "tricia@hog.org")); + + SQLiteQueryableStore store = new StoreTestBuilder(connectionString, Group.class.getName()).withIds(); + + List all = store.query( + GROUP_QUERY_FIELD.eq("42") + ) + .findAll(); + + assertThat(all) + .extracting("displayName") + .containsExactly("Tricia"); + } + + @Test + void shouldHandleContainsCondition() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("tricia", new User("trillian", "Tricia", "tricia@hog.org")); + store.put("McMillan", new User("trillian", "Trillian McMillan", "mcmillan@gmail.com")); + store.put("dent", new User("arthur", "Arthur Dent", "arthur@hog.org")); + + List all = store.query( + USER_NAME_QUERY_FIELD.contains("ri") + ) + .findAll(); + + assertThat(all).hasSize(2); + } + + @Test + void shouldHandleIsNullCondition() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("tricia", new User("trillian", null, "tricia@hog.org")); + store.put("dent", new User("arthur", "Arthur Dent", "arthur@hog.org")); + + List all = store.query( + DISPLAY_NAME_QUERY_FIELD.isNull() + ) + .findAll(); + + assertThat(all) + .extracting("name") + .containsExactly("trillian"); + } + + @Test + void shouldHandleNotNullCondition() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("tricia", new User("trillian", null, "tricia@hog.org")); + store.put("dent", new User("arthur", "Arthur Dent", "arthur@hog.org")); + + List all = store.query( + Conditions.not(DISPLAY_NAME_QUERY_FIELD.isNull()) + ) + .findAll(); + + assertThat(all) + .extracting("name") + .containsExactly("arthur"); + } + + @Test + void shouldHandleOr() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("tricia", new User("trillian", "Tricia", "tricia@hog.org")); + store.put("McMillan", new User("trillian", "Trillian McMillan", "mcmillan@gmail.com")); + store.put("dent", new User("arthur", "Arthur Dent", "arthur@hog.org")); + + List all = store.query( + Conditions.or( + DISPLAY_NAME_QUERY_FIELD.eq("Tricia"), + DISPLAY_NAME_QUERY_FIELD.eq("Trillian McMillan") + ) + ) + .findAll(); + + assertThat(all) + .extracting("name") + .containsExactly("trillian", "trillian"); + } + + + @Test + void shouldHandleOrWithMultipleStores() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Group").withIds("CoolGroup"); + User tricia = new User("trillian", "Tricia", "tricia@hog.org"); + User mcmillan = new User("trillian", "Trillian McMillan", "mcmillan@gmail.com"); + User dent = new User("arthur", "Arthur Dent", "arthur@hog.org"); + store.put("tricia", tricia); + store.put("McMillan", mcmillan); + store.put("dent", dent); + + SQLiteQueryableMutableStore parallelStore = new StoreTestBuilder(connectionString, "sonia.Group").withIds("LameGroup"); + parallelStore.put("tricia", new User("trillian", "Trillian IAMINAPARALLELSTORE McMillan", "mcmillan@gmail.com")); + + List result = store.query( + Conditions.or( + new LeafCondition<>(new QueryableStore.StringQueryField<>("mail"), Operator.EQ, "arthur@hog.org"), + new LeafCondition<>(new QueryableStore.StringQueryField<>("mail"), Operator.EQ, "mcmillan@gmail.com")) + ).findAll(); + + assertThat(result).containsExactlyInAnyOrder(dent, mcmillan); + } + + @Test + void shouldHandleGroup() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Group") + .withIds("42"); + store.put("tricia", new User("trillian", "Tricia", "tricia@hog.org")); + new StoreTestBuilder(connectionString, "sonia.Group") + .withIds("1337") + .put("tricia", new User("trillian", "Trillian McMillan", "tricia@hog.org")); + + List all = store.query().findAll(); + + assertThat(all) + .extracting("displayName") + .containsExactly("Tricia"); + } + + @Test + void shouldHandleGroupWithCondition() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Group") + .withIds("42"); + store + .put("tricia", new User("trillian", "Tricia", "tricia@hog.org")); + new StoreTestBuilder(connectionString, "sonia.Group") + .withIds("1337") + .put("tricia", new User("trillian", "Trillian McMillan", "tricia@hog.org")); + + List all = store.query(USER_NAME_QUERY_FIELD.eq("trillian")).findAll(); + + assertThat(all) + .extracting("displayName") + .containsExactly("Tricia"); + } + + @Test + void shouldHandleInArrayCondition() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put(new User("trillian", "McMillan", "tricia@hog.org")); + store.put(new User("arthur", "Dent", "arthur@hog.org")); + store.put(new User("zaphod", "Beeblebrox", "zaphod@hog.org")); + + List all = store.query( + USER_NAME_QUERY_FIELD.in("trillian", "arthur") + ) + .findAll(); + + assertThat(all) + .extracting("name") + .containsExactly("trillian", "arthur"); + } + } + + @Test + void shouldFindAllObjectsWithoutParentWithoutConditions() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("tricia", new User("trillian")); + + List all = store.query().findAll(); + + assertThat(all).hasSize(1); + } + + @Test + void shouldFindAllObjectsWithoutParentWithCondition() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("tricia", new User("trillian")); + store.put("dent", new User("arthur")); + + List all = store.query(USER_NAME_QUERY_FIELD.eq("trillian")).findAll(); + assertThat(all).hasSize(1); + } + + @Test + void shouldFindAllObjectsWithOneParentAndMultipleConditions() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, "sonia.Group").withIds("CoolGroup"); + User tricia = new User("trillian", "Tricia", "tricia@hog.org"); + User mcmillan = new User("trillian", "Trillian McMillan", "mcmillan@gmail.com"); + User dent = new User("arthur", "Arthur Dent", "arthur@hog.org"); + store.put("tricia", tricia); + store.put("McMillan", mcmillan); + store.put("dent", dent); + + List result = store.query( + Conditions.or( + new LeafCondition<>(new QueryableStore.StringQueryField<>("mail"), Operator.EQ, "arthur@hog.org"), + new LeafCondition<>(new QueryableStore.StringQueryField<>("mail"), Operator.EQ, "mcmillan@gmail.com")) + ).findAll(); + + assertThat(result).containsExactlyInAnyOrder(dent, mcmillan); + } + + @Test + void shouldFindAllObjectsWithoutParentWithMultipleConditions() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put("tricia", new User("trillian", "Tricia", "tricia@hog.org")); + store.put("McMillan", new User("trillian", "Trillian McMillan", "mcmillan@gmail.com")); + store.put("dent", new User("arthur", "Arthur Dent", "arthur@hog.org")); + + List all = store.query( + USER_NAME_QUERY_FIELD.eq("trillian"), + DISPLAY_NAME_QUERY_FIELD.eq("Tricia") + ) + .findAll(); + + assertThat(all).hasSize(1); + } + + @Test + void shouldReturnIds() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString, Spaceship.class.getName()) + .withIds("hog"); + store.put("tricia", new User("trillian", "Tricia", "tricia@hog.org")); + + List> results = store + .query() + .withIds() + .findAll(); + + assertThat(results).hasSize(1); + QueryableStore.Result result = results.get(0); + assertThat(result.getParentId(Spaceship.class)).contains("hog"); + assertThat(result.getId()).isEqualTo("tricia"); + assertThat(result.getEntity().getName()).isEqualTo("trillian"); + } + } + + @Nested + class FindOne { + @Test + void shouldReturnEmptyOptionalIfNoResultFound() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + assertThat(store.query(SPACESHIP_NAME_QUERY_FIELD.eq("Heart Of Gold")).findOne()).isEmpty(); + } + + @Test + void shouldReturnOneResultIfOneIsGiven() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + Spaceship expectedShip = new Spaceship("Heart Of Gold", Range.INNER_GALACTIC); + store.put(expectedShip); + Spaceship ship = store.query(SPACESHIP_NAME_QUERY_FIELD.eq("Heart Of Gold")).findOne().get(); + + assertThat(ship).isEqualTo(expectedShip); + } + + @Test + void shouldThrowErrorIfMoreThanOneResultIsSaved() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).forClassWithIds(Spaceship.class); + Spaceship expectedShip = new Spaceship("Heart Of Gold", Range.INNER_GALACTIC); + Spaceship localShip = new Spaceship("Heart Of Gold", Range.SOLAR_SYSTEM); + store.put(expectedShip); + store.put(localShip); + assertThatThrownBy(() -> store.query(SPACESHIP_NAME_QUERY_FIELD.eq("Heart Of Gold")).findOne().get()) + .isInstanceOf(QueryableStore.TooManyResultsException.class); + } + } + + @Nested + class FindFirst { + @Test + void shouldFindFirst() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + User expectedUser = new User("trillian", "Tricia", "tricia@hog.org"); + + store.put("1", expectedUser); + store.put("2", new User("trillian", "Trillian McMillan", "mcmillan@gmail.com")); + store.put("3", new User("arthur", "Arthur Dent", "arthur@hog.org")); + + Optional user = store.query( + USER_NAME_QUERY_FIELD.eq("trillian") + ) + .findFirst(); + + assertThat(user).isEqualTo(Optional.of(expectedUser)); + } + + @Test + void shouldFindFirstWithMatchingCondition() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + User expectedUser = new User("trillian", "Trillian McMillan", "mcmillan-alternate@gmail.com"); + + store.put("1", new User("trillian", "Tricia", "tricia@hog.org")); + store.put("2", new User("trillian", "Trillian McMillan", "mcmillan@gmail.com")); + store.put("3", expectedUser); + store.put("4", new User("arthur", "Arthur Dent", "arthur@hog.org")); + + Optional user = store.query( + USER_NAME_QUERY_FIELD.eq("trillian"), + MAIL_QUERY_FIELD.eq("mcmillan-alternate@gmail.com") + ) + .findFirst(); + + assertThat(user).isEqualTo(Optional.of(expectedUser)); + } + + + @Test + void shouldFindFirstWithMatchingLogicalCondition() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + User expectedUser = new User("trillian", "Trillian McMillan", "mcmillan@gmail.com"); + + store.put("1", new User("trillian-old", "Tricia", "tricia@hog.org")); + store.put("2", new User("trillian", "Trillian McMillan", "mcmillan@gmail.com")); + store.put("3", expectedUser); + store.put("4", new User("arthur", "Arthur Dent", "arthur@hog.org")); + store.put("5", new User("arthur", "Trillian McMillan", "mcmillan@gmail.com")); + + Optional user = store.query( + Conditions.and( + Conditions.and( + DISPLAY_NAME_QUERY_FIELD.eq("Trillian McMillan"), + MAIL_QUERY_FIELD.eq("mcmillan@gmail.com") + ), + Conditions.not( + ID_QUERY_FIELD.eq("1") + ) + ) + ).findFirst(); + + assertThat(user).isEqualTo(Optional.of(expectedUser)); + } + + @Test + void shouldReturnEmptyOptionalIfNoResultFound() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + Optional user = store.query( + USER_NAME_QUERY_FIELD.eq("dave") + ) + .findFirst(); + assertThat(user).isEmpty(); + } + } + + @Nested + class ForMaintenance { + @Test + void shouldUpdateRawJson() throws Exception { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + User user = new User("trillian", "Trillian McMillan", "mcmillan@gmail.com"); + store.put("1", user); + + try (MaintenanceIterator iterator = store.iterateAll()) { + assertThat(iterator.hasNext()).isTrue(); + MaintenanceStoreEntry entry = iterator.next(); + assertThat(entry.getId()).isEqualTo("1"); + + User userFromIterator = entry.get(); + userFromIterator.setName("dent"); + entry.update(userFromIterator); + + assertThat(iterator.hasNext()).isFalse(); + } + User changedUser = store.get("1"); + assertThat(changedUser.getName()).isEqualTo("dent"); + } + + @Test + void shouldUpdateRawJsonForItemWithParent() throws Exception { + SQLiteQueryableMutableStore subStore = new StoreTestBuilder(connectionString, Group.class.getName()).withIds("hitchhiker"); + User user = new User("trillian", "Trillian McMillan", "mcmillan@gmail.com"); + subStore.put("1", user); + + QueryableMaintenanceStore maintenanceStore = new StoreTestBuilder(connectionString, Group.class.getName()).forMaintenanceWithSubIds(); + try (MaintenanceIterator iterator = maintenanceStore.iterateAll()) { + assertThat(iterator.hasNext()).isTrue(); + MaintenanceStoreEntry entry = iterator.next(); + assertThat(entry.getId()).isEqualTo("1"); + + User userFromIterator = entry.get(); + userFromIterator.setName("dent"); + entry.update(userFromIterator); + + assertThat(iterator.hasNext()).isFalse(); + } + User changedUser = subStore.get("1"); + assertThat(changedUser.getName()).isEqualTo("dent"); + } + + @Test + void shouldRemoveFromIteratorWithoutParent() { + SQLiteQueryableMutableStore store = new StoreTestBuilder(connectionString).withIds(); + store.put(new User("trillian", "Trillian McMillan", "mcmillan@gmail.com")); + store.put(new User("dent", "Arthur Dent", "dent@gmail.com")); + + for (MaintenanceIterator iter = store.iterateAll(); iter.hasNext(); ) { + MaintenanceStoreEntry next = iter.next(); + if (next.get().getName().equals("dent")) { + iter.remove(); + } + } + + assertThat(store.getAll()) + .values() + .extracting("name") + .containsExactly("trillian"); + } + + @Test + void shouldRemoveFromIteratorWithParents() { + StoreTestBuilder testStoreBuilder = new StoreTestBuilder(connectionString, Repository.class.getName(), Group.class.getName()); + SQLiteQueryableMutableStore hogStore = testStoreBuilder.withIds("42", "hog"); + hogStore.put("trisha", new User("trillian", "Trillian McMillan", "mcmillan@hog.com")); + hogStore.put("dent", new User("dent", "Arthur Dent", "dent@hog.com")); + + SQLiteQueryableMutableStore earthStore = testStoreBuilder.withIds("42", "earth"); + earthStore.put("dent", new User("dent", "Arthur Dent", "dent@gmail.com")); + + QueryableMaintenanceStore store = testStoreBuilder.forMaintenanceWithSubIds("42"); + + for (MaintenanceIterator iter = store.iterateAll(); iter.hasNext(); ) { + MaintenanceStoreEntry next = iter.next(); + if (next.get().getName().equals("dent") && next.getParentId(Group.class).get().equals("hog")) { + iter.remove(); + } + } + + assertThat(testStoreBuilder.withIds("42", "hog").getAll()) + .values() + .extracting("name") + .containsExactly("trillian"); + assertThat(testStoreBuilder.withIds("42", "earth").getAll()) + .values() + .extracting("name") + .containsExactly("dent"); + } + + @Test + void shouldReadAll() { + StoreTestBuilder testStoreBuilder = new StoreTestBuilder(connectionString, Repository.class.getName()); + SQLiteQueryableMutableStore hogStore = testStoreBuilder.withIds("42"); + hogStore.put("trisha", new User("trillian", "Trillian McMillan", "mcmillan@hog.com")); + hogStore.put("dent", new User("dent", "Arthur Dent", "dent@hog.com")); + + QueryableMaintenanceStore store = testStoreBuilder.forMaintenanceWithSubIds("42"); + + Collection> rows = store.readAll(); + + assertThat(rows) + .extracting("id") + .containsExactlyInAnyOrder("dent", "trisha"); + assertThat(rows) + .extracting(QueryableMaintenanceStore.Row::getParentIds) + .allSatisfy(strings -> assertThat(strings).containsExactly("42")); + assertThat(rows) + .extracting(QueryableMaintenanceStore.Row::getValue) + .extracting("name") + .containsExactlyInAnyOrder("trillian", "dent"); + } + + @Test + void shouldWriteAllForNewParent() { + StoreTestBuilder testStoreBuilder = new StoreTestBuilder(connectionString, Repository.class.getName()); + + QueryableMaintenanceStore store = testStoreBuilder.forMaintenanceWithSubIds("42"); + store.writeAll( + List.of( + new QueryableMaintenanceStore.Row<>(new String[]{"23"}, "trisha", new User("trillian", "Trillian McMillan", "trisha@hog.com")) + ) + ); + + SQLiteQueryableMutableStore hogStore = testStoreBuilder.withIds("42"); + Collection> allValues = hogStore.readAll(); + assertThat(allValues) + .extracting("value") + .extracting("name") + .containsExactly("trillian"); + } + + @Test + void shouldWriteRawForNewParent() { + StoreTestBuilder testStoreBuilder = new StoreTestBuilder(connectionString, Repository.class.getName()); + + QueryableMaintenanceStore store = testStoreBuilder.forMaintenanceWithSubIds("42"); + store.writeRaw( + List.of( + new QueryableMaintenanceStore.RawRow(new String[]{"23"}, "trisha", "{ \"name\": \"trillian\", \"displayName\": \"Trillian McMillan\", \"mail\": \"mcmillan@hog.com\" }") + ) + ); + + SQLiteQueryableMutableStore hogStore = testStoreBuilder.withIds("42"); + Collection> allValues = hogStore.readAll(); + assertThat(allValues) + .extracting("value") + .extracting("name") + .containsExactly("trillian"); + } + } + + private static final QueryableStore.IdQueryField ID_QUERY_FIELD = + new QueryableStore.IdQueryField<>(); + private static final QueryableStore.IdQueryField GROUP_QUERY_FIELD = + new QueryableStore.IdQueryField<>(Group.class); + private static final QueryableStore.StringQueryField USER_NAME_QUERY_FIELD = + new QueryableStore.StringQueryField<>("name"); + private static final QueryableStore.StringQueryField DISPLAY_NAME_QUERY_FIELD = + new QueryableStore.StringQueryField<>("displayName"); + private static final QueryableStore.StringQueryField MAIL_QUERY_FIELD = + new QueryableStore.StringQueryField<>("mail"); + private static final QueryableStore.NumberQueryField CREATION_DATE_QUERY_FIELD = + new QueryableStore.NumberQueryField<>("creationDate"); + private static final QueryableStore.NumberQueryField CREATION_DATE_AS_INTEGER_QUERY_FIELD = + new QueryableStore.NumberQueryField<>("creationDate"); + private static final QueryableStore.BooleanQueryField ACTIVE_QUERY_FIELD = + new QueryableStore.BooleanQueryField<>("active"); + + enum Range { + SOLAR_SYSTEM, INNER_GALACTIC, INTER_GALACTIC + } + + private static final QueryableStore.StringQueryField SPACESHIP_NAME_QUERY_FIELD = + new QueryableStore.StringQueryField<>("name"); + private static final QueryableStore.EnumQueryField SPACESHIP_RANGE_ENUM_QUERY_FIELD = + new QueryableStore.EnumQueryField<>("range"); + private static final QueryableStore.CollectionQueryField SPACESHIP_CREW_QUERY_FIELD = + new QueryableStore.CollectionQueryField<>("crew"); + private static final QueryableStore.CollectionSizeQueryField SPACESHIP_CREW_SIZE_QUERY_FIELD = + new QueryableStore.CollectionSizeQueryField<>("crew"); + private static final QueryableStore.MapQueryField SPACESHIP_DESTINATIONS_QUERY_FIELD = + new QueryableStore.MapQueryField<>("destinations"); + private static final QueryableStore.MapSizeQueryField SPACESHIP_DESTINATIONS_SIZE_QUERY_FIELD = + new QueryableStore.MapSizeQueryField<>("destinations"); + private static final QueryableStore.InstantQueryField SPACESHIP_INSERVICE_QUERY_FIELD = + new QueryableStore.InstantQueryField<>("inServiceSince"); +} diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteStoreMetaDataProviderTest.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteStoreMetaDataProviderTest.java new file mode 100644 index 0000000000..5557054e0b --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/SQLiteStoreMetaDataProviderTest.java @@ -0,0 +1,94 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.plugin.ExtensionProcessor; +import sonia.scm.plugin.PluginLoader; +import sonia.scm.plugin.QueryableTypeDescriptor; +import sonia.scm.store.QueryableType; + +import java.util.Collection; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class SQLiteStoreMetaDataProviderTest { + + @Mock + private PluginLoader pluginLoader; + @Mock + private ExtensionProcessor extensionProcessor; + @Mock + private QueryableTypeDescriptor descriptor1; + @Mock + private QueryableTypeDescriptor descriptor2; + + private SQLiteStoreMetaDataProvider metaDataProvider; + + @BeforeEach + void setUp() { + when(descriptor1.getTypes()).thenReturn(new String[]{"sonia.scm.store.sqlite.TestParent1.class"}); + when(descriptor1.getClazz()).thenReturn("sonia.scm.store.sqlite.TestChildWithOneParent"); + + when(descriptor2.getTypes()).thenReturn(new String[]{"sonia.scm.store.sqlite.TestParent1.class", "sonia.scm.store.sqlite.TestParent2.class"}); + when(descriptor2.getClazz()).thenReturn("sonia.scm.store.sqlite.TestChildWithTwoParent"); + + when(extensionProcessor.getQueryableTypes()).thenReturn(List.of(descriptor1, descriptor2)); + + when(pluginLoader.getUberClassLoader()).thenReturn(this.getClass().getClassLoader()); + when(pluginLoader.getExtensionProcessor()).thenReturn(extensionProcessor); + + metaDataProvider = new SQLiteStoreMetaDataProvider(pluginLoader); + } + + @Test + void testInitializeType() { + Collection> parent1Types = metaDataProvider.getTypesWithParent(TestParent1.class); + assertThat(parent1Types) + .extracting("name") + .containsExactly( + "sonia.scm.store.sqlite.TestChildWithOneParent", + "sonia.scm.store.sqlite.TestChildWithTwoParent" + ); + + Collection> parent2Types = metaDataProvider.getTypesWithParent(TestParent1.class, TestParent2.class); + assertThat(parent2Types) + .extracting("name") + .containsExactly("sonia.scm.store.sqlite.TestChildWithTwoParent"); + } +} + +class TestParent1 { +} + +class TestParent2 { +} + +@QueryableType(TestParent1.class) +class TestChildWithOneParent { +} + +@QueryableType({TestParent1.class, TestParent2.class}) +class TestChildWithTwoParent { +} diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/Spaceship.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/Spaceship.java new file mode 100644 index 0000000000..7241f37b99 --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/Spaceship.java @@ -0,0 +1,98 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import jakarta.xml.bind.annotation.XmlAccessType; +import jakarta.xml.bind.annotation.XmlAccessorType; +import jakarta.xml.bind.annotation.XmlRootElement; +import lombok.EqualsAndHashCode; +import sonia.scm.store.QueryableType; + +import java.time.Instant; +import java.util.Arrays; +import java.util.Collection; +import java.util.Map; + +@XmlRootElement +@XmlAccessorType(XmlAccessType.FIELD) +@QueryableType +@EqualsAndHashCode +class Spaceship { + String name; + SQLiteQueryableStoreTest.Range range; + Collection crew; + Map destinations; + Instant inServiceSince; + + public Spaceship() { + } + + public Spaceship(String name, SQLiteQueryableStoreTest.Range range) { + this.name = name; + this.range = range; + } + + public Spaceship(String name, String... crew) { + this.name = name; + this.crew = Arrays.asList(crew); + } + + public Spaceship(String name, Map destinations) { + this.name = name; + this.destinations = destinations; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public SQLiteQueryableStoreTest.Range getRange() { + return range; + } + + public void setRange(SQLiteQueryableStoreTest.Range range) { + this.range = range; + } + + public Collection getCrew() { + return crew; + } + + public void setCrew(Collection crew) { + this.crew = crew; + } + + public Map getDestinations() { + return destinations; + } + + public void setDestinations(Map destinations) { + this.destinations = destinations; + } + + public Instant getInServiceSince() { + return inServiceSince; + } + + public void setInServiceSince(Instant inServiceSince) { + this.inServiceSince = inServiceSince; + } +} diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/StoreTestBuilder.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/StoreTestBuilder.java new file mode 100644 index 0000000000..2574344690 --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/StoreTestBuilder.java @@ -0,0 +1,84 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule; +import sonia.scm.security.UUIDKeyGenerator; +import sonia.scm.store.QueryableMaintenanceStore; +import sonia.scm.store.QueryableStore; +import sonia.scm.user.User; + +import java.util.List; + +import static com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES; +import static com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATES_AS_TIMESTAMPS; +import static com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS; +import static sonia.scm.store.sqlite.QueryableTypeDescriptorTestData.createDescriptor; + +class StoreTestBuilder { + + private final ObjectMapper mapper = getObjectMapper(); + + private static ObjectMapper getObjectMapper() { + ObjectMapper objectMapper = new ObjectMapper(); + objectMapper.registerModule(new JavaTimeModule()); + objectMapper.configure(WRITE_DATES_AS_TIMESTAMPS, true); + objectMapper.configure(WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS, false); + objectMapper.configure(FAIL_ON_UNKNOWN_PROPERTIES, false); + return objectMapper; + } + + private final String connectionString; + private final String[] parentClasses; + + StoreTestBuilder(String connectionString, String... parentClasses) { + this.connectionString = connectionString; + this.parentClasses = parentClasses; + } + + SQLiteQueryableMutableStore withIds(String... ids) { + return forClassWithIds(User.class, ids); + } + + QueryableStore withSubIds(String... ids) { + if (ids.length > parentClasses.length) { + throw new IllegalArgumentException("id length should be at most " + parentClasses.length); + } + return createStoreFactory(User.class).getReadOnly(User.class, ids); + } + + QueryableMaintenanceStore forMaintenanceWithSubIds(String... ids) { + if (ids.length > parentClasses.length) { + throw new IllegalArgumentException("id length should be at most " + parentClasses.length); + } + return createStoreFactory(User.class).getForMaintenance(User.class, ids); + } + + SQLiteQueryableMutableStore forClassWithIds(Class clazz, String... ids) { + return createStoreFactory(clazz).getMutable(clazz, ids); + } + + private SQLiteQueryableStoreFactory createStoreFactory(Class clazz) { + return new SQLiteQueryableStoreFactory( + connectionString, + mapper, + new UUIDKeyGenerator(), + List.of(createDescriptor(clazz.getName(), parentClasses)) + ); + } +} diff --git a/scm-persistence/src/test/java/sonia/scm/store/sqlite/TableCreatorTest.java b/scm-persistence/src/test/java/sonia/scm/store/sqlite/TableCreatorTest.java new file mode 100644 index 0000000000..a21a0a6657 --- /dev/null +++ b/scm-persistence/src/test/java/sonia/scm/store/sqlite/TableCreatorTest.java @@ -0,0 +1,151 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store.sqlite; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.plugin.QueryableTypeDescriptor; +import sonia.scm.store.StoreException; + +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.util.LinkedHashMap; +import java.util.Map; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.fail; +import static org.mockito.Mockito.when; +import static sonia.scm.store.sqlite.QueryableTypeDescriptorTestData.createDescriptor; + +@ExtendWith(MockitoExtension.class) +class TableCreatorTest { + + private final Connection connection = DriverManager.getConnection("jdbc:sqlite::memory:");; + + private final TableCreator tableCreator = new TableCreator(connection); + + TableCreatorTest() throws SQLException { + } + + @Test + void shouldCreateTableWithoutParents() throws SQLException { + QueryableTypeDescriptor descriptor = createDescriptor(new String[0]); + + tableCreator.initializeTable(descriptor); + + assertThat(getColumns("com_cloudogu_space_to_be_Spaceship_STORE")) + .containsEntry("ID", "TEXT") + .containsEntry("payload", "JSONB"); + } + + @Test + void shouldCreateNamedTableWithoutParents() throws SQLException { + QueryableTypeDescriptor descriptor = createDescriptor(new String[0]); + when(descriptor.getName()).thenReturn("ships"); + + tableCreator.initializeTable(descriptor); + + assertThat(getColumns("ships_STORE")) + .containsEntry("ID", "TEXT") + .containsEntry("payload", "JSONB"); + } + + @Test + void shouldCreateTableWithSingleParent() throws SQLException { + QueryableTypeDescriptor descriptor = createDescriptor(new String[]{"sonia.scm.repo.Repository.class"}); + + tableCreator.initializeTable(descriptor); + + assertThat(getColumns("com_cloudogu_space_to_be_Spaceship_STORE")) + .containsEntry("Repository_ID", "TEXT") + .containsEntry("ID", "TEXT") + .containsEntry("payload", "JSONB"); + } + + @Test + void shouldCreateTableWithMultipleParents() throws SQLException { + QueryableTypeDescriptor descriptor = createDescriptor(new String[]{"sonia.scm.repo.Repository.class", "sonia.scm.user.User"}); + + tableCreator.initializeTable(descriptor); + + assertThat(getColumns("com_cloudogu_space_to_be_Spaceship_STORE")) + .containsEntry("Repository_ID", "TEXT") + .containsEntry("User_ID", "TEXT") + .containsEntry("ID", "TEXT") + .containsEntry("payload", "JSONB"); + } + + @Test + void shouldFailIfTableExistsWithoutIdColumn() throws SQLException { + QueryableTypeDescriptor descriptor = createDescriptor(new String[0]); + try { + connection.createStatement().execute("CREATE TABLE com_cloudogu_space_to_be_Spaceship_STORE (payload JSONB)"); + tableCreator.initializeTable(descriptor); + fail("exception expected"); + } catch (StoreException e) { + assertThat(e.getMessage()).contains("does not contain ID column"); + } + } + + @Test + void shouldFailIfTableExistsWithoutPayloadColumn() throws SQLException { + QueryableTypeDescriptor descriptor = createDescriptor(new String[0]); + try { + connection.createStatement().execute("CREATE TABLE com_cloudogu_space_to_be_Spaceship_STORE (ID TEXT)"); + tableCreator.initializeTable(descriptor); + fail("exception expected"); + } catch (StoreException e) { + assertThat(e.getMessage()).contains("does not contain payload column"); + } + } + + @Test + void shouldFailIfTableExistsWithoutParentColumn() throws SQLException { + QueryableTypeDescriptor descriptor = createDescriptor(new String[]{"sonia.scm.repo.Repository.class"}); + try { + connection.createStatement().execute("CREATE TABLE com_cloudogu_space_to_be_Spaceship_STORE (ID TEXT, payload JSONB)"); + tableCreator.initializeTable(descriptor); + fail("exception expected"); + } catch (StoreException e) { + assertThat(e.getMessage()).contains("does not contain column Repository_ID"); + } + } + + @Test + void shouldFailIfTableExistsWithTooManyParentColumns() throws SQLException { + QueryableTypeDescriptor descriptor = createDescriptor(new String[]{"sonia.scm.repo.Repository.class"}); + try { + connection.createStatement().execute("CREATE TABLE com_cloudogu_space_to_be_Spaceship_STORE (ID TEXT, Repository_ID, User_ID, payload JSONB)"); + tableCreator.initializeTable(descriptor); + fail("exception expected"); + } catch (StoreException e) { + assertThat(e.getMessage()).contains("but has too many columns"); + } + } + + private Map getColumns(String expectedTableName) throws SQLException { + ResultSet resultSet = connection.createStatement().executeQuery("PRAGMA table_info("+ expectedTableName +")"); + Map columns = new LinkedHashMap<>(); + while (resultSet.next()) { + columns.put(resultSet.getString("name"), resultSet.getString("type")); + } + return columns; + } +} diff --git a/scm-dao-xml/src/test/java/sonia/scm/update/xml/XmlV1PropertyDAOTest.java b/scm-persistence/src/test/java/sonia/scm/update/xml/XmlV1PropertyDAOTest.java similarity index 94% rename from scm-dao-xml/src/test/java/sonia/scm/update/xml/XmlV1PropertyDAOTest.java rename to scm-persistence/src/test/java/sonia/scm/update/xml/XmlV1PropertyDAOTest.java index f3444d781b..4fce89011e 100644 --- a/scm-dao-xml/src/test/java/sonia/scm/update/xml/XmlV1PropertyDAOTest.java +++ b/scm-persistence/src/test/java/sonia/scm/update/xml/XmlV1PropertyDAOTest.java @@ -23,8 +23,9 @@ import sonia.scm.SCMContextProvider; import sonia.scm.Stage; import sonia.scm.repository.RepositoryReadOnlyChecker; import sonia.scm.security.KeyGenerator; -import sonia.scm.store.JAXBConfigurationEntryStoreFactory; -import sonia.scm.store.StoreCacheConfigProvider; +import sonia.scm.store.file.JAXBConfigurationEntryStoreFactory; +import sonia.scm.store.file.StoreCacheConfigProvider; +import sonia.scm.store.file.StoreCacheFactory; import sonia.scm.update.RepositoryV1PropertyReader; import java.io.File; @@ -105,7 +106,7 @@ class XmlV1PropertyDAOTest { Path propFile = configPath.resolve("repository-properties-v1.xml"); Files.write(propFile, PROPERTIES.getBytes()); RepositoryReadOnlyChecker readOnlyChecker = mock(RepositoryReadOnlyChecker.class); - XmlV1PropertyDAO dao = new XmlV1PropertyDAO(new JAXBConfigurationEntryStoreFactory(new SimpleContextProvider(temp), null, new SimpleKeyGenerator(), readOnlyChecker, new StoreCacheConfigProvider(false))); + XmlV1PropertyDAO dao = new XmlV1PropertyDAO(new JAXBConfigurationEntryStoreFactory(new SimpleContextProvider(temp), null, new SimpleKeyGenerator(), readOnlyChecker, new StoreCacheFactory(new StoreCacheConfigProvider(false)))); dao.getProperties(new RepositoryV1PropertyReader()) .forEachEntry((key, prop) -> { diff --git a/scm-dao-xml/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker b/scm-persistence/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker similarity index 100% rename from scm-dao-xml/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker rename to scm-persistence/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker diff --git a/scm-dao-xml/src/test/resources/sonia/scm/store/fixed.format.xml b/scm-persistence/src/test/resources/sonia/scm/store/fixed.format.xml similarity index 100% rename from scm-dao-xml/src/test/resources/sonia/scm/store/fixed.format.xml rename to scm-persistence/src/test/resources/sonia/scm/store/fixed.format.xml diff --git a/scm-dao-xml/src/test/resources/sonia/scm/store/repositoryDaoMetadata.xml b/scm-persistence/src/test/resources/sonia/scm/store/repositoryDaoMetadata.xml similarity index 100% rename from scm-dao-xml/src/test/resources/sonia/scm/store/repositoryDaoMetadata.xml rename to scm-persistence/src/test/resources/sonia/scm/store/repositoryDaoMetadata.xml diff --git a/scm-dao-xml/src/test/resources/sonia/scm/store/wrong.format.xml b/scm-persistence/src/test/resources/sonia/scm/store/wrong.format.xml similarity index 100% rename from scm-dao-xml/src/test/resources/sonia/scm/store/wrong.format.xml rename to scm-persistence/src/test/resources/sonia/scm/store/wrong.format.xml diff --git a/scm-plugins/scm-integration-test-plugin/build.gradle b/scm-plugins/scm-integration-test-plugin/build.gradle index 72902677e9..214047573d 100644 --- a/scm-plugins/scm-integration-test-plugin/build.gradle +++ b/scm-plugins/scm-integration-test-plugin/build.gradle @@ -15,10 +15,12 @@ */ plugins { - id 'org.scm-manager.smp' version '0.17.0' + id 'org.scm-manager.smp' version '0.18.0' } dependencies { + annotationProcessor project(':scm-annotation-processor') + annotationProcessor project(':scm-core-annotation-processor') } scmPlugin { diff --git a/scm-plugins/scm-integration-test-plugin/src/main/java/sonia/scm/it/resource/IntegrationTestResource.java b/scm-plugins/scm-integration-test-plugin/src/main/java/sonia/scm/it/resource/IntegrationTestResource.java index 9cad654e8c..b5357d7648 100644 --- a/scm-plugins/scm-integration-test-plugin/src/main/java/sonia/scm/it/resource/IntegrationTestResource.java +++ b/scm-plugins/scm-integration-test-plugin/src/main/java/sonia/scm/it/resource/IntegrationTestResource.java @@ -24,14 +24,24 @@ import jakarta.ws.rs.Consumes; import jakarta.ws.rs.GET; import jakarta.ws.rs.POST; import jakarta.ws.rs.Path; +import jakarta.ws.rs.PathParam; import jakarta.ws.rs.Produces; import lombok.Getter; import lombok.Setter; import sonia.scm.api.v2.resources.LinkBuilder; import sonia.scm.api.v2.resources.ScmPathInfoStore; +import sonia.scm.repository.NamespaceAndName; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryManager; +import store.RepositoryTestData; +import store.RepositoryTestDataStoreFactory; + +import java.util.Collection; import static de.otto.edison.hal.Embedded.embeddedBuilder; import static de.otto.edison.hal.Links.linkingTo; +import static sonia.scm.ContextEntry.ContextBuilder.entity; +import static sonia.scm.NotFoundException.notFound; /** * Web Service Resource to support integration tests. @@ -43,11 +53,15 @@ public class IntegrationTestResource { private final ScmPathInfoStore scmPathInfoStore; private final MergeDetectionHelper mergeDetectionHelper; + private final RepositoryManager repositoryManager; + private final RepositoryTestDataStoreFactory testDataStoreFactory; @Inject - public IntegrationTestResource(ScmPathInfoStore scmPathInfoStore, MergeDetectionHelper mergeDetectionHelper) { + public IntegrationTestResource(ScmPathInfoStore scmPathInfoStore, MergeDetectionHelper mergeDetectionHelper, RepositoryManager repositoryManager, RepositoryTestDataStoreFactory testDataStoreFactory) { this.scmPathInfoStore = scmPathInfoStore; this.mergeDetectionHelper = mergeDetectionHelper; + this.repositoryManager = repositoryManager; + this.testDataStoreFactory = testDataStoreFactory; } @GET @@ -71,6 +85,26 @@ public class IntegrationTestResource { mergeDetectionHelper.initialize(mergeDetectionConfiguration.getTarget(), mergeDetectionConfiguration.getBranch()); } + @GET + @Path("{namespace}/{name}/test-data") + @Produces("application/json") + public Collection getData(@PathParam("namespace") String namespace, @PathParam("name") String name) { + Repository repository = repositoryManager.get(new NamespaceAndName(namespace, name)); + return testDataStoreFactory.get(repository).query().findAll(); + } + + @POST + @Path("{namespace}/{name}/test-data") + @Consumes("application/json") + public void storeData(@PathParam("namespace") String namespace, @PathParam("name") String name, RepositoryTestData testData) { + NamespaceAndName namespaceAndName = new NamespaceAndName(namespace, name); + Repository repository = repositoryManager.get(namespaceAndName); + if (repository == null) { + throw notFound(entity(namespaceAndName)); + } + testDataStoreFactory.getMutable(repository).put(testData); + } + private String self() { LinkBuilder linkBuilder = new LinkBuilder(scmPathInfoStore.get(), IntegrationTestResource.class); return linkBuilder.method("get").parameters().href(); diff --git a/scm-plugins/scm-integration-test-plugin/src/main/java/sonia/scm/it/resource/RepositoryIntegrationTestLinkEnricher.java b/scm-plugins/scm-integration-test-plugin/src/main/java/sonia/scm/it/resource/RepositoryIntegrationTestLinkEnricher.java new file mode 100644 index 0000000000..7a41824b26 --- /dev/null +++ b/scm-plugins/scm-integration-test-plugin/src/main/java/sonia/scm/it/resource/RepositoryIntegrationTestLinkEnricher.java @@ -0,0 +1,48 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.it.resource; + +import jakarta.inject.Inject; +import jakarta.inject.Provider; +import sonia.scm.api.v2.resources.Enrich; +import sonia.scm.api.v2.resources.HalAppender; +import sonia.scm.api.v2.resources.HalEnricher; +import sonia.scm.api.v2.resources.HalEnricherContext; +import sonia.scm.api.v2.resources.LinkBuilder; +import sonia.scm.api.v2.resources.ScmPathInfoStore; +import sonia.scm.plugin.Extension; +import sonia.scm.repository.Repository; + +@Extension +@Enrich(Repository.class) +public class RepositoryIntegrationTestLinkEnricher implements HalEnricher { + + private final Provider pathInfoStore; + + @Inject + public RepositoryIntegrationTestLinkEnricher(Provider pathInfoStore) { + this.pathInfoStore = pathInfoStore; + } + + @Override + public void enrich(HalEnricherContext context, HalAppender appender) { + Repository repository = context.oneRequireByType(Repository.class); + LinkBuilder linkBuilder = new LinkBuilder(pathInfoStore.get().get(), IntegrationTestResource.class); + String dataUrl = linkBuilder.method("getData").parameters(repository.getNamespace(), repository.getName()).href(); + appender.appendLink("test-data", dataUrl); + } +} diff --git a/scm-plugins/scm-integration-test-plugin/src/main/java/store/RepositoryTestData.java b/scm-plugins/scm-integration-test-plugin/src/main/java/store/RepositoryTestData.java new file mode 100644 index 0000000000..364a8cda40 --- /dev/null +++ b/scm-plugins/scm-integration-test-plugin/src/main/java/store/RepositoryTestData.java @@ -0,0 +1,27 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package store; + +import lombok.Data; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableType; + +@Data +@QueryableType(Repository.class) +public class RepositoryTestData { + private String value; +} diff --git a/scm-queryable-test/build.gradle b/scm-queryable-test/build.gradle new file mode 100644 index 0000000000..0d0f8f05e0 --- /dev/null +++ b/scm-queryable-test/build.gradle @@ -0,0 +1,35 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +plugins { + id 'java-library' + id 'org.scm-manager.java' +} + +dependencies { + api platform(project(':')) + api project(':scm-core') + api project(':scm-persistence') + + api libraries.junitJupiterApi + api libraries.mockitoCore + implementation libraries.jacksonDatatypeJsr310 + + // tests + testImplementation project(':scm-test') + testImplementation libraries.junitPioneer + testAnnotationProcessor project(':scm-core-annotation-processor') +} diff --git a/scm-queryable-test/src/main/java/sonia/scm/store/QueryableStoreExtension.java b/scm-queryable-test/src/main/java/sonia/scm/store/QueryableStoreExtension.java new file mode 100644 index 0000000000..f02d569296 --- /dev/null +++ b/scm-queryable-test/src/main/java/sonia/scm/store/QueryableStoreExtension.java @@ -0,0 +1,150 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.databind.DeserializationFeature; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.extension.AfterEachCallback; +import org.junit.jupiter.api.extension.BeforeEachCallback; +import org.junit.jupiter.api.extension.ExtensionContext; +import org.junit.jupiter.api.extension.ParameterContext; +import org.junit.jupiter.api.extension.ParameterResolutionException; +import org.junit.jupiter.api.extension.ParameterResolver; +import sonia.scm.plugin.QueryableTypeDescriptor; +import sonia.scm.security.UUIDKeyGenerator; +import sonia.scm.store.sqlite.SQLiteQueryableStoreFactory; +import sonia.scm.util.IOUtil; + +import java.io.IOException; +import java.lang.annotation.Retention; +import java.lang.reflect.Constructor; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashSet; +import java.util.Set; + +import static java.util.Arrays.stream; +import static org.mockito.Mockito.lenient; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +/** + * Loads {@link QueryableTypes} into a JUnit test suite. + *
+ * This extension also includes support for {@link Nested} classes: {@link QueryableTypes} attached to a nested class + * are loaded before the types of its parent. + */ +public class QueryableStoreExtension implements ParameterResolver, BeforeEachCallback, AfterEachCallback { + private final ObjectMapper mapper = getObjectMapper(); + private final Set> storeFactoryClasses = new HashSet<>(); + private Path tempDirectory; + private Collection queryableTypeDescriptors; + private SQLiteQueryableStoreFactory storeFactory; + + private static ObjectMapper getObjectMapper() { + return new ObjectMapper() + .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) + .registerModule(new JavaTimeModule()) + .configure(JsonParser.Feature.IGNORE_UNDEFINED, true) + .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); + } + + @Override + public void beforeEach(ExtensionContext context) throws IOException { + tempDirectory = Files.createTempDirectory("test"); + String connectionString = "jdbc:sqlite:" + tempDirectory.toString() + "/test.db"; + queryableTypeDescriptors = new ArrayList<>(); + addDescriptors(context); + storeFactory = new SQLiteQueryableStoreFactory( + connectionString, + mapper, + new UUIDKeyGenerator(), + queryableTypeDescriptors + ); + } + + @Override + public void afterEach(ExtensionContext context) throws IOException { + IOUtil.delete(tempDirectory.toFile()); + } + + private void addDescriptors(ExtensionContext context) { + context.getTestClass().ifPresent( + testClass -> { + QueryableTypes annotation = testClass.getAnnotation(QueryableTypes.class); + if (annotation != null) { + queryableTypeDescriptors.addAll(stream( + annotation + .value() + ).map(this::createDescriptor).toList()); + } + } + ); + + context.getParent().ifPresent(this::addDescriptors); + } + + private QueryableTypeDescriptor createDescriptor(Class clazz) { + QueryableTypeDescriptor descriptor = mock(QueryableTypeDescriptor.class); + QueryableType queryableAnnotation = clazz.getAnnotation(QueryableType.class); + when(descriptor.getTypes()).thenReturn(stream(queryableAnnotation.value()).map(Class::getName).toArray(String[]::new)); + lenient().when(descriptor.getClazz()).thenReturn(clazz.getName()); + when(descriptor.getName()).thenReturn(queryableAnnotation.name()); + try { + Class storeFactoryClass = Class.forName(clazz.getName() + "StoreFactory"); + storeFactoryClasses.add(storeFactoryClass); + } catch (ClassNotFoundException e) { + throw new RuntimeException("class for store factory not found", e); + } + return descriptor; + } + + @Override + public boolean supportsParameter(ParameterContext parameterContext, ExtensionContext extensionContext) throws ParameterResolutionException { + Class requestedParameterType = parameterContext.getParameter().getType(); + return requestedParameterType.equals(QueryableStoreFactory.class) + || storeFactoryClasses.contains(requestedParameterType); + } + + @Override + public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext) throws ParameterResolutionException { + Class requestedParameterType = parameterContext.getParameter().getType(); + if (requestedParameterType.equals(QueryableStoreFactory.class)) { + return storeFactory; + } else if (storeFactoryClasses.contains(requestedParameterType)) { + try { + Constructor constructor = requestedParameterType.getDeclaredConstructor(QueryableStoreFactory.class); + constructor.setAccessible(true); + return constructor.newInstance(storeFactory); + } catch (Exception e) { + throw new RuntimeException("failed to instantiate store factory", e); + } + } else { + throw new ParameterResolutionException("unsupported parameter type"); + } + } + + @Retention(java.lang.annotation.RetentionPolicy.RUNTIME) + public @interface QueryableTypes { + Class[] value(); + } +} diff --git a/scm-queryable-test/src/test/java/sonia/scm/store/QueryableStoreExtensionTest.java b/scm-queryable-test/src/test/java/sonia/scm/store/QueryableStoreExtensionTest.java new file mode 100644 index 0000000000..78f387b310 --- /dev/null +++ b/scm-queryable-test/src/test/java/sonia/scm/store/QueryableStoreExtensionTest.java @@ -0,0 +1,53 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +@ExtendWith(QueryableStoreExtension.class) +@QueryableStoreExtension.QueryableTypes(Spaceship.class) +class QueryableStoreExtensionTest { + + @Test + void shouldProvideQueryableStoreFactory(QueryableStoreFactory storeFactory) { + QueryableMutableStore store = storeFactory.getMutable(Spaceship.class); + store.put(new Spaceship("Heart Of Gold")); + assertEquals(1, store.getAll().size()); + } + + @Test + void shouldProvideTypeRelatedStoreFactory(SpaceshipStoreFactory storeFactory) { + QueryableMutableStore store = storeFactory.getMutable(); + store.put(new Spaceship("Heart Of Gold")); + assertEquals(1, store.getAll().size()); + } +} + +@QueryableType +class Spaceship { + String name; + + Spaceship() { + } + + Spaceship(String name) { + this.name = name; + } +} diff --git a/scm-webapp/build.gradle b/scm-webapp/build.gradle index 8509fdbc6a..b7554817ab 100644 --- a/scm-webapp/build.gradle +++ b/scm-webapp/build.gradle @@ -50,9 +50,11 @@ dependencies { assets project(path: ':scm-ui', configuration: 'assets') implementation project(':scm-core') - implementation project(':scm-dao-xml') + implementation project(':scm-persistence') testImplementation project(':scm-test') + testImplementation project(':scm-queryable-test') annotationProcessor project(':scm-annotation-processor') + testAnnotationProcessor project(':scm-core-annotation-processor') // servlet api providedCompile libraries.servletApi diff --git a/scm-webapp/src/main/java/sonia/scm/group/GroupDeletionNotifier.java b/scm-webapp/src/main/java/sonia/scm/group/GroupDeletionNotifier.java new file mode 100644 index 0000000000..49d748a0bf --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/group/GroupDeletionNotifier.java @@ -0,0 +1,40 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.group; + +import com.github.legman.ReferenceType; +import com.github.legman.Subscribe; +import sonia.scm.HandlerEventType; +import sonia.scm.plugin.Extension; +import sonia.scm.store.StoreDeletionNotifier; + +@Extension +public class GroupDeletionNotifier implements StoreDeletionNotifier { + private DeletionHandler handler; + + @Override + public void registerHandler(DeletionHandler handler) { + this.handler = handler; + } + + @Subscribe(referenceType = ReferenceType.STRONG) + public void onDelete(GroupEvent event) { + if (handler != null && event.getEventType() == HandlerEventType.DELETE) { + handler.notifyDeleted(Group.class, event.getItem().getId()); + } + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryExporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryExporter.java index 8f4ce649bf..f97bfecca2 100644 --- a/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryExporter.java +++ b/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryExporter.java @@ -37,6 +37,7 @@ import java.io.FileOutputStream; import java.io.IOException; import java.io.OutputStream; import java.nio.file.Files; +import java.nio.file.Path; import java.nio.file.Paths; import static sonia.scm.ContextEntry.ContextBuilder.entity; @@ -47,6 +48,7 @@ public class FullScmRepositoryExporter implements FullRepositoryExporter { static final String SCM_ENVIRONMENT_FILE_NAME = "scm-environment.xml"; static final String METADATA_FILE_NAME = "metadata.xml"; static final String STORE_DATA_FILE_NAME = "store-data.tar"; + static final String QUERYABLE_STORE_DATA_FILE_NAME = "queryable-store-data.tar"; private final EnvironmentInformationXmlGenerator environmentGenerator; private final RepositoryMetadataXmlGenerator metadataGenerator; private final RepositoryServiceFactory serviceFactory; @@ -55,17 +57,20 @@ public class FullScmRepositoryExporter implements FullRepositoryExporter { private final RepositoryExportingCheck repositoryExportingCheck; private final RepositoryImportExportEncryption repositoryImportExportEncryption; private final ExportNotificationHandler notificationHandler; - private final AdministrationContext administrationContext; + private final RepositoryQueryableStoreExporter queryableStoreExporter; @Inject - public FullScmRepositoryExporter(EnvironmentInformationXmlGenerator environmentGenerator, - RepositoryMetadataXmlGenerator metadataGenerator, - RepositoryServiceFactory serviceFactory, - TarArchiveRepositoryStoreExporter storeExporter, - WorkdirProvider workdirProvider, - RepositoryExportingCheck repositoryExportingCheck, - RepositoryImportExportEncryption repositoryImportExportEncryption, ExportNotificationHandler notificationHandler, AdministrationContext administrationContext) { + FullScmRepositoryExporter(EnvironmentInformationXmlGenerator environmentGenerator, + RepositoryMetadataXmlGenerator metadataGenerator, + RepositoryServiceFactory serviceFactory, + TarArchiveRepositoryStoreExporter storeExporter, + WorkdirProvider workdirProvider, + RepositoryExportingCheck repositoryExportingCheck, + RepositoryImportExportEncryption repositoryImportExportEncryption, + ExportNotificationHandler notificationHandler, + AdministrationContext administrationContext, + RepositoryQueryableStoreExporter queryableStoreExporter) { this.environmentGenerator = environmentGenerator; this.metadataGenerator = metadataGenerator; this.serviceFactory = serviceFactory; @@ -75,6 +80,7 @@ public class FullScmRepositoryExporter implements FullRepositoryExporter { this.repositoryImportExportEncryption = repositoryImportExportEncryption; this.notificationHandler = notificationHandler; this.administrationContext = administrationContext; + this.queryableStoreExporter = queryableStoreExporter; } public void export(Repository repository, OutputStream outputStream, String password) { @@ -95,11 +101,12 @@ public class FullScmRepositoryExporter implements FullRepositoryExporter { BufferedOutputStream bos = new BufferedOutputStream(outputStream); OutputStream cos = repositoryImportExportEncryption.optionallyEncrypt(bos, password); GzipCompressorOutputStream gzos = new GzipCompressorOutputStream(cos); - TarArchiveOutputStream taos = Archives.createTarOutputStream(gzos); + TarArchiveOutputStream taos = Archives.createTarOutputStream(gzos) ) { writeEnvironmentData(repository, taos); writeMetadata(repository, taos); writeStoreData(repository, taos); + writeQueryableStoreData(repository, taos); writeRepository(service, taos); taos.finish(); } catch (IOException e) { @@ -136,20 +143,13 @@ public class FullScmRepositoryExporter implements FullRepositoryExporter { } private void writeRepository(RepositoryService service, TarArchiveOutputStream taos) throws IOException { - File newWorkdir = workdirProvider.createNewWorkdir(service.getRepository().getId()); - try { + createAndAddFromTemporaryDirectory(service.getRepository(), taos, createRepositoryEntryName(service), newWorkdir -> { File repositoryFile = Files.createFile(Paths.get(newWorkdir.getPath(), "repository")).toFile(); try (FileOutputStream repositoryFos = new FileOutputStream(repositoryFile)) { service.getBundleCommand().bundle(repositoryFos); } - TarArchiveEntry entry = new TarArchiveEntry(createRepositoryEntryName(service)); - entry.setSize(repositoryFile.length()); - taos.putArchiveEntry(entry); - Files.copy(repositoryFile.toPath(), taos); - taos.closeArchiveEntry(); - } finally { - IOUtil.deleteSilently(newWorkdir); - } + return repositoryFile; + }); } private String createRepositoryEntryName(RepositoryService service) { @@ -157,19 +157,46 @@ public class FullScmRepositoryExporter implements FullRepositoryExporter { } private void writeStoreData(Repository repository, TarArchiveOutputStream taos) throws IOException { - File newWorkdir = workdirProvider.createNewWorkdir(repository.getId()); - try { + createAndAddFromTemporaryDirectory(repository, taos, STORE_DATA_FILE_NAME, newWorkdir -> { File metadata = Files.createFile(Paths.get(newWorkdir.getPath(), "metadata")).toFile(); try (FileOutputStream metadataFos = new FileOutputStream(metadata)) { storeExporter.export(repository, metadataFos); } - TarArchiveEntry entry = new TarArchiveEntry(STORE_DATA_FILE_NAME); - entry.setSize(metadata.length()); - taos.putArchiveEntry(entry); - Files.copy(metadata.toPath(), taos); - taos.closeArchiveEntry(); + return metadata; + }); + } + + private void writeQueryableStoreData(Repository repository, TarArchiveOutputStream taos) throws IOException { + createAndAddFromTemporaryDirectory(repository, taos, QUERYABLE_STORE_DATA_FILE_NAME, newWorkdir -> { + Path queryableTarFilePath = Paths.get(newWorkdir.getPath(), QUERYABLE_STORE_DATA_FILE_NAME); + File queryableTarFile = Files.createFile(queryableTarFilePath).toFile(); + try (FileOutputStream fos = new FileOutputStream(queryableTarFile); + TarArchiveOutputStream tempTaos = Archives.createTarOutputStream(fos)) { + queryableStoreExporter.addQueryableStoreDataToArchive(repository, newWorkdir, tempTaos); + } + return queryableTarFile; + }); + } + + private void createAndAddFromTemporaryDirectory(Repository repository, TarArchiveOutputStream taos, String entryName, PackFileProducer packFileProducer) throws IOException { + File newWorkdir = workdirProvider.createNewWorkdir(repository.getId()); + try { + File tempFile = packFileProducer.packFile(newWorkdir); + addToTar(entryName, tempFile, taos); } finally { IOUtil.deleteSilently(newWorkdir); } } + + private static void addToTar(String storeDataFileName, File metadata, TarArchiveOutputStream taos) throws IOException { + TarArchiveEntry entry = new TarArchiveEntry(storeDataFileName); + entry.setSize(metadata.length()); + taos.putArchiveEntry(entry); + Files.copy(metadata.toPath(), taos); + taos.closeArchiveEntry(); + } + + private interface PackFileProducer { + File packFile(File newWorkdir) throws IOException; + } } diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryImporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryImporter.java index b641e78c84..d9914d3524 100644 --- a/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryImporter.java +++ b/scm-webapp/src/main/java/sonia/scm/importexport/FullScmRepositoryImporter.java @@ -26,6 +26,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; import sonia.scm.ContextEntry; import sonia.scm.event.ScmEventBus; +import sonia.scm.repository.ClearRepositoryCacheEvent; import sonia.scm.repository.FullRepositoryImporter; import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryImportEvent; @@ -33,6 +34,7 @@ import sonia.scm.repository.RepositoryManager; import sonia.scm.repository.RepositoryPermission; import sonia.scm.repository.RepositoryPermissions; import sonia.scm.repository.api.ImportFailedException; +import sonia.scm.update.UpdateEngine; import java.io.BufferedInputStream; import java.io.IOException; @@ -53,21 +55,31 @@ public class FullScmRepositoryImporter implements FullRepositoryImporter { private final RepositoryImportExportEncryption repositoryImportExportEncryption; private final ScmEventBus eventBus; private final RepositoryImportLoggerFactory loggerFactory; + private final UpdateEngine updateEngine; @Inject - public FullScmRepositoryImporter(EnvironmentCheckStep environmentCheckStep, + FullScmRepositoryImporter(EnvironmentCheckStep environmentCheckStep, MetadataImportStep metadataImportStep, StoreImportStep storeImportStep, + QueryableStoreImportStep queryableStoreImportStep, RepositoryImportStep repositoryImportStep, RepositoryManager repositoryManager, RepositoryImportExportEncryption repositoryImportExportEncryption, RepositoryImportLoggerFactory loggerFactory, - ScmEventBus eventBus) { + ScmEventBus eventBus, + UpdateEngine updateEngine) { this.repositoryManager = repositoryManager; this.loggerFactory = loggerFactory; this.repositoryImportExportEncryption = repositoryImportExportEncryption; this.eventBus = eventBus; - importSteps = new ImportStep[]{environmentCheckStep, metadataImportStep, storeImportStep, repositoryImportStep}; + this.updateEngine = updateEngine; + importSteps = new ImportStep[]{ + environmentCheckStep, + metadataImportStep, + storeImportStep, + queryableStoreImportStep, + repositoryImportStep + }; } public Repository importFromStream(Repository repository, InputStream inputStream, String password) { @@ -122,11 +134,17 @@ public class FullScmRepositoryImporter implements FullRepositoryImporter { logger.repositoryCreated(state.getRepository()); try { TarArchiveEntry tarArchiveEntry; - while ((tarArchiveEntry = tais.getNextTarEntry()) != null) { + while ((tarArchiveEntry = tais.getNextEntry()) != null) { LOG.trace("Trying to handle tar entry '{}'", tarArchiveEntry.getName()); handle(tais, state, tarArchiveEntry); } + stream(importSteps).forEach(step -> step.finish(state)); + + eventBus.post(new ClearRepositoryCacheEvent(createdRepository)); + updateEngine.update(repository.getId()); + eventBus.post(new ClearRepositoryCacheEvent(createdRepository)); + state.getLogger().finished(); return state.getRepository(); } catch (RuntimeException | IOException e) { diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/NoneClosingTarArchiveInputStream.java b/scm-webapp/src/main/java/sonia/scm/importexport/NoneClosingTarArchiveInputStream.java new file mode 100644 index 0000000000..0fd5ea4730 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/NoneClosingTarArchiveInputStream.java @@ -0,0 +1,34 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import org.apache.commons.compress.archivers.tar.TarArchiveInputStream; + +import java.io.IOException; +import java.io.InputStream; + +class NoneClosingTarArchiveInputStream extends TarArchiveInputStream { + + NoneClosingTarArchiveInputStream(InputStream is) { + super(is); + } + + @Override + public void close() throws IOException { + // Do not close this input stream + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/QueryableStoreImportStep.java b/scm-webapp/src/main/java/sonia/scm/importexport/QueryableStoreImportStep.java new file mode 100644 index 0000000000..46bc1505c9 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/QueryableStoreImportStep.java @@ -0,0 +1,85 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import jakarta.inject.Inject; +import lombok.extern.slf4j.Slf4j; +import org.apache.commons.compress.archivers.tar.TarArchiveEntry; +import org.apache.commons.compress.archivers.tar.TarArchiveInputStream; +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.repository.api.ImportFailedException; + +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.nio.file.Files; +import java.nio.file.Path; + +import static sonia.scm.ContextEntry.ContextBuilder.entity; +import static sonia.scm.importexport.FullScmRepositoryExporter.QUERYABLE_STORE_DATA_FILE_NAME; + +@Slf4j +class QueryableStoreImportStep implements ImportStep { + private final RepositoryQueryableStoreExporter queryableStoreExporter; + private final RepositoryLocationResolver locationResolver; + + @Inject + QueryableStoreImportStep(RepositoryQueryableStoreExporter queryableStoreExporter, RepositoryLocationResolver locationResolver) { + this.queryableStoreExporter = queryableStoreExporter; + this.locationResolver = locationResolver; + } + + @Override + public boolean handle(TarArchiveEntry entry, ImportState state, InputStream inputStream) { + if (entry.getName().equals(QUERYABLE_STORE_DATA_FILE_NAME) && !entry.isDirectory()) { + log.trace("Importing store from tar"); + state.getLogger().step("importing queryable stores"); + + Path repositoryPath = locationResolver + .forClass(Path.class) + .getLocation(state.getRepository().getId()); + + try { + extractTarToDirectory(inputStream, repositoryPath.toFile()); + queryableStoreExporter.importStores(state.getRepository().getId(), repositoryPath.toFile()); + + return true; + } catch (IOException e) { + throw new ImportFailedException(entity(state.getRepository()).build(), "Failed to extract TAR content", e); + } + } + return false; + } + + private void extractTarToDirectory(InputStream inputStream, File outputDir) throws IOException { + try (TarArchiveInputStream tarInput = new NoneClosingTarArchiveInputStream(inputStream)) { + TarArchiveEntry entry; + while ((entry = tarInput.getNextEntry()) != null) { + if (entry.isDirectory()) { + continue; + } + File outputFile = new File(outputDir, entry.getName()); + outputFile.getParentFile().mkdirs(); + + try (OutputStream outputStream = Files.newOutputStream(outputFile.toPath())) { + tarInput.transferTo(outputStream); + } + } + } + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/RemainingQueryableStoreImporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/RemainingQueryableStoreImporter.java new file mode 100644 index 0000000000..7d64186a88 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/RemainingQueryableStoreImporter.java @@ -0,0 +1,81 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import jakarta.inject.Inject; +import lombok.extern.slf4j.Slf4j; +import sonia.scm.repository.RepositoryLocationResolver; +import sonia.scm.repository.xml.PathBasedRepositoryLocationResolver; + +import java.io.File; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +/** + * The job of this class is to check for remaining queryable store data from former imports, that have not been + * imported yet. This can happen if a repository was imported, when not all plugins were installed but those plugins + * are installed now. After this is done, the update process has to be run for all repositories. + */ +@Slf4j +public class RemainingQueryableStoreImporter { + + private final RepositoryLocationResolver.RepositoryLocationResolverInstance repositoryLocationResolverInstance; + private final RepositoryQueryableStoreExporter queryableStoreExporter; + + @Inject + public RemainingQueryableStoreImporter(PathBasedRepositoryLocationResolver repositoryLocationResolver, + RepositoryQueryableStoreExporter queryableStoreExporter) { + this.repositoryLocationResolverInstance = repositoryLocationResolver.create(Path.class); + this.queryableStoreExporter = queryableStoreExporter; + } + + public void onInitializationCompleted() { + log.info("Starting import of remaining queryable store data for all repositories."); + + repositoryLocationResolverInstance.forAllLocations((repositoryId, repositoryPath) -> { + File repoDir = repositoryPath.toFile(); + File dataDir = new File(repoDir, "queryable-store-data"); + + if (dataDir.exists() && dataDir.isDirectory()) { + List xmlFiles = getXmlFiles(dataDir); + if (!xmlFiles.isEmpty()) { + log.info("Found {} XML files in repository {} - importing...", xmlFiles.size(), repositoryId); + queryableStoreExporter.importStores(repositoryId, repoDir); + } + } + }); + + log.info("Finished importing queryable store data."); + } + + private List getXmlFiles(File directory) { + try (Stream files = Files.list(directory.toPath())) { + return files + .map(Path::toFile) + .filter(file -> file.getName().endsWith(".xml")) + .collect(Collectors.toList()); + } catch (IOException e) { + log.error("Error reading directory {}: {}", directory.getAbsolutePath(), e.getMessage()); + return List.of(); + } + } +} + diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/RepositoryQueryableStoreExporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/RepositoryQueryableStoreExporter.java new file mode 100644 index 0000000000..ff68856d65 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/importexport/RepositoryQueryableStoreExporter.java @@ -0,0 +1,146 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import jakarta.inject.Inject; +import jakarta.xml.bind.JAXBContext; +import jakarta.xml.bind.JAXBException; +import jakarta.xml.bind.Marshaller; +import jakarta.xml.bind.Unmarshaller; +import jakarta.xml.bind.annotation.XmlAccessType; +import jakarta.xml.bind.annotation.XmlAccessorType; +import jakarta.xml.bind.annotation.XmlRootElement; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.apache.commons.compress.archivers.tar.TarArchiveEntry; +import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableMaintenanceStore; +import sonia.scm.store.QueryableStoreFactory; +import sonia.scm.store.StoreMetaDataProvider; + +import java.io.File; +import java.io.IOException; +import java.nio.file.Files; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + +@Slf4j +public class RepositoryQueryableStoreExporter { + + private final StoreMetaDataProvider metaDataProvider; + private final QueryableStoreFactory storeFactory; + + + @Inject + RepositoryQueryableStoreExporter(StoreMetaDataProvider metaDataProvider, + QueryableStoreFactory storeFactory) { + this.metaDataProvider = metaDataProvider; + this.storeFactory = storeFactory; + } + + void addQueryableStoreDataToArchive(Repository repository, File newWorkdir, TarArchiveOutputStream tempTaos) throws IOException { + TarArchiveEntry dirEntry = new TarArchiveEntry("queryable-store-data/"); + tempTaos.putArchiveEntry(dirEntry); + tempTaos.closeArchiveEntry(); + + File dataDir = new File(newWorkdir, "queryable-store-data"); + if (!dataDir.mkdirs()) { + throw new RuntimeException("Could not create temp directory: " + dataDir.getAbsolutePath()); + } + + exportStores(repository.getId(), dataDir); + + File[] xmlFiles = dataDir.listFiles(); + if (xmlFiles != null) { + for (File xmlFile : xmlFiles) { + TarArchiveEntry fileEntry = new TarArchiveEntry("queryable-store-data/" + xmlFile.getName()); + fileEntry.setSize(xmlFile.length()); + tempTaos.putArchiveEntry(fileEntry); + Files.copy(xmlFile.toPath(), tempTaos); + tempTaos.closeArchiveEntry(); + } + } + tempTaos.finish(); + } + + void exportStores(String repositoryId, File workdir) { + try { + JAXBContext jaxbContext = JAXBContext.newInstance(StoreExport.class); + Marshaller marshaller = jaxbContext.createMarshaller(); + for (Class type : metaDataProvider.getTypesWithParent(Repository.class)) { + Collection rows = storeFactory.getForMaintenance(type, repositoryId).readRaw(); + StoreExport export = new StoreExport(type, rows); + marshaller.marshal(export, new File(workdir, type.getName() + ".xml")); + } + } catch (JAXBException e) { + throw new RuntimeException(e); + } + } + + void importStores(String repositoryId, File workdir) { + try { + File dataDir = new File(workdir, "queryable-store-data"); + if (!dataDir.exists() || !dataDir.isDirectory()) { + throw new RuntimeException("Directory 'queryable-store-data' not found in workdir: " + workdir.getAbsolutePath()); + } + + JAXBContext jaxbContext = JAXBContext.newInstance(StoreExport.class); + Unmarshaller unmarshaller = jaxbContext.createUnmarshaller(); + + for (Class type : metaDataProvider.getTypesWithParent(Repository.class)) { + File file = new File(dataDir, type.getName() + ".xml"); + if (!file.exists() || file.length() == 0) { + continue; + } + + StoreExport export = (StoreExport) unmarshaller.unmarshal(file); + Collection rows = export.getRows(); + if (rows == null) { + continue; + } + + storeFactory.getForMaintenance(type, repositoryId).writeRaw(rows); + + try { + Files.delete(file.toPath()); + log.trace("Deleted imported file: {}", file.getAbsolutePath()); + } catch (IOException e) { + log.error("Failed to delete imported file: {} - {}", file.getAbsolutePath(), e.getMessage()); + } + } + } catch (JAXBException e) { + throw new RuntimeException(e); + } + } + + @Getter + @XmlRootElement + @NoArgsConstructor + @XmlAccessorType(XmlAccessType.FIELD) + private static class StoreExport { + private String type; + private Collection rows = new ArrayList<>(); + + StoreExport(Class type, Collection rows) { + this.type = type.getName(); + this.rows = rows != null ? rows : new ArrayList<>(); + } + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/StoreImportStep.java b/scm-webapp/src/main/java/sonia/scm/importexport/StoreImportStep.java index 477f495ffc..44e0b079a0 100644 --- a/scm-webapp/src/main/java/sonia/scm/importexport/StoreImportStep.java +++ b/scm-webapp/src/main/java/sonia/scm/importexport/StoreImportStep.java @@ -21,7 +21,6 @@ import org.apache.commons.compress.archivers.tar.TarArchiveEntry; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import sonia.scm.repository.Repository; -import sonia.scm.update.UpdateEngine; import java.io.InputStream; @@ -32,12 +31,10 @@ class StoreImportStep implements ImportStep { private static final Logger LOG = LoggerFactory.getLogger(StoreImportStep.class); private final TarArchiveRepositoryStoreImporter storeImporter; - private final UpdateEngine updateEngine; @Inject - StoreImportStep(TarArchiveRepositoryStoreImporter storeImporter, UpdateEngine updateEngine) { + StoreImportStep(TarArchiveRepositoryStoreImporter storeImporter) { this.storeImporter = storeImporter; - this.updateEngine = updateEngine; } @Override @@ -47,15 +44,11 @@ class StoreImportStep implements ImportStep { state.getLogger().step("importing stores"); // Inside the repository tar archive stream is another tar archive. // The nested tar archive is wrapped in another TarArchiveInputStream inside the storeImporter - importStores(state.getRepository(), inputStream, state.getLogger()); + Repository repository = state.getRepository(); + storeImporter.importFromTarArchive(repository, inputStream, state.getLogger()); state.storeImported(); return true; } return false; } - - private void importStores(Repository repository, InputStream inputStream, RepositoryImportLogger logger) { - storeImporter.importFromTarArchive(repository, inputStream, logger); - updateEngine.update(repository.getId()); - } } diff --git a/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporter.java b/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporter.java index cf60ca8890..23541a2d0b 100644 --- a/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporter.java +++ b/scm-webapp/src/main/java/sonia/scm/importexport/TarArchiveRepositoryStoreImporter.java @@ -110,16 +110,4 @@ public class TarArchiveRepositoryStoreImporter { private boolean isConfigStore(String storeType) { return storeType.equals(StoreType.CONFIG.getValue()) || storeType.equals(StoreType.CONFIG_ENTRY.getValue()); } - - static class NoneClosingTarArchiveInputStream extends TarArchiveInputStream { - - public NoneClosingTarArchiveInputStream(InputStream is) { - super(is); - } - - @Override - public void close() throws IOException { - // Do not close this input stream - } - } } diff --git a/scm-webapp/src/main/java/sonia/scm/lifecycle/BootstrapContextListener.java b/scm-webapp/src/main/java/sonia/scm/lifecycle/BootstrapContextListener.java index 989e37bb31..9ea57034cf 100644 --- a/scm-webapp/src/main/java/sonia/scm/lifecycle/BootstrapContextListener.java +++ b/scm-webapp/src/main/java/sonia/scm/lifecycle/BootstrapContextListener.java @@ -27,6 +27,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; import sonia.scm.SCMContext; import sonia.scm.config.LoggingConfiguration; +import sonia.scm.importexport.RemainingQueryableStoreImporter; import sonia.scm.lifecycle.classloading.ClassLoaderLifeCycle; import sonia.scm.lifecycle.modules.ApplicationModuleProvider; import sonia.scm.lifecycle.modules.BootstrapModule; @@ -172,6 +173,9 @@ public class BootstrapContextListener extends GuiceServletContextListener { private void processUpdates(PluginLoader pluginLoader, Injector bootstrapInjector) { Injector updateInjector = bootstrapInjector.createChildInjector(new UpdateStepModule(pluginLoader)); + RemainingQueryableStoreImporter importer = updateInjector.getInstance(RemainingQueryableStoreImporter.class); + importer.onInitializationCompleted(); + UpdateEngine updateEngine = updateInjector.getInstance(UpdateEngine.class); updateEngine.update(); } diff --git a/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/BootstrapModule.java b/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/BootstrapModule.java index 71a21e2c0d..f6e30c6988 100644 --- a/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/BootstrapModule.java +++ b/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/BootstrapModule.java @@ -16,6 +16,7 @@ package sonia.scm.lifecycle.modules; +import com.fasterxml.jackson.databind.ObjectMapper; import com.google.inject.AbstractModule; import com.google.inject.TypeLiteral; import com.google.inject.multibindings.Multibinder; @@ -25,6 +26,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; import sonia.scm.SCMContext; import sonia.scm.SCMContextProvider; +import sonia.scm.api.rest.ObjectMapperProvider; import sonia.scm.cache.CacheManager; import sonia.scm.cache.GuavaCacheManager; import sonia.scm.io.DefaultFileSystem; @@ -52,15 +54,19 @@ import sonia.scm.store.ConfigurationStore; import sonia.scm.store.ConfigurationStoreDecoratorFactory; import sonia.scm.store.ConfigurationStoreFactory; import sonia.scm.store.DataStoreFactory; -import sonia.scm.store.DefaultBlobDirectoryAccess; -import sonia.scm.store.FileBlobStoreFactory; -import sonia.scm.store.FileNamespaceUpdateIterator; -import sonia.scm.store.FileRepositoryUpdateIterator; import sonia.scm.store.FileStoreUpdateStepUtilFactory; -import sonia.scm.store.JAXBConfigurationEntryStoreFactory; -import sonia.scm.store.JAXBConfigurationStoreFactory; -import sonia.scm.store.JAXBDataStoreFactory; -import sonia.scm.store.JAXBPropertyFileAccess; +import sonia.scm.store.QueryableStoreFactory; +import sonia.scm.store.StoreMetaDataProvider; +import sonia.scm.store.file.DefaultBlobDirectoryAccess; +import sonia.scm.store.file.FileBlobStoreFactory; +import sonia.scm.store.file.FileNamespaceUpdateIterator; +import sonia.scm.store.file.FileRepositoryUpdateIterator; +import sonia.scm.store.file.JAXBConfigurationEntryStoreFactory; +import sonia.scm.store.file.JAXBConfigurationStoreFactory; +import sonia.scm.store.file.JAXBDataStoreFactory; +import sonia.scm.store.file.JAXBPropertyFileAccess; +import sonia.scm.store.sqlite.SQLiteQueryableStoreFactory; +import sonia.scm.store.sqlite.SQLiteStoreMetaDataProvider; import sonia.scm.update.BlobDirectoryAccess; import sonia.scm.update.DefaultRepositoryPermissionUpdater; import sonia.scm.update.NamespaceUpdateIterator; @@ -116,6 +122,8 @@ public class BootstrapModule extends AbstractModule { bind(ConfigurationEntryStoreFactory.class, JAXBConfigurationEntryStoreFactory.class); bind(DataStoreFactory.class, JAXBDataStoreFactory.class); bind(BlobStoreFactory.class, FileBlobStoreFactory.class); + bind(QueryableStoreFactory.class, SQLiteQueryableStoreFactory.class); + bind(StoreMetaDataProvider.class, SQLiteStoreMetaDataProvider.class); bind(PluginLoader.class).toInstance(pluginLoader); bind(V1PropertyDAO.class, XmlV1PropertyDAO.class); bind(PropertyFileAccess.class, JAXBPropertyFileAccess.class); @@ -123,8 +131,9 @@ public class BootstrapModule extends AbstractModule { bind(RepositoryUpdateIterator.class, FileRepositoryUpdateIterator.class); bind(NamespaceUpdateIterator.class, FileNamespaceUpdateIterator.class); bind(StoreUpdateStepUtilFactory.class, FileStoreUpdateStepUtilFactory.class); - bind(RepositoryPermissionUpdater.class, DefaultRepositoryPermissionUpdater.class); + bind(ObjectMapper.class).toProvider(ObjectMapperProvider.class); bind(new TypeLiteral>() {}).to(new TypeLiteral() {}); + bind(RepositoryPermissionUpdater.class, DefaultRepositoryPermissionUpdater.class); // bind metrics bind(MeterRegistry.class).toProvider(MeterRegistryProvider.class).asEagerSingleton(); diff --git a/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/ScmServletModule.java b/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/ScmServletModule.java index 48fad066da..1f138fd791 100644 --- a/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/ScmServletModule.java +++ b/scm-webapp/src/main/java/sonia/scm/lifecycle/modules/ScmServletModule.java @@ -16,7 +16,6 @@ package sonia.scm.lifecycle.modules; -import com.fasterxml.jackson.databind.ObjectMapper; import com.google.inject.Provider; import com.google.inject.multibindings.Multibinder; import com.google.inject.servlet.RequestScoped; @@ -31,7 +30,6 @@ import sonia.scm.PushStateDispatcherProvider; import sonia.scm.RootURL; import sonia.scm.Undecorated; import sonia.scm.admin.ScmConfigurationStore; -import sonia.scm.api.rest.ObjectMapperProvider; import sonia.scm.api.v2.resources.BranchLinkProvider; import sonia.scm.api.v2.resources.DefaultBranchLinkProvider; import sonia.scm.api.v2.resources.DefaultRepositoryLinkProvider; @@ -110,7 +108,7 @@ import sonia.scm.security.LoginAttemptHandler; import sonia.scm.security.RepositoryPermissionProvider; import sonia.scm.security.SecuritySystem; import sonia.scm.store.ConfigurationStoreDecoratorFactory; -import sonia.scm.store.FileStoreExporter; +import sonia.scm.store.file.FileStoreExporter; import sonia.scm.store.StoreExporter; import sonia.scm.template.MustacheTemplateEngine; import sonia.scm.template.TemplateEngine; @@ -258,7 +256,6 @@ class ScmServletModule extends ServletModule { bind(TemplateEngine.class).annotatedWith(Default.class).to( MustacheTemplateEngine.class); bind(TemplateEngineFactory.class); - bind(ObjectMapper.class).toProvider(ObjectMapperProvider.class); // bind events diff --git a/scm-webapp/src/main/java/sonia/scm/plugin/DefaultExtensionProcessor.java b/scm-webapp/src/main/java/sonia/scm/plugin/DefaultExtensionProcessor.java index b2090475bf..12ec40a3a8 100644 --- a/scm-webapp/src/main/java/sonia/scm/plugin/DefaultExtensionProcessor.java +++ b/scm-webapp/src/main/java/sonia/scm/plugin/DefaultExtensionProcessor.java @@ -82,9 +82,13 @@ public class DefaultExtensionProcessor implements ExtensionProcessor { return collector.getIndexedTypes(); } + @Override + public Iterable getQueryableTypes() { + return collector.getQueryableTypes(); + } + @Override public Iterable getConfigBindings() { return configBindings; } - } diff --git a/scm-webapp/src/main/java/sonia/scm/plugin/ExtensionCollector.java b/scm-webapp/src/main/java/sonia/scm/plugin/ExtensionCollector.java index 5c57f4a058..138ef60cd7 100644 --- a/scm-webapp/src/main/java/sonia/scm/plugin/ExtensionCollector.java +++ b/scm-webapp/src/main/java/sonia/scm/plugin/ExtensionCollector.java @@ -50,6 +50,7 @@ public final class ExtensionCollector { private final Set configElements = Sets.newHashSet(); private final Multimap extensions = HashMultimap.create(); private final Map extensionPointIndex = Maps.newHashMap(); + private final Set queryableTypes = Sets.newHashSet(); public ExtensionCollector(ClassLoader moduleClassLoader, Set modules, Set installedPlugins) { this.pluginIndex = createPluginIndex(installedPlugins); @@ -144,6 +145,10 @@ public final class ExtensionCollector { return indexedTypes; } + public Iterable getQueryableTypes() { + return queryableTypes; + } + private void appendExtension(Class extension) { boolean found = false; @@ -221,6 +226,16 @@ public final class ExtensionCollector { return true; } + private Collection collectQueryableTypes(ClassLoader defaultClassLoader, Iterable descriptors) { + Set queryableTypes = new HashSet<>(); + for (QueryableTypeDescriptor descriptor : descriptors) { + if (isRequirementFulfilled(descriptor)) { + queryableTypes.add(descriptor); + } + } + return queryableTypes; + } + private void collectRootElements(ClassLoader classLoader, ScmModule module) { for (ExtensionPointElement epe : module.getExtensionPoints()) { extensionPointIndex.put(epe.getClazz(), epe); @@ -233,5 +248,6 @@ public final class ExtensionCollector { webElements.addAll(collectWebElementExtensions(classLoader, module.getWebElements())); indexedTypes.addAll(collectIndexedTypes(classLoader, module.getIndexedTypes())); Iterables.addAll(configElements, module.getConfigElements()); + queryableTypes.addAll(collectQueryableTypes(classLoader, module.getQueryableTypes())); } } diff --git a/scm-webapp/src/main/java/sonia/scm/repository/RepositoryDeletionNotifier.java b/scm-webapp/src/main/java/sonia/scm/repository/RepositoryDeletionNotifier.java new file mode 100644 index 0000000000..85a4525161 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/repository/RepositoryDeletionNotifier.java @@ -0,0 +1,39 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.repository; + +import com.github.legman.ReferenceType; +import com.github.legman.Subscribe; +import sonia.scm.HandlerEventType; +import sonia.scm.plugin.Extension; +import sonia.scm.store.StoreDeletionNotifier; + +@Extension +class RepositoryDeletionNotifier implements StoreDeletionNotifier { + private DeletionHandler handler; + @Override + public void registerHandler(DeletionHandler handler) { + this.handler = handler; + } + + @Subscribe(referenceType = ReferenceType.STRONG) + public void onDelete(RepositoryEvent event) { + if (handler != null && event.getEventType() == HandlerEventType.DELETE) { + handler.notifyDeleted(Repository.class, event.getItem().getId()); + } + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/store/QueryableStoreDeletionHandler.java b/scm-webapp/src/main/java/sonia/scm/store/QueryableStoreDeletionHandler.java new file mode 100644 index 0000000000..c0082d2471 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/store/QueryableStoreDeletionHandler.java @@ -0,0 +1,51 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.store; + +import jakarta.inject.Inject; +import sonia.scm.EagerSingleton; +import sonia.scm.plugin.Extension; + +import java.util.Collection; +import java.util.Set; + +@Extension +@EagerSingleton +class QueryableStoreDeletionHandler implements StoreDeletionNotifier.DeletionHandler { + + private final StoreMetaDataProvider metaDataProvider; + private final QueryableStoreFactory storeFactory; + + @Inject + QueryableStoreDeletionHandler(Set notifiers, StoreMetaDataProvider metaDataProvider, QueryableStoreFactory storeFactory) { + this.metaDataProvider = metaDataProvider; + this.storeFactory = storeFactory; + notifiers.forEach(notifier -> notifier.registerHandler(this)); + } + + @Override + public void notifyDeleted(StoreDeletionNotifier.ClassWithId... classWithIds) { + Class[] classes = new Class[classWithIds.length]; + String[] ids = new String[classWithIds.length]; + for (int i = 0; i < classWithIds.length; i++) { + classes[i] = classWithIds[i].clazz(); + ids[i] = classWithIds[i].id(); + } + Collection> typesWithParent = metaDataProvider.getTypesWithParent(classes); + typesWithParent.forEach(type -> storeFactory.getForMaintenance(type, ids).clear()); + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/update/group/XmlGroupV1UpdateStep.java b/scm-webapp/src/main/java/sonia/scm/update/group/XmlGroupV1UpdateStep.java index c23333fcea..7db54d0966 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/group/XmlGroupV1UpdateStep.java +++ b/scm-webapp/src/main/java/sonia/scm/update/group/XmlGroupV1UpdateStep.java @@ -33,7 +33,7 @@ import sonia.scm.migration.UpdateStep; import sonia.scm.plugin.Extension; import sonia.scm.store.ConfigurationEntryStore; import sonia.scm.store.ConfigurationEntryStoreFactory; -import sonia.scm.store.StoreConstants; +import sonia.scm.store.file.StoreConstants; import sonia.scm.update.V1Properties; import sonia.scm.version.Version; diff --git a/scm-webapp/src/main/java/sonia/scm/update/index/RemoveCombinedIndex.java b/scm-webapp/src/main/java/sonia/scm/update/index/RemoveCombinedIndex.java index 1225b5cc0c..3a858fc856 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/index/RemoveCombinedIndex.java +++ b/scm-webapp/src/main/java/sonia/scm/update/index/RemoveCombinedIndex.java @@ -29,8 +29,8 @@ import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; -import static sonia.scm.store.StoreConstants.DATA_DIRECTORY_NAME; -import static sonia.scm.store.StoreConstants.VARIABLE_DATA_DIRECTORY_NAME; +import static sonia.scm.store.file.StoreConstants.DATA_DIRECTORY_NAME; +import static sonia.scm.store.file.StoreConstants.VARIABLE_DATA_DIRECTORY_NAME; @Extension public class RemoveCombinedIndex implements UpdateStep { diff --git a/scm-webapp/src/main/java/sonia/scm/update/repository/AnonymousModeUpdateStep.java b/scm-webapp/src/main/java/sonia/scm/update/repository/AnonymousModeUpdateStep.java index bb08bf5de5..bba18585e1 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/repository/AnonymousModeUpdateStep.java +++ b/scm-webapp/src/main/java/sonia/scm/update/repository/AnonymousModeUpdateStep.java @@ -31,7 +31,7 @@ import sonia.scm.plugin.Extension; import sonia.scm.security.AnonymousMode; import sonia.scm.store.ConfigurationStore; import sonia.scm.store.ConfigurationStoreFactory; -import sonia.scm.store.StoreConstants; +import sonia.scm.store.file.StoreConstants; import sonia.scm.version.Version; import java.nio.file.Path; diff --git a/scm-webapp/src/main/java/sonia/scm/update/repository/V1RepositoryHelper.java b/scm-webapp/src/main/java/sonia/scm/update/repository/V1RepositoryHelper.java index 765e63e4c1..149932e5be 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/repository/V1RepositoryHelper.java +++ b/scm-webapp/src/main/java/sonia/scm/update/repository/V1RepositoryHelper.java @@ -23,7 +23,7 @@ import jakarta.xml.bind.annotation.XmlAccessorType; import jakarta.xml.bind.annotation.XmlElement; import jakarta.xml.bind.annotation.XmlRootElement; import sonia.scm.SCMContextProvider; -import sonia.scm.store.StoreConstants; +import sonia.scm.store.file.StoreConstants; import java.io.File; import java.nio.file.Paths; diff --git a/scm-webapp/src/main/java/sonia/scm/update/repository/XmlRepositoryFileNameUpdateStep.java b/scm-webapp/src/main/java/sonia/scm/update/repository/XmlRepositoryFileNameUpdateStep.java index 8cdd376c31..a3367464aa 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/repository/XmlRepositoryFileNameUpdateStep.java +++ b/scm-webapp/src/main/java/sonia/scm/update/repository/XmlRepositoryFileNameUpdateStep.java @@ -24,7 +24,7 @@ import sonia.scm.migration.UpdateStep; import sonia.scm.plugin.Extension; import sonia.scm.repository.xml.PathBasedRepositoryLocationResolver; import sonia.scm.repository.xml.XmlRepositoryDAO; -import sonia.scm.store.StoreConstants; +import sonia.scm.store.file.StoreConstants; import sonia.scm.version.Version; import java.io.IOException; diff --git a/scm-webapp/src/main/java/sonia/scm/update/repository/XmlRepositoryV1UpdateStep.java b/scm-webapp/src/main/java/sonia/scm/update/repository/XmlRepositoryV1UpdateStep.java index 5cafb26094..4a9aaf1322 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/repository/XmlRepositoryV1UpdateStep.java +++ b/scm-webapp/src/main/java/sonia/scm/update/repository/XmlRepositoryV1UpdateStep.java @@ -29,7 +29,7 @@ import sonia.scm.repository.RepositoryPermission; import sonia.scm.repository.xml.XmlRepositoryDAO; import sonia.scm.store.ConfigurationEntryStore; import sonia.scm.store.ConfigurationEntryStoreFactory; -import sonia.scm.store.StoreConstants; +import sonia.scm.store.file.StoreConstants; import sonia.scm.update.CoreUpdateStep; import sonia.scm.update.V1Properties; import sonia.scm.version.Version; diff --git a/scm-webapp/src/main/java/sonia/scm/update/security/XmlSecurityV1UpdateStep.java b/scm-webapp/src/main/java/sonia/scm/update/security/XmlSecurityV1UpdateStep.java index 7f47d31e80..05d3474af1 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/security/XmlSecurityV1UpdateStep.java +++ b/scm-webapp/src/main/java/sonia/scm/update/security/XmlSecurityV1UpdateStep.java @@ -31,7 +31,7 @@ import sonia.scm.plugin.Extension; import sonia.scm.security.AssignedPermission; import sonia.scm.store.ConfigurationEntryStore; import sonia.scm.store.ConfigurationEntryStoreFactory; -import sonia.scm.store.StoreConstants; +import sonia.scm.store.file.StoreConstants; import sonia.scm.version.Version; import java.io.File; diff --git a/scm-webapp/src/main/java/sonia/scm/update/store/DifferentiateBetweenConfigAndConfigEntryUpdateStep.java b/scm-webapp/src/main/java/sonia/scm/update/store/DifferentiateBetweenConfigAndConfigEntryUpdateStep.java index c2e4ab7a57..5370672bdb 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/store/DifferentiateBetweenConfigAndConfigEntryUpdateStep.java +++ b/scm-webapp/src/main/java/sonia/scm/update/store/DifferentiateBetweenConfigAndConfigEntryUpdateStep.java @@ -21,7 +21,7 @@ import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.xml.sax.SAXException; import sonia.scm.migration.UpdateException; -import sonia.scm.store.CopyOnWrite; +import sonia.scm.CopyOnWrite; import sonia.scm.version.Version; import sonia.scm.xml.XmlStreams; import sonia.scm.xml.XmlStreams.AutoCloseableXMLReader; @@ -41,7 +41,7 @@ import java.nio.file.Files; import java.nio.file.Path; import java.util.stream.Stream; -import static sonia.scm.store.CopyOnWrite.compute; +import static sonia.scm.CopyOnWrite.compute; abstract class DifferentiateBetweenConfigAndConfigEntryUpdateStep { diff --git a/scm-webapp/src/main/java/sonia/scm/update/user/XmlUserV1UpdateStep.java b/scm-webapp/src/main/java/sonia/scm/update/user/XmlUserV1UpdateStep.java index 91b4417a99..92030fa929 100644 --- a/scm-webapp/src/main/java/sonia/scm/update/user/XmlUserV1UpdateStep.java +++ b/scm-webapp/src/main/java/sonia/scm/update/user/XmlUserV1UpdateStep.java @@ -32,7 +32,7 @@ import sonia.scm.plugin.Extension; import sonia.scm.security.AssignedPermission; import sonia.scm.store.ConfigurationEntryStore; import sonia.scm.store.ConfigurationEntryStoreFactory; -import sonia.scm.store.StoreConstants; +import sonia.scm.store.file.StoreConstants; import sonia.scm.update.V1Properties; import sonia.scm.user.User; import sonia.scm.user.xml.XmlUserDAO; diff --git a/scm-webapp/src/main/java/sonia/scm/user/UserDeletionNotifier.java b/scm-webapp/src/main/java/sonia/scm/user/UserDeletionNotifier.java new file mode 100644 index 0000000000..d04eb8faa9 --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/user/UserDeletionNotifier.java @@ -0,0 +1,40 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.user; + +import com.github.legman.ReferenceType; +import com.github.legman.Subscribe; +import sonia.scm.HandlerEventType; +import sonia.scm.plugin.Extension; +import sonia.scm.store.StoreDeletionNotifier; + +@Extension +public class UserDeletionNotifier implements StoreDeletionNotifier { + private DeletionHandler handler; + + @Override + public void registerHandler(DeletionHandler handler) { + this.handler = handler; + } + + @Subscribe(referenceType = ReferenceType.STRONG) + public void onDelete(UserEvent event) { + if (handler != null && event.getEventType() == HandlerEventType.DELETE) { + handler.notifyDeleted(User.class, event.getItem().getId()); + } + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryExporterTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryExporterTest.java index bb333a6d2e..e8eb93cedc 100644 --- a/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryExporterTest.java +++ b/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryExporterTest.java @@ -16,6 +16,7 @@ package sonia.scm.importexport; +import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; @@ -36,6 +37,7 @@ import sonia.scm.web.security.PrivilegedAction; import java.io.ByteArrayOutputStream; import java.io.File; +import java.io.FileWriter; import java.io.IOException; import java.io.OutputStream; import java.nio.file.Files; @@ -48,6 +50,7 @@ import static org.assertj.core.api.Assertions.assertThat; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.anyString; +import static org.mockito.Mockito.doAnswer; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; @@ -76,6 +79,8 @@ class FullScmRepositoryExporterTest { private AdministrationContext administrationContext; @Mock private RepositoryImportExportEncryption repositoryImportExportEncryption; + @Mock + private RepositoryQueryableStoreExporter queryableStoreExporter; @InjectMocks private FullScmRepositoryExporter exporter; @@ -88,10 +93,23 @@ class FullScmRepositoryExporterTest { when(metadataGenerator.generate(REPOSITORY)).thenReturn(new byte[0]); when(repositoryExportingCheck.withExportingLock(any(), any())).thenAnswer(invocation -> invocation.getArgument(1, Supplier.class).get()); when(repositoryImportExportEncryption.optionallyEncrypt(any(), any())).thenAnswer(invocation -> invocation.getArgument(0)); + + doAnswer(invocation -> { + File directory = invocation.getArgument(1, File.class); + File dummyFile = new File(directory, "dummy.xml"); + try (FileWriter writer = new FileWriter(dummyFile)) { + writer.write("Dummy content for testing"); + } + return null; + }).when(queryableStoreExporter).addQueryableStoreDataToArchive( + any(Repository.class), + any(File.class), + any(TarArchiveOutputStream.class) + ); } @Test - void shouldExportEverythingAsTarArchive(@TempDir Path temp) { + void shouldExportEverythingAsTarArchive(@TempDir Path temp) throws IOException { BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class); when(repositoryService.getBundleCommand()).thenReturn(bundleCommandBuilder); when(repositoryService.getRepository()).thenReturn(REPOSITORY); @@ -104,6 +122,8 @@ class FullScmRepositoryExporterTest { verify(metadataGenerator, times(1)).generate(REPOSITORY); verify(bundleCommandBuilder, times(1)).bundle(any(OutputStream.class)); verify(repositoryExportingCheck).withExportingLock(eq(REPOSITORY), any()); + verify(queryableStoreExporter, times(1)) + .addQueryableStoreDataToArchive(eq(REPOSITORY), any(File.class), any(TarArchiveOutputStream.class)); workDirsCreated.forEach(wd -> assertThat(wd).doesNotExist()); } diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryImporterTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryImporterTest.java index eecc497121..123b4fbefe 100644 --- a/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryImporterTest.java +++ b/scm-webapp/src/test/java/sonia/scm/importexport/FullScmRepositoryImporterTest.java @@ -32,6 +32,7 @@ import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; import sonia.scm.event.ScmEventBus; +import sonia.scm.repository.ClearRepositoryCacheEvent; import sonia.scm.repository.ImportRepositoryHookEvent; import sonia.scm.repository.Repository; import sonia.scm.repository.RepositoryHookEvent; @@ -62,7 +63,6 @@ import static org.junit.jupiter.api.Assertions.assertThrows; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.argThat; import static org.mockito.ArgumentMatchers.eq; -import static org.mockito.Mockito.doAnswer; import static org.mockito.Mockito.doNothing; import static org.mockito.Mockito.doThrow; import static org.mockito.Mockito.lenient; @@ -114,6 +114,8 @@ class FullScmRepositoryImporterTest { private StoreImportStep storeImportStep; @InjectMocks private RepositoryImportStep repositoryImportStep; + @InjectMocks + private QueryableStoreImportStep queryableStoreImportStep; @Mock private RepositoryHookEvent event; @@ -129,11 +131,13 @@ class FullScmRepositoryImporterTest { environmentCheckStep, metadataImportStep, storeImportStep, + queryableStoreImportStep, repositoryImportStep, repositoryManager, repositoryImportExportEncryption, loggerFactory, - eventBus); + eventBus, + updateEngine); } @BeforeEach @@ -256,17 +260,30 @@ class FullScmRepositoryImporterTest { fullImporter.importFromStream(REPOSITORY, stream, null); - assertThat(capturedEvents.getAllValues()).hasSize(2); - assertThat(capturedEvents.getAllValues()).anyMatch( - event -> - event instanceof ImportRepositoryHookEvent && - ((ImportRepositoryHookEvent) event).getRepository().equals(REPOSITORY) - ); - assertThat(capturedEvents.getAllValues()).anyMatch( - event -> - event instanceof RepositoryImportEvent && - ((RepositoryImportEvent) event).getItem().equals(REPOSITORY) - ); + assertThat(capturedEvents.getAllValues()).hasSize(4); + assertThat(capturedEvents.getAllValues()) + .satisfiesExactlyInAnyOrder( + event -> + assertThat(event) + .isInstanceOf(ClearRepositoryCacheEvent.class) + .extracting("repository") + .isEqualTo(REPOSITORY), + event -> + assertThat(event) + .isInstanceOf(ClearRepositoryCacheEvent.class) + .extracting("repository") + .isEqualTo(REPOSITORY), + event -> + assertThat(event) + .isInstanceOf(RepositoryImportEvent.class) + .extracting("item") + .isEqualTo(REPOSITORY), + event -> + assertThat(event) + .isInstanceOf(ImportRepositoryHookEvent.class) + .extracting("repository") + .isEqualTo(REPOSITORY) + ); } @Test diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/QueryableStoreImportStepTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/QueryableStoreImportStepTest.java new file mode 100644 index 0000000000..78859e96ad --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/QueryableStoreImportStepTest.java @@ -0,0 +1,103 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import com.google.common.io.Resources; +import org.apache.commons.compress.archivers.tar.TarArchiveEntry; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.Repository; +import sonia.scm.repository.RepositoryLocationResolver; + +import java.io.File; +import java.io.FileInputStream; +import java.io.InputStream; +import java.nio.file.Path; + +import static org.assertj.core.api.AssertionsForClassTypes.assertThat; +import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.doNothing; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class QueryableStoreImportStepTest { + private static final String QUERYABLE_STORE_DATA_FILE_NAME = "queryable-store-data.tar"; + + @Mock + private RepositoryQueryableStoreExporter queryableStoreExporter; + @Mock + private ImportState importState; + @Mock + private RepositoryImportLogger logger; + @Mock + private Repository repository; + @Mock + private RepositoryLocationResolver locationResolver; + @Mock + private RepositoryLocationResolver.RepositoryLocationResolverInstance forClass; + + @InjectMocks + private QueryableStoreImportStep queryableStoreImportStep; + + private File tarFile; + @TempDir + private File tempWorkDir; + + @BeforeEach + void setUp() { + when(importState.getRepository()).thenReturn(repository); + when(importState.getLogger()).thenReturn(logger); + when(repository.getId()).thenReturn("42"); + doNothing().when(logger).step(anyString()); + + when(locationResolver.forClass(Path.class)).thenReturn(forClass); + when(forClass.getLocation(anyString())).thenReturn(tempWorkDir.toPath()); + + tarFile = new File(Resources.getResource("sonia/scm/importexport/queryable-store-data.tar").getFile()); + } + + @Test + void shouldHandleQueryableStoreTarFileCorrectly() throws Exception { + TarArchiveEntry entry = new TarArchiveEntry(tarFile, QUERYABLE_STORE_DATA_FILE_NAME); + entry.setSize(tarFile.length()); + + doAnswer( + invocation -> { + assertThat(tempWorkDir.listFiles()) + .containsExactlyInAnyOrder( + new File(tempWorkDir, "sonia.scm.importexport.SimpleType.xml"), + new File(tempWorkDir, "sonia.scm.importexport.SimpleTypeWithTwoParents.xml") + ); + return null; + } + ).when(queryableStoreExporter).importStores("42", tempWorkDir); + + try (InputStream inputStream = new FileInputStream(tarFile)) { + boolean result = queryableStoreImportStep.handle(entry, importState, inputStream); + assertThat(result).isTrue(); + + verify(queryableStoreExporter).importStores("42", tempWorkDir); + } + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/RemainingQueryableStoreImporterTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/RemainingQueryableStoreImporterTest.java new file mode 100644 index 0000000000..48812c3969 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/RemainingQueryableStoreImporterTest.java @@ -0,0 +1,108 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.RepositoryLocationResolver.RepositoryLocationResolverInstance; +import sonia.scm.repository.xml.PathBasedRepositoryLocationResolver; + +import java.io.File; +import java.io.FileWriter; +import java.io.IOException; +import java.nio.file.Path; +import java.util.function.BiConsumer; + +import static org.mockito.Mockito.any; +import static org.mockito.Mockito.anyString; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class RemainingQueryableStoreImporterTest { + + @TempDir + private File tempDir; + + @Mock + private PathBasedRepositoryLocationResolver repositoryLocationResolver; + @Mock + private RepositoryLocationResolverInstance repositoryLocationResolverInstance; + @Mock + private RepositoryQueryableStoreExporter queryableStoreExporter; + + private RemainingQueryableStoreImporter listener; + + @BeforeEach + void setUp() throws IOException { + when(repositoryLocationResolver.create(Path.class)).thenReturn(repositoryLocationResolverInstance); + + listener = new RemainingQueryableStoreImporter(repositoryLocationResolver, queryableStoreExporter); + + File queryableStoreDir = new File(tempDir, "queryable-store-data"); + queryableStoreDir.mkdirs(); + + createXmlFile(new File(queryableStoreDir, "sonia.scm.importexport.SimpleType.xml")); + createXmlFile(new File(queryableStoreDir, "sonia.scm.importexport.SimpleTypeWithTwoParents.xml")); + } + + @Test + void shouldImportXmlFilesIfExist() { + Path repoPath = tempDir.toPath(); + + doAnswer(invocation -> { + @SuppressWarnings("unchecked") + BiConsumer consumer = invocation.getArgument(0); + consumer.accept("test-repo", repoPath); + return null; + }).when(repositoryLocationResolverInstance).forAllLocations(any()); + + listener.onInitializationCompleted(); + + verify(queryableStoreExporter).importStores("test-repo", tempDir); + } + + @Test + void shouldNotImportIfNoXmlFiles() { + File emptyRepoDir = new File(tempDir, "empty-repo"); + emptyRepoDir.mkdirs(); + Path emptyRepoPath = emptyRepoDir.toPath(); + + doAnswer(invocation -> { + @SuppressWarnings("unchecked") + BiConsumer consumer = invocation.getArgument(0); + consumer.accept("empty-repo", emptyRepoPath); + return null; + }).when(repositoryLocationResolverInstance).forAllLocations(any()); + + listener.onInitializationCompleted(); + + verify(queryableStoreExporter, never()).importStores(anyString(), any(File.class)); + } + + private void createXmlFile(File file) throws IOException { + try (FileWriter writer = new FileWriter(file)) { + writer.write("data"); + } + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/RepositoryQueryableStoreExporterTest.java b/scm-webapp/src/test/java/sonia/scm/importexport/RepositoryQueryableStoreExporterTest.java new file mode 100644 index 0000000000..fd75182e12 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/RepositoryQueryableStoreExporterTest.java @@ -0,0 +1,164 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import com.google.common.io.Resources; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableStoreExtension; +import sonia.scm.store.QueryableStoreFactory; +import sonia.scm.store.StoreMetaDataProvider; + +import java.io.File; +import java.io.IOException; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.mockito.Mockito.lenient; + +@ExtendWith({QueryableStoreExtension.class, MockitoExtension.class}) +@QueryableStoreExtension.QueryableTypes({SimpleType.class, SimpleTypeWithTwoParents.class}) +class RepositoryQueryableStoreExporterTest { + + @Mock + private StoreMetaDataProvider storeMetaDataProvider; + + @BeforeEach + void initMetaDataProvider() { + lenient().when(storeMetaDataProvider.getTypesWithParent(Repository.class)).thenReturn(List.of(SimpleType.class, SimpleTypeWithTwoParents.class)); + } + + @Nested + class ExportStores { + @Test + void shouldExportSimpleType(QueryableStoreFactory storeFactory, SimpleTypeStoreFactory simpleTypeStoreFactory, @TempDir java.nio.file.Path tempDir) { + simpleTypeStoreFactory.getMutable("23").put("1", new SimpleType("hack")); + simpleTypeStoreFactory.getMutable("42").put("1", new SimpleType("hitchhike")); + simpleTypeStoreFactory.getMutable("42").put("2", new SimpleType("heart of gold")); + + RepositoryQueryableStoreExporter exporter = new RepositoryQueryableStoreExporter(storeMetaDataProvider, storeFactory); + + exporter.exportStores("42", tempDir.toFile()); + + assertThat(tempDir).isNotEmptyDirectory(); + } + + @Test + void shouldExportTypeWithTwoParents(QueryableStoreFactory storeFactory, SimpleTypeWithTwoParentsStoreFactory simpleTypeStoreFactory, @TempDir java.nio.file.Path tempDir) { + simpleTypeStoreFactory.getMutable("23", "1").put("1", new SimpleTypeWithTwoParents("hack")); + simpleTypeStoreFactory.getMutable("42", "1").put("1", new SimpleTypeWithTwoParents("hitchhike")); + simpleTypeStoreFactory.getMutable("42", "1").put("2", new SimpleTypeWithTwoParents("heart of gold")); + + RepositoryQueryableStoreExporter exporter = new RepositoryQueryableStoreExporter(storeMetaDataProvider, storeFactory); + + exporter.exportStores("42", tempDir.toFile()); + + assertThat(tempDir).isNotEmptyDirectory(); + } + } + + @Nested + class ImportStores { + + private File queryableStoreDir; + + @TempDir + private File tempDir; + + @BeforeEach + void prepareImportDirectory() throws IOException { + queryableStoreDir = new File(tempDir, "queryable-store-data"); + Files.createDirectories(queryableStoreDir.toPath()); + } + + @Test + void shouldImportSimpleType(QueryableStoreFactory storeFactory, SimpleTypeStoreFactory simpleTypeStoreFactory) throws IOException { + simpleTypeStoreFactory.getMutable("23").put("1", new SimpleType("hack")); + URL url = Resources.getResource("sonia/scm/importexport/SimpleType.xml"); + + Files.createFile(queryableStoreDir.toPath().resolve("sonia.scm.importexport.SimpleType.xml")); + Files.writeString(queryableStoreDir.toPath().resolve("sonia.scm.importexport.SimpleType.xml"), Resources.toString(url, StandardCharsets.UTF_8)); + + RepositoryQueryableStoreExporter exporter = new RepositoryQueryableStoreExporter(storeMetaDataProvider, storeFactory); + + exporter.importStores("42", tempDir); + + assertThat(simpleTypeStoreFactory.getMutable("42").getAll()).hasSize(2); + } + + @Test + void shouldImportTypeWithTwoParents(QueryableStoreFactory storeFactory, SimpleTypeWithTwoParentsStoreFactory simpleTypeStoreFactory) throws IOException { + simpleTypeStoreFactory.getMutable("23", "1").put("1", new SimpleTypeWithTwoParents("hack")); + URL url = Resources.getResource("sonia/scm/importexport/SimpleTypeWithTwoParents.xml"); + Files.writeString(queryableStoreDir.toPath().resolve("sonia.scm.importexport.SimpleTypeWithTwoParents.xml"), Resources.toString(url, StandardCharsets.UTF_8)); + + RepositoryQueryableStoreExporter exporter = new RepositoryQueryableStoreExporter(storeMetaDataProvider, storeFactory); + exporter.importStores("42", tempDir); + + assertThat(simpleTypeStoreFactory.getMutable("42", "1").getAll()).hasSize(2); + } + + @Test + void shouldNotImportWhenFileDoesNotExist(QueryableStoreFactory storeFactory, SimpleTypeStoreFactory simpleTypeStoreFactory) { + simpleTypeStoreFactory.getMutable("23").put("1", new SimpleType("hack")); + + File nonExistentFile = queryableStoreDir.toPath().resolve("sonia.scm.importexport.SimpleType.xml").toFile(); + assertThat(nonExistentFile).doesNotExist(); + + RepositoryQueryableStoreExporter exporter = new RepositoryQueryableStoreExporter(storeMetaDataProvider, storeFactory); + exporter.importStores("42", tempDir); + + assertThat(simpleTypeStoreFactory.getMutable("42").getAll()).isEmpty(); + } + + @Test + void shouldThrowExceptionForMalformedXML(QueryableStoreFactory storeFactory) throws IOException { + Files.writeString(queryableStoreDir.toPath().resolve("sonia.scm.importexport.SimpleType.xml"), ""); + + RepositoryQueryableStoreExporter exporter = new RepositoryQueryableStoreExporter(storeMetaDataProvider, storeFactory); + + assertThrows(RuntimeException.class, () -> exporter.importStores("42", tempDir)); + } + + @Test + void shouldNotImportFromEmptyFile(QueryableStoreFactory storeFactory, SimpleTypeStoreFactory simpleTypeStoreFactory) throws IOException { + simpleTypeStoreFactory.getMutable("42").put("1", new SimpleType("existing data")); + + Files.createFile(queryableStoreDir.toPath().resolve("sonia.scm.importexport.SimpleType.xml")); + + RepositoryQueryableStoreExporter exporter = new RepositoryQueryableStoreExporter(storeMetaDataProvider, storeFactory); + exporter.importStores("42", tempDir); + + SimpleType simpleType = simpleTypeStoreFactory.getMutable("42").get("1"); + + assertThat(simpleType) + .extracting("someField") + .isEqualTo("existing data"); + } + } +} + diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/SimpleType.java b/scm-webapp/src/test/java/sonia/scm/importexport/SimpleType.java new file mode 100644 index 0000000000..a93b4756f0 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/SimpleType.java @@ -0,0 +1,31 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.NoArgsConstructor; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableType; + +@Data +@NoArgsConstructor +@AllArgsConstructor +@QueryableType(Repository.class) +class SimpleType { + private String someField; +} diff --git a/scm-webapp/src/test/java/sonia/scm/importexport/SimpleTypeWithTwoParents.java b/scm-webapp/src/test/java/sonia/scm/importexport/SimpleTypeWithTwoParents.java new file mode 100644 index 0000000000..5ce85c5336 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/importexport/SimpleTypeWithTwoParents.java @@ -0,0 +1,31 @@ +/* + * Copyright (c) 2020 - present Cloudogu GmbH + * + * This program is free software: you can redistribute it and/or modify it under + * the terms of the GNU Affero General Public License as published by the Free + * Software Foundation, version 3. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS + * FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more + * details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see https://www.gnu.org/licenses/. + */ + +package sonia.scm.importexport; + +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.NoArgsConstructor; +import sonia.scm.repository.Repository; +import sonia.scm.store.QueryableType; + +@Data +@NoArgsConstructor +@AllArgsConstructor +@QueryableType({Repository.class, SimpleType.class}) +class SimpleTypeWithTwoParents { + private String someField; +} diff --git a/scm-webapp/src/test/java/sonia/scm/security/DefaultSecuritySystemTest.java b/scm-webapp/src/test/java/sonia/scm/security/DefaultSecuritySystemTest.java index 27fbdf2cea..2e4d0e6cff 100644 --- a/scm-webapp/src/test/java/sonia/scm/security/DefaultSecuritySystemTest.java +++ b/scm-webapp/src/test/java/sonia/scm/security/DefaultSecuritySystemTest.java @@ -27,8 +27,9 @@ import org.mockito.MockitoAnnotations; import sonia.scm.AbstractTestBase; import sonia.scm.auditlog.Auditor; import sonia.scm.plugin.PluginLoader; -import sonia.scm.store.JAXBConfigurationEntryStoreFactory; -import sonia.scm.store.StoreCacheConfigProvider; +import sonia.scm.store.file.JAXBConfigurationEntryStoreFactory; +import sonia.scm.store.file.StoreCacheConfigProvider; +import sonia.scm.store.file.StoreCacheFactory; import sonia.scm.util.ClassLoaders; import sonia.scm.util.MockUtil; @@ -60,7 +61,7 @@ public class DefaultSecuritySystemTest extends AbstractTestBase public void createSecuritySystem() { jaxbConfigurationEntryStoreFactory = - spy(new JAXBConfigurationEntryStoreFactory(contextProvider , repositoryLocationResolver, new UUIDKeyGenerator(), null, new StoreCacheConfigProvider(false)) {}); + spy(new JAXBConfigurationEntryStoreFactory(contextProvider , repositoryLocationResolver, new UUIDKeyGenerator(), null, new StoreCacheFactory(new StoreCacheConfigProvider(false))) {}); pluginLoader = mock(PluginLoader.class); when(pluginLoader.getUberClassLoader()).thenReturn(ClassLoaders.getContextClassLoader(DefaultSecuritySystem.class)); diff --git a/scm-webapp/src/test/java/sonia/scm/update/repository/DefaultMigrationStrategyDAOTest.java b/scm-webapp/src/test/java/sonia/scm/update/repository/DefaultMigrationStrategyDAOTest.java index 4a92d88752..f5bb861484 100644 --- a/scm-webapp/src/test/java/sonia/scm/update/repository/DefaultMigrationStrategyDAOTest.java +++ b/scm-webapp/src/test/java/sonia/scm/update/repository/DefaultMigrationStrategyDAOTest.java @@ -26,8 +26,9 @@ import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; import sonia.scm.SCMContextProvider; import sonia.scm.store.ConfigurationStoreFactory; -import sonia.scm.store.JAXBConfigurationStoreFactory; -import sonia.scm.store.StoreCacheConfigProvider; +import sonia.scm.store.file.JAXBConfigurationStoreFactory; +import sonia.scm.store.file.StoreCacheConfigProvider; +import sonia.scm.store.file.StoreCacheFactory; import java.nio.file.Path; import java.util.Optional; @@ -47,7 +48,7 @@ class DefaultMigrationStrategyDAOTest { @BeforeEach void initStore(@TempDir Path tempDir) { when(contextProvider.getBaseDirectory()).thenReturn(tempDir.toFile()); - storeFactory = new JAXBConfigurationStoreFactory(contextProvider, null, null, emptySet(), new StoreCacheConfigProvider(false)); + storeFactory = new JAXBConfigurationStoreFactory(contextProvider, null, null, emptySet(), new StoreCacheFactory(new StoreCacheConfigProvider(false))); } @Test diff --git a/scm-webapp/src/test/java/sonia/scm/user/DefaultUserManagerTest.java b/scm-webapp/src/test/java/sonia/scm/user/DefaultUserManagerTest.java index 82208d0d3f..14805cfcf8 100644 --- a/scm-webapp/src/test/java/sonia/scm/user/DefaultUserManagerTest.java +++ b/scm-webapp/src/test/java/sonia/scm/user/DefaultUserManagerTest.java @@ -25,8 +25,9 @@ import org.junit.Rule; import org.junit.Test; import org.mockito.ArgumentCaptor; import sonia.scm.NotFoundException; -import sonia.scm.store.JAXBConfigurationStoreFactory; -import sonia.scm.store.StoreCacheConfigProvider; +import sonia.scm.store.file.JAXBConfigurationStoreFactory; +import sonia.scm.store.file.StoreCacheConfigProvider; +import sonia.scm.store.file.StoreCacheFactory; import sonia.scm.user.xml.XmlUserDAO; import static java.util.Collections.emptySet; @@ -153,6 +154,6 @@ public class DefaultUserManagerTest extends UserManagerTestBase { } private XmlUserDAO createXmlUserDAO() { - return new XmlUserDAO(new JAXBConfigurationStoreFactory(contextProvider, locationResolver, null, emptySet(), new StoreCacheConfigProvider(false))); + return new XmlUserDAO(new JAXBConfigurationStoreFactory(contextProvider, locationResolver, null, emptySet(), new StoreCacheFactory(new StoreCacheConfigProvider(false)))); } } diff --git a/scm-webapp/src/test/resources/sonia/scm/importexport/SimpleType.xml b/scm-webapp/src/test/resources/sonia/scm/importexport/SimpleType.xml new file mode 100644 index 0000000000..3e74fcd727 --- /dev/null +++ b/scm-webapp/src/test/resources/sonia/scm/importexport/SimpleType.xml @@ -0,0 +1,30 @@ + + + + + sonia.scm.importexport.SimpleType + + 42 + 1 + {"someField":"hitchhike"} + + + 42 + 2 + {"someField":"heart of gold"} + + diff --git a/scm-webapp/src/test/resources/sonia/scm/importexport/SimpleTypeWithTwoParents.xml b/scm-webapp/src/test/resources/sonia/scm/importexport/SimpleTypeWithTwoParents.xml new file mode 100644 index 0000000000..cfb882067f --- /dev/null +++ b/scm-webapp/src/test/resources/sonia/scm/importexport/SimpleTypeWithTwoParents.xml @@ -0,0 +1,32 @@ + + + + + sonia.scm.importexport.SimpleTypeWithTwoParents + + 42 + 1 + 1 + {"someField":"hitchhike"} + + + 42 + 1 + 2 + {"someField":"heart of gold"} + + diff --git a/scm-webapp/src/test/resources/sonia/scm/importexport/queryable-store-data.tar b/scm-webapp/src/test/resources/sonia/scm/importexport/queryable-store-data.tar new file mode 100644 index 0000000000..880905e0d4 Binary files /dev/null and b/scm-webapp/src/test/resources/sonia/scm/importexport/queryable-store-data.tar differ diff --git a/settings.gradle b/settings.gradle index 8798244476..79a449545e 100644 --- a/settings.gradle +++ b/settings.gradle @@ -48,14 +48,16 @@ includeBuild 'build-plugins' include 'scm-annotations' include 'scm-annotation-processor' include 'scm-core' +include 'scm-core-annotation-processor' include 'scm-test' +include 'scm-queryable-test' include 'scm-ui' include 'scm-plugins:scm-git-plugin' include 'scm-plugins:scm-hg-plugin' include 'scm-plugins:scm-svn-plugin' include 'scm-plugins:scm-legacy-plugin' include 'scm-plugins:scm-integration-test-plugin' -include 'scm-dao-xml' +include 'scm-persistence' include 'scm-webapp' include 'scm-server' include 'scm-it'