diff --git a/docs/de/user/admin/assets/cache_invalidation.png b/docs/de/user/admin/assets/cache_invalidation.png new file mode 100644 index 0000000000..f252eb1692 Binary files /dev/null and b/docs/de/user/admin/assets/cache_invalidation.png differ diff --git a/docs/de/user/admin/assets/rebuild_index.png b/docs/de/user/admin/assets/rebuild_index.png new file mode 100644 index 0000000000..bd8a7f638e Binary files /dev/null and b/docs/de/user/admin/assets/rebuild_index.png differ diff --git a/docs/de/user/admin/index.md b/docs/de/user/admin/index.md index fd4e75c3f4..cda00b99a4 100644 --- a/docs/de/user/admin/index.md +++ b/docs/de/user/admin/index.md @@ -9,6 +9,7 @@ Im Bereich Administration kann die SCM-Manager Instanz administriert werden. Von * [Berechtigungsrollen](roles/) * [Einstellungen](settings/) * [Git](git/) +* [Fehlerbehebung](troubleshooting/) ### Information diff --git a/docs/de/user/admin/troubleshooting.md b/docs/de/user/admin/troubleshooting.md new file mode 100644 index 0000000000..927258dcf9 --- /dev/null +++ b/docs/de/user/admin/troubleshooting.md @@ -0,0 +1,30 @@ +--- +title: Fehlerbehebung +--- + +## Caches invalidieren + +Um die Performance des SCM-Managers zu verbessern, werden viele Daten zusätzlich als Cache im Arbeitsspeicher gehalten. +Es kann passieren, dass die Daten im Cache nicht invalidiert werden, obwohl sich die zugrundeliegenden Daten geändert +haben. Dies kann zu Fehlern führen, z. B. könnten manche Ansichten versuchen ein Repository zu laden, welches bereits +gelöscht wurde. Um dieses Problem manuell zu lösen, können Administratoren den internen Cache des SCM-Managers +invalidieren. Allerdings kann diese Operation den SCM-Manager für eine Zeit verlangsamen. Dementsprechend sollte diese +Operation nur bedacht genutzt werden. + +Die Option zur Invalidierung findet sich in den generellen Einstellungen: + +![Screenshot der generellen Einstellungen für die Cache Invalidierung](assets/cache_invalidation.png) + +## Suchindex neu aufbauen + +Unter hoher Server-Last kann es passieren, dass der Suchindex nicht korrekt invalidiert wird, obwohl sich die +zugrundeliegenden Daten geändert haben. Dementsprechend kann es passieren, dass veraltete Daten gefunden werden. Dies +kann zu Fehlern in der Suchkomponente führen. Um dieses Problem manuell zu lösen, können Administratoren den Suchindex +neu erstellen lassen. Allerdings ist diese Operation zeitaufwändig und könnte den SCM-Manager für eine Zeit +verlangsamen. Dementsprechend sollte diese Operation nur bedacht genutzt werden. Wenn die Probleme bei der Suche nur ein +Repository betrifft, dann sollten Administratoren stattdessen nur den Suchindex für dieses Repository neu aufbauen +lassen. Dies kann in den generellen Einstellungen des Repositories gemacht werden. + +Die Option zum Neuaufbau findet sich in den generellen Einstellungen: + +![Screenshot der generellen Einstellungen für das erneute Aufbauen des Suchindex](assets/rebuild_index.png) diff --git a/docs/en/user/admin/assets/cache_invalidation.png b/docs/en/user/admin/assets/cache_invalidation.png new file mode 100644 index 0000000000..006416e3e0 Binary files /dev/null and b/docs/en/user/admin/assets/cache_invalidation.png differ diff --git a/docs/en/user/admin/assets/rebuild_index.png b/docs/en/user/admin/assets/rebuild_index.png new file mode 100644 index 0000000000..5b54cc6953 Binary files /dev/null and b/docs/en/user/admin/assets/rebuild_index.png differ diff --git a/docs/en/user/admin/index.md b/docs/en/user/admin/index.md index af1b9fd116..3bca1872b0 100644 --- a/docs/en/user/admin/index.md +++ b/docs/en/user/admin/index.md @@ -8,6 +8,7 @@ The SCM-Manager instance can be administered in the Administration area. From he * [Permission Roles](roles/) * [Settings](settings/) * [Git](git/) +* [Troubleshooting](troubleshooting/) ### Information On the information page in the administration area you can find the version of your SCM-Manager instance and helpful links to get in touch with the SCM-Manager support team. If there is a newer version for SCM-Manager, it will be shown with the link to the download section on the official SCM-Manager homepage. diff --git a/docs/en/user/admin/troubleshooting.md b/docs/en/user/admin/troubleshooting.md new file mode 100644 index 0000000000..f3e832c2b7 --- /dev/null +++ b/docs/en/user/admin/troubleshooting.md @@ -0,0 +1,28 @@ +--- +title: Fehlerbehebung +--- + +## Invalidate Caches + +To improve the performance of the SCM-Manager, many data is additionally kept as cache in the main memory. It can happen +that the data in the cache is not invalidated, although the underlying data has changed. This can lead to errors, e.g. +some views could try to load a repository that has already been deleted. To solve this problem manually, administrators +can invalidate the internal cache of the SCM-Manager. However, this operation can slow down the SCM-Manager for a while. +Accordingly, this operation should only be used with caution. + +The option for invalidation can be found in the general settings: + +![Screenshot of the general settings for cache invalidation](assets/cache_invalidation.png) + +## Rebuild Search Index + +Under high server load, it can happen that the search index is not invalidated correctly, although the underlying data +has changed. Accordingly, it can happen that outdated data is found. This can lead to errors in the search component. To +solve this problem manually, administrators can have the search index rebuilt. However, this operation is time-consuming +and could slow down the SCM-Manager for a while. Accordingly, this operation should only be used with caution. If the +problems with the search only affect one repository, then administrators should instead have the search index for this +repository rebuilt. This can be done in the general settings of the repository. + +The option for rebuilding can be found in the general settings: + +![Screenshot of the general Settings to rebuild the search index](assets/rebuild_index.png) diff --git a/gradle/changelog/invalidation.yaml b/gradle/changelog/invalidation.yaml new file mode 100644 index 0000000000..75ebe1327b --- /dev/null +++ b/gradle/changelog/invalidation.yaml @@ -0,0 +1,2 @@ +- type: added + description: Invalidation of caches and search index diff --git a/scm-core/src/main/java/sonia/scm/cache/CacheManager.java b/scm-core/src/main/java/sonia/scm/cache/CacheManager.java index b39579a93c..4cb4fa2b4a 100644 --- a/scm-core/src/main/java/sonia/scm/cache/CacheManager.java +++ b/scm-core/src/main/java/sonia/scm/cache/CacheManager.java @@ -48,4 +48,10 @@ public interface CacheManager extends Closeable { * @return the cache with the specified types and name */ Cache getCache(String name); + + /** + * Clears (aka invalidates) all caches. + * @since 2.48.0 + */ + void clearAllCaches(); } diff --git a/scm-test/src/main/java/sonia/scm/cache/MapCacheManager.java b/scm-test/src/main/java/sonia/scm/cache/MapCacheManager.java index d5633de9c3..b6cc38558e 100644 --- a/scm-test/src/main/java/sonia/scm/cache/MapCacheManager.java +++ b/scm-test/src/main/java/sonia/scm/cache/MapCacheManager.java @@ -75,6 +75,13 @@ public class MapCacheManager return (MapCache) cacheMap.computeIfAbsent(name, k -> new MapCache()); } + @Override + public void clearAllCaches() { + for(MapCache cache : cacheMap.values()) { + cache.clear(); + } + } + //~--- fields --------------------------------------------------------------- /** Field description */ diff --git a/scm-ui/ui-api/src/index.ts b/scm-ui/ui-api/src/index.ts index 5ae171362d..8f108ec90d 100644 --- a/scm-ui/ui-api/src/index.ts +++ b/scm-ui/ui-api/src/index.ts @@ -61,6 +61,7 @@ export * from "./contentType"; export * from "./annotations"; export * from "./search"; export * from "./loginInfo"; +export * from "./useInvalidation"; export * from "./usePluginCenterAuthInfo"; export * from "./compare"; export * from "./utils"; diff --git a/scm-ui/ui-api/src/useInvalidation.tsx b/scm-ui/ui-api/src/useInvalidation.tsx new file mode 100644 index 0000000000..070dab2847 --- /dev/null +++ b/scm-ui/ui-api/src/useInvalidation.tsx @@ -0,0 +1,50 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +import { useMutation } from "react-query"; +import { apiClient } from "./apiclient"; +import { useRequiredIndexLink } from "./base"; + +const useInvalidation = (link: string) => { + const { mutate, isLoading, error, isSuccess } = useMutation((link) => + apiClient.post(link, {}) + ); + + return { + invalidate: () => mutate(link), + isLoading, + isSuccess, + error, + }; +}; + +export const useInvalidateAllCaches = () => { + const invalidateCacheLink = useRequiredIndexLink("invalidateCaches"); + return useInvalidation(invalidateCacheLink); +}; + +export const useInvalidateSearchIndices = () => { + const invalidateSearchIndexLink = useRequiredIndexLink("invalidateSearchIndex"); + return useInvalidation(invalidateSearchIndexLink); +}; diff --git a/scm-ui/ui-webapp/public/locales/de/config.json b/scm-ui/ui-webapp/public/locales/de/config.json index d2128a9a19..fd0798b7c1 100644 --- a/scm-ui/ui-webapp/public/locales/de/config.json +++ b/scm-ui/ui-webapp/public/locales/de/config.json @@ -93,6 +93,18 @@ "login-attempt-limit-invalid": "Dies ist keine Zahl", "plugin-url-invalid": "Dies ist keine gültige URL" }, + "invalidateCaches": { + "success": "Invalidierung von Caches war erfolgreich", + "subtitle": "Caches invalidieren", + "description": "Invalidieren sie Caches manuell, um bestimmte Probleme zu beheben. Achtung: Nach der Invalidierung ist der SCM-Manager verlangsamt.", + "button": "Invalidierung von Caches starten" + }, + "invalidateSearchIndex": { + "success": "Neuaufbau vom Suchindex erfolgreich angestoßen", + "subtitle": "Suchindex neu aufbauen", + "description": "Bauen Sie den Suchindex neu auf, um Probleme mit den Suchergebnissen zu beheben. Achtung: während des Neuaufbaus ist der SCM-Manager verlangsamt.", + "button": "Neuaufbau vom Suchindex starten" + }, "help": { "realmDescriptionHelpText": "Beschreibung des Authentication Realm.", "dateFormatHelpText": "Moments Datumsformat. Zulässige Formate sind in der MomentJS Dokumentation beschrieben.", diff --git a/scm-ui/ui-webapp/public/locales/en/config.json b/scm-ui/ui-webapp/public/locales/en/config.json index 437c5ae12c..e320746c3d 100644 --- a/scm-ui/ui-webapp/public/locales/en/config.json +++ b/scm-ui/ui-webapp/public/locales/en/config.json @@ -93,6 +93,18 @@ "login-attempt-limit-invalid": "This is not a number", "plugin-url-invalid": "This is not a valid url" }, + "invalidateCaches": { + "success": "Successfully invalidated caches", + "subtitle": "Invalidate Caches", + "description": "Invalidate caches manually to fix certain issues. Warning: After invalidation the SCM-Manager is slowed down.", + "button": "Start cache invalidation" + }, + "invalidateSearchIndex": { + "success": "Rebuild of the search index has been triggered", + "subtitle": "Rebuild Search Index", + "description": "Rebuild the search index to fix certain issues with search results. Warning: While rebuilding the search index the SCM-Manager is slowed down.", + "button": "Start recreation of search index" + }, "help": { "realmDescriptionHelpText": "Enter authentication realm description.", "dateFormatHelpText": "Moments date format. Please have a look at the MomentJS documentation.", diff --git a/scm-ui/ui-webapp/src/admin/components/form/ConfigForm.tsx b/scm-ui/ui-webapp/src/admin/components/form/ConfigForm.tsx index ed28589d05..1bcd46cfd3 100644 --- a/scm-ui/ui-webapp/src/admin/components/form/ConfigForm.tsx +++ b/scm-ui/ui-webapp/src/admin/components/form/ConfigForm.tsx @@ -23,7 +23,7 @@ */ import React, { FC, FormEvent, useEffect, useState } from "react"; import { useTranslation } from "react-i18next"; -import { Config, ConfigChangeHandler, NamespaceStrategies } from "@scm-manager/ui-types"; +import { Config, ConfigChangeHandler, Link, NamespaceStrategies } from "@scm-manager/ui-types"; import { Level, Notification, SubmitButton } from "@scm-manager/ui-components"; import ProxySettings from "./ProxySettings"; import GeneralSettings from "./GeneralSettings"; @@ -31,6 +31,8 @@ import BaseUrlSettings from "./BaseUrlSettings"; import LoginAttempt from "./LoginAttempt"; import PluginSettings from "./PluginSettings"; import FunctionSettings from "./FunctionSettings"; +import InvalidateCaches from "./InvalidateCaches"; +import InvalidateSearchIndex from "./InvalidateSearchIndex"; type Props = { submitForm: (p: Config) => void; @@ -39,6 +41,8 @@ type Props = { configReadPermission: boolean; configUpdatePermission: boolean; namespaceStrategies?: NamespaceStrategies; + invalidateCachesLink?: Link; + invalidateSearchIndexLink?: Link; }; const ConfigForm: FC = ({ @@ -48,6 +52,8 @@ const ConfigForm: FC = ({ configReadPermission, configUpdatePermission, namespaceStrategies, + invalidateCachesLink, + invalidateSearchIndexLink, }) => { const [t] = useTranslation("config"); const [innerConfig, setInnerConfig] = useState({ @@ -196,6 +202,18 @@ const ConfigForm: FC = ({ hasUpdatePermission={configUpdatePermission} />
+ {invalidateCachesLink ? ( + <> + +
+ + ) : null} + {invalidateSearchIndexLink ? ( + <> + +
+ + ) : null} { + const { invalidate, isLoading, error, isSuccess } = useInvalidateAllCaches(); + const [t] = useTranslation("config"); + + return ( +
+ + {isSuccess ? {t("invalidateCaches.success")} : null} + +

{t("invalidateCaches.description")}

+ + {t("invalidateCaches.button")} + + } + /> +
+ ); +}; + +export default InvalidateCaches; diff --git a/scm-ui/ui-webapp/src/admin/components/form/InvalidateSearchIndex.tsx b/scm-ui/ui-webapp/src/admin/components/form/InvalidateSearchIndex.tsx new file mode 100644 index 0000000000..86e08d158e --- /dev/null +++ b/scm-ui/ui-webapp/src/admin/components/form/InvalidateSearchIndex.tsx @@ -0,0 +1,52 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +import React, { FC } from "react"; +import { ErrorNotification, Level, Notification, Subtitle } from "@scm-manager/ui-components"; +import { useTranslation } from "react-i18next"; +import { Button, ButtonVariants } from "@scm-manager/ui-buttons"; +import { useInvalidateSearchIndices } from "@scm-manager/ui-api"; + +const InvalidateSearchIndex: FC = () => { + const { invalidate, isLoading, error, isSuccess } = useInvalidateSearchIndices(); + const [t] = useTranslation("config"); + + return ( +
+ + {isSuccess ? {t("invalidateSearchIndex.success")} : null} + +

{t("invalidateSearchIndex.description")}

+ + {t("invalidateSearchIndex.button")} + + } + /> +
+ ); +}; + +export default InvalidateSearchIndex; diff --git a/scm-ui/ui-webapp/src/admin/containers/GlobalConfig.tsx b/scm-ui/ui-webapp/src/admin/containers/GlobalConfig.tsx index 35a05cc9a7..507607c131 100644 --- a/scm-ui/ui-webapp/src/admin/containers/GlobalConfig.tsx +++ b/scm-ui/ui-webapp/src/admin/containers/GlobalConfig.tsx @@ -26,15 +26,16 @@ import { useTranslation } from "react-i18next"; import { Link } from "@scm-manager/ui-types"; import { ErrorNotification, Loading, Title } from "@scm-manager/ui-components"; import ConfigForm from "../components/form/ConfigForm"; -import { useConfig, useNamespaceStrategies, useUpdateConfig } from "@scm-manager/ui-api"; +import { useConfig, useIndexLinks, useNamespaceStrategies, useUpdateConfig } from "@scm-manager/ui-api"; const GlobalConfig: FC = () => { + const indexLinks = useIndexLinks(); const { data: config, error: configLoadingError, isLoading: isLoadingConfig } = useConfig(); const { isLoading: isUpdating, error: updateError, isUpdated, update, reset } = useUpdateConfig(); const { data: namespaceStrategies, error: namespaceStrategiesLoadingError, - isLoading: isLoadingNamespaceStrategies + isLoading: isLoadingNamespaceStrategies, } = useNamespaceStrategies(); const [t] = useTranslation("config"); const error = configLoadingError || namespaceStrategiesLoadingError || updateError || undefined; @@ -67,6 +68,8 @@ const GlobalConfig: FC = () => { namespaceStrategies={namespaceStrategies} configUpdatePermission={canUpdateConfig} configReadPermission={!!config} + invalidateCachesLink={indexLinks.invalidateCaches as Link | undefined} + invalidateSearchIndexLink={indexLinks.invalidateSearchIndex as Link | undefined} /> ); diff --git a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/IndexDtoGenerator.java b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/IndexDtoGenerator.java index e29e1687ba..968171159e 100644 --- a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/IndexDtoGenerator.java +++ b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/IndexDtoGenerator.java @@ -135,6 +135,10 @@ public class IndexDtoGenerator extends HalAppenderMapper { } if (ConfigurationPermissions.list().isPermitted()) { builder.single(link("config", resourceLinks.config().self())); + if (ConfigurationPermissions.write(configuration.getId()).isPermitted()) { + builder.single(link("invalidateCaches", resourceLinks.invalidationLinks().caches())); + builder.single(link("invalidateSearchIndex", resourceLinks.invalidationLinks().searchIndex())); + } if (!Strings.isNullOrEmpty(configuration.getReleaseFeedUrl())) { builder.single(link("updateInfo", resourceLinks.adminInfo().updateInfo())); } diff --git a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/InvalidationResource.java b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/InvalidationResource.java new file mode 100644 index 0000000000..2ac041e7ca --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/InvalidationResource.java @@ -0,0 +1,102 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.api.v2.resources; + +import io.swagger.v3.oas.annotations.OpenAPIDefinition; +import io.swagger.v3.oas.annotations.Operation; +import io.swagger.v3.oas.annotations.media.Content; +import io.swagger.v3.oas.annotations.media.Schema; +import io.swagger.v3.oas.annotations.responses.ApiResponse; +import io.swagger.v3.oas.annotations.tags.Tag; +import sonia.scm.cache.CacheManager; +import sonia.scm.config.ConfigurationPermissions; +import sonia.scm.search.IndexRebuilder; +import sonia.scm.web.VndMediaType; + +import javax.inject.Inject; +import javax.ws.rs.POST; +import javax.ws.rs.Path; + +@OpenAPIDefinition(tags = { + @Tag(name = "Invalidations", description = "Invalidations of different resources like caches and search index") +}) +@Path("v2/invalidations") +public class InvalidationResource { + + private final CacheManager cacheManager; + private final IndexRebuilder indexRebuilder; + + @Inject + public InvalidationResource(CacheManager cacheManager, IndexRebuilder indexRebuilder) { + this.cacheManager = cacheManager; + this.indexRebuilder = indexRebuilder; + } + + @POST + @Path("/caches") + @Operation( + summary = "Invalidates the caches of every store", + description = "Deletes every cached object of every store from the cache", + tags = "Invalidations" + ) + @ApiResponse(responseCode = "204", description = "Invalidated cache successfully") + @ApiResponse(responseCode = "401", description = "not authenticated / invalid credentials") + @ApiResponse(responseCode = "403", description = "not authorized, the current user does not have the \"configuration:write:global\" privilege") + @ApiResponse( + responseCode = "500", + description = "internal server error", + content = @Content( + mediaType = VndMediaType.ERROR_TYPE, + schema = @Schema(implementation = ErrorDto.class) + ) + ) + public void invalidateCaches() { + ConfigurationPermissions.write("global").check(); + cacheManager.clearAllCaches(); + } + + @POST + @Path("/search-index") + @Operation( + summary = "Invalidates the search index", + description = "Invalidates the search index, by completely recreating it", + tags = "Invalidations" + ) + @ApiResponse(responseCode = "204", description = "Invalidated search index successfully") + @ApiResponse(responseCode = "401", description = "not authenticated / invalid credentials") + @ApiResponse(responseCode = "403", description = "not authorized, the current user does not have the \"configuration:write:global\" privilege") + @ApiResponse( + responseCode = "500", + description = "internal server error", + content = @Content( + mediaType = VndMediaType.ERROR_TYPE, + schema = @Schema(implementation = ErrorDto.class) + ) + ) + public void invalidateSearchIndex() { + ConfigurationPermissions.write("global").check(); + indexRebuilder.rebuildAll(); + } +} diff --git a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/ResourceLinks.java b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/ResourceLinks.java index a66fefc751..064b8d2288 100644 --- a/scm-webapp/src/main/java/sonia/scm/api/v2/resources/ResourceLinks.java +++ b/scm-webapp/src/main/java/sonia/scm/api/v2/resources/ResourceLinks.java @@ -347,6 +347,27 @@ class ResourceLinks { } } + InvalidationLinks invalidationLinks() { + return new InvalidationLinks(accessScmPathInfoStore().get()); + } + + static class InvalidationLinks { + private final LinkBuilder invalidationLinkBuilder; + + InvalidationLinks(ScmPathInfo pathInfo) { + this.invalidationLinkBuilder = new LinkBuilder(pathInfo, InvalidationResource.class); + } + + String caches() { + return invalidationLinkBuilder.method("invalidateCaches").parameters().href(); + } + + + String searchIndex() { + return invalidationLinkBuilder.method("invalidateSearchIndex").parameters().href(); + } + } + AdminInfoLinks adminInfo() { return new AdminInfoLinks(accessScmPathInfoStore().get()); } diff --git a/scm-webapp/src/main/java/sonia/scm/cache/GuavaCacheManager.java b/scm-webapp/src/main/java/sonia/scm/cache/GuavaCacheManager.java index cdbdc787ab..35a8a4e4f3 100644 --- a/scm-webapp/src/main/java/sonia/scm/cache/GuavaCacheManager.java +++ b/scm-webapp/src/main/java/sonia/scm/cache/GuavaCacheManager.java @@ -79,6 +79,13 @@ public class GuavaCacheManager implements CacheManager, org.apache.shiro.cache.C }); } + @Override + public void clearAllCaches() { + for(GuavaCache cache : caches.values()) { + cache.clear(); + } + } + @Override public void close() throws IOException { LOG.info("close guava cache manager"); diff --git a/scm-webapp/src/main/java/sonia/scm/search/IndexRebuilder.java b/scm-webapp/src/main/java/sonia/scm/search/IndexRebuilder.java new file mode 100644 index 0000000000..51fdbf25ec --- /dev/null +++ b/scm-webapp/src/main/java/sonia/scm/search/IndexRebuilder.java @@ -0,0 +1,46 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.search; + +import javax.inject.Inject; +import java.util.Set; + +public class IndexRebuilder { + + private final SearchEngine searchEngine; + private final Set indexers; + + @Inject + public IndexRebuilder(SearchEngine searchEngine, Set indexers) { + this.searchEngine = searchEngine; + this.indexers = indexers; + } + + public void rebuildAll() { + for (Indexer indexer : indexers) { + searchEngine.forType(indexer.getType()).update(indexer.getReIndexAllTask()); + } + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/IndexDtoGeneratorTest.java b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/IndexDtoGeneratorTest.java index 51bb031d57..425eb593ed 100644 --- a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/IndexDtoGeneratorTest.java +++ b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/IndexDtoGeneratorTest.java @@ -155,6 +155,68 @@ class IndexDtoGeneratorTest { Link.linkBuilder("search", "/api/v2/search/query/group").withName("group").build() ); } + + @Nested + class InvalidationLinks { + @Test + void shouldAppendInvalidationLinks() { + when(subject.isAuthenticated()).thenReturn(true); + when(subject.isPermitted("configuration:list")).thenReturn(true); + when(subject.isPermitted("configuration:write:1")).thenReturn(true); + mockOtherPermissions(); + when(configuration.getId()).thenReturn("1"); + + IndexDto dto = generator.generate(); + assertThat(dto.getLinks().getLinkBy("invalidateCaches")).contains( + Link.linkBuilder("invalidateCaches", "/api/v2/invalidations/caches").build() + ); + assertThat(dto.getLinks().getLinkBy("invalidateSearchIndex")).contains( + Link.linkBuilder("invalidateSearchIndex", "/api/v2/invalidations/search-index").build() + ); + } + + @Test + void shouldNotAppendInvalidationsIfWritePermissionIsMissing() { + when(subject.isAuthenticated()).thenReturn(true); + when(subject.isPermitted("configuration:list")).thenReturn(true); + when(subject.isPermitted("configuration:write:1")).thenReturn(false); + mockOtherPermissions(); + when(configuration.getId()).thenReturn("1"); + + IndexDto dto = generator.generate(); + assertThat(dto.getLinks().getLinkBy("invalidateCaches")).isEmpty(); + assertThat(dto.getLinks().getLinkBy("invalidateSearchIndex")).isEmpty(); + } + + @Test + void shouldNotAppendInvalidationsIfListPermissionIsMissing() { + when(subject.isAuthenticated()).thenReturn(true); + when(subject.isPermitted("configuration:list")).thenReturn(false); + mockOtherPermissions(); + + IndexDto dto = generator.generate(); + assertThat(dto.getLinks().getLinkBy("invalidateCaches")).isEmpty(); + assertThat(dto.getLinks().getLinkBy("invalidateSearchIndex")).isEmpty(); + } + + @Test + void shouldNotAppendInvalidationsIfUnauthenticated() { + when(subject.isAuthenticated()).thenReturn(false); + + IndexDto dto = generator.generate(); + assertThat(dto.getLinks().getLinkBy("invalidateCaches")).isEmpty(); + assertThat(dto.getLinks().getLinkBy("invalidateSearchIndex")).isEmpty(); + } + + private void mockOtherPermissions() { + when(subject.isPermitted("plugin:read")).thenReturn(false); + when(subject.isPermitted("plugin:write")).thenReturn(false); + when(subject.isPermitted("user:list")).thenReturn(false); + when(subject.isPermitted("user:autocomplete")).thenReturn(false); + when(subject.isPermitted("group:autocomplete")).thenReturn(false); + when(subject.isPermitted("group:list")).thenReturn(false); + } + } } private SearchableType searchableType(String name) { diff --git a/scm-webapp/src/test/java/sonia/scm/api/v2/resources/InvalidationResourceTest.java b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/InvalidationResourceTest.java new file mode 100644 index 0000000000..30e80216e9 --- /dev/null +++ b/scm-webapp/src/test/java/sonia/scm/api/v2/resources/InvalidationResourceTest.java @@ -0,0 +1,126 @@ +/* + * MIT License + * + * Copyright (c) 2020-present Cloudogu GmbH and Contributors + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package sonia.scm.api.v2.resources; + +import org.github.sdorra.jse.ShiroExtension; +import org.github.sdorra.jse.SubjectAware; +import org.jboss.resteasy.mock.MockHttpRequest; +import org.jboss.resteasy.mock.MockHttpResponse; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import sonia.scm.cache.CacheManager; +import sonia.scm.search.IndexRebuilder; +import sonia.scm.web.RestDispatcher; + +import java.net.URISyntaxException; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoInteractions; + + +@ExtendWith({MockitoExtension.class, ShiroExtension.class}) +@SubjectAware("TrainerRed") +class InvalidationResourceTest { + + @Mock + private CacheManager cacheManager; + @Mock + private IndexRebuilder indexRebuilder; + + private RestDispatcher dispatcher; + + private final String basePath = "/v2/invalidations"; + + @BeforeEach + void init() { + InvalidationResource invalidationResource = new InvalidationResource(cacheManager, indexRebuilder); + + dispatcher = new RestDispatcher(); + dispatcher.addSingletonResource(invalidationResource); + } + + @Nested + class InvalidateCaches { + + @Test + void shouldReturnForbiddenBecauseOfMissingPermission() throws URISyntaxException { + MockHttpResponse response = invokeInvalidateCaches(); + assertThat(response.getStatus()).isEqualTo(403); + verifyNoInteractions(cacheManager); + } + + @Test + @SubjectAware(permissions = {"configuration:write:global"}) + void shouldClearCaches() throws URISyntaxException { + MockHttpResponse response = invokeInvalidateCaches(); + assertThat(response.getStatus()).isEqualTo(204); + verify(cacheManager).clearAllCaches(); + } + + private MockHttpResponse invokeInvalidateCaches() throws URISyntaxException { + MockHttpRequest request = MockHttpRequest.post(basePath + "/caches"); + MockHttpResponse response = new MockHttpResponse(); + + dispatcher.invoke(request, response); + return response; + } + } + + @Nested + class ReIndex { + + @Test + void shouldReturnForbiddenBecauseOfMissingPermission() throws URISyntaxException { + MockHttpResponse response = invokeReIndex(); + + assertThat(response.getStatus()).isEqualTo(403); + + verifyNoInteractions(indexRebuilder); + } + + @Test + @SubjectAware(permissions = {"configuration:write:global"}) + void shouldReIndexAll() throws URISyntaxException { + MockHttpResponse response = invokeReIndex(); + + assertThat(response.getStatus()).isEqualTo(204); + + verify(indexRebuilder).rebuildAll(); + } + + private MockHttpResponse invokeReIndex() throws URISyntaxException { + MockHttpRequest request = MockHttpRequest.post(basePath + "/search-index"); + MockHttpResponse response = new MockHttpResponse(); + + dispatcher.invoke(request, response); + return response; + } + } +} diff --git a/scm-webapp/src/test/java/sonia/scm/cache/CacheManagerTestBase.java b/scm-webapp/src/test/java/sonia/scm/cache/CacheManagerTestBase.java index f3a4b28fe0..c06de3877f 100644 --- a/scm-webapp/src/test/java/sonia/scm/cache/CacheManagerTestBase.java +++ b/scm-webapp/src/test/java/sonia/scm/cache/CacheManagerTestBase.java @@ -87,6 +87,20 @@ public abstract class CacheManagerTestBase assertIsSame(c1, c2); } + @Test + public void shouldClearCache() { + Cache c1 = cacheManager.getCache("test-1"); + c1.put("key1", "value1"); + + Cache c2 = cacheManager.getCache("test-2"); + c2.put("key2", "value2"); + + cacheManager.clearAllCaches(); + + assertEquals(c1.size(), 0); + assertEquals(c2.size(), 0); + } + /** * Method description *