diff --git a/CHANGELOG.md b/CHANGELOG.md index 57e365aada..6ef07d9123 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -281,7 +281,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [2.9.0] - 2020-11-06 ### Added -- Tracing api ([#1393](https://github.com/scm-manager/scm-manager/pull/#1393)) +- Tracing api ([#1393](https://github.com/scm-manager/scm-manager/pull/1393)) - Automatic user converter for external users ([#1380](https://github.com/scm-manager/scm-manager/pull/1380)) - Create _authenticated group on setup ([#1396](https://github.com/scm-manager/scm-manager/pull/1396)) - The name of the initial git branch can be configured and is set to `main` by default ([#1399](https://github.com/scm-manager/scm-manager/pull/1399)) diff --git a/docs/en/administration/workdir_caching.md b/docs/en/administration/workdir_caching.md new file mode 100644 index 0000000000..7df2b557e0 --- /dev/null +++ b/docs/en/administration/workdir_caching.md @@ -0,0 +1,20 @@ +--- +title: Caching for Working Directories +--- + +SCM-Manager offers commands to modify repositories on the server side. For example this is used by the +[Editor Plugin](https://www.scm-manager.org/plugins/scm-editor-plugin/) and the +[Review Plugin](https://www.scm-manager.org/plugins/scm-review-plugin/). Without further configuration, this is done +by cloning/checking out the repository temporarily, performing the change, creating a commit and pushing the changes +back to the central repository. The larger the repositories, the longer this may take. + +To speed up such changes a lot, SCM-Manager offers a strategy where the local clones will be cached and reused for +subsequent requests. This strategy caches up to a configurable amount of clones (but at most one per repository). +To enable this strategy, add the system property `scm.workingCopyPoolStrategy` to the value +`sonia.scm.repository.work.SimpleCachingWorkingCopyPool`: + +```bash +-Dscm.workingCopyPoolStrategy=sonia.scm.repository.work.SimpleCachingWorkingCopyPool +``` + +The maximum capacity of the cache can be set using the property `scm.workingCopyPoolSize` (the default is 5). diff --git a/docs/en/navigation.yml b/docs/en/navigation.yml index e0edf3418d..bdae6704ae 100644 --- a/docs/en/navigation.yml +++ b/docs/en/navigation.yml @@ -22,6 +22,7 @@ - /administration/logging/ - /administration/scm-server/ - /administration/reverse-proxies/ + - /administration/workdir_caching/ - section: Development entries: diff --git a/gradle/changelog/simple_workdir_cache.yaml b/gradle/changelog/simple_workdir_cache.yaml new file mode 100644 index 0000000000..7ff1960e21 --- /dev/null +++ b/gradle/changelog/simple_workdir_cache.yaml @@ -0,0 +1,2 @@ +- type: changed + description: The simple workdir cache has a maximum size, an lru semantic and blocks on parallel requests ([#1735](https://github.com/scm-manager/scm-manager/pull/1735)) diff --git a/scm-core/src/main/java/sonia/scm/repository/work/SimpleCachingWorkingCopyPool.java b/scm-core/src/main/java/sonia/scm/repository/work/SimpleCachingWorkingCopyPool.java index a66e55aa4e..d57b3cfeed 100644 --- a/scm-core/src/main/java/sonia/scm/repository/work/SimpleCachingWorkingCopyPool.java +++ b/scm-core/src/main/java/sonia/scm/repository/work/SimpleCachingWorkingCopyPool.java @@ -24,35 +24,53 @@ package sonia.scm.repository.work; +import com.google.common.annotations.VisibleForTesting; import com.google.common.base.Stopwatch; +import io.micrometer.core.instrument.Counter; +import io.micrometer.core.instrument.MeterRegistry; +import io.micrometer.core.instrument.Timer; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import sonia.scm.util.IOUtil; import javax.inject.Inject; import java.io.File; +import java.util.LinkedHashMap; import java.util.Map; +import java.util.Optional; import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.locks.Lock; +import java.util.concurrent.locks.ReentrantLock; + +import static java.lang.Integer.getInteger; +import static java.util.Optional.empty; +import static java.util.Optional.of; /** * This class is a simple implementation of the {@link WorkingCopyPool} to demonstrate, - * how caching can work in the simplest way. For the first time a {@link WorkingCopy} is + * how caching can work in an LRU style. For the first time a {@link WorkingCopy} is * requested for a repository with {@link #getWorkingCopy(SimpleWorkingCopyFactory.WorkingCopyContext)}, * this implementation fetches a new directory from the {@link WorkdirProvider}. * On {@link #contextClosed(SimpleWorkingCopyFactory.WorkingCopyContext, File)}, - * the directory is not deleted, but put into a map with the repository id as key. + * the directory is not deleted, but put into a cache with the repository id as key. * When a working copy is requested with {@link #getWorkingCopy(SimpleWorkingCopyFactory.WorkingCopyContext)} * for a repository with such an existing directory, it is taken from the map, reclaimed and * returned as {@link WorkingCopy}. * If for one repository a working copy is requested, while another is in use already, - * a second directory is requested from the {@link WorkdirProvider} for the second one. - * If a context is closed with {@link #contextClosed(SimpleWorkingCopyFactory.WorkingCopyContext, File)} - * and there already is a directory stored in the map for the repository, - * the directory from the closed context simply is deleted. + * the process will wait until the other process has finished. + * The number of directories cached is limited. By default, directories are cached for + * {@value DEFAULT_WORKING_COPY_POOL_SIZE} repositories. This can be changes with the system + * property '{@value WORKING_COPY_POOL_SIZE_PROPERTY}' (if this is set to zero, no caching will + * take place; to cache the directories for each repository without eviction simply set this to a + * high enough value). *
- * In general, this implementation should speed up things a bit, but one has to take into - * account, that there is no monitoring of diskspace. So you have to make sure, that - * there is enough space for a clone of each repository in the working dir. + * The usage of this pool has to be enabled by setting the system property `scm.workingCopyPoolStrategy` + * to 'sonia.scm.repository.work.SimpleCachingWorkingCopyPool'. + *
+ * In general, this implementation should speed up modifications inside SCM-Manager performed by + * the editor plugin or the review plugin, but one has to take into + * account, that the space needed for repositories is multiplied. So you have to make sure, that + * there is enough space for clones of the repository. *
* Possible enhancements: *