Improve server config documentation and cleanup

This commit is contained in:
Eduard Heimbuch
2023-12-01 14:34:50 +01:00
committed by René Pfeuffer
parent abe0a62cb4
commit d0c43dd9f4
32 changed files with 431 additions and 303 deletions

View File

@@ -5,15 +5,18 @@ displayToc: true
---
SCM-Manager can be configured in several ways. We recommend using `config.yml` to have most of the settings in
one place.
However, if required, each option in this configuration can also be set via environment variables.
one place. However, if required, each option in this configuration can also be set via environment variables.
See the relevant topics below for more information.
## Change log level
The log level can be configured in the `config.yml`.
You may either change the root log level to change the log level globally for all loggers.
You can change the root log level to change the log level globally for all loggers.
Also, new specific logger can be added to control logging in a fine-grained style.
We provide two types of log appender.
`fileAppender` which writes the logging output to a defined log file and the `consoleAppender` which simply prints
everything to your (bash/shell) console.
Depending on which platform your scm-server is running, we already configured the recommended logging settings.
#### Example
@@ -21,6 +24,8 @@ Also, new specific logger can be added to control logging in a fine-grained styl
log:
# General logging level
rootLevel: WARN
enableFileAppender: true
enableConsoleAppender: true
# Custom specific loggers
# The "name" has to be the path of the classes to be logged with this logger
@@ -34,6 +39,8 @@ log:
To override this config with environment variables you could set it like:
`SCM_LOG_ROOT_LEVEL` to one of the log levels, like `DEBUG`
`SCM_LOG_FILE_APPENDER_ENABLED` to one of the log levels, like `true`
`SCM_LOG_CONSOLE_APPENDER_ENABLED` to one of the log levels, like `false`
`SCM_LOG_LOGGER` with a comma-separated list of your loggers, like `sonia.scm:DEBUG,com.cloudogu.scm:TRACE`
Supported log levels are: TRACE, DEBUG, INFO, WARN, ERROR
@@ -98,9 +105,10 @@ If the logback configuration is enabled, the log configuration of the `config.ym
</configuration>
```
## Change host and port
## Change host, port and context path
The listener host and port of your SCM-Server can directly be edited in the top level of your `config.yml`.
If you want your server without a context path (use `root path`), you can change this option to be `/`.
#### Example
@@ -109,26 +117,13 @@ The listener host and port of your SCM-Server can directly be edited in the top
addressBinding: 0.0.0.0
# This is the exposed port for your application
port: 8080
```
To override this config with environment variables you could set it like:
`SCM_SERVER_PORT` to your port
`SCM_SERVER_ADDRESS_BINDING` to the destination ip / hostname
## Change context path
SCM-Server context path can be set directly in the top level of your `config.yml`.
If you want your server without a context path (use `root`), you can change this option to be `/`.
#### Example
```yaml
contextPath: /
```
To override this config with environment variables you could set it like:
`SCM_SERVER_PORT` to your port, like `8081`
`SCM_SERVER_ADDRESS_BINDING` to the destination ip / hostname, like `0.0.0.0`
`SCM_SERVER_CONTEXT_PATH` to `/myContextPath`
## SSL
@@ -212,12 +207,80 @@ https:
## Change directories
The default directories are platform-specific and therefore could be different if you try scm-server on different
operation systems. Paths starting with `/` are absolute to your file system. If you use relative paths without a
starting `/`, your configured path will be located under the base directory of your scm-server.
operational systems. Paths starting with `/` are absolute to your file system. If you use relative paths without a
starting `/`, your configured path will be located relative to your base directory of your scm-server (
like `/opt/scm-server`
on unix-based packages).
For technical reasons the tempDir is located at the top level of your config.yml. All other path-based config options
are located under `webapp`.
#### Example
```yaml
tempDir: /tmp
homeDir: scm-home
webapp:
homeDir: ./scm-home
workDir:
```
To override this config with environment variables you could set it like:
`SCM_TEMP_DIR` to your port, like `/tmp`
`SCM_WEBAPP_HOMEDIR` to home dir (aka `scm-home`), like `./myHomeDir`
`SCM_WEBAPP_WORKDIR` to `/scm-work`
## Reverse proxy
If your SCM-Manager instance is behind a reverse proxy like NGINX, you most likely will have to enable
X-Forward-Headers.
These HTTP headers are being appended to the requests which are redirected by your reverse proxy server. Without
this option set, your SCM-Server may run into connection issues. This option is disabled by default, because without a
reverse proxy it could cause security issues.
#### Example
```yaml
forwardHeadersEnabled: true
```
## Webapp
The webapp configuration consists of anything that is not webserver or logging related.
Most of the available options should be set to the recommended values of your default `config.yml` file.
```yaml
webapp:
## Sets explicit working directory for internal processes, empty means default java temp dir
workDir:
## Home directory "scm-home" which is also set for classpath
homeDir: /var/lib/scm
cache:
dataFile:
enabled: true
store:
enabled: true
## Warning: Enabling this option can lead to security issue.
endlessJwt: false
## Number of async threads
asyncThreads: 4
## Max seconds to abort async execution
maxAsyncAbortSeconds: 60
## Amount of central work queue workers
centralWorkQueue:
workers: 4
## Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
workingCopyPoolStrategy: sonia.scm.repository.work.SimpleCachingWorkingCopyPool
## Amount of "cached" working copies
workingCopyPoolSize: 5
```
To override this config with environment variables can set the options following the schema `SCM_WEBAPP_PROPERTYNAME`.
All hierarchy levels have to be separated by underscores `_`.
Like:
- `SCM_WEBAPP_CACHE_DATAFILE_ENABLED` = `true`
- `SCM_WEBAPP_MAXASYNCABORTSECONDS` = `45`
- `SCM_WEBAPP_CENTRALWORKQUEUE_WORKERS` = `42`

View File

@@ -25,6 +25,7 @@
package sonia.scm;
import com.google.common.base.Strings;
import lombok.extern.slf4j.Slf4j;
import sonia.scm.config.WebappConfigProvider;
import sonia.scm.util.SystemUtil;
@@ -41,6 +42,7 @@ import java.util.Properties;
*
* @since 2.0.0
*/
@Slf4j
final class BaseDirectory {
/**
@@ -93,7 +95,7 @@ final class BaseDirectory {
if (Strings.isNullOrEmpty(directory)) {
directory = getOsSpecificDefault();
}
log.info("using directory {} as scm home directory", directory);
return Paths.get(directory);
}
@@ -116,14 +118,21 @@ final class BaseDirectory {
} else if (platform.isWindows()) {
return getWindowsDefault();
}
return getUnixDefault();
}
private String getUnixDefault() {
log.debug("using Unix default for home directory");
return systemProperties.getProperty("user.home") + "/.scm";
}
private String getOsxDefault() {
log.debug("using OS/X default for home directory");
return systemProperties.getProperty("user.home") + "/Library/Application Support/SCM-Manager";
}
private String getWindowsDefault() {
log.debug("using Windows default for home directory");
return environment.get("APPDATA") + "\\SCM-Manager";
}

View File

@@ -24,18 +24,32 @@
package sonia.scm.config;
import java.util.HashMap;
import lombok.extern.slf4j.Slf4j;
import java.util.Map;
import java.util.Optional;
public class WebappConfigProvider {
import static java.util.Optional.empty;
private WebappConfigProvider() {}
@Slf4j
public final class WebappConfigProvider {
private static Map<String, String> configBindings = new HashMap<>();
private static WebappConfigProvider instance;
private final Map<String, String> configBindings;
private final Map<String, String> environment;
private WebappConfigProvider(Map<String, String> configBindings, Map<String, String> environment) {
this.configBindings = configBindings;
this.environment = environment;
}
public static void setConfigBindings(Map<String, String> newBindings) {
configBindings = newBindings;
WebappConfigProvider.setConfigBindings(newBindings, System.getenv());
}
static void setConfigBindings(Map<String, String> newBindings, Map<String, String> environment) {
instance = new WebappConfigProvider(newBindings, environment);
}
public static Optional<String> resolveAsString(String key) {
@@ -55,6 +69,25 @@ public class WebappConfigProvider {
}
private static Optional<String> resolveConfig(String key) {
return Optional.ofNullable(configBindings.get(key));
if (instance == null) {
return empty();
}
return instance.resolveConfigInternal(key);
}
private Optional<String> resolveConfigInternal(String key) {
String envValue = environment.get("SCM_WEBAPP_" + key.replace('.', '_').toUpperCase());
if (envValue != null) {
log.debug("resolve config for key '{}' to value '{}' from environment", key, envValue);
return Optional.of(envValue);
}
String value = instance.configBindings.get(key);
if (value == null) {
log.debug("could not resolve config for key '{}'", key);
return empty();
} else {
log.debug("resolve config for key '{}' to value '{}'", key, value);
return Optional.of(value);
}
}
}

View File

@@ -21,7 +21,7 @@
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.plugin;
//~--- JDK imports ------------------------------------------------------------
@@ -35,23 +35,21 @@ import java.nio.file.Path;
* @author Sebastian Sdorra
* @since 2.0.0
*/
public final class InstalledPlugin implements Plugin
{
public final class InstalledPlugin implements Plugin {
public static final String UNINSTALL_MARKER_FILENAME = "uninstall";
public static final String COMPATIBILITY_MARKER_FILENAME = ".jakarta-compatible";
/**
* Constructs a new plugin wrapper.
* @param descriptor wrapped plugin
* @param classLoader plugin class loader
*
* @param descriptor wrapped plugin
* @param classLoader plugin class loader
* @param webResourceLoader web resource loader
* @param directory plugin directory
* @param core marked as core or not
* @param directory plugin directory
* @param core marked as core or not
*/
public InstalledPlugin(InstalledPluginDescriptor descriptor, ClassLoader classLoader,
WebResourceLoader webResourceLoader, Path directory, boolean core)
{
WebResourceLoader webResourceLoader, Path directory, boolean core) {
this.descriptor = descriptor;
this.classLoader = classLoader;
this.webResourceLoader = webResourceLoader;
@@ -64,56 +62,46 @@ public final class InstalledPlugin implements Plugin
/**
* Returns plugin class loader.
*
*
* @return plugin class loader
*/
public ClassLoader getClassLoader()
{
public ClassLoader getClassLoader() {
return classLoader;
}
/**
* Returns plugin directory.
*
*
* @return plugin directory
*/
public Path getDirectory()
{
public Path getDirectory() {
return directory;
}
/**
* Returns the id of the plugin.
*
*
* @return id of plugin
*/
public String getId()
{
public String getId() {
return descriptor.getInformation().getId();
}
/**
* Returns the plugin descriptor.
*
*
* @return plugin descriptor
*/
@Override
public InstalledPluginDescriptor getDescriptor()
{
public InstalledPluginDescriptor getDescriptor() {
return descriptor;
}
/**
* Returns the {@link WebResourceLoader} for this plugin.
*
*
* @return web resource loader
*/
public WebResourceLoader getWebResourceLoader()
{
public WebResourceLoader getWebResourceLoader() {
return webResourceLoader;
}
@@ -139,16 +127,24 @@ public final class InstalledPlugin implements Plugin
//~--- fields ---------------------------------------------------------------
/** plugin class loader */
/**
* plugin class loader
*/
private final ClassLoader classLoader;
/** plugin directory */
/**
* plugin directory
*/
private final Path directory;
/** plugin */
/**
* plugin
*/
private final InstalledPluginDescriptor descriptor;
/** plugin web resource loader */
/**
* plugin web resource loader
*/
private final WebResourceLoader webResourceLoader;
private final boolean core;

View File

@@ -52,7 +52,7 @@ public class WorkdirProvider implements ServletContextListener {
@Inject
public WorkdirProvider(
@ConfigValue(key = "workdir", defaultValue = "", description = "Working directory for internal repository operations") String workDir,
@ConfigValue(key = "workDir", defaultValue = "", description = "Working directory for internal repository operations") String workDir,
RepositoryLocationResolver repositoryLocationResolver
) {
this(new File(!Strings.isNullOrEmpty(workDir) ? workDir : System.getProperty("java.io.tmpdir") , "scm-work"), repositoryLocationResolver, workDir == null);

View File

@@ -0,0 +1,80 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.config;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.junit.jupiter.MockitoExtension;
import java.util.Map;
import static java.util.Collections.emptyMap;
import static org.assertj.core.api.Assertions.assertThat;
@ExtendWith(MockitoExtension.class)
class WebappConfigProviderTest {
@Test
void shouldHandleEmptyConfig() {
WebappConfigProvider.setConfigBindings(emptyMap());
assertThat(WebappConfigProvider.resolveAsString("key")).isEmpty();
}
@Test
void shouldResolveStringValueFromConfig() {
WebappConfigProvider.setConfigBindings(Map.of("key", "value"));
assertThat(WebappConfigProvider.resolveAsString("key")).contains("value");
}
@Test
void shouldResolveNumberValueFromConfig() {
WebappConfigProvider.setConfigBindings(Map.of("key", "42"));
assertThat(WebappConfigProvider.resolveAsInteger("key")).contains(42);
}
@Test
void shouldResolveBooleanValueFromConfig() {
WebappConfigProvider.setConfigBindings(Map.of("key", "true"));
assertThat(WebappConfigProvider.resolveAsBoolean("key")).contains(true);
}
@Test
void shouldLetEnvironmentOverrideConfig() {
WebappConfigProvider.setConfigBindings(Map.of("key", "value"), Map.of("SCM_WEBAPP_KEY", "env"));
assertThat(WebappConfigProvider.resolveAsString("key")).contains("env");
}
@Test
void shouldCreateCorrectKeyForEnvironment() {
WebappConfigProvider.setConfigBindings(Map.of("some.more.complex.key", "value"), Map.of("SCM_WEBAPP_SOME_MORE_COMPLEX_KEY", "env"));
assertThat(WebappConfigProvider.resolveAsString("some.more.complex.key")).contains("env");
}
}

View File

@@ -59,6 +59,7 @@ public class DataFileCache {
DataFileCache(Cache<File, Object> cache, boolean cacheEnabled) {
this.cache = cache;
this.cacheEnabled = cacheEnabled;
LOG.debug("data file cache enabled: {}", cacheEnabled);
}
DataFileCacheInstance instanceFor(Class<?> type) {

View File

@@ -32,6 +32,7 @@ log:
sonia.scm: INFO
com.cloudogu.scm: INFO
# webapp config
webapp:
## Sets explicit working directory for internal processes, empty means default java temp dir
workDir:
@@ -44,14 +45,14 @@ webapp:
enabled: true
## Warning: Enabling this option can lead to security issue.
endlessJwt: false
#### Number of async threads
## Number of async threads
asyncThreads: 4
#### Max seconds to abort async execution
## Max seconds to abort async execution
maxAsyncAbortSeconds: 60
#### Amount of central work queue workers
central-work-queue:
## Amount of central work queue workers
centralWorkQueue:
workers: 4
#### Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
## Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
workingCopyPoolStrategy: sonia.scm.repository.work.SimpleCachingWorkingCopyPool
#### Amount of "cached" working copies
## Amount of "cached" working copies
workingCopyPoolSize: 5

View File

@@ -30,6 +30,7 @@ log:
sonia.scm: INFO
com.cloudogu.scm: INFO
# webapp config
webapp:
## Sets explicit working directory for internal processes, empty means default java temp dir
workDir:
@@ -42,14 +43,14 @@ webapp:
enabled: true
## Warning: Enabling this option can lead to security issue.
endlessJwt: false
#### Number of async threads
## Number of async threads
asyncThreads: 4
#### Max seconds to abort async execution
## Max seconds to abort async execution
maxAsyncAbortSeconds: 60
#### Amount of central work queue workers
central-work-queue:
## Amount of central work queue workers
centralWorkQueue:
workers: 4
#### Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
## Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
workingCopyPoolStrategy: sonia.scm.repository.work.SimpleCachingWorkingCopyPool
#### Amount of "cached" working copies
## Amount of "cached" working copies
workingCopyPoolSize: 5

View File

@@ -80,14 +80,14 @@ data:
enabled: true
## Warning: Enabling this option can lead to security issue.
endlessJwt: false
#### Number of async threads
## Number of async threads
asyncThreads: 4
#### Max seconds to abort async execution
## Max seconds to abort async execution
maxAsyncAbortSeconds: 60
#### Amount of central work queue workers
## Amount of central work queue workers
central-work-queue:
workers: 4
#### Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
## Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
workingCopyPoolStrategy: sonia.scm.repository.work.SimpleCachingWorkingCopyPool
#### Amount of "cached" working copies
## Amount of "cached" working copies
workingCopyPoolSize: 5

View File

@@ -33,6 +33,7 @@ log:
sonia.scm: INFO
com.cloudogu.scm: INFO
# webapp config
webapp:
## Sets explicit working directory for internal processes, empty means default java temp dir
workDir:
@@ -45,14 +46,14 @@ webapp:
enabled: true
## Warning: Enabling this option can lead to security issue.
endlessJwt: false
#### Number of async threads
## Number of async threads
asyncThreads: 4
#### Max seconds to abort async execution
## Max seconds to abort async execution
maxAsyncAbortSeconds: 60
#### Amount of central work queue workers
central-work-queue:
## Amount of central work queue workers
centralWorkQueue:
workers: 4
#### Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
## Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
workingCopyPoolStrategy: sonia.scm.repository.work.SimpleCachingWorkingCopyPool
#### Amount of "cached" working copies
## Amount of "cached" working copies
workingCopyPoolSize: 5

View File

@@ -5,7 +5,7 @@ port: 8080
contextPath: /scm
## Evaluates headers set by a reverse proxy like X-Forwarded-For, X-Forwarded-Proto and X-Forwarded-Host
forwardHeadersEnabled: false
forwardHeadersEnabled: true
## increase http header size for mercurial
httpHeaderSize: 16384
@@ -20,12 +20,12 @@ https:
redirectHttpToHttps: false
# Temp directory used for jetty webserver. The temporary directory for internal operations can be configured as "workDir" in webapp.
tempDir: /var/cache/scm/work
tempDir: ./work
# logging
log:
## Destination of logging files
logDir: /var/log/scm
logDir: ./logs
rootLevel: WARN
enableFileAppender: true
enableConsoleAppender: true
@@ -37,7 +37,7 @@ webapp:
## Sets explicit working directory for internal processes, empty means default java temp dir
workDir:
## Home directory "scm-home" which is also set for classpath
homeDir: /var/lib/scm
homeDir:
cache:
dataFile:
enabled: true
@@ -45,14 +45,14 @@ webapp:
enabled: true
## Warning: Enabling this option can lead to security issue.
endlessJwt: false
#### Number of async threads
## Number of async threads
asyncThreads: 4
#### Max seconds to abort async execution
## Max seconds to abort async execution
maxAsyncAbortSeconds: 60
#### Amount of central work queue workers
central-work-queue:
## Amount of central work queue workers
centralWorkQueue:
workers: 4
#### Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
## Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
workingCopyPoolStrategy: sonia.scm.repository.work.SimpleCachingWorkingCopyPool
#### Amount of "cached" working copies
## Amount of "cached" working copies
workingCopyPoolSize: 5

View File

@@ -33,6 +33,7 @@ log:
sonia.scm: INFO
com.cloudogu.scm: INFO
# webapp config
webapp:
## Sets explicit working directory for internal processes, empty means default java temp dir
workDir:
@@ -45,14 +46,14 @@ webapp:
enabled: true
## Warning: Enabling this option can lead to security issue.
endlessJwt: false
#### Number of async threads
## Number of async threads
asyncThreads: 4
#### Max seconds to abort async execution
## Max seconds to abort async execution
maxAsyncAbortSeconds: 60
#### Amount of central work queue workers
central-work-queue:
## Amount of central work queue workers
centralWorkQueue:
workers: 4
#### Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
## Strategy for the working copy pool implementation [sonia.scm.repository.work.NoneCachingWorkingCopyPool, sonia.scm.repository.work.SimpleCachingWorkingCopyPool]
workingCopyPoolStrategy: sonia.scm.repository.work.SimpleCachingWorkingCopyPool
#### Amount of "cached" working copies
## Amount of "cached" working copies
workingCopyPoolSize: 5

View File

@@ -142,7 +142,7 @@ public class ServerConfigYaml {
}
public boolean isForwardHeadersEnabled() {
return getEnvWithDefault("FORWARD_REMOTE_ADDRESS", forwardHeadersEnabled);
return getEnvWithDefault("FORWARD_HEADERS_ENABLED", forwardHeadersEnabled);
}
public void setForwardHeadersEnabled(boolean forwardHeadersEnabled) {

View File

@@ -38,6 +38,8 @@ import org.eclipse.jetty.util.resource.ResourceCollection;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.eclipse.jetty.webapp.WebAppContext;
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.nio.file.Path;
import java.nio.file.Paths;
@@ -49,16 +51,15 @@ public final class ServerConfiguration {
@SuppressWarnings("java:S1075") // not a real uri
private static final String DEFAULT_CONTEXT_PATH = "/scm";
private final ServerConfigYaml configYaml;
private Path testTempDir;
private final String baseDir = System.getProperty("basedir", ".");
public ServerConfiguration() {
this.configYaml = new ServerConfigParser().parse();
}
// Visible for testing
public ServerConfiguration(URL configFile, Path tempDir) {
public ServerConfiguration(URL configFile) {
this.configYaml = new ServerConfigParser().parse(configFile);
this.testTempDir = tempDir;
}
public void configureServer(Server server) {
@@ -138,23 +139,45 @@ public final class ServerConfiguration {
// disable directory listings
webApp.setInitParameter("org.eclipse.jetty.servlet.Default.dirAllowed", "false");
String baseDir = resolveBaseDir();
webApp.setWar(baseDir + "/var/webapp/scm-webapp.war");
String baseDir = this.baseDir;
String warFile = asCanonicalFile(Paths.get(baseDir, "/var/webapp/scm-webapp.war")).toString();
System.out.println("Set webapp war file to " + warFile);
webApp.setWar(warFile);
String tempDir = configYaml.getTempDir();
webApp.setTempDirectory(tempDir.startsWith("/") ? Paths.get(tempDir, "webapp").toFile() : Paths.get(baseDir, tempDir).toFile());
File webappTempDir =
asCanonicalFile(tempDir.startsWith("/") ?
Paths.get(tempDir, "webapp") :
Paths.get(baseDir, tempDir, "scm")
);
System.out.printf("Set webapp temp directory to %s%n", webappTempDir);
webApp.setTempDirectory(webappTempDir);
return webApp;
}
private WebAppContext createDocRoot() {
WebAppContext docRoot = new WebAppContext();
docRoot.setContextPath("/");
String baseDir = resolveBaseDir();
String baseDir = this.baseDir;
docRoot.setBaseResource(new ResourceCollection(new String[]{baseDir + "/var/webapp/docroot"}));
String tempDir = configYaml.getTempDir();
docRoot.setTempDirectory(tempDir.startsWith("/") ? Paths.get(tempDir, "work/docroot").toFile() : Paths.get(baseDir, tempDir).toFile());
File docRootTempDir =
asCanonicalFile(tempDir.startsWith("/") ?
Paths.get(tempDir, "work/docroot") :
Paths.get(baseDir, tempDir, "docroot")
);
System.out.printf("Set docroot temp directory to %s%n", docRootTempDir);
docRoot.setTempDirectory(docRootTempDir);
return docRoot;
}
private File asCanonicalFile(Path path) {
try {
return path.toFile().getAbsoluteFile().getCanonicalFile();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private HttpConfiguration createCustomHttpConfig() {
HttpConfiguration httpConfig = new HttpConfiguration();
System.out.println("Set http request header size to " + configYaml.getHttpHeaderSize());
@@ -195,11 +218,6 @@ public final class ServerConfiguration {
return listeners;
}
private String resolveBaseDir() {
return testTempDir != null ? testTempDir.toString() : System.getProperty("baseDir", ".");
}
private String findContextPath(Handler[] handlers) {
for (Handler handler : handlers) {
if (handler instanceof WebAppContext) {

View File

@@ -41,7 +41,7 @@ class ServerConfigurationTest {
@Test
void shouldThrowServerConfigurationExceptionIfConfigYamlNotFound() {
assertThrows(ServerConfigurationException.class, () -> new ServerConfiguration(null, null));
assertThrows(ServerConfigurationException.class, () -> new ServerConfiguration(null));
}
@Test
@@ -101,7 +101,8 @@ class ServerConfigurationTest {
Files.createDirectories(tempDir.resolve("var/webapp/docroot"));
Files.createFile(tempDir.resolve("var/webapp/scm-webapp.war"));
return new ServerConfiguration(path.toUri().toURL(), tempDir);
System.setProperty("basedir", tempDir.toString());
return new ServerConfiguration(path.toUri().toURL());
}
}

View File

@@ -60,12 +60,12 @@ public class LoggingConfiguration {
root.setLevel(Level.valueOf(config.getRootLevel()));
configureSpecificLoggers();
if (config.isEnableFileAppender()) {
if (config.isFileAppenderEnabled()) {
RollingFileAppender fileAppender = configureFileLogger();
root.addAppender(fileAppender);
}
if (config.isEnableConsoleAppender()) {
if (config.isConsoleAppenderEnabled()) {
ConsoleAppender consoleAppender = configureConsoleLogger();
root.addAppender(consoleAppender);
}

View File

@@ -39,8 +39,8 @@ public class ServerConfigYaml {
private String logDir = "";
private String rootLevel = "INFO";
private Map<String, String> logger = new HashMap<>();
private boolean enableFileAppender = true;
private boolean enableConsoleAppender = true;
private boolean fileAppenderEnabled = true;
private boolean consoleAppenderEnabled = true;
private LogConfig() {}
@@ -67,7 +67,7 @@ public class ServerConfigYaml {
continue;
}
String[] envLoggerEntryParts = envLoggerEntry.trim().split(":");
loggerMap.put(envLoggerEntryParts[0], envLoggerEntryParts[1]);
loggerMap.put(envLoggerEntryParts[0].trim(), envLoggerEntryParts[1].trim());
}
return loggerMap;
}
@@ -82,20 +82,20 @@ public class ServerConfigYaml {
this.logger = logger;
}
public boolean isEnableFileAppender() {
return getEnvWithDefault("LOG_ENABLE_FILE_APPENDER", enableFileAppender);
public boolean isFileAppenderEnabled() {
return getEnvWithDefault("LOG_FILE_APPENDER_ENABLED", fileAppenderEnabled);
}
public void setEnableFileAppender(boolean enableFileAppender) {
this.enableFileAppender = enableFileAppender;
public void setFileAppenderEnabled(boolean fileAppenderEnabled) {
this.fileAppenderEnabled = fileAppenderEnabled;
}
public boolean isEnableConsoleAppender() {
return getEnvWithDefault("LOG_ENABLE_CONSOLE_APPENDER", enableConsoleAppender);
public boolean isConsoleAppenderEnabled() {
return getEnvWithDefault("LOG_CONSOLE_APPENDER_ENABLED", consoleAppenderEnabled);
}
public void setEnableConsoleAppender(boolean enableConsoleAppender) {
this.enableConsoleAppender = enableConsoleAppender;
public void setConsoleAppenderEnabled(boolean consoleAppenderEnabled) {
this.consoleAppenderEnabled = consoleAppenderEnabled;
}
}

View File

@@ -46,8 +46,6 @@ import sonia.scm.plugin.InstalledPluginDescriptor;
import sonia.scm.plugin.PluginException;
import sonia.scm.plugin.PluginLoadException;
import sonia.scm.plugin.PluginLoader;
import sonia.scm.plugin.PluginTransformException;
import sonia.scm.plugin.PluginTransformer;
import sonia.scm.plugin.PluginsInternal;
import sonia.scm.plugin.SmpArchive;
import sonia.scm.util.IOUtil;
@@ -109,35 +107,12 @@ public final class PluginBootstrap {
LOG.info("core plugin extraction is disabled");
}
uninstallMarkedPlugins(pluginDirectory.toPath());
transformIncompatiblePlugins(pluginDirectory.toPath());
return PluginsInternal.collectPlugins(classLoaderLifeCycle, pluginDirectory.toPath());
} catch (IOException ex) {
throw new PluginLoadException("could not load plugins", ex);
}
}
private void transformIncompatiblePlugins(Path pluginsDirectory) {
try (Stream<Path> list = java.nio.file.Files.list(pluginsDirectory)) {
list
.filter(java.nio.file.Files::isDirectory)
.filter(this::isIncompatiblePlugin)
.forEach(plugin -> {
PluginTransformer.transform(plugin);
try {
Files.touch(plugin.resolve(InstalledPlugin.COMPATIBILITY_MARKER_FILENAME).toFile());
} catch (IOException e) {
throw new PluginTransformException("Failed to create marker file for jakarta compatibility", e);
}
});
} catch (IOException e) {
LOG.warn("error occurred while checking for plugins that should be transformed", e);
}
}
private boolean isIncompatiblePlugin(Path path) {
return !new File(path.toFile(), InstalledPlugin.COMPATIBILITY_MARKER_FILENAME).exists();
}
private void uninstallMarkedPlugins(Path pluginDirectory) {
try (Stream<Path> list = java.nio.file.Files.list(pluginDirectory)) {
list

View File

@@ -54,7 +54,7 @@ public class PluginWizardStartupAction implements InitializationStep {
@Override
public boolean done() {
return WebappConfigProvider.resolveAsString("initialPassword").orElse(null) != null || store.getPluginSets().isPresent();
return WebappConfigProvider.resolveAsString("initialPassword").isPresent() || store.getPluginSets().isPresent();
}
}

View File

@@ -70,7 +70,11 @@ public class ConfigurationResolver {
Map<String, String> configurationFile = new HashMap<>();
rootNode.fields().forEachRemaining(entry -> {
if (entry.getValue().isValueNode()) {
configurationFile.put(prefix + entry.getKey(), entry.getValue().asText());
if (entry.getValue().isNull()) {
configurationFile.put(prefix + entry.getKey(), null);
} else {
configurationFile.put(prefix + entry.getKey(), entry.getValue().asText());
}
} else {
configurationFile.putAll(readConfigurationFile(entry.getValue(), prefix + entry.getKey() + "."));
}

View File

@@ -44,20 +44,17 @@ import java.util.Objects;
*
* @author Sebastian Sdorra
*/
public final class ExplodedSmp
{
public final class ExplodedSmp {
private static final Logger logger = LoggerFactory.getLogger(ExplodedSmp.class);
/**
* Constructs ...
*
*
* @param path
* @param plugin
*/
ExplodedSmp(Path path, InstalledPluginDescriptor plugin)
{
ExplodedSmp(Path path, InstalledPluginDescriptor plugin) {
logger.trace("create exploded scm for plugin {} and dependencies {}", plugin.getInformation().getName(), plugin.getDependencies());
this.path = path;
this.plugin = plugin;
@@ -68,16 +65,12 @@ public final class ExplodedSmp
/**
* Creates a new ExplodedSmp object.
*
*
* @param directory directory containing an extracted SCM-Manager plugin.
*
* @return ExplodedSmp object
*
* @throws PluginException if the path does not contain an plugin descriptor
* or the plugin descriptor could not be parsed
* or the plugin descriptor could not be parsed
*/
public static ExplodedSmp create(Path directory)
{
public static ExplodedSmp create(Path directory) {
Path desc = directory.resolve(PluginConstants.FILE_DESCRIPTOR);
return new ExplodedSmp(directory, Plugins.parsePluginDescriptor(desc));
@@ -87,6 +80,7 @@ public final class ExplodedSmp
/**
* Returns {@code true} if the exploded smp contains a core plugin
*
* @return {@code true} for a core plugin
* @since 2.30.0
*/
@@ -98,22 +92,18 @@ public final class ExplodedSmp
/**
* Returns the path to the plugin directory.
*
*
* @return to plugin directory
*/
public Path getPath()
{
public Path getPath() {
return path;
}
/**
* Returns parsed plugin descriptor.
*
*
* @return plugin descriptor
*/
public InstalledPluginDescriptor getPlugin()
{
public InstalledPluginDescriptor getPlugin() {
return plugin;
}
@@ -141,24 +131,19 @@ public final class ExplodedSmp
/**
* Transforms {@link Path} to {@link ExplodedSmp}.
*/
public static class PathTransformer implements Function<Path, ExplodedSmp>
{
public static class PathTransformer implements Function<Path, ExplodedSmp> {
/**
* Transforms {@link Path} to {@link ExplodedSmp}. The path must contain an
* extracted SCM-Manager plugin.
*
*
* @param directory directory containing exploded plugin
*
* @return exploded smp object
*
* @throws PluginException if the path does not contain an extracted
* SCM-Manager plugin.
* SCM-Manager plugin.
*/
@Override
public ExplodedSmp apply(Path directory)
{
public ExplodedSmp apply(Path directory) {
return ExplodedSmp.create(directory);
}
}
@@ -166,9 +151,13 @@ public final class ExplodedSmp
//~--- fields ---------------------------------------------------------------
/** directory */
/**
* directory
*/
private final Path path;
/** plugin object */
/**
* plugin object
*/
private final InstalledPluginDescriptor plugin;
}

View File

@@ -174,13 +174,22 @@ public final class PluginProcessor
Set<ExplodedSmp> installedPlugins = findInstalledPlugins();
logger.debug("found {} installed plugins", installedPlugins.size());
for (ExplodedSmp installedPlugin : installedPlugins) {
if (installedPlugin.getPlugin().getScmVersion() < 3) {
logger.debug("start jakarta transformation of already installed plugin: {}", installedPlugin.getPlugin().getInformation().getName());
PluginTransformer.transform(installedPlugin.getPath());
}
}
Set<ExplodedSmp> newlyInstalledPlugins = installPending(installedPlugins);
logger.debug("finished installation of {} plugins", newlyInstalledPlugins.size());
for (ExplodedSmp newInstalledSmp : newlyInstalledPlugins) {
PluginTransformer.transform(newInstalledSmp.getPath());
if (newInstalledSmp.getPlugin().getScmVersion() < 3) {
logger.debug("start jakarta transformation of newly installed smp: {}", newInstalledSmp.getPlugin().getInformation().getName());
PluginTransformer.transform(newInstalledSmp.getPath());
}
}
logger.debug("finished jakarta transformation of {} plugins", newlyInstalledPlugins.size());
Set<ExplodedSmp> plugins = concat(installedPlugins, newlyInstalledPlugins);

View File

@@ -40,7 +40,7 @@ import java.util.List;
*/
public final class PluginTree {
private static final int SCM_VERSION = 2;
private static final int SCM_VERSION = 3;
/**
* the logger for PluginTree
@@ -72,16 +72,6 @@ public final class PluginTree {
private void checkIfSupported(ExplodedSmp smp) {
InstalledPluginDescriptor plugin = smp.getPlugin();
if (plugin.getScmVersion() != SCM_VERSION) {
throw new PluginException(
String.format(
"scm version %s of plugin %s does not match, required is version %s",
plugin.getScmVersion(), plugin.getInformation().getId(), SCM_VERSION
)
);
}
checkIfConditionsMatch(smp, plugin);
}

View File

@@ -39,11 +39,9 @@ import java.nio.file.Path;
import java.util.Set;
/**
*
* @author Sebastian Sdorra
*/
public final class PluginsInternal
{
public final class PluginsInternal {
/**
* the logger for PluginsInternal
@@ -51,30 +49,27 @@ public final class PluginsInternal
private static final Logger logger =
LoggerFactory.getLogger(PluginsInternal.class);
private PluginsInternal() {}
private PluginsInternal() {
}
public static Set<InstalledPlugin> collectPlugins(ClassLoaderLifeCycle classLoaderLifeCycle,
Path directory)
throws IOException
{
throws IOException {
PluginProcessor processor = new PluginProcessor(classLoaderLifeCycle, directory);
return processor.collectPlugins(classLoaderLifeCycle.getBootstrapClassLoader());
}
public static File createPluginDirectory(File parent, InstalledPluginDescriptor plugin)
{
public static File createPluginDirectory(File parent, InstalledPluginDescriptor plugin) {
PluginInformation info = plugin.getInformation();
return new File(parent, info.getName());
}
public static void extract(SmpArchive archive, String checksum,
File directory, File checksumFile, boolean core)
throws IOException
{
if (directory.exists())
{
File directory, File checksumFile, boolean core)
throws IOException {
if (directory.exists()) {
logger.debug("delete directory {} for plugin extraction",
archive.getPlugin().getInformation().getName(false));
IOUtil.delete(directory);
@@ -87,31 +82,25 @@ public final class PluginsInternal
archive.extract(directory);
Files.write(checksum, checksumFile, Charsets.UTF_8);
if (core)
{
if (!new File(directory, PluginConstants.FILE_CORE).createNewFile())
{
if (core) {
if (!new File(directory, PluginConstants.FILE_CORE).createNewFile()) {
throw new IOException("could not create core plugin marker");
}
}
}
public static Iterable<InstalledPluginDescriptor> unwrap(Iterable<InstalledPlugin> wrapped)
{
public static Iterable<InstalledPluginDescriptor> unwrap(Iterable<InstalledPlugin> wrapped) {
return Iterables.transform(wrapped, new Unwrap());
}
public static File getChecksumFile(File pluginDirectory)
{
public static File getChecksumFile(File pluginDirectory) {
return new File(pluginDirectory, PluginConstants.FILE_CHECKSUM);
}
private static class Unwrap implements Function<InstalledPlugin, InstalledPluginDescriptor>
{
private static class Unwrap implements Function<InstalledPlugin, InstalledPluginDescriptor> {
@Override
public InstalledPluginDescriptor apply(InstalledPlugin wrapper)
{
@Override
public InstalledPluginDescriptor apply(InstalledPlugin wrapper) {
return wrapper.getDescriptor();
}
}

View File

@@ -68,7 +68,7 @@ public class DefaultCentralWorkQueue implements CentralWorkQueue, Closeable {
@Inject
public DefaultCentralWorkQueue(
@ConfigValue(key = "centralWorkQueue.workers", defaultValue = "8", description = "") Integer workers,
@ConfigValue(key = "centralWorkQueue.workers", defaultValue = "0", description = "Sets the number of threads for the central work queue. If set to 0, the default will be used.") Integer workers,
Injector injector,
Persistence persistence,
MeterRegistry meterRegistry

View File

@@ -34,11 +34,8 @@ public class ThreadCountProvider implements IntSupplier {
private static final Logger LOG = LoggerFactory.getLogger(ThreadCountProvider.class);
@VisibleForTesting
static final String PROPERTY = "scm.central-work-queue.workers";
private final IntSupplier cpuCountProvider;
private final Integer workers;
private final int workers;
public ThreadCountProvider(Integer workers) {
this(() -> Runtime.getRuntime().availableProcessors(), workers);
@@ -47,18 +44,24 @@ public class ThreadCountProvider implements IntSupplier {
@VisibleForTesting
ThreadCountProvider(IntSupplier cpuCountProvider, Integer workers) {
this.cpuCountProvider = cpuCountProvider;
this.workers = workers;
this.workers = sanitizeWorkerCount(workers);
}
@Override
public int getAsInt() {
if (isInvalid(workers)) {
return workers;
}
private int sanitizeWorkerCount(Integer workers) {
if (workers == null || workers == 0) {
return deriveFromCPUCount();
} else if (isInvalid(workers)) {
LOG.warn(
"config value {} contains a invalid value {}, fall back and derive worker count from cpu count",
"central-work-queue.workers", workers
"config value 'centralWorkQueue.workers' contains an invalid value {}, fall back and derive worker count from cpu count", workers
);
return deriveFromCPUCount();
}
LOG.debug("using {} workers for central work queue", workers);
return workers;
}
@@ -69,8 +72,10 @@ public class ThreadCountProvider implements IntSupplier {
private int deriveFromCPUCount() {
int cpus = cpuCountProvider.getAsInt();
if (cpus > 1) {
LOG.debug("using 4 workers for central work queue");
return 4;
}
LOG.debug("using 2 workers for central work queue");
return 2;
}
}

View File

@@ -77,4 +77,12 @@ class ConfigurationResolverTest {
assertThat(WebappConfigProvider.resolveAsBoolean("redirect")).contains(true);
assertThat(WebappConfigProvider.resolveAsInteger("https.port")).contains(42);
}
@Test
void shouldReadNullValuesFromConfigYaml() {
new ConfigurationResolver(Collections.emptyMap(), "sonia/scm/plugin/configWithNull.yml");
assertThat(WebappConfigProvider.resolveAsBoolean("redirect")).contains(true);
assertThat(WebappConfigProvider.resolveAsInteger("https.keyType")).isEmpty();
}
}

View File

@@ -45,7 +45,6 @@ import static org.hamcrest.Matchers.contains;
import static org.hamcrest.Matchers.containsInAnyOrder;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertThat;
import static org.junit.Assert.assertTrue;
/**
* @author Sebastian Sdorra
@@ -104,15 +103,6 @@ public class PluginTreeTest {
assertThat(nodes, containsInAnyOrder("a", "b", "c"));
}
@Test(expected = PluginException.class)
public void testScmVersion() throws IOException {
InstalledPluginDescriptor plugin = new InstalledPluginDescriptor(1, createInfo("a", "1"), null, null, false,
null, null);
ExplodedSmp smp = createSmp(plugin);
new PluginTree(Stage.PRODUCTION, smp).getLeafLastNodes();
}
@Test
public void testSimpleDependencies() throws IOException {
ExplodedSmp[] smps = new ExplodedSmp[]{

View File

@@ -61,7 +61,7 @@ class ThreadCountProviderTest {
}
@ParameterizedTest
@ValueSource(strings = {"-1", "0", "100", "a", ""})
@ValueSource(strings = {"-1", "0", "100"})
void shouldUseDefaultForInvalidValue(String value) {
ThreadCountProvider provider = new ThreadCountProvider(() -> 1, Integer.parseInt(value));
assertThat(provider.getAsInt()).isEqualTo(2);

View File

@@ -0,0 +1,27 @@
#
# MIT License
#
# Copyright (c) 2020-present Cloudogu GmbH and Contributors
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
webapp:
https:
keyType:

View File

@@ -1,63 +0,0 @@
# Webapp config options
#### Disables feedback links on frontend page footer
disableFeedback: false
#### Makes jwt token endless to prevent that logged-in users get automatically logged off. Warning: Enabling this option can lead to security issue.
endlessJwt: false
#### Set app stage, e.g. DEVELOPMENT, PRODUCTION, TESTING
stage: PRODUCTION
#### Override app version (for testing purposes only)
versionOverride:
#### Sets path for the working directory which is used for internal repository operations. Empty string defaults to java tmpdir
workdir:
#### Parent config to configure caches
cache:
- dataFileCache:
enabled: true
- externalGroups:
maximumSize: 42
#### Username of initial admin user
initialUser
#### Password of initial admin user
initialPassword
#### skip initial admin creation
skipAdminCreation: false
#### Number of async threads
asyncThreads: 4
#### Max seconds to abort async execution
maxAsyncAbortSeconds: 60
#### Central work queue
central-work-queue:
workers: 4
#### Strategy for the working copy pool implementation [NoneCachingWorkingCopyPool, SimpleCachingWorkingCopyPool]
workingCopyPoolStrategy: NoneCachingWorkingCopyPool
#### Amount of "cached" working copies
workingCopyPoolSize: 5
#### Cache xml stores in memory
storeCache:
enabled: true