Welcome to the Duplicati Documentation! This site contains documentation for using the open source Duplicati client, including best practice, pro tips, and trouble shooting.

Jump right in

With a configured backup, you can have a schedule that runs the backup automatically each day. Having the backup run automatically is recommended because it makes it less likely that the backups are not recent if they are needed.

Even if the backup already has a schedule there may be times where you want to manually run a backup. If you have just configured a backup, you may want to run it ahead of the scheduled next run. If you are within the UI you can click the "Run now" link for the backup.

Once the backup is running, the top area will act as a progress bar that shows how the backup progresses. Note that the first run of a backup is the slowest run because it needs to process every file and folder that is part of the source. On later runs it will recognize what parts have changed and only process the new and changed data.

After running a backup, the view will change slightly and show some information about the backup.

The secret provider was introduced in Duplicati version 2.0.9.109 and aims to reduce the possibility of leaking passwords from Duplicati by not storing the passwords inside Duplicati.

To start using a secret provider you need to set only a single option:

--secret-provider=<url>

This will make the secret provider available for the remainder of the application.

You can then insert placeholder values where you want secrets to appear but without storing the actual secret in Duplicati. For commandline users, the secrets can appear in both the backend destination or in the options.

As an example:

duplicati backup \
  s3://example-bucket?auth-username=$s3-user&password=$s3-pass \
  --passphrase=$passphrase 

The secret provider will find the three keys prefixed with $ and look them up with the secret provider. The provider will then be invoked to obtain the real values and the values will be replaced before running the operation. If the secret provider has these values:

s3-user=user
s3-pass=pass
passphrase=my-password

The example from above will then be updated internally, but without having the keys written on disk:

duplicati backup \
  s3://example-bucket?auth-username=user&password=pass \
  --passphrase=my-password

To ensure you never run with an empty string or a placeholder instead of the real value, all values requested needs to be in the storage provider, or the operation will fail with a message indicating which key was not found.

When using Duplicati, you need to decide on what type of instance you want to use. Duplicati is designed to be flexible and work with many different setups, but generally you can use this overview to decide what is best for you:

The TrayIcon

When running, the TrayIcon gives a visual indication of the current status, and provides access to the visual user interface by opening a browser window.

The Server

When running the server it will emit log messages to the system log and it will expose a web server that can be accessed via a browser. Beware that if you are running the Server as root/Administrator you are also running a web server with the same privileges that you need to protect.

The Agent

If you have multiple machines to manage, using the console enables you to access all the backups, settings, logs, controls, etc. from one place.

The Command Line Interface (CLI)

Mixing types

For some additional flexibility in configurations it is also possible to combine the different types in some ways.

Combining Server and TrayIcon

It the server is used primarily to elevate privileges, it is possible to have the TrayIcon run in the local user desktop and connect to an already running Server. To do this, change the TrayIcon commandline and add additional arguments:

duplicati --no-hosted-server 
  --hosturl=http://localhost:8200 
  --webservice-password=<password>

The --no-hosted-server argument disables launching another (competing) server, and the two other arguments will give information on how to reach the running server.

Triggering Server jobs externally

Using the CLI for on Server backups

Setting up and using either of the vaults described here is outside the scope of this document.

HashiCorp Vault

To connect to the vault, provide the url as part of the configuration:

--secret-provider=
  hcv://localhost:8200?token=<token>&secrets=app1,app2

The url is converted to the url used to connect to the vault (e.g., https://localhost:8200 in this example). The token is used to authenticate, and the secrets are the vaults that secrets are read from.

In the cloud-based offering the "secrets" values shown here are referred to as Apps and in the CLI as "mount points". When more than one value is supplied, the vaults are tried in order and stops once all secrets are resolved. This means that if the same secret key is found in two vaults, the value will be used from the first vault examined.

Other options for hcv://

For development purposes, the url can use a http connection by setting &connection-type=http , but this should not be used in production.

To connect using a credential pair instead of the token, the credentials can be provided with the values client-id and client-secret , but should be passed via the environment variables:

export HCP_CLIENT_ID=<client-id>
export HCP_CLIENT_SECRET=<secret>

--secret-provider=hcv://localhost:8200?secrets=app1

By default, the key lookup is done case-insensitive but can be toggled case-sensitive with the option &case-sensitive=true.

Amazon Secret Manager

export AWS_ACCESS_KEY_ID=<id>
export AWS_SECRET_ACCESS_KEY=<key>

--secret-provider=awssm://?region=us-east-1&secrets=vault1,vault2

The secrets values name the vaults to use (called "Secret Name" in the AWS Console). When more than one value is supplied, the vaults are tried in order and stops once all secrets are resolved. This means that if the same secret key is found in two vaults, the value will be used from the first vault examined.

Instead of suppling the region the entire service point url can also be provided via &service-url=.

By default, the key lookup is done case-insensitive but can be toggled case-sensitive with the option &case-sensitive=true.

Google Cloud Secret Manager

--secret-provider=gcsm://?project-id=<projectid>

--secret-provider=gcsm://?project-id=<projectid>&token=<token>

Additional options for gcsm://

By default, the screts are accessed with the version set to latest but this can be changed with &version=. Finally, the communication protocol can be changed from gRPC to https with by adding &api-type=Rest.

Azure Key Vault

--secret-provider=azkv://?keyvault-name=keyvault

Instead of supplying the name of the keyvault, the full vault url can be supplied with &vault-uri=.

Manually authenticating

Instead of relying on the autmated login handling, it is possible to authenticate with either a client credential, or a username/password pair.

For authenticating with client credentials, use:

--secret-provider=azkv://?keyvault-name=keyvault
  &auth-type=ClientSecret
  &tenant-id=<tenantid>
  &client-id=<clientid>
  &client-secret=<secret>

And for username/password, use:

--secret-provider=azkv://?keyvault-name=keyvault
  &auth-type=UsernamePassword
  &tenant-id=<tenantid>
  &client-id=<clientid>
  &username=<username>
  &password=<password>

The Duplicati package types

For desktop and laptop users, the most common application type is called the "GUI" package, which is short for Graphical User Interface. The GUI package includes the core components, a webserver to show the user interface and a tray icon (also called a status bar icon).

For users installing in environments without a desktop or screen, there are also commandline only, remote management and Docker versions. Depending on your setup, you may also want to use one of those packages on a desktop or laptop.

This page covers only the GUI installation.

Jump to the section that is relevant to you:

Install Duplicati on Windows

The most common installation format on Windows is the MSI package. To install on Windows you need to know what kind of processor is on your system. If you are unsure, you are most likely using the 64-bit processor, also known as x64. There is also a version supporting Arm64 processors, and a version for legacy 32-bit Windows called x86.

Install Duplicati on MacOS

For MacOS the common installation method is to use a DMG file with the application file. Most modern MacOS machines are using the Apple Silicon which is called Arm64 in Duplicati's packages. If you are on an older Mac that has a 64-bit Intel processor, you can use the x64 package instead.

Install Duplicati on Linux

Most Linux distributions work well with Duplicati but there are only packages for Debian based distributions (Ubuntu, Mint, etc) and for RedHat based based distributions (Fedora, SUSE, etc). For other distributions you may need to manually install some dependencies.

For Linux distributions there are packages for the most common 64-bit based system with x64, support for Arm64 and the predecessor Arm7 aka ArmHF which are commonly found in NAS boxes and the older Raspberry Pi series.

Install on Debian-based Linux (Ubuntu, Mint, etc)

To install Duplicati on a Debian based system, first download the .deb package matching the system architecture, then run:

sudo dpkg -i duplicati-version-arch.deb

Install on RedHat-based Linux (Fedora, SUSE, etc)

To install Duplicati on a RedHat-based system, first download the .rpm package matching the system architecture, then run:

sudo yum -i duplicati-version-arch.rpm

Install on another Linux distribution

For other linux distributions you can use the .zip file that matches your system architecture. Inside the zip files are all the binaries that are needed, and you can simply place them in a folder that works for your system. Generally, all dependecies are inlcuded in the packages so unless you are using a very slimmed down setup, it should work without additional packages.

In the UI, start by clicking "Add backup", and choose the option "Configure a new backup":

To set up a new backup there are some details that are required, and these are divided into 5 steps:

1. Basic configuration

For the basic configuration, you need to provide a name and setup encryption:

The name and description fields can be any text you like, and is only used to display the backup configuration in lists so you can differentiate if you have multiple backups.

The encryption setup allows you to choose an encryption method and a passphrase. Encryption adds a minor overhead to the processing, but is generally a good idea to add. If you opt out of encryption, make sure you control the storage destination and have adequate protections in place.

To avoid weak passphrases, Duplicati has a built-in passphrase generator as well as a passphrase strength measurer.

2. Storage destination

The storage destination is arguably the most technical step because it is where you specify how to connect to the storage provider you want to hold your information. Some destinations require only a single setting, where others require multiple.

When the details are entered, it is recommended that you use the "Test" button which will perform some connection tests that helps reveal any issues with the entered information.

When the destination is configured as desired, click the "Next" button.

3. Source data

In the third step you need to define what data should be backed up. This part depends on your use. If you are a home user, you may want to back up images and documents. An IT professional may want to back up databases.

In the source picker view you can choose the files and folders you would like to back up. If you pick a folder, all subfolders and files in that folder will be included. You can use the UI to uncheck some items that you want to exclude, and they will show up with a red X.

Once you are satisfied with the source view, click the "Next" button to continue to the schedule step.

4. Schedule

Having an outdated backup is rarely an ideal solution, but remembering to run backups is also tedious and easy to forget. To ensure you have up-to-date backups, there is a built-in scheduler in Duplicati that you can enable to have Duplicati run automatically.

Once satisfied with the schedule, click "Next".

5. Retention and miscelaneous

Even though Duplicati has deduplication and compression to reduce the stored data, it is inevitable that old data is stored that will take up space, but is not needed for restore. In this final configuration step you can decide when old versions are removed and what size of files to store on the destination.

For the retention setting, it is inveitable that the backups will grow as new and changed data is added to the backups. If nothing is ever deleted, the backup size will keep growing in size. With the retention settings you can choose how to automatically remove older versions.

The setting "Smart backup retention" is meant to be useful for most users where it keeps one daily backup and then gradually fewer versions going back in time.

Once you are satisfied with the settings, click the "Save" button.

You have now configured your backup! 🎉

Sharing the secret provider

This sharing simplifies the setup by only having a single secret provider configuration and then letting each of the other parts access secrets without further configuration. If needed, the secret providers can be specified for the individual backups, such that it is possible to opt-out of using the shared secret provider.

How to avoid passing credentials on the commandline

To make passing arguments to the application a bit harder to obtain, the value for --secret-provider is treated as an environment variable if:

• It starts with $ optionally with curly brackets {}:

  • $secretprovider

  • ${secretprovider}

• If it starts and ends with % :

  • %secretprovider%

No expansion is done on environment variables, so the entire provider string is required to be set as an environment variable.

How to protect against secret provider outages

If you run an operation and the secret provider is unavailable when the secrets are requested, the operation will fail. For most uses the occurence of an outage is so rare that this situation is acceptable.

However, for some uses it is important that the backups keep running, even in the face out outages. To handle this need, Duplicati supports an optional cache strategy:

Storing the secrets somewhere makes it more likely that it is eventually leaked. For that reason, the default is to use the cache setting None which turns off the caching fully and only relies on the provider.

Finally, the Persistent option will write secrets to disk, so it can handle situations where the provider is unavailable during startup, or where a shared provider does not work.

As the purpose of the secret provider is to prevent the secrets from being written to disk, the secrets are written to disk using a passhprase derived from the secret provider url. If the secret provider url does not contain a strong secret already, it is possible to add any parameter to the url to increase the strength of the key.

If the secret provider url changes, it is no longer possible to retrieve the cached values, and the next run will fail if the provider is unavailable, but will otherwise write a new encrypted cache file to disk.

The most important reason to make a backup is the ability to recover the data at a later stage, usually due to some unforseen incident. Depending on the incident, the original configuration may not be available.

To start a restore process in Duplicati, start on the "Restore" page.

The restore and browsing process are fastest when using a configured backup, because Duplicati can query a local database with information. If the local database is not present, Duplicati needs to fetch enough information from the remote storage to build a partial database when performing the restore.

Direct restore from backup files

To restore files from the backup, Duplicati needs only to know how to access the files and the encryption passphrase (if any). If you do not have the passphrase, it is not possible to restore.

To restore directly from the backup files, the first step is to provide the destination details. These details are the same as you supplied initially when creating the backup. if you are using a cloud provider, you can usually get the needed information via your account on the vendors website.

Once the details are entered, it is recommended to use the "Test connection" button to ensure that the connection is working correctly. Then click the "Next" button.

Restore from configuration

In the dialog, provide the exported configuration file and the configuration file's encryption passphrase. Note that the passphrase the configuration file is encrypted with is not neccesarily the same as the passphrase used to encrypt the backup with.

Once the configuration is correct, click the "Import" button and you are given the option to correct the settings before starting the restore process. If you do not need to change anything, click "Next" and then "Connect".

Choosing files to restore

Once Duplicati has a connection to the remote destination it will find all the backups that were made. It will then choose the most recent version and list the files from within that version. Use the "Restore from" dropdown to select the version to restore from, and use the search field to highlight files matching the expression. Click the "Search" button to list only files matching the criteria.

Check the files or folders that you want to restore and then click "Continue".

Choosing restore options

When restoring there are a few options that control how the files are restored.

If you want to restore a file to a previous state, you can leave the settings to their defaults. If you are unsure if you want to revert, or need to examine the files before replacing the current versions, you can choose to restore to another destination. If the folder you are restoring to is not empty, you can choose to store multiple versions of the files by appending the restore timestamp to the filename. This is especially useful if you are restoring multiple versions into a target folder for comparison.

Duplicati will not restore permissions by default because the users and groups that were present on the machine that made the backup may not be present on the machine being restored to. Restoring the permissions can cause you to be unable to access the restored files, if your user does not have the necessary permissions.

When satisfied with the settings, click the "Restore" button and the restore process will restore the files.

The Environment Variable provider

The simplest provider is the env:// provider, which simply extracts environment variables and replaces those. There is no configuration needed for this provider, and the syntax for adding it is simply:

The File Secret provider

The file-secret:// provider supports reading secrets from a file containing a JSON encoded dictionary of key/value pairs. As an example, a file could look like:

Credential Manager (Windows)

Using libsecret (Linux)

Using the pass secret provider (Linux)

Using the KeyChain (MacOS)

For more advanced uses the options account and service can be used to narrow down what secrets can be extracted.

Despite all efforts to make Duplicati as robust as possible against failures, it is not possible to handle every possible problem that may arise after the initial setup. Common failure causes is revoked credentials, filled storage, missing provider updates, etc.

To avoid discovering too late that the backup had stopped working for some reason, it is highly recommended to set up automated monitoring of backups. Duplicati has a number of ways that you can use to send reports into a monitoring solution:

Register the local installation

In a default installation, Duplicati will serve up a UI using an internal webserver. This setup works well for workstations and laptops but can be challenging when the machine is not always connected to a display. To securely connect the instance to the Duplicati Console, go to the settings page and find the "Remote access control" section.

Click the button "Register for remote control" to start the registration process. After a short wait, the machine will obtain a registration link:

Registering on the Console

Click the registration link to open a browser and claim the machine in the Duplicati Console:

Click "Register machine" to add it to your account, then return to the Duplicati Settings page where the machine is now registered and ready to connect:

Click the "Enable remote control" button and see the machine is now connected to the Duplicati Console:

Connecting to the machine

You can now click "Connect" to access the machine directly from the portal!

As long as the Agent is not registered, restarting it will make it attempt to connect again.

Simplified registration

Any machine can now use this pre-authorized url to add machines to your organization in the Console. You can click the "Copy" button to get the link to your clipboard and paste it in when registering a machine. Do not share this link with anyone as it could allow them to add machines to your account.

To revoke a link, simply delete it from within the portal. This will prevent new machines from registering, but existing registered machines will remain there.

With the registration link, start the Agent with a commandline such as:

duplicati-agent --registration-url=<copied-url>

This will cause the Agent to immediately show up in the Console. Future invocations of the agent will not require the registration url, but should the Agent somehow be de-registered, it will re-reregister if the url is set and the link is still valid.

Registration with deployment

{
  "args": {
    "agent": [ "--registration-url=<copied-url>" ],
  }
}

These options allow you to integrate custom scripts with Duplicati operations, providing automation capabilities before and after backups, restores, or other tasks.

Pre and Post Operation Scripts Run custom scripts before an operation starts or after it completes. Use these to perform preparation tasks (like database locking), cleanup actions, or to trigger notifications based on operation results.

Control Flow Management Configure whether operations should continue or abort based on script execution status, with customizable timeout settings to prevent operation blocking.

Script Output Processing Post-operation scripts receive operation results via standard output, enabling conditional processing based on success or failure.

Scripting options

--run-script-before(Path) Run a script on startup. Executes a script before performing an operation. The operation will block until the script has completed or timed out.

--run-script-after(Path) Run a script on exit. Executes a script after performing an operation. The script will receive the operation results written to stdout.

--run-script-before-required(Path) Run a required script on startup. Executes a script before performing an operation. The operation will block until the script has completed or timed out. If the script returns a non-zero error code or times out, the operation will be aborted.

--run-script-timeout(Timespan) Sets the script timeout. Sets the maximum time a script is allowed to execute. If the script has not completed within this time, it will continue to execute but the operation will continue too, and no script output will be processed. Default value: 60s

Script Output Integration with Duplicati Logging

You can add custom entries directly to Duplicati's log system from your scripts by using special prefixes in stdout messages. This allows script events to appear in both the Duplicati Log and Reports alongside native application events.

Supported Log Level Prefixes:

• LOG:INFO - For general information and success notifications

• LOG:WARN - For potential issues that didn't prevent completion

• LOG:ERROR - For critical failures that require attention

Example Usage:

echo "LOG:INFO Preparation tasks completed successfully"
echo "LOG:WARN Database backup older than 24 hours detected"
echo "LOG:ERROR Unable to lock database, backup may contain inconsistent data"

These messages will be captured with their appropriate severity levels and integrated into Duplicati's logging system, making script events traceable within the same monitoring interfaces you use for Duplicati itself.

Sample Scripts

run-script-example.bat (Windows)

@echo off

REM ###############################################################################
REM How to run scripts before or after backups
REM ###############################################################################

REM Duplicati is able to run scripts before and after backups. This 
REM functionality is available in the advanced options of any backup job (UI) or
REM as option (CLI). The (advanced) options to run scripts are
REM --run-script-before = <filename>
REM --run-script-before-required = <filename>
REM --run-script-timeout = <time>
REM --run-script-after = <filename>
REM
REM --run-script-before = <filename>
REM Duplicati will run the script before the backup job and waits for its 
REM completion for 60 seconds (default timeout value). After a timeout a 
REM warning is logged and the backup is started.
REM The following exit codes are supported:
REM
REM - 0: OK, run operation
REM - 1: OK, don't run operation
REM - 2: Warning, run operation
REM - 3: Warning, don't run operation
REM - 4: Error, run operation
REM - 5: Error don't run operation
REM - other: Error don't run operation
REM
REM --run-script-before-required = <filename>
REM Duplicati will run the script before the backup job and wait for its 
REM completion for 60 seconds (default timeout value). The backup will only be
REM run if the script completes with the exit code 0. Other exit codes or a
REM timeout will cancel the backup job.
REM
REM --run-script-timeout = <time>
REM Specify a new value for the timeout. Default is 60s. Accepted values are
REM e.g. 30s, 1m15s, 1h12m03s, and so on. To turn off the timeout set the value 
REM to 0. Duplicati will then wait endlessly for the script to finish.
REM
REM --run-script-after = <filename>
REM Duplicati will run the script after the backup job and wait for its 
REM completion for 60 seconds (default timeout value). After a timeout a 
REM warning is logged.
REM The same exit codes as in --run-script-before are supported, but
REM the operation will always continue (i.e. 1 => 0, 3 => 2, 5 => 4)
REM as it has already completed so stopping it during stop is useless.



REM ###############################################################################
REM Changing options from within the script 
REM ###############################################################################

REM Within a script, all Duplicati options are exposed as environment variables
REM with the prefix "DUPLICATI__". Please notice that the dash (-) character is
REM not allowed in environment variable keys, so it is replaced with underscore
REM (_). For a list of available options, have a look at the output of
REM "duplicati.commandline.exe help".
REM
REM For instance the current value of the option --encryption-module can be 
REM accessed in the script by
REM ENCRYPTIONMODULE=%DUPLICATI__encryption_module%

REM All Duplicati options can be changed by the script by writing options to
REM stdout (with echo or similar). Anything not starting with a double dash (--)
REM will be ignored:
REM echo "Hello! -- test, this line is ignored"
REM echo --new-option=This will be a setting

REM Filters are supplied in the DUPLICATI__FILTER variable.
REM The variable contains all filters supplied with --include and --exclude,
REM combined into a single string, separated with semicolon (;).
REM Filters set with --include will be prefixed with a plus (+),
REM and filters set with --exclude will be prefixed with a minus (-).
REM
REM Example:
REM     --include=*.txt --exclude=[.*\.abc] --include=*
REM 
REM Will be encoded as:
REM     DUPLICATI__FILTER=+*.txt;-[.*\.abc];+*
REM
REM You can set the filters by writing --filter=<new filter> to stdout.
REM You may want to append to the existing filter like this:
REM     echo "--filter=+*.123;%DUPLICATI__FILTER%;-*.xyz"


REM ###############################################################################
REM Special Environment Variables
REM ###############################################################################

REM DUPLICATI__EVENTNAME
REM Eventname is BEFORE if invoked as --run-script-before, and AFTER if 
REM invoked as --run-script-after. This value cannot be changed by writing
REM it back!

REM DUPLICATI__OPERATIONNAME
REM Operation name can be any of the operations that Duplicati supports. For
REM example it can be "Backup", "Cleanup", "Restore", or "DeleteAllButN".
REM This value cannot be changed by writing it back!

REM DUPLICATI__RESULTFILE
REM If invoked as --run-script-after this will contain the name of the 
REM file where result data is placed. This value cannot be changed by 
REM writing it back!

REM DUPLICATI__REMOTEURL
REM This is the remote url for the target backend. This value can be changed by
REM echoing --remoteurl = "new value".

REM DUPLICATI__LOCALPATH
REM This is the path to the folders being backed up or restored. This variable
REM is empty operations  other than backup or restore. The local path can 
REM contain : to separate multiple folders. This value can be changed by echoing
REM --localpath = "new value".

REM DUPLICATI__PARSED_RESULT
REM This is a value indicating how well the operation was performed.
REM It can take the values: Unknown, Success, Warning, Error, Fatal.


REM ###############################################################################
REM Example script
REM ###############################################################################

REM We read a few variables first.
SET EVENTNAME=%DUPLICATI__EVENTNAME%
SET OPERATIONNAME=%DUPLICATI__OPERATIONNAME%
SET REMOTEURL=%DUPLICATI__REMOTEURL%
SET LOCALPATH=%DUPLICATI__LOCALPATH%

REM Basic setup, we use the same file for both before and after,
REM so we need to figure out which event has happened
if "%EVENTNAME%" == "BEFORE" GOTO ON_BEFORE
if "%EVENTNAME%" == "AFTER" GOTO ON_AFTER

REM This should never happen, but there may be new operations
REM in new version of Duplicati
REM We write this to stderr, and it will show up as a warning in the logfile
echo Got unknown event "%EVENTNAME%", ignoring 1>&2
GOTO end

:ON_BEFORE

REM If the operation is a backup starting, 
REM then we check if the --dblock-size option is unset
REM or 50mb, and change it to 25mb, otherwise we 
REM leave it alone

IF "%OPERATIONNAME%" == "Backup" GOTO ON_BEFORE_BACKUP
REM This will be ignored
echo Got operation "%OPERATIONNAME%", ignoring
GOTO end

:ON_BEFORE_BACKUP
REM Check if volsize is either not set, or set to 50mb
IF "%DUPLICATI__dblock_size%" == "" GOTO SET_VOLSIZE
IF "%DUPLICATI__dblock_size%" == "50mb" GOTO SET_VOLSIZE

REM We write this to stderr, and it will show up as a warning in the logfile
echo Not setting volumesize, it was already set to %DUPLICATI__dblock_size% 1>&2
GOTO end

:SET_VOLSIZE
REM Write the option to stdout to change it
echo --dblock-size=25mb
GOTO end


:ON_AFTER

IF "%OPERATIONNAME%" == "Backup" GOTO ON_AFTER_BACKUP
REM This will be ignored
echo "Got operation "%OPERATIONNAME%", ignoring	"
GOTO end

:ON_AFTER_BACKUP

REM Basic email setup		
SET EMAIL="admin@example.com"		
SET SUBJECT="Duplicati backup"

REM We use a temp file to store the email body
SET MESSAGE="%TEMP%\duplicati-mail.txt"
echo Duplicati finished a backup. > %MESSAGE%
echo This is the result : >> %MESSAGE%
echo.  >> %MESSAGE%

REM We append the results to the message
type "%DUPLICATI__RESULTFILE%" >> %MESSAGE%

REM If the log-file is enabled, we append that as well
IF EXIST "%DUPLICATI__log_file%" type "%DUPLICATI__log_file%" >> %MESSAGE%

REM If the backend-log-database file is enabled, we append that as well
IF EXIST "%DUPLICATI__backend_log_database%" type "%DUPLICATI__backend_log_database%" >> %MESSAGE%

REM Finally send the email using a fictive sendmail program
sendmail %SUBJECT% %EMAIL% < %MESSAGE%

GOTO end

:end

REM We want the exit code to always report success.
REM For scripts that can abort execution, use the option
REM --run-script-on-start-required = <filename> when running Duplicati
exit /B 0

run-script-example.sh (Linux)

#!/bin/bash

###############################################################################
# How to run scripts before or after backups
###############################################################################

# Duplicati is able to run scripts before and after backups. This 
# functionality is available in the advanced options of any backup job (UI) or
# as option (CLI). The (advanced) options to run scripts are
# --run-script-before = <filename>
# --run-script-before-required = <filename>
# --run-script-timeout = <time>
# --run-script-after = <filename>
#
# --run-script-before = <filename>
# Duplicati will run the script before the backup job and waits for its 
# completion for 60 seconds (default timeout value). After a timeout a 
# warning is logged and the backup is started.
# The following exit codes are supported:
#
# - 0: OK, run operation
# - 1: OK, don't run operation
# - 2: Warning, run operation
# - 3: Warning, don't run operation
# - 4: Error, run operation
# - 5: Error don't run operation
# - other: Error don't run operation
#
# --run-script-before-required = <filename>
# Duplicati will run the script before the backup job and wait for its 
# completion for 60 seconds (default timeout value). The backup will only be
# run if the script completes with the exit code 0. Other exit codes or a
# timeout will cancel the backup job.
#
# --run-script-timeout = <time>
# Specify a new value for the timeout. Default is 60s. Accepted values are
# e.g. 30s, 1m15s, 1h12m03s, and so on. To turn off the timeout set the value 
# to 0. Duplicati will then wait endlessly for the script to finish.
#
# --run-script-after = <filename>
# Duplicati will run the script after the backup job and wait for its 
# completion for 60 seconds (default timeout value). After a timeout a 
# warning is logged.
# The same exit codes as in --run-script-before are supported, but
# the operation will always continue (i.e. 1 => 0, 3 => 2, 5 => 4)
# as it has already completed so stopping it during stop is useless.


###############################################################################
# Changing options from within the script 
###############################################################################

# Within a script, all Duplicati options are exposed as environment variables
# with the prefix "DUPLICATI__". Please notice that the dash (-) character is
# not allowed in environment variable keys, so it is replaced with underscore
# (_). For a list of available options, have a look at the output of
# "duplicati.commandline.exe help".
#
# For instance the current value of the option --encryption-module can be 
# accessed in the script by
# ENCRYPTIONMODULE=$DUPLICATI__encryption_module

# All Duplicati options can be changed by the script by writing options to
# stdout (with echo or similar). Anything not starting with a double dash (--)
# will be ignored:
# echo "Hello! -- test, this line is ignored"
# echo "--new-option=\"This will be a setting\""

# Filters are supplied in the DUPLICATI__FILTER variable.
# The variable contains all filters supplied with --include and --exclude,
# combined into a single string, separated with colon (:).
# Filters set with --include will be prefixed with a plus (+),
# and filters set with --exclude will be prefixed with a minus (-).
#
# Example:
#     --include=*.txt --exclude=[.*\.abc] --include=*
# 
# Will be encoded as:
#     DUPLICATI__FILTER=+*.txt:-[.*\.abc]:+*
#
# You can set the filters by writing --filter=<new filter> to stdout.
# You may want to append to the existing filter like this:
#     echo "--filter=+*.123:%DUPLICATI__FILTER%:-*.xyz"


###############################################################################
# Special Environment Variables
###############################################################################

# DUPLICATI__EVENTNAME
# Eventname is BEFORE if invoked as --run-script-before, and AFTER if 
# invoked as --run-script-after. This value cannot be changed by writing
# it back!

# DUPLICATI__OPERATIONNAME
# Operation name can be any of the operations that Duplicati supports. For
# example it can be "Backup", "Cleanup", "Restore", or "DeleteAllButN".
# This value cannot be changed by writing it back!

# DUPLICATI__RESULTFILE
# If invoked as --run-script-after this will contain the name of the 
# file where result data is placed. This value cannot be changed by 
# writing it back!

# DUPLICATI__REMOTEURL
# This is the remote url for the target backend. This value can be changed by
# echoing --remoteurl = "new value".

# DUPLICATI__LOCALPATH
# This is the path to the folders being backed up or restored. This variable
# is empty operations  other than backup or restore. The local path can 
# contain : to separate multiple folders. This value can be changed by echoing
# --localpath = "new value".

# DUPLICATI__PARSED_RESULT
# This is a value indicating how well the operation was performed.
# It can take the values: Unknown, Success, Warning, Error, Fatal.



###############################################################################
# Example script
###############################################################################

# We read a few variables first.
EVENTNAME=$DUPLICATI__EVENTNAME
OPERATIONNAME=$DUPLICATI__OPERATIONNAME
REMOTEURL=$DUPLICATI__REMOTEURL
LOCALPATH=$DUPLICATI__LOCALPATH

# Basic setup, we use the same file for both before and after,
# so we need to figure out which event has happened
if [ "$EVENTNAME" == "BEFORE" ]
then
	# If the operation is a backup starting, 
	# then we check if the --dblock-size option is unset
	# or 50mb, and change it to 25mb, otherwise we 
	# leave it alone
	
	if [ "$OPERATIONNAME" == "Backup" ]
	then
		if [ "$DUPLICATI__dblock_size" == "" ] || ["$DUPLICATI__dblock_size" == "50mb"]
		then
			# Write the option to stdout to change it
			echo "--dblock-size=25mb"
		else
			# We write this to stderr, and it will show up as a warning in the logfile
			echo "Not setting volumesize, it was already set to $DUPLICATI__dblock_size" >&2
		fi
	else
		# This will be ignored
		echo "Got operation \"OPERATIONNAME\", ignoring"	
	fi

elif [ "$EVENTNAME" == "AFTER" ]
then

	# If this is a finished backup, we send an email
	if [ "$OPERATIONNAME" == "Backup" ]
	then

		# Basic email setup		
		EMAIL="admin@example.com"		
		SUBJECT="Duplicati backup"
		
		# We use a temp file to store the email body
		MESSAGE="/tmp/duplicati-mail.txt"
		echo "Duplicati finished a backup."> $MESSAGE
		echo "This is the result :" >> $MESSAGE
		echo "" >> $MESSAGE

		# We append the result of the operation to the email
		cat "$DUPLICATI__RESULTFILE" >> $MESSAGE

		# If the log-file is enabled, we append it
		if [ -f "$DUPLICATI__log_file" ]
		then
			echo "The log file looks like this: " >> $MESSAGE
			cat "$DUPLICATI__log_file" >> $MESSAGE
		fi
		
		# If the backend-log-database file is enabled, we append that as well
		if [ -f "$DUPLICATI__backend_log_database" ]
		then
			echo "The backend-log file looks like this: " >> $MESSAGE
			cat "$DUPLICATI__backend_log_database" >> $MESSAGE
		fi

		# Finally send the email using /bin/mail
		/bin/mail -s "$SUBJECT" "$EMAIL" < $MESSAGE
	else
		# This will be ignored
		echo "Got operation \"OPERATIONNAME\", ignoring"	
	fi
else
	# This should never happen, but there may be new operations
	# in new version of Duplicati
	# We write this to stderr, and it will show up as a warning in the logfile
	echo "Got unknown event \"$EVENTNAME\", ignoring" >&2
fi

# We want the exit code to always report success.
# For scripts that can abort execution, use the option
# --run-script-before-required = <filename> when running Duplicati
exit 0

Note: it is possible to restore files across operating systems, but due to path differences it is not possible to continue a backup made on Windows on a Linux/MacOS based operating system and vice versa.

Note: do not attempt to run backups from two different machines to the same destination. Before migrating, make sure the previous machine is no longer running backups automatically. If both machines run backups, one instance will detect that the remote destination has been modified and will refuse to continue until the local database has been rebuilt.

Previous machine is still available

If the previous machine is still accessible, you can copy over the contents of the Duplicati folder containing the the configuration database Duplicati-server.sqlite and the other support database. This approach is by far the fastest as Duplicati has all the information and does not need to check up with the remote storage.

Make sure to stop Duplicati before moving in the folder into the same location on the new machine. After moving in the folder, you can start Duplicati again and everything will be working as before. If it has been a while since the previous instance was running, this may trigger the scheduled backups on startup. Use the option --startup-delay=5min to start Duplicati in pause mode for 5 minutes if you want to check up before it starts running.

Backup configurations are available

Once the local database has been recreated, it is then possible to run the backup as before with no modifications required.

Previous machine and configurations are unavailable

If you do not have access to the previous setup, you can still continue the backups, but this requires that you re-create the backups manually. You need at least the storage destination details, the passphrase and to select the sources.

Once the local database has been recreated, it is then possible to run the backup as before with no modifications required.

To use the option, you only need to provide the url to send to:

Besides the URL it is also possible to configure:

• The message body and type (JSON is supported)

• The HTTP verb used

• Conditions on when to send emails

• Conditions on what log elements to include

New in 2.0.9.106

You can now specify multiple urls, using the options:

--send-http-form-urls=
--send-http-json-urls=

These two options greatly simplify sending notifications to multiple destinations. Additionally, the options make it possible to send both the form-encoded result in text format as well as in JSON format.

Besides the connection details, you also need to provide the recipient email address. Note that SMTP servers may sometimes restrict what recipients they allow, but generally using the provider SMTP server will allow sending to your own account.

In the UI you can configure these mandatory values as well as the optional values.

The basic options for sending email can be entered into the general settings, which will then apply to each backup. It is also possible to apply or change the settings for the individual backups by editing the advanced options. Here is how it looks when editing it in the user interface:

You can toggle between the two views using the "Edit as list" and "Edit as text" links.

Besides the mandatory options, it is also possible to configure:

• Email sender address

• The subject line

• The email body

• Conditions on when to send emails

To send a notification via XMPP you need to supply one or more recipientes, an XMPP username and a password.

In the UI you can configure these mandatory values as well as the optional values.

The basic options for sending XMPP notifications can be entered into the general settings, which will then apply to each backup. It is also possible to apply or change the settings for the individual backups by editing the advanced options. Here is how it looks when editing it in the user interface:

You can toggle between the two views using the "Edit as list" and "Edit as text" links.

Besides the mandatory options, it is also possible to configure:

• The notification message and format

• Conditions on when to send emails

• Conditions on what log elements to include

The template system used in Duplicati is quite simple, as it will essentially expand Windows-style environment placeholders, %EXAMPLE%, into values. The same replace logic works for both the subject line (if applicable) and the message body.

Duplicati has defaults for the body and subject line, but you can specify a custom string here. For convenience, the string can also be a path to a file on the machine, which contains the template.

An example custom template could look like:

Duplicati %OPERATIONNAME% for %backup-name% on %machine-name%

The %OPERATIONNAME% operation has completed with the result: %PARSEDRESULT%

Source folders: 
%LOCALPATH%

Encryption module: %ENCRYPTION-MODULE%
Compression module: %COMPRESSION-MODULE%

%RESULT%

The template engine supports reporting any setting by using the setting name as the template value. Besides the options, there are also a few variables that can be used to extract information more easily:

%PARSEDRESULT%
  The parsed result op the operation: Success, Warning, Error
%RESULT%
  When used in the body, this is the result/log of the backup, 
  When used in the subject line, this is the same as %PARSEDRESULT%
%OPERATIONNAME%
  The name of the operation, usually "backup", but could also be "restore" etc.
%REMOTEURL%
  The backend url
%LOCALPATH%
  The path to the local folders involved (i.e. the folders being backed up)
%machine-id%
  The assigned unique random identifier for the current machine. 
  Can be overridden with --machine-id
%backup-id%
  The assigned id for the backup. Can be overridden with --backup-id
%backup-name%
  The name of the backup. Can be overridden with --backup-name
%machine-name%
  The name of the machine. Can be overridden with --machine-name

JSON output

If the output is JSON it needs to be handled different than regular text, to ensure the result is valid. The logic for this is to re-use the templating concept, but only as a lookup, to figure out what keys to include in the results.

An example template could be:

%OPERATIONNAME% 
%backup-name% 
%machine-name% 
%PARSEDRESULT% 
%LOCALPATH%
%ENCRYPTION-MODULE%
%COMPRESSION-MODULE%

This will ensure that each of those values will be included in the extra element in the JSON output. The default template for JSON output includes all fields listed above, but no options are included by default.

To send a notification via Telegram you need to supply a channel id, a bot token and a an api key.

After obtaining the bot token you can obtain the channel id with a cURL script:

BOT_TOKEN="YOURBOTTOKEN" curl -s "https://api.telegram.org/bot$BOT_TOKEN/getUpdates" \
  | grep -o '"id":[0-9]*' | head -1 | cut -d':' -f2

With all required values obtained, you can set up the Telegram notifications in the general settings:

You can toggle between the two views using the "Edit as list" and "Edit as text" links.

Besides the mandatory options, it is also possible to configure:

• The notification message and format

• Conditions on when to send emails

• Conditions on what log elements to include

Telegram Notification Options

Bot Configuration

--send-telegram-bot-id (String) - The Telegram bot ID that will send messages

--send-telegram-api-key (String) - The API key for authenticating your Telegram bot

Message Destination

--send-telegram-channel-id (String) - The channel ID where messages will be sent

--send-telegram-topid-id (String) - Topic ID for posting in specific topics within Telegram groups

Notification Content

--send-telegram-message (String) - Template for message content with support for variables like %OPERATIONNAME%, %REMOTEURL%, %LOCALPATH%, and %PARSEDRESULT%

--send-telegram-result-output-format (format) - Format for presenting operation results

• Duplicati

• Json

Notification Filtering

--send-telegram-level (level) - Controls which result types trigger notifications:

• Success - Only successful operations

• Warning - Operations that completed with warnings

• Error - Operations that failed with recoverable errors

• Fatal - Operations that failed with critical errors

• All - All operation results regardless of status

--send-telegram-any-operation (Boolean) - When enabled, sends notifications for all operations, not just backups

--send-telegram-log-level (Enumeration) - Sets minimum severity level for included log entries:

• ExplicitOnly - Show only explicitly requested messages

• Profiling - Include performance measurement data

• Verbose - Include detailed diagnostic information

• Retry - Include information about retry attempts

• Information - Include general status messages

• DryRun - Include simulation mode outputs

• Warning - Include potential issues that didn't prevent completion

• Error - Include critical failures that require attention

--send-telegram-log-filter (String) - Filters log entries based on specified patterns

--send-telegram-max-log-lines (Integer) - Limits the number of log lines included in notifications

Securing the database

Due to the nature of Duplicati, this database can contain a few secrets that are vital to ensuring the integrity and security of the backups and also the Duplicati server itself. These secrets include both the user-provided secrets, such as the backup encryption passphrase and the connection credentials, but also server-provided secrets, such as the token signing keys, and optionally an SSL certificate password.

Even though the database is located on the machine that makes the backup, it is important to prevent unauthorized access to the database, as it could be used for privilege escalation. And should the database ever be leaked, it is also important to ensure the contents are not accessible.

To protect the database, Duplicati has support for a field-level encryption password. When activated, any setting that is deemed sensitive will be encrypted before being written to the database. This method ensures that the SQLite database itself is still readable, but the secrets are not readable without the encryption passphrase.

To supply the field-level encryption password, start the Server, TrayIcon, or Agent with the commandline option --settings-encryption-key=<key>. As the commandline can usually be read by other processes, it is also possible to supply this key via the environment variable SETTINGS_ENCRYPTION_KEY=<key>.

If you are aware of the risks, you can also set the commandline argument --disable-db-encryption=true instead of the key. This will remove existing encryption and not warn that the database is not encrypted.

The simplest way to apply an encryption key, is to locate the server database, and create the file preload.json if it does not already exist. The file should contain the following:

Database location

When running Duplicati for the first time, it will find a place where it can store the configuration database. Some versions of Duplicati change the location where it looks for the databases, but this is always done backwards compatible, so new versions will also find the database in previous locations. Due to this logic, the locations can change a bit depending on what version of Duplicati was originally installed.

It is possible to pick a different location for the database with the commandline option --server-datafolder=<path> or use the environment variable DUPLICATI_HOME.

To change the folder of an existing instance of Duplicati, perform these steps:

1. Stop Duplicati

2. Move the Duplicati folder from the old location to the new location

3. Change the startup parameters (environment variables, commandline arguments, or preload.json)

4. Start Duplicati again

Database location on Windows

The default location for users running Duplicati is %LOCALAPPDATA%\Duplicati which usually resolves to something like C:\Users\username\AppData\Local\Duplicati. This folder is the non-roaming folder. Older versions of Duplicati used %APPDATA%\Duplicati which is the roaming folder, causing files to be synchronized across machines. However, since Duplicati is not meant to be an app that is useful for roaming, it is now using the non-roaming folder.

When running Duplicati as a Windows Service, the %LOCALAPPDATA%\Duplicati folder resolves to:

Since this folder is under C:\Windows the contents may be deleted on major Windows upgrades (usually when the version number changes). For that reason, Duplicati will detect an attempt to store files in the C:\Windows folder and emit a warning. From version 2.1.0.108 and forward, Duplicati will choose to use C:\Users\LocalService\Duplicati as the storage folder, if it would otherwise be under C:\Windows.

Database location on Linux

The default location when running Duplicati on Linux is ~/.config/Duplicati. For most distros, running Duplicati as a service means running it as the root users, resulting in /root/.config/Duplicati.

However, due to some compatibility mapping, the mapping is sometimes missing the prefix, causing Duplicati data to be stored in /Duplicati. From version 2.1.0.108, this location is avoided and the location /var/lib/Duplicati is used instead, if possible.

Database location on MacOS

The default location when running Duplicati on MacOS is ~/Library/Application Support/Duplicati. Duplicati version 2.0.8.1 and older used the Linux-style ~/.config/Duplicati but this is avoided since version 2.1.0.2.

While it is not required that you keep a copy of the backup configuration, it can sometimes be convenient to have all settings related to a backup stored in a single file.

Export

To export from within the user interface, expand the backup configuration and click "Export ..."

You then need to decide on how to handle secrets stored in the configuration. Since these secrets include both the credentials to connect to the remote destination as well as the encryption passphrase, it is important that the exported file is protected.

You can choose to not include any secrets by unchecking the "Export passwords" option. The resulting file will then not contain the secrets and you need to store them in a different place (credential vault, keychain, etc).

You can also choose to encrypt the file before exporting it. If you choose this option, make sure you choose a strong unique passphrase, and store that passphrase in a safe location.

If you choose to export with passwords but without encryption, you will be warned that this is insecure:

Import configuration

With an exported configuration, you can delete an existing configuration and re-create it by importing the configuration. You can optionally edit the parameters so the re-created backup configuration differs from the original.

To import a configuration, go to the "Add backup" page and choose "Import from file":

Pick the file or drag-n-drop it on the file chooser. If the file is encrypted, provide the file encryption passphrase here as well.

The option to "Import metadata" will create the new backup configuration and restore the statistics, including backup size, number of versions, etc. from the data in the file. If not checked, these will not be filled, and will be updated when the first backup is executed.

If the option "Save immediately" is checked, the backup will be created when clicking import, skipping the option to edit the backup configuration.

If you are starting Duplicati for the first time, it will ask you to pick a password. Picking a strong password is important to ensure unwanted access to Duplicati from other processes on the system. By default, Duplicati has chosen a strong random password and it is recommended for most users to not change the random password. It is not possible to extract the current password in any way and it is not possible to disable the password.

Access from the TrayIcon

This mechanism works for most default installations and is secure as long as the desktop is not compromised. This signin process is the reason that the default random password is prefered, because it is not possible to leak the password.

The downside is that you can bookmark the Duplicati page, but you may be asked for a password that you do not know when accessing the page. In this case, re-launching from the TrayIcon will log you in again.

If you prefer, it is possible to choose the password so you can enter it when asked. Optionally, you can also choose to disable the feature that allows the TrayIcon to sign in without a password, through the settings page.

Login with the TrayIcon is shown here for MacOS, but the same works on Linux and Windows:

Temporary signin token

Note that the regular output from journalctl is capped in width, so you cannot see the whole token. Pipe to a file or another program as shown above to get the full output.

Once you have obtained the link, simply click it or paste it into a browser. Note that the sign-in token has a short lifetime to prevent it being used to gain unathorized access from someone who obtains the logs. If the link has expired, simply restart the service or application and a new link will be generated.

After a password has been set, the link will no longer be generated.

Change password with ServerUtil

This works by reading the same database as the server is using and extracting the keys used to sign a sign-in token, and then creating a sign-in token. This sign-in token works the same way as the TrayIcon's signin feature. Note that the password itself cannot be extracted from the database, it can only be verified.

After obtaining a sign-in token, ServerUtil can then change the password in the running instance.

This only works if:

• The database is readable from the process running ServerUtil

• The database field encryption password is available to the process running ServerUtil

If these constraints are satisfied, it is possible to reset the server password by running:

If ServerUtil is launched in a similar environment (i.e., same user, same environment variables) this would allow access in most cases. There are a number of commandline options that can be used to guide ServerUtil in case the environments are not entirely the same.

Example change with a different context

If you need to change the password for a Windows Service instance running in the service context, you can use a command such as this:

Similarly, if the service is running as root on Linux:

Change password from the Server

Since commandline arguments and environment variables can be viewed through various system tools, it is recommended that the option is not set on every launch. A prefered way to set this would be to stop all running instances, start once with the new password from a commandline terminal, shut down, and then start again normally.

Disable sign-in tokens

It is possible to disable the use of sign-in tokens completely, which can increase security further. This is done by passing the option:

The database is essentially a compact view of what data is stored at the remote destination, and as such it can always be created from the remote data. The only information that is lost if the database is recreated are log messages and the hashes of the remote volumes. The log messages are mostly important for error-tracing but the hashes of the remote volumes are important if the files are not encrypted, as this helps to ensure the backup integrity.

Prior to running a backup, Duplicati will do a quick scan of the remote destination to ensure it looks as expected. This check is important, as making a backup with the assumption that data exists, could result in backups that can only be partially restored. If for the check fails for some reason, Duplicati will exit with an error message explaining the problem.

In rare cases, the database itself may become corrupted or defect. If this seems to be the case, it is safe to delete the local database and run the repair command. Note that it may take a while to recreate the database, but no data is lost in the process, and restores are possible without the database.

Duplicati uses the same setup for filters to select individual files. It is most prominent when choosing the sources, but can be applied in other places where individual files can be selected.

Path representations

Internally, Duplicati represents folders with a trailing path separator, which makes it easy to distinguish between the two types. This distinction is important when constructing filters, as Duplicati requires a full match, including the trailing path separator, before a match is considered. An example for Windows and Linux/MacOS:

• Windows

  • Folders

    • C:\Users\john\

    • X:\data\

  • Files

    • C:\Users\myfile

    • X:\data\file.bin

• Linux/MacOS

  • Folders

    • /home/john/

    • /usr/share/

  • Files

    • /home/myfile

    • /usr/file.bin

For brewity, the remainder of this page will only use the Linux/MacOS format in examples, but the same can be applied to the Windows paths.

Filter types

Duplicati supports 4 different kinds of filters: paths, globbing, regex, and predefined groups. The simplest type of filter is the path. To use a path-type filter, simply provide the full path to the file or folder to target.

Globbing expressions

An example of glob expressions:

The first expression matches files with the 4 ? characters replaced by any character, and the second expression matches the Download folder for any user, and the third matches any files with the .iso extension.

Regular expressions

Regular expressions are provided by wrapping the expressions with hard braces [ ]:

Note that for Windows, the path separators must be escaped with a backslash, \ so each separator becomes a double backslash \\ .

Predefined filter groups

Some files are commonly excluded on many systems, and to make it easier to exclude such files, Duplicati has a number of built in filter groups:

• SystemFiles

  Files that are not real files, such as /proc or System Volume Information.

• OperatingSystem

  Files that are provided by the operating system, such as /bin or C:\Windows\

• CacheFiles

  Files that are part of application or operating system caches, such as the browser cache.

• TemporaryFiles

  Files that are stored temporarily by applications as part of normal operations

• Applications

  Binary applications, such as /lib/ or C:\Program files\

• DefaultExcludes

  All the above filters in one group

To use a filter group, supply one or more names inside curly braces { }, separated with commas. As an example:

Apply filters

By default, Duplicati will recurse the source folders and include every file and folder found. For this reason, most of the filters will be exclude filters that removes something from the backup. Include filters are prefixed with a + and exclude filters are prefixed with a -.

When Duplicati is evaluating filters, it will consider only the first full match, and not evaluate further. It will also evaluate folders before files, meaning that it is not possible to include a file, if the parent folder is excluded. Importantly, the filters are processed in the order they are supplied, which makes it possible to supply advanced rules. As an example:

In the example, the first rule is applied before the second rule, which means that all .txt files in /usr/share/ are included, but any other .txt files are excluded. The inverse goes for the .bin files, because the exclude rule is before the last rule, the files will be matched as exclude, even though there is an include rule.

If we append a rule:

Even if this rule is last, it will exclude the entire folder. Since the folder is excluded, the match on the include rule is never evaluated. This cut-off at the folder level makes it possible to fully avoid processing subfolders, which could otherwise be time consuming.

The preload settings allow configuring machine-wide or enterprise-wide default settings with a single file. Because of this usecase, all settings are applied only if they are not already present. This means a commandline argument could be set up to change the default blocksize, but if the user has applied another setting via the commandline or parameters-file, the preload setting has no effect.

To support different ways of deploying the settings file, 3 locations are checked:

• %CommonApplicationData%\Duplicati\preload.json

  • Linux: /usr/share/Duplicati/preload.json

  • MacOS: /usr/local/share/Duplicati/preload.json

  • Windows:C:\ProgramData\Duplicati\preload.json

• Inside the installation folder

• The file pointed to by DUPLICATI_PRELOAD_SETTINGS

For security reasons, all these paths are expected to be writeable only by Administrator/root so unprivileged users cannot modify the values. If the settings contains secrets, make sure that only the relevant users can read them.

The loading of the files is default silent, even if the parsing fails, but the environment variable DUPLICATI_PRELOAD_SETTINGS_DEBUG=1 will toggle loader debug information to help investigate issues.

The implementation here follows the format:

{
  "env": {
    "*": {
      "TEMP": "/mnt/tmp",
      "LOGGING": "false"
   },
   "tray": {
      "LOG": "1"
    },
    "server": {
        "DUPLICATI__WEBSERVICE_ALLOWED_HOSTNAMES": "m1"
    }
  },

  "db": {
    "server": {
      "--compression-module": "zip",
      "--send-http-result-output-format": "Json"
     }
  },

  "args": {
    "tray": [ "--hosturl=http://m1:8299" ],
    "server": [ "--webservice-port=8299" ]
  }
}

The file has 3 sections that are all similar and all optional: env, db, and args. Each section can apply to all executables (*) or a specific executable. The executable names can be seen in the source, but the most common ones are tray and server.

In the case where the * section and specific executable has the same variable, the specific one is used. If multiple settings files are found, they are loaded in the order described above. Here the last file loaded will be able to overwrite the others. The * settings are collected from all three files, as are the executable specific options, and only after all parsing is done, are the specific executable options applied (see below for an example).

Note that some executables will load others, such that TrayIcon, Service, and WindowsServer will load Server.

Environment variables - env

The env section contains environment variables that are applied inside the process, after starting. Each entry under an executable is a key-value pair, where the key is the name of the environment variable, and the value will be the contents of the environment variable.

The environment variables are only set if they are not already set, allowing a custom base set, but prefers local machine variables.

In the case where one binary loads another, the starting application environment variables are applied first, and then any unset environment variables are applied for the loaded executable.

Database settings - db

For the db section it is possible to use * but the settings are currently only applied when running the server, so for future compatibility this section should use server only. The settings under an executable in the db section are automatically prefixed with -- to ensure they are valid options and are saved as the "application wide" settings, also visible in the UI under Settings -> Advanced Options.

The settings here are applied to the database if they are changed, meaning a change to the settings will overwrite settings the user has already applied. This check is performed on startup.

The database settings are not passed on from a binary when it loads another, so the only database settings that are loaded are done by Server, even if any are supplied by tray (may change in the future).

Commandline arguments

The commandline arguments supports both the * and specific executable name. The arguments are expected to be switches in the format --name=value but can be any commandline argument. The general logic in Duplicati is that "last option wins", so the resolver logic for that is applied to try to get the most logical combination of arguments.

Resolution with conflicts

If the following fragment is supplied:

"env": {
  "*": {
    "E1": "a",
    "E2": "b"
  },
  "tray": {
    "E1": "c",
    "E3": "d"
  }
}

The Server executable will get the settings from * and the TrayIcon will get the values: "E1=c E2=b E3=d".

If the above fragment is found in the first file, but this fragment is found in a later file:

"env": {
  "*": {
    "E3": "f"
  },
  "tray": {
    "E1": "g"
  }
}

First the * variables are collected, giving "E1=a E2=b E3=f", then the tray variables give "E1=g E3=d", and then they are combined to give "E1=g E2=b E3=d" for tray.

The same combination logic is applied for both the db and args sections, but since the args section are not key-value pairs, and their order matter, it is done by collecting the arguments first, and then reducing them:

"args": {
  "*": ["--test=1", "--abc=123"],
  "server": ["--xyz=z", "--test=1", "--test=2"]
}

In this case the arguments are collected, with * first, then the executable specifics, giving:

["--test=1", "--abc=123", "--xyz=z", "--test=1", "--test=2"]

Since this contains 3 options named --test, they are reduced and appended so it ends up with:

["--abc=123", "--xyz=z", "--test=2"]

The intention here is to stay as close as possible to the original line that was entered. If the commandline arguments already contains --test, the values are not applied.

Configure the image

The Duplicati Docker images are using /data inside the container to store configurations and any files that should persist between container restarts. Note that other images may choose a different location to store data, so be sure to follow the instructions if using a different image.

You also need a way to sign in to the server after it has started. You can either watch the log output, which will emit a special signin url with a token that expires a few minutes after the server has started, or provide the password from within the configuration file.

To ensure that any secrets configured within the application are not stored in plain text, it is also important to set up the database encryption key.

Managing secrets in Docker

Ideally, you need at least the settings encryption key provided to the container, but perhaps also the webservice password. You can easily provide this via a regular environment variable:

Using a preload file

To use the preload approach, prepare a preload.json file with your encryption key:

You can then configure this in the compose file:

Using a secret manager

Setting up the secret manager is a bit more work, but it has the benefit of being able to configure multiple secrets in a single place. To configure the file-based secret provider, you need to create a secrets.json file such as this:

Then set it up in the compose file:

It is also possible to use one of the other secret providers, such as one that fetches secrets from a secure key vault. In this case, you do not need the secrets.json file, but can just configure the provider.

Read locked files

Duplicati has support for LVM-based snapshots which is the recommended way for getting a consistent point-in-time copy of the disk. For some uses, it is not possible to configure LVM snapshots, and this can cause problems due to some files being locked. By default, Duplicati will respect the advisory file locking and fail to open locked files, as the lock is usually an indication that the files are in use, and reading it may not result in a meaningful copy.

If you prefer to make a best-effort backup, which was the default in Duplicati v2.0.8.1 and older, you can disable advisory file locking for individual jobs with the advanced option: --ignore-advisory-locking=true. You can also disable file locking support entirely in Duplicati:

Before you can install Duplicati, you need to decide on three different parameters:

• Your package manager: apt, yum or something else.

• You machine CPU type: x64, Arm64 or Arm7

Deciding on type

Determine package manager

Next step is checking what Linux distribution you are using. Duplicati supports running on most Linux distros, but does not yet support FreeBSD.

If you are using a Debian-based operating system, such as Ubuntu or Mint, you can use the .deb package, and for RedHat-based operating system, such as Fedora or SUSE, you can use the .rpm packages.

For other operating systems you can use the .zip package, or check if your package manager already carries Duplicati.

Determine CPU architecture

Finally you need to locate information on what CPU architecture you are using:

• x64: 64bit Intel or AMD based CPU. This is the most common CPU at this time.

• Arm64: 64bit ARM based CPU. Used in Raspberry Pi Model 4 and some Laptops and Servers.

• Arm7: 32bit ARM based CPU. Used in Raspberry Pi Model 3 and older, and some NAS devices.

Installing the package

Using the TrayIcon

When running the TrayIcon in a user context, it will create a folder in your home folder, typically ~/.config/Duplicati where it stores the local databases and the Server database with the backup configurations.

Using the Server

Using Server as a Service

If you need to pass options to the server, edit the settings file, usually at /etc/default/duplicati. Make sure you only edit the configuration file and not the service file as it will be overwritten when a new version is installed. The settings file should look something like this:

You can use DAEMON_OPTS to pass arguments to duplicati-server, such as --webservice-password=<passsword>.

To enable the service to auto-start, reload configurations, start the service and report the status, run the following commands:

The server is now running and will automatically start when you restart the machine.

Note: the service runs in the root user context, so files will be stored in /root/.config/Duplicati on most systems, but in /Duplicati on other systems. Use the DAEMON_OPTS to add --server-datafolder=<path to storage folder> if you want a specific location.

To check the logs (and possibly obtain a signin link), the following command can usually be used:

Using the Agent

When the Agent starts, it will emit a registration link to the log, and you can usually see it with the following command:

After registration is complete, restart the service to pick up the new credentials:

Using the CLI

Using the CLI is simply a matter of invoking the binary:

If you specify the --dbpath parameter, it will not use the dbconfig.json file and it will not store anything in the local datafolder.

Using the support programs

Before you can install Duplicati, you need to decide on three different parameters:

• You machine CPU type: x64, Arm64, or x86 (32 bit)

Deciding on type

Determine CPU architecture

Finally you need to locate information on what CPU architecture you are using:

• x64: 64bit Intel or AMD based CPU. This is the most common CPU at this time.

• Arm64: 64bit ARM based CPU. Some laptops, tablets and servers use it.

• x86: 32bit Intel or AMD based CPU. Note that Windows 10 was the last version to support 32 bit processors.

Installing the package

Using the TrayIcon

C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.exe

When running the TrayIcon in a user context, it will create a folder in your home folder, typically C:\Users\<username>\AppData\Local\Duplicati where it stores the local databases and the Server database with the backup configurations.

Using the Server

C:\Program Files\Duplicati 2\Duplicati.Server.exe

Running the Server as a Windows Service

If you want to run Duplicati as a Windows Service, you can use the bundled service tool to install/uninstall the service:

C:\Program Files\Duplicati 2\Duplicati.WindowsService.exe INSTALL
C:\Program Files\Duplicati 2\Duplicati.WindowsService.exe UNINSTALL

When installing the Service it will automatically start, and likewise, uninstalling it will stop the service. If you need to pass options to the server, you can provide them to the INSTALL command:

C:\Program Files\Duplicati 2\Duplicati.WindowsService.exe INSTALL --webservice-port=8100 --server-datafolder=<path>

Note: When running the Windows Service it will default to use port 8200 and fail it that port is not available. If you are running the TrayIcon, that will run a different instance, usually at port 8300. If you want to connect the TrayIcon to the Windows Service, edit the shortcut to Duplicati:

C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.exe --no-hosted-server --host-url=http://localhost:8200 --webservice-password=<password>

Using the Agent

You can also register the Agent, using the Agent executable:

C:\Program Files\Duplicati 2\Duplicati.Agent.exe register <registration url>

After the Agent has been registered, restart the service and it will now be available on the Duplicati Console.

{
  "args": {
    "agent": [ "--agent-registration-url=<registration-url>" ]
  }
}

Using the CLI

Using the CLI is simply a matter of invoking the binary:

C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe help

If you specify the --dbpath parameter, it will not use the dbconfig.json file and it will not store anything in the local datafolder.

Using the support programs

 C:\Program Files\Duplicati 2\Duplicati.CommandLine.ServerUtil.exe help

Even though Duplicati tries hard to reduce storage use as much as possible, it is inevitable that the remotely stored data grows as new versions of files are added. To avoid running out of space or paying for excessive storage use, it is important that unnecessary backups are removed regularly.

After deleting one or more versions, Duplicati will mark any data that can no longer be referenced as waste, and may occasionally choose to run a compact process that deletes unused volumes and creates new volumes with no wasted space.

Despite all deletion rules, Duplicati will never delete the last version, keeping at least one version available.

Delete older than

The most intuitive option is to choose a period that data is stored, and then to consider everything older than this period as stale data. The actual period depends on the actual use, but it could be 7 days, 1 year or 5 years for example.

This option is usually the prefered choice if the backups happen regularly, such as a backup each day, and then keep the last 3 months.

Keep versions

If the backups are running irregularly, where the backups are triggered by some external event, there may be long periods where there are no backups. For this case you can choose a number of versions to keep and Duplicati will consider anything outside that count as outdated.

Another special case is that if the source data has not changed at all, which is uncommon, Duplicati will not make a new version, as it would be identical to the previous version. In such a setup, it may be preferable to use a version count, despite regularly scheduled backups.

Retention policy

7D:U,1Y:1W

The first bucket is defined as being 7 days, and the value U means unlimited the number of backups in this bucket. In other words: for the most recent 7 days, keep all backups.

The second bucket is defined as 1 year, keeping a backup for each 1 week, resulting in rougly 52 backups after the first 7 days.

Any backups outside the buckets are deleted, meaning anything older than a year would be removed.

In the UI, a helpful default is called "Smart retention" which sets the following retention policy:

1W:1D,4W:1W,12M:1M

Translated, this policy means that:

• For the most-recent week, store 1 backup each day

• For the last 4 weeks, store 1 backup each week

• For the last 12 months, store 1 backup each month

In normal Duplicati operations, the files at the remote destination should never be handled by anything but Duplicati. Changing the remote files will always result in warnings or errors when Duplicati needs to access those files.

However, in certain exceptional scenarios, it may be required that the file contents are accessed manually.

Processing files encrypted with AES encryption

Processing files encrypted with GPG encryption

gpg -d volume.zip.gpg -o volume.zip

And similarly, to encrypt a file, you can use:

gpg --symmetric volume.zip -o volume.zip.gpg

Re-compress and re-encrypt

Before you can install Duplicati, you need to decide on two different parameters:

• You machine CPU type: Arm64 or x64

Deciding on type

Determine CPU architecture

Your Mac is most likely using Arm64 with one of the M1, M2, M3, or M4 chips. If you have an older Mac, it may use the Intel x64 chipset. To see what CPU you have, click the Apple icon and choose "About this Mac". In the field labelled "Chip" it will either show Intel (x64) or M1, M2, M3, M4 (Arm64).

Installing the package

If you are using the .dmg package the installation works similar to other application, simply open the .dmg file and drag Duplicati into Applications. Note that with the .dmg package, Duplicati is not set to start automatically with your Mac, but if you restart with the option to re-open running programs, Duplicati will start on login.

If you are using the .pkg package, Duplicati will install a launchAgent that ensures Duplicati starts on reboots. The CLI package installs a stub file that is not active, so you can edit the launchAgent and have it start the Server if you prefer.

Using the TrayIcon

If you have installed the GUI package, you will have Duplicati installed in /Applications and it can be started like any other application. Once Duplicati is started, it will place itself in the menu bar near the clock and battery icons. Because Duplicati is meant to be a background program, there is no Duplicati icon in the dock.

On the first start Duplicati will also open your browser and allow you to configure your backups. If you need access to the UI again later, locate the TrayIcon in the status bar, click it and click "Open". If you install the CLI or Agent packages, the Duplicati application is not available.

Using the Server

If you install the CLI package, Duplicati binaries are placed in /usr/local/duplicati and symlinked into /usr/local/bin and you can start the server simply by running:

duplicati-server

Note: If you install the GUI package or install from homebrew, Duplicati's binaries are not symlinked into the paths searched by MacOS. You can invoke the binaries by supplying the full path:

/Applications/Duplicati.app/Contents/MacOS/duplicati-server

Using the Agent

If the Agent is not registered with the Console, it will open the default browser and ask to be registered. Once registered, it will run in the background and be avilable on the Duplicati Console for management.

{
  "args": {
    "agent": [ "--agent-registration-url=<registration-url>" ]
  }
}

Using the CLI

Using the CLI is simply a matter of invoking the binary:

duplicati-cli help

If you specify the --dbpath parameter, it will not use the dbconfig.json file and it will not store anything in the local datafolder.

Note: If you install the GUI package or install from homebrew, Duplicati's binaries are not symlinked into the paths searched by MacOS. You can invoke the binaries by supplying the full path:

/Applications/Duplicati.app/Contents/MacOS/duplicati-cli help

Using the support programs

duplicati-server-util help

Note: If you install the GUI package or install from homebrew, Duplicati's binaries are not symlinked into the paths searched by MacOS. You can invoke the binaries by supplying the full path:

/Applications/Duplicati.app/Contents/MacOS/duplicati-server-util help

If you are using one of the backends that requires login via OAuth (Google, Dropbox, OneDrive, etc) you will need to obtain a "clientId" and a "clientSecret". These are given by the service providers when you are logged in, and are usually free.

If you prefer to avoid the hassle of setting this up, you can opt to use the Duplicati provided OAuth server, where Duplicati's team will handle the configuration. This OAuth server is the default way to authenticate. If you prefer to be more in control of the full infrastructure, you can use this guide to set up and use your own self-hosted OAuth Server.

For example, this guide will show how to set up an OAuth server for internal use in an organization, granting Duplicati instances full access to the Google Drive files.

Getting access to Google Cloud Services

Once you have create a project where the OAuth settings can live in, you need to enable the "Google Drive API". Go to the top-left menu, choose "API & Services" and then "Enabled APIs & Services". From here search for "Google Drive API", click it and enable:

Before you can get the values you need to configure the consent screen that is shown when users log in with your OAuth Service. You can choose "Internal" here, unless you need to provide access to people outside your organization. Choosing "External" also requires a Google review. On the consent screen, you only need to fill in the required fields, the app name and some contact information:

The last step in the consent is choosing the scopes (meaning the permissions) it is possible to grant with this setup. In this example we choose the auth/drive scope, granting full access to all files in the users Drive. For regular uses, it is safest to use auth/drive.file which will only grant Duplicati access to files created by Duplicati. However, in some cases Google Drive will drop your permissions and refuse to let Duplicati access the files. There is no way to change the permissions on the files, so if this happens, your only choice is to use auth/drive and obtain full access:

You can now click update and save the consent screen and proceed to setting up the credentials needed. Click "Create Credentials" and choose "OAuth client ID". On the next page, choose the type "Web application". In the "Authorized redirect URIs" field you need to enter the url for the server that is being called after login. The Duplicati OAuth server uses a path of /logged-in so make sure it ends with that. In the screenshot, the server is hosted on a single machine, so the setup is for https://localhost:8080/logged-in:

When you are done, click "Save" and a popup will show the credentials that are generated. Use the convenient copy buttons to get "Client ID" and "Client secret" or download the JSON file containing them. If you loose them, you can get then again via the "Credentials" page. The credentials shown here are redacted:

Setting up the configuration

With the credentials available, create a JSON text file similar to this:

{
  "GD_CLIENT_ID": "<Put Client ID here>",
  "GD_CLIENT_SECRET": "<Put Client secret here>"
}

Docker based setup

- ASPNETCORE_URLS: "http://localhost:8080"
- HOSTNAME: "localhost:8080"
- SECRETS: "/path/to/secrets.json.aes"
- SECRETS_PASSPHRASE: "<encryption passphrase>"
- STORAGE: "file:///path/to/persisted/folder"
- SERVICES: "googledocs"

The hostname here MUST match the one set as the redirect URI or the authorization will fail. The URLs parameter is what the internal Docker engine thinks it is running. For this setup there is no TLS/SSL certificate, so the URL here is http but note that we used https in the redirect URI and these two must match in the end. Here I am assuming some other service is providing the SSL layer.

If you need to serve the certificate directly from the Docker container, generate a certificate .pfx file and use a configuration such as:

- ASPNETCORE_URLS: "https://localhost:8080"
- HOSTNAME: "localhost:8080"
- SECRETS: "/path/to/secrets.json.aes"
- SECRETS_PASSPHRASE: "<encryption passphrase>"
- STORAGE: "file:///path/to/persisted/folder"
- SERVICES: "googledocs"
- ASPNETCORE_Kestrel__Certificates__Default__Path: "/path/to/certificate.pfx"
- ASPNETCORE_Kestrel__Certificates__Default__Password: "<certificate password>"

Local machine setup

To run the server, invoke it with a setup like this:

OAuthServer run 
  --listen-urls=http://localhost:8080 
  --hostname=localhost:8080
  --storage=file:///path/to/persisted/folder
  --secrets=/path/to/secrets.json.aes
  --secrets-passphrase=<encryption passphrase>
  --services=googledocs

The hostname here MUST match the one set as the redirect URI or the authorization will fail. The URLs parameter is what the process thinks it is running locally. For this setup there is no TLS/SSL certificate, so the URL here is http but note that we used https in the redirect URI and these two must match in the end. Here I am assuming some proxy service is providing the SSL certificate.

If you need to serve the certificate directly from the the binary, generate a certificate .pfx file and use a configuration such as:

OAuthServer run 
  --listen-urls=https://localhost:8080 
  --hostname=localhost:8080
  --storage=file:///path/to/persisted/folder
  --secrets=/path/to/secrets.json.aes
  --secrets-passphrase=<encryption passphrase>
  --services=googledocs
  --certificate-path=/path/to/certificate.pfx
  --certificate-password=<certificate password>

Issuing an AuthID

Once the service is running, you can navigate to the page and generate an AuthID:

Using the self-hosted OAuth server in Duplicati

The final step is to instruct Duplicati to use the self-hosted OAuth server instead of the regular instance. This is done by visiting the "Settings" page in the Duplicati UI and adding the advanced option --oauth-url=https://localhost:8080/refresh:

Don't forget to click "OK" to save the settings. Once configured, the "AuthID" links in the UI will point to your self-hosted OAuth server, and all authorization is done purely through the self-hosted OAuth server.

The most basic destination in Duplicati is the file backend. This backend simply stores the backup data somewhere that is reachable from the file system. The destination can be a network based storage as long as it is mounted when needed, a fixed disk, or a removable media.

The file backend can be chosen with the file:// prefix where the rest of the destination url is the path.

Windows example:

file://C:\Data
file://\\server\share\folder

Linux/MacOS example:

file:///home/user

For most cases it will also work without the file:// prefix, but adding the prefix makes the intention clear.

Improving speed for local filesystems

Since Duplicati is intended to be used with remote systems, it will make a temporary file, and then copy the temporary file to the new location. This enables various retry mechanisms, progress reporting and failure handling that may not be desired with local filesystems.

To change this logic to instead use the operating system movecommand to move the file into place, avoiding a copy, set the option --use-move-for-put, on the file backend and also set --disable-streaming-transfers. With these two options, all special handling will be removed and the transfer speed should be the optimal possible with the current operating system. Note that setting --disable-streaming-transferswill not show any progress during transfers, if you are using the UI, because the underlying copy or move method cannot be monitored.

Disabling length verification

Because a local storage destination is expected to have a very low latency, the file backend will verify the length of the file after copy. This additional call is usually very fast and does not impact transfers speeds, but can be disabled for slightly faster uploads with --disable-length-verification.

Removable drives (mostly Windows)

For removable drives, the mount path can sometimes change when inserting the drive. This is most prominent on WIndows, where the drive letters are assigned based on what order the drives are connected. To support different paths, you can supply multiple alternate paths with --alternate-target-paths, where each path is separated with the system path separator (;on Windows, :on Linux/MacOS):

// Note, the paths are URL encoded here: E:\backupdata;G:\backupdata
file://F:\backupdata?alternate-target-paths=E%3A%5Cbackupdata%3BG%3A%5Cbackupdata

If you would like to support any drive letter, you can also use * as the drive letter (Windows only):

file://*:\backupdata

Because using multiple paths could end up attempting to make a backup to the wrong drive, you can use the option --alternate-destination-marker to provide a unique marker filename that needs to exist on the destination:

file://F:\backupdata?alternate-destination-marker=<filename>

Using this option will scan all paths provided, either using the * drive letter or --alternate-target-paths, and check if the folder contains a folder with the given filename.

Authentication (Windows Only)

To use authentication, provide the --auth-username and --auth-passwordarguments to the query. Since the authentication in Windows is tied to the current user context, it is possible that the share is already mounted with different credentials, that may not have the correct permissions.

To guard against this, it is possible to drop the current authentication and re-authenticate prior to acessing the share. This can be done by adding the --force-smb-authentication option.

The Simple Storage Service, S3, was originally described, developed and offered by Amazon via AWS. Since then, numerous other providers have adopted the protocol and offer S3-compatible services. While these services are mostly compatible with the core S3 protocol, a number of additional AWS-specific settings are usually not supported and will be ignored.

When storing data in S3, the storage is divided into a top-level "folder" called a "bucket", and each bucket has "objects", similar to files. For most providers, an object name with /characters will be interpreted as subfolders in some way.

In the original S3 specification, the bucket name was used as part of the hostname, causing some issues with bucket names that are not valid hostnames, and some delays for new buckets caused by DNS update speeds. Newer solutions use a single shared hostname and provide the bucket name as a parameter.

To use S3 as the storage destination, us a format such as:

s3://<bucket name>/<prefix>
  ?aws-access-key-id=<account id or username>
  &aws-secret-access-key=<account key or password>
  &s3-servername=<server ip or hostname>
  &use-ssl=true

Note that the default for S3 is to use unencrypted connections. The connections are secured with signatures, but all data transfered can be captured through the network. If the provider supports SSL/TLS, which most do, make sure to add --use-ssl=trueto also encrypt the connection.

Choosing the client

Generally, both libraries will work with most providers, but the AWS library has some defaults that may not be compatible with other providers. While you can configure the settings, it may be simpler to use Minio with the default settings.

Creating the bucket

Since the bucket defines the place where data is stored, a bucket needs to be created before it can be used. All providers will offer a way to do this through their UI, and allows you to set various options, such as which geographical region the bucket is located in.

If you use Duplicati to create the bucket, you can also set the option --s3-location-contraintto provide the desired location. Support for this, and available regions, depends on the provider.

Storage class

With S3 it is also possible to set the storage class which is sometimes used to fine-tune the cost/performance/durability of the files. The storage class is set with --s3-storage-class, but the possible settings depends on the provider.

Duplicati makes backups of files, called the source, and places the backup data at a destination chosen by the user. To make Duplicati as versatile as possible, each of the destinations are implemented as a "destination" (or "backend"), each with different properties.

Some storage providers support multiple protocols with each their strenghts, and you can generally pick which storage destination provider you like, but if there is a specific implementation for a given storage provider, that is usually the best pick.

Each storage destination has a number of options that can be provided via a URL like format. The options should preferably be provided as part of the URL, but can also be provided via regular commandline options. For instance, the --use-ssl=true flag can also be added to the URL with &use-ssl=true. If both are provided, the URL value is used.

Standard based destinations

Destinations in this category are general purpose enough, or commonly used, so they can be used across a range of storage providers. Destinations in this category are:

Provider specific destinations

Storage destinations in this category are specific to one particular provider and implemented using either their public API description, or by using libraries implemented for that provider. Destinations in this category are:

File synchronization providers

Storage destinations in this category are also specific to one particular provider, but these storage provider products are generally intended to be used as file synchronization storage. When they are used with Duplicati, the backup files will generally be visible as part of the synchronization files. Destinations in this category are:

Decentralized providers

Storage destinations in this category are utilizing a decentralized storage strategy and requires knowledge about each system to have it working. Some of these may require additional servers or intermediary providers and may have different speed characteristics, compared to other storage providers. Destinations in this category are:

To use the FTP backend, you can use a URL such as:

Despite FTP being a well documented standard, there are many different implementations of the protocol, so the FTP backend supports a variety of settings for configuring the connection. You can use a non-standard port through the hostname, such as ftp://hostname:2121 .

Connection mode

Due to the way FTP is working, it requires multiple connections to transfer data, and the method for selecting which mode has a number of quirks. The default setting is "AutoPassive" which works great for most setups, leaving the burden of configuring the firewall to the server.

Use the option --ftp-data-connection-type to choose a specific connection mode if the default does not work for your setup.

Encryption mode

To enable encrypted connections, you can use the option --ftp-encryption-mode and setting it to either Implicit or Explicit. The Implicit setting creates a TLS connection and everything is encrypted, where Explicit is more commonly used, and creates an unencrypted connection and then upgrades to an encrypted session.

The default setting is --ftp-encryption-mode=None which uses unencrypted FTP connections.

The setting --ftp-encryption-mode=Auto is the most compatible setting, but also insecure, as it connects in unencrypted mode and then attempts to switch to encrypted, but will continue in unencrypted mode if this fails.

To further lock down the encryption mode, the option --ftp-ssl-protocols can be used to limit the accepted protocols. Note: that due to unfortunate naming in .NET, the option --ftp-ssl-protocols=None means "use the system defaults".

Self-signed certificates

To support self signed certificates, the FTP destination also supports the --accept-specified-ssl-hash option is also supported which takes an SHA1 certificate digest and approves the certificate if it matches that hash. This is similar to a manual certificate pinning and allows trusting a specific certificate outside the operating systems normal trust chain.

For testing, it is also possible to use --accept-any-ssl-certificate which will bypass certificate checks completely and enable man-in-the-middle attacks on the connection.

Path resolution

The FTP protcol is tied to a Posix-style path where / is the root folder and subfolders are described using the forward-slash separator. On some systems the filesystem is virtual, so the user can only see the root path, but has no knowledge of the underlying real filesystem. On others, the paths are mapped directly to the user home, like /home/user.

Use the option --ftp-absolute-path to treat the source path as an absolute path, meaning that folder maps to /folder and not to /home/user/folder.

A related option is the --ftp-use-cwd-names option that makes Duplicati keep track of the working directory and uses the FTP server's CD command to set the working folder prior to making a request.

Verification of uploads

To verify that uploads actually work, the FTP connection will request the file after it has been uploaded to check that it exists and has the correct file size. This check is usually quite fast and does not impact backup speeds, but if needed it can be disabled with --disable-upload-verify.

A related setting --ftp-upload-delay adjusts the delay that is inserted after the upload but before verifying the file exists, which is required on some servers to ensure the file is fully flushed before validating the existence.

Debugging commands

Notes on aFTP

With Duplicati 2.1.0.2 the codebase was upgraded to .NET8 which means that FtpWebRequest is now deprecated. For that reason, the FTP backend was converted to also be based on FluentFTP, so both FTP backends are currently using the same library.

The aFTP backend is still available for backwards compatibility, but is the same as the FTP backend, with some different defaults. The aFTP backend will likely be marked deprecated in a future version, and eventually removed.

The WebDAV protocol is a minor extension to the HTTP protocol used for web requests. Because it is compatible with HTTP it also supports SSL/TLS certificates and verification similar to what websites are using.

To use the WebDAV destination, you can use a url such as:

webdav://<hostname>/<path>
  ?auth-username=<username>
  &auth-password=<password>

You can supply a port through the hostname, such as webdav://hostname:8080/path.

Authentication method

There are three different authentication methods supported with WebDAV:

• Integrated Authentication (mostly on Windows)

  • Use --integrated-authentication=trueto enable. This works for some hosts on Windows and most likely has no effect on other systems as it requires a Windows-only extension to the request and a server that supports it.

• Digest Authentication

  • Use --force-digest-authentication=true to use Digest-based authentication

• Basic Authentication

  • Sending the username and password in plain-text. This is the default, but is insecure if not using an SSL/TLS encrypted connection.

You need to examine your destination servers documentation to find the supported and recommended authentication method.

Encryption and Certificates

To use an encrypted connection, add the option --use-ssl=truesuch as:

webdav://<hostname>/<path>
  ?auth-username=<username>
  &auth-password=<password>
  &use-ssl=true

This will then use an HTTPS secured connection subject to the operating system certificate validation rules. If you need to use a self-signed certificate that is not trusted by the operating system, you can use the option --accept-specified-ssl-hash=<hash> to specifically trust a certain certificate. The hash value is reported if you attempt to connect and the certificate is not trusted.

This technique is similar to certificate pinning and prevents rotating the certificate and blocks man-in-the-middle attacks.

For testing setups you can also use --accept-any-ssl-certificate that will disable certificate validation. As this enables various attacks it is not recommended besides for testing.

If you are using Rclone, some features, such as bandwidth limits and transfer progress do not work.

Duplicati does not bundle Rclone, so you need to download and install the appropriate binaries before you can use this backend. The URL format for the Rclone destination is:

rclone://<remote repo>/<remote path>
  ?rclone-executable=<path to rclone executable>

If the remote repo is not a valid hostname, you can instead use this format:

rclone://
  ?rclone-remote-repository=<remote repo>
  &rclone-remote-path=<remote path>
  &rclone-executable=<path to rclone executable>

Advanced options

If you need to change the Rclone local repo you can use the option --rclone-local-repository which will otherwise be set to local, which works for most setups.

If you need to supply options to Rclone, these can be passed via --rclone-option. Note that the values must be url encoded, and multiple options can be passed by separating them with spaces, before encoding.

As an example adding "--opt1=a --opt2=b" needs to url encoded and results in:

rclone://<remote repo>/<remote path>
  ?rclone-option=--opt1%3Da%20--opt2%3Db

Duplicati supports storing files with OpenStack, which is a large-scale object storage, similar to S3. With OpenStack you store "objects" (similar to files) in "containers" which define various properties shared between the objects. If you use a / in the object prefix, they can be displayed as virtual folders when listing them.

OpenStack v2

If you are using OpenStack with version 2 of the protocol, you can either use an API key or a username/password/tenant combination. To use the password based authentication, use a URL format like this:

If you are using an API key, leave out the --auth-password and --openstack-tenant-name parameters and add in --openstack-apikey=<apikey>.

OpenStack v3

If you are using OpenStack with version 3 of the protocol, you must supply: username, password, domain, and tenant name:

Region selection

The authentication response will contain a set of endpoints to be used for actual transfers. In some cases, this response can contain multiple possible endpoints, each with a different region. To prefer a specific region, supply this with --openstack-region. If any of the returned endpoints have the same region (case-insensitive compare), the first endpoint matching will be selected. If no region is specified, or no region matches, the first region in the response is used.

To use the SFTP destination you can use a URL such as:

You can supply a non-standard port through the hostname, such as ssh://hostname:2222/folder.

Using key-based authentication

It is very common, and more secure, to use key-based authentication, and Duplicati supports this as well. You can either provide the entire key as part of the URL or give a path to the key file. If the key is encrypted, you can supply the encryption key with --auth-password.

To use a private key inline, you need to url encode it first and then pass it to --ssh-key. An example with an inline private key:

Note that you need both the prefix sshkey:// and you need to URL encode the contents.

If you have the SSH keyfile installed in your home folder, you can use the file directly with --ssh-keyfile:

Note that Duplicati does not currently support key agents so you must pass the password here.

For best security it is recommended to use a separate identity and key files for the user, so a compromise of the keys does not grant more permissions than what is required.

Validating the host key

Since SSH does not have a global key registry, like for HTTPS, it is possible to launch a man-in-the-middle attack on an SSH connection. To prevent this, Duplicati and other SSH clients will use certificate pinning where the previously recorded host certificate hash is saved and changes to the host certificate must be manually handled by the user.

On the first connection to the SSH server, Duplicati will throw an exception that explains how to trust the server host key, including the host key fingerprint. Once you obtain the host key fingerprint, you can supply it with the --ssh-fingerprint option.

If the host key changes, you will get a different message, but also reporting the new host key, so you can update it. The option --ssh-accept-any-fingerprints=true is only recommended for testing and not for production setups as it will disable the man-in-the-middle protection.

If you are using the UI, you can click the "Test connection" button and it will guide you to set the host key parameters based on what the server reports.

Timeout and keep-alive

By default, Duplicati will assume that the connection works once it has been established. If the SSH server is malfunctioning it may cause operations to hang. To guard against this case, you can set the --ssh-operation-timeout option to enforce a maximum time the operation may take.

A different kind of timeout is when firewalls and other network equipment monitors the connections and closes them if there is no activity. Because Duplicati may open a connection and then perform a long operation locally, it may cause the connection to be closed due to inactivity. The option --ssh-keepalive can be used to define a keep-alive interval where messages are sent if there is no other activity.

Both options are default disabled and should only be enabled if there are special conditions in a setup where the options are needed.

The Common Internet File System (CIFS) backend provides native support for accessing shared network resources using the CIFS/SMB protocol. This backend enables direct interaction with Windows shares and other CIFS-compatible network storage systems.

To use the CIFS destination, you can use a url such as:

cifs://<hostname>/<share>/<path>
  ?auth-username=<username>
  &auth-password=<password>
  &transport=directtcp

Transport

CIFS supports two distinct transport protocols, each with its own characteristics:

DirectTCP (directtcp)

• Port: 445

• Characteristics:

  • Faster performance

  • Modern implementation

  • Preferred for newer systems

  • Direct TCP/IP connection

  • Lower overhead

NetBIOS over TCP (netbios)

• Port: 139

• Characteristics:

  • Legacy support

  • Compatible with older systems

  • Additional protocol overhead

  • Slower performance

  • Uses NetBIOS naming service

Advanced Options

--read-buffer-size

Defines the read buffer size, in bytes, for SMB operations (Will be capped automatically by SMB negotiated values, values bellow 10000 bytes will be ignored)

--write-buffer-size

Defines the write buffer size, in bytes, for SMB operations (Will be capped automatically by SMB negotiated values, values bellow 10000 bytes will be ignored)

To use box.com, use the following URL format:

box://<folder>/<subfolder>?authid=<authid>

Fully delete files

When files are deleted from your box.com account, they will be placed in the trash folder. To avoid old files taking up storage in your account, you can add --box-delete-from-trash which will then also remove the file from the trash folder.

Duplicati supports storing files with Rackspace CloudFiles, which is a large-scale object storage, similar to S3. With CloudFiles you store "objects" (similar to files) in "containers" which define various properties shared between the objects. If you use a / in the object prefix, they can be displayed as virtual folders when listing them.

To use CloudFiles, you can use the following URL format:

cloudfiles://<container>/<prefix>
  ?cloudfiles-username=<username>
  &cloudfiles-accesskey=<access key>

Using a different API endpoint

The default authentication will use the US endpoint, which will not work if you are a customer of the UK service. To choose the UK account, add --cloudfiles-uk-account=true to the request:

cloudfiles://<container>/<prefix>
  ?cloudfiles-username=<username>
  &cloudfiles-accesskey=<access key>
  &cloudfiles-uk-account=true

If you need to use a specific host, you can also provide the authentication URL directly with the --cloudfiles-authentication-url option. If you are providing the URL, the --cloudfiles-uk-account option will be ignored.

Note that the bucket id is globally unique, so it is recommended using a name that is not likely to conflict with other users, such as prefixing the bucket with the project id or a similar unique value. If you use a simple name, like data or backup it is likely already associated with another project and you will get permission errors when attempting to use it.

To use iDrive e2, you can use the following URL format:

e2://<bucket>/<prefix>
  ?access_key_id=<Access key id>
  &access_secret_key=<Access secret key>

Note that the bucket id is globally unique, so it is recommended using a name that is not likely to conflict with other users, such as prefixing the bucket with the project id or a similar unique value. If you use a simple name, like data or backup it is likely already associated with another project and you will get permission errors when attempting to use it.

To use Aliyun OSS, you can use the following URL format:

aliyunoss://<prefix>
  ?oss-bucket=<Bucket name>
  &oss-endpoint=<Endpoint>
  &oss-access-key-id=<Access Key Id>
  &oos-access-key-secret=<Access Key Secret>

mega://<folder>/<subfolder>
  ?auth-username=<username>
  &auth-password=<password>

Two-factor authorization

It is possible to provide a two-factor key with the option --auth-two-factor-key but since this value changes often, it is not suitable to use in most automated backup settings. This is a design choice from Mega.nz and cannot be fixed by Duplicati.

To use GCS, you can use the following URL format:

cos://<prefix>
  ?cos-bucket=<Bucket name>
  &cos-region=<Bucket region>
  &cos-app-id=<Account AppId>
  &cos-secret-id=<API Secret Id>
  &cos-secret-key=<API Secret Key>

Note that the bucket must be created from within the Cloud Console prior to use.

Storage class

NOTE: The ARCHIVE and DEEP_ARCHIVE storage does not work well with Duplicati. Because Duplicati really likes to verify that things are working as expected you need to disable these checks. You also need to disable cleanup of data after deleting versions. Restores are tricky, because you need to manually restore data to the standard storage class before Duplicati can access it.

jottacloud://<folder>/<subfolder>
  ?authid=<authid>

Device and mount point

Within Jottacloud, each machine registered is a device that can be used for storage, and within each device you an choose the mount point. By default, Duplicati will use the special device Jotta and the mount point Archive.

If you need to store data on another device, you can use the options --jottacloud-device and --jottacloud-mountpoint to set the device and mount point. If you only set the device, the mount point will be set to Duplicati.

Performance tuning

If you need to tune the performance and resource usage to match your specific setup, you can adjust the two parameters:

• --jottacloud-threads: The number of threads used to fetch chunks with

• --jottacloud-chunksize: The size of chunks to download with each thread

Duplicati supports storing files with Backblaze B2, which is a large-scale object storage, similar to S3. With B2 you store "objects" (similar to files) in "buckets" which define various properties shared between the objects. If you use a / in the object prefix, they can be displayed as virtual folders when listing them.

To use the B2 storage destination, use the following URL format:

Create a bucket

You can use the Backblaze UI to create your buckets, but if you need to create buckets with Duplicati, this is also possible. The default is to create private buckets, but you can create public buckets with --b2-create-bucket-type=allPublic.

Performance tuning

You can change the size of file listings to better match pricing and speed through --b2-page-size, which is default set to 500, meaning you will have a list request for each 500 objects. Note that setting this higher may cause the number of requests to go down, but each requests may be priced as a more expensive request.

If you prefer downloads from you custom domain name, you can supply this with --b2-download-url. This setting does not affect uploads.

The pCloud provider was added in Duplicati v2.1.0.100, and is not yet included in a stable release.

To use pCloud, use the following URL format:

pcloud://<host>/<folder>/<subfolder>?authid=<authid>

The <host> value must be one of:

• api.pcloud.com for US based access

• eapi.pcloud.com for EU based access

Due to the way the pCloud authentication system is implemented, the generated AuthID is not stored by the OAuth server and cannot be revoked via the OAuth server. To revoke the token, you must revoke the Duplicati app from your pCloud account, which will revoke all issued tokens.

This also means that after issuing the pCloud token, you do not need to contact the OAuth server again, unlike other OAuth solutions.

To use the Azure Blob Storage destination, you can use the following URL format:

azure://<container>/<prefix>
  ?azure-account-name=<account id>
  &azure-access-key=<access key>

Create container

If you use the UI, the "Test connection" button will prompt you if the container needs to be created.

Using a Shared Access Signature (SAS) token

Instead of using a traditional Access Key, you can also use a SAS token. To use this, supply it instead of the access key, for example:

azure://<container>/<prefix>
  ?azure-account-name=<account id>
  &azure-access-sas-token=<SAS token>