# Duplicati Documentation Welcome to the Duplicati Documentation! This site contains documentation for using the open source Duplicati client, including best practice, pro tips, and trouble shooting. If you cannot find an answer on this site, you can always ask a question on our [helpful forum](https://forum.duplicati.com) 🤗. ### Jump right in
InstallationInstall the Duplicati client/pages/VlBDTwgLOnzSHRawpK4N
Set up a backupConfigure your first backup/pages/lnymEVlRr8jm7zWZdKZR
Configuring a destinationShow all destinations/pages/olm0G2WA0iEPz7p004R5
### Contributing If you spot an error or want to add something to the documentation, head over to the [documentation repository](https://github.com/duplicati/documentation) and open an issue or pull request. # Installation This page describes how to install Duplicati on the various supported platforms ## The Duplicati package types For desktop and laptop users, the most common application type is called the "GUI" package, which is short for Graphical User Interface. The GUI package includes the core components, a webserver to show the user interface and a tray icon (also called a status bar icon). For users installing in environments without a desktop or screen, there are also commandline only, remote management and Docker versions. Depending on your setup, you may also want to use one of those packages on a desktop or laptop. This page covers only the GUI installation. Jump to the section that is relevant to you: * [Install on Windows](#install-duplicati-on-windows) * [Install on Linux](#install-duplicati-on-linux) * [Install on MacOS](#install-duplicati-on-macos) ## Install Duplicati on Windows The most common installation format on Windows is the MSI package. To install on Windows you need to know what kind of processor is on your system. If you are unsure, you are most likely using the 64-bit processor, also known as `x64`. There is also a version supporting `Arm64` processors, and a version for legacy 32-bit Windows called `x86`. Simply head over to the [Duplicati download page](https://duplicati.com/download) and download the relevant MSI package. Once downloaded, double-click the installer. The installation dialog lets you adjust settings to your liking and will install Duplicati. The first time Duplicati starts up, it will open the user interface in your browser. At this point you are ready to [set up a backup](/getting-started/set-up-a-backup-in-the-ui). ## Install Duplicati on MacOS For MacOS the common installation method is to use a DMG file with the application file. Most modern MacOS machines are using the Apple Silicon which is called `Arm64` in Duplicati's packages. If you are on an older Mac that has a 64-bit Intel processor, you can use the `x64` package instead. Once you know which kind of Mac you have, header over to the [Duplicati download page](https://duplicati.com/download) and download the relevant DMG file. Open the file and drag Duplicati into the Application folder, and then you can start Duplicati. The first time Duplicati starts up, it will open the user interface in your browser. At this point you are ready to [set up a backup](/getting-started/set-up-a-backup-in-the-ui). ## Install Duplicati on Linux Most Linux distributions work well with Duplicati but there are only packages for Debian based distributions (Ubuntu, Mint, etc) and for RedHat based based distributions (Fedora, SUSE, etc). For other distributions you may need to manually install some dependencies. For Linux distributions there are packages for the most common 64-bit based system with `x64`, support for `Arm64` and the predecessor `Arm7` aka `ArmHF` which are commonly found in NAS boxes and the older Raspberry Pi series. ### Install on Debian-based Linux (Ubuntu, Mint, etc) To install Duplicati on a Debian based system, first download the `.deb` package matching the system architecture, then run: ```sh sudo dpkg -i duplicati-version-arch.deb ``` This will install all dependencies and place Duplicati in the default location on the target system. The first time Duplicati starts up, it will open the user interface in your browser. At this point you are ready to [set up a backup](/getting-started/set-up-a-backup-in-the-ui). ### Install on RedHat-based Linux (Fedora, SUSE, etc) To install Duplicati on a RedHat-based system, first download the `.rpm` package matching the system architecture, then run: ```sh sudo yum -i duplicati-version-arch.rpm ``` This will install all dependencies and place Duplicati in the default location on the target system. The first time Duplicati starts up, it will open the user interface in your browser. At this point you are ready to [set up a backup](/getting-started/set-up-a-backup-in-the-ui). ### Install on another Linux distribution For other linux distributions you can use the `.zip` file that matches your system architecture. Inside the zip files are all the binaries that are needed, and you can simply place them in a folder that works for your system. Generally, all dependecies are inlcuded in the packages so unless you are using a very slimmed down setup, it should work without additional packages. The first time Duplicati starts up, it will open the user interface in your browser. At this point you are ready to [set up a backup](/getting-started/set-up-a-backup-in-the-ui). ## Upgrading an existing installation If you already have Duplicati installed, the regular way to upgrade is to uninstall the previous version and then install the new one. Duplicati does not delete your configurations or backup databases during uninstall, so your settings and jobs will remain intact. For more details, see [Upgrading and downgrading](/installation-details/release-channels-and-versions/upgrading-and-downgrading). # Set up a backup in the UI Describes how to configure a backup in Duplicati Once Duplicati is running, you can set up a backup through the UI. If the UI is not showing, you can use the [TrayIcon](/duplicati-programs/trayicon) in your system menu bar and choose "Open". If you are asked for a password before logging in to the UI, see [how to access without a password](/configuration-and-management/duplicati-access-password). In the UI, start by clicking "Add backup", and choose the option "Add a new backup":

Configuring a new backup

If you have an existing backup configuration you want to load in, see the [section on import/export](/configuration-and-management/import-and-export-backup-configurations). To set up a new backup there are some details that are required, and these are divided into 5 steps: 1. [Basic configuration](#basic-configuration) (descriptive name, passphrase) 2. [Storage destination](#storage-destination) (where to store the backups) 3. [Source data](#source-data) (what data should be backed up) 4. [Schedule](#schedule) (automatically run backups) 5. [Retention and miscelaneous](#retention-and-miscelaneous) (when to delete old backups and more) ## 1. Basic configuration For the basic configuration, you need to provide a name and setup encryption:

Basic configuration page

The name and description fields can be any text you like, and is only used to display the backup configuration in lists so you can differentiate if you have multiple backups. The encryption setup allows you to choose an encryption method and a passphrase. Encryption adds a minor overhead to the processing, but is generally a good idea to add. If you opt out of encryption, make sure you control the storage destination and have adequate protections in place. {% hint style="warning" %} **Be sure to store the chosen or generated passphrase in a safe location** as it is not possible to recover anything if this passphrase is lost! {% endhint %} To avoid weak passphrases, Duplicati has a built-in passphrase generator as well as a passphrase strength measurer. ## 2. Storage destination The storage destination is arguably the most technical step because it is where you specify how to connect to the storage provider you want to hold your information. Some destinations require only a single setting, where others require multiple. {% hint style="warning" %} Each backup created by Duplicati **requires a separate folder**. Do not create two backups that use the same destination folder as they will keep breaking each other. {% endhint %} Due to the number of supported backends, this page does not contain the instructions for how to configure it. Instead, each of the supported destinations is described in detail on the [destination overview page](/backup-destinations/destination-overview).

Storage destination list

When the details are entered, it is recommended that you use the "Test destination" button which will perform some connection tests that helps reveal any issues with the entered information. When the destination is configured as desired, click the "Continue" button. ## 3. Source data In the third step you need to define what data should be backed up. This part depends on your use. If you are a home user, you may want to back up images and documents. An IT professional may want to back up databases. In the source picker view you can choose the files and folders you would like to back up. If you pick a folder, all subfolders and files in that folder will be included. You can use the UI to unselect some items that you want to exclude, and they will show up without a selection marker.

Selecting source folders

For more advanced uses, you can also use the filters to set up rules for what to include and exclude. See the section on [how filters are evaluated in Duplicati](/security-and-secrets/filters-in-duplicati) if your have advanced needs. Once you are satisfied with the source view, click the "Continue" button to continue to the schedule step. ## 4. Schedule Having an outdated backup is rarely an ideal solution, but remembering to run backups is also tedious and easy to forget. To ensure you have up-to-date backups, there is a built-in scheduler in Duplicati that you can enable to have Duplicati run automatically.

Choosing a schedule to run on

If you prefer to run the backups manually, disable the scheduler, and you can use [ServerUtil](/duplicati-programs/command-line-interface-cli-1/serverutil) to trigger the backups as needed. Once satisfied with the schedule, click "Continue". ## 5. Retention and miscelaneous Even though Duplicati has deduplication and compression to reduce the stored data, it is inevitable that old data is stored that will take up space, but is not needed for restore. In this final configuration step you can decide when old versions are removed and what size of files to store on the destination.

Choosing backup retention

The size of remote volumes is meant for a balanced size usable with cloud-storage and a limited network connection. If you have a fast connection or store files on a local network, consider increasing the size of the remote volumes. For more information see [this page on the tradeoffs between sizes](https://forum.duplicati.com/t/choosing-sizes-in-duplicati/17683). For the retention setting, it is inveitable that the backups will grow as new and changed data is added to the backups. If nothing is ever deleted, the backup size will keep growing in size. With the retention settings you can choose how to automatically remove older versions. The setting "Smart backup retention" is meant to be useful for most users where it keeps one daily backup and then gradually fewer versions going back in time. Once you are satisfied with the settings, click the "Save" button. You have now configured your backup! 🎉 # Running a backup This page describes how to run a backup outside of an automatic schedule With a configured backup, you can have a schedule that runs the backup automatically each day. Having the backup run automatically is recommended because it makes it less likely that the backups are not recent if they are needed. Even if the backup already has a schedule there may be times where you want to manually run a backup. If you have just configured a backup, you may want to run it ahead of the scheduled next run. If you are within the UI you can click the "Start" button for the backup.

A newly configured backup

Once the backup is running, the top area will act as a progress bar that shows how the backup progresses. Note that the first run of a backup is the slowest run because it needs to process every file and folder that is part of the source. On later runs it will recognize what parts have changed and only process the new and changed data. If you need to automate starting a backup without using the UI, you can use [ServerUtil](/duplicati-programs/command-line-interface-cli-1/serverutil) to trigger backups from the commandline. After running a backup, the view will change slightly and show some information about the backup.

After the backup has completed it shows backup details

# Restoring files This page describes how to restore files using the Duplicati user interface The most important reason to make a backup is the ability to recover the data at a later stage, usually due to some unforseen incident. Depending on the incident, the original configuration may not be available. To start a restore process in Duplicati, start on the "Restore" page.

Choosing a restore approach

If the backup configuration is already existing on the machine, you can choose it from the list. In this case you can click "Restore" and skip to the section on [choosing the files to restore](#choosing-files-to-restore). The restore and browsing process are fastest when using a configured backup, because Duplicati can query a local database with information. If the local database is not present, Duplicati needs to fetch enough information from the remote storage to build a partial database when performing the restore. If you have exported the backup configuration and have the configuration available, you can click the "Start" button on "Restore from configuration" and skip to the [restore from configuration section](#restore-from-configuration). You can also read up on how to [import and export configurations](/configuration-and-management/import-and-export-backup-configurations). ## Direct restore from backup files To restore files from the backup, Duplicati needs only to know how to access the files and the encryption passphrase (if any). If you do not have the passphrase, it is not possible to restore. {% hint style="info" %} This option is useful restoring from another machine than the one that made the backup originally, but is not recommended for regular operations due to the need to download and process more data. {% endhint %} To restore directly from the backup files, the first step is to provide the destination details. These details are the same as you supplied initially when creating the backup. if you are using a cloud provider, you can usually get the needed information via your account on the vendors website. You can see a list of the [destinations that are supported by Duplicati](/backup-destinations/destination-overview).

Supply storage destination details

Once the details are entered, it is recommended to use the "Test connection" button to ensure that the connection is working correctly. Then click the "Continue" button.

Supply the encryption passphrase

If the backup is not encrypted, leave the field empty. When ready, click "Continue" and Duplicati will examine the remote destination and figure out what backups are present. After working through the information, you can [choose files to restore](#choosing-files-to-restore). ## Restore from configuration If you have a configuration file you can use the information in that file to avoid entering it manually. If you need to restore more than once, it may be faster to [import the configuration](/configuration-and-management/import-and-export-backup-configurations) and [rebuild the local database](/database-and-storage/the-local-database). After the database is built, you can choose the configuration from the list and skip to [choosing files to restore](#choosing-files-to-restore).

Restoring with a backup configuration

In the dialog, provide the exported configuration file. If the file is encrypted, you will be asked to enter the file passphrase. {% hint style="info" %} The passphrase the configuration file is encrypted with is not neccesarily the same as the passphrase used to encrypt the backup with. {% endhint %} Once the configuration is correct, click the "Restore" button and you will be ready to choose files to restore. ## Choosing files to restore Once Duplicati has a connection to the remote destination it will find all the backups that were made. It will then choose the most recent version and list the files from within that version. Use the dropdown to select the version to restore from, and then choose the files that you want to restore.

Choosing files to restore

Click the files or folders that you want to restore and then click "Continue". ## Choosing restore options When restoring there are a few options that control how the files are restored.

Choosing restore options

If you want to restore a file to a previous state, you can leave the settings to their defaults. If you are unsure if you want to revert, or need to examine the files before replacing the current versions, you can choose to restore to another destination. If the folder you are restoring to is not empty, you can choose to store multiple versions of the files by appending the restore timestamp to the filename. This is especially useful if you are restoring multiple versions into a target folder for comparison. Duplicati will not restore permissions by default because the users and groups that were present on the machine that made the backup may not be present on the machine being restored to. Restoring the permissions can cause you to be unable to access the restored files, if your user does not have the necessary permissions. When satisfied with the settings, click the "Submit" button and the restore process will restore the files. Once the restore has completed, you will see the restore summary page:
# Choosing Duplicati Type This page describes the different ways to run Duplicati When using Duplicati, you need to decide on what type of instance you want to use. Duplicati is designed to be flexible and work with many different setups, but generally you can use this overview to decide what is best for you: * Home user, single desktop machine: [TrayIcon](/duplicati-programs/trayicon) or [Agent](/duplicati-programs/agent) * Server backup or headless: [Server](/duplicati-programs/server), [CLI](/duplicati-programs/command-line-interface-cli) or [Agent](/duplicati-programs/agent) * Multiple machines: [Server](/duplicati-programs/server), [CLI](/duplicati-programs/command-line-interface-cli) or [Agent](/duplicati-programs/agent) ## Duplicati Package Types The [Duplicati installation packages](/installation-details/package-options) are available in the following formats: * **GUI** includes: [TrayIcon](/duplicati-programs/trayicon), [Server](/duplicati-programs/server), and [CLI](/duplicati-programs/command-line-interface-cli-1) * **CLI** includes: [Server](/duplicati-programs/server) and [CLI](/duplicati-programs/command-line-interface-cli-1) * **Agent** includes: [Agent](/duplicati-programs/agent) and [CLI](/duplicati-programs/command-line-interface-cli-1) ## The TrayIcon The [TrayIcon](/duplicati-programs/trayicon) is meant to be the simplest way to run Duplicati with the minimal amount of effort required. The TrayIcon starts as a single process, registers with the machine desktop environment and shows a small icon in the system status bar (usually to the right, either top or bottom of the screen). When running, the TrayIcon gives a visual indication of the current status, and provides access to the visual user interface by opening a browser window. ## The Server The [Server](/duplicati-programs/server) mode is intended for users who want to run the full Duplicati with a user interface, but without a desktop connection. When running the Server it is usually running as a system service so it has elevated privileges and is started automatically with the system. When running the server it will emit log messages to the system log and it will expose a web server that can be accessed via a browser. **Beware** that if you are running the Server as root/Administrator you are also running a web server with the same privileges that you need to protect. When the Server is running it will lock down access to only listen on the loopback adapter and refuse connections not using an IP address as the hostname. If you need to access the Server from another machine, make sure you protect it and [enable remote access](/duplicati-programs/server#external-access-to-the-server) and also add [HTTPS protection](/duplicati-programs/server#ssl-tls-support). When running the Server you also need to [configure a password](/configuration-and-management/duplicati-access-password), either by getting a [signing token from the logs](/configuration-and-management/duplicati-access-password#temporary-signin-token), [changing the password](/configuration-and-management/duplicati-access-password#change-password-with-serverutil), or [setting one explicitly](/duplicati-programs/server#configuring-the-server-password). ## The Agent The [Agent](/duplicati-programs/agent) mode is intended for users who wants to run Duplicati with remote access through the [Duplicati Console](/duplicati-console/using-the-duplicati-console). The benefit from this is that you do not need to provide any local access as all access is protected with HTTPS and additional channel encryption from the Agent to the browser you are using. {% hint style="info" %} The Agent mode is only available on the Enterprise plan {% endhint %} If you have multiple machines to manage, using the console enables you to access all the backups, settings, logs, controls, etc. from one place. ## The Command Line Interface (CLI) The [CLI](/duplicati-programs/command-line-interface-cli) mode is intended for advanced users who prefer to manage and configure each of the backups manually. The typical use for this is a server-like setup where the backups are running as cron scheduled tasks or triggered with some external tool. ## Mixing types For some additional flexibility in configurations it is also possible to combine the different types in some ways. ### Combining Server and TrayIcon It the server is used primarily to elevate privileges, it is possible to have the TrayIcon run in the local user desktop and connect to an already running Server. To do this, change the TrayIcon commandline and add additional arguments: ``` duplicati --no-hosted-server --hosturl=http://localhost:8200 --webservice-password= ``` The `--no-hosted-server` argument disables launching another (competing) server, and the two other arguments will give information on how to reach the running server. ### Triggering Server jobs externally If you prefer to use the Server (or TrayIcon) but would like to trigger the backups with an external scheduler or event system, you can use the [ServerUtil](/duplicati-programs/command-line-interface-cli-1/serverutil) to trigger a backup or pause/resume the server. ### Using the CLI for on Server backups If you are using the Server (or TrayIcon) but you want to run a command that is not in the UI, it is possible to use the CLI to run commands on the backups defined in the Server. Note that the Server and CLI use different ways of keeping track of the [local database](https://github.com/duplicati/documentation/blob/main/database-and-storage/the-local-database.md), so you need to obtain the storage destination url and the database path from the Server and then run the CLI. # Using Duplicati with Linux This page describes how to use Duplicati with Linux Before you can install Duplicati, you need to decide on three different parameters: * The type you want: [GUI](/duplicati-programs/trayicon), [Server](/duplicati-programs/server), [Agent](/duplicati-programs/agent), [CLI](/duplicati-programs/command-line-interface-cli). * Your package manager: `apt`, `yum` or something else. * You machine CPU type: x64, Arm64 or Arm7 ## Deciding on type To use Duplicati on Linux, you first need to decide which kind of instance you want: GUI (aka [TrayIcon](/duplicati-programs/trayicon)), [Server](/duplicati-programs/server), [Agent](/duplicati-programs/agent), [CLI](/duplicati-programs/command-line-interface-cli)). The section on [Choosing Duplicati Type](/platform-specific-guides/choosing-duplicati-type) has more details on each of the different types. ## Determine package manager Next step is checking what Linux distribution you are using. Duplicati supports running on most Linux distros, but does not yet support FreeBSD. If you are using a Debian-based operating system, such as Ubuntu or Mint, you can use the `.deb` package, and for RedHat-based operating system, such as Fedora or SUSE, you can use the `.rpm` packages. For other operating systems you can use the `.zip` package, or check if your package manager already carries Duplicati. ## Determine CPU architecture Finally you need to locate information on what CPU architecture you are using: * x64: 64bit Intel or AMD based CPU. This is the most common CPU at this time. * Arm64: 64bit ARM based CPU. Used in Raspberry Pi Model 4 and some Laptops and Servers. * Arm7: 32bit ARM based CPU. Used in Raspberry Pi Model 3 and older, and some NAS devices. ## Installing the package Once you have decided the on (type, distro, cpu) combination you are ready to download the package. The full list of packages can be obtained via the main [download page](https://duplicati.com/download), and then clicking "Other versions". Refer to the [installation page](/getting-started/installation) for details on how to install the packages, or simply use the package manager in your system. ## Using the TrayIcon For users with a desktop environment and no special requirements, the [TrayIcon](/duplicati-programs/trayicon) instance is the recommended way to run Duplicati. If you are using either `.deb` or `.rpm` you should see Duplicati in the program menu, and you can launch it from there. If you do not see Duplicati in the program menu, you can start it with: ``` duplicati ``` When running the TrayIcon in a user context, it will create a folder in your home folder, typically `~/.config/Duplicati` where it stores the local databases and the Server database with the backup configurations. ## Using the Server The [Server](/duplicati-programs/server) is a regular executable and can simply be invoked with: ``` duplicati-server ``` When invoked as a regular user, it will use the same folder, `~/.config/Duplicati`, as the [TrayIcon](/duplicati-programs/trayicon) and share the configuration. Besides the configuration listed below, it is also possible to run [Duplicati in Docker](/platform-specific-guides/using-duplicati-from-docker). ### Using Server as a Service If you would like to run the Server as a service the `.rpm` and `.deb` packages includes a regular systemd service. If you are installing from the `.zip` package, you can grab the [service file from the source code](https://github.com/duplicati/duplicati/tree/master/ReleaseBuilder/Resources/debian/systemd) and install it manually on your system. If you need to pass options to the server, edit the settings file, usually at `/etc/default/duplicati`. Make sure you only edit the configuration file and not the service file as it will be overwritten when a new version is installed. The settings file should look something like this: ``` # Defaults for duplicati initscript # sourced by /etc/init.d/duplicati # installed at /etc/default/duplicati by the maintainer scripts # # This is a POSIX shell fragment # # Additional options that are passed to the Daemon. DAEMON_OPTS="" ``` You can use `DAEMON_OPTS` to pass arguments to `duplicati-server`, such as `--webservice-password=`. To enable the service to auto-start, reload configurations, start the service and report the status, run the following commands: ```sh sudo systemctl enable duplicati.service sudo systemctl daemon-reload sudo systemctl start duplicati.service sudo systemctl status duplicati.service ``` The server is now running and will automatically start when you restart the machine. **Note:** the service runs in the `root` user context, so files will be stored in `/root/.config/Duplicati` on most systems, but in `/Duplicati` on other systems. Use the `DAEMON_OPTS` to add `--server-datafolder=` if you want a specific location. To check the logs (and possibly obtain a signin link), the following command can usually be used: ```sh sudo journalctl --unit=duplicati ``` #### Linux systemd service and supplementary groups When Duplicati runs under a dedicated service account on Linux, systemd does not automatically include that user's supplementary groups. If you add the service account to additional groups (for example, to access NFS or Samba shares) you should explicitly configure the unit file so systemd grants those memberships when the service starts. To supply the groups, use the edit functionality: ```bash sudo systemctl edit duplicati.service ``` Then edit the file and add the supplementary groups: ``` [Service] SupplementaryGroups=group1 group2 ``` When you save and exit, an override file will be created, typically in `/etc/systemd/system/duplicati.service.d/override.conf` . This method ensures that a package upgrade does not erase you edits. Finally reload and restart the service so the new group membership takes effect: ```sh sudo systemctl daemon-reload sudo systemctl restart duplicati ``` ## Using the Agent With the [Agent](/duplicati-programs/agent) there is a minimal setup required, which is to register the machine with the Duplicati Console. When installing either the `.rpm` or `.deb` packages, it will automatically register the `duplicati-agent.service` for startup. If you are using the `.zip` installation, you can find the [agent service in the source code](https://github.com/duplicati/duplicati/tree/master/ReleaseBuilder/Resources/debian/systemd) and manually register it: ```sh sudo systemctl enable duplicati-agent.service sudo systemctl start duplicati-agent.service ``` When the Agent starts, it will emit a registration link to the log, and you can usually see it with the following command: ```sh sudo journalctl --unit=duplicati ``` If you are using a [pre-authenticated link](/duplicati-programs/agent#registering-the-machine), you can run the following command to activate the registration: ```sh duplicati-agent register "" ``` After registration is complete, restart the service to pick up the new credentials: ```sh sudo systemctl restart duplicati-agent ``` ## Using the CLI Using the CLI is simply a matter of invoking the binary: ```sh duplicati-cli help ``` Since the CLI also needs a local database for each backup, it will use the same location as [described for the Server](#using-the-server) above to place databases. In addition to this, it will keep a small file called `dbconfig.json` in the storage folder where it maps URLs to databases. The intention of this is to avoid manually specifying the `--dbpath` parameter on every invocation. If you specify the `--dbpath` parameter, it will not use the `dbconfig.json` file and it will not store anything in the local datafolder. ## Using the support programs Each package of Duplicati contains a number of support utilities, such as the [RecoveryTool](/duplicati-programs/command-line-interface-cli-1/recoverytool). Each of these can be invoked from the commandline with a `duplicati-*` name and all contain built-in help. For example, to invoke [ServerUtil](/duplicati-programs/command-line-interface-cli-1/serverutil), run: ```sh duplicati-server-util help ``` ## Handling locked files By default, Duplicati will honor Linux advisory locking and refuse to open files that are locked. The logic for this is that it is not guaranteed that reading the file while it is locked, will result in a useful file when restored. However, many Linux applications ignore the locks because the default file operations ignores them as well. If you prefer that Duplicati ignores locked files, and just reads what it finds, you can set the advanced option: ``` --ignore-advisory-locking=true ``` # Using Duplicati with MacOS This page describes common scenarios for configuring Duplicati with MacOS Before you can install Duplicati, you need to decide on two different parameters: * The type you want: [GUI](/duplicati-programs/trayicon), [Server](/duplicati-programs/server), [Agent](/duplicati-programs/agent), [CLI](/duplicati-programs/command-line-interface-cli). * You machine CPU type: Arm64 or x64 ## Deciding on type To use Duplicati on MacOS, you first need to decide which kind of instance you want: GUI (aka [TrayIcon](/duplicati-programs/trayicon)), [Server](/duplicati-programs/server), [Agent](/duplicati-programs/agent), [CLI](/duplicati-programs/command-line-interface-cli). The section on [Choosing Duplicati Type](/platform-specific-guides/choosing-duplicati-type) has more details on each of the different types. For home users, the common choice is the GUI package in `.dmg`format. For enterprise rollouts, you can choose the `.pkg` packages. ## Determine CPU architecture Your Mac is most likely using Arm64 with one of the M1, M2, M3, or M4 chips. If you have an older Mac, it may use the Intel x64 chipset. To see what CPU you have, click the Apple icon and choose "About this Mac". In the field labelled "Chip" it will either show Intel (x64) or M1, M2, M3, M4 (Arm64). ## Installing the package The packages can be obtained via the [main download page](https://duplicati.com/download). The default package shown on the page is the MacOS Arm64 GUI package in `.dmg` format. If you need another version click the "Other versions" link at the bottom of the page. If you are using the `.dmg` package the installation works similar to other application, simply open the `.dmg` file and drag Duplicati into Applications. Note that with the `.dmg` package, Duplicati is not set to start automatically with your Mac, but if you restart with the option to re-open running programs, Duplicati will start on login. If you are using the `.pkg` package, Duplicati will install a `launchAgent` that ensures Duplicati starts on reboots. The CLI package installs a stub file that is not active, so you can edit the `launchAgent` and have it start the Server if you prefer. ## Using the TrayIcon If you have installed the GUI package, you will have Duplicati installed in `/Applications` and it can be started like any other application. Once Duplicati is started, it will place itself in the menu bar near the clock and battery icons. Because Duplicati is meant to be a background program, there is no Duplicati icon in the dock. On the first start Duplicati will also open your browser and allow you to configure your backups. If you need access to the UI again later, locate the TrayIcon in the status bar, click it and click "Open". If you install the CLI or Agent packages, the Duplicati application is not available. ## Using the Server If you install the CLI package, Duplicati binaries are placed in `/usr/local/duplicati` and symlinked into `/usr/local/bin` and you can start the server simply by running: ```sh duplicati-server ``` When invoked as a regular user, it will use the same folder, `~/Library/Application Support/Duplicati`, as the [TrayIcon](/duplicati-programs/trayicon) and share the configuration. **Note**: If you install the GUI package or install from homebrew, Duplicati's binaries are not symlinked into the paths searched by MacOS. You can invoke the binaries by supplying the full path: ```sh /Applications/Duplicati.app/Contents/MacOS/duplicati-server ``` ## Using the Agent With the [Agent](/duplicati-programs/agent) there is a minimal setup required, which is to register the machine with the Duplicati Console. When installing the Agent package, it will automatically register the Duplicati agent with a `launchAgent` that starts Duplicati in an Agent mode. If the Agent is not registered with the Console, it will open the default browser and ask to be registered. Once registered, it will run in the background and be avilable on the Duplicati Console for management. If you have a [pre-authenticated link](/duplicati-programs/agent#registering-the-machine) for registering the machine, you can place a file in the installation folder (or [data folder](/database-and-storage/the-server-database)) with content similar to: ```json { "args": { "agent": [ "--agent-registration-url=" ] } } ``` ## Using the CLI Using the CLI is simply a matter of invoking the binary: ```sh duplicati-cli help ``` Since the CLI also needs a local database for each backup, it will use the same location as [described for the Server](#using-the-server) above to place databases. In addition to this, it will keep a small file called `dbconfig.json` in the storage folder where it maps URLs to databases. The intention of this is to avoid manually specifying the `--dbpath` parameter on every invocation. If you specify the `--dbpath` parameter, it will not use the `dbconfig.json` file and it will not store anything in the local datafolder. **Note**: If you install the GUI package or install from homebrew, Duplicati's binaries are not symlinked into the paths searched by MacOS. You can invoke the binaries by supplying the full path: ```sh /Applications/Duplicati.app/Contents/MacOS/duplicati-cli help ``` ## Using the support programs Each package of Duplicati contains a number of support utilities, such as the [RecoveryTool](/duplicati-programs/command-line-interface-cli-1/recoverytool). Each of these can be invoked from the commandline with a `duplicati-*` name and all contain built-in help. For example, to invoke [ServerUtil](/duplicati-programs/command-line-interface-cli-1/serverutil), run: ```sh duplicati-server-util help ``` **Note**: If you install the GUI package or install from homebrew, Duplicati's binaries are not symlinked into the paths searched by MacOS. You can invoke the binaries by supplying the full path: ```sh /Applications/Duplicati.app/Contents/MacOS/duplicati-server-util help ``` # Using Duplicati with Windows This page describes common scenarios for configuring Duplicati with Windows Before you can install Duplicati, you need to decide on three different parameters: * The type you want: [GUI](/duplicati-programs/trayicon), [Server](/duplicati-programs/server), [Agent](/duplicati-programs/agent), [CLI](/duplicati-programs/command-line-interface-cli). * You machine CPU type: x64, Arm64, or x86 (32 bit) ## Deciding on type To use Duplicati on Windows, you first need to decide which kind of instance you want: GUI (aka [TrayIcon](/duplicati-programs/trayicon)), [Server](/duplicati-programs/server), [Agent](/duplicati-programs/agent), [CLI](/duplicati-programs/command-line-interface-cli). The section on [Choosing Duplicati Type](/platform-specific-guides/choosing-duplicati-type) has more details on each of the different types. ## Determine CPU architecture Finally you need to locate information on what CPU architecture you are using: * x64: 64bit Intel or AMD based CPU. This is the most common CPU at this time. * Arm64: 64bit ARM based CPU. Some laptops, tablets and servers use it. * x86: 32bit Intel or AMD based CPU. Note that Windows 10 was the last version to support 32 bit processors. If you are in doubt, you can try the x64 version, or use [Microsofts guide for determining the CPU](https://support.microsoft.com/en-us/windows/32-bit-and-64-bit-windows-frequently-asked-questions-c6ca9541-8dce-4d48-0415-94a3faa2e13d). ## Installing the package Once you have decided the on package you want, you are ready to download the package. The default version shown on the main [download page](https://duplicati.com/download) is the x64 GUI version in `.msi` format. The full list of packages can be obtained via the main [download page](https://duplicati.com/download), and then clicking "Other versions". ## Using the TrayIcon For users with a desktop environment and no special requirements, the [TrayIcon](/duplicati-programs/trayicon) instance is the recommended way to run Duplicati. If you are using the `.msi` package to install Duplicati, you will see an option to automatically start Duplicati, as well as create a shortcut on your desktop and in the start menu. If you need to manually start Duplicati, you can find the executable in: ``` C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.exe ``` When running the TrayIcon in a user context, it will create a folder in your home folder, typically `C:\Users\\AppData\Local\Duplicati` where it stores the local databases and the Server database with the backup configurations. ## Using the Server The [Server](/duplicati-programs/server) is a regular executable and can simply be invoked with: ``` C:\Program Files\Duplicati 2\Duplicati.Server.exe ``` When invoked as a regular user, it will use the same folder, `C:\Users\\AppData\Local\Duplicati`, as the [TrayIcon](/duplicati-programs/trayicon) and share the configuration. ### Running the Server as a Windows Service If you want to run Duplicati as a Windows Service, you can use the bundled service tool to install/uninstall the service: ``` C:\Program Files\Duplicati 2\Duplicati.WindowsService.exe INSTALL C:\Program Files\Duplicati 2\Duplicati.WindowsService.exe UNINSTALL ``` When installing the Service it will automatically start, and likewise, uninstalling it will stop the service. If you need to pass options to the server, you can provide them to the INSTALL command: {% code overflow="wrap" %} ``` C:\Program Files\Duplicati 2\Duplicati.WindowsService.exe INSTALL --webservice-port=8100 --server-datafolder= ``` {% endcode %} You can also use the [preload.json](/configuration-and-management/preload-settings) file to pass settings to the Server when running as a service, which allows you to change the settings without the uninstall/install cycle (you still need to restart the service). **Note**: When running the Windows Service it will default to use port 8200 and fail it that port is not available. If you are running the TrayIcon, that will run a **different** instance, usually at port 8300. If you want to connect the TrayIcon to the Windows Service, edit the shortcut to Duplicati: {% code overflow="wrap" %} ``` C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.exe --no-hosted-server --host-url=http://localhost:8200 --webservice-password= ``` {% endcode %} ## Using the Agent With the [Agent](/duplicati-programs/agent) there is a minimal setup required, which is to register the machine with the Duplicati Console. The default installation is to install the Agent as a Windows Service, meaning it will run in the LocalService system account, instead of the local user. Due to this, it will not be able to open the browser and start the registration process for you. Instead, you must look into the Windows Event Viewer and extract the registration link from there. You can also register the Agent, using the Agent executable: ```sh C:\Program Files\Duplicati 2\Duplicati.Agent.exe register ``` After the Agent has been registered, restart the service and it will now be available on the Duplicati Console. If you have a [pre-authenticated link](/duplicati-programs/agent#registering-the-machine) for registering the machine, and would like to automate the process, you can place a file in the installation folder (or [data folder](/database-and-storage/the-server-database)) with content similar to: ```json { "args": { "agent": ["--agent-registration-url="] } } ``` ## Using the CLI Using the CLI is simply a matter of invoking the binary: ```sh C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe help ``` Since the CLI also needs a local database for each backup, it will use the same location as [described for the Server](#using-the-server) above to place databases. In addition to this, it will keep a small file called `dbconfig.json` in the storage folder where it maps URLs to databases. The intention of this is to avoid manually specifying the `--dbpath` parameter on every invocation. If you specify the `--dbpath` parameter, it will not use the `dbconfig.json` file and it will not store anything in the local datafolder. ## Using the support programs Each package of Duplicati contains a number of support utilities, such as the [RecoveryTool](/duplicati-programs/command-line-interface-cli-1/recoverytool). Each of these can be invoked from the commandline with their executable name and all contain built-in help. For example, to invoke [ServerUtil](/duplicati-programs/command-line-interface-cli-1/serverutil), run: ```sh C:\Program Files\Duplicati 2\Duplicati.CommandLine.ServerUtil.exe help ``` ## Handling locked files On Windows, file locking is an integrated part of how programs expect the system to work. This is problematic for backup systems that want to read all the files, locked or not, to ensure they are backed up. Windows offers two primary ways to read locked files: VSS and BackupRead. Both ways require that the caller has the SeBackupPrivilege enabled and will then permit the caller to read the files. Unfortunately, this is implemented with a permission override as well, meaning that a caller with the SeBackupPrivilege has access to read any file on the system. In other words: it is not possible to read the user's own locked files without also granting full read access to the system. Since it is a security issue to have a regular account that has access to all files, the two options are: run as a user with elevated privileges (e.g., Administrator) or ignore locked files. For simplicity, most users prefer to [run Duplicati as a service](#running-the-server-as-a-windows-service), which will run in the LocalSystem account that already has the SeBackupPrivilege set. ### Volume Snapshot Service (VSS) The most robust and heavy handed way of making a backup on Windows is to use VSS to create a snapshot of the current disk, and then read this. The implementation on Windows signals all programs that want to write to disk that a snapshot is being prepared, waits for them to flush to disk, makes the snapshot and lets all programs continue. This approach ensures that the disk snapshot is in a consistent state so there are not partial writes being backed up. To enable VSS, set the advanced option `--snapshot-policy=required` . If you are using Duplicati 2.1.1.0 or later, VSS will be enabled by default if the user context has the SeBackupPrivilege. ### BackupRead The BackupRead method does not create a snapshot and instead relies on a Windows API call that allows a program to read files for backup purposes. The benefit from this is that you do not need to create disk snapshots, which requires extra disk space and co-operation from other programs. To enable BackupRead, set `--backupread-policy=required` and `--snapshot-policy=off` to ensure you are only using BackupRead. Note that the `--backupread-policy` option is currently only available in the [canary builds](https://github.com/duplicati/documentation/tree/main/detailed-descriptions/installation-details/release-channels-and-versions/README.md). # Run Duplicati as a Windows Service This page explains when you need to run Duplicati as a service and how to configure it {% hint style="warning" %} The setup described here was introduced in Canary 2.3.0.104 and is not yet available in a stable release {% endhint %} On Windows the operating system will lock files while they are in use. While there are advantages to this, it is problematic for a backup system because the files cannot be read by the backup tool either. To get access to locked files you need Administrator privileges, meaning you need to run Duplicati in a special way. Once way of running Duplicati with elevated privileges is to run it as a service. The benefit from this is that the backup runs even if no users are logged in to the machine. When Duplicati runs as a Windows Service it will change the default to use the VSS service when running backup, such that locked files can be read. {% hint style="danger" %} When running Duplicati as a Windows Service this means Duplicati has full access to the machine, and you must be careful not to allow access to the service. Make sure that you do not store the password insecurely and do not enable remote access to Duplicati. {% endhint %} ## Using the Windows Installer When running the Windows Installer package, simply check the "Install Duplicati as a Windows Service" option:
After the installation has completed, you will get the option to copy the generated password by clicking "Copy password":
The installer will automatically configure the current user, such that it can connect securely to the service. If you need to let another user access the service, or need to use some of the command line tools, you can copy the password by clicking the "Copy password" button. When you start Duplicati it will initially start in a disconnected state, and then connect to the server using the password stored in the Windows Credential Manager. The password is only applied on the first install and will not be retrivable later, including if you uninstall Duplicati. See the [password reset section below](#reset-password-for-windows-service) if you need to change the password or later want to use the password. ## Automated installation with a service By default, Duplicati will install as a regular application and will be able to run as the logged in user. To configure the installer to install the service you need to pass options to `msiexec` explaining what you want. A few examples: **Install Service + all shortcuts** {% code overflow="wrap" %} ``` msiexec /i Duplicati.msi /qn ADDLOCAL=DuplicatiCore,DuplicatiServiceFeature,DuplicatiDesktopShortCutFeature,DuplicatiProgramMenuShortCutFeature,DuplicatiStartupShortCutFeature ``` {% endcode %} **Install Service only and no shortcuts:** {% code overflow="wrap" %} ``` msiexec /i Duplicati.msi /qn ADDLOCAL=DuplicatiCore,DuplicatiServiceFeature REMOVE=DuplicatiDesktopShortCutFeature,DuplicatiProgramMenuShortCutFeature,DuplicatiStartupShortCutFeature ``` {% endcode %} **Install Service + only desktop shortcut** {% code overflow="wrap" %} ``` msiexec /i Duplicati.msi /qn ADDLOCAL=DuplicatiCore,DuplicatiServiceFeature,DuplicatiDesktopShortCutFeature REMOVE=DuplicatiProgramMenuShortCutFeature,DuplicatiStartupShortCutFeature ``` {% endcode %} In the examples, the `/qn` switch toggles unattended install which means no user interface will be shown. It also means that all actions will run in the installer context so the local user will not be configured to access Duplicati. You can provide the desired password to the installer as an option: {% code overflow="wrap" %} ``` msiexec /i Duplicati.msi DUPLICATI_SERVICE_PASSWORD= ``` {% endcode %} Note that providing the password to the installer may cause it to be logged. If you do not provide a password for unattended installs, Duplicati will default to writing a login link to the Windows Event log that can be used to log in. See also the [section on the access password](/configuration-and-management/duplicati-access-password) for other ways to manage the password. You will need the password to access the UI and to allow the TrayIcon to connect to the service. ## Installing a [preload.json](/configuration-and-management/preload-settings) file If you want to configure the installation, you can place a `preload.json` file in the same folder where the MSI is placed, and then launch with the option `INSTALL_PRELOAD=true` to have the installer pick up the `preload.json` file and install it along with the rest of the files: {% code overflow="wrap" %} ``` msiexec /i Duplicati.msi INSTALL_PRELOAD=true ``` {% endcode %} Without the option, the preload file is not installed even if it is found. This design prevents situations where a malicious file is placed in the source folder and unexpectedly configures the application. ## Reset password for Windows Service The [`WindowsService.exe`](/duplicati-programs/service) program supports the command `reset-password` which will set a password and then restart the service. This provides a secure way to set a new password without risking that the password is shown in logs. If Duplicati was installed with a silent install, this option allows you to change the password via a command line operation: {% code overflow="wrap" %} ``` "C:\Program Files\Duplicati 2\Duplicati.WindowsService.exe" reset-password ``` {% endcode %} The program will ask you to enter a password or offer to generate one. You can also see the [section about access password](/configuration-and-management/duplicati-access-password) for more ways to set the password. If you are using the [TrayIcon](/duplicati-programs/trayicon) to connect to the service, right-click the TrayIcon and choose "Change password", then enter the updated password. Manual install If you need additional customization beyond what the MSI package offers, see the [`WindowsService.exe`](/duplicati-programs/service) page for details on how to configure the service from a commandline or scripts. # Hyper-V backup This page describes how to make backups of Hyper-V machines On Windows, Duplicati has built-in support for finding Hyper-V machines if running with sufficient privileges. To include a Hyper-V machine in a backup, visit the [Source Data](https://docs.duplicati.com/platform-specific-guides/using-duplicati-with-windows/pages/lnymEVlRr8jm7zWZdKZR#id-3.-source-data) step, and find "Hyper-V Machines" in the treeview:
By choosing the top-level "Hyper-V Machines", all Hyper-V machines will be included in the backup. If new machines are added later, these will automatically be included. ## Exclude a VM If you want to exclude one or more machines, expand the "Hyper-V Machines" entry, and click any VM to toggle its inclusion status:
This way of setting it up will mean that new machines will automatically be backed up, but the items you deselected will not be backed up. Note the filters to the right are automatically populated when clicking the VMs. The filter relies on the Hyper-V VMs internal ID, so recreating a VM will usually generate a new ID for the machine. ## Include only specific VMs If you prefer to manually specify VMs to include so new VMs are not automatically included, make sure the "Hyper-V Machines" element is not selected, then expand the element and click the specific VMs that you want to include in the backup:
This will ensure that new machines will not automatically be included. Notice the "Paths" section on the right is automatically filled with the VMs ID. Beware that recreating the VM will usually cause it to be assigned a new ID. ## Backup of running Hyper-V machines To support backup of running Hyper-V machines, Duplicati relies on [Volume Shadow Copy Service (VSS)](https://learn.microsoft.com/en-us/windows-server/storage/file-server/volume-shadow-copy-service). The VSS setup is a series of steps that ensures that the data Duplicati backs up is consistent and can be restored without issues. {% hint style="warning" %} Using VSS requires the `SE_BACKUP` privilege which often means running Duplicati as Administrator, or as a [Windows Service](/platform-specific-guides/using-duplicati-with-windows/run-duplicati-as-a-windows-service) {% endhint %} In short, the sequence is as follows: * Duplicati asks for a snapshot * VSS asks all VSS-aware application to flush to disk * Applications flush and refrain from updating the disk * VSS creates a snapshot, and tells all applications to resume * Anything written to disk while the snapshot is active uses Copy-on-write * Duplicati reads the consistent snapshot and releases it after the backup ends This requires a small amount of extra disk and memory to store the data that changes while the snapshot is active, but this is released once the snapshot is released. The end result is that Hyper-V machines can continue to run with no downtime, and Duplicati can read a consistent copy. To ensure that Hyper-V machines are correctly backed up, change the advanced option `--snapshot-policy` to `Required`:
This setting makes Duplicati abort if it is not able to secure a VSS snapshot. # MSSQL backup This page describes how to make backups of MSSQL databases On Windows, Duplicati has built-in support for finding MSSQL Servers if running with sufficient privileges. To include MSSQL databases in a backup, visit the [Source Data](https://docs.duplicati.com/platform-specific-guides/using-duplicati-with-windows/pages/lnymEVlRr8jm7zWZdKZR#id-3.-source-data) step, and find "Microsoft SQL Servers" in the treeview:
By choosing the top-level "Microsoft SQL Servers", all MSSQL databases will be included in the backup. If new databases are added later, these will automatically be included. ## Exclude a Database If you want to exclude one or more databases, expand the "Microsoft SQL Servers" entry, then expand the Server entry, and click any database to toggle its inclusion status:
This way of setting it up will mean that new databases will automatically be backed up, but the items you deselected will not be backed up. Note the filters to the right are automatically populated when clicking the databases. The filter relies on the name of both the server and the database, so renaming either will require the setup to be changed. ## Include only specific databases If you prefer to manually specify databases to include so new databases are not automatically included, make sure the "Microsoft SQL Servers" element is not selected, then expand the element, expand the server name, and click the specific databases that you want to include in the backup:
This will ensure that new databases will not automatically be included. Notice the "Paths" section on the right is automatically filled with the server- and database names. If you rename either the server or the database, you will need to edit the backup. ## Backup of active MSSQL databases To support backup of active MSSQL databases, Duplicati relies on [Volume Shadow Copy Service (VSS)](https://learn.microsoft.com/en-us/windows-server/storage/file-server/volume-shadow-copy-service). The VSS setup is a series of steps that ensures that the data Duplicati backs up is consistent and can be restored without issues. {% hint style="warning" %} Using VSS requires the `SE_BACKUP` privilege which often means running Duplicati as Administrator, or as a [Windows Service](/platform-specific-guides/using-duplicati-with-windows/run-duplicati-as-a-windows-service) {% endhint %} In short, the sequence is as follows: * Duplicati asks for a snapshot * VSS asks all VSS-aware application to flush to disk * Applications flush and refrain from updating the disk * VSS creates a snapshot, and tells all applications to resume * Anything written to disk while the snapshot is active uses Copy-on-write * Duplicati reads the consistent snapshot and releases it after the backup ends This requires a small amount of extra disk and memory to store the data that changes while the snapshot is active, but this is released once the snapshot is released. The end result is that MSSQL databases can continue to run with no downtime, and Duplicati can read a consistent copy. To ensure that MSSQL databases are correctly backed up, change the advanced option `--snapshot-policy` to `Required`:
This setting makes Duplicati abort if it is not able to secure a VSS snapshot. # Using Duplicati from Docker This page describes common scenarios for configuring Duplicati with Docker The Duplicati Docker images are available from [DockerHub](https://hub.docker.com/r/duplicati/duplicati) and are released as part of the regular releases. The Docker images provided by Duplicati are quite minimal and includes only the binaries required to run Duplicati. There are also variations of the [Duplicati images provided by third parties](https://hub.docker.com/search?q=duplicati), including the popular [linuxserver/duplicati](https://hub.docker.com/r/linuxserver/duplicati) variant. ## Configure the image The Duplicati Docker images are using `/data` inside the container to store configurations and any files that should persist between container restarts. Note that other images may choose a different location to store data, so be sure to follow the instructions if using a different image. You also need a way to sign in to the server after it has started. You can either watch the log output, which will emit a special signin url with a token that expires a few minutes after the server has started, or provide the password from within the configuration file. To ensure that any secrets configured within the application are not stored in plain text, it is also important to set up the database encryption key. See also the DockerHub page for details on how to configure the image: ### Data folder permissions Starting with version **2.3.1.0**, Duplicati requires that the data folder has the exact expected permissions, or it will refuse to use it. Some Docker setups may not be able to set the permissions due to the mount setup and will need to opt out of the permission check by setting the environment variable `DUPLICATI__ALLOW_INSECURE_DATAFOLDER=true` in the image: ```yaml environment: DUPLICATI__ALLOW_INSECURE_DATAFOLDER: "true" ``` See the [server database documentation](/database-and-storage/the-server-database#limited-access-to-the-database-folder) for more details on the data folder permission check. ## Hostname access Duplicati's server allows access from IP-based requests, but disallows access from requests that use a hostname. This is done to prevent certain DNS-based attacks, but will also block attempts to use a correct hostname. To avoid this, set the environment variable: ```yaml environment: DUPLICATI__WEBSERVICE_ALLOWED_HOSTNAMES: ; ``` Setting this environment variable will enable using desired hostnames instead of IP addresses only. The special hostname `*` will disable the protection and allow any hostname, but this is not recommended for security reasons. ## Managing secrets in Docker Ideally, you need at least the settings encryption key provided to the container, but perhaps also the webservice password. You can easily provide this via a regular environment variable: ```yaml services: myapp: image: duplicati/duplicati:latest volumes: - ./data:/data environment: SETTINGS_ENCRYPTION_KEY: "" DUPLICATI__WEBSERVICE_PASSWORD: "" ``` But you can make it a bit more secure by using [Docker secrets](https://docs.docker.com/compose/how-tos/use-secrets/) which are abstracted as files that are mounted under `/run/secrets/`. Since Duplicati does not support reading files in place of the environment variables, you can either use a [preload configuration file](/configuration-and-management/preload-settings) or use one of [the secret providers](/security-and-secrets/using-the-secret-provider). ### Using a preload file To use the preload approach, prepare a `preload.json` file with your encryption key: ```json { "env": { "server": { "SETTINGS_ENCRYPTION_KEY": "", "DUPLICATI__WEBSERVICE_PASSWORD": "" } } } ``` You can then configure this in the compose file: ```yml services: myapp: image: duplicati/duplicati:latest volumes: - ./data:/data environment: DUPLICATI_PRELOAD_SETTINGS: /run/secrets/preloadsettings secrets: - preloadsettings secrets: preloadsettings: file: ./preload.json ``` ### Using a secret manager Setting up the secret manager is a bit more work, but it has the benefit of being able to configure multiple secrets in a single place. To configure the file-based secret provider, you need to create a `secrets.json` file such as this: ```json { "settings-key": "", "ui-password": "" } ``` Then set it up in the compose file: ```yml services: myapp: image: duplicati/duplicati:latest volumes: - ./data:/data environment: SETTINGS_ENCRYPTION_KEY: "$$settings-key" DUPLICATI__SECRET_PROVIDER: file-secret:///run/secrets/secretprovider DUPLICATI__WEBSERVICE_PASSWORD: "$$ui-password" secrets: - secretprovider secrets: secretprovider: file: ./secrets.json ``` It is also possible to use one of the other secret providers, such as one that fetches secrets from a secure key vault. In this case, you do not need the `secrets.json` file, but can just configure the provider. ## Read locked files Duplicati has support for LVM-based snapshots which is the recommended way for getting a consistent point-in-time copy of the disk. For some uses, it is not possible to configure LVM snapshots, and this can cause problems due to some files being locked. By default, Duplicati will respect the advisory file locking and fail to open locked files, as the lock is usually an indication that the files are in use, and reading it may not result in a meaningful copy. If you prefer to make a best-effort backup, which was the default in Duplicati v2.0.8.1 and older, you can disable advisory file locking for individual jobs with the advanced option: `--ignore-advisory-locking=true`. You can also disable file locking support entirely in Duplicati: ```yaml services: myapp: image: duplicati/duplicati:latest volumes: - ./data:/data environment: SETTINGS_ENCRYPTION_KEY: "" DUPLICATI__WEBSERVICE_PASSWORD: "" DOTNET_SYSTEM_IO_DISABLEFILELOCKING: true ``` ## Running behind a proxy If you want to run Duplicati behind an nginx proxy, you can use a `docker-compose` configuration like this example ```yaml services: nginx: image: nginx:alpine ports: - "8200:8200" volumes: - ./nginx.conf:/etc/nginx/nginx.conf:ro depends_on: - duplicati restart: unless-stopped duplicati: image: duplicati/duplicati:latest environment: - DUPLICATI__WEBSERVICE_PASSWORD: "" volumes: - ./data:/data restart: unless-stopped ``` And then use an `nginx.conf` file like this example: ```nginx events { worker_connections 1024; } http { server { listen 8200; server_name localhost; location / { proxy_pass http://duplicati:8200; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } } ``` ### Pre-authenticated with reverse proxy If your proxy setup already authenticates the user and you prefer not having to use another password to access Duplicati, you can configure the proxy to forward a preconfigured authentication header. It is not possible to disable authentication for Duplicati, as that would make it possible to accidentially expose the server without access control. To avoid being asked for a password on each accss, you need to generate a random token that you can pass from the nginx server to Duplicati that serves as authentication and grants access to Duplicati. {% hint style="danger" %} This setup bypasses the Duplicati authentication so make sure your authentication system is sufficiently secure before deploying it. {% endhint %} When you have a secure random token, make Duplicati trust it via the pre-authenticated header: ```yaml services: myapp: image: duplicati/duplicati:latest volumes: - ./data:/data environment: SETTINGS_ENCRYPTION_KEY: "" DUPLICATI__WEBSERVICE_PRE_AUTH_TOKENS: "" ``` Then make the nginx proxy forward the header on each request: ```nginx http { server { listen 8200; server_name localhost; location / { proxy_pass http://duplicati:8200; proxy_set_header PreAuth ; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } } ``` # Configuring HTTPS This page explains how to configure HTTPS for secure web UI access in Duplicati. {% hint style="info" %} **Available since:** Canary 2.2.0.106 {% endhint %} Duplicati can automatically generate HTTPS certificates for secure web UI access. This document explains the security model and provides platform-specific instructions for configuring HTTPS. ## Certificate Approach Browsers currently reject certificates that have a validity period for more than 90 days, with the logic that rotation should be frequent and automated to prevent long-term exposure to potential vulnerabilities. Since Duplicati uses localhost serving by default, there is no Certificate Authority (CA) to request certificates from. If Duplicati creates a self-signed certificate, it would need to be re-authorized after 90 days, which would require admin permissions and possibly manual intervention every 90 days. To avoid this, Duplicati generates its own Certificate Authority (CA) and uses it to sign server certificates. The CA private key is stored in the Duplicati database file, which is encrypted with the database encryption key. When a new certificate is needed, Duplicati generates a new certificate signed by the CA. Since the CA is local and not shared with any external service, this approach provides a secure way to manage certificates without requiring external dependencies. As the CA is trusted by the user or system, this enables Duplicati to automatically renew certificates without user intervention, in a way that is trusted by the user's browser. {% hint style="warning" %} **Security Warning:** While the CA is local, it is still a CA and can be used to sign certificates for other domains. If someone gains access to the Duplicati database, they can use the CA to sign certificates for other domains, essentially providing an undetected man-in-the-middle attack. {% endhint %} If you prefer providing your own certificate, you can do so by setting the `server-ssl-certificate` and `server-ssl-certificatepassword` settings. This will not activate auto-renewal or generate a CA. ## Certificate Authority (CA) Security Model When HTTPS is configured, Duplicati creates a local Certificate Authority (CA) with the following security properties: ### Local-Only CA * The CA is generated locally on your machine and is not shared with any external service * The CA certificate is installed only in your system's local trust store * Other machines do not trust this CA unless explicitly configured to do so * The CA should never be exported or shared with other systems, unless you understand the security implications ### Certificate Validity Periods * **CA Certificate:** Valid for approximately 10 years * **Server Certificate:** Valid for 90 days * **Auto-renewal:** Server certificates are automatically renewed 30 days before expiration ## CA Key Storage Security The CA private key is stored with multiple layers of protection: 1. **Encryption:** The private key is encrypted using AES-256 with a password-derived key 2. **Password Separation:** The encryption password is stored separately from the encrypted key 3. **Database Encryption:** If database field encryption is enabled, an additional encryption layer is applied {% hint style="warning" %} **Important:** The security of your HTTPS certificates depends on the security of your Duplicati database file. Ensure that database encryption is enabled. {% endhint %} ## Platform-Specific Configuration ### Windows On Windows, the CA certificate is installed in the certificate store. By default, it uses the LocalMachine store when running as Administrator, or CurrentUser store when running as a regular user. A dialog is shown to confirm the installation location when installing without Administrator privileges. #### Generating Certificates on Windows Open Command Prompt or PowerShell as Administrator (for system-wide trust) or as a regular user (for user-only trust): ```cmd duplicati-configure https generate ``` To specify a particular certificate store: ```cmd duplicati-configure https generate --store=local ``` Valid store values are: * `local` or `machine` - Install in the LocalMachine store (requires Administrator privileges) * `user` or `currentuser` - Install in the CurrentUser store #### Trust Store Location The CA certificate is installed in: * **LocalMachine:** `Cert:\LocalMachine\Root` (system-wide trust) * **CurrentUser:** `Cert:\CurrentUser\Root` (user-only trust) ### Linux On Linux, the CA certificate is installed in distribution-specific certificate directories. The ConfigureTool will automatically detect the correct location for your distribution. #### Generating Certificates on Linux Run the ConfigureTool with appropriate privileges (typically requires sudo for system-wide trust): ```bash sudo duplicati-configure https generate ``` To specify a custom certificate directory: ```bash sudo duplicati-configure https generate --cert-dir=/usr/local/share/ca-certificates/ ``` #### Common Certificate Directories Different Linux distributions use different paths for CA certificates: * Debian/Ubuntu: `/usr/local/share/ca-certificates/` * RHEL/CentOS/Fedora: `/etc/pki/ca-trust/source/anchors/` * SUSE: `/etc/pki/trust/anchors/` * Alpine: `/usr/local/share/ca-certificates/` The ConfigureTool will automatically detect the correct location for your distribution. #### Chrome and Firefox on Linux Chrome and Firefox on Linux maintain their own certificate stores and may not automatically trust the system CA. This is especially true for sandboxed installations (Snap, Flatpak). **Exporting the CA Certificate** First, export the CA certificate for manual import: ```bash duplicati-configure https export-ca --file ~/duplicati-ca.crt ``` **Importing to Firefox** 1. Open Firefox and go to **Settings** → **Privacy & Security** 2. Scroll to **Certificates** and click **View Certificates** 3. Select the **Authorities** tab 4. Click **Import** and select the exported `duplicati-ca.crt` file 5. Check **"Trust this CA to identify websites"** and click OK **Importing to Chrome** 1. Open Chrome and go to **Settings** → **Privacy and security** → **Security** 2. Click **Manage certificates** 3. Select the **Authorities** tab 4. Click **Import** and select the exported `duplicati-ca.crt` file 5. Check **"Trust this certificate for identifying websites"** and click OK {% hint style="info" %} For Flatpak or Snap installations, browsers run in a sandbox that may restrict file access. If you encounter "error reading file" during import, copy the certificate to a location accessible by the sandbox: ```bash cp ~/duplicati-ca.crt /tmp/duplicati-ca.crt chmod 644 /tmp/duplicati-ca.crt # Then import from /tmp in the browser ``` {% endhint %} **Avoiding System Store Trust** Since Flatpak or Snap browsers don't use the system certificate store anyway, you can avoid installing the CA in the system trust store by using the `--no-trust` flag: ```bash duplicati-configure https generate --no-trust ``` This generates a CA certificate without requiring elevated privileges. You can then export and manually import it into your browser. ### macOS On macOS, the CA certificate is installed in the System keychain by default, which requires administrator privileges. #### Generating Certificates on macOS Run the ConfigureTool with administrator privileges: ```bash sudo duplicati-configure https generate ``` To specify a custom keychain: ```bash duplicati-configure https generate --keychain=/Library/Keychains/System.keychain ``` #### Trust Store Location By default, the CA certificate is installed in: * System keychain: `/Library/Keychains/System.keychain` (requires administrator privileges) The ConfigureTool uses the `security add-trusted-cert` command to add the certificate to the trust store. ## Revocation and Compromise Response If you suspect your CA private key has been compromised: 1. **Immediate Action:** Remove the certificates using: ```bash duplicati-configure https remove ``` 2. **Regenerate:** Create a new CA with: ```bash duplicati-configure https regenerate-ca ``` 3. **Review:** Check recent server logs and backup history for unauthorized access 4. **Monitor:** Set up alerts for unexpected certificate changes ## Certificate Pinning Considerations For high-security environments, consider implementing certificate pinning: * Extract the CA certificate thumbprint after generation * Configure clients or monitoring systems to expect only this specific CA * This prevents acceptance of certificates signed by other CAs that might be installed on the system ## Viewing Certificate Status To check the current status of your HTTPS certificates: ```bash duplicati-configure https show ``` This displays: * CA certificate details (subject, issuer, validity dates) * Whether the CA is installed in the trust store * Server certificate details (subject, issuer, validity dates, DNS names, IP addresses) * Database encryption status * Certificate expiration status ## Renewing Certificates Server certificates are automatically renewed 30 days before expiration. To manually renew the server certificate: ```bash duplicati-configure https renew ``` ## Removing HTTPS Configuration To remove HTTPS certificates and stop using HTTPS: ```bash duplicati-configure https remove ``` This will: * Remove the CA certificate from the system trust store * Delete all certificate data from the Duplicati database ## Using Custom Certificates If you prefer to use your own certificates instead of the auto-generated CA: 1. Obtain a certificate and private key from a trusted CA (or create your own) 2. Configure Duplicati Server with the `server-ssl-certificate` and `server-ssl-certificatepassword` options 3. Note that auto-renewal will not be available when using custom certificates See the [Server documentation](https://github.com/duplicati/documentation/blob/main/detailed-descriptions/duplicati-programs/server.md) for details on configuring custom SSL certificates. ## See Also * [ConfigureTool Reference](/duplicati-programs/command-line-interface-cli-1/configuretool) - Complete command reference * [Duplicati Access Password](/configuration-and-management/duplicati-access-password) - Managing server access * [The Server Database](/database-and-storage/the-server-database) - Database configuration # Import and export backup configurations This page describes how to import and export configurations from Duplicati While it is not required that you keep a copy of the backup configuration, it can sometimes be convenient to have all settings related to a backup stored in a single file. ## Export To export from within the user interface, expand the backup configuration and click "Export ..." On this page you should select "To File", which is default. The option to export "As commandline..." is not covered here, but allows you to get a string that can be used with the [Duplicati CLI executable](/duplicati-programs/command-line-interface-cli). You then need to decide on how to handle secrets stored in the configuration. Since these secrets include both the credentials to connect to the remote destination as well as the encryption passphrase, it is important that the exported file is protected. You can choose to not include any secrets by unchecking the "Export passwords" option. The resulting file will then not contain the secrets and you need to store them in a different place (credential vault, keychain, etc). You can also choose to encrypt the file before exporting it. If you choose this option, make sure you choose a strong unique passphrase, and store that passphrase in a safe location. After completing the export, you will get a file containing the backup configuration. The file is in JSON format and optionally encrypted with [AESCrypt](/using-tools/encrypting-and-decrypting-files). ## Import configuration With an exported configuration, you can delete an existing configuration and re-create it by importing the configuration. You can optionally edit the parameters so the re-created backup configuration differs from the original. To import a configuration, go to the "Add backup" page and choose "Import from file": Pick the file or drag-n-drop it on the file chooser. If the file is encrypted, provide the file encryption passphrase here as well. The option to "Import metadata" will create the new backup configuration and restore the statistics, including backup size, number of versions, etc. from the data in the file. If not checked, these will not be filled, and will be updated when the first backup is executed. If the option "Save immediately" is checked, the backup will be created when clicking import, skipping the option to edit the backup configuration. When all is configured as desired, click the "Import" button. If you have not checked "Save immediately", the flow will look like it does when [setting up the initial backup](/getting-started/set-up-a-backup-in-the-ui). If the backup was created with the [store-task-config](#storing-configuration-with-the-backup) option enabled, you can also retrieve the configuration directly from the backup destination when restoring or setting up a new machine. ## Storing configuration with the backup Duplicati can automatically store backup configurations together with the backup data. This makes it easier to recreate a backup job when restoring on a new machine or after losing the local Duplicati database. {% hint style="info" %} Avaliable from Canary 2.3.0.104 {% endhint %} The behavior is controlled by the `--store-task-config` advanced option, which supports the following modes: | Mode | Description | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | `Auto` (default) | For encrypted backups, behaves as `Self`. For unencrypted backups, behaves as `None`. | | `None` | No configuration is stored with the backup. | | `Self` | Stores the current backup configuration with all secrets unconditionally removed. | | `All` | Stores the configurations of **all** backups defined in Duplicati, with all secrets unconditionally removed. | | `SelfWithSecrets` | Stores the current backup configuration with all secrets included. If the backup is not encrypted, Duplicati logs a warning and reverts to `Self`. | | `AllWithSecrets` | Stores the configurations of **all** backups with all secrets included. If the backup is not encrypted, Duplicati logs a warning and reverts to `All`. | | `SelfWithUnencryptedSecrets` | Stores the current backup configuration with all secrets included, even if the backup is not encrypted. | | `AllWithUnencryptedSecrets` | Stores the configurations of **all** backups with all secrets included, even if the backup is not encrypted. | When a configuration is stored with the backup, it is saved as a control file named `task-setup.json` and uploaded alongside the backup data. You can later retrieve and import these stored configurations through the Duplicati user interface when connecting to the backup destination. For backwards compatibility, the option also accepts boolean values: `true` is treated as `Self`, and `false` is treated as `None`. ### Security considerations Storing backup configurations in the remote backup introduces security risks that should be carefully evaluated before changing the default settings. **Leaked secrets:** Backup configurations can contain sensitive data such as destination credentials, encryption passphrases, API keys, SMTP credentials for notifications, and paths to remote sources. If these secrets are included in a stored configuration, anyone who gains access to the backup data can extract them. Even if secrets are removed, the configuration may still reveal server names, paths, and other infrastructure details that aid an attacker. **Cross-contamination between backups:** If you use the `All`, `AllWithSecrets`, or `AllWithUnencryptedSecrets` mode, a single compromised backup destination can expose the complete set of backup configurations. An attacker who obtains access to one backup can extract the credentials for every other backup destination, notification service, and remote source configured in Duplicati. This cross-contamination means that one leak can cascade into a compromise of multiple systems. **Additional targets and remote sources:** A backup configuration can reference additional storage targets, secondary destinations, SMTP servers for notifications, and remote sources. When secrets are included in the stored configuration—such as with `SelfWithSecrets` or any of the unencrypted-secrets modes—a single leaked backup can provide an attacker with credentials to access these connected systems. This means compromising one backup destination could allow an attacker to disable secondary targets, read notification credentials, or access remote source locations. **Default protection:** The default `Auto` mode is designed to minimize these risks. When the backup is encrypted, it stores only the current job's configuration and unconditionally strips all secrets (`Self`). When the backup is unencrypted, it stores nothing (`None`). This default ensures that a leaked backup does not contain credentials that can be used to access other systems. If you choose to override the defaults—especially by using `All`, `AllWithSecrets`, `AllWithUnencryptedSecrets`, `SelfWithUnencryptedSecrets`, or any mode that includes secrets in an unencrypted backup, ensure that the backup destination is strongly protected and that you fully understand the risk of cross-contamination across all your backups, sources, and notification services. # Migrating Duplicati to a new machine This page describes how to best migrate a Duplicati instance to a new machine If you have moved to a new machine and want to restore files to the new machine, you can follow the steps outlined in [Restoring files](/getting-started/restoring-files). If instead, you have already moved files to the new machine and would like to set up the new machine to continue backups made on the previous machine, there are a few ways to do this. **Note:** it is possible to restore files across operating systems, but due to path differences it is not possible to continue a backup made on Windows on a Linux/MacOS based operating system and vice versa. **Note:** do not attempt to run backups from two different machines to the same destination. Before migrating, make sure the previous machine is no longer running backups automatically. If both machines run backups, one instance will detect that the remote destination has been modified and will refuse to continue until the local database has been rebuilt. If you have access to backup configurations, jump to the [section for moving with backup configurations](#backup-configurations-are-available). And if you have no configurations, jump to the [manual setup section](#previous-machine-and-configurations-are-unavailable). ## Previous machine is still available If the previous machine is still accessible, you can copy over the contents of the `Duplicati` folder containing the the configuration database `Duplicati-server.sqlite` and the other support database. This approach is by far the fastest as Duplicati has all the information and does not need to check up with the remote storage. {% hint style="info" %} From version 2.3 the default is to encrypt the configuration database, and you need to decrypt it before copying. If you have disabled database encryption, you can skip the decryption step. {% endhint %} Before making a copy you need to decrypt the settings database so it can be read on the new machine. The new machine will encrypt it when Duplicati starts, using a new unique key for that machine. To decrypt the database, follow these steps on the machine you are moving from: 1. Stop Duplicati 2. Start Duplicati with the commandline option `--disable-db-encryption=true` 3. Stop Duplicati again Make sure Duplicati is stopped before moving in the folder into the same location on the new machine. After moving in the folder, you can start Duplicati on the new machine and everything will be working as before. If it has been a while since the previous instance was running, this may trigger the scheduled backups on startup. Use the option `--startup-delay=5min` to start Duplicati in pause mode for 5 minutes if you want to check up before it starts running. ## Backup configurations are available If you don't have the backup configurations, see the section on [import/export configuration](/configuration-and-management/import-and-export-backup-configurations) for a guide on how to create the backup jobs from the configuration files. With the backup configurations, it is possible to re-create the backup configurations. The flow allows you to modify set setup before saving the configuration, in case some details have changed. Once the backup is re-created, it is required that you run the repair operation to make Duplicati recreate the [local database](/database-and-storage/the-local-database) for the backup. Once the local database has been recreated, it is then possible to run the backup as before with no modifications required. ## Previous machine and configurations are unavailable If you do not have access to the previous setup, you can still continue the backups, but this requires that you re-create the backups manually. You need at least the storage destination details, the passphrase and to select the sources. Once the backup configuration has been created it works the same as if you had imported it from a file. Before running a backup, you need to run the repair operation to make Duplicati recreate the [local database](https://github.com/duplicati/documentation/blob/main/detailed-descriptions/configuration-and-management/the-local-database.md) for the backup. Once the local database has been recreated, it is then possible to run the backup as before with no modifications required. # Preload settings This page describes how Preload settings are applied The preload settings allow configuring machine-wide or enterprise-wide default settings with a single file. Because of this usecase, all settings are applied only if they are not already present. This means a commandline argument could be set up to change the default blocksize, but if the user has applied another setting via the commandline or parameters-file, the preload setting has no effect. For single-machine users, the preload settings are a convenient way to change the arguments passed to either [TrayIcon](/duplicati-programs/trayicon), [Server](/duplicati-programs/server), or [Agent](/duplicati-programs/agent), without needing to edit shortcuts or service files. To support different ways of deploying the settings file, 3 locations are checked: * Inside the installation folder * The file pointed to by `DUPLICATI_PRELOAD_SETTINGS` * Inside the data folder For security reasons, all these paths are expected to be writeable only by Administrator/root so unprivileged users cannot modify the values. If the settings contains secrets, make sure that only the relevant users can read them. {% hint style="warning" %} Starting with version **2.3.1.0**, a `preload.json` placed inside the data folder is only loaded if the folder passes the [data folder permission check](/database-and-storage/the-server-database#limited-access-to-the-database-folder), or if one of the opt-out methods for insecure permissions is activated. If the folder permissions do not match the expected values and the check is not bypassed, the `preload.json` file inside the data folder is ignored. Additionally, the previously trusted paths `/usr/local/share/Duplicati/preload.json` (macOS) and `C:\ProgramData\Duplicati\preload.json` (Windows) are no longer supported, as they cannot be guaranteed to be locked down. {% endhint %} The loading of the files is default silent, even if the parsing fails, but the environment variable `DUPLICATI_PRELOAD_SETTINGS_DEBUG=1` will toggle loader debug information to help investigate issues. The implementation here follows the format: ```json { "env": { "*": { "TEMP": "/mnt/tmp", "LOGGING": "false" }, "tray": { "LOG": "1" }, "server": { "DUPLICATI__WEBSERVICE_ALLOWED_HOSTNAMES": "m1" } }, "db": { "server": { "--compression-module": "zip", "--send-http-result-output-format": "Json" } }, "args": { "tray": [ "--hosturl=http://m1:8299" ], "server": [ "--webservice-port=8299" ] } } ``` The file has 3 sections that are all similar and all optional: `env`, `db`, and `args`. Each section can apply to all executables (`*`) or a specific executable. The executable names can be seen in the source, but the most common ones are `tray` and `server`. In the case where the `*` section and specific executable has the same variable, the specific one is used. If multiple settings files are found, they are loaded in the order described above. Here the last file loaded will be able to overwrite the others. The `*` settings are collected from all three files, as are the executable specific options, and only after all parsing is done, are the specific executable options applied (see below for an example). Note that some executables will load others, such that TrayIcon, Service, and WindowsServer will load Server. ## Environment variables - `env` The `env` section contains environment variables that are applied inside the process, after starting. Each entry under an executable is a key-value pair, where the key is the name of the environment variable, and the value will be the contents of the environment variable. The environment variables are only set if they are not already set, allowing a custom base set, but prefers local machine variables. In the case where one binary loads another, the starting application environment variables are applied first, and then any unset environment variables are applied for the loaded executable. ## Database settings - `db` For the `db` section it is possible to use `*` but the settings are currently only applied when running the server, so for future compatibility this section should use `server` only. The settings under an executable in the `db` section are automatically prefixed with `--` to ensure they are valid options and are saved as the "application wide" settings, also visible in the UI under Settings -> Advanced Options. The settings here are applied to the database if they are changed, meaning a change to the settings will overwrite settings the user has already applied. This check is performed on startup. The database settings are not passed on from a binary when it loads another, so the only database settings that are loaded are done by Server, even if any are supplied by `tray` (may change in the future). ## Commandline arguments The commandline arguments supports both the `*` and specific executable name. The arguments are expected to be switches in the format `--name=value` but can be any commandline argument. The general logic in Duplicati is that "last option wins", so the resolver logic for that is applied to try to get the most logical combination of arguments. ## Resolution with conflicts If the following fragment is supplied: ```json "env": { "*": { "E1": "a", "E2": "b" }, "tray": { "E1": "c", "E3": "d" } } ``` The Server executable will get the settings from `*` and the TrayIcon will get the values: "`E1=c E2=b E3=d`". If the above fragment is found in the first file, but this fragment is found in a later file: ```json "env": { "*": { "E3": "f" }, "tray": { "E1": "g" } } ``` First the `*` variables are collected, giving "`E1=a E2=b E3=f`", then the `tray` variables give "`E1=g E3=d`", and then they are combined to give "`E1=g E2=b E3=d`" for `tray`. The same combination logic is applied for both the `db` and `args` sections, but since the `args` section are not key-value pairs, and their order matter, it is done by collecting the arguments first, and then reducing them: ```json "args": { "*": ["--test=1", "--abc=123"], "server": ["--xyz=z", "--test=1", "--test=2"] } ``` In this case the arguments are collected, with `*` first, then the executable specifics, giving: ```json ["--test=1", "--abc=123", "--xyz=z", "--test=1", "--test=2"] ``` Since this contains 3 options named `--test`, they are reduced and appended so it ends up with: ```json ["--abc=123", "--xyz=z", "--test=2"] ``` The intention here is to stay as close as possible to the original line that was entered. If the commandline arguments already contains `--test`, the values are not applied. # Retention settings This page describes the different retention settings available in Duplicati Even though Duplicati tries hard to reduce storage use as much as possible, it is inevitable that the remotely stored data grows as new versions of files are added. To avoid running out of space or paying for excessive storage use, it is important that unnecessary backups are removed regularly. In Duplicati there are a few different settings that can be used to configure when a "snapshot" is removed. All of these options are invoked automatically at the end of a backup to ensure that removal follows a new version. If you use the [Command Line Interface](/duplicati-programs/command-line-interface-cli), it is possible to disable the removal and run the delete command as a separate step. After deleting one or more versions, Duplicati will mark any data that can no longer be referenced as waste, and may occasionally choose to run a compact process that deletes unused volumes and creates new volumes with no wasted space. Despite all deletion rules, Duplicati will never delete the last version, keeping at least one version available. ## Delete older than The most intuitive option is to choose a period that data is stored, and then to consider everything older than this period as stale data. The actual period depends on the actual use, but it could be 7 days, 1 year or 5 years for example. This option is usually the prefered choice if the backups happen regularly, such as a backup each day, and then keep the last 3 months. ## Keep versions If the backups are running irregularly, where the backups are triggered by some external event, there may be long periods where there are no backups. For this case you can choose a number of versions to keep and Duplicati will consider anything outside that count as outdated. Another special case is that if the source data has not changed at all, which is uncommon, Duplicati will not make a new version, as it would be identical to the previous version. In such a setup, it may be preferable to use a version count, despite regularly scheduled backups. ## Retention policy The retention policy is a "bucket" based strategy, where you define how many backups to keep in each "bucket" and what a "bucket" covers. With this strategy, it is possible to get something similar to [grandfather-father-son](https://en.wikipedia.org/wiki/Backup_rotation_scheme#Grandfather-father-son) style backup rotations. The syntax for the rentention policy uses the [time format](/technical-details/option-formats#timespans-timestamps-and-durations) to define the bucket and contents in that bucket. The bucket size is first, then a colon separator, and then the duration in the bucket. Multiple buckets can be defined with commas. As an example: ``` 7D:U,1Y:1W ``` The first bucket is defined as being 7 days, and the value `U` means unlimited the number of backups in this bucket. In other words: for the most recent 7 days, keep all backups. The second bucket is defined as 1 year, keeping a backup for each 1 week, resulting in rougly 52 backups after the first 7 days. Any backups outside the buckets are deleted, meaning anything older than a year would be removed. In the UI, a helpful default is called "Smart retention" which sets the following retention policy: ``` 1W:1D,4W:1W,12M:1M ``` Translated, this policy means that: * For the most-recent week, store 1 backup each day * For the last 4 weeks, store 1 backup each week * For the last 12 months, store 1 backup each month # Duplicati Access Password This page describes how the authentication is working with Duplicati and how to regain access if the password is lost or unknown If you are starting Duplicati for the first time, it will ask you to pick a password. Picking a strong password is important to ensure unwanted access to Duplicati from other processes on the system. By default, Duplicati has chosen a strong random password and it is recommended for most users to not change the random password. It is not possible to extract the current password in any way and it is not possible to disable the password. ## Access from the TrayIcon The [TrayIcon](/duplicati-programs/trayicon) process will usually host the [Server](/duplicati-programs/server) that presents the UI. Since the two parts are within the same process they can communicate securely, and this setup enables the TrayIcon to negotiate a short-term signin token with the server, even though it does not know the password. This mechanism works for most default installations and is secure as long as the desktop is not compromised. This signin process is the reason that the default random password is prefered, because it is not possible to leak the password. The downside is that you can bookmark the Duplicati page, but you may be asked for a password that you do not know when accessing the page. In this case, re-launching from the TrayIcon will log you in again. If you prefer, it is possible to choose the password so you can enter it when asked. Optionally, you can also choose to disable the feature that allows the TrayIcon to sign in without a password, through the settings page. Login with the TrayIcon is shown here (click to choose operating system): {% tabs %} {% tab title="Windows" %}
{% endtab %} {% tab title="MacOS" %}
{% endtab %} {% tab title="Linux (Ubuntu)" %}
{% endtab %} {% endtabs %} ## Temporary signin token When Duplicati starts up with the randomly generated password it will attempt to emit a temporary sign-in url. If you run either the [TrayIcon](/duplicati-programs/trayicon) or [Server](/duplicati-programs/server) in a terminal, most systems will show the link here. If you are running Duplicati as a service with no console attached, the link will end up in the system logs. On Windows you can use the [Event Viewer](https://learn.microsoft.com/en-us/shows/inside/event-viewer) utility to find the message with a sign-in url. For Linux you can view the system logs, usually: ```bash sudo journalctl --unit=duplicati | less ``` Note that the regular output from `journalctl` is capped in width, so you cannot see the whole token. Pipe to a file or another program as shown above to get the full output. For MacOS you can use the [Console app](https://support.apple.com/lt-lt/guide/console/welcome/mac). Once you have obtained the link, simply click it or paste it into a browser. Note that the sign-in token has a short lifetime to prevent it being used to gain unathorized access from someone who obtains the logs. If the link has expired, simply restart the service or application and a new link will be generated. After a password has been set, the link will no longer be generated. ## Change password with ServerUtil If you are not using the TrayIcon or you have disabled the signin feature, but lost the password somehow, you can change the password with [ServerUtil](/duplicati-programs/command-line-interface-cli-1/serverutil) in some cases. This works by reading the same database as the server is using and extracting the keys used to sign a sign-in token, and then creating a sign-in token. This sign-in token works the same way as the TrayIcon's signin feature. Note that the password itself cannot be extracted from the database, it can only be verified. After obtaining a sign-in token, ServerUtil can then change the password in the running instance. This only works if: * The database is readable from the process running ServerUtil * The database field encryption password is available to the process running ServerUtil If these constraints are satisfied, it is possible to reset the server password by running: ```shell > duplicati-server-util change-password ``` If ServerUtil is launched in a similar environment (i.e., same user, same environment variables) this would allow access in most cases. There are a number of commandline options that can be used to guide ServerUtil in case the environments are not entirely the same. For Linux user, you can usually use `su` or `sudo`to enter the correct user context, but some additional environment variables may be needed. The default location for the database is described in the [data location section](https://github.com/duplicati/documentation/blob/main/detailed-descriptions/duplicati-programs/server.md#storing-data-in-different-places), and a different location can be provided with `--server-datafolder.` ### Example change with a different context If you need to change the password for a Windows Service instance running in the service context, you can use a command such as this: ``` Duplicati.CommandLine.ServerUtil change-password \ --server-datafolder "C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati" ``` Similarly, if the service is running as root on Linux: ``` duplicati-server-util change-password \ --server-datafolder=/root/.config/Duplicati ``` ## Change password from the Server If the other options are not available, it is possible to restart the [Server](/duplicati-programs/server) process and supply the commandline option: ``` --webservice-password= ``` This will write a hashed ([PBKDF](https://en.wikipedia.org/wiki/PBKDF2)) version of the new password to the database and use this going forward. This process requires restarting the server, but is persisted in the database, so it is only required to start the server once with with the `--webservice-password` option and future starts can be done without the password. Since commandline arguments and environment variables can be viewed through various system tools, it is recommended that the option is not set on every launch. A prefered way to set this would be to stop all running instances, start once with the new password from a commandline terminal, shut down, and then start again normally. The option can also be supplied to the [TrayIcon](/duplicati-programs/trayicon) and [Agent](/duplicati-programs/agent) processes, which will pass it on to their internal instance of the Server. ### Change the password for a service To change the password for a running service, you can grab the [temporary signin token from the logs as explained above](#temporary-signin-token). If that approach is not possible, and the [ServerUtil approach](#change-password-with-serverutil) is not an option either, you can also instruct the service to change the password. This works by passing the `--webservice-password` command to the service, instructing it to reset the password. To avoid leaking the actual password, it is recommended that you choose a temporary password and run the following steps (example for Windows): ``` Duplicati.WindowsService.exe UNINSTALL Duplicati.WindowsService.exe INSTALL --webservice-password= Duplicati.WindowsService.exe UNINSTALL Duplicati.WindowsService.exe INSTALL ``` After this sequence, you can now log in to Duplicati using the temporary password, and change it to a real password. It is recommended that you use a temporary password because there is a chance that the password given to `INSTALL` will be written to a log. By using a temporary password you ensure that the real password is never persisted. Since the `UNINSTALL` and `INSTALL` steps will automatically stop/start the service, this sequence will make the service start once with the `--webservice-password=` option, and then restart without the option. If you only run the first two steps, the service will continue to reset the password on each startup, and your password will be present in the service commandline and possibly also captured in logs. ## Disable sign-in tokens It is possible to disable the use of sign-in tokens completely, which can increase security further. This is done by passing the option: ```sh --webservice-disable-signin-tokens=true ``` This will make the [Server](/duplicati-programs/server) reject any sign-in tokens and prevent the access from the TrayIcon and ServerUtil without explicitly passing the password. With this option, it will require write access to the database to create a new token, but it will also require handling the password in a safe manner from all instances where this is needed. This option can also be supplied to the [TrayIcon](/duplicati-programs/trayicon) process and is default enabled by the [Agent](/duplicati-programs/agent). # Running a self-hosted OAuth Server This page describes how to set up and run a self-hosted OAuth Server If you are using one of the backends that requires login via OAuth (Google, Dropbox, OneDrive, etc) you will need to obtain a "clientId" and a "clientSecret". These are given by the service providers when you are logged in, and are usually free. If you prefer to avoid the hassle of setting this up, you can opt to use the Duplicati provided OAuth server, where Duplicati's team will handle the configuration. This OAuth server is the default way to authenticate. If you prefer to be more in control of the full infrastructure, you can use this guide to set up and use your own self-hosted OAuth Server. For example, this guide will show how to set up an OAuth server for internal use in an organization, granting Duplicati instances full access to the Google Drive files. If you need to set up another provider than Google, see [the configuration defaults that has links to the pages where the Client ID and Client secret can be found for other services](https://github.com/duplicati/oauth-server/blob/main/ConfigurationLoader.Defaults.cs). ## Getting access to Google Cloud Services The first step is to [sign up for Google Cloud Services](https://console.cloud.google.com/) if you are not already a customer. Once you are signed up, you can create a new project as shown here: Once you have create a project where the OAuth settings can live in, you need to enable the "**Google Drive API**". Go to the top-left menu, choose "**API & Services**" and then "**Enabled APIs & Services**". From here search for "**Google Drive API**", click it and enable: Before you can get the values you need to configure the consent screen that is shown when users log in with your OAuth Service. You can choose "**Internal**" here, unless you need to provide access to people outside your organization. Choosing "External" also requires a Google review. On the consent screen, you only need to fill in the required fields, the app name and some contact information: The last step in the consent is choosing the scopes (meaning the permissions) it is possible to grant with this setup. In this example we choose the `auth/drive` scope, granting full access to all files in the users Drive. For regular uses, it is safest to use `auth/drive.file` which will only grant Duplicati access to files created by Duplicati. However, in some cases Google Drive will drop your permissions and refuse to let Duplicati access the files. There is no way to change the permissions on the files, so if this happens, your only choice is to use `auth/drive` and obtain full access: You can now click update and save the consent screen and proceed to setting up the credentials needed. Click "**Create Credentials**" and choose "**OAuth client ID**". On the next page, choose the type "**Web application**". In the "**Authorized redirect URIs**" field you need to enter the url for the server that is being called after login. The Duplicati OAuth server uses a path of `/logged-in` so make sure it ends with that. In the screenshot, the server is hosted on a single machine, so the setup is for `https://localhost:8080/logged-in`: When you are done, click "Save" and a popup will show the credentials that are generated. Use the convenient copy buttons to get "**Client ID**" and "**Client secret**" or download the JSON file containing them. If you loose them, you can get then again via the "Credentials" page. The credentials shown here are redacted: ## Setting up the configuration With the credentials available, create a JSON text file similar to this: ```json { "GD_CLIENT_ID": "", "GD_CLIENT_SECRET": "" } ``` If you are setting up a secure server, you should use [SharpAESCrypt](/duplicati-programs/command-line-interface-cli-1/sharpaescrypt) to encrypt the file after you have created it. If you do, make a note of the passphrase used. Save the file either as `secrets.json` or `secrets.json.aes` if you have encrypted it. In the following, we will only set up Full Access Google Drive, which for legacy reasons is called "`googledocs`" in the OAuth server. If you are looking to set up one of the other services, see [the configuration document](https://github.com/duplicati/oauth-server/blob/main/config.json), and pick the ids you need. In the following, the `services` are configured to just `googledocs` but it can be a comma separated list of services if you want to enable multiple. The storage is here simply a local folder that stores encrypted tokens, but you can also use an S3 compatible storage if needed. See the [OAuth server readme](https://github.com/duplicati/oauth-server/blob/main/README.md) for more details. ## Docker based setup If you are using Docker, you can run the [OAuth server image](https://hub.docker.com/r/duplicati/oauth-server) directly and simply add environment variables: ```yaml - ASPNETCORE_URLS: "http://localhost:8080" - HOSTNAME: "localhost:8080" - SECRETS: "/path/to/secrets.json.aes" - SECRETS_PASSPHRASE: "" - STORAGE: "file:///path/to/persisted/folder" - SERVICES: "googledocs" ``` The hostname here **MUST** match the one set as the redirect URI or the authorization will fail. The URLs parameter is what the internal Docker engine thinks it is running. For this setup there is no TLS/SSL certificate, so the URL here is `http` but note that we used `https` in the redirect URI and these two must match in the end. Here I am assuming some other service is providing the SSL layer. If you need to serve the certificate directly from the Docker container, generate a certificate `.pfx` file and use a configuration such as: ```yaml - ASPNETCORE_URLS: "https://localhost:8080" - HOSTNAME: "localhost:8080" - SECRETS: "/path/to/secrets.json.aes" - SECRETS_PASSPHRASE: "" - STORAGE: "file:///path/to/persisted/folder" - SERVICES: "googledocs" - ASPNETCORE_Kestrel__Certificates__Default__Path: "/path/to/certificate.pfx" - ASPNETCORE_Kestrel__Certificates__Default__Password: "" ``` ## Local machine setup To run without Docker, first you need to [download the OAuth Server binaries for your operating system](https://github.com/duplicati/oauth-server/releases) and extract them to a suitable place. The binaries are self-contained so the will run without any additional framework installation. To run the server, invoke it with a setup like this: ``` OAuthServer run --listen-urls=http://localhost:8080 --hostname=localhost:8080 --storage=file:///path/to/persisted/folder --secrets=/path/to/secrets.json.aes --secrets-passphrase= --services=googledocs ``` The hostname here **MUST** match the one set as the redirect URI or the authorization will fail. The URLs parameter is what the process thinks it is running locally. For this setup there is no TLS/SSL certificate, so the URL here is `http` but note that we used `https` in the redirect URI and these two must match in the end. Here I am assuming some proxy service is providing the SSL certificate. If you need to serve the certificate directly from the the binary, generate a certificate `.pfx` file and use a configuration such as: ``` OAuthServer run --listen-urls=https://localhost:8080 --hostname=localhost:8080 --storage=file:///path/to/persisted/folder --secrets=/path/to/secrets.json.aes --secrets-passphrase= --services=googledocs --certificate-path=/path/to/certificate.pfx --certificate-password= ``` ## Issuing an AuthID Once the service is running, you can navigate to the page and generate an AuthID: ## Using the self-hosted OAuth server in Duplicati The final step is to instruct Duplicati to use the self-hosted OAuth server instead of the regular instance. This is done by visiting the "Settings" page in the Duplicati UI and adding the advanced option `--oauth-url=https://localhost:8080/refresh`: Don't forget to click "OK" to save the settings. Once configured, the "AuthID" links in the UI will point to your self-hosted OAuth server, and all authorization is done purely through the self-hosted OAuth server. # Using the secret provider This page describes how to use the secret provider. The secret provider was introduced in Duplicati version 2.0.9.109 and aims to reduce the possibility of leaking passwords from Duplicati by not storing the passwords inside Duplicati. To start using a secret provider you need to set only a single option: ```sh --secret-provider= ``` This will make the secret provider available for the remainder of the application. You can then insert placeholder values where you want secrets to appear but without storing the actual secret in Duplicati. For commandline users, the secrets can appear in both the backend destination or in the options. As an example: ``` duplicati backup \ s3://example-bucket?auth-username=$s3-user&password=$s3-pass \ --passphrase=$passphrase ``` The secret provider will find the three keys prefixed with `$` and look them up with the secret provider. The provider will then be invoked to obtain the real values and the values will be replaced before running the operation. If the secret provider has these values: ``` s3-user=user s3-pass=pass passphrase=my-password ``` The example from above will then be updated internally, but without having the keys written on disk: ``` duplicati backup \ s3://example-bucket?auth-username=user&password=pass \ --passphrase=my-password ``` To ensure you never run with an empty string or a placeholder instead of the real value, all values requested needs to be in the storage provider, or the operation will fail with a message indicating which key was not found. ## Using secret provider in the user interface To set up a secret provider for the [TrayIcon](/duplicati-programs/trayicon), [Agent](/duplicati-programs/agent) or [Server](/duplicati-programs/server) setups, you can add the `--secret-provider=url` to their respective commandlines, or you can add it via the user interface. Set it up in "Settings" -> "Advanced options" to have it apply to all backups, or via "Advanced options" on the individual backups.
Remember to click "Save settings" after configuring the provider. # Local providers This page describes the providers that operate locally on the machine they are running ## The Environment Variable provider The simplest provider is the `env://` provider, which simply extracts environment variables and replaces those. There is no configuration needed for this provider, and the syntax for adding it is simply: ``` --secret-provider=env:// ``` ## The File Secret provider The `file-secret://` provider supports reading secrets from a file containing a JSON encoded dictionary of key/value pairs. As an example, a file could look like: ```json { "key1": "value1", "passphrase": "my password" } ``` The file provider also supports files encrypted with [AESCrypt](/using-tools/encrypting-and-decrypting-files) and you supply the decryption key with the option `passphrase`. Suppose the file is encrypted with the key `mypassword` you can then configure the provider: ``` --secret-provider=file-secret:///home/user/secrets.json.aes?passphrase=my-password ``` To avoid passing the encryption key via a commandline, see [the section on how to inject the secret provider configuration via an environment variable](/security-and-secrets/using-the-secret-provider/advanced-configurations#how-to-avoid-passing-credentials-on-the-commandline). To encrypt the file, you can use the [AESCrypt tool](/duplicati-programs/command-line-interface-cli-1/sharpaescrypt) provided with Duplicati: ``` Duplicati.CommandLine.SharpAESCrypt.exe e my-password source.json destination.json.aes ``` ## Credential Manager (Windows) On Windows XP and later, the [Credential Manager](https://support.microsoft.com/en-us/windows/accessing-credential-manager-1b5c916a-6a16-889f-8581-fc16e8165ac0) can be used to securely store secrets. As the credentials are protected by the account login, there is no configuration needed, so the setup is simply: ``` --secret-provider=wincred:// ``` The Windows Credential Manager can be accessed from the Control Panel and looks like this:
The secrets created from Duplicati will show up as "Generic Credentials" but Duplicati can also read the Windows Credentials. You can click "Add a generic credential" to create a new secret that can be used by Duplicati. ## Using `libsecret` (Linux) The[ `libsecret` implementation](https://gnome.pages.gitlab.gnome.org/libsecret/) stores various credentials on Linux and integrates with various UI applications to let the user approve or reject attempts to read secrets. The `libsecret` provider supports a single optional setting, `collection`, which indicates what collection to read from. If not supplied the default collection is used. The default collection is usually `login` on Gnome-based systems and `kdewallet` on KDE-based systems, and will typically be unlocked when the user logs in. To use the `libsecret` provider, use this argument: ``` --secret-provider=libsecret:// ``` If you are using a system with a Gnome-based desktop, such as Ubuntu, you can use the [Seahorse](https://wiki.gnome.org/Apps/Seahorse) application to manage your passwords. On KDE-based desktops you can use [KeepSecret](https://apps.kde.org/keepsecret/) or [KWalletManager](https://apps.kde.org/kwalletmanager5/). On KDE, make sure you have enabled the "KDE Wallet Subsystem" and also checked the option to "Use KWallet for the Secret Service interface":
### Seahorse In Seahorse the passwords are shown in the wallet in a list and are not grouped:
You can add new secrets and they can be used by Duplicati. ### KeepSecret With KeepSecret the passwords created from Duplicati are grouped under "Duplicati Secrets":
You can add new secrets and they can be read by Duplicati even if they are not under the "Duplicati Secrets" group. ### KWalletManager In KWalletManager the secrets are shown in a tree-structure, and for Plasma 6 they show up under "Duplicati Secrets":
The UI does not always update when dealing with multiple secrets, so you may need to close and open the wallet for the display to show correctly. You can add secrets here and they can be used in Duplicati even if they are not added under "Duplicati Secrets". With Plasma 5, there is a compatibility layer that makes the secrets created by Duplicati show up under "Secret Service":
This is purely cosmetic and they will show up under "Duplicati Secrets" if you upgrade to Plasma 6 or later. ## Using the `pass` secret provider (Linux) The [`pass` command](https://www.passwordstore.org) is a project that implements a secure password storage solution on Linux system, backed by GPG. Duplicati can use `pass` as the secret provider: ``` --secret-provider=pass:// ``` If you want to use `pass`, make sure it is installed on the system. You also need a GPG key, and you can create one with: ``` gpg --full-generate-key ``` As part of the key generation process, you are asked to enter an email address that will later be used to identify the key. Once you have the GPG key you can initialize pass with: ``` pass init ``` ## Using the KeyChain (MacOS) For MacOS users the standard password storage is the [KeyChain Access](https://support.apple.com/lt-lt/guide/keychain-access/kyca1083/mac) program. The secrets stored here as application passwords can be used by Duplicati. The KeyChain can be enabled as a secret provider with: ``` --secret-provider=keychain:// ``` For more advanced uses the options `account` and `service` can be used to narrow down what secrets can be extracted. # Cloud providers This page lists the cloud providers supported as secret providers For cloud-based providers there is generally a need to pass some kind of credentials to access the storage as well as the possibility of a provider being unavailable for a shorter period. To address these two issues, see [the section on how to avoid passing credentials on the commandline](/security-and-secrets/using-the-secret-provider/advanced-configurations#how-to-avoid-passing-credentials-on-the-commandline) and [how to protect against outages](/security-and-secrets/using-the-secret-provider/advanced-configurations#how-to-protect-against-secret-provider-outages). Setting up and using either of the vaults described here is outside the scope of this document. ## HashiCorp Vault The implementation for [HashiCorp Vault](https://www.vaultproject.io) supports both the cloud-based offering as well as the self-hosted version as sources. To connect to the vault, provide the url as part of the configuration: ``` --secret-provider= hcv://localhost:8200?token=&secrets=app1,app2 ``` The url is converted to the url used to connect to the vault (e.g., `https://localhost:8200` in this example). The token is used to authenticate, and the secrets are the vaults that secrets are read from. In the cloud-based offering the "secrets" values shown here are referred to as Apps and in the CLI as "mount points". When more than one value is supplied, the vaults are tried in order and stops once all secrets are resolved. This means that if the same secret key is found in two vaults, the value will be used from the first vault examined. ### Other options for `hcv://` For development purposes, the url can use a `http` connection by setting `&connection-type=http` , but this should not be used in production. To connect using a credential pair instead of the token, the credentials can be provided with the values `client-id` and `client-secret` , but should be passed via the environment variables: ``` export HCP_CLIENT_ID= export HCP_CLIENT_SECRET= --secret-provider=hcv://localhost:8200?secrets=app1 ``` By default, the key lookup is done case-insensitive but can be toggled case-sensitive with the option `&case-sensitive=true`. ## Amazon Secret Manager The provider for [AWS Secret Manager](https://aws.amazon.com/secrets-manager/) supports the AWS hosted vault. The credentials for the vault are the regular Access Key Id and Access Key Secret. While these can be provided via the secret provider url as `access-id` and `secret-key`, they should be passed via the environment variables: ``` export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= --secret-provider=awssm://?region=us-east-1&secrets=vault1,vault2 ``` The `secrets` values name the vaults to use (called "Secret Name" in the AWS Console). When more than one value is supplied, the vaults are tried in order and stops once all secrets are resolved. This means that if the same secret key is found in two vaults, the value will be used from the first vault examined. Instead of suppling the region the entire service point url can also be provided via `&service-url=`. By default, the key lookup is done case-insensitive but can be toggled case-sensitive with the option `&case-sensitive=true`. If you use IAM to create an account for this, you can use the policy `SecretsManagerReadWrite`. {% hint style="warning" %} You need to create the "Secret Name" in AWS console before you can use it. The secrets in AWS should be key/value and can contain multiple values. Setting a value will always append it to the first secret provided. {% endhint %} ## Google Cloud Secret Manager The secret provider for [Google Cloud Secret Manager](https://cloud.google.com/security/products/secret-manager?hl=en) relies on the [Google Cloud SDK](https://cloud.google.com/sdk?hl=en) to handle the authentication. Follow [the steps to get the environment authenticated](https://cloud.google.com/sdk/gcloud/reference/auth/login) with Google. After the athentication is complete, the configuration is: ``` --secret-provider=gcsm://?project-id= ``` If you need to integrate with a different flow you can also [supply an access token](https://cloud.google.com/sdk/gcloud/reference/auth/print-access-token), but notice that the token may be short-lived and you cannot change the token after configuring the secret provider: ``` --secret-provider=gcsm://?project-id=&token= ``` ### Additional options for `gcsm://` By default, the secrets are accessed with the version set to `latest` but this can be changed with `&version=`. The communication protocol can be changed from gRPC to https with by adding `&api-type=Rest`. It is also possible to use either `service-account-file` or `service-account-json` to authenticate with a service account. ### Configure a service account for Google Cloud Secret Manager To configure a service account for Google Cloud Secret Manager, visit [IAM ](https://console.cloud.google.com/iam-admin/serviceaccounts)→[ Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts) → Create. Use the role `Secret Accessor` for read-only access, and add `Secret Version Adder` if the account should be able to update the secrets. Then create the key with: Service account → Keys → Add key → Create new key → JSON. This will generate a key that will be downloaded. You can either use the key-file with `service-account-file=`, or you can url-encode the contents and supply it to `service-account-json=`. ## Azure Key Vault With [Azure Key Vault](https://azure.microsoft.com/en-us/products/key-vault) as the provider there are several options for authenticating, where the most secure method is to use the [Azure CLI login](https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli) that handles all the details. Since this method is the default, the secret provider can be configured as: ``` --secret-provider=azkv://?keyvault-name=keyvault ``` Instead of supplying the name of the keyvault, the full vault url can be supplied with `&vault-uri=`. ### Manually authenticating Instead of relying on the autmated login handling, it is possible to authenticate with either a client credential, or a username/password pair. For authenticating with client credentials, use: ``` --secret-provider=azkv://?keyvault-name=keyvault &auth-type=ClientSecret &tenant-id= &client-id= &client-secret= ``` And for username/password, use: ``` --secret-provider=azkv://?keyvault-name=keyvault &auth-type=UsernamePassword &tenant-id= &client-id= &username= &password= ``` ### How to configure manual access to an Azure Key Vault There are several ways as shown above, and the recommended is to use the CI method, but here is a description of a manual way: creating a client secret. First, visit Portal → Microsoft Entra ID → App registrations → New registration. Enter a name for the application and create it. On the application screen, record the Application Id and Directory Id:
Then visit: App → Certificates & secrets → Client secrets → New client secret. Create a new client secret, and make sure you copy the secret as it will not be shown again:
Event with all the information ,you still need to grant permissions for the app/client to access the key vault: Key Vault → Access control (IAM) → Add role assignment. If you need read-only access, use the role `Key Vault Secrets User` and if you need read/write acces, use the role `Key Vault Secrets Officer`. On the member page, add the app/client. Now you have all the details to connect to the vault, add them as shown above, and make sure to url-encode the values, particularly the secret as it may contain characters that are not url-safe. You need to encode: `keyvault-name`, `tenant-id`, `client-id`, and `client-secret`. # Advanced configurations ## Sharing the secret provider If the secret provider is configured for the entry application (e.g., the [TrayIcon](/duplicati-programs/trayicon), [Server](/duplicati-programs/server) or [Agent](/duplicati-programs/agent)) it will naturally work for that application, but will also be shared within that process. For the [Agent](/duplicati-programs/agent), this means that setting the secret provider for the agent, will also let the server that it hosts use the same secret provider. When a backup or other operation is then executed by the server it will also have access to the same secret provider. This sharing simplifies the setup by only having a single secret provider configuration and then letting each of the other parts access secrets without further configuration. If needed, the secret providers can be specified for the individual backups, such that it is possible to opt-out of using the shared secret provider. ## How to avoid passing credentials on the commandline To make passing arguments to the application a bit harder to obtain, the value for `--secret-provider` is treated as an environment variable if: * It starts with `$` optionally with curly brackets `{}`: * `$secretprovider` * `${secretprovider}` * If it starts and ends with `%` : * `%secretprovider%` No expansion is done on environment variables, so the entire provider string is required to be set as an environment variable. ## How to protect against secret provider outages If you run an operation and the secret provider is unavailable when the secrets are requested, the operation will fail. For most uses the occurence of an outage is so rare that this situation is acceptable. However, for some uses it is important that the backups keep running, even in the face out outages. To handle this need, Duplicati supports an optional cache strategy: ``` --secret-provider-cache=None --secret-provider-cache=InMemory --secret-provider-cache=Persistent ``` Storing the secrets somewhere makes it more likely that it is eventually leaked. For that reason, the default is to use the cache setting `None` which turns off the caching fully and only relies on the provider. The `InMemory` setting is the least intrusive version as it only stores the secrets in the process memory. This option is most useful when using a [shared provider](#sharing-the-secret-provider) such that it stays in memory between runs. Finally, the `Persistent` option will write secrets to disk, so it can handle situations where the provider is unavailable during startup, or where a shared provider does not work. As the purpose of the secret provider is to prevent the secrets from being written to disk, the secrets are written to disk using a passhprase derived from the secret provider url. If the secret provider url does not contain a strong secret already, it is possible to add any parameter to the url to increase the strength of the key. If the secret provider url changes, it is no longer possible to retrieve the cached values, and the next run will fail if the provider is unavailable, but will otherwise write a new encrypted cache file to disk. # Filters in Duplicati This page describes how filters are evaluated inside Duplicati and how to construct them Duplicati uses the same setup for filters to select individual files. It is most prominent when choosing the sources, but can be applied in other places where individual files can be selected. ## Path representations Internally, Duplicati represents folders with a trailing path separator, which makes it easy to distinguish between the two types. This distinction is important when constructing filters, as Duplicati requires a full match, including the trailing path separator, before a match is considered. An example for Windows and Linux/MacOS: * Windows * Folders * `C:\Users\john\` * `X:\data\` * Files * `C:\Users\myfile` * `X:\data\file.bin` * Linux/MacOS * Folders * `/home/john/` * `/usr/share/` * Files * `/home/myfile` * `/usr/file.bin` For brewity, the remainder of this page will only use the Linux/MacOS format in examples, but the same can be applied to the Windows paths. ## Filter types Duplicati supports 4 different kinds of filters: paths, globbing, regex, and predefined groups. The simplest type of filter is the path. To use a path-type filter, simply provide the full path to the file or folder to target. ### Globbing expressions While it would be possible to maintain an ever growing list of paths in a filter, it can quickly become hard to manage. For cases where there is some similarity between multiple files or folder paths, it is possible to target multiple paths with a [file-globbing syntax](https://en.wikipedia.org/wiki/Glob_\(programming\)). The wildcard character `*` matches any length of characters (including zero) and the character `?` matches a exactly one character. Unlike other glob implementations, the path separator is also matched in Duplicati filters. An example of glob expressions: ``` /usr/share/IMG_????.jpeg /home/*/Download/ *.iso ``` The first expression matches files with the 4 `?` characters replaced by any character, and the second expression matches the `Download` folder for any user, and the third matches any files with the `.iso` extension. ### Regular expressions If the paths to match are more complicated than what can be expressed with globbing, it is also possible to use [regular expressions](https://en.wikipedia.org/wiki/Regular_expression), which are a common way of expressing a string pattern. Understanding regular expressions and applying them can be a challenging task, and will most often require some testing to ensure it is working as expected. Also note that since Duplicati is written in C#, it uses the [.NET variant of regular expressions](https://learn.microsoft.com/en-us/dotnet/standard/base-types/regular-expressions). Regular expressions are provided by wrapping the expressions with hard braces `[ ]`: ``` [/usr/share/IMG_\d{4}\.jpeg] [/home/[^/]+/Download/] [.*\.iso] ``` Note that for Windows, the path separators must be escaped with a backslash, `\` so each separator becomes a double backslash `\\` . ### Predefined filter groups Some files are commonly excluded on many systems, and to make it easier to exclude such files, Duplicati has a number of built in filter groups: * SystemFiles Files that are not real files, such as `/proc` or System Volume Information. * OperatingSystem Files that are provided by the operating system, such as `/bin` or `C:\Windows\` * CacheFiles Files that are part of application or operating system caches, such as the browser cache. * TemporaryFiles Files that are stored temporarily by applications as part of normal operations * Applications Binary applications, such as `/lib/` or `C:\Program files\` * DefaultExcludes All the above filters in one group To use a filter group, supply one or more names inside curly braces `{ }`, separated with commas. As an example: ``` {CacheFiles,TemporaryFiles} ``` ## Apply filters By default, Duplicati will recurse the source folders and include every file and folder found. For this reason, most of the filters will be exclude filters that removes something from the backup. Include filters are prefixed with a `+` and exclude filters are prefixed with a `-`. When Duplicati is evaluating filters, it will consider only the first full match, and not evaluate further. It will also evaluate folders before files, meaning that it is not possible to include a file, if the parent folder is excluded. Importantly, the filters are processed in the order they are supplied, which makes it possible to supply advanced rules. As an example: ``` +/usr/share/*.txt -*.txt -*.bin +/usr/share/*.bin ``` In the example, the first rule is applied before the second rule, which means that all `.txt` files in `/usr/share/` are included, but any other `.txt` files are excluded. The inverse goes for the `.bin` files, because the exclude rule is before the last rule, the files will be matched as exclude, even though there is an include rule. If we append a rule: ``` -/usr/share/ ``` Even if this rule is last, it will exclude the entire folder. Since the folder is excluded, the match on the include rule is never evaluated. This cut-off at the folder level makes it possible to fully avoid processing subfolders, which could otherwise be time consuming. # Using remote file locking Duplicati supports locking remote files to prevent deletion and this page describes how and when you should use remove locking. Duplicati supports Object Locking, also known as WORM (Write Once, Read Many), for compatible storage providers. When enabled, this feature ensures that your backup files cannot be deleted, modified, or overwritten for a specific period, even if your backup credentials are compromised. {% hint style="info" %} Remote file locking was introduced in Canary 2.2.0.103 {% endhint %} *** ### Why Use Remote File Locking? Remote file locking provides a "last line of defense" for your data. In a standard backup scenario, if an attacker gains access to your backup credentials, they can delete your offsite backups before encrypting your local data. Object locking prevents this. #### Use Cases * Ransomware Protection: Even if a ransomware strain finds your storage credentials, it cannot delete the locked backup volumes. Your data remains safe on the server until the lock expires. * Legal Hold & Compliance: For industries with strict data retention requirements (like HIPAA or GDPR),Compliance Mode ensures that data is preserved exactly as it was recorded for a mandatory duration. * Protection Against Accidental Deletion: Prevents accidental cleanup or bucket emptying by administrators. *** ### How it Works When you enable locking, Duplicati instructs the storage provider to apply a retention period to every file it uploads (`dblock`, `dindex`, and `dlist`). Since the storage provider is responsible for the actual locking, Duplicati and the machine it runs on does not have any way to unlock the files. #### Retention Modes | Mode | Description | Flexibility | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | |

Governance
(Unlocked)

|

Files are locked, and cannot be deleted even if the storage credentials are leaked.

Accounts with specific high-level permissions (Administrsators) can still bypass the lock if necessary.

| High (Good for testing) | |

Compliance
(Locked)

| No one can delete the files, including the root account holder. The lock cannot be shortened. | None (Strict security) | {% hint style="danger" %} WARNING: Use Compliance Mode with extreme caution. If you set a 10-year lock in Compliance Mode, those files cannot be deleted by you, the storage provider, or anyone else until the time expires. You will be billed for that storage for the entire duration, and there is no "undo" button. {% endhint %} *** ### Configuration Guide #### 1. Prerequisites Ensure your storage provider and bucket support Object Locking. This must often be enabled at the time of bucket creation. For instance, AWS S3 and B2 requires that Versioning is enabled for the bucket before locking can be enabled. Supported Backends: * Amazon S3 (and S3-compatible like IDrive e2) * Backblaze B2 * Azure Blob Storage * Google Cloud Storage #### 2. Enabling Locking in a Backup Job To enable locking for a backup, add the following option to your configuration (Step 5 "Options" in the WebUI): `--remote-file-lock-duration=30D` This example locks every new file for 30 days. You can use units like `H` (hours), `D` (days), `W` (weeks), or `M` (months). {% hint style="info" %} Always test this with compliance mode (default) and use a short duration until you are sure it works as intended. {% endhint %} #### 3. Provider-Specific Settings You can specify the mode depending on your provider: * S3: `--s3-object-lock-mode=governance` (default) or `compliance` * Backblaze B2: `--b2-retention-mode=governance` or `compliance` * Azure: `--azure-blob-immutability-policy-mode=unlocked` or `locked` * Google Cloud Storage: `--gcs-retention-policy-mode=unlocked` or `locked` {% hint style="info" %} **Google Cloud Storage** also requires using `--service-account-file` or `--service-account-json` as the default OAuth flow does not grant permissions to lock objects {% endhint %} *** ### Managing Locks via CLI If you have existing backups that you want to lock retrospectively, or if you need to audit your locks, use the Duplicati Command Line. You can also use the Commandline part of the UI to execute these commands. #### Apply locks to existing versions To lock files belonging to the most recent backup: ```bash duplicati-cli set-locks --remote-file-lock-duration=1M --version=0 ``` #### Sync lock status to local database If you changed locks manually on the server, update Duplicati’s local records: ```shellscript duplicati-cli read-lock-info --refresh-lock-info-complete=true ``` *** ### Important Storage Considerations #### Soft Deletion (S3, B2 and others) In S3 and B2, the buckets have versioning enabled for locking to work. When Duplicati "deletes" a file (e.g., during a compaction process), the provider creates a Delete Marker. The file seems to disappear from your file list, but the locked data is still there taking up space. This happens regardless of the file being locked or not. To avoid paying for data that should have been deleted, you must configure Lifecycle Rules on your bucket to permanently remove non-current versions once the lock expires. #### Direct Deletion (Azure and others) In Azure, the file remains visible and cannot be deleted. If Duplicati attempts to delete a locked file, the operation will simply fail because the blob itself is protected. In this case, no lifecycle rules are needed, because the file will actually be deleted when requested. #### Mixed (Google Cloud Storage) Google Cloud Storage can use both a WORM approach as well as a soft-delete and/or versioning approach. Depending on the bucket settings you will get either a soft-delete approach or a WORM setup. If you disable versioning and soft-delete, but enable object lock retention, this gives a WORM behavior that rejects deletes. Enabling soft-delete will allow the files to be marked as deleted but they are not actually deleted before the lock expires. Additionally, the soft-delete rules may keep the objects in the bucket even after the lock expires. # Telemetry collection This page describes the telemetry collected by Duplicati and how to opt out When Duplicat is running, it collects some basic non-identifying telemetry, such as what version of Duplicati is running, what operating system it is running on and similar values. When running a backup it also collects the type of connection, the duration, the size of the remote and local filesets. The purpose of this data collection is to give us an insight into how Duplicati is used and determine how we can best focus on making Duplicati better. We are sharing the aggregated data that we collect on a public page: [https://usage-reporter.duplicati.com](https://usage-reporter.duplicati.com/) ## Opting out Even though we try very hard to only collect non-sensitive information, you are in control and can choose to opt out of data collection if you want to. The easy way to opt out when using the UI is to visit the settings page, and simply choose "None / disabled":
However, since Duplicati also collects the event of starting Duplicati, you will still be sending at least one telemetry data point. If you prefer not to send anything at all, you need to set an environment variable before starting Duplicati: ```bash USAGEREPORTER_Duplicati_LEVEL=none ``` If this is set, no telemetry is sent, and it is not possible to enable telemetry from within the UI. # Sending reports Describes how to send reports with Duplicati Duplicati strives to make it as easy as possible to [set up backups](/getting-started/set-up-a-backup-in-the-ui), and using the built-in scheduler makes it easy to ensure that backups are running regularly. Because it is easy to set up a backup and forget about, it is possible to have a backup running with little interaction. Despite all efforts to make Duplicati as robust as possible against failures, it is not possible to handle every possible problem that may arise after the initial setup. Common failure causes is revoked credentials, filled storage, missing provider updates, etc. To avoid discovering too late that the backup had stopped working for some reason, it is highly recommended to set up automated monitoring of backups. Duplicati has a number of ways that you can use to send reports into a monitoring solution: * [Duplicati Console](/monitoring-and-notifications/sending-reports-via-email/monitoring-with-duplicati-console) * [Send emails](/monitoring-and-notifications/sending-reports-via-email/sending-reports-with-email) * [Send Jabber/XMPP](/monitoring-and-notifications/sending-reports-via-email/sending-jabber-xmpp-notifications) * [Send HTTP message](/monitoring-and-notifications/sending-reports-via-email/sending-http-notifications) * [Send Telegram message](/monitoring-and-notifications/sending-reports-via-email/sending-telegram-notifications) # Monitoring with Duplicati Console This page describes how to set up monitoring with Duplicati consoel The Duplicati console is a [paid option](https://duplicati.com/pricing) for handling monitoring of Duplicati backups, but has a free usage tier. To get started with the console, head over to the [Duplicati Console](https://app.duplicati.com/app/getting-started/connection-key) page and sign up or log in. If you connect your local instance to the Duplicati Console, this will automatically enable reporting to be sent from the Duplicati client to the console. Within the console, visit the [Alert Center](https://app.duplicati.com/app/alert-center/settings) to configure what notifications you want to received.

Duplicati Console screenshot

# Sending reports with email Describes the how to configure sending emails with backup details Sending emails is supported by having access to an SMTP server that will accept the inbound emails. From on your SMTP/email server provider you need to get a url, a username, and a password. If you are a [GMail or Google Workspace user, use the Google SMTP guide](https://support.google.com/a/answer/176600), otherwise consult your provider for these details. Besides the connection details, you also need to provide the recipient email address. Note that SMTP servers may sometimes restrict what recipients they allow, but generally using the provider SMTP server will allow sending to your own account. In the UI you can configure these mandatory values as well as the optional values. The basic options for sending email can be entered into the general settings, which will then apply to each backup. It is also possible to apply or change the settings for the individual backups by editing the advanced options. Here is how it looks when editing it in the user interface:

Set up email with the default options editor

Set up email option with a text field

You can toggle between the two views using the "Edit as list" and "Edit as text" links. Besides the mandatory options, it is also possible to configure: * Email sender address * The subject line * The email body * Conditions on when to send emails For details on how to customize the subject line and message body, see the [section on customizing message content](/monitoring-and-notifications/sending-reports-via-email/custom-message-content). If you prefer email logs, but would also like to get reports, check out the community provided [dupReport](https://forum.duplicati.com/t/announcing-dupreport-a-duplicati-email-report-summary-generator/1116) tool that can summarize the emails into overviews. # Sending Jabber/XMPP notifications Describes the how to configure sending notifications via Jabber/XMPP One of the supported notification methods in Duplicati is the open-source [XMPP protocol](https://en.wikipedia.org/wiki/XMPP), supported by a variety of projects, including commercial enterprise offerings. To send a notification via XMPP you need to supply one or more recipientes, an XMPP username and a password. In the UI you can configure these mandatory values as well as the optional values. The basic options for sending XMPP notifications can be entered into the general settings, which will then apply to each backup. It is also possible to apply or change the settings for the individual backups by editing the advanced options. Here is how it looks when editing it in the user interface:

Set up XMPP notifications with the default options editor

Set up XMPP option with a text field

You can toggle between the two views using the "Edit as list" and "Edit as text" links. Besides the mandatory options, it is also possible to configure: * The notification message and format * Conditions on when to send emails * Conditions on what log elements to include For details on how to customize the notification message, see the [section on customizing message content](/monitoring-and-notifications/sending-reports-via-email/custom-message-content). # Sending HTTP notifications This page describes how to send reports via the HTTP protocol The most versatile reporting option is the ability to send messages via the HTTP(s) protocol. By default messages are sent as a [form url encoded](https://en.wikipedia.org/wiki/Percent-encoding#The_application.2Fx-www-form-urlencoded_type) body in a request with the [POST](https://en.wikipedia.org/wiki/POST_\(HTTP\)) verb. To use the option, you only need to provide the url to send to:

Configuring a HTTP notification

Besides the URL it is also possible to configure: * The message body and type (JSON is supported) * The HTTP verb used * Conditions on when to send emails * Conditions on what log elements to include For details on how to customize the notification message, see the [section on customizing message content](/monitoring-and-notifications/sending-reports-via-email/custom-message-content). ### New in 2.0.9.106 You can now specify multiple urls, using the options: ``` --send-http-form-urls= --send-http-json-urls= ``` These two options greatly simplify sending notifications to multiple destinations. Additionally, the options make it possible to send both the form-encoded result in text format as well as in JSON format. # Sending Telegram notifications Describes the how to configure sending notifications via Telegram To send a notification via Telegram you need to supply a channel id, a bot token and a an api key. To obtain the bot token (aka bot id), message the `@BotFather` bot. After creating the bot, send a message to the bot, so it can reply. For more details on Telegram bots, see the [Telegram bot documentation](https://core.telegram.org/bots). After obtaining the bot token you can obtain the channel id with a cURL script: ``` BOT_TOKEN="YOURBOTTOKEN" curl -s "https://api.telegram.org/bot$BOT_TOKEN/getUpdates" \ | grep -o '"id":[0-9]*' | head -1 | cut -d':' -f2 ``` To obtain the API key, follow the [Telegram guide to creating an application](https://core.telegram.org/api/obtaining_api_id). With all required values obtained, you can set up the Telegram notifications in the general settings:

Set up Telegram notifications with the default options editor

Set up Telegram option with a text field

You can toggle between the two views using the "Edit as list" and "Edit as text" links. Besides the mandatory options, it is also possible to configure: * The notification message and format * Conditions on when to send emails * Conditions on what log elements to include ### Telegram Notification Options **Bot Configuration** **`--send-telegram-bot-id`**` `` ``(String) ` - The Telegram bot ID that will send messages **`--send-telegram-api-key`**` `` ``(String) ` - The API key for authenticating your Telegram bot **Message Destination** **`--send-telegram-channel-id`**` `` ``(String) ` - The channel ID where messages will be sent **`--send-telegram-topid-id`**` `` ``(String) ` - Topic ID for posting in specific topics within Telegram groups **Notification Content** **`--send-telegram-message`**` `` ``(String) ` - Template for message content with support for variables like %OPERATIONNAME%, %REMOTEURL%, %LOCALPATH%, and %PARSEDRESULT% **`--send-telegram-result-output-format`**` `` ``(format) ` - Format for presenting operation results * Duplicati * Json **Notification Filtering** **`--send-telegram-level`**` `` ``(level) ` - Controls which result types trigger notifications: * Success - Only successful operations * Warning - Operations that completed with warnings * Error - Operations that failed with recoverable errors * Fatal - Operations that failed with critical errors * All - All operation results regardless of status **`--send-telegram-any-operation`**` `` ``(Boolean) ` - When enabled, sends notifications for all operations, not just backups **`--send-telegram-log-level`**` `` ``(Enumeration) ` - Sets minimum severity level for included log entries: * ExplicitOnly - Show only explicitly requested messages * Profiling - Include performance measurement data * Verbose - Include detailed diagnostic information * Retry - Include information about retry attempts * Information - Include general status messages * DryRun - Include simulation mode outputs * Warning - Include potential issues that didn't prevent completion * Error - Include critical failures that require attention **`--send-telegram-log-filter`**` `` ``(String) ` - Filters log entries based on specified patterns **`--send-telegram-max-log-lines`**` `` ``(Integer) ` - Limits the number of log lines included in notifications For details on how to customize the notification message, see the [section on customizing message content](/monitoring-and-notifications/sending-reports-via-email/custom-message-content). # Custom message content This page describes the template system used to format text messages sent The template system used in Duplicati is quite simple, as it will essentially expand Windows-style environment placeholders, `%EXAMPLE%`, into values. The same replace logic works for both the subject line (if applicable) and the message body. **Note**: The description here only covers the text-based output (such as emails, etc). The t[emplate system for JSON](#json-output) is a bit different. Duplicati has defaults for the body and subject line, but you can specify a custom string here. For convenience, the string can also be a path to a file on the machine, which contains the template. An example custom template could look like: ``` Duplicati %OPERATIONNAME% for %backup-name% on %machine-name% The %OPERATIONNAME% operation has completed with the result: %PARSEDRESULT% Source folders: %LOCALPATH% Encryption module: %ENCRYPTION-MODULE% Compression module: %COMPRESSION-MODULE% %RESULT% ``` The template engine supports reporting any setting by using the setting name as the template value. Besides the options, there are also a few variables that can be used to extract information more easily: ``` %PARSEDRESULT% The parsed result op the operation: Success, Warning, Error %RESULT% When used in the body, this is the result/log of the backup, When used in the subject line, this is the same as %PARSEDRESULT% %OPERATIONNAME% The name of the operation, usually "backup", but could also be "restore" etc. %REMOTEURL% The backend url %LOCALPATH% The path to the local folders involved (i.e. the folders being backed up) %machine-id% The assigned unique random identifier for the current machine. Can be overridden with --machine-id %backup-id% The assigned id for the backup. Can be overridden with --backup-id %backup-name% The name of the backup. Can be overridden with --backup-name %machine-name% The name of the machine. Can be overridden with --machine-name ``` ## JSON output If the output is JSON it needs to be handled different than regular text, to ensure the result is valid. The logic for this is to re-use the templating concept, but only as a lookup, to figure out what keys to include in the results. An example template could be: ``` %OPERATIONNAME% %backup-name% %machine-name% %PARSEDRESULT% %LOCALPATH% %ENCRYPTION-MODULE% %COMPRESSION-MODULE% ``` This will ensure that each of those values will be included in the `extra` element in the JSON output. The default template for JSON output includes all fields listed above, but no options are included by default. # The local database This page describes the local database associated with a backup Duplicati uses [two databases](/technical-details/database-versions), one for the [Server](/duplicati-programs/server) and one for each backup. This page describes the overall purpose of the local database and how to work with it. The database itself is stored in the same folder as the server database and has a randomly generated name. If you have access to the backup files generated by Duplicati, you only need the passphrase to restore files. As described in the [migration section](/configuration-and-management/migrating-duplicati-to-a-new-machine), it is also everything that is needed to continue the backup. But to increase the performance and reduce the number of remote calls required during regular operations, Duplicati relies on a database with some well-structured data. The database is essentially a compact view of what data is stored at the remote destination, and as such it can always be created from the remote data. The only information that is lost if the database is recreated are log messages and the hashes of the remote volumes. The log messages are mostly important for error-tracing but the hashes of the remote volumes are important if the files are not encrypted, as this helps to ensure the backup integrity. Prior to running a backup, Duplicati will do a quick scan of the remote destination to ensure it looks as expected. This check is important, as making a backup with the assumption that data exists, could result in backups that can only be partially restored. If for the check fails for some reason, Duplicati will exit with an error message explaining the problem. For some errors it is possible to run the repair command and have the problem resolved. This works if all data required is still present on the system, but may fail if there is no real way to recover. If this is the case, there may be additional options in the section on [recovering from failure](/using-tools/recovering-from-failure). In rare cases, the database itself may become corrupted or defect. If this seems to be the case, it is safe to delete the local database and run the repair command. Note that it may take a while to recreate the database, but no data is lost in the process, and restores are possible without the database. # The server database This page describes the database kept by the Duplicati Server When the [Server](/duplicati-programs/server) is running, either stand-alone or as part of the [TrayIcon](/duplicati-programs/trayicon) or [Agent](/duplicati-programs/agent), it needs a place to store the configuration. All configuration data, logs and settings are stored inside the file `Duplicati-server.sqlite`. As the file extension reveals, this is an [SQLite](https://www.sqlite.org) database file and as such can be viewed and updated by any tool that works with SQLite databases. The database file is by default located in a folder that belongs to the user account running it. See the section on the [database location](/duplicati-programs/server#storing-data-in-different-places) for details on where this is and how to change it. ## Securing the database Due to the nature of Duplicati, this database can contain a few secrets that are vital to ensuring the integrity and security of the backups and also the Duplicati server itself. These secrets include both the user-provided secrets, such as the backup encryption passphrase and the connection credentials, but also server-provided secrets, such as the token signing keys, and optionally an SSL certificate password. Even though the database is located on the machine that makes the backup, it is important to prevent unauthorized access to the database, as it could be used for privilege escalation. And should the database ever be leaked, it is also important to ensure the contents are not accessible. To protect the database, Duplicati has support for a field-level encryption password. When activated, any setting that is deemed sensitive will be encrypted before being written to the database. This method ensures that the SQLite database itself is still readable, but the secrets are not readable without the encryption passphrase. To supply the field-level encryption password, start the Server, TrayIcon, or Agent with the commandline option `--settings-encryption-key=`. As the commandline can usually be read by other processes, it is also possible to supply this key via the environment variable `SETTINGS_ENCRYPTION_KEY=`. If you are aware of the risks, you can also set the commandline argument `--disable-db-encryption=true` instead of the key. This will remove existing encryption and not warn that the database is not encrypted. The simplest way to apply an encryption key, is to locate the server database, and create the file `preload.json` if it does not already exist. The file should contain the following: ```json { "env": { "*": { "SETTINGS_ENCRYPTION_KEY": "" } } } ``` Both the commandline arguments and environment variables can be set with the [Preload settings](/configuration-and-management/preload-settings) file, which makes it simpler to apply the same settings across executables, and removes the need for changing the service or launcher files. For additional protection of the encryption key, the [operating system Keychain, or an external secret provider](/security-and-secrets/using-the-secret-provider), can be used to further secure the encryption key. ## Database location When running Duplicati for the first time, it will find a place where it can store the configuration database. Some versions of Duplicati change the location where it looks for the databases, but this is always done backwards compatible, so new versions will also find the database in previous locations. Due to this logic, the locations can change a bit depending on what version of Duplicati was originally installed. It is possible to pick a different location for the database with the commandline option `--server-datafolder=` or use the environment variable `DUPLICATI_HOME`. To change the folder of an existing instance of Duplicati, perform these steps: 1. Stop Duplicati 2. Move the `Duplicati` folder from the old location to the new location 3. Change the startup parameters (environment variables, commandline arguments, or preload.json) 4. Start Duplicati again ### Limited access to the database folder To limit unauthorized access to the server database and other settings, Duplicati will enforce access to the folder for only the account that is currently running Duplicati. On Windows the permissions are set to include the current user, the Administrator and the System account. On Linux/MacOS the permissions are set to the current user only, as `root` always has access. Starting with version **2.3.1.0**, Duplicati **requires** that the data folder has the exact expected permissions, or it will refuse to use it. Duplicati will automatically lock down folders it creates, so for most users no change is required. However, if you rely on lax folder permissions, such as access from a different user account, you need to opt out of the permission check. You can opt out using any of these methods: * Pass the commandline option `--allow-insecure-datafolder` * Set the environment variable `DUPLICATI__ALLOW_INSECURE_DATAFOLDER=true` * Place a file named `insecure-permissions.txt` in the installation folder (it can be an empty file) {% hint style="warning" %} The previous method of placing `insecure-permissions.txt` in the data folder is no longer supported. The file must now be placed in the installation folder. {% endhint %} To force the correct permissions on an existing data folder, use the [ConfigureTool](/duplicati-programs/command-line-interface-cli-1/configuretool#securing-the-data-folder) `secure-datafolder` command. Some Docker setups may not be able to set the permissions and will need to apply `DUPLICATI__ALLOW_INSECURE_DATAFOLDER=true` in the image to run without the protections. ### Database location on Windows The default location for users running Duplicati is `%LOCALAPPDATA%\Duplicati` which usually resolves to something like `C:\Users\username\AppData\Local\Duplicati`. This folder is the non-roaming folder. Older versions of Duplicati used `%APPDATA%\Duplicati` which is the roaming folder, causing files to be synchronized across machines. However, since Duplicati is not meant to be an app that is useful for roaming, it is now using the non-roaming folder. When running Duplicati as a Windows Service, the `%LOCALAPPDATA%\Duplicati` folder resolves to: ``` C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati ``` Since this folder is under `C:\Windows` the contents may be deleted on major Windows upgrades (usually when the version number changes). For that reason, Duplicati will detect an attempt to store files in the `C:\Windows` folder and emit a warning. From version **2.1.0.108** and forward, Duplicati will avoid using a folder under `C:\Windows` and instead choose to use: ``` C:\ProgramData\Duplicati ``` ### Database location on Linux The default location when running Duplicati on Linux is `~/.config/Duplicati`. For most distros, running Duplicati as a service means running it as the root users, resulting in `/root/.config/Duplicati`. However, due to some compatibility mapping, the mapping is sometimes missing the prefix, causing Duplicati data to be stored in `/Duplicati`. From version 2.1.0.108, this location is avoided and the location `/var/lib/Duplicati` is used instead, if possible. ### Database location on MacOS The default location when running Duplicati on MacOS is `~/Library/Application Support/Duplicati`. Duplicati version 2.0.8.1 and older used the Linux-style `~/.config/Duplicati` but this is avoided since version 2.1.0.2. # Scripts These options allow you to integrate custom scripts with Duplicati operations, providing automation capabilities before and after backups, restores, or other tasks. **Pre and Post Operation Scripts** Run custom scripts before an operation starts or after it completes. Use these to perform preparation tasks (like database locking), cleanup actions, or to trigger notifications based on operation results. **Control Flow Management** Configure whether operations should continue or abort based on script execution status, with customizable timeout settings to prevent operation blocking. **Script Output Processing** Post-operation scripts receive operation results via standard output, enabling conditional processing based on success or failure. ### Scripting options **`--run-script-before`**`(Path)` Run a script on startup. Executes a script before performing an operation. The operation will block until the script has completed or timed out. **`--run-script-after`**`(Path)` Run a script on exit. Executes a script after performing an operation. The script will receive the operation results written to stdout. **`--run-script-before-required`**`(Path)` Run a required script on startup. Executes a script before performing an operation. The operation will block until the script has completed or timed out. If the script returns a non-zero error code or times out, the operation will be aborted. **`--run-script-timeout`**`(Timespan)` Sets the script timeout. Sets the maximum time a script is allowed to execute. If the script has not completed within this time, it will continue to execute but the operation will continue too, and no script output will be processed. Default value: `60s` ### Script Output Integration with Duplicati Logging You can add custom entries directly to Duplicati's log system from your scripts by using special prefixes in stdout messages. This allows script events to appear in both the Duplicati Log and Reports alongside native application events. **Supported Log Level Prefixes:** * `LOG:INFO` - For general information and success notifications * `LOG:WARN` - For potential issues that didn't prevent completion * `LOG:ERROR` - For critical failures that require attention **Example Usage (Linux / MacOS):** ```bash echo "LOG:INFO Preparation tasks completed successfully" echo "LOG:WARN Database backup older than 24 hours detected" echo "LOG:ERROR Unable to lock database, backup may contain inconsistent data" ``` **Example Usage (Windows):** ```batch echo LOG:INFO Preparation tasks completed successfully echo LOG:WARN Database backup older than 24 hours detected echo LOG:ERROR Unable to lock database, backup may contain inconsistent data ``` These messages will be captured with their appropriate severity levels and integrated into Duplicati's logging system, making script events traceable within the same monitoring interfaces you use for Duplicati itself. ### Sample Scripts ### run-script-example.bat (Windows) ```nohighlight @echo off REM ############################################################################### REM How to run scripts before or after backups REM ############################################################################### REM Duplicati is able to run scripts before and after backups. This REM functionality is available in the advanced options of any backup job (UI) or REM as option (CLI). The (advanced) options to run scripts are REM --run-script-before = REM --run-script-before-required = REM --run-script-timeout =