LogoLogo
  • Duplicati Documentation
  • Getting Started
    • Installation
    • Set up a backup in the UI
    • Running a backup
    • Restoring files
  • Detailed descriptions
    • Choosing Duplicati Type
    • Using the secret provider
      • Local providers
      • Cloud providers
      • Advanced configurations
    • Using remote management
      • Using remote control with agent
    • Migrating Duplicati to a new machine
    • Scripts
    • Sending reports
      • Monitoring with Duplicati Console
      • Sending reports with email
      • Sending Jabber/XMPP notifications
      • Sending HTTP notifications
      • Sending Telegram notifications
      • Custom message content
    • Duplicati Access Password
    • Import and export backup configurations
    • Filters in Duplicati
    • The local database
    • The server database
    • Preload settings
    • Retention settings
    • Using Duplicati with Linux
    • Using Duplicati from Docker
    • Using Duplicati with MacOS
    • Using Duplicati with Windows
    • Running a self-hosted OAuth Server
  • Using tools
    • Encrypting and decrypting files
    • Using Duplicati from the Command Line
    • Recovering from failure
    • Disaster recovery
  • Backup destinations
    • Destination overview
    • Standard based destinations
      • File Destination
      • S3-compatible Destination
      • FTP Destination
      • SFTP (SSH) Destination
      • WebDAV Destination
      • OpenStack Destination
      • Rclone Destination
      • CIFS (aka SMB) Destination
    • Provider specific destinations
      • Backblaze B2 Destination
      • Box.com Destination
      • Rackspace CloudFiles Destination
      • IDrive e2 Destination
      • Mega.nz Destination
      • Aliyun OSS Destination
      • Tencent COS Destination
      • Jottacloud Destination
      • pCloud Destination
      • Azure Blob Storage Destination
      • Google Cloud Storage Destination
      • Microsoft Group Destination
      • SharePoint Destination
      • SharePoint v2 (Graph API)
      • Amazon S3 destination
    • File synchronization providers
      • Dropbox Destination
      • Google Drive Destination
      • OneDrive Destination
      • OneDrive For Business Destination
    • Decentralized providers
      • Sia Destination
      • Storj Destination
      • TahoeLAFS destination
  • Duplicati Programs
    • TrayIcon
    • Server
    • Command Line Interface CLI
    • Service and WindowsService
    • Command Line Tools
      • AutoUpdater
      • BackendTester
      • BackendTool
      • RecoveryTool
      • SecretTool
      • SharpAESCrypt
      • Snapshots
      • ServerUtil
    • Agent
    • LICENSE
      • Duplicati Inc & Open Source
      • License Agreement
    • OAuth Server
  • SUPPORT
  • Installation details
    • Release channels and versions
      • Upgrading and downgrading
      • Downgrade from 2.1.0.2 to 2.0.8.1
    • Package options
    • Developer
  • TECHNICAL DETAILS
    • Architecture Premises
    • Understanding Backup
      • How Backup Works
      • Encryption Algorithms
      • Backup size parameters
    • Understanding Restore
      • How Restore Works
      • Disaster Recovery
    • Database versions
    • Server authentication model
    • Option formats
Powered by GitBook
On this page

Was this helpful?

Export as PDF
  1. TECHNICAL DETAILS

Architecture Premises

This page describes the overall design goals for the Duplicati client

PreviousTECHNICAL DETAILSNextUnderstanding Backup

Last updated 6 months ago

Was this helpful?

At the core, Duplicati is designed to make a "copy" of files on a machine, and later be able to restore these files as they looked at the time they were "copied".

Most data to be "copied" is expected to change slowly, such that each new "copy" contains add/delete/modifications applied to various files, but overall contains a significant part that is unchanged from last "copy".

The best storage destination is a choice that depends on multiple factors which can change over time. And to cater to these changes in storage solutions, new solutions may appear, having unique properties.

Since the storage destination may be remote, it is assumed that the transfer link can be slow and have random errors. It is also assumed that the remote destination is not fully infallible.

It is also assumed that the remote destination is not owned by the entity making the backup, so confidentiality of stored files cannot be guaranteed by the storage provider.

Overall design

With the above assumptions and observations, the core of Duplicati was designed with specfic choices:

  • Each backup "copy" should a snapshot of the system

  • Files are treated as a stream of blocks

  • Data is broken into fixed-size blocks

  • Blocks are only stored once to reduce storage space (deduplication)

  • Blocks are combined into volumes to reduce the number of remote storage requests

  • Volumes are compressed to save storage space (compression)

  • Volumes are encrypted to ensure integrity and confidentiality (encryption)

  • No storage destination server/service is required (client-only)

These choices are the foundation from which the rest of Duplicati is built. With the use of client-side encryption, Duplicati can be classified as a system, where the possesion of the encryption key decides who has access to the data.

To ensure that Duplicati can work with different storage destinations, such as S3 and a network share, any interaction with the storage destination is limited to 4 commands: GET, PUT, LIST, and DELETE. Any storage system that supports these 4 commands can work with Duplicati.

The deduplication and block-level tracking makes it possible to avoid the traditional full+incremental cycle, where the full backup is always required to get to a specific version.

To ensure the remote storage is consistent, Duplicati uses a local database, where all files, blocks, volumes, etc. are tracked. This database makes it possible to detect remote storage problems early, and makes it possible to run backups without downloading any volumes from the remote storage.

Trust-no-one / Zero-Trust