diff options
Diffstat (limited to 'docs/usage')
53 files changed, 2945 insertions, 649 deletions
diff --git a/docs/usage/benchmark.rst b/docs/usage/benchmark.rst new file mode 100644 index 000000000..27436a9b1 --- /dev/null +++ b/docs/usage/benchmark.rst @@ -0,0 +1 @@ +.. include:: benchmark_crud.rst.inc diff --git a/docs/usage/benchmark_crud.rst.inc b/docs/usage/benchmark_crud.rst.inc new file mode 100644 index 000000000..b76c091dc --- /dev/null +++ b/docs/usage/benchmark_crud.rst.inc @@ -0,0 +1,85 @@ +.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! + +.. _borg_benchmark_crud: + +borg benchmark crud +------------------- +.. code-block:: none + + borg [common options] benchmark crud [options] REPO PATH + +.. only:: html + + .. class:: borg-options-table + + +-------------------------------------------------------+----------+------------------------------------------+ + | **positional arguments** | + +-------------------------------------------------------+----------+------------------------------------------+ + | | ``REPO`` | repo to use for benchmark (must exist) | + +-------------------------------------------------------+----------+------------------------------------------+ + | | ``PATH`` | path were to create benchmark input data | + +-------------------------------------------------------+----------+------------------------------------------+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-------------------------------------------------------+----------+------------------------------------------+ + + .. raw:: html + + <script type='text/javascript'> + $(document).ready(function () { + $('.borg-options-table colgroup').remove(); + }) + </script> + +.. only:: latex + + REPO + repo to use for benchmark (must exist) + PATH + path were to create benchmark input data + + + :ref:`common_options` + | + +Description +~~~~~~~~~~~ + +This command benchmarks borg CRUD (create, read, update, delete) operations. + +It creates input data below the given PATH and backups this data into the given REPO. +The REPO must already exist (it could be a fresh empty repo or an existing repo, the +command will create / read / update / delete some archives named borg-test-data\* there. + +Make sure you have free space there, you'll need about 1GB each (+ overhead). + +If your repository is encrypted and borg needs a passphrase to unlock the key, use: + +BORG_PASSPHRASE=mysecret borg benchmark crud REPO PATH + +Measurements are done with different input file sizes and counts. +The file contents are very artificial (either all zero or all random), +thus the measurement results do not necessarily reflect performance with real data. +Also, due to the kind of content used, no compression is used in these benchmarks. + +C- == borg create (1st archive creation, no compression, do not use files cache) + C-Z- == all-zero files. full dedup, this is primarily measuring reader/chunker/hasher. + C-R- == random files. no dedup, measuring throughput through all processing stages. + +R- == borg extract (extract archive, dry-run, do everything, but do not write files to disk) + R-Z- == all zero files. Measuring heavily duplicated files. + R-R- == random files. No duplication here, measuring throughput through all processing + stages, except writing to disk. + +U- == borg create (2nd archive creation of unchanged input files, measure files cache speed) + The throughput value is kind of virtual here, it does not actually read the file. + U-Z- == needs to check the 2 all-zero chunks' existence in the repo. + U-R- == needs to check existence of a lot of different chunks in the repo. + +D- == borg delete archive (delete last remaining archive, measure deletion + compaction) + D-Z- == few chunks to delete / few segments to compact/remove. + D-R- == many chunks to delete / many segments to compact/remove. + +Please note that there might be quite some variance in these measurements. +Try multiple measurements and having a otherwise idle machine (and network, if you use it).
\ No newline at end of file diff --git a/docs/usage/break-lock.rst.inc b/docs/usage/break-lock.rst.inc index 5fa1cda55..17d5ce105 100644 --- a/docs/usage/break-lock.rst.inc +++ b/docs/usage/break-lock.rst.inc @@ -4,20 +4,44 @@ borg break-lock --------------- -:: +.. code-block:: none - borg break-lock <options> REPOSITORY + borg [common options] break-lock [options] [REPOSITORY] + +.. only:: html + + .. class:: borg-options-table + + +-------------------------------------------------------+----------------+-----------------------------------------+ + | **positional arguments** | + +-------------------------------------------------------+----------------+-----------------------------------------+ + | | ``REPOSITORY`` | repository for which to break the locks | + +-------------------------------------------------------+----------------+-----------------------------------------+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-------------------------------------------------------+----------------+-----------------------------------------+ + + .. raw:: html + + <script type='text/javascript'> + $(document).ready(function () { + $('.borg-options-table colgroup').remove(); + }) + </script> + +.. only:: latex -positional arguments REPOSITORY repository for which to break the locks -`Common options`_ - | + + :ref:`common_options` + | Description ~~~~~~~~~~~ This command breaks the repository and cache locks. Please use carefully and only while no borg process (on any machine) is -trying to access the Cache or the Repository. +trying to access the Cache or the Repository.
\ No newline at end of file diff --git a/docs/usage/change-passphrase.rst.inc b/docs/usage/change-passphrase.rst.inc index 3bb827a4f..1114d3503 100644 --- a/docs/usage/change-passphrase.rst.inc +++ b/docs/usage/change-passphrase.rst.inc @@ -4,19 +4,43 @@ borg change-passphrase ---------------------- -:: +.. code-block:: none - borg change-passphrase <options> REPOSITORY + borg [common options] change-passphrase [options] [REPOSITORY] + +.. only:: html + + .. class:: borg-options-table + + +-------------------------------------------------------+----------------+--+ + | **positional arguments** | + +-------------------------------------------------------+----------------+--+ + | | ``REPOSITORY`` | | + +-------------------------------------------------------+----------------+--+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-------------------------------------------------------+----------------+--+ + + .. raw:: html + + <script type='text/javascript'> + $(document).ready(function () { + $('.borg-options-table colgroup').remove(); + }) + </script> + +.. only:: latex -positional arguments REPOSITORY -`Common options`_ - | + + :ref:`common_options` + | Description ~~~~~~~~~~~ The key files used for repository encryption are optionally passphrase -protected. This command can be used to change this passphrase. +protected. This command can be used to change this passphrase.
\ No newline at end of file diff --git a/docs/usage/check.rst b/docs/usage/check.rst new file mode 100644 index 000000000..143b0b4c0 --- /dev/null +++ b/docs/usage/check.rst @@ -0,0 +1 @@ +.. include:: check.rst.inc diff --git a/docs/usage/check.rst.inc b/docs/usage/check.rst.inc index 17d6cf492..cc61a76ea 100644 --- a/docs/usage/check.rst.inc +++ b/docs/usage/check.rst.inc @@ -4,34 +4,80 @@ borg check ---------- -:: +.. code-block:: none - borg check <options> REPOSITORY_OR_ARCHIVE + borg [common options] check [options] [REPOSITORY_OR_ARCHIVE] + +.. only:: html + + .. class:: borg-options-table + + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **positional arguments** | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``REPOSITORY_OR_ARCHIVE`` | repository or archive to check consistency of | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **optional arguments** | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--repository-only`` | only perform repository checks | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--archives-only`` | only perform archives checks | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--verify-data`` | perform cryptographic archive data integrity verification (conflicts with ``--repository-only``) | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--repair`` | attempt to repair any inconsistencies found | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--save-space`` | work slower, but using less space | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Archive filters** — Archive filters can be applied to repository targets. | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-P PREFIX``, ``--prefix PREFIX`` | only consider archive names starting with this prefix. | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-a GLOB``, ``--glob-archives GLOB`` | only consider archive names matching the glob. sh: rules apply, see "borg help patterns". ``--prefix`` and ``--glob-archives`` are mutually exclusive. | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--sort-by KEYS`` | Comma-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--first N`` | consider first N archives after other filters were applied | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--last N`` | consider last N archives after other filters were applied | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + + .. raw:: html + + <script type='text/javascript'> + $(document).ready(function () { + $('.borg-options-table colgroup').remove(); + }) + </script> + +.. only:: latex -positional arguments REPOSITORY_OR_ARCHIVE repository or archive to check consistency of -optional arguments - ``--repository-only`` - | only perform repository checks - ``--archives-only`` - | only perform archives checks - ``--verify-data`` - | perform cryptographic archive data integrity verification (conflicts with --repository-only) - ``--repair`` - | attempt to repair any inconsistencies found - ``--save-space`` - | work slower, but using less space - ``--last N`` - | only check last N archives (Default: all) - ``-P``, ``--prefix`` - | only consider archive names starting with this prefix - ``-p``, ``--progress`` - | show progress display while checking - -`Common options`_ - | + + optional arguments + --repository-only only perform repository checks + --archives-only only perform archives checks + --verify-data perform cryptographic archive data integrity verification (conflicts with ``--repository-only``) + --repair attempt to repair any inconsistencies found + --save-space work slower, but using less space + + + :ref:`common_options` + | + + Archive filters + -P PREFIX, --prefix PREFIX only consider archive names starting with this prefix. + -a GLOB, --glob-archives GLOB only consider archive names matching the glob. sh: rules apply, see "borg help patterns". ``--prefix`` and ``--glob-archives`` are mutually exclusive. + --sort-by KEYS Comma-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp + --first N consider first N archives after other filters were applied + --last N consider last N archives after other filters were applied + Description ~~~~~~~~~~~ @@ -50,7 +96,7 @@ First, the underlying repository data files are checked: stored in the segments. - If you use a remote repo server via ssh:, the repo check is executed on the repo server without causing significant network traffic. -- The repository check can be skipped using the --archives-only option. +- The repository check can be skipped using the ``--archives-only`` option. Second, the consistency and correctness of the archive metadata is verified: @@ -72,16 +118,12 @@ Second, the consistency and correctness of the archive metadata is verified: decryption and this is always done client-side, because key access will be required). - The archive checks can be time consuming, they can be skipped using the - --repository-only option. + ``--repository-only`` option. -The --verify-data option will perform a full integrity verification (as opposed to +The ``--verify-data`` option will perform a full integrity verification (as opposed to checking the CRC32 of the segment) of data, which means reading the data from the repository, decrypting and decompressing it. This is a cryptographic verification, which will detect (accidental) corruption. For encrypted repositories it is tamper-resistant as well, unless the attacker has access to the keys. -It is also very slow. - ---verify-data only verifies data used by the archives specified with --last, ---prefix or an explicitly named archive. If none of these are passed, -all data in the repository is verified. +It is also very slow.
\ No newline at end of file diff --git a/docs/usage/common-options.rst.inc b/docs/usage/common-options.rst.inc index 0a3c3c3ed..093041c47 100644 --- a/docs/usage/common-options.rst.inc +++ b/docs/usage/common-options.rst.inc @@ -1,24 +1,18 @@ - ``-h``, ``--help`` - | show this help message and exit - ``--critical`` - | work on log level CRITICAL - ``--error`` - | work on log level ERROR - ``--warning`` - | work on log level WARNING (default) - ``--info``, ``-v``, ``--verbose`` - | work on log level INFO - ``--debug`` - | enable debug output, work on log level DEBUG - ``--lock-wait N`` - | wait for the lock, but max. N seconds (default: 1). - ``--show-version`` - | show/log the borg version - ``--show-rc`` - | show/log the return code (rc) - ``--no-files-cache`` - | do not load/update the file metadata cache used to detect unchanged files - ``--umask M`` - | set umask to M (local and remote, default: 0077) - ``--remote-path PATH`` - | set remote path to executable (default: "borg")
\ No newline at end of file +-h, --help show this help message and exit +--critical work on log level CRITICAL +--error work on log level ERROR +--warning work on log level WARNING (default) +--info, -v, --verbose work on log level INFO +--debug enable debug output, work on log level DEBUG +--debug-topic TOPIC enable TOPIC debugging (can be specified multiple times). The logger path is borg.debug.<TOPIC> if TOPIC is not fully qualified. +-p, --progress show progress information +--log-json Output one JSON object per log line instead of formatted text. +--lock-wait SECONDS wait at most SECONDS for acquiring a repository/cache lock (default: 1). +--show-version show/log the borg version +--show-rc show/log the return code (rc) +--no-files-cache do not load/update the file metadata cache used to detect unchanged files +--umask M set umask to M (local and remote, default: 0077) +--remote-path PATH use PATH as borg executable on the remote (default: "borg") +--remote-ratelimit RATE set remote network upload rate limit in kiByte/s (default: 0=unlimited) +--consider-part-files treat part files like normal files (e.g. to list/extract them) +--debug-profile FILE Write execution profile in Borg format into FILE. For local use a Python-compatible file can be generated by suffixing FILE with ".pyprof". diff --git a/docs/usage/create.rst b/docs/usage/create.rst new file mode 100644 index 000000000..ec49f5fa8 --- /dev/null +++ b/docs/usage/create.rst @@ -0,0 +1,69 @@ +.. include:: create.rst.inc + +Examples +~~~~~~~~ +:: + + # Backup ~/Documents into an archive named "my-documents" + $ borg create /path/to/repo::my-documents ~/Documents + + # same, but list all files as we process them + $ borg create --list /path/to/repo::my-documents ~/Documents + + # Backup ~/Documents and ~/src but exclude pyc files + $ borg create /path/to/repo::my-files \ + ~/Documents \ + ~/src \ + --exclude '*.pyc' + + # Backup home directories excluding image thumbnails (i.e. only + # /home/<one directory>/.thumbnails is excluded, not /home/*/*/.thumbnails etc.) + $ borg create /path/to/repo::my-files /home \ + --exclude 'sh:/home/*/.thumbnails' + + # Backup the root filesystem into an archive named "root-YYYY-MM-DD" + # use zlib compression (good, but slow) - default is lz4 (fast, low compression ratio) + $ borg create -C zlib,6 /path/to/repo::root-{now:%Y-%m-%d} / --one-file-system + + # Backup a remote host locally ("pull" style) using sshfs + $ mkdir sshfs-mount + $ sshfs root@example.com:/ sshfs-mount + $ cd sshfs-mount + $ borg create /path/to/repo::example.com-root-{now:%Y-%m-%d} . + $ cd .. + $ fusermount -u sshfs-mount + + # Make a big effort in fine granular deduplication (big chunk management + # overhead, needs a lot of RAM and disk space, see formula in internals + # docs - same parameters as borg < 1.0 or attic): + $ borg create --chunker-params 10,23,16,4095 /path/to/repo::small /smallstuff + + # Backup a raw device (must not be active/in use/mounted at that time) + $ dd if=/dev/sdx bs=10M | borg create /path/to/repo::my-sdx - + + # No compression (none) + $ borg create --compression none /path/to/repo::arch ~ + + # Super fast, low compression (lz4, default) + $ borg create /path/to/repo::arch ~ + + # Less fast, higher compression (zlib, N = 0..9) + $ borg create --compression zlib,N /path/to/repo::arch ~ + + # Even slower, even higher compression (lzma, N = 0..9) + $ borg create --compression lzma,N /path/to/repo::arch ~ + + # Only compress compressible data with lzma,N (N = 0..9) + $ borg create --compression auto,lzma,N /path/to/repo::arch ~ + + # Use short hostname, user name and current time in archive name + $ borg create /path/to/repo::{hostname}-{user}-{now} ~ + # Similar, use the same datetime format as borg 1.1 will have as default + $ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S} ~ + # As above, but add nanoseconds + $ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S.%f} ~ + + # Backing up relative paths by moving into the correct directory first + $ cd /home/user/Documents + # The root directory of the archive will be "projectA" + $ borg create /path/to/repo::daily-projectA-{now:%Y-%m-%d} projectA diff --git a/docs/usage/create.rst.inc b/docs/usage/create.rst.inc index 6e176a52e..c70e2a8f1 100644 --- a/docs/usage/create.rst.inc +++ b/docs/usage/create.rst.inc @@ -4,93 +4,230 @@ borg create ----------- -:: +.. code-block:: none - borg create <options> ARCHIVE PATH + borg [common options] create [options] ARCHIVE [PATH...] + +.. only:: html + + .. class:: borg-options-table + + +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | **positional arguments** | + +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``ARCHIVE`` | name of archive to create (must be also a valid directory name) | |