build_usage

8 files changed, 138 insertions, 95 deletions

diff --git a/docs/usage/create.rst.inc b/docs/usage/create.rst.inc
index e6fa4e3a1..cb377b2ac 100644
--- a/docs/usage/create.rst.inc
+++ b/docs/usage/create.rst.inc
@@ -260,16 +260,15 @@ how much your repository will grow. Please note that the "All archives" stats re
 the state after creation. Also, the ``--stats`` and ``--dry-run`` options are mutually
 exclusive because the data is not actually compressed and deduplicated during a dry run.
 
-See the output of the "borg help patterns" command for more help on exclude patterns.
+For more help on include/exclude patterns, see the :ref:`borg_patterns` command output.
 
-See the output of the "borg help placeholders" command for more help on placeholders.
+For more help on placeholders, see the :ref:`borg_placeholders` command output.
 
 .. man NOTES
 
 The ``--exclude`` patterns are not like tar. In tar ``--exclude`` .bundler/gems will
 exclude foo/.bundler/gems. In borg it will not, you need to use ``--exclude``
-'\*/.bundler/gems' to get the same effect. See ``borg help patterns`` for
-more information.
+'\*/.bundler/gems' to get the same effect.
 
 In addition to using ``--exclude`` patterns, it is possible to use
 ``--exclude-if-present`` to specify the name of a filesystem object (e.g. a file
diff --git a/docs/usage/delete.rst.inc b/docs/usage/delete.rst.inc
index e058804f1..0cf94191f 100644
--- a/docs/usage/delete.rst.inc
+++ b/docs/usage/delete.rst.inc
@@ -112,7 +112,7 @@ Please note that the "All archives" stats refer to the state after deletion.
 You can delete multiple archives by specifying their common prefix, if they
 have one, using the ``--prefix PREFIX`` option. You can also specify a shell
 pattern to match multiple archives using the ``--glob-archives GLOB`` option
-(for more info on these patterns, see ``borg help patterns``). Note that these
+(for more info on these patterns, see :ref:`borg_patterns`). Note that these
 two options are mutually exclusive.
 
 To avoid accidentally deleting archives, especially when using glob patterns,
diff --git a/docs/usage/diff.rst.inc b/docs/usage/diff.rst.inc
index ec3bffc29..267aad605 100644
--- a/docs/usage/diff.rst.inc
+++ b/docs/usage/diff.rst.inc
@@ -102,4 +102,4 @@ If you did not create the archives with different chunker params,
 pass ``--same-chunker-params``.
 Note that the chunker params changed from Borg 0.xx to 1.0.
 
-See the output of the "borg help patterns" command for more help on exclude patterns.
\ No newline at end of file
+For more help on include/exclude patterns, see the :ref:`borg_patterns` command output.
\ No newline at end of file
diff --git a/docs/usage/export-tar.rst.inc b/docs/usage/export-tar.rst.inc
index c898d7b51..c02a04ee6 100644
--- a/docs/usage/export-tar.rst.inc
+++ b/docs/usage/export-tar.rst.inc
@@ -112,7 +112,7 @@ By default the entire archive is extracted but a subset of files and directories
 can be selected by passing a list of ``PATHs`` as arguments.
 The file selection can further be restricted by using the ``--exclude`` option.
 
-See the output of the "borg help patterns" command for more help on exclude patterns.
+For more help on include/exclude patterns, see the :ref:`borg_patterns` command output.
 
 ``--progress`` can be slower than no progress display, since it makes one additional
 pass over the archive metadata.
\ No newline at end of file
diff --git a/docs/usage/extract.rst.inc b/docs/usage/extract.rst.inc
index ce6ede4cc..c157c768a 100644
--- a/docs/usage/extract.rst.inc
+++ b/docs/usage/extract.rst.inc
@@ -106,7 +106,7 @@ archive is extracted but a subset of files and directories can be selected
 by passing a list of ``PATHs`` as arguments. The file selection can further
 be restricted by using the ``--exclude`` option.
 
-See the output of the "borg help patterns" command for more help on exclude patterns.
+For more help on include/exclude patterns, see the :ref:`borg_patterns` command output.
 
 By using ``--dry-run``, you can do all extraction steps except actually writing the
 output data: reading metadata and data chunks from the repo, checking the hash/hmac,
diff --git a/docs/usage/help.rst.inc b/docs/usage/help.rst.inc
index cd7682db0..39dcca80f 100644
--- a/docs/usage/help.rst.inc
+++ b/docs/usage/help.rst.inc
@@ -20,10 +20,15 @@ store all files as `some/path/.../file.ext` and ``borg create
 /path/to/repo /home/user`` will store all files as
 `home/user/.../file.ext`.
 
+A directory exclusion pattern can end either with or without a slash ('/').
+If it ends with a slash, such as `some/path/`, the directory will be
+included but not its content. If it does not end with a slash, such as
+`some/path`, both the directory and content will be excluded.
+
 File patterns support these styles: fnmatch, shell, regular expressions,
 path prefixes and path full-matches. By default, fnmatch is used for
 ``--exclude`` patterns and shell-style is used for the ``--pattern``
- option. For commands that support patterns in their ``PATH`` argument
+option. For commands that support patterns in their ``PATH`` argument
 like (``borg list``), the default pattern is path prefix.
 
 Starting with Borg 1.2, for all but regular expression pattern matching
@@ -144,64 +149,85 @@ Examples::
     EOF
     $ borg create --exclude-from exclude.txt backup /
 
-    A more general and easier to use way to define filename matching patterns exists
-    with the ``--pattern`` and ``--patterns-from`` options. Using these, you may
-    specify the backup roots (starting points) and patterns for inclusion/exclusion.
-    A root path starts with the prefix `R`, followed by a path (a plain path, not a
-    file pattern). An include rule starts with the prefix +, an exclude rule starts
-    with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern.
-
-    .. note::
-
-        Via ``--pattern`` or ``--patterns-from`` you can define BOTH inclusion and exclusion
-        of files using pattern prefixes ``+`` and ``-``. With ``--exclude`` and
-        ``--exclude-from`` ONLY excludes are defined.
-
-    Inclusion patterns are useful to include paths that are contained in an excluded
-    path. The first matching pattern is used so if an include pattern matches before
-    an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches
-    a directory, it won't recurse into it and won't discover any potential matches for
-    include rules below that directory.
-
-    .. note::
-
-        It's possible that a sub-directory/file is matched while parent directories are not.
-        In that case, parent directories are not backed up thus their user, group, permission,
-        etc. can not be restored.
-
-    Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is
-    shell style (`sh:`), so those patterns behave similar to rsync include/exclude
-    patterns. The pattern style can be set via the `P` prefix.
-
-    Patterns (``--pattern``) and excludes (``--exclude``) from the command line are
-    considered first (in the order of appearance). Then patterns from ``--patterns-from``
-    are added. Exclusion patterns from ``--exclude-from`` files are appended last.
-
-    Examples::
-
-        # backup pics, but not the ones from 2018, except the good ones:
-        # note: using = is essential to avoid cmdline argument parsing issues.
-        borg create --pattern=+pics/2018/good --pattern=-pics/2018 repo::arch pics
-
-        # use a file with patterns:
-        borg create --patterns-from patterns.lst repo::arch
-
-    The patterns.lst file could look like that::
-
-        # "sh:" pattern style is the default, so the following line is not needed:
-        P sh
-        R /
-        # can be rebuild
-        - /home/*/.cache
-        # they're downloads for a reason
-        - /home/*/Downloads
-        # susan is a nice person
-        # include susans home
-        + /home/susan
-        # don't backup the other home directories
-        - /home/*
-        # don't even look in /proc
-        ! /proc
+A more general and easier to use way to define filename matching patterns exists
+with the ``--pattern`` and ``--patterns-from`` options. Using these, you may
+specify the backup roots (starting points) and patterns for inclusion/exclusion.
+A root path starts with the prefix `R`, followed by a path (a plain path, not a
+file pattern). An include rule starts with the prefix +, an exclude rule starts
+with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern.
+
+.. note::
+
+    Via ``--pattern`` or ``--patterns-from`` you can define BOTH inclusion and exclusion
+    of files using pattern prefixes ``+`` and ``-``. With ``--exclude`` and
+    ``--exclude-from`` ONLY excludes are defined.
+
+Inclusion patterns are useful to include paths that are contained in an excluded
+path. The first matching pattern is used so if an include pattern matches before
+an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches
+a directory, it won't recurse into it and won't discover any potential matches for
+include rules below that directory.
+
+.. note::
+
+    It's possible that a sub-directory/file is matched while parent directories are not.
+    In that case, parent directories are not backed up thus their user, group, permission,
+    etc. can not be restored.
+
+Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is
+shell style (`sh:`), so those patterns behave similar to rsync include/exclude
+patterns. The pattern style can be set via the `P` prefix.
+
+Patterns (``--pattern``) and excludes (``--exclude``) from the command line are
+considered first (in the order of appearance). Then patterns from ``--patterns-from``
+are added. Exclusion patterns from ``--exclude-from`` files are appended last.
+
+Examples::
+
+    # backup pics, but not the ones from 2018, except the good ones:
+    # note: using = is essential to avoid cmdline argument parsing issues.
+    borg create --pattern=+pics/2018/good --pattern=-pics/2018 repo::arch pics
+
+    # use a file with patterns:
+    borg create --patterns-from patterns.lst repo::arch
+
+The patterns.lst file could look like that::
+
+    # "sh:" pattern style is the default, so the following line is not needed:
+    P sh
+    R /
+    # can be rebuild
+    - /home/*/.cache
+    # they're downloads for a reason
+    - /home/*/Downloads
+    # susan is a nice person
+    # include susans home
+    + /home/susan
+    # also back up this exact file
+    + pf:/home/bobby/specialfile.txt
+    # don't backup the other home directories
+    - /home/*
+    # don't even look in /proc
+    ! /proc
+
+You can specify recursion roots either on the command line or in a patternfile::
+
+    # these two commands do the same thing
+    borg create --exclude /home/bobby/junk repo::arch /home/bobby /home/susan
+    borg create --patterns-from patternfile.lst repo::arch
+
+The patternfile::
+
+    # note that excludes use fm: by default and patternfiles use sh: by default.
+    # therefore, we need to specify fm: to have the same exact behavior.
+    P fm
+    R /home/bobby
+    R /home/susan
+
+    - /home/bobby/junk
+
+This allows you to share the same patterns between multiple repositories
+without needing to specify them on the command line.
 
 .. _borg_placeholders:
 
@@ -223,11 +249,11 @@ and ``--remote-path`` values support these placeholders:
 
 {now}
     The current local date and time, by default in ISO-8601 format.
-    You can also supply your own `format string <https://docs.python.org/3.5/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {now:%Y-%m-%d_%H:%M:%S}
+    You can also supply your own `format string <https://docs.python.org/3.8/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {now:%Y-%m-%d_%H:%M:%S}
 
 {utcnow}
     The current UTC date and time, by default in ISO-8601 format.
-    You can also supply your own `format string <https://docs.python.org/3.5/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {utcnow:%Y-%m-%d_%H:%M:%S}
+    You can also supply your own `format string <https://docs.python.org/3.8/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {utcnow:%Y-%m-%d_%H:%M:%S}
 
 {user}
     The user name (or UID, if no name is available) of the user running borg.
diff --git a/docs/usage/init.rst.inc b/docs/usage/init.rst.inc
index aa95281aa..d3d3f301d 100644
--- a/docs/usage/init.rst.inc
+++ b/docs/usage/init.rst.inc
@@ -62,31 +62,41 @@ Description
 This command initializes an empty repository. A repository is a filesystem
 directory containing the deduplicated data from zero or more archives.
 
-Encryption can be enabled at repository init time. It cannot be changed later.
+Encryption mode TLDR
+++++++++++++++++++++
 
-It is not recommended to work without encryption. Repository encryption protects
-you e.g. against the case that an attacker has access to your backup repository.
+The encryption mode can only be configured when creating a new repository -
+you can neither configure it on a per-archive basis nor change the
+encryption mode of an existing repository.
 
-Borg relies on randomly generated key material and uses that for chunking, id
-generation, encryption and authentication. The key material is encrypted using
-the passphrase you give before it is stored on-disk.
+Use ``repokey``::
 
-You need to be careful with the key / the passphrase:
+    borg init --encryption repokey /path/to/repo
 
-If you want "passphrase-only" security, use one of the repokey modes. The
-key will be stored inside the repository (in its "config" file). In above
-mentioned attack scenario, the attacker will have the key (but not the
-passphrase).
+Or ``repokey-blake2`` depending on which is faster on your client machines (see below)::
 
-If you want "passphrase and having-the-key" security, use one of the keyfile
-modes. The key will be stored in your home directory (in .config/borg/keys).
-In the attack scenario, the attacker who has just access to your repo won't
-have the key (and also not the passphrase).
+    borg init --encryption repokey-blake2 /path/to/repo
 
-Make a backup copy of the key file (keyfile mode) or repo config file
-(repokey mode) and keep it at a safe place, so you still have the key in
-case it gets corrupted or lost. Also keep the passphrase at a safe place.
-The backup that is encrypted with that key won't help you with that, of course.
+Borg will:
+
+1. Ask you to come up with a passphrase.
+2. Create a borg key (which contains 3 random secrets. See :ref:`key_files`).
+3. Encrypt the key with your passphrase.
+4. Store the encrypted borg key inside the repository directory (in the repo config).
+   This is why it is essential to use a secure passphrase.
+5. Encrypt and sign your backups to prevent anyone from reading or forging them unless they
+   have the key and know the passphrase. Make sure to keep a backup of
+   your key **outside** the repository - do not lock yourself out by
+   "leaving your keys inside your car" (see :ref:`borg_key_export`).
+   For remote backups the encryption is done locally - the remote machine
+   never sees your passphrase, your unencrypted key or your unencrypted files.
+   Chunking and id generation are also based on your key to improve
+   your privacy.
+6. Use the key when extracting files to decrypt them and to verify that the contents of
+   the backups have not been accidentally or maliciously altered.
+
+Picking a passphrase
+++++++++++++++++++++
 
 Make sure you use a good passphrase. Not too short, not too simple. The real
 encryption / decryption key is encrypted with / locked by your passphrase.
@@ -106,14 +116,22 @@ a different keyboard layout.
 You can change your passphrase for existing repos at any time, it won't affect
 the encryption/decryption key or other secrets.
 
-Encryption modes
-++++++++++++++++
+More encryption modes
++++++++++++++++++++++
+
+Only use ``--encryption none`` if you are OK with anyone who has access to
+your repository being able to read your backups and tamper with their
+contents without you noticing.
+
+If you want "passphrase and having-the-key" security, use ``--encryption keyfile``.
+The key will be stored in your home directory (in ``~/.config/borg/keys``).
 
-You can choose from the encryption modes seen in the table below on a per-repo
-basis. The mode determines encryption algorithm, hash/MAC algorithm and also the
-key storage location.
+If you do **not** want to encrypt the contents of your backups, but still
+want to detect malicious tampering use ``--encryption authenticated``.
 
-Example: `borg init --encryption repokey ...`
+If ``BLAKE2b`` is faster than ``SHA-256`` on your hardware, use ``--encryption authenticated-blake2``,
+``--encryption repokey-blake2`` or ``--encryption keyfile-blake2``. Note: for remote backups
+the hashing is done on your local machine.
 
 .. nanorst: inline-fill
 
diff --git a/docs/usage/list.rst.inc b/docs/usage/list.rst.inc
index 456f42a66..2c1bdc14e 100644
--- a/docs/usage/list.rst.inc
+++ b/docs/usage/list.rst.inc
@@ -105,7 +105,7 @@ Description
 
 This command lists the contents of a repository or an archive.
 
-See the "borg help patterns" command for more help on exclude patterns.
+For more help on include/exclude patterns, see the :ref:`borg_patterns` command output.
 
 .. man NOTES