path: root/docs/usage/help.rst.inc

.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit!

.. _borg_patterns:

borg help patterns
~~~~~~~~~~~~~~~~~~


The path/filenames used as input for the pattern matching start from the
currently active recursion root. You usually give the recursion root(s)
when invoking borg and these can be either relative or absolute paths.

So, when you give `relative/` as root, the paths going into the matcher
will look like `relative/.../file.ext`. When you give `/absolute/` as
root, they will look like `/absolute/.../file.ext`.

File paths in Borg archives are always stored normalized and relative.
This means that e.g. ``borg create /path/to/repo ../some/path`` will
store all files as `some/path/.../file.ext` and ``borg create
/path/to/repo /home/user`` will store all files as
`home/user/.../file.ext`.

File patterns support these styles: fnmatch, shell, regular expressions,
path prefixes and path full-matches. By default, fnmatch is used for
``--exclude`` patterns and shell-style is used for the experimental
``--pattern`` option.

Starting with Borg 1.2, for all but regular expression pattern matching
styles, all paths are treated as relative, meaning that a leading path
separator is removed after normalizing and before matching. This allows
you to use absolute or relative patterns arbitrarily.

If followed by a colon (':') the first two characters of a pattern are
used as a style selector. Explicit style selection is necessary when a
non-default style is desired or when the desired pattern starts with
two alphanumeric characters followed by a colon (i.e. `aa:something/*`).

`Fnmatch <https://docs.python.org/3/library/fnmatch.html>`_, selector `fm:`
    This is the default style for ``--exclude`` and ``--exclude-from``.
    These patterns use a variant of shell pattern syntax, with '\*' matching
    any number of characters, '?' matching any single character, '[...]'
    matching any single character specified, including ranges, and '[!...]'
    matching any character not specified. For the purpose of these patterns,
    the path separator (backslash for Windows and '/' on other systems) is not
    treated specially. Wrap meta-characters in brackets for a literal
    match (i.e. `[?]` to match the literal character `?`). For a path
    to match a pattern, the full path must match, or it must match
    from the start of the full path to just before a path separator. Except
    for the root path, paths will never end in the path separator when
    matching is attempted.  Thus, if a given pattern ends in a path
    separator, a '\*' is appended before matching is attempted. A leading
    path separator is always removed.

Shell-style patterns, selector `sh:`
    This is the default style for ``--pattern`` and ``--patterns-from``.
    Like fnmatch patterns these are similar to shell patterns. The difference
    is that the pattern may include `**/` for matching zero or more directory
    levels, `*` for matching zero or more arbitrary characters with the
    exception of any path separator. A leading path separator is always removed.

Regular expressions, selector `re:`
    Regular expressions similar to those found in Perl are supported. Unlike
    shell patterns regular expressions are not required to match the full
    path and any substring match is sufficient. It is strongly recommended to
    anchor patterns to the start ('^'), to the end ('$') or both. Path
    separators (backslash for Windows and '/' on other systems) in paths are
    always normalized to a forward slash ('/') before applying a pattern. The
    regular expression syntax is described in the `Python documentation for
    the re module <https://docs.python.org/3/library/re.html>`_.

Path prefix, selector `pp:`
    This pattern style is useful to match whole sub-directories. The pattern
    `pp:root/somedir` matches `root/somedir` and everything therein. A leading
    path separator is always removed.

Path full-match, selector `pf:`
    This pattern style is (only) useful to match full paths.
    This is kind of a pseudo pattern as it can not have any variable or
    unspecified parts - the full path must be given. `pf:root/file.ext` matches
    `root/file.ext` only. A leading path separator is always removed.

    Implementation note: this is implemented via very time-efficient O(1)
    hashtable lookups (this means you can have huge amounts of such patterns
    without impacting performance much).
    Due to that, this kind of pattern does not respect any context or order.
    If you use such a pattern to include a file, it will always be included
    (if the directory recursion encounters it).
    Other include/exclude patterns that would normally match will be ignored.
    Same logic applies for exclude.

.. note::

    `re:`, `sh:` and `fm:` patterns are all implemented on top of the Python SRE
    engine. It is very easy to formulate patterns for each of these types which
    requires an inordinate amount of time to match paths. If untrusted users
    are able to supply patterns, ensure they cannot supply `re:` patterns.
    Further, ensure that `sh:` and `fm:` patterns only contain a handful of
    wildcards at most.

Exclusions can be passed via the command line option ``--exclude``. When used
from within a shell the patterns should be quoted to protect them from
expansion.

The ``--exclude-from`` option permits loading exclusion patterns from a text
file with one pattern per line. Lines empty or starting with the number sign
('#') after removing whitespace on both ends are ignored. The optional style
selector prefix is also supported for patterns loaded from a file. Due to
whitespace removal paths with whitespace at the beginning or end can only be
excluded using regular expressions.

To test your exclusion patterns without performing an actual backup you can
run ``borg create --list --dry-run ...``.

Examples::

    # Exclude '/home/user/file.o' but not '/home/user/file.odt':
    $ borg create -e '*.o' backup /

    # Exclude '/home/user/</