Move the documentation to the docs

1 files changed, 167 insertions, 0 deletions

diff --git a/doc/src/02000-store.md b/doc/src/02000-store.md
index eb4fa860..2f484758 100644
--- a/doc/src/02000-store.md
+++ b/doc/src/02000-store.md
@@ -146,3 +146,170 @@ moment. If the module "B" gets updated, it might update its entries in the store
 as well. The link from the "a" should never get invalid in this case, though it
 is not ensured by the core of imag itself.
 
+## Backends {#sec:thestore:backends}
+
+The store itself also has a backend. This backend is the "filesystem
+abstraction" code.
+
+Note: This is a very core thing. Casual users might want to skip this section.
+
+### Problem {#sec:thestore:backends:problem}
+
+First, we had a compiletime backend for the store.
+This means that the actual filesystem operations were compiled into the stores
+either as real filesystem operations (in a normal debug or release build) but as
+a in-memory variant in the 'test' case.
+So tests did not hit the filesystem when running.
+This gave us us the possibility to run tests concurrently with multiple stores
+that did not interfere with eachother.
+
+This approach worked perfectly well until we started to test not the
+store itself but crates that depend on the store implementation.
+When running tests in a crate that depends on the store, the store
+itself was compiled with the filesystem-hitting-backend.
+This was problematic, as tests could not be implemented without hitting
+the filesystem.
+
+Hence we implemented this.
+
+### Implementation {#sec:thestore:backends:implementation}
+
+The filesystem is abstracted via a trait `FileAbstraction` which
+contains the essential functions for working with the filesystem.
+
+Two implementations are provided in the code:
+
+* FSFileAbstraction
+* InMemoryFileAbstraction
+
+whereas the first actually works with the filesystem and the latter
+works with an in-memory HashMap that is used as filesystem.
+
+Further, the trait `FileAbstractionInstance` was introduced for
+functions which are executed on actual instances of content from the
+filesystem, which was previousely tied into the general abstraction
+mechanism.
+
+So, the `FileAbstraction` trait is for working with the filesystem, the
+`FileAbstractionInstance` trait is for working with instances of content
+from the filesystem (speak: actual Files).
+
+In case of the `FSFileAbstractionInstance`, which is the implementation
+of the `FileAbstractionInstance` for the actual filesystem-hitting code,
+the underlying resource is managed like with the old code before.
+The `InMemoryFileAbstractionInstance` implementation is corrosponding to
+the `InMemoryFileAbstraction` implementation - for the in-memory
+"filesystem".
+
+The implementation of the `get_file_content()` function had to be
+changed to return a `String` rather than a `&mut Read` because of
+lifetime issues.
+This change is store-internally and the API of the store itself was not
+affected.
+
+## The StdIo backend {#sec:thestore:backends:stdio}
+
+Sidenote: The name is "StdIo" because its main purpose is Stdin/Stdio, but it
+is abstracted over Read/Write actually, so it is also possible to use this
+backend in other ways, too.
+
+### Why? {#sec:thestore:backends:stdio:why}
+
+This is a backend for the imag store which is created
+from stdin, by piping contents into the store (via JSON or TOML) and piping the
+store contents (as JSON or TOML) to stdout when the the backend is destructed.
+
+This is one of some components which make command-chaining in imag possible.
+With this, the application does not have to know whether the store actually
+lives on the filesystem or just "in memory".
+
+### Mappers {#sec:thestore:backends:stdio:mappers}
+
+The backend contains a "Mapper" which defines how the contents get mapped into
+the in-memory store representation: A JSON implementation or a TOML
+implementation are possible.
+
+The following section assumes a JSON mapper.
+
+The backends themselves do not know "header" and "content" - they know only
+blobs which live in pathes.
+Indeed, this "backend" code does not serve "content" or "header" to the `Store`
+implementation, but only a blob of bytes.
+Anyways, the JSON-protocol for passing a store around _does_ know about content
+and header (see @sec:thestore:backends:stdio:json for the JSON format).
+
+So the mapper reads the JSON, parses it (thanks serde!) and translates it to
+TOML, because TOML is the Store representation of a header.
+But because the backend does not serve header and content, but only a blob,
+this TOML is then translated (with the content of the respective file) to a
+blob.
+
+This is then made available to the store codebase.
+This is complex and probably slow, we know.
+
+To summarize what we do right now, lets have a look at the awesome ascii-art
+below:
+
+```
+                            libimag*
+                               |
+                               v
+ IO     Mapper        FS     Store      FS         Mapper     IO
++--+-------------+---------+--------+---------+--------------+--+
+|  |             |         |        |         |              |  |
+    JSON -> TOML -> String -> Entry -> String -> TOML -> JSON
+                              + TOML
+                              + Content
+```
+
+This is what gets translated where for one imag call with a stdio store backend.
+
+The rationale behind this implementation is that this is the best implementation
+we can have in a relatively short amount of time.
+
+### The JSON Mapper {#sec:thestore:backends:stdio:json}
+
+The JSON mapper maps JSON which is read from a source into a HashMap which
+represents the in-memory filesystem.
+
+The strucure is as follows:
+
+```json
+{
+    "version": "0.3.0",
+    "store": {
+        "/example": {
+            "header": {
+                "imag": {
+                    "version": "0.3.0",
+                },
+            },
+            "content": "hi there!",
+        },
+    },
+}
+```
+
+### TODO {#sec:thestore:backends:todo}
+
+Of course, the above is not optimal.
+The TODO here is almost visible: Implement a proper backend where we do not need
+to translate between types all the time.
+
+The first goal would be to reduce the above figure to:
+
+```
+           libimag*
+              |
+              v
+ IO Mapper  Store    Mapper IO
++--+------+--------+-------+--+
+|  |      |        |       |  |
+    JSON -> Entry -> JSON
+            + TOML
+            + Content
+```
+
+and the second step would be to abstract all the things away so the `libimag*`
+crates handle the header without knowing whether it is JSON or TOML.
+