diff options
| author | Matthias Beyer <mail@beyermatthias.de> | 2017-08-25 21:27:41 +0200 |
|---|---|---|
| committer | Matthias Beyer <mail@beyermatthias.de> | 2017-08-25 21:27:41 +0200 |
| commit | 1c4b4c85e9798035936608b983983e53a0d7fb09 (patch) | |
| tree | fd8fe2ef4a1bd1169985dd6187f9001aae06259f | |
| parent | 0e6bfe7259ef29f2967d2d257957bd0109fd6e03 (diff) | |
| parent | d56b20b28d938775bdd9dd70c4a074fd985c67bb (diff) | |
Merge branch 'imag-0.3.0'
| -rw-r--r-- | config.toml | 2 | ||||
| -rw-r--r-- | content/blog/2017-08-25-imag-0-3-0.md | 218 | ||||
| -rw-r--r-- | content/doc.md | 14 | ||||
| -rw-r--r-- | content/doc/pdf/0.3.0.pdf | bin | 0 -> 203005 bytes | |||
| -rw-r--r-- | content/doc/web/0.3.0/index.html | 786 | ||||
| -rw-r--r-- | content/releases.md | 82 | ||||
| -rw-r--r-- | content/who.md | 20 |
7 files changed, 1112 insertions, 10 deletions
diff --git a/config.toml b/config.toml index 20a0694..8684219 100644 --- a/config.toml +++ b/config.toml @@ -58,7 +58,7 @@ url = "/mailinglist/" name = "Docs" weight = -70 identifier = "docs" -url = "https://matthiasbeyer.github.io/imag/imag_documentation/index.html" +url = "/doc/" [[menu.main]] name = "Code" diff --git a/content/blog/2017-08-25-imag-0-3-0.md b/content/blog/2017-08-25-imag-0-3-0.md new file mode 100644 index 0000000..9a2abd2 --- /dev/null +++ b/content/blog/2017-08-25-imag-0-3-0.md @@ -0,0 +1,218 @@ +--- +title: "imag 0.3.0" +slug: "imag-0-3-0" +date: "2017-08-25T21:22:06" +tags: [ "linux", " open source", " programming", " rust", " software", " tools", " imag" ] +--- + +I did it! Just hours ago I released imag in version 0.3.0. + +<!-- more --> + +**First of all (like with the 0.2.0 release): This is not production-ready +software. Use at your own +risk! This is a release for people to notice that some things work and one +could start to play around with it. Do not trust imag with your personal data +if you do not have backups. There _are_ bugs. This is _not_ perfect. This is +_alpha quality or pre-alpha quality software_!** + +<small> + imag is a personal information management suite for the commandline. + Its target audience are commandline- and power-users. + It does not reimplement personal information management (PIM) aspects, but + re-uses existing tools and standards to be an <i>addition</i> to an existing + workflow, so one does not have to learn a new tool before beeing productive + again. + Some simple PIM aspects are implemented as imag modules, though. + It gives the user the power to <i>connect</i> data from different existing + tools and <i>add meta-information</i> to these connections, so one can do + data-mining on PIM data. +</small> + +# How the release was done + +What did I do to get to the release? Well, not that much. I checked out the +`master` branch and created a `v0.3.0` branch on it, then started to fix +things up for the release. + +Then I manually published `libimagutil` and `libimagerror` because they do not +depend on other imag crates themselves. +After that I `cargo publish`ed all crates one by one until all crates were +published, fixing errors while doing so and when all crates were published I +tagged the release and pushed out my code to all remote repositories. + +The `.imag-documentation` crate was not published this time, I even plan +removing this crate from the imag codebase as there's no purpose on keeping it +right now. + +I removed the `v0.3.0` branch and created an annotated tag for the release. + +# Changelog / What's in there + +In this release we have merged ___ merges, from pull request number 821 to +1010, some still pending though. The release itself is clearly too big and I +really want to make the next release smaller and maybe even within this year. +But I won't commit to that to hard, because we all know how little time I will +have compared to what amount of time I _want_ to invest into the project. + +So, a full list of merges goes here. The pull request number is linked to the +github issue, while the hash is linked to the commit in our [git web +interface](http://git.imag-pim.org/imag/). + +What we have by now: + +* imag - The imag binary itself. The purpose of this is to be able to call + `imag view` instead of `imag-view`. +* imag-bookmark - A tool to keep and organize your bookmarks +* imag-counter - A simple counter tool. Create, delete, increment and + decrement counters +* imag-diary - A diary +* imag-link - Link imag entries with other imag entries +* imag-mail - This is not implemented at all and shouldn't even be usable + right now. +* imag-notes - Create notes +* imag-ref - Reference files on your Filesystem with imag +* imag-store - Core plumbing tool. Do not use at all if you do not know what + this is for. +* imag-tag - Tag imag entries +* imag-todo - Todo management with imag, using taskwarrior as backend. This is + _not_ a replacement for taskwarrior, but can be used to create entries in + the imag store for each taskwarrior task and then these entries can be + linked with `imag-link`. +* imag-view - View entries from the imag store + +Clearly this is not everything. We have a huge number of libraries which work +under the hood of these executables, but users might not care about this. + +# What had to be fixed for the release + +Some changes had to be done before the release could be made, unfortunately. +This includes the `path`-dependency specifications to be rewritten, but +unfortunately also more. + +The path dependencies had to be rewritten because we develop the imag codebase +with each crate depending on other crates in the repository (besides external +crates). And because we do not want to release imag crates all the time, we do +this by depending on a path: + +```toml +[dependencies.libimagstore] +path = "../libimagstore" +``` + +for example. Because we want to depend on actual "crates.io.crates" when + publishing, we had to rewrite all path specifications to: + +```toml +libimagstore = "0.3.0" +``` + +(for example). + +Because we are then only possible to publish crates which have no dependencies +or dependencies which are already on crates.io, we had to publish the imag +crates in a specific order: + +``` +./libimagutil +./libimagerror +./libimagstore +./libimagentrylink +./libimagentrylist +./libimagentrymarkdown +./libimagentrytag +./libimagrt +./libimagentrycategory +./libimagentrydatetime +./libimagentryedit +./libimagentryfilter +./libimagentrytimetrack +./libimagentryview +./libimagbookmark +./libimagcounter +./libimagdiary +./libimaginteraction +./libimagnotes +./libimagref +./libimagtimeui +./libimagtodo +./libimagannotation +./libimagmail +./imag-bookmark +./imag-counter +./imag-diary +./imag-link +./imag-notes +./imag-ref +./imag-store +./imag-tag +./imag-timetrack +./imag-view +./imag-mail +./imag-todo +./bin (the imag binary) +``` + +And because some of these could not be published right away (see the order, +first I published the "core" crates which do not depend on any other crate in +the list, then the store, then the `libimagentry*` crates, after that the +`libimag*` crates and the binaries as the last step, finishing with the `imag` +binary itself). + +Besides the path dependency specification rewrite commit, we have a bunch of +other commits in the branch from `master..v0.3.0`: + +http://git.imag-pim.org/imag/commit/?id= + +* [71ac310a1ffe88357c3a7e7fa6bd16231fd34275](http://git.imag-pim.org/imag/commit/?id=71ac310a1ffe88357c3a7e7fa6bd16231fd34275) + fixed the dependencies which were in the wrong section of the + `libimagentrycategory` crate +* [7dfa6de55d745507f536ce60ca468b0139486992](http://git.imag-pim.org/imag/commit/?id=7dfa6de55d745507f536ce60ca468b0139486992) + fixed the dependencies which were in the wrong section of the + `libimagentrydatetime` crate +* [0134ef91a8f0eecc94858bc974f87720999a1f7b](http://git.imag-pim.org/imag/commit/?id=0134ef91a8f0eecc94858bc974f87720999a1f7b) + added missing data in the `Cargo.toml` file of the `libimagannotation` crate +* [f8a6dd18d5779144bbe6d1c8dbbc257f3aa3c2ec](http://git.imag-pim.org/imag/commit/?id=f8a6dd18d5779144bbe6d1c8dbbc257f3aa3c2ec) + added missing data in the `Cargo.toml` file of the `imag-mail` crate +* [3dd4e632dcf119a18ba0d64748204c7b62019cbb](http://git.imag-pim.org/imag/commit/?id=3dd4e632dcf119a18ba0d64748204c7b62019cbb) + fixed a dependency which was accidentially removed in one of the prior + commits in the `Cargo.toml` file of the `imag-todo` crate +* [a2046bf178931dfd43924c42713e2841e89c571d](http://git.imag-pim.org/imag/commit/?id=a2046bf178931dfd43924c42713e2841e89c571d) + tried to fix the build script for the `imag` crate (in the `./bin/` + directory), which includes `src/ui.rs` files from other crates to generate + commandline completion scripts with the `clap` crate - which is a hack +* [4b17701c19f1a77407d6eae32e0e0a7593f86fb8](http://git.imag-pim.org/imag/commit/?id=4b17701c19f1a77407d6eae32e0e0a7593f86fb8) + removed the build.rs script of the `imag` crate because a fix was not + possible at this moment. + +The last commit is also the one that holds the `v0.3.0` tag and marks the +release of the imag core distribution therefore. + +# What's coming + +Well, what's coming for the next release? The +[0.4.0 milestone](https://github.com/matthiasbeyer/imag/milestone/4) +already exists and there are a lot of things already registered in it. + +When will I be there? The next version will be there if its ready. +Honestly I don't know. We have a lot of things to tackle and this is growing +to be a _really_ huge project, especially for a one-man-show, which it almost +is, still. + +# Can I use imag libraries in my project? + +No. + +Well, you can, but I do not guarantee any stability or API consistency or +something. +But if you want, you clearly can use one of the imag libraries +(they are on crates.io, so why not). +Just keep in mind that interfaces _will_ change and things _will_ break and +bugs _will_ happen. +We are 0.3.0 here, not 1.x.y. +Keep that in mind. + +That being said... feel free to play around by installing the imag crates +(and having backups) and maybe you will be in the `git log` of the next +release? + diff --git a/content/doc.md b/content/doc.md new file mode 100644 index 0000000..a7630e2 --- /dev/null +++ b/content/doc.md @@ -0,0 +1,14 @@ +--- +title: imag - User and developer documentation +--- + +The imag documentation is only published for releases, whereas the +v0.3.0 +is the latest release for which documentation is published. + +The source-documentation is not included (yet). + +* v0.3.0 + * [pdf](/doc/pdf/0.3.0.pdf) + * [html](/doc/web/0.3.0/) + diff --git a/content/doc/pdf/0.3.0.pdf b/content/doc/pdf/0.3.0.pdf Binary files differnew file mode 100644 index 0000000..dc75477 --- /dev/null +++ b/content/doc/pdf/0.3.0.pdf diff --git a/content/doc/web/0.3.0/index.html b/content/doc/web/0.3.0/index.html new file mode 100644 index 0000000..29cd380 --- /dev/null +++ b/content/doc/web/0.3.0/index.html @@ -0,0 +1,786 @@ +<!DOCTYPE html> +<html lang="de"> +<head> + <meta charset="utf-8"> + <meta name="generator" content="pandoc"> + <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes"> + + <meta name="dcterms.date" content="August 2017"> + + <title>imag User Documentation</title> + <style type="text/css">code{white-space: pre;}</style> + <!--[if lt IE 9]> + <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script> + <![endif]--> + + <style type="text/css"> + div.sourceCode { overflow-x: auto; } + table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode { + margin: 0; padding: 0; vertical-align: baseline; border: none; } + table.sourceCode { width: 100%; line-height: 100%; } + td.lineNumbers { text-align: right; padding-right: 4px; padding-left: 4px; color: #aaaaaa; border-right: 1px solid #aaaaaa; } + td.sourceCode { padding-left: 5px; } + code > span.kw { color: #007020; font-weight: bold; } /* Keyword */ + code > span.dt { color: #902000; } /* DataType */ + code > span.dv { color: #40a070; } /* DecVal */ + code > span.bn { color: #40a070; } /* BaseN */ + code > span.fl { color: #40a070; } /* Float */ + code > span.ch { color: #4070a0; } /* Char */ + code > span.st { color: #4070a0; } /* String */ + code > span.co { color: #60a0b0; font-style: italic; } /* Comment */ + code > span.ot { color: #007020; } /* Other */ + code > span.al { color: #ff0000; font-weight: bold; } /* Alert */ + code > span.fu { color: #06287e; } /* Function */ + code > span.er { color: #ff0000; font-weight: bold; } /* Error */ + code > span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */ + code > span.cn { color: #880000; } /* Constant */ + code > span.sc { color: #4070a0; } /* SpecialChar */ + code > span.vs { color: #4070a0; } /* VerbatimString */ + code > span.ss { color: #bb6688; } /* SpecialString */ + code > span.im { } /* Import */ + code > span.va { color: #19177c; } /* Variable */ + code > span.cf { color: #007020; font-weight: bold; } /* ControlFlow */ + code > span.op { color: #666666; } /* Operator */ + code > span.bu { } /* BuiltIn */ + code > span.ex { } /* Extension */ + code > span.pp { color: #bc7a00; } /* Preprocessor */ + code > span.at { color: #7d9029; } /* Attribute */ + code > span.do { color: #ba2121; font-style: italic; } /* Documentation */ + code > span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */ + code > span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */ + code > span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */ + </style> + + <link rel="stylesheet" href="/home/m/archive/development/rust/imag/doc/templates/css/table.css"> + +</head> + <body style="margin-left: 20%; margin-right: 20%;"> + <header> + <h1 class="title">imag User Documentation</h1> + <small class="date">August 2017</small> + + <small class="date">Version: 0.3.0</small> + </header> + <hr /> + + <h1 id="sec:introduction">Introduction</h1> + <p>This document is the user documentation for imag, the personal information management suite for the commandline. Besides beeing a documentation, it servey also as "roadmap" where this project should go. Parts which are not yet implemented might be documented already, therefore. A list on what is implemented and what is not can be found at the end of this document.</p> + <p>If you have any objections, suggestions for improvements, bugs, etc, please file them. A way to reach out to the imag project maintainer(s) is described in the CONTRIBUTING file of the repository or in this document, on the appropriate section.</p> + <h2 id="sec:intro:problem">The Problem</h2> + <p>The problem imag wants to solve is rather simple. When the project was initiated, there was no PIM-Suite available which</p> + <ul> + <li>was for this domain of users ("power-users", "commandline users")</li> + <li>uses plain text as storage format</li> + <li>was scriptable</li> + <li>was modular</li> + <li>contained functionality to link content</li> + </ul> + <p>The latter point is the bigger one: "imag" wants to offer the ability for users to link content. This means not only that a contact may be linked to a date, but that <em>all things</em> can be linked together. For example that a wiki article can be linked to a date which is linked to a todo which is linked to a note which is linked to a contact.</p> + <h2 id="sec:intro:approach">The Approach</h2> + <p>The approach "imag" takes on solving this problem is to store content in a "store" and persisting content in a unified way. Meta-Information is attached to the content which can be used to store structured data. This can be used to implement a variety of "domain modules" using the store. While content is stored in <em>one</em> place, imag does not duplicate content. imag does not copy or move icalendar files, emails, vcard files, music or movies to the store, but indexes them and stores the meta-information in the store.</p> + <p>Detailed explanation on this approach follows in the chapters of this work.</p> + <h2 id="sec:intro:implementation">Implementation</h2> + <p>The program is written in the Rust programming language.</p> + <p>The program consists of libraries which can be re-used by other projects to implement and adapt imag functionality. An external program may use a library of the imag distribution to store content in the store of imag and make it visible to imag this way.</p> + <p>This is a technical detail a user does not necessarily need to know, but as imag is intended for power-users anyways, we could say it fits here.</p> + <h1 id="architecture-of-the-imag-code">Architecture of the imag code</h1> + <p>The imag codebase has a rather simple overall architecture. But before introducing the reader to it, a few things have to be introduced.</p> + <h2 id="crate-types">Crate types</h2> + <p>There are different types of crates in the imag world. A crate is a rust project.</p> + <p>First of all, there are core crates. These crates provide the very core of imag and almost all other crates use them:</p> + <ul> + <li>libimagstore - The imag store is the abstraction overbthe filesystem. It provides primitives to get, write and manipulate store entries and their header information.</li> + <li>libimagrt - The runtime library, which provides functionality to create a store object from libimagstore, helps with configurarion loading and commandline argument handling (through the external "clap" crate).</li> + <li>libimagerror - Error handling library for handling errors the imag way. Used in all other crates, even the store itself. It also offers functionality to log and trace errors as well as exiting the application, if necessary.</li> + <li>libimagutil - Utilities.</li> + </ul> + <p>The next type of imag crates are entry extension libraries. Those provide extensional functionality for the types from libimagstore. For example, there is "libimagentrylink" which provides functionality to link two entries in the store.</p> + <p>The next kind of crate is the one that offers end-user functionality for a imag aspect, for example "libimagtodo" provides functionality to track todos.</p> + <p>And last, but not least, the commandline frontend crates provide the user interface. These are the kind of crates that are not library crates, but binaries.</p> + <h2 id="architecture-of-an-imag-module">Architecture of an imag module</h2> + <p>With the things from above, a module could have the following architecture:</p> + <pre><code>+---------------------------------------------+ + | imag-foo | + +-----------------------------------+---------+ + | libimagfoo | | + +-----------------+-----------------+ | + | | | | + | libimagentrybar | libimagentrybaz | | + | | | lib | + +-----------------+-----------------+ | + | | | + | ... | | + | | imag | + +-----------------------------------+ | + | | | + | libimagrt | | + | | error | + +-----------------------------------+ | + | | | + | libimagstore | | + | | | + +-----------------------------------+---------+</code></pre> + <p>The foundation of all imag modules is the store, as one can see from the image. Above this there is the libimagrt, which provides the basic runtime and access to the <code>Store</code> object. Cross-cutting, there is the error library (and possibly the util library, but we do not care about this one here), which is used through all levels. The highest level of all imag modules is the commandline interface on top of the domain library. In between can be any number of entry extension libraries, or none if not needed.</p> + <h2 id="types">Types</h2> + <p>The imag core, hence the libimagstore, libimagrt and libimagerror, provide a set of types that a user (as in a library writer) should be aware of.</p> + <p>First of all, there is the Runtime type which is provided by the libimagrt. It provides basic access to whether debugging or verbosity is enabled as well as the most important core object: The <code>Store</code>.</p> + <p>It is provided by the libimagstore library, the heart of everything.</p> + <p>When interacting with the store, two types are visible: <code>FileLockEntry</code> and <code>Entry</code> whereas the former derefs to the latter, which basically means that the former wraps the latter. The <code>FileLockEntry</code> is a necessary wrapper for ensuring that when working concurrently with the store, an entry is only <em>borrowed</em> once from the store. It also ensures that the object is alive as long as the store is.</p> + <p>The <code>Entry</code> type provides functionality like reading the actual content, its header and so on. Extensions for its functionality are implemented on this type, not on the <code>FileLockEntry</code>.</p> + <p>The <code>Entry</code> provides access to its header, which is a <code>toml::Value</code>, where toml is the toml-rs crate (external project). Conveniance functionality is provided via the <code>toml-query</code> crate, which is an external project which was initiated and extracted from the imag project.</p> + <p>Error types are also important. All errors in the imag prijects are generated by libimagerror usage and are interoperable. User code hardly handles the actual <code>Error</code> type but its inner one, <code>ErrorKind</code>, which is an enum where each member can be mapped to a representing text. Imag error types do never contain payload (they can via extensions, though it is not really practical and there is no usage for it).</p> + <h1 id="sec:thestore">The Store</h1> + <p>The store is where all the good things happen. The store is basically just a directory on the filesystem imag manages and keeps its state in.</p> + <p>One could say that the store is simply a databases, and it really is. We opted to go for plain text, though, as we believe that plain text is the only sane way to do such a thing. A user should always be able to read her data without great effort and putting everything in a <em>real</em> database like sqlite or even postgresql would need a user to install additional software just to read his own data. We don't want that. Text is readable until the worlds end and we think it is therefor better to store the data in plain text.</p> + <p>The following sections describe the store and the file format we use to store data. One may skip the following sections, they are included for users who want to dig into the store with their editors.</p> + <h2 id="sec:thestore:fileformat">File Format</h2> + <p>The contents of the store are encoded in either UTF-8 or ASCII. Either way, a normal text editor (like <code>vim</code> or the other one) will always be sufficient to dog into the store and modify files. For simple viewing even a pager (like <code>less</code>) is sufficient.</p> + <p>Each entry in the store consists of two parts:</p> + <ol style="list-style-type: decimal"> + <li>Header</li> + <li>Content</li> + </ol> + <p>The following section describe their purpose.</p> + <h3 id="sec:thestore:fileformat:header">Header Format</h3> + <p>The header format is where imag stores its data. The header is an area at the top of every file which is seperated from the content part by three dashes (<code>---</code>). Between these three dashes there is structured data. imag uses <code>TOML</code> as data format for this structured data, because it fits best and the available <code>TOML</code> parser for the rust programming language is really good.</p> + <p>The header can contain any amount of data, but modules (see Section 5) are restricted in their way of altering the data.</p> + <p>So normally there are several sections in the header. One section (<code>[imag]</code>) is always present. It contains a <code>version</code> field, which tells imag which version this file was created with (the version information is <em>also</em> encoded in the filename, just in case things change in the future). It also contains a <code>links</code> field which is an Array of values. This <code>links</code> field is for linking (see Section 3.6) to other entries in the store.</p> + <p>Other sections are named like the modules which created them. Every module is allowed to store arbitrary data under its own section and a module may never read other sections than its own. This is not enforced by imag itself, though.</p> + <h3 id="sec:thestore:fileformat:content">Content Format</h3> + <p>The content is the part of the file where the user is free to enter any textual content. The content may be rendered as Markdown or other markup format for the users convenience. The store does never expect and specific markup and actually the markup implementation is not inside the very code of imag.</p> + <p>Technically it would be possible that the content part of a file is used to store binary data. We don't want this, though.</p> + <h3 id="sec:thestore:fileformat:example">Example</h3> + <p>An example for a file in the store follows.</p> + <pre class="text"><code> + --- + [imag] + links = ["/home/user/more_kittens.mpeg"] + version = "0.3.0" + + [note] + name = "foo" + --- + + This is an example text, written by the user. + </code></pre> + <h2 id="sec:thestore:fileorganization">File organization</h2> + <p>The "Entries" are stored as files in the "Store", which is a directory the user has access to. The store may exist in the users Home-directory or any other directory the user has read-write-Access to.</p> + <p>Each module stores its data in an own subdirectory in the store. This is because we like to keep things ordered and clean, not because it is technically necessary.</p> + <p>We name the path to a file in the store "Store id" or "Storepath" and we often refer to it by using the store location as root. So if the store exists in <code>/home/user/store/</code>, a file with the storepath <code>/example.file</code> is (on the filesystem) located at <code>/home/user/store/example.file</code>.</p> + <p>A storepath contains predefined parts:</p> + <ul> + <li>The module name of the Module the Entry belongs to, as said above. This part is always a directory.</li> + <li>The version (semantic versioning applies) of the module storing the entry. This part is a postfix to the filename.</li> + </ul> + <p>The pattern for the storepath is</p> + <pre><code>/<module name>/<optional sub-folders>/<file name>~<sem version></code></pre> + <p>So if a module named "example-module" with version "0.1.0" stores a file in the Store, the storepath for a file with the name "example" is "/example-module/example~0.1.0".</p> + <p>Any number of subdirectories may be used, so creating folder hierarchies is possible and valid. A file "example" for a module "module" in version "0.1.0" would be stored in sub-folders like this:</p> + <pre><code>/module/some/sub/folder/example~0.1.0</code></pre> + <p>For example, it is valid if these files exist at the same time:</p> + <ul> + <li>/foo/bar~0.2</li> + <li>/foo/bar~1.3</li> + </ul> + <p>It might not be sane, though.</p> + <p>To future-proof the system it is necessary to provide a way for modules to differentiate in their versions on the store level. Thus if a module wants to retrieve a file from the store it must at least accept files from it's current advertised version. It may accept older files and it may transform them and resubmit them in the newer version.</p> + <h2 id="sec:thestore:links">Store path links</h2> + <p>Linking entries is version independent.</p> + <p>This means if an entry "a" from a module "A" gets written to the store, it may link to an entry "b" from a module "B", which is in version "0.1.0" at the moment. If the module "B" gets updated, it might update its entries in the store as well. The link from the "a" should never get invalid in this case, though it is not ensured by the core of imag itself.</p> + <h2 id="sec:thestore:backends">Backends</h2> + <p>The store itself also has a backend. This backend is the "filesystem abstraction" code.</p> + <p>Note: This is a very core thing. Casual users might want to skip this section.</p> + <h3 id="sec:thestore:backends:problem">Problem</h3> + <p>First, we had a compiletime backend for the store. This means that the actual filesystem operations were compiled into the stores either as real filesystem operations (in a normal debug or release build) but as a in-memory variant in the 'test' case. So tests did not hit the filesystem when running. This gave us us the possibility to run tests concurrently with multiple stores that did not interfere with eachother.</p> + <p>This approach worked perfectly well until we started to test not the store itself but crates that depend on the store implementation. When running tests in a crate that depends on the store, the store itself was compiled with the filesystem-hitting-backend. This was problematic, as tests could not be implemented without hitting the filesystem.</p> + <p>Hence we implemented this.</p> + <h3 id="sec:thestore:backends:implementation">Implementation</h3> + <p>The filesystem is abstracted via a trait <code>FileAbstraction</code> which contains the essential functions for working with the filesystem.</p> + <p>Two implementations are provided in the code:</p> + <ul> + <li>FSFileAbstraction</li> + <li>InMemoryFileAbstraction</li> + </ul> + <p>whereas the first actually works with the filesystem and the latter works with an in-memory HashMap that is used as filesystem.</p> + <p>Further, the trait <code>FileAbstractionInstance</code> was introduced for functions which are executed on actual instances of content from the filesystem, which was previousely tied into the general abstraction mechanism.</p> + <p>So, the <code>FileAbstraction</code> trait is for working with the filesystem, the <code>FileAbstractionInstance</code> trait is for working with instances of content from the filesystem (speak: actual Files).</p> + <p>In case of the <code>FSFileAbstractionInstance</code>, which is the implementation of the <code>FileAbstractionInstance</code> for the actual filesystem-hitting code, the underlying resource is managed like with the old code before. The <code>InMemoryFileAbstractionInstance</code> implementation is corrosponding to the <code>InMemoryFileAbstraction</code> implementation - for the in-memory "filesystem".</p> + <h2 id="sec:thestore:backends:stdio">The StdIo backend</h2> + <p>Sidenote: The name is "StdIo" because its main purpose is Stdin/Stdio, but it is abstracted over Read/Write actually, so it is also possible to use this backend in other ways, too.</p> + <h3 id="sec:thestore:backends:stdio:why">Why?</h3> + <p>This is a backend for the imag store which is created from stdin, by piping contents into the store (via JSON or TOML) and piping the store contents (as JSON or TOML) to stdout when the the backend is destructed.</p> + <p>This is one of some components which make command-chaining in imag possible. With this, the application does not have to know whether the store actually lives on the filesystem or just "in memory".</p> + <h3 id="sec:thestore:backends:stdio:mappers">Mappers</h3> + <p>The backend contains a "Mapper" which defines how the contents get mapped into the in-memory store representation: A JSON implementation or a TOML implementation are possible.</p> + <p>The following section assumes a JSON mapper.</p> + <p>The mapper reads the JSON, parses it (thanks serde!) and translates it to a <code>Entry</code>, which is the in-memory representation of the files. The <code>Entry</code> contains a <code>Header</code> part and a <code>Content</code> part.</p> + <p>This is then made available to the store codebase.</p> + <p>To summarize what we do right now, lets have a look at the awesome ascii-art below:</p> + <pre><code> libimag* + | + v + IO Mapper Store Mapper IO + +--+---------+----------------+--------+--+ + | | | | | | + JSON -> Entry -> JSON + + Header + + Content</code></pre> + <p>This is what gets translated where for one imag call with a stdio store backend.</p> + <h3 id="sec:thestore:backends:stdio:json">The JSON Mapper</h3> + <p>The JSON mapper maps JSON which is read from a source into a HashMap which represents the in-memory filesystem.</p> + <p>The strucure is as follows:</p> + <div class="sourceCode"><pre class="sourceCode json"><code class="sourceCode json"><span class="fu">{</span> + <span class="dt">"version"</span><span class="fu">:</span> <span class="st">"0.3.0"</span><span class="fu">,</span> + <span class="dt">"store"</span><span class="fu">:</span> <span class="fu">{</span> + <span class="dt">"example"</span><span class="fu">:</span> <span class="fu">{</span> + <span class="dt">"header"</span><span class="fu">:</span> <span class="fu">{</span> + <span class="dt">"imag"</span><span class="fu">:</span> <span class="fu">{</span> + <span class="dt">"version"</span><span class="fu">:</span> <span class="st">"0.3.0"</span><span class="fu">,</span> + <span class="fu">},</span> + <span class="fu">},</span> + <span class="dt">"content"</span><span class="fu">:</span> <span class="st">"hi there!"</span><span class="fu">,</span> + <span class="fu">},</span> + <span class="fu">},</span> + <span class="fu">}</span></code></pre></div> + <h3 id="sec:thestore:backends:todo">TODO</h3> + <p>If you look at the version history of this file you will see that this implementation has grown from something complex and probably slow to what we have today.</p> + <p>Still, there's one improvement we could make: abstract all the things away so the <code>libimag*</code> crates handle the header without knowing whether it is JSON or TOML. With this, we would not even have to translate JSON to TOML anymore. We should measure whether this would have actually any performance impact before implementing it.</p> + <h2 id="sec:thestore:linking">Linking from an store entry</h2> + <p>As described in Section 1.1 the purpose of imag is to <em>link</em> content together. The following section describes, from a technical view, how this is done in imag.</p> + <p>There are two ways of linking in imag. You can either link internally or externally. The following sections describe the differences.</p> + <h3 id="sec:thestore:linking:internal">Linking to internal content</h3> + <p>Internal links are links between store entries themselfes. This means that one store entry can link to another. Actually, links are not pointers but rather tries between entries, meaning that an link is not directed, but always a two-way pointer.</p> + <p>How linking works from the user interface is described in Section 5.17.</p> + <h3 id="sec:thestore:linking:external">Linking to external content</h3> + <p>Linking to external content means linking to files or directories which do not live inside the store itself but outside of it.</p> + <p>Each store entry can store <em>one link to external content at most</em>.</p> + <p>External linking should not be used from the user interface but rather the <code>ref</code> feature (Section 5.28) should be used. Section 5.28 describes why that is.</p> + <h1 id="conventions-best-practices">Conventions, best practices</h1> + <p>This section goes about best practices in the imag codebase. It is mainly focused on developers, but a user may read it for getting to know how imag works.</p> + <p>Lets work our way up from the store and how to extend it to the commandline user interface.</p> + <h2 id="store-and-entry-functionality">Store and Entry functionality</h2> + <p>A <code>Entry</code> does not offer much functionality by itself. So its the job of libraries to <em>extend</em> their functionality. This should never be done by wrapping the <code>Entry</code> type itself but by providing and implementing an extension trait on it.</p> + <p>Same goes for extending the <code>Store</code> type: never wrap it, always provide an extension trait for it.</p> + <p>These two rules ensure that the type does not lose any functionality from a wrapping. <code>Deref</code> could do that, but not over muliple levels, so extension traits it is. It also most likely results in functions inside the extension trait which all return a <code>Result<_, _></code>.</p> + <h2 id="libraries">Libraries</h2> + <p>In the next few sections, conventions and best practices for writing a imag library are written down.</p> + <p>A developer of imag should read this carefully, a user may skip this section or cross-read it for better understanding of the imag project.</p> + <h3 id="library-naming">Library naming</h3> + <p>Libraries which provide functionality for entries or the store (most likely entries or both) should be named "libimagentrything" whereas "thing" stands for what the library provides.</p> + <p>All other libraries should be prefixed with "libimag" at least. Most likely, one will not write such a library but rather a "libimagentrything" library.</p> + <h3 id="library-scope">Library scope</h3> + <p>A library should never introduce utility functionality which could be useful for other libraries as well. If there is no such functionality available, the "libimagutil" might be a place where such a function would be put, or, if not yet available, a "libimagentryutil" could be created.</p> + <p>If a library has to introduce free functions in its public interface, one should think hard whether this is really necessary.</p> + <h3 id="library-error-typeskinds">Library error types/kinds</h3> + <p>Libraries must use the "libimagerror" tools to create error types and kinds. Most likely, a library needs some kinds for wrapping the errors from underlying libraries, such as the store itself.</p> + <p>A library must <em>never</em> introduce multiple error types, but is free to introduce as many error kinds as required. Indeed, more kinds is better than fewer.</p> + <h3 id="libraries-with-commandline-frontends">Libraries with commandline frontends</h3> + <p>Libraries with commandline frontends provide end-user functionality. Normally, they depend on one or more "libimagentrything" libraries. They should be named "libimagthing", though. For example: "libimagdiary", "libimagtimetrack" or "libimagwiki", whereas the commandline frontend |