1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
|
<img align="left" width="183" height="154" src="logo.svg" />
# up - the Ultimate Plumber
**up** is the **Ultimate Plumber**, a tool for writing Linux pipes in a
terminal-based UI interactively, with instant live preview of command results.
The main **goal** of the Ultimate Plumber is to help **interactively and
incrementally explore textual data** in Linux, by making it easier to quickly
build complex pipelines, thanks to a **fast feedback loop**. This is achieved
by boosting any typical **Linux text-processing utils** such as `grep`, `sort`,
`cut`, `paste`, `awk`, `wc`, `perl`, etc., etc., by providing a quick,
**interactive, scrollable preview** of their results.
[](https://asciinema.org/a/208538)
## Usage
**[Download *up* for Linux](https://github.com/akavel/up/releases/latest/download/up)**
| [ArchLinux](https://wiki.archlinux.org/index.php/Arch_User_Repository): [`aur/up`](https://aur.archlinux.org/packages/up/)
| FreeBSD: [`pkg install up`](https://www.freshports.org/textproc/up)
| macOS: [`brew install up`](https://formulae.brew.sh/formula/up)
| [Other OSes](https://github.com/akavel/up/releases)
To start using **up**, redirect any text-emitting command (or pipeline) into it
— for example:
$ lshw |& ./up
then:
- use ***PgUp/PgDn*** and ***Ctrl-[←]/Ctrl-[→]*** for basic browsing through
the command output;
- in the input box at the top of the screen, start **writing any bash
pipeline**; then **press Enter to execute the command you typed**,
and the Ultimate Plumber will immediately show you the output of
the pipeline in the **scrollable window** below (replacing any
earlier contents)
- For example, you can try writing:
`grep network -A2 | grep : | cut -d: -f2- | paste - -`
— on my computer, after pressing *Enter*, the screen then shows
the pipeline and a scrollable preview of its output like below:
| grep network -A2 | grep : | cut -d: -f2- | paste - -
Wireless interface Centrino Advanced-N 6235
Ethernet interface RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
- **WARNING: Please be careful when using it! It could be dangerous.**
In particular, writing "rm" or "dd" into it could be like running around
with a chainsaw. But you'd be careful writing "rm" anywhere in Linux
anyway, no?
- when you are satisfied with the result, you can **press *Ctrl-X* to exit**
the Ultimate Plumber, and the command you built will be **written into
`up1.sh` file** in the current working directory (or, if it already existed,
`up2.sh`, etc., until 1000, based on [Shlemiel the Painter's
algorithm](https://www.joelonsoftware.com/2001/12/11/back-to-basics/)).
Alternatively, you can press ***Ctrl-C*** to quit without saving.
- If the command you piped into *up* is long-running (in such case you will see
a tilde `~` indicator character in the top-left corner of the screen, meaning
that *up* is still waiting for more input), you may need to press
***Ctrl-S*** to temporarily freeze *up*'s input buffer (a freeze will be
indicated by a `#` character in top-left corner), which will inject a fake
EOF into the pipeline; otherwise, some commands in the pipeline may not print
anything, waiting for full input (especially commands like `wc` or `sort`,
but `grep`, `perl`, etc. may also show incomplete results). To unfreeze back,
press ***Ctrl-Q***.
## Additional Notes
- The pipeline is passed verbatim to a `bash -c` command, so any bash-isms should work.
- The input buffer of the Ultimate Plumber is currently fixed at **40 MB**. If
you reach this limit, a `+` character should get displayed in the top-left
corner of the screen. (This is intended to be changed to a
dynamically/manually growable buffer in a future version of *up*.)
- **MacOSX support:** I don't have a Mac, thus I have no idea if it works on
one. You are welcome to try, and also to send PRs. If you're interested in
me providing some kind of official-like support for MacOSX, please consider
trying to find a way to send me some usable-enough Mac computer. Please note
I'm not trying to "take advantage" of you by this, as I'm actually not at all
interested in achieving a Mac otherwise. (Also, trying to commit to this kind
of support will be an extra burden and obligation on me. Knowing someone out
there cares enough to do a fancy physical gesture would really help alleviate
this.) If you're serious enough to consider this option, please contact me by
email (mailto:czapkofan@gmail.com) or keybase (https://keybase.io/akavel), so
that we could try to research possible ways to achieve this.
Thanks for understanding!
- **Prior art:** I was surprised no one seemed to write a similar tool before,
that I could find. It should have been possible to write this since the dawn
of Unix already, or earlier! And indeed, after I announced *up*, I got enough
publicity that my attention was directed to one such earlier project already:
**[Pipecut](http://pipecut.org/index.html)**. Looks interesting! You may like
to check it too! (Thanks [@TronDD](https://lobste.rs/s/acpz00/up_tool_for_writing_linux_pipes_with#c_qxrgoa).)
- **Other influences:** I don't remember the fact too well already, but I'm
rather sure that this must have been inspired in big part by The Bret Victor's Talk(s).
## Future Ideas
- I have quite a lot of ideas for further experimentation of development of
*up*, including but not limited to:
- [RIIR](https://rust-lang.org) (once I learn enough of Rust... at some
point in future... maybe...) — esp. to hopefully make *up* be a smaller
binary (and also to maybe finally learn some Rust); though I'm somewhat
afraid if it might ossify the codebase and make harder to develop
further..? ...but maybe actually converse?...
- Maybe it could be made into an UI-less, RPC/REST/socket/text-driven
service, like gocode or [Language Servers](https://langserver.org/), for
integration with editors/IDEs (emacs? vim? VSCode?...) I'd be especially
interested in eventually merging it into [Luna
Studio](https://luna-lang.org/); RIIR may help in this. (Before this, as
a simpler approach, multi-line editing may be needed, or at least
left&right scrolling of the command editor input box. Also, some kind of
jumping between words in the command line; readline's *Alt-b* & *Alt-f*?)
- Make it possible to [capture output of already running
processes](https://stackoverflow.com/a/19584979/98528)! (But maybe that
could be better made as a separate, composable tool! In Rust?)
- Adding tests... (ahem; see also
[#1](https://github.com/akavel/up/issues/1)) ...also write `--help`...
- Making it work on Windows,
somehow[?](https://github.com/mattn/go-shellwords) Also, obviously, would
be nice to have some CI infrastructure enabling porting it to MacOSX,
BSDs, etc., etc...
- Integration with [fzf](https://github.com/junegunn/fzf) and other TUI
tools? I only have some vague thoughts and ideas about it as of now, not
even sure how this could look like.
- Adding more previews, for each `|` in the pipeline; also forking of
pipelines, merging, feedback loops, and other mixing and matching (though
I'd strongly prefer if [Luna](https://luna-lang.org) was to do it
eventually).
- If you are interested in financing my R&D work, contact me by email at:
czapkofan@gmail.com, or [on keybase.io as akavel](https://keybase.io/akavel).
I suppose I will probably be developing the Ultimate Plumber further anyway,
but at this time it's purely a hobby project, with all the fun and risks this
entails.
— *Mateusz Czapliński*
*October 2018*
*PS. The UP logo was conceived and generously sponsored
by [Thoai Nguyen](https://github.com/thoaionline)
and [GPU Exchange](https://gpu.exchange/),
with a helping hand from [Many Pixels](https://www.manypixels.co/).*
|
