diff options
| author | Jesse Duffield <jessedduffield@gmail.com> | 2023-07-10 21:41:29 +1000 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-07-10 21:41:29 +1000 |
| commit | 2dddd906f8f8d14f9c76c304b8b57bcb10bf9ebe (patch) | |
| tree | 9f8e2061837ae982467fdf1137dc3bfde4a28268 | |
| parent | 585ea361f6703526038f50cddf4e1f050fe0c3d0 (diff) | |
| parent | 16ed3c23773d5e7b0573c5af805f6a8cb86512ac (diff) | |
Track busy/idle state for integration tests (#2765)
91 files changed, 1081 insertions, 402 deletions
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 05a5a0f73..3494e6a40 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -68,9 +68,8 @@ jobs: restore-keys: | ${{runner.os}}-go- - name: Test code - # LONG_WAIT_BEFORE_FAIL means that for a given test assertion, we'll wait longer before failing run: | - LONG_WAIT_BEFORE_FAIL=true go test pkg/integration/clients/*.go + go test pkg/integration/clients/*.go build: runs-on: ubuntu-latest env: diff --git a/docs/Integration_Tests.md b/docs/Integration_Tests.md deleted file mode 100644 index fab7bb984..000000000 --- a/docs/Integration_Tests.md +++ /dev/null @@ -1 +0,0 @@ -see new docs [here](https://github.com/jesseduffield/lazygit/blob/master/pkg/integration/README.md) diff --git a/docs/README.md b/docs/README.md index c0c8191d2..acce8338c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,3 +6,4 @@ * [Keybindings](./keybindings)
* [Undo/Redo](./Undoing.md)
* [Searching/Filtering](./Searching.md)
+* [Dev docs](./dev)
diff --git a/docs/dev/Busy.md b/docs/dev/Busy.md new file mode 100644 index 000000000..309f2d25d --- /dev/null +++ b/docs/dev/Busy.md @@ -0,0 +1,78 @@ +# Knowing when Lazygit is busy/idle + +## The use-case + +This topic deserves its own doc because there there are a few touch points for it. We have a use-case for knowing when Lazygit is idle or busy because integration tests follow the following process: +1) press a key +2) wait until Lazygit is idle +3) run assertion / press another key +4) repeat + +In the past the process was: +1) press a key +2) run assertion +3) if assertion fails, wait a bit and retry +4) repeat + +The old process was problematic because an assertion may give a false positive due to the contents of some view not yet having changed since the last key was pressed. + +## The solution + +First, it's important to distinguish three different types of goroutines: +* The UI goroutine, of which there is only one, which infinitely processes a queue of events +* Worker goroutines, which do some work and then typically enqueue an event in the UI goroutine to display the results +* Background goroutines, which periodically spawn worker goroutines (e.g. doing a git fetch every minute) + +The point of distinguishing worker goroutines from background goroutines is that when any worker goroutine is running, we consider Lazygit to be 'busy', whereas this is not the case with background goroutines. It would be pointless to have background goroutines be considered 'busy' because then Lazygit would be considered busy for the entire duration of the program! + +In gocui, the underlying package we use for managing the UI and events, we keep track of how many busy goroutines there are using the `Task` type. A task represents some work being done by lazygit. The gocui Gui struct holds a map of tasks and allows creating a new task (which adds it to the map), pausing/continuing a task, and marking a task as done (which removes it from the map). Lazygit is considered to be busy so long as there is at least one busy task in the map; otherwise it's considered idle. When Lazygit goes from busy to idle, it notifies the integration test. + +It's important that we play by the rules below to ensure that after the user does anything, all the processing that follows happens in a contiguous block of busy-ness with no gaps. + +### Spawning a worker goroutine + +Here's the basic implementation of `OnWorker` (using the same flow as `WaitGroup`s): + +```go +func (g *Gui) OnWorker(f func(*Task)) { + task := g.NewTask() + go func() { + f(task) + task.Done() + }() +} +``` + +The crucial thing here is that we create the task _before_ spawning the goroutine, because it means that we'll have at least one busy task in the map until the completion of the goroutine. If we created the task within the goroutine, the current function could exit and Lazygit would be considered idle before the goroutine starts, leading to our integration test prematurely progressing. + +You typically invoke this with `self.c.OnWorker(f)`. Note that the callback function receives the task. This allows the callback to pause/continue the task (see below). + +### Spawning a background goroutine + +Spawning a background goroutine is as simple as: + +```go +go utils.Safe(f) +``` + +Where `utils.Safe` is a helper function that ensures we clean up the gui if the goroutine panics. + +### Programmatically enqueing a UI event + +This is invoked with `self.c.OnUIThread(f)`. Internally, it creates a task before enqueuing the function as an event (including the task in the event struct) and once that event is processed by the event queue (and any other pending events are processed) the task is removed from the map by calling `task.Done()`. + +### Pressing a key + +If the user presses a key, an event will be enqueued automatically and a task will be created before (and `Done`'d after) the event is processed. + +## Special cases + +There are a couple of special cases where we manually pause/continue the task directly in the client code. These are subject to change but for the sake of completeness: + +### Writing to the main view(s) + +If the user focuses a file in the files panel, we run a `git diff` command for that file and write the output to the main view. But we only read enough of the command's output to fill the view's viewport: further loading only happens if the user scrolls. Given that we have a background goroutine for running the command and writing more output upon scrolling, we create our own task and call `Done` on it as soon as the viewport is filled. + +### Requesting credentials from a git command + +Some git commands (e.g. git push) may request credentials. This is the same deal as above; we use a worker goroutine and manually pause continue its task as we go from waiting on the git command to waiting on user input. This requires passing the task through to the `Push` method so that it can be paused/continued. diff --git a/docs/dev/Integration_Tests.md b/docs/dev/Integration_Tests.md new file mode 100644 index 000000000..df10c3f8f --- /dev/null +++ b/docs/dev/Integration_Tests.md @@ -0,0 +1 @@ +see new docs [here](../../pkg/integration/README.md) diff --git a/docs/dev/README.md b/docs/dev/README.md new file mode 100644 index 000000000..9b66032de --- /dev/null +++ b/docs/dev/README.md @@ -0,0 +1,4 @@ +# Dev Documentation Overview + +* [Busy/Idle tracking](./Busy.md). +* [Integration Tests](../../pkg/integration/README.md) @@ -18,7 +18,7 @@ require ( github.com/integrii/flaggy v1.4.0 github.com/jesseduffield/generics v0.0.0-20220320043834-727e535cbe68 github.com/jesseduffield/go-git/v5 v5.1.2-0.20221018185014-fdd53fef665d - github.com/jesseduffield/gocui v0.3.1-0.20230702054502-d6c452fc12ce + github.com/jesseduffield/gocui v0.3.1-0.20230710004407-9bbfd873713b github.com/jesseduffield/kill v0.0.0-20220618033138-bfbe04675d10 github.com/jesseduffield/lazycore v0.0.0-20221012050358-03d2e40243c5 github.com/jesseduffield/minimal/gitignore v0.3.3-0.20211018110810-9cde264e6b1e @@ -67,8 +67,8 @@ require ( golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 // indirect golang.org/x/exp v0.0.0-20220318154914-8dddf5d87bd8 // indirect golang.org/x/net v0.7.0 // indirect - golang.org/x/sys v0.9.0 // indirect - golang.org/x/term v0.9.0 // indirect - golang.org/x/text v0.10.0 // indirect + golang.org/x/sys v0.10.0 // indirect + golang.org/x/term v0.10.0 // indirect + golang.org/x/text v0.11.0 // indirect gopkg.in/warnings.v0 v0.1.2 // indirect ) @@ -72,8 +72,8 @@ github.com/jesseduffield/generics v0.0.0-20220320043834-727e535cbe68 h1:EQP2Tv8T github.com/jesseduffield/generics v0.0.0-20220320043834-727e535cbe68/go.mod h1:+LLj9/WUPAP8LqCchs7P+7X0R98HiFujVFANdNaxhGk= github.com/jesseduffield/go-git/v5 v5.1.2-0.20221018185014-fdd53fef665d h1:bO+OmbreIv91rCe8NmscRwhFSqkDJtzWCPV4Y+SQuXE= github.com/jesseduffield/go-git/v5 v5.1.2-0.20221018185014-fdd53fef665d/go.mod h1:nGNEErzf+NRznT+N2SWqmHnDnF9aLgANB1CUNEan09o= -github.com/jesseduffield/gocui v0.3.1-0.20230702054502-d6c452fc12ce h1:Xgm21B1an/outcRxnkDfMT6wKb6SKBR05jXOyfPA8WQ= -github.com/jesseduffield/gocui v0.3.1-0.20230702054502-d6c452fc12ce/go.mod |