summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2021-06-12grep-printer-0.1.6grep-printer-0.1.6Andrew Gallant
2021-06-12deps/searcher: update minimal versionsAndrew Gallant
2021-06-12grep-searcher-0.1.8grep-searcher-0.1.8Andrew Gallant
2021-06-12deps/pcre2: update minimal versionsAndrew Gallant
2021-06-12grep-pcre2-0.1.5grep-pcre2-0.1.5Andrew Gallant
2021-06-12pcre2: update minimal version to 0.2.3Andrew Gallant
2021-06-12deps/regex: update minimal versionsAndrew Gallant
2021-06-12grep-regex-0.1.9grep-regex-0.1.9Andrew Gallant
2021-06-12deps/matcher: update minimal versionsAndrew Gallant
2021-06-12grep-matcher-0.1.5grep-matcher-0.1.5Andrew Gallant
2021-06-12deps/cli: update minimal versionsAndrew Gallant
2021-06-12grep-cli-0.1.6grep-cli-0.1.6Andrew Gallant
2021-06-12deps/ignore: update minimal versionsAndrew Gallant
2021-06-12release: add step about making sure 'master' is in syncAndrew Gallant
Otherwise, if we start doing crate releases from the local checkout (with git tags) and it turns out that origin/master has newer commits, rebasing local master will then invalidate those tags.
2021-06-12ignore-0.4.18ignore-0.4.18Andrew Gallant
2021-06-12deps/globset: update minimal versionsAndrew Gallant
2021-06-12globset-0.4.7globset-0.4.7Andrew Gallant
2021-06-12release: tweak 'cargo outdated' adviceAndrew Gallant
I do run --aggressive, although I've been ignoring the clap 3 update for what seems like forever since it's still in beta.
2021-06-12deps: update to memmap2Andrew Gallant
Looking at the changelog for memmap2, the only breaking change was to MmapOptions, which we don't use. So no migration is needed.
2021-06-12deps: updates libc and synAndrew Gallant
2021-06-01msrv: bump to Rust 1.52.1Andrew Gallant
This matches the latest stable release of Rust.
2021-06-01deps: update everythingAndrew Gallant
Removes two dependencies! autocfg and byteorder.
2021-06-01edition: manual changesAndrew Gallant
This is mostly just about removing 'extern crate' everywhere and fixing the fallout.
2021-06-01edition: run 'cargo fix --edition --edition-idioms --all'Andrew Gallant
2021-06-01edition: set edition=2018Andrew Gallant
2021-06-01edition: initial 'cargo fix --edition' runAndrew Gallant
2021-06-01changelog: fix typo and add Ruby to type improvement listAndrew Gallant
2021-06-01ignore/types: config.ru and *.rbw RubyUlysse Buonomo
PR #1886
2021-06-01changelog: a bit of polishAndrew Gallant
I think I'm just waiting on the CVE to be published at this point.
2021-06-01github: add note about file typesAndrew Gallant
2021-05-31ci: re-work github actions releaseAndrew Gallant
This combines the tips from #1820 and the patch submitted in #1675. The latter wasn't taken as-is because I didn't agree with some of the changes, and in particular, it removed the ability to easily test the release on a branch with a dummy tag name. I've tried to add that back here with the 'rg_version' output. Overall though, using outputs is indeed much simpler. Closes #1675, Closes #1820
2021-05-31changelog: prep for ripgrep 13 releaseAndrew Gallant
2021-05-31printer: trim line terminator before doing replacementsAndrew Gallant
This is basically the same bug as #1401, but applied to replacements instead of --only-matching. Fixes #1739
2021-05-31changelog: fish completions are stayingAndrew Gallant
In a previous release, I announced that Fish completions were being removed. But the Fish project decided to remove theirs and have ripgrep's stay. Closes #1577
2021-05-31globset: expand docs and impl Default for GlobSetMartin Pool
Closes #1882, Closes #1883
2021-05-31printer: trim line terminator before finding submatchesAndrew Gallant
This fixes a bug where PCRE2 look-around could change the result of a match if it observed a line terminator in the printer. And in particular, this is precisely how the searcher operates: the line is considered unto itself *without* the line terminator. Fixes #1401
2021-05-31grep: fix bugs in handling multi-line look-aroundAndrew Gallant
This commit hacks in a bug fix for handling look-around across multiple lines. The main problem is that by the time the matching lines are sent to the printer, the surrounding context---which some look-behind or look-ahead might have matched---could have been dropped if it wasn't part of the set of matching lines. Therefore, when the printer re-runs the regex engine in some cases (to do replacements, color matches, etc etc), it won't be guaranteed to see the same matches that the searcher found. Overall, this is a giant clusterfuck and suggests that the way I divided the abstraction boundary between the printer and the searcher is just wrong. It's likely that the searcher needs to handle more of the work of matching and pass that info on to the printer. The tricky part is that this additional work isn't always needed. Ultimately, this means a serious re-design of the interface between searching and printing. Sigh. The way this fix works is to smuggle the underlying buffer used by the searcher through into the printer. Since these bugs only impact multi-line search (otherwise, searches are only limited to matches across a single line), and since multi-line search always requires having the entire file contents in a single contiguous slice (memory mapped or on the heap), it follows that the buffer we pass through when we need it is, in fact, the entire haystack. So this commit refactors the printer's regex searching to use that buffer instead of the intended bundle of bytes containing just the relevant matching portions of that same buffer. There is one last little hiccup: PCRE2 doesn't seem to have a way to specify an ending position for a search. So when we re-run the search to find matches, we can't say, "but don't search past here." Since the buffer is likely to contain the entire file, we really cannot do anything here other than specify a fixed upper bound on the number of bytes to search. So if look-ahead goes more than N bytes beyond the match, this code will break by simply being unable to find the match. In practice, this is probably pretty rare. I believe that if we did a better fix for this bug by fixing the interfaces, then we'd probably try to have PCRE2 find the pertinent matches up front so that it never needs to re-discover them. Fixes #1412
2021-05-31printer: fix multi-line replacement bugAndrew Gallant
This commit fixes a subtle bug in multi-line replacement of line terminators. The problem is that even though ripgrep supports multi-line searches, it is *still* line oriented. It still needs to print line numbers, for example. For this reason, there are various parts in the printer that iterate over lines in order to format them into the desired output. This turns out to be problematic in some cases. #1311 documents one of those cases (with line numbers enabled to highlight a point later): $ printf "hello\nworld\n" | rg -n -U "\n" -r "?" 1:hello? 2:world? But the desired output is this: $ printf "hello\nworld\n" | rg -n -U "\n" -r "?" 1:hello?world? At first I had thought that the main problem was that the printer was taking ownership of writing line terminators, even if the input already had them. But it's more subtle than that. If we fix that issue, we get output like this instead: $ printf "hello\nworld\n" | rg -n -U "\n" -r "?" 1:hello?2:world? Notice how '2:' is printed before 'world?'. The reason it works this way is because matches are reported to the printer in a line oriented way. That is, the printer gets a block of lines. The searcher guarantees that all matches that start or end in any of those lines also end or start in another line in that same block. As a result, the printer uses this assumption: once it has processed a block of lines, the next match will begin on a new and distinct line. Thus, things like '2:' are printed. This is generally all fine and good, but an impedance mismatch arises when replacements are used. Because now, the replacement can be used to change the "block of lines" approach. Now, in terms of the output, the subsequent match might actually continue the current line since the replacement might get rid of the concept of lines altogether. We can sometimes work around this. For example: $ printf "hello\nworld\n" | rg -U "\n(.)?" -r '?$1' hello?world? Why does this work? It's because the '(.)' after the '\n' causes the match to overlap between lines. Thus, the searcher guarantees that the block sent to the printer contains every line. And there in lay the solution: all we need to do is tweak the multi-line searcher so that it combines lines with matches that directly adjacent, instead of requiring at least one byte of overlap. Fixing that solves the issue above. It does cause some tests to fail: * The binary3 test in the searcher crate fails because adjacent line matches are now one part of block, and that block is scanned for binary data. To preserve the essence of the test, we insert a couple dummy lines to split up the blocks. * The JSON CRLF test. It was testing that we didn't output any messages with an empty 'submatches' array. That is indeed still the case. The difference is that the messages got combined because of the adjacent line merging behavior. This is a slight change to the output, but is still correct. Fixes #1311
2021-05-31printer: vimgrep now only prints one lineAndrew Gallant
It turns out that the vimgrep format really only wants one line per match, even when that match spans multiple lines. We continue to support the previous behavior (print all lines in a match) in the `grep-printer` crate. We add a new option to enable the "only print the first line" behavior, and unconditionally enable it in ripgrep. We can do that because the option has no effect in single-line mode, since, well, in that case matches are guaranteed to span one line anyway. Fixes #1866
2021-05-31cli: add --field-{context,match}-separator flagsAnthony Huang
These flags permit configuring the bytes used to delimit fields in match or context lines, where "fields" are things like the file path, line number, column number and the match/context itself. Fixes #1842, Closes #1871
2021-05-31cargo: statically link binary on Windows/MSVCAustin Wise
Before this change, rg.exe depended on vcruntime140.dll, which does not exist on a fresh install of Windows. Closes #1613
2021-05-31globset: fix recursive suffix over matchingAndres Suarez
Previous, 'foo/**' would match 'foo', but it shouldn't have. In this case, not matching 'foo' is what is documented and also seems consistent with other recursive globbing implementations (like that in zsh). This also updates the prefix extractor to pull 'foo/' out of 'foo/**'. Closes #1756
2021-05-31ignore: check ignore rules before issuing stat callsRichard Khoury
This seems like an obvious optimization but becomes critical when filesystem operations even as simple as stat can result in significant overheads; an example of this was a bespoke filesystem layer in Windows that hosted files remotely and would download them on-demand when particular filesystem operations occurred. Users of this system who ensured correct file-type fileters were being used could still get unnecessary file access resulting in large downloads. Fixes #1657, Closes #1660
2021-05-31printer: fix context bug when --max-count is usedPen Tree
In the case where after-context is requested with a match count limit, we need to be careful not to reset the state tracking the remaining context lines. Fixes #1380, Closes #1642
2021-05-31searcher: do UTF-8 BOM sniffing like UTF-16Alessandro Menezes
Previously, we were only looking for the UTF-16 BOM for determining whether to do transcoding or not. But we should also look for the UTF-8 BOM as well. Fixes #1638, Closes #1697
2021-05-31ignore/types: add redRaimon Grau
See: https://www.red-lang.org/ Closes #1663
2021-05-31ignore/types: replace duplicate glob with *.aspx.vbSimon Morgan
*.aspx.cs was listed twice and the VB variant is missing. Closes #1683
2021-05-31doc: explain ignore rules a bit moretillyboy
Closes #1600
2021-05-31cli: add -. as short option for --hiddenJoão Marcos
This is somewhat non-standard, but it seems nice on the surface: short flag names are in short supply, --hidden is probably somewhat common and -. has an obvious connection with how hidden files are named on Unix. Closes #1680
2021-05-31printer: fix \r\n line terminator handlingAndrew Gallant
This fixes a bug where it was assumed that 'is_suffix' when CRLF handling was enabled mean that '\r\n' was present. But that's not the case, and it is intentional that 'is_suffix' only looks for '\n'. (Which is why #1803 wasn't taken, which tries to fix this by changing 'is_suffix'.) Fixes #1765, Closes #1803