diff options
| author | Andrew Gallant <jamslam@gmail.com> | 2019-01-25 17:18:57 -0500 |
|---|---|---|
| committer | Andrew Gallant <jamslam@gmail.com> | 2019-01-25 17:18:57 -0500 |
| commit | 276e2c9b9ab3dac7643033d8b9d6c272d5dac1d5 (patch) | |
| tree | 781a5da81a73e541d869accfcdce15e5df518de3 /grep-searcher | |
| parent | 9a9f54d44ce45c6c3c5bfa31ad1a08fda57cdad7 (diff) | |
searcher: always strip BOM
This fixes a bug where a BOM prefix was included. While this was somewhat
intentional in order to have a faithful "UTF8 passthru" option, in
practice, this causes problems such as breaking patterns like `^` in a
really non-obvious way.
The actual fix was to add a new API to encoding_rs_io, which this commit
brings in.
Fixes #1163
Diffstat (limited to 'grep-searcher')
| -rw-r--r-- | grep-searcher/Cargo.toml | 2 | ||||
| -rw-r--r-- | grep-searcher/src/searcher/mod.rs | 1 |
2 files changed, 2 insertions, 1 deletions
diff --git a/grep-searcher/Cargo.toml b/grep-searcher/Cargo.toml index 4cf5fde3..1d977b45 100644 --- a/grep-searcher/Cargo.toml +++ b/grep-searcher/Cargo.toml @@ -15,7 +15,7 @@ license = "Unlicense/MIT" [dependencies] bytecount = "0.5" encoding_rs = "0.8.14" -encoding_rs_io = "0.1.3" +encoding_rs_io = "0.1.4" grep-matcher = { version = "0.1.1", path = "../grep-matcher" } log = "0.4.5" memchr = "2.1" diff --git a/grep-searcher/src/searcher/mod.rs b/grep-searcher/src/searcher/mod.rs index bc428b68..c70b3a0e 100644 --- a/grep-searcher/src/searcher/mod.rs +++ b/grep-searcher/src/searcher/mod.rs @@ -307,6 +307,7 @@ impl SearcherBuilder { decode_builder .encoding(self.config.encoding.as_ref().map(|e| e.0)) .utf8_passthru(true) + .strip_bom(true) .bom_override(true); Searcher { config: config, |