diff options
| author | Harel Ben-Attia <harelba@gmail.com> | 2022-01-16 08:30:51 +0200 |
|---|---|---|
| committer | Harel Ben-Attia <harelba@gmail.com> | 2022-01-16 08:30:51 +0200 |
| commit | ce8733e74b35eb560b9f6ec8ce0c5fab94560f39 (patch) | |
| tree | 8a4b63f0941803febd5d9daaec3345530bd3aadd /test | |
| parent | a8b671ffcea5b52195485fd78e8e547d2b2d652d (diff) | |
update benchmark doc
Diffstat (limited to 'test')
| -rw-r--r-- | test/BENCHMARK.md | 7 |
1 files changed, 5 insertions, 2 deletions
diff --git a/test/BENCHMARK.md b/test/BENCHMARK.md index 2162e38..c8ec866 100644 --- a/test/BENCHMARK.md +++ b/test/BENCHMARK.md @@ -15,14 +15,17 @@ The following table shows the impact of using caching in q: | 100,000 | 100 | 99MB | 5.2 seconds | 0.141 seconds | x36 | | 100,000 | 50 | 48MB | 2.7 seconds | 0.105 seconds | x25 | -Effectively, `.qsql` files are just standard sqlite3 files, with an additional metadata table that is used for detecting changes in the original delimited file. This means that any tool that can read sqlite3 files can use these files directly. -As a side-effect from this addition, q knows how to directly query multi-file sqlite3 databases, which means that the user can query any sqlite3 database, or the `.qsql` file, even when the original file doesn't exist anymore. For example: +Effectively, `.qsql` files are just standard sqlite3 files, with an additional metadata table that is used for detecting changes in the original delimited file. This means that any tool that can read sqlite3 files can use these files directly. The tradeoff is of course the additional disk usage that the cache files take. + +A good side-effect to this addition, is that q now knows how to directly query multi-file sqlite3 databases. This means that the user can query any sqlite3 database file, or the `.qsql` file itself, even when the original file doesn't exist anymore. For example: ```bash q "select a.*,b.* from my_file.csv.qsql a left join some-sqlite3-database:::some_table_name b on (a.id = b.id)" ``` +NOTE: In the current version, caching is not enabled by default - Use `-C readwrite` to enable reading+writing cache files, or `-C read` to just read any existing cache files. A `~/.qrc` file can be added in order to make these options the default if you want. + The benchmark results below reflect the peformance without the caching, e.g. directly reading the delimited files, parsing them and performing the query. I'll update benchmark results later on to provide cached results as well. |