diff options
| author | Mynacol <Mynacol@users.noreply.github.com> | 2023-06-21 16:34:28 +0200 |
|---|---|---|
| committer | Benjamin Brahmer <info@b-brahmer.de> | 2023-06-22 11:31:42 +0200 |
| commit | 48e1ad158358b0a9c556d76b18f84984e984bf13 (patch) | |
| tree | 3ed71b26f58aa2207906bb14b9ea0c5f69b09254 /lib | |
| parent | 6410e4f272dad7fc9a8d39ccebfa05a431ce1d2e (diff) | |
Use HTTP compression for downloads
As long as we use Feed-io < 6, we always use Guzzle (with Feed-io 6, we
need to choose a [HTTPlug](https://httplug.io/) library ourselves). Guzzle
[supports](https://docs.guzzlephp.org/en/stable/request-options.html#decode-content)
transparently decompressing gzip or deflate compressed responses, which is enabled by default.
Feed-io does this [by default](https://github.com/alexdebril/feed-io/pull/311/commits/ffef9eaabefcfc66bca3bb598b952d13842f14b5),
but as we override the headers, we have to add a fitting `Accept-Encoding` header as well.
Previously, my feed collection caused up to 45.9 MB download traffic per cronjob,
with this commit it is as low as 23.6 MB.
Signed-off-by: Mynacol <Mynacol@users.noreply.github.com>
Diffstat (limited to 'lib')
| -rw-r--r-- | lib/Config/FetcherConfig.php | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/lib/Config/FetcherConfig.php b/lib/Config/FetcherConfig.php index ce2e7db8d..797dae49e 100644 --- a/lib/Config/FetcherConfig.php +++ b/lib/Config/FetcherConfig.php @@ -103,7 +103,11 @@ class FetcherConfig { $config = [ 'timeout' => $this->client_timeout, - 'headers' => ['User-Agent' => static::DEFAULT_USER_AGENT, 'Accept' => static::DEFAULT_ACCEPT], + 'headers' => [ + 'User-Agent' => static::DEFAULT_USER_AGENT, + 'Accept' => static::DEFAULT_ACCEPT, + 'Accept-Encoding' => 'gzip, deflate', + ], ]; if (!is_null($this->proxy)) { |