summaryrefslogtreecommitdiffstats
path: root/lib/Fetcher
AgeCommit message (Collapse)Author
2023-10-22Merge remote-tracking branch 'nextcloud/master' into master-mergeDevlin Junker
Signed-off-by: Devlin Junker <devlin.junker@gmail.com>
2023-10-11Log more useful things when checking a logoBenjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-08-26use unique name for cache folderBenjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-08-25Check available compression types of curl (#2328)Benjamin Brahmer
Check an use available compression types of curl use compression when downloading feed logo Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com> Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-08-09upmerged from masterDevlin Junker
Signed-off-by: Devlin Junker <devlin.junker@gmail.com>
2023-08-09Upgrade feed-io to v5.1.3Benjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-03-23If timestamp is null use timestamp "one year ago"Benjamin Brahmer
if timestamp is not set during creation of a feed use date one year ago code fixes and linting fixes. Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com> Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-03-23Use httpLastModified field for If-Modified-Since header when fetching feed ↵Robert Wunderer
updates Signed-off-by: Robert Wunderer <robert.wunderer@caprisys.at>
2023-03-23store useragent in one placeBenjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-02-14fix: do not request favicon for empty base URLSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2023-01-12don't process link if it's nullBenjamin Brahmer
don't process body of feed if it's null Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com> Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2022-11-17upmerging master into vue-rewriteDevlin Junker
Signed-off-by: Devlin Junker <devlin.junker@gmail.com>
2022-08-19Fix deprecationsBenjamin Brahmer
add phpstan deprecation rules Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com> Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2022-05-31If items of feed do not provide an author fallback to feed author (#1803)Benjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2022-05-24Use Feed Link as GUID when Feed omits Guid. (#1785)Accalia Elementia
* Use Feed Link as GUID when Feed omits Guid. As noted in nextcloud/news#1702 some feeds omit the GUID and are therefore not a valid RSS feed. nextcloud/news#1738 resolved the issue to allow valid feeds to update correctly when an invalid feed is present. This commit allows parsing of the invalid feed as well by assuming that the item link of the feed is unique to the feed and using it in place of the GUID when the feed omits the GUID. This will allow NextCloud News to accept and behave like many other popular feed aggregators when presented with such an invalid feed. Signed-off-by: Accalia <Accalia@Elementia.me> * Add basic Logging when using fallback guid Signed-off-by: Accalia <Accalia@Elementia.me> * Add basic Logging when using fallback guid - Fix Fatfinger Typo Signed-off-by: Accalia <Accalia@Elementia.me> * Add basic Logging when using fallback guid - Update tests to account for additional logging Signed-off-by: Accalia <Accalia@Elementia.me>
2022-04-18Fix malformed feeds (without GUIDs) stopping the update process (fixes #1738)nextcloud486153
Signed-off-by: nextcloud486153 <78801830+nextcloud486153@users.noreply.github.com>
2022-02-27Use rawurlencode as it encodes spaces correctly as %20Benjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-11-30Fix catching connect exceptions while fetching feed logos #1570Michael Hamann
Display the message instead of the response when catching an exception during feed logo fetching. As there is no response and thus no method `getResponse()` in `ConnectException`, this fixes a "call to undefined method"-error that prevented feeds from fetching if fetching the logo resulted in a `ConnectException`. Signed-off-by: Michael Hamann <michael@content-space.de>
2021-11-14Catch network errors while fetching feed logos, fixes #1570Benjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-10-18Smarter error logging Benjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de> Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
2021-10-18make use of constant Benjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de> Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
2021-10-18make use of base_url to handle relative urlsBenjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-10-18Download feed logos via guzzle to have better error handlingBenjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-05-03[Backport] Prevent some of the favicon fetching errorsBenjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-04-24replace deprecated getDescription callanoy
Signed-off-by: anoy <anoymouserver+github@mailbox.org>
2021-04-08Null check category label before trying to decode (#1282)Chris Danser
* Null check category label before trying to decode Signed-off-by: skiingwiz <skiingwiz@gmail.com>
2021-03-25Make PHPstan stricterSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2021-03-17🎨 Item: store categories as json & add helper fnsMarco Nassabain
+ added setter/getters that work with arrays to simplify use case Signed-off-by: Marco Nassabain <marco.nassabain@hotmail.com>
2021-03-17♻️ Renaming 'tags' into 'categories'Jimmy Huynh
Signed-off-by: Jimmy Huynh <jimmy.huynh@etu.unistra.fr>
2021-03-17✨ Update FeedFetcher: import 'tags' to 'item'Jimmy Huynh
Signed-off-by: Jimmy Huynh <jimmy.huynh@etu.unistra.fr>
2021-02-16remove unused lastModified from feed fetcherBenjamin Brahmer
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-02-16prefer the feeds logo over the faviconBenjamin Brahmer
The logo of the feed is prefered if it is a square picture, else the favicon is returned. Signed-off-by: Benjamin Brahmer <info@b-brahmer.de> Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
2021-02-13Remove V1 item APISean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2021-01-20Fetcher: Update client and add testSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2021-01-12Fix psalm issuesSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-12-30Remove PHPunit integration testsSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-12-15Remove LastModified-based cursor when updating feedsKevin Decherf
We remove the call to readSince() as some feeds push new articles with pubDate prior to the lastModified time stored for these feeds (e.g. lemonde.fr). As we go through all items of a feed again and again, we prevent the constant update of an item's lastModified timestamp by keeping the previous one if its fingerprint does not change. Fixes #921 Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2020-11-07remove deprecated YouTube playlist APIanoy
Signed-off-by: anoy <anoymouserver+github@mailbox.org>
2020-10-12Allow titles to be nullSean Molenaar
Issue GH-867 Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-29Add feed autodiscoverySean Molenaar
Issue GH-415 Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-29OPML export command and fixesSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-29Fix repair step and test itSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-27Create V2 mapper, Service and management commandsSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-25Move to nextcloud config and update phpunitSean Molenaar
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-08-31Fix crash if full-text is enabled and the news item does not provide an URL ↵Gioele Falcetti
to be fetched Signed-off-by: Gioele Falcetti <thegio.f@gmail.com>
2020-04-28fix 'If-Modified-Since' causing BAD REQUEST (#684)Chris Noxz
fix 'If-Modified-Since' causing BAD REQUEST Signed-off-by: Chris Noxz <chris@noxz.tech> Co-authored-by: kesselb <mail@danielkesselberg.de>
2020-02-16Basic Media-RSS support (#599)Éloi Rivard
Signed-off-by: Éloi Rivard <azmeuk@gmail.com>
2019-12-27Update httpLastModified from the feed response (#594)kesselb
LastModified is used (in a feed context to see if a user edited a feed). httpLastModified to store the last-modified response from the source. Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-12-24Reimplement full-text scraping (#563)DriverXX
Add readability.php scraper Fixes #482 Signed-off-by: Gioele Falcetti <thegio.f@gmail.com>
2019-11-16Get content:encoded of item if available (#565)b_b
Get content:encoded of item if available fixes #564 Signed-off-by: brunob <bruno@eliaz.fr>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-20 01:09:43 +0000