summaryrefslogtreecommitdiffstats
path: root/hugolib/resource_chain_test.go
AgeCommit message (Collapse)Author
2023-06-18Replace the old log setup, with structured logging etc.Bjørn Erik Pedersen
Fixes #11124
2023-03-01Replace deprecated ioutil with io and osOleksandr Redko
https://pkg.go.dev/io/ioutil is deprecated since Go 1.16.
2023-01-25Only invoke a given cached partial onceBjørn Erik Pedersen
Note that this is backed by a LRU cache (which we soon shall see more usage of), so if you're a heavy user of cached partials it may be evicted and refreshed if needed. But in most cases every partial is only invoked once. This commit also adds a timeout (the global `timeout` config option) to make infinite recursion in partials easier to reason about. ``` name old time/op new time/op delta IncludeCached-10 8.92ms ± 0% 8.48ms ± 1% -4.87% (p=0.016 n=4+5) name old alloc/op new alloc/op delta IncludeCached-10 6.65MB ± 0% 5.17MB ± 0% -22.32% (p=0.002 n=6+6) name old allocs/op new allocs/op delta IncludeCached-10 117k ± 0% 71k ± 0% -39.44% (p=0.002 n=6+6) ``` Closes #4086 Updates #9588
2022-09-14Support PostProcess for all file typesBjørn Erik Pedersen
Not just HTML. Fixes #10269
2022-04-08Rework the Destination filesystem to make --renderStaticToDisk workBjørn Erik Pedersen
See #9626
2022-04-05resources: Create a common ResourceFinder interfaceBjørn Erik Pedersen
And make both .Resources and resources implement it. This gets us 2 new methods/functions, so you can now also do: * .Resources.Get * resources.ByType Note that GetRemote is not covered by this interface, as that is only available as a global template function. Fixes #8653
2022-03-25resources: Add more details to .ErrBjørn Erik Pedersen
This commit adds a .Data object (a map with `Body`, `StatusCode` etc.) to the .Err returned from `resources.GetRemote`, which means you can now do: ``` {{ with .Err }} {{ range $k, $v := .Data }} {{ end }} {{ end }} ``` Fixes #9708
2022-03-17all: gofmt -w -r 'interface{} -> any' .Bjørn Erik Pedersen
Updates #9687
2022-02-15deps: Update github.com/tdewolff/minify/v2 v2.9.22 => v2.9.29Joe Mooring
Fixes #9244 Fixes #9132 Fixes https://discourse.gohugo.io/t/36523
2022-02-09postcss: Move integration test to its own packageBjørn Erik Pedersen
2022-02-09minifier: Port integration tests to its packageBjørn Erik Pedersen
2022-02-09templates: Port integration test to its packageBjørn Erik Pedersen
2022-02-09tocss: Port integration tests to their packageBjørn Erik Pedersen
2021-12-22media: Also consider extension in FromContentBjørn Erik Pedersen
As used in `resources.GetRemote`. This will now reject image files with text and text files with images.
2021-12-17Add resources.GetRemoteBjørn Erik Pedersen
In Hugo 0.89 we added remote support to `resources.Get`. In hindsight that was not a great idea, as a poll from many Hugo users showed. See Issue #9285 for more details. After this commit `resources.Get` only supports local resource lookups. If you want to support both, you need to use a construct similar to: Also improve some option case handling. ``` {{ resource := "" }} {{ if (urls.Parse $url).IsAbs }} {{ $resource = resources.GetRemote $url }} {{ else }} {{ $resource = resources.Get $url }} {{ end }} ``` Fixes #9285 Fixes #9296
2021-12-16Add some basic security policies with sensible defaultsBjørn Erik Pedersen
This ommmit contains some security hardening measures for the Hugo build runtime. There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers". For `asciidoctor` and some others we use Go's `os/exec` package to start a new process. These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off. You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do. The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all. ```toml [security] enableInlineShortcodes = false [security.exec] allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$'] osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$'] [security.funcs] getenv = ['^HUGO_'] [security.http] methods = ['(?i)GET|POST'] urls = ['.*'] ```
2021-12-10Allow user to handle/ignore errors in resources.GetBjørn Erik Pedersen
In Hugo 0.90.0 we introduced remote support in `resources.Get`. But with remote resources comes with a higher chance of failing a build (network issues, remote server down etc.). Before this commit we always failed the build on any unexpected error. This commit allows the user to check for any error (and potentially fall back to a default local resource): ```htmlbars {{ $result := resources.Get "https://gohugo.io/img/hugo-logo.png" }} {{ with $result }} {{ if .Err }} {{/* log the error, insert a default image etc. *}} {{ else }} <img src="{{ .RelPermalink }}" width="{{ .Width }}" height="{{ .Height }}" alt=""> {{ end }} {{ end }} ``` Note that the default behaviour is still to fail the build, but we will delay that error until you start using the `Resource`. Fixes #9529
2021-12-10Make resource.Get return nil on 404 not foundBjørn Erik Pedersen
This is in line with the interface declaration and also how local lookups work. Fixes #9267
2021-12-02Implement XML data supportPaul van Brouwershaven
Example: ``` {{ with resources.Get "https://example.com/rss.xml" | transform.Unmarshal }} {{ range .channel.item }} <strong>{{ .title | plainify | htmlUnescape }}</strong><br /> <p>{{ .description | plainify | htmlUnescape }}</p> {{ $link := .link | plainify | htmlUnescape }} <a href="{{ $link }}">{{ $link }}</a><br /> <hr> {{ end }} {{ end }} ``` Closes #4470
2021-11-30Add remote support to resources.GetPaul van Brouwershaven
Closes #5255 Supports #9044
2021-10-18Add a cross process build lock and use it in the archetype content builderBjørn Erik Pedersen
Fixes #9048
2021-10-10Fix value of useResourceCacheWhen in TestResourceChainPostCSSJoe Mooring
2021-10-09Skip a test assertion on CIBjørn Erik Pedersen
Need to investigate later, but got to get the tests green.
2021-09-22Pass minification errors to the userPaul Gottschling
Previously, *minifyTransformation.Transform suppressed the error returned by t.m.Minify. This meant that when minification returned an error, the error would not reach the user. Instead, minification would silently fail. For example, if a JavaScript file included a call to the Date constructor with: new Date(2020, 04, 02) The package that the minification library uses to parse JS files, github.com/tdewolff/parse would return an error, since "04" would be parsed as a legacy octal. However, the JS file would remain un-minified with no error. Fixing this is not as simple as replacing "_" with an "err" in *minifyTransformation.Transform, however (though this is necessary). If we only returned this error from Transform, then hugolib.TestResourceMinifyDisabled would fail. Instead of being a no-op, as TestResourceMinifyDisabled expects, using the "minify" template function with a "disableXML=true" config setting instead returns the error, "minifier does not exist for mimetype." The "minifier does not exist" error is returned because of the way minifiers.New works. If the user's config disables minification for a particular MIME type, minifiers.New does not add it to the resulting Client's *minify.M. However, this also means that when the "minify" template function is executed, a *resourceAdapter's transformations still add a minification. When it comes time to call the minify.Minifier for a specific MIME type via *M.MinifyMimetype, the github.com/tdewolff/minify library throws the "does not exist" error for the missing MIME type. The solution was to change minifiers.New so, instead of skipping a minifier for each disabled MIME type, it adds a NoOpMinifier, which simply copies the source to the destination without minification. This means that when the "minify" template function is used for a particular resource, and that resource's MIME type has minification disabled, minification is genuinely skipped, and does not result in an error. In order to add this, I've fixed a possibly unwanted interaction between minifiers.TestConfigureMinify and hugolib.TestResourceMinifyDisabled. The latter disables minification and expects minification to be a no-op. The former disables minification and expects it to result in an error. The only reason hugolib.TestResourceMinifyDisabled passes in the original code is that the "does not exist" error is suppressed. However, we shouldn't suppress minification errors, since they can leave users perplexed. I've changed the test assertion in minifiers.TestConfigureMinify to expect no errors and a no-op if minification is disabled for a particular MIME type. Fixes #8954
2021-08-22Prevent minifier from removing quoutes around post-processed attributesBjørn Erik Pedersen
Fixes #8884
2021-07-19minifiers: Make keepWhitespace = true default for HTMLBjørn Erik Pedersen
Fixes #8771
2021-06-14Misc config loading fixesBjørn Erik Pedersen
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster: ``` BenchmarkDefaultConfigProvider/Viper-16 252418 4546 ns/op 2720 B/op 30 allocs/op BenchmarkDefaultConfigProvider/Custom-16 450756 2651 ns/op 1008 B/op 6 allocs/op ``` Fixes #8633 Fixes #8618 Fixes #8630 Updates #8591 Closes #6680 Closes #5192
2021-02-18deps: Update github.com/tdewolff/minify/v2 v2.6.2 => v2.9.13Bjørn Erik Pedersen
Fixes #8258
2021-02-01github: Enable NPM tests on WindowsBjørn Erik Pedersen
Fixes #8196
2020-12-30Add Dart Sass supportBjørn Erik Pedersen
But note that the Dart Sass Embedded Protocol is still in beta (beta 5), a main release scheduled for Q1 2021. Fixes #7380 Fixes #8102
2020-12-23para: Skip para test when not on CIBjørn Erik Pedersen
Fixes #6963
2020-12-19Improve LookPathBjørn Erik Pedersen
2020-12-17Fix Resource.ResourceType so it always returns MIME's main typeBjørn Erik Pedersen
The one exception being for the Page, which does not have a MIME type, in which you will get the value `page`. Fixes #8052
2020-12-03all: Format code with gofumptBjørn Erik Pedersen
See https://github.com/mvdan/gofumpt
2020-10-07Revert "deps: Update to github.com/tdewolff/minify v2.9.4"Bjørn Erik Pedersen
Closes #7792 This reverts commit b254532b52785954c98a473a635b9cea016d8565.
2020-09-19deps: Update to github.com/tdewolff/minify v2.9.4Bjørn Erik Pedersen
2020-09-13Add "hugo mod npm pack"Bjørn Erik Pedersen
This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into: ``` assets/_jsconfig ´`` These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename. But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific. This commit also adds adds `workDir/node_modules` to `NODE_PATH` and `HUGO_WORKDIR` to the env when running the JS tools above. Fixes #7644 Fixes #7656 Fixes #7675
2020-08-20Improve stderr logging for PostCSS and simlilarBjørn Erik Pedersen
Fixes #7584
2020-06-18Rename taxonomy kinds from taxonomy to term, taxonomyTerm to taxonomyBjørn Erik Pedersen
And we have taken great measures to limit potential site breakage: * For `disableKinds` and `outputs` we try to map from old to new values if possible, if not we print an ERROR that can be toggled off if not relevant. * The layout lookup is mostly compatible with more options for the new `term` kind. That leaves: * Where queries in site.Pages using taxonomy/taxonomyTerm Kind values as filter. * Other places where these kind value are used in the templates (classes etc.) Fixes #6911 Fixes #7395
2020-04-21Fix broken testBjørn Erik Pedersen
2020-04-07Add basic "post resource publish support"Bjørn Erik Pedersen
Fixes #7146
2020-03-20Some minify configuration adjustmentsBjørn Erik Pedersen
2020-03-16Add workaround for regular CSS imports in SCSSBjørn Erik Pedersen
Fixes #7059
2020-03-11Improve Tailwind/PostCSS error messagesBjørn Erik Pedersen
Fixes #7041 Fixes #7042
2020-03-03Add build.UseResourceCacheWhenBjørn Erik Pedersen
Fixes #6993
2020-02-27resources: Add basic @import support to resources.PostCSSBjørn Erik Pedersen
This commit also makes the HUGO_ENVIRONMENT environment variable available to Node. Fixes #6957 Fixes #6961
2019-12-19Fix abs path handling in module mountsBjørn Erik Pedersen
Fixes #6622
2019-11-26Fix language handling in ExecuteAsTemplateBjørn Erik Pedersen
Fixes #6331
2019-10-09deps: Update minify to v2.5.2Anthony Fok
- v2.5.1 removes import comments, solving a build error with Go 1.13 in GOPATH mode (used Debian packaging for example) - v2.5.2 no longer converts polyline/rect/polygon/line to path as it has been reported to break a SVG referenced by CSS, see tdewolff/minify#260 The test case for Min SVG in TestResourceChains is updated accordingly. Fixes pocc/tshark.dev#33
2019-09-05Fix concat with fingerprint regressionBjørn Erik Pedersen
In Hugo 0.58 we optimized the transformers that only adjusted metadata, e.g. the fingerprint. This depended on the source readers implementing `io.ReadSeeker`. The reader produced by `concat` did that, but the implementation was buggy. This commit fixes that. Fixes #6309