summaryrefslogtreecommitdiffstats
path: root/langs
AgeCommit message (Collapse)Author
2024-02-05Filter dot files etc. in i18nBjørn Erik Pedersen
Closes #11993
2024-01-28all: Run gofumpt -l -w .Bjørn Erik Pedersen
2024-01-28testing: Simplify some integration testsBjørn Erik Pedersen
2024-01-28testing: Rename integration_test.go to PACKAGE_integration_test.goBjørn Erik Pedersen
Primary motivation making them easier to find in the code editor.
2024-01-27all: Rework page store, add a dynacache, improve partial rebuilds, and some ↵develop2024Bjørn Erik Pedersen
general spring cleaning There are some breaking changes in this commit, see #11455. Closes #11455 Closes #11549 This fixes a set of bugs (see issue list) and it is also paying some technical debt accumulated over the years. We now build with Staticcheck enabled in the CI build. The performance should be about the same as before for regular sized Hugo sites, but it should perform and scale much better to larger data sets, as objects that uses lots of memory (e.g. rendered Markdown, big JSON files read into maps with transform.Unmarshal etc.) will now get automatically garbage collected if needed. Performance on partial rebuilds when running the server in fast render mode should be the same, but the change detection should be much more accurate. A list of the notable new features: * A new dependency tracker that covers (almost) all of Hugo's API and is used to do fine grained partial rebuilds when running the server. * A new and simpler tree document store which allows fast lookups and prefix-walking in all dimensions (e.g. language) concurrently. * You can now configure an upper memory limit allowing for much larger data sets and/or running on lower specced PCs. We have lifted the "no resources in sub folders" restriction for branch bundles (e.g. sections). Memory Limit * Hugos will, by default, set aside a quarter of the total system memory, but you can set this via the OS environment variable HUGO_MEMORYLIMIT (in gigabytes). This is backed by a partitioned LRU cache used throughout Hugo. A cache that gets dynamically resized in low memory situations, allowing Go's Garbage Collector to free the memory. New Dependency Tracker: Hugo has had a rule based coarse grained approach to server rebuilds that has worked mostly pretty well, but there have been some surprises (e.g. stale content). This is now revamped with a new dependency tracker that can quickly calculate the delta given a changed resource (e.g. a content file, template, JS file etc.). This handles transitive relations, e.g. $page -> js.Build -> JS import, or $page1.Content -> render hook -> site.GetPage -> $page2.Title, or $page1.Content -> shortcode -> partial -> site.RegularPages -> $page2.Content -> shortcode ..., and should also handle changes to aggregated values (e.g. site.Lastmod) effectively. This covers all of Hugo's API with 2 known exceptions (a list that may not be fully exhaustive): Changes to files loaded with template func os.ReadFile may not be handled correctly. We recommend loading resources with resources.Get Changes to Hugo objects (e.g. Page) passed in the template context to lang.Translate may not be detected correctly. We recommend having simple i18n templates without too much data context passed in other than simple types such as strings and numbers. Note that the cachebuster configuration (when A changes then rebuild B) works well with the above, but we recommend that you revise that configuration, as it in most situations should not be needed. One example where it is still needed is with TailwindCSS and using changes to hugo_stats.json to trigger new CSS rebuilds. Document Store: Previously, a little simplified, we split the document store (where we store pages and resources) in a tree per language. This worked pretty well, but the structure made some operations harder than they needed to be. We have now restructured it into one Radix tree for all languages. Internally the language is considered to be a dimension of that tree, and the tree can be viewed in all dimensions concurrently. This makes some operations re. language simpler (e.g. finding translations is just a slice range), but the idea is that it should also be relatively inexpensive to add more dimensions if needed (e.g. role). Fixes #10169 Fixes #10364 Fixes #10482 Fixes #10630 Fixes #10656 Fixes #10694 Fixes #10918 Fixes #11262 Fixes #11439 Fixes #11453 Fixes #11457 Fixes #11466 Fixes #11540 Fixes #11551 Fixes #11556 Fixes #11654 Fixes #11661 Fixes #11663 Fixes #11664 Fixes #11669 Fixes #11671 Fixes #11807 Fixes #11808 Fixes #11809 Fixes #11815 Fixes #11840 Fixes #11853 Fixes #11860 Fixes #11883 Fixes #11904 Fixes #7388 Fixes #7425 Fixes #7436 Fixes #7544 Fixes #7882 Fixes #7960 Fixes #8255 Fixes #8307 Fixes #8863 Fixes #8927 Fixes #9192 Fixes #9324
2023-07-08Restore language.disabled configBjørn Erik Pedersen
Fixes #11219
2023-06-30Only print the path warnings onceBjørn Erik Pedersen
We could reset and rerun it on server rebuilds, but that report needs a full build to make sense. Also clean up the config vs flags in this area: Make all config settings match the flags e.g. `printPathWarnings`, but set up aliases for the old. Fixes #11187
2023-06-19all: Fix typos in function names and commentsOleksandr Redko
2023-06-18Replace the old log setup, with structured logging etc.Bjørn Erik Pedersen
Fixes #11124
2023-05-30Avoid panic in invalid language configBjørn Erik Pedersen
Fixes #11046
2023-05-29Fix potential deadlock in ByParamBjørn Erik Pedersen
Fixes #11039
2023-05-27langs: Remove the Language.Params deprecation message for nowBjørn Erik Pedersen
Closes #11025
2023-05-27Add language.LanguageCodeBjørn Erik Pedersen
But keep an alias at Site Closes #11027
2023-05-20Add hugo.WorkingDirBjørn Erik Pedersen
Fixes #10969
2023-05-18all: Fix comments for exported functions and packagesOleksandr Redko
2023-05-17Handle transient errors in config loading etc.Bjørn Erik Pedersen
As in: Get the Kubernetes site to build with the new Hugo version. Updates #10947
2023-05-17Deprecate site.Language.Params and some other fixesBjørn Erik Pedersen
Updates #10947
2023-05-16Create a struct with all of Hugo's config optionsBjørn Erik Pedersen
Primary motivation is documentation, but it will also hopefully simplify the code. Also, * Lower case the default output format names; this is in line with the custom ones (map keys) and how it's treated all the places. This avoids doing `stringds.EqualFold` everywhere. Closes #10896 Closes #10620
2023-04-24langs/i18n: Fallback to defaultContentLanguage instead of EnglishJoe Mooring
Co-authored-by: 641bill <wo23636@126.com> Fixes #9216
2023-03-04Page context handling in i18nBjørn Erik Pedersen
This is a workaround. We need to improve on this, but not today. Fixes #10782
2023-03-02Correct typos in Go commentsOleksandr Redko
2023-01-04Misc doc, code refactoring to improve documentationBjørn Erik Pedersen
2022-05-15Improve SASS errorsBjørn Erik Pedersen
Fixes #9897
2022-05-15postcss: Fix import error handlingBjørn Erik Pedersen
Note that we will now fail if `inlineImports` is enabled and we cannot resolve an import. You can work around this by either: * Use url imports or imports with media queries. * Set `skipInlineImportsNotFound=true` in the options Also get the argument order in the different NewFileError* funcs in line. Fixes #9895
2022-05-14errors: Misc improvementsBjørn Erik Pedersen
* Redo the server error template * Always add the content file context if relevant * Remove some now superflous error string matching * Move the server error template to _server/error.html * Add file context (with position) to codeblock render blocks * Improve JS build errors Fixes #9892 Fixes #9891 Fixes #9893
2022-05-06Improve error messages, esp. when the server is runningBjørn Erik Pedersen
* Add file context to minifier errors when publishing * Misc fixes (see issues) * Allow custom server error template in layouts/server/error.html To get to this, this commit also cleans up and simplifies the code surrounding errors and files. This also removes the usage of `github.com/pkg/errors`, mostly because of https://github.com/pkg/errors/issues/223 -- but also because most of this is now built-in to Go. Fixes #9852 Fixes #9857 Fixes #9863
2022-04-27Some godoc adjustments and image struct renamesBjørn Erik Pedersen
2022-04-12Make string sorting (e.g. ByTitle, ByLinkTitle and ByParam) language awareBjørn Erik Pedersen
Fixes #2180
2022-04-10Replace all usage of CopyOnWriteFs with OverlayFsBjørn Erik Pedersen
Fixes #9761
2022-04-08Rework the Destination filesystem to make --renderStaticToDisk workBjørn Erik Pedersen
See #9626
2022-04-05Localize all the GroupBy*Date methodsBjørn Erik Pedersen
Fixes #9745
2022-03-17all: gofmt -w -r 'interface{} -> any' .Bjørn Erik Pedersen
Updates #9687
2022-03-08Cache reflect.MethodByNameBjørn Erik Pedersen
The isolated benchmark for the function is obviously much faster: ```bash name old time/op new time/op delta GetMethodByName-10 1.21µs ± 7% 0.23µs ± 5% -81.42% (p=0.029 n=4+4) name old alloc/op new alloc/op delta GetMethodByName-10 680B ± 0% 0B -100.00% (p=0.029 n=4+4) name old allocs/op new allocs/op delta GetMethodByName-10 20.0 ± 0% 0.0 -100.00% (p=0.029 n=4+4) ``` But more pleasing is the overall performance looking at the site benchmarks: ```bash name old time/op new time/op delta SiteNew/Regular_Bundle_with_image-10 6.25ms ± 2% 6.10ms ± 2% ~ (p=0.057 n=4+4) SiteNew/Regular_Bundle_with_JSON_file-10 6.30ms ± 2% 5.66ms ±11% ~ (p=0.057 n=4+4) SiteNew/Regular_Tags_and_categories-10 22.2ms ± 2% 17.4ms ± 1% -21.88% (p=0.029 n=4+4) SiteNew/Regular_Canonify_URLs-10 108ms ± 0% 107ms ± 0% -1.20% (p=0.029 n=4+4) SiteNew/Regular_Deep_content_tree-10 36.1ms ± 1% 33.8ms ± 1% -6.44% (p=0.029 n=4+4) SiteNew/Regular_TOML_front_matter-10 24.9ms ± 1% 22.6ms ± 1% -9.30% (p=0.029 n=4+4) SiteNew/Regular_Many_HTML_templates-10 17.9ms ± 1% 16.7ms ± 1% -6.43% (p=0.029 n=4+4) SiteNew/Regular_Page_collections-10 23.3ms ± 1% 22.0ms ± 0% -5.58% (p=0.029 n=4+4) SiteNew/Regular_List_terms-10 8.00ms ± 1% 7.63ms ± 0% -4.62% (p=0.029 n=4+4) name old alloc/op new alloc/op delta SiteNew/Regular_Bundle_with_image-10 2.10MB ± 0% 2.07MB ± 0% -1.46% (p=0.029 n=4+4) SiteNew/Regular_Bundle_with_JSON_file-10 1.88MB ± 0% 1.85MB ± 0% -1.76% (p=0.029 n=4+4) SiteNew/Regular_Tags_and_categories-10 13.5MB ± 0% 11.6MB ± 0% -13.99% (p=0.029 n=4+4) SiteNew/Regular_Canonify_URLs-10 96.1MB ± 0% 95.8MB ± 0% -0.40% (p=0.029 n=4+4) SiteNew/Regular_Deep_content_tree-10 28.4MB ± 0% 27.3MB ± 0% -3.83% (p=0.029 n=4+4) SiteNew/Regular_TOML_front_matter-10 16.9MB ± 0% 15.1MB ± 0% -10.58% (p=0.029 n=4+4) SiteNew/Regular_Many_HTML_templates-10 8.98MB ± 0% 8.44MB ± 0% -6.04% (p=0.029 n=4+4) SiteNew/Regular_Page_collections-10 17.1MB ± 0% 16.5MB ± 0% -3.91% (p=0.029 n=4+4) SiteNew/Regular_List_terms-10 3.92MB ± 0% 3.72MB ± 0% -5.03% (p=0.029 n=4+4) name old allocs/op new allocs/op delta SiteNew/Regular_Bundle_with_image-10 25.8k ± 0% 24.9k ± 0% -3.49% (p=0.029 n=4+4) SiteNew/Regular_Bundle_with_JSON_file-10 25.8k ± 0% 24.9k ± 0% -3.49% (p=0.029 n=4+4) SiteNew/Regular_Tags_and_categories-10 288k ± 0% 233k ± 0% -18.90% (p=0.029 n=4+4) SiteNew/Regular_Canonify_URLs-10 375k ± 0% 364k ± 0% -2.80% (p=0.029 n=4+4) SiteNew/Regular_Deep_content_tree-10 314k ± 0% 283k ± 0% -9.77% (p=0.029 n=4+4) SiteNew/Regular_TOML_front_matter-10 302k ± 0% 252k ± 0% -16.55% (p=0.029 n=4+4) SiteNew/Regular_Many_HTML_templates-10 133k ± 0% 117k ± 0% -11.81% (p=0.029 n=4+4) SiteNew/Regular_Page_collections-10 202k ± 0% 183k ± 0% -9.55% (p=0.029 n=4+4) SiteNew/Regular_List_terms-10 55.6k ± 0% 49.8k ± 0% -10.40% (p=0.029 n=4+4) ``` Thanks to @quasilyte for the suggestion. Fixes 9386
2021-12-02Validate private use language tagsJoe Mooring
Fixes #9119
2021-10-16Reimplement archetypesBjørn Erik Pedersen
The old implementation had some issues, mostly related to the context (e.g. name, file paths) passed to the template. This new implementation is using the exact same code path for evaluating the pages as in a regular build. This also makes it more robust and easier to reason about in a multilingual setup. Now, if you are explicit about the target path, Hugo will now always pick the correct mount and language: ```bash hugo new content/en/posts/my-first-post.md ``` Fixes #9032 Fixes #7589 Fixes #9043 Fixes #9046 Fixes #9047
2021-08-02Reduce binary size vs locale, update to CLDR v36.1Bjørn Erik Pedersen
Test building with `go build -ldflags="-s -w"` Hugo 0.86.2: 46MB Before this commit: 77MB After this commit: 54MB Fixes #8839 Fixes #8841
2021-08-01Fail on invalid time zoneBjørn Erik Pedersen
Fixes #8832
2021-07-29tpl/lang: Add new localized versions of lang.FormatNumber etc.Bjørn Erik Pedersen
Fixes #8820
2021-07-28Switch to go-toml v2Bjørn Erik Pedersen
We have been using `go-toml` for language files only. This commit makes it the only TOML library. It's spec compliant and very fast. A benchark building a site with 200 pages with TOML front matter: ```bash name old time/op new time/op delta SiteNew/Regular_TOML_front_matter-16 48.5ms ± 1% 47.1ms ± 1% -2.85% (p=0.029 n=4+4) name old alloc/op new alloc/op delta SiteNew/Regular_TOML_front_matter-16 16.9MB ± 0% 16.7MB ± 0% -1.56% (p=0.029 n=4+4) name old allocs/op new allocs/op delta SiteNew/Regular_TOML_front_matter-16 302k ± 0% 296k ± 0% -2.20% (p=0.029 n=4+4) ``` Note that the front matter unmarshaling is only a small part of building a site, so the above is very good. Fixes #8801
2021-07-27Add timezone support for front matter dates without oneBjørn Erik Pedersen
Fixes #8810
2021-07-27Localize time.FormatBjørn Erik Pedersen
Fixes #8797
2021-07-15hugofs: Make FileMeta a structBjørn Erik Pedersen
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct. This is easier to reason about, and it's more effective: ``` name old time/op new time/op delta SiteNew/Regular_Deep_content_tree-16 71.5ms ± 3% 69.4ms ± 5% ~ (p=0.200 n=4+4) name old alloc/op new alloc/op delta SiteNew/Regular_Deep_content_tree-16 29.7MB ± 0% 27.9MB ± 0% -5.82% (p=0.029 n=4+4) name old allocs/op new allocs/op delta SiteNew/Regular_Deep_content_tree-16 313k ± 0% 303k ± 0% -3.35% (p=0.029 n=4+4) ``` See #8749
2021-06-18Split out the puthe path/filepath functions into common/pathsBjørn Erik Pedersen
So they can be used from the config package without cyclic troubles. Updates #8654
2021-06-14Misc config loading fixesBjørn Erik Pedersen
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster: ``` BenchmarkDefaultConfigProvider/Viper-16 252418 4546 ns/op 2720 B/op 30 allocs/op BenchmarkDefaultConfigProvider/Custom-16 450756 2651 ns/op 1008 B/op 6 allocs/op ``` Fixes #8633 Fixes #8618 Fixes #8630 Updates #8591 Closes #6680 Closes #5192
2021-06-07tpl/fmt: Add erroridf template funcBjørn Erik Pedersen
Fixes #8613
2021-05-02langs/i18n: Fix warning regression in i18nBjørn Erik Pedersen
Fix this by 1. Making sure that only numerical values are treated as plural counts 2. Making sure that `i18n.pluralFormNotFoundError` is not logged as a warning if `other` resolved. Note that 2. isn't a new problem, but became visible because of the plural improvements in Hugo `0.83.0`. Fixes #8492
2021-04-30langs/i18n: Fix multiple unknown language codesBjørn Erik Pedersen
This fixes a bug upstream by temporarily using a patched version of go-i18n. Closes #7838
2021-04-25langs/i18n: Improve plural handling of floatsBjørn Erik Pedersen
The go-i18n library expects plural counts with floats to be represented as strings. Fixes #8464
2021-04-23langs/i18n: Revise the plural implementationBjørn Erik Pedersen
There were some issues introduced with the plural counting when we upgraded from v1 to v2 of go-i18n. This commit improves that situation given the following rules: * A single integer argument is used as plural count and passed to the i18n template as a int type with a `.Count` method. The latter is to preserve compability with v1. * Else the plural count is either fetched from the `Count`/`count` field/method/map or from the value itself. * Any data type is accepted, if it can be converted to an integer, that value is used. The above means that you can now do pass a single integer and both of the below will work: ``` {{ . }} minutes to read {{ .Count }} minutes to read ``` Fixes #8454 Closes #7822 See https://github.com/gohugoio/hugoDocs/issues/1410
2021-02-06langs/i18n: Support translation files with suffix *.ymlBjørn Erik Pedersen
Fixes #8212
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-22 04:53:20 +0000