Add a documentation download widget - #3828
Merged
Merged
Conversation
Readers can download whole products of the documentation as a single .tar.gz, choosing products, a version for each versioned product, and a format: Markdown one file per page, one Markdown file per product, HTML, or JSON. The picker is a dialog carried by every documentation page, opened from a button at the bottom of the sidebar or from the page-meta column, where it replaces the old per-page "Download Markdown" link. That one-page shortcut now lives only in the URL convention (add .md to any page), which ai-agent-resources.md documents alongside the direct archive URLs. How it fits together data/doc_bundles.json is the single source of truth for products, their paths and the formats. make_doc_bundles.py names the archives from it and partials/download-docs.html renders the picker from it, so the two cannot drift apart -- don't hard-code product ids in either. CI packages one archive per product+version+format from the finished build and uploads them to the docs bucket, so nothing is assembled at request time. The versioned products are built by separate matrix jobs, so each job packages what it has and every deploy job writes into the same bucket directory. That is why deploy_latest's destructive mirror now excludes ^downloads/bundles/: without it, -d would delete every versioned archive, because that job does not have them. Multi-product downloads are stitched in the browser -- inflate each bundle, copy its entries up to its end-of-archive marker, deflate the result -- so streams keep memory flat whether one product is picked or all thirteen. Every bundle roots its files at redis-docs/<product>-<version>/, so merged archives never collide, not even two versions of the same product. Bundles are byte-for-byte reproducible: fixed mtimes, sorted entries, and no build timestamp anywhere inside. An unchanged product produces identical bytes, so gsutil rsync -c skips it rather than re-uploading the whole set on every push. Offline HTML The published pages address everything from the site root, and under file:// the root is the filesystem root, so an untouched copy opens unstyled with dead links. HTML bundles therefore rewrite every URL against the depth of the page holding it, carry the assets their pages reference (following stylesheets, which pull in fonts and background images of their own), prune the sidebar to the product the bundle holds, drop the version level from versioned bundles, empty the site header, remove whatever the templates mark data-offline-hide, and carry a notice below the header saying this is a copy, with a link to the live page. Traps worth remembering - gsutil guesses "Content-Encoding: gzip" from the .tar.gz extension, so browsers inflate the archive in transit and save a plain tar under a .tar.gz name. The upload pins Content-Type and clears the guessed encoding. - Reading a Response whose body stream errored does not reliably reject with that stream's reason: browsers substitute "The operation was aborted." mergedBlob keeps hold of the reason and rethrows it, or a missing archive reports nothing a reader can act on. - Guard a browser entry point on `document`, not on `module` being undefined. Any other script defining a `module` global otherwise stops the whole file running. - The nav templates never close their <li> elements. That is legal HTML, but an item ends at the next item at its level or at the end of its list, not at a </li> -- looking for one finds nothing and strips the opening tags instead. - Pages are never assets. Archived versions and excluded child products sit in the build one link away, and following references would drag them back in. - Some pages hand-write <img src="../../../../../images/x.png">, which climbs to the site root from where the page sits on redis.io but overshoots in a bundle rooted three levels shallower. Relative URLs are resolved against the page's own place on the site for that reason. Those paths are also broken on redis.io itself in every archived version, which wants fixing in the content. Covered by build/test_make_doc_bundles.py (29 cases: archive layout, exclusions, reproducibility, and the offline HTML rewriting) and build/test_download_docs.cjs (7 cases, merging real archives and checking them with the system tar). The merge tests matter more than they look: a tar ends with zero blocks and every reader stops at the first set it meets, so a bad seam silently yields an archive holding only the first product. `make serve_downloads` drives the whole feature locally, on the same URLs and content type as production, with no extra server. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e pages reaching the archives The picker now lists what the sidebar lists, in the sidebar's order: Develop with Redis, Libraries and tools, RedisVL, the eight Redis products, Glossary, and Command reference last, where the sidebar keeps Commands in its own box. Get started and APIs are gone -- both were single link-hub pages. Redis Iris context engine and Redis Feature Form are now offered. They were missing because the first manifest was written by reading a directory listing and keeping the familiar names, and nothing compared that list against the site. test_every_product_is_offered is that comparison: every directory under content/operate and content/integrate must be covered by some docset, or the tests fail. Redis Data Integration has no archived versions, so its entry no longer declares any and the row shows no dropdown. That removal is quietly dangerous on its own -- iter_pages only skips version directories for products that declare "versions", so if RDI versions return and the entry is not updated, every archived version gets packaged into the latest download. test_version_directories_require_a_versions_key catches exactly that. Both new guards were checked by breaking them deliberately; neither passes vacuously. Alias redirects are no longer shipped as pages. A renamed page leaves a meta-refresh stub at its old URL, which Hugo writes as html only, so an html bundle carried the page twice, once under each name, and counted both. Redis Software went from 2397 "pages" to 607, matching its md bundle exactly; Develop from 1907 to 497. make serve_downloads now builds with --cleanDestinationDir. Hugo leaves orphans behind, so a public/ rebuilt in place still holds pages whose source was renamed or deleted, and the packager cannot tell them from real ones: content/operate/rs/8.2 had not existed for weeks and was still producing four phantom bundles, and a page renamed in July was still shipping under its old name. CI never sees this, building from a fresh checkout, so it was purely a local trap -- and one that made local verification lie. Smaller things: the version dropdowns take a fixed width and centre their label, so the column lines up whether it reads "latest" or "v0.25.0"; rows without a description are vertically centred so a one-line name sits level with the dropdown beside it, while rows with one stay top-aligned so the checkbox does not float between the two lines; and Redis Open Source lost a description that only restated its own title. The two descriptions left both say which child product is *not* included, which is the one thing a reader cannot infer from the row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The dialog gains a small table above the products, offering the one page the reader is on as a single published file rather than an archive. This restores what the old per-page "Download Markdown" link did before the dialog replaced it, and it has its own button so it stays clear of the product selection and the select-all checkbox. Its title and filename are filled in by the script from location rather than rendered by Hugo. The dialog is included with partialCached, so a single copy of its markup serves every page in the site and nothing page-specific can be baked into it. pageFileFor() maps a URL to the file Hugo published for it and is unit-tested, because its edge cases are quiet ones: a URL without its trailing slash, the site root having no last segment to name a file after, and md-single collapsing to Markdown since one page is already one file. Single-page HTML is declined rather than served. A page's stylesheet, fonts, and links all live at the site root, so a lone index.html opens unstyled with dead links -- which is precisely why the html format ships a whole product together with its assets. With HTML selected the row explains that and disables its button, instead of handing over a file that looks broken. Around 40 of Redis Software's pages publish no index.html.md at all, using custom layouts, so the button asks with a HEAD first and names the format that is missing. Verified against /operate/rs/databases/durability-ha/db-availability/, which is one of them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
main gained DOC-6979 (versioned pages in sitemap.xml, #3818), which appends to the same two places in each versioned build job that the download bundles do: after `hugo -d "output"`, and after the artifact upload. Eight conflicts, all of that shape, and none of them a real disagreement -- both sides are additive, so both are kept. Order matters in the first of each pair. main's sitemap generation is a shell command continuing the existing `run:` block, whereas the bundle packaging is a new step, so the sitemap lines stay attached to the build step and the packaging step follows it. In the second of each pair both sides are steps, and the `retention-days: 1` that followed the conflict completes whichever comes last, so the bundle upload gained its own. Checked rather than assumed: - All four versioned build jobs still generate a sitemap, package bundles, and upload both artifacts. Every upload step still carries retention-days. - The artifact name patterns do not collide: deploy_complete_sitemap collects sitemap-*, the deploy jobs collect bundles-*. - The version sitemap is now written inside the versioned subtree before packaging runs, so it could have been swept into the archives. It is not: nothing references it, so the html bundle does not collect it as an asset, and it is not a page directory, so no format packages it. Verified by planting one and packaging. - Both test suites on both sides pass on the merged tree, and the site builds clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
Staging links: |
Contributor
🧠 Redis MemoryFound 5 related items from repository history:
Memory updated at e3e7402 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
… the sidebar The sidebar nav closed a div it had never opened, between the Commands box and </nav>. A browser resolves an unmatched end tag against the nearest matching open element, which here was the outer sidebar wrapper -- so it closed that, and <nav> with it, and treated the literal </nav></div> that followed as stray tags to ignore. The tag predates this branch (it is in c82e18a~1) and was harmless while nothing came after it. Putting the download trigger and dialog there made it matter: both landed outside the sidebar column, under <main>, so `md:mt-auto` had nothing to pin them to the bottom of. Confirmed with a parser that mimics a browser's end-tag recovery rather than by reading the template. Before: the trigger's ancestors were html > body > main > div and the dialog's html > body > main. After: both are inside nav#sidebar. The collapsed nav was checked too and was never affected -- its trigger already sat correctly inside its nav. Reported by Cursor Bugbot. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e3e7402. Configure here.
dwdougherty
approved these changes
Aug 18, 2026
dwdougherty
left a comment
dwdougherty
left a comment
Collaborator
There was a problem hiding this comment.
This is really great, @paoloredis! It works a treat. Approved. I'll add docs back in as a reviewer in case there are others who should review.
Contributor
|
When downloading from the drop-down menu (I chose the Glossary page), is the tar file expected? I was a little surprised by the folder structure for just one page, but I suppose it would make more sense if I selected multiple. |
Contributor
|
I love that you can download the entire set of versioned docs with two clicks! This is AMAZING! |
kaitlynmichael
approved these changes
Aug 18, 2026
kaitlynmichael
left a comment
kaitlynmichael
left a comment
Contributor
There was a problem hiding this comment.
Works when i tested both the top button to download current page with plain .md and the drop down button for a single page and whole version of RS docs. Amazing work @paoloredis
Collaborator
Author
Yes, all product downloads are in tar files, that's expected. It perhaps feels overkill for the |
Collaborator
|
this would fix #2154 as well (not PDF but actually better) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change adds a documentation download widget which allows the user to download documentation for one or more pages in different formats
Preview link:
https://redis.io/docs/staging/DOC-6969/develop/
Note
Medium Risk
Touches production GCS deploy sync rules and ships large generated artifacts; mistakes could break downloads or delete versioned bundles, but changes are scoped to docs hosting with extensive tests.
Overview
Adds a sidebar/page-meta download widget so readers can grab the current page or whole doc products as
.tar.gzarchives in Markdown, single-file Markdown, offline HTML, or JSON.Packaging & data: New
data/doc_bundles.jsondrives both the picker andbuild/make_doc_bundles.py, which builds reproducible per-product bundles (with offline HTML rewriting, exclusions for nested products/archived versions, etc.).Makefilegainsbundlesandserve_downloadsfor local testing.CI/deploy: Each build job packages bundles into separate artifacts; deploy jobs upload them under
docs/.../downloads/bundles/, exclude that path from destructiversync, and pin GCS metadata so.tar.gzfiles are not served with mistakenContent-Encoding: gzip.UI: Replaces the old per-page “Download Markdown” link with the shared dialog; nav/meta links use
data-offline-hideso offline HTML bundles stay usable. Docs updated inai-agent-resources.mdandfor-ais-only/DOC_DOWNLOADS.md, with Python and Node tests for packaging and multi-bundle merge.Reviewed by Cursor Bugbot for commit e3e7402. Bugbot is set up for automated code reviews on this repo. Configure here.