[bradystroud]

Description

✏️ Relates to #2801. The production server renders a rule page again on almost each request, and each render sends 17 requests to TinaCloud. After this change, the server renders a page only after a content change (Tina webhook) or in a deploy.

Problem. Two settings together put all rule pages on a 60 second revalidation cycle.

• The layout query. components/layout/layout.tsx sent the global settings query with next: { revalidate: 60 }.
  • The value is not a decision for this site. It came with the Tina starter template in the first commit (8d96be7), and the file is the same in tinacms/tina-nextjs-starter today. Fix: add diagnostics for Rule loading failures #2705 only moved the lines.
  • In a build, the Tina client disk cache (cacheDir) answers this query for almost all pages. Next.js then does not see the option. In a local build of main, 20 of 4,192 pages got the 60 seconds.
  • At run time that cache is not available. Each render sends the query, so a page that the server renders one time gets s-maxage=60.
• The giscus cache. giscusDiscussions.service.ts cached the discussion numbers with revalidate: 6 hours (fix: map giscus comments by discussion number instead of search #2776, in production from 12 Nov). Next.js copies a numeric revalidate of unstable_cache to each page that calls the function. Each rule page calls it.
  • Each rule page thus became stale 6 hours after the build. The first request after that time rendered the page at run time, and the page got the 60 seconds.
• A category page calls neither, so it keeps s-maxage=31536000. It gets the 60 seconds when the webhook revalidates it.
• The home, categories, search, user, archived and orphaned pages declare 5 minutes or 6 hours. Production serves each of them with s-maxage=60.
• The global settings are not shown on a public page. The only reader is the icon field of the Tina admin. global/index.json changed 2 times in 12 months.

Changes

• Commit 1, lib/services/github/giscusDiscussions.service.ts: the cache has no period. The numbers are read again in each build, and when the webhook revalidates a rule page. That render does not use a cache entry that is older than the revalidation (measured: 2 GitHub requests in that render).
• Commit 2:
  • lib/services/global/globalSettings.service.ts reads the global settings through unstable_cache with the tag global-settings and no period.
    • It uses unstable_cache and not fetch options, because the Tina disk cache hides fetch options from Next.js in a build. With unstable_cache, 4,180 of 4,180 pages that have the layout carry the tag.
    • unstable_cache stores only a result. A throttled TinaCloud response (HTTP 200 with an error) is thus not stored, which was possible with the fetch option.
  • components/layout/layout.tsx uses that function.
  • app/api/revalidate/route.ts marks the tag stale when a file in global/ changes. It uses "max" (stale while revalidate), because the tag is on each page. A visitor gets the current page, and Next.js renders the new one in the background.

Checks

Local production builds with the local Tina server (tinacms build --local, client with the disk cache as in CI), 4,192 pages, standalone server. A local stub answers the giscus lookup, so the lookup works as in CI.

	Before (2741dc4)	After
Rule pages with a 6 hour period from the build	❌ 3,913	✅ 0
Rule pages with a 60 second period from the build	❌ 20	✅ 0
Rule pages with no period (s-maxage=31536000)	1	✅ 3,934
Category pages with no period	238	238
Rule page, 7 hours after the build	❌ STALE, 17 Tina requests, then s-maxage=60	✅ HIT, 0 requests
Same page, more than 60 seconds later	❌ STALE, 17 Tina requests	✅ HIT, 0 requests
Rule or category edit + webhook	new content, then s-maxage=60	✅ new content, s-maxage=31536000 stays
Global settings edit + webhook	not handled (60 second cycle)	✅ each page: STALE one time, then the new settings
/search, /user, /archived, /orphaned after a run time render	❌ s-maxage=60	✅ s-maxage=300
/ and /categories after a run time render	❌ s-maxage=60	✅ s-maxage=21600
Tina requests in the build	8,167	8,171

• The header before the change is the same as on production: cache-control: s-maxage=60, stale-while-revalidate=31535940.
• Production, 12 Nov 22:07 to 22:15 UTC: 3 of 3 rule pages had s-maxage=60 and STALE. The home, categories, search, user and archived pages had s-maxage=60.
• One change only, 60 to 137 in the compiled layout chunk of main, changed the page header to s-maxage=137.
• 16 new unit tests. The full suite passes (248 tests). biome lint and tsc --noEmit pass. tina-lock.json has no diff.

Options for the period (product decision)

Option	Editors	Visitors and TinaCloud load
A. Commit 1 only. The layout query keeps 60 seconds.	No change from before 12 Nov. A page that the server rendered at run time renders again each 60 seconds on request, so a wrong render repairs itself.	No render for a rule page that nobody edited. The 60 second cycle stays for edited pages, for unknown paths and for the 6 other routes.
B. Both commits (this PR). No period.	See the next section.	No render for a page that did not change.
C. One hour for run time renders only (next: { revalidate: 3600 } on the query).	As A, with 1 hour.	An edited page renders again each hour on request. It depends on the Tina disk cache that hides the option in the build.
D. A period for all pages (a number in unstable_cache).	Each change shows after the period with no webhook.	Each page renders again after the period. In the current image (see below), a page that is past its period and not in memory renders on each request. 24 hours is the time of the daily build. Not recommended.

What changes for editors and visitors (option B)

• No change: a rule edit, a new rule and a category edit show on the next request after the webhook.
• New: a global settings edit shows after the webhook. Each page renders one time, on its next request.
• Slower: content that a page shows but the webhook does not revalidate for that page. Examples: the mega menu, the home page text (homepage/index.json), the title of a rule in a category list or in a related rule card.
  • Before: approximately 60 seconds on a page that was in the cycle, the next deploy on other pages.
  • After: the next deploy (06:00 UTC each day) for rule and category pages. The period that the page declares for the other routes (6 hours for / and /categories, 5 minutes for search, user, archived and orphaned).
• Risk: a page that the server renders wrong at run time now stays until the next webhook for that page or the next deploy. Before, the 60 seconds repaired it. See the merge order.
• Accepted trade: a restart brings back build-time HTML. A page that the webhook rendered again lives only in the container (no storage mount, and the server cannot write to /app). After a container restart, an edited page shows its build-time HTML again until the next webhook for that page or the next deploy. Before, the 6 hour and 60 second periods repaired that. The daily 06:00 UTC deploy limits the window to about 24 hours. The alternative is a long period, for example 24 hours, on the rule pages.

Effect of commit 1 on the comments (giscus)

The stated intent in giscusDiscussions.service.ts was: "the number refreshes on each deploy, or when a rule edit revalidates its page". Commit 1 makes the code do that.

• Before: the page rendered again 6 hours after the build and then each 60 seconds on request. Each render used a list of discussion numbers that was 6 hours old at most. A new discussion number thus got to a requested rule page in 6 hours or less.
• After: a new discussion number gets to the rule page in the next deploy (06:00 UTC each day), or when the webhook revalidates that rule. That render reads the list from GitHub again.
• A rule whose discussion is not in the list has discussionNumber: null. components/Discussion.tsx then uses mapping="specific" with the rule GUID, which is the behavior before fix: map giscus comments by discussion number instead of search #2776. The comments show when the GitHub discussion search works. A rule gets its discussion with its first comment, so this applies to a rule that got its first comment after the last deploy.

Merge order

• Commit 1 is safe alone. It puts back the behavior of the builds before 12 Nov.
• Merge commit 2 after 🐛 Rule pages - Retry throttled Tina requests and render the fallback only for a missing record #2802. On main, a failed Tina request at run time gives a NOT FOUND fallback page. With commit 2 that page has no period. 🐛 Rule pages - Retry throttled Tina requests and render the fallback only for a missing record #2802 stops the fallback for a failed request.
• After 🐛 Rule pages - Retry throttled Tina requests and render the fallback only for a missing record #2802, one case remains: a record that TinaCloud does not have when the page renders (for example a new rule that is not indexed). A 60 second period for the fallback render only solves it. That is a change in app/[filename]/page.tsx, which 🐛 Rule pages - Retry throttled Tina requests and render the fallback only for a missing record #2802 owns.

Not in this PR

• The Docker image cannot write the page cache. The runner stage has USER nextjs before COPY --from=builder with no --chown, so /app belongs to root. The server keeps a re-rendered page only in memory (50 MB). An edited page that is not in memory renders again on each request until the next deploy.
  • --chown=nextjs:nodejs on the three COPY --from=builder lines fixes it (tested in a small image that has the same commands).
  • Risk 1: if the server can create /app/tina/__generated__/.cache, the Tina client answers each query from disk, and an edit does not show after the webhook (measured in the local build). With --chown on the COPY lines, /app itself stays owned by root and the folder cannot be created (tested in the small image). This holds only while the standalone output has no tina folder.
  • Risk 2: with the page code on main, a fallback page from a failed Tina request is then written to disk. Make this change after 🐛 Rule pages - Retry throttled Tina requests and render the fallback only for a missing record #2802.
• 12 of the 17 Tina requests of a rule page render are the category lookup: getFullRelativePathFromFilename (app/[filename]/page.tsx:19-56) reads all 6 pages of topCategoryWithIndexQuery for a path that is not a category, and the page and generateMetadata each call it. 🐛 Rule pages - Retry throttled Tina requests and render the fallback only for a missing record #2802 makes it 6. category-uri-title-map.json cannot replace it as it is: its keys are the uri field and not the file name, it has no top category folder, and it has 238 of the 239 categories.
• /latest-rules reads searchParams, so it is dynamic and its revalidate = 300 has no effect.

Screenshot (optional)

✏️ Not applicable. The pages do not change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UTtNDpWnuaUUQ2winSSgMb

[bradystroud]

🎬 Explainer video for this pull request (about 40 seconds, no sound): the problem, the change, and how it helps.

🤖 Generated with Claude Code

pr-2805.mp4

[bradystroud]

Updated to the latest `main` (9de9506).

• `components/layout/layout.tsx` had a conflict with 🐛 Rule pages - Retry throttled Tina requests and render the fallback only for a missing record #2802. The layout now calls `getGlobalSettings()` as before in this pull request.
• `fetchGlobalSettings` now sends the query through `withTinaRetry` and `tinaRetryOptions` from 🐛 Rule pages - Retry throttled Tina requests and render the fallback only for a missing record #2802, so a throttled global settings query is tried again. It has no `revalidate`, as before.
• The test `rejects when the Tina request fails` is now `tries a throttled request again, then rejects`. It checks 5 attempts with fake timers.

🤖 Generated with Claude Code