fix(docs): auto-inject git-derived last-updated into markdown frontmatter

[aditya-arolkar-swe]

Description

Requested by: @aditya-arolkar-swe
Link to Devin Session

During fern docs publish, automatically sets the last-updated frontmatter field on markdown pages using per-file git history. This field is already read by the platform to populate <lastmod> in sitemaps — this PR automates keeping it current.

Behavior with existing last-updated frontmatter

Scenario	Behavior
Page has no last-updated	Injected from git
Page has last-updated and git history exists	Overwritten with git date
Page has last-updated but no git history (untracked file, non-git env)	Left unchanged
API-generated page (OpenAPI tag descriptions)	Skipped entirely — never injected

Git is the single source of truth. User-specified last-updated values are always replaced when git history exists, so the field never goes stale. Whitespace-only commits (formatting, indentation) are ignored via git log -G'[^\s]' to avoid SEO noise from trivial edits.

Design decisions

• Whitespace-aware: git log -1 -G'[^\s]' --format=%cs — only non-whitespace changes count
• Content-type filtering: OpenAPI-generated pages excluded via excludePaths (one spec → N pages with same timestamp = Google distrust)
• Concurrency-limited: asyncPool (cap: 50) to avoid EMFILE on large doc sites
• In-memory only: Source .mdx files on disk are never modified

Changes Made

• injectLastUpdated.ts (new) — Core utility: getGitLastModifiedDate() queries git with -G'[^\s]' whitespace filtering, injectLastUpdatedIntoMarkdown() / replaceLastUpdatedInMarkdown() handle frontmatter manipulation (including CRLF), injectLastUpdatedDates() orchestrates everything
• asyncPool.ts (new) — Concurrency-limited async task runner with .finally() cleanup
• DocsDefinitionResolver.ts — Tracks apiGeneratedPagePaths, calls injectLastUpdatedDates() after image-path replacement
• injectLastUpdated.test.ts (new) — 24 unit tests
• versions.yml — Added v4.20.5 changelog entry
• Updated README.md generator (if applicable) — N/A

Testing

• Unit tests added/updated — 24 passing
• All CI checks passing (35/35)
• Manual testing completed

Review checklist:

• Verify downstream sitemap generator converts "Month Day, Year" → W3C Datetime (YYYY-MM-DD) for <lastmod> XML. If not, all <lastmod> values will be invalid.
• Confirm -G '[^\s]' performance is acceptable on large doc repos (more expensive than plain git log -1)
• Confirm always-overwriting user-specified last-updated with git dates is acceptable

[github-actions]

🌱 Seed Test Selector

Select languages to run seed tests for:

• Python
• TypeScript
• Java
• Go
• Ruby
• C#
• PHP
• Swift
• Rust
• OpenAPI
• Postman

---

How to use: Click the ⋯ menu above → "Edit" → check the boxes you want → click "Update comment". Tests will run automatically and snapshots will be committed to this PR.

[graphite-app]

The regex and string manipulation logic has a bug with CRLF line endings. The regex allows \r?\n for line endings, but line 56 uses lastIndexOf("\n---") which doesn't account for \r\n. On Windows systems with CRLF line endings, if frontmatter is ---\r\ntitle: Foo\r\n---, the lastIndexOf("\n---") will find the wrong position because it searches for \n--- but the actual pattern is \r\n---. This causes the last-updated field to be inserted at the wrong position, potentially corrupting the frontmatter.

Fix:

const closingIdx = matchStr.lastIndexOf("---");
const insertPos = frontmatterMatch.index + closingIdx;

Or handle both line ending types:

const closingMatch = /\r?\n---$/.exec(matchStr);
const insertPos = frontmatterMatch.index + (closingMatch?.index ?? 0);

Suggested change

	const frontmatterMatch = /^---\r?\n([\s\S]*?\r?\n)?---(\r?\n|$)/.exec(markdown);
	if (frontmatterMatch != null) {
	// Find the last occurrence of '\n---' within the match to locate the closing delimiter
	const matchStr = frontmatterMatch[0];
	const closingIdx = matchStr.lastIndexOf("\n---");
	const insertPos = frontmatterMatch.index + closingIdx;
	return markdown.slice(0, insertPos) + `\nlast-updated: ${date}` + markdown.slice(insertPos);
	const frontmatterMatch = /^---\r?\n([\s\S]*?\r?\n)?---(\r?\n|$)/.exec(markdown);
	if (frontmatterMatch != null) {
	// Find the last occurrence of '\n---' within the match to locate the closing delimiter
	const matchStr = frontmatterMatch[0];
	const closingMatch = /\r?\n---$/.exec(matchStr);
	const insertPos = frontmatterMatch.index + (closingMatch?.index ?? 0);
	return markdown.slice(0, insertPos) + `\nlast-updated: ${date}` + markdown.slice(insertPos);

Spotted by Graphite



Is this helpful? React 👍 or 👎 to let us know.

[graphite-app]

Critical bug: Promise cleanup fails on rejection

The .then() callback only runs on successful promise resolution. If fn(item, i) throws an error or the git command fails, the promise will reject and never be removed from the executing set. This causes:

1. Memory leak - rejected promises accumulate in the set
2. Incorrect concurrency limiting - the pool thinks it's at capacity when rejected promises still occupy slots
3. Potential hang - if enough promises fail, Promise.race() keeps racing the same rejected promises

Fix: Use .finally() instead of .then() to ensure cleanup happens on both success and failure:

const promise = (async () => {
    const result = await fn(item, i);
    results[i] = result;
})().finally(() => {
    executing.delete(promise);
});

This is critical because git commands can fail (permissions, corrupted repo, missing files), and the concurrent processing will break when processing large documentation sites.

Suggested change

	const promise = (async () => {
	const result = await fn(item, i);
	results[i] = result;
	})().then(() => {
	executing.delete(promise);
	});
	const promise = (async () => {
	const result = await fn(item, i);
	results[i] = result;
	})().finally(() => {
	executing.delete(promise);
	});

Spotted by Graphite



Is this helpful? React 👍 or 👎 to let us know.

[graphite-app]

Critical logic bug: The code unconditionally overwrites existing last-updated dates without checking if the git date is actually newer.

The PR description states: "Git timestamps override stale user-specified dates when the file has been modified more recently." However, the implementation always replaces the existing date with the git date, even if the user-specified date is newer.

Fix: Parse and compare dates before overwriting:

if (hasLastUpdated(markdown)) {
    const existingDate = getExistingLastUpdated(markdown);
    if (existingDate != null) {
        const existingParsed = new Date(existingDate);
        const gitParsed = new Date(date);
        // Only replace if git date is newer than existing date
        if (!isNaN(gitParsed.getTime()) && !isNaN(existingParsed.getTime()) && gitParsed > existingParsed) {
            result[key] = replaceLastUpdatedInMarkdown(markdown, date);
        } else {
            result[key] = markdown; // Keep existing date
        }
    } else {
        result[key] = markdown;
    }
} else {
    result[key] = injectLastUpdatedIntoMarkdown(markdown, date);
}

Without this fix, manually-set future dates (e.g., for scheduled content) will be incorrectly overwritten with older git dates.

Suggested change

	// Set (or overwrite) last-updated with the git-derived date.
	if (hasLastUpdated(markdown)) {
	result[key] = replaceLastUpdatedInMarkdown(markdown, date);
	} else {
	result[key] = injectLastUpdatedIntoMarkdown(markdown, date);
	}
	// Set (or overwrite) last-updated with the git-derived date.
	if (hasLastUpdated(markdown)) {
	const existingDate = getExistingLastUpdated(markdown);
	if (existingDate != null) {
	const existingParsed = new Date(existingDate);
	const gitParsed = new Date(date);
	// Only replace if git date is newer than existing date
	if (!isNaN(gitParsed.getTime()) && !isNaN(existingParsed.getTime()) && gitParsed > existingParsed) {
	result[key] = replaceLastUpdatedInMarkdown(markdown, date);
	} else {
	result[key] = markdown; // Keep existing date
	}
	} else {
	result[key] = markdown;
	}
	} else {
	result[key] = injectLastUpdatedIntoMarkdown(markdown, date);
	}

Spotted by Graphite



Is this helpful? React 👍 or 👎 to let us know.