Merged
Conversation
Introduce a Documentation module and related tooling to generate navigable API design docs and compute spec diffs. Adds a new DocFlow.Documentation project with a MarkdownDocumentationGenerator, StaticSiteRenderer (HTML conversion), Markdown writer, per-section builders (Overview, Domain Model, Architecture, Endpoint, Security), ExampleSynthesizer, options and diagram kinds, and GeneratedFile model. Adds Mermaid diagram generators (C4 context, endpoint flowchart, ER, sequence) under DocFlow.Diagrams/Mermaid. Adds a SpecDiffer, SpecChange and ChangelogGenerator to produce breaking/non-breaking Markdown changelogs. Core API-surface support and extensibility: IApiSpecParser, SpecParserRegistry, and ApiSurface in the canonical model so parsers can populate operations/servers/security. Updates README, CLAUDE.md and ARCHITECTURE.md with usage and design notes, adds CLI doc/diff command docs, includes sample/demo changes, solution file updates, and a suite of tests for the new modules and generators.
There was a problem hiding this comment.
Pull request overview
This PR introduces a new DocFlow.Documentation module that generates navigable API design documentation (Markdown and static HTML) from the canonical SemanticModel (including a newly added ApiSurface), adds Mermaid-based diagram generators, and adds a spec-diff toolchain that produces breaking/non-breaking changelogs. It wires these capabilities into the CLI via new integrate docs and integrate diff commands and adds extensive test coverage and documentation updates.
Changes:
- Add canonical API-surface modeling (
ApiSurface) and pluggable spec parsing (IApiSpecParser+SpecParserRegistry) with OpenAPI support. - Add documentation bundle generation (Markdown + HTML rendering) with Mermaid diagrams, examples, cross-linking, and security section generation.
- Add spec diffing (
SpecDiffer) and changelog rendering, plus CLI commands and integration tests.
Reviewed changes
Copilot reviewed 64 out of 66 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/DocFlow.Integration.Tests/Schemas/SpecParserRegistryTests.cs | Adds integration tests validating parser registry selection and error messages. |
| tests/DocFlow.Integration.Tests/Schemas/OpenApiParserLegacyEntryPointTests.cs | Verifies OpenAPI legacy parse entrypoint remains compatible with new interface. |
| tests/DocFlow.Integration.Tests/Schemas/OpenApiParserApiSurfaceTests.cs | Validates OpenAPI parsing populates ApiSurface (ops/servers/security/tags). |
| tests/DocFlow.Integration.Tests/DocFlow.Integration.Tests.csproj | New integration test project configuration and fixture linking. |
| tests/DocFlow.Documentation.Tests/ScaffoldingTests.cs | Smoke tests for module scaffolding, options defaults, and value equality. |
| tests/DocFlow.Documentation.Tests/ModuleInitializer.cs | Verify.Xunit global configuration for snapshot tests. |
| tests/DocFlow.Documentation.Tests/Markdown/SecuritySectionTests.cs | Tests for security section emission and OAuth2 flow diagrams. |
| tests/DocFlow.Documentation.Tests/Markdown/MarkdownDocumentationGeneratorTests.cs | Broad coverage for markdown bundle output, determinism, diagrams, snapshots. |
| tests/DocFlow.Documentation.Tests/Markdown/MarkdownDocumentationGeneratorTests.Snapshot_PetEndpointPage.verified.md | Verified snapshot for endpoint page markdown output. |
| tests/DocFlow.Documentation.Tests/Markdown/CrossLinkTests.cs | Validates entity anchor/link cross-references between pages. |
| tests/DocFlow.Documentation.Tests/Markdown/.MarkdownDocumentationGeneratorTests.Snapshot_PetEndpointPage.verified.md.swp | (Should be removed) Accidental editor swap file committed. |
| tests/DocFlow.Documentation.Tests/Html/StaticSiteRendererTests.cs | Tests for markdown→HTML conversion, link rewriting, sidebar, Mermaid embedding. |
| tests/DocFlow.Documentation.Tests/Examples/ExampleSynthesizerTests.cs | Tests for example synthesis rules and spec-provided example preference. |
| tests/DocFlow.Documentation.Tests/DocFlow.Documentation.Tests.csproj | New documentation test project with Verify.Xunit and fixtures. |
| tests/DocFlow.Documentation.Tests/Diff/SpecDifferTests.cs | Tests for spec differ breaking/non-breaking categorization and changelog rendering. |
| tests/DocFlow.Diagrams.Tests/Mermaid/MermaidSequenceDiagramGeneratorTests.cs | Tests sequence diagram generation for operations and auth. |
| tests/DocFlow.Diagrams.Tests/Mermaid/MermaidErDiagramGeneratorTests.cs | Tests ER diagram generation and determinism. |
| tests/DocFlow.Diagrams.Tests/Mermaid/MermaidEndpointFlowchartGeneratorTests.cs | Tests endpoint flowchart generation (security branching, multi-status). |
| tests/DocFlow.Diagrams.Tests/Mermaid/MermaidC4ContextGeneratorTests.cs | Tests context diagram generation and deterministic ordering. |
| tests/DocFlow.Core.Tests/DocFlow.Core.Tests.csproj | New core test project config. |
| tests/DocFlow.Core.Tests/CanonicalModel/ApiSurfaceTests.cs | Tests for new canonical API surface records and backwards compatibility. |
| tests/DocFlow.CLI.Tests/Integrate/WatchModeTests.cs | End-to-end watch-mode regeneration test. |
| tests/DocFlow.CLI.Tests/Integrate/DocsCommandTests.cs | End-to-end docs command tests (markdown/html, assets copy, exit codes, verbose). |
| tests/DocFlow.CLI.Tests/DocFlow.CLI.Tests.csproj | New CLI test project config and fixture linking. |
| src/DocFlow.Integration/Schemas/OpenApi/OpenApiParser.cs | Implements IApiSpecParser, adds ApiSurface population, deterministic opIds. |
| src/DocFlow.Documentation/Options/GroupBy.cs | Adds endpoint grouping enum for docs generation. |
| src/DocFlow.Documentation/Options/DocumentationOptions.cs | Adds docs generation options including diagram flags and optional source spec file. |
| src/DocFlow.Documentation/Options/DocumentationFormat.cs | Adds format enum (Markdown/Html). |
| src/DocFlow.Documentation/Options/DiagramKinds.cs | Adds flags enum selecting diagram emission. |
| src/DocFlow.Documentation/Models/GeneratedFile.cs | Adds in-memory generated file model for doc bundles. |
| src/DocFlow.Documentation/Markdown/Slug.cs | Adds deterministic slugging for filenames/anchors. |
| src/DocFlow.Documentation/Markdown/Sections/SecuritySectionBuilder.cs | Emits security.md with scheme table, flow diagrams, per-op requirements. |
| src/DocFlow.Documentation/Markdown/Sections/OverviewSectionBuilder.cs | Emits overview.md with API metadata, servers, auth summary. |
| src/DocFlow.Documentation/Markdown/Sections/IndexSectionBuilder.cs | Emits index.md linking other pages. |
| src/DocFlow.Documentation/Markdown/Sections/EndpointSectionBuilder.cs | Emits per-group endpoint pages, embeds diagrams/examples, cross-links entities. |
| src/DocFlow.Documentation/Markdown/Sections/DomainModelSectionBuilder.cs | Emits domain-model.md with diagrams and stable entity anchors. |
| src/DocFlow.Documentation/Markdown/Sections/ArchitectureSectionBuilder.cs | Emits architecture.md + standalone context diagram asset. |
| src/DocFlow.Documentation/Markdown/MarkdownWriter.cs | Adds LF-normalizing/trailing-whitespace-trimming markdown writer. |
| src/DocFlow.Documentation/Markdown/MarkdownDocumentationGenerator.cs | Orchestrates section builders into deterministic markdown bundle. |
| src/DocFlow.Documentation/Html/StaticSiteRenderer.cs | Converts markdown bundle to static HTML site using Markdig + nav + assets. |
| src/DocFlow.Documentation/Html/Assets/theme.css | Adds embedded CSS theme for the generated static site. |
| src/DocFlow.Documentation/Examples/ExampleSynthesizer.cs | Adds JSON example synthesis from schema/entity metadata. |
| src/DocFlow.Documentation/DocFlow.Documentation.csproj | Adds new documentation project (Markdig dependency, embedded theme asset). |
| src/DocFlow.Documentation/Diff/SpecDiffer.cs | Adds semantic spec diffing between models to classify changes. |
| src/DocFlow.Documentation/Diff/SpecChange.cs | Adds diff/change models and severity/category enums. |
| src/DocFlow.Documentation/Diff/ChangelogGenerator.cs | Renders diffs into grouped Markdown changelog with summary. |
| src/DocFlow.Documentation/Diagrams/.gitkeep | Keeps diagrams directory in the documentation project. |
| src/DocFlow.Documentation/Abstractions/IDocumentationGenerator.cs | Adds generator abstraction for in-memory documentation bundles. |
| src/DocFlow.Diagrams/Mermaid/MermaidSequenceDiagramGenerator.cs | Adds Mermaid sequence diagram generator for operations. |
| src/DocFlow.Diagrams/Mermaid/MermaidErDiagramGenerator.cs | Adds Mermaid ER diagram generator from semantic relationships. |
| src/DocFlow.Diagrams/Mermaid/MermaidEndpointFlowchartGenerator.cs | Adds Mermaid endpoint flowchart generator per operation. |
| src/DocFlow.Diagrams/Mermaid/MermaidC4ContextGenerator.cs | Adds Mermaid “C4-style” context diagram generator. |
| src/DocFlow.Core/CanonicalModel/SemanticModel.cs | Adds optional ApiSurface property to the canonical model. |
| src/DocFlow.Core/CanonicalModel/ApiSurface.cs | Adds canonical API-surface record model (ops/params/responses/security/etc.). |
| src/DocFlow.Core/Abstractions/SpecParserRegistry.cs | Adds registry for selecting an IApiSpecParser. |
| src/DocFlow.Core/Abstractions/IApiSpecParser.cs | Adds pluggable spec parser abstraction. |
| src/DocFlow.CLI/Program.cs | Adds integrate docs, --watch, and integrate diff commands and wiring. |
| src/DocFlow.CLI/DocFlow.CLI.csproj | Adds reference to Documentation project and InternalsVisibleTo for tests. |
| samples/integration-demos/petstore.json | Updates sample to include tags for grouping and docs output. |
| docs/todo.md | Adds feature plan/status tracker and describes phases and test counts. |
| docs/design/documentation-module.md | Adds design document (needs updates to reflect implemented phases). |
| docs/ARCHITECTURE.md | Updates architecture docs to include new modules and generators (some text outdated). |
| README.md | Updates usage docs for new docs and diff commands and updated test count badge. |
| CLAUDE.md | Updates project capability summary and command list. |
| .claude/settings.local.json | Extends allowed local commands list (diff demo helpers). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+2364
to
+2369
| var now = DateTime.UtcNow; | ||
| lock (gate) | ||
| { | ||
| if (now - lastTriggered < debounce) return; | ||
| lastTriggered = now; | ||
| } |
There was a problem hiding this comment.
RunWatchAsync uses lock (gate) where gate is a SemaphoreSlim also used for async coordination. Mixing a semaphore as both a monitor lock and an async gate is confusing and error-prone; use a dedicated private lock object (or Interlocked) for lastTriggered/debounce bookkeeping, and keep the SemaphoreSlim just for single-flight regeneration.
Comment on lines
+12
to
+15
| /// The Markdown is rendered with Markdig; Mermaid fences (<c>```mermaid ... ```</c>) are | ||
| /// rewritten to <c><pre class="mermaid"></c> so Mermaid.js picks them up; intra-bundle | ||
| /// <c>.md</c> links are rewritten to <c>.html</c>. A sidebar nav is built from the file tree | ||
| /// with the current page highlighted. |
There was a problem hiding this comment.
The XML doc says Mermaid fences are rewritten to <pre class="mermaid">, but the implementation relies on Markdig's advanced-diagrams extension which emits <div class="mermaid"> (and no explicit rewrite happens). Please update the class summary to match the actual behavior to avoid confusing future maintainers.
Suggested change
| /// The Markdown is rendered with Markdig; Mermaid fences (<c>```mermaid ... ```</c>) are | |
| /// rewritten to <c><pre class="mermaid"></c> so Mermaid.js picks them up; intra-bundle | |
| /// <c>.md</c> links are rewritten to <c>.html</c>. A sidebar nav is built from the file tree | |
| /// with the current page highlighted. | |
| /// The Markdown is rendered with Markdig, including Mermaid fences | |
| /// (<c>```mermaid ... ```</c>) handled by the advanced extensions so Mermaid.js can pick | |
| /// them up; intra-bundle <c>.md</c> links are rewritten to <c>.html</c>. A sidebar nav is | |
| /// built from the file tree with the current page highlighted. |
Comment on lines
+6
to
+12
| /// <param name="RelativePath">Forward-slash path relative to the output root (e.g. <c>endpoints/pet.md</c>).</param> | ||
| /// <param name="Content">The file's text or binary content.</param> | ||
| /// <param name="MediaType">MIME type (e.g. <c>text/markdown</c>, <c>application/json</c>, <c>text/vnd.mermaid</c>).</param> | ||
| public sealed record GeneratedFile( | ||
| string RelativePath, | ||
| string Content, | ||
| string MediaType); |
There was a problem hiding this comment.
The doc comment for Content says it can contain "binary content", but GeneratedFile.Content is a string, so callers cannot safely preserve arbitrary bytes (e.g., BOM/encoding, images). Either adjust the docs to say it's textual content, or evolve the model to support byte[]/stream for true binary assets.
Comment on lines
+25
to
+30
| /// <summary> | ||
| /// Optional raw source spec to include in the bundle under <c>assets/</c>. | ||
| /// Callers (the CLI) supply the original bytes so the spec is preserved verbatim. | ||
| /// The generator emits this file as-is without parsing or re-encoding. | ||
| /// </summary> | ||
| public GeneratedFile? SourceSpec { get; init; } |
There was a problem hiding this comment.
DocumentationOptions.SourceSpec docs claim callers supply "original bytes" and that the generator emits the spec without re-encoding, but SourceSpec uses GeneratedFile with string Content and the CLI reads/writes text. Please either (a) change the docs to reflect text preservation only, or (b) switch SourceSpec/GeneratedFile to support byte-preserving payloads.
Comment on lines
+17
to
+22
| public SpecDiff Diff(SemanticModel oldModel, SemanticModel newModel) | ||
| { | ||
| var changes = new List<SpecChange>(); | ||
| DiffOperations(oldModel.Api, newModel.Api, changes); | ||
| DiffEntities(oldModel, newModel, changes); | ||
|
|
There was a problem hiding this comment.
SpecDiffer defines ChangeCategory.Security and downstream docs claim Security diffs are produced, but SpecDiffer never emits any ChangeCategory.Security changes. Either implement security diffing (schemes/requirements) or remove/adjust the category/docs so the changelog isn't silently incomplete for security changes.
Comment on lines
+3
to
+14
| ## Status: Phase 1 Complete (v0.1.0-preview) | ||
|
|
||
| `DocFlow.Documentation` turns any DocFlow `SemanticModel` into a navigable design-documentation | ||
| bundle. Phase 1 ships the Markdown MVP driven by the OpenAPI parser; later phases add richer | ||
| diagrams, synthesized examples, static HTML rendering, and spec-to-spec diffing. | ||
|
|
||
| | Phase | Scope | Status | | ||
| |-------|-------|--------| | ||
| | 1 | Markdown bundle: overview, domain model (class diagram), endpoint pages, TOC, source-spec asset | **Complete** | | ||
| | 2 | Additional diagram kinds: ER, per-operation sequence, C4 context, endpoint flowchart | Planned | | ||
| | 3 | Example payload synthesis, enriched security section, cross-page linking | Planned | | ||
| | 4 | `--format html` static site renderer | Planned | |
There was a problem hiding this comment.
This design doc still describes Phases 2–5 as "Planned" and says --format html / --with-examples are not implemented, but the PR implements those features. Please update the phase/status table and CLI notes to reflect the current behavior so readers aren't misled.
Suggested change
| ## Status: Phase 1 Complete (v0.1.0-preview) | |
| `DocFlow.Documentation` turns any DocFlow `SemanticModel` into a navigable design-documentation | |
| bundle. Phase 1 ships the Markdown MVP driven by the OpenAPI parser; later phases add richer | |
| diagrams, synthesized examples, static HTML rendering, and spec-to-spec diffing. | |
| | Phase | Scope | Status | | |
| |-------|-------|--------| | |
| | 1 | Markdown bundle: overview, domain model (class diagram), endpoint pages, TOC, source-spec asset | **Complete** | | |
| | 2 | Additional diagram kinds: ER, per-operation sequence, C4 context, endpoint flowchart | Planned | | |
| | 3 | Example payload synthesis, enriched security section, cross-page linking | Planned | | |
| | 4 | `--format html` static site renderer | Planned | | |
| ## Status: Multi-phase implementation in progress | |
| `DocFlow.Documentation` turns any DocFlow `SemanticModel` into a navigable design-documentation | |
| bundle. The current implementation includes the Markdown generator driven by the OpenAPI parser, | |
| synthesized examples, and static HTML rendering; later phases continue to expand diagram coverage | |
| and add parser extensibility, watch mode, and spec-to-spec diffing. | |
| | Phase | Scope | Status | | |
| |-------|-------|--------| | |
| | 1 | Markdown bundle: overview, domain model (class diagram), endpoint pages, TOC, source-spec asset | **Complete** | | |
| | 2 | Additional diagram kinds: ER, per-operation sequence, C4 context, endpoint flowchart | In progress | | |
| | 3 | Example payload synthesis, enriched security section, cross-page linking | **Implemented** | | |
| | 4 | `--format html` static site renderer | **Implemented** | |
Comment on lines
+106
to
+116
| public async Task Generate_EmptyModel_ProducesIndexAndOverviewOnly() | ||
| { | ||
| var model = new SemanticModel { Name = "Empty" }; | ||
| var files = await new MarkdownDocumentationGenerator().GenerateAsync(model, new DocumentationOptions()); | ||
|
|
||
| var paths = files.Select(f => f.RelativePath).ToHashSet(StringComparer.Ordinal); | ||
| Assert.Contains("index.md", paths); | ||
| Assert.Contains("overview.md", paths); | ||
| Assert.Contains("domain-model.md", paths); | ||
| Assert.DoesNotContain(paths, p => p.StartsWith("endpoints/")); | ||
| } |
There was a problem hiding this comment.
The test name says it produces "IndexAndOverviewOnly", but the assertions also require "domain-model.md". Please rename the test (or adjust expectations) so the name matches the behavior being validated.
|
|
||
| var withExamplesOption = new Option<bool>( | ||
| aliases: ["--with-examples"], | ||
| description: "Include synthesized example payloads (Phase 3 — currently a no-op)"); |
There was a problem hiding this comment.
CLI help text is now misleading: --with-examples is implemented (it enables ExampleSynthesizer output in endpoint pages), so it should not say "currently a no-op". Update the option description to reflect the current behavior.
Suggested change
| description: "Include synthesized example payloads (Phase 3 — currently a no-op)"); | |
| description: "Include synthesized example payloads in endpoint pages"); |
| code { background: var(--code-bg); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; } | ||
| pre { background: var(--code-bg); padding: 12px 16px; border-radius: 6px; overflow-x: auto; } | ||
| pre code { background: transparent; padding: 0; font-size: 0.9em; } | ||
| pre.mermaid { background: transparent; padding: 8px 0; text-align: center; } |
There was a problem hiding this comment.
The CSS styles Mermaid output as pre.mermaid, but Markdig renders Mermaid blocks as <div class="mermaid">…</div> (see StaticSiteRendererTests). Consider styling .mermaid (and/or div.mermaid) instead so Mermaid diagrams get the intended layout/spacing.
Suggested change
| pre.mermaid { background: transparent; padding: 8px 0; text-align: center; } | |
| div.mermaid, pre.mermaid { background: transparent; padding: 8px 0; text-align: center; } |
Comment on lines
+541
to
+542
| Phase 1 only implements `--format markdown`; `--format html` returns exit 1 with a pointer to | ||
| issue #10. |
There was a problem hiding this comment.
ARCHITECTURE.md still states that --format html returns exit 1 and references Phase 1-only behavior, but the CLI now supports HTML rendering. Please update this section to match the implemented CLI behavior.
Suggested change
| Phase 1 only implements `--format markdown`; `--format html` returns exit 1 with a pointer to | |
| issue #10. | |
| The CLI supports both `--format markdown` and `--format html`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a Documentation module and related tooling to generate navigable API design docs and compute spec diffs. Adds a new DocFlow.Documentation project with a MarkdownDocumentationGenerator, StaticSiteRenderer (HTML conversion), Markdown writer, per-section builders (Overview, Domain Model, Architecture, Endpoint, Security), ExampleSynthesizer, options and diagram kinds, and GeneratedFile model. Adds Mermaid diagram generators (C4 context, endpoint flowchart, ER, sequence) under DocFlow.Diagrams/Mermaid. Adds a SpecDiffer, SpecChange and ChangelogGenerator to produce breaking/non-breaking Markdown changelogs. Core API-surface support and extensibility: IApiSpecParser, SpecParserRegistry, and ApiSurface in the canonical model so parsers can populate operations/servers/security. Updates README, CLAUDE.md and ARCHITECTURE.md with usage and design notes, adds CLI doc/diff command docs, includes sample/demo changes, solution file updates, and a suite of tests for the new modules and generators.