Skip to content

Support collection-level markdown documentation#835

Open
jviotti wants to merge 2 commits intomainfrom
markdown-docs
Open

Support collection-level markdown documentation#835
jviotti wants to merge 2 commits intomainfrom
markdown-docs

Conversation

@jviotti
Copy link
Copy Markdown
Member

@jviotti jviotti commented Apr 22, 2026

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

Comment thread collections/self/v1/schemas/api/list/response.json Outdated
Comment thread docs/api.md Outdated
Comment thread docs/api.md Outdated
Comment thread docs/api.md Outdated
Comment thread docs/api.md Outdated
Comment thread docs/commercial.md Outdated
Comment thread docs/configuration.md Outdated
Comment thread src/build/include/sourcemeta/one/build.h Outdated
Comment thread src/build/delta.cc Outdated
Comment thread src/build/delta.cc Outdated
Comment thread test/cli/index/community/fail-documentation-enterprise-only.sh
Comment thread test/cli/index/enterprise/documentation-basic.sh Outdated
Comment thread test/cli/index/enterprise/documentation-two-collections-same-file.sh Outdated
Comment thread test/cli/index/enterprise/rebuild-documentation-add.sh Outdated
Comment thread test/cli/index/enterprise/rebuild-documentation-modify.sh Outdated
Comment thread test/cli/index/enterprise/rebuild-documentation-modify.sh
Comment thread test/cli/index/enterprise/rebuild-documentation-remove.sh Outdated
Comment thread test/cli/index/enterprise/rebuild-documentation-remove.sh Outdated
Comment thread test/cli/index/enterprise/rebuild-documentation-shared-modify.sh Outdated
Comment thread test/cli/index/enterprise/rebuild-documentation-shared-remove-one.sh Outdated
Comment thread test/cli/index/enterprise/rebuild-documentation-shared-remove-one.sh Outdated
Comment thread test/unit/build/build_delta_test.cc
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Index (enterprise)

Details
Benchmark suite Current: e6ec31e Previous: d78e942 Ratio
Add one schema (0 existing) 21 ms 15 ms 1.40
Add one schema (100 existing) 26 ms 131 ms 0.20
Add one schema (1000 existing) 81 ms 61 ms 1.33
Add one schema (10000 existing) 702 ms 699 ms 1.00
Update one schema (1 existing) 19 ms 14 ms 1.36
Update one schema (101 existing) 26 ms 19 ms 1.37
Update one schema (1001 existing) 80 ms 60 ms 1.33
Update one schema (10001 existing) 673 ms 699 ms 0.96
Cached rebuild (1 existing) 10 ms 8 ms 1.25
Cached rebuild (101 existing) 13 ms 10 ms 1.30
Cached rebuild (1001 existing) 35 ms 27 ms 1.30
Cached rebuild (10001 existing) 282 ms 224 ms 1.26
Index 100 schemas 109 ms 88 ms 1.24
Index 1000 schemas 1245 ms 1035 ms 1.20
Index 10000 schemas 14512 ms 18847 ms 0.77

This comment was automatically generated by workflow using github-action-benchmark.

Comment thread docs/configuration.md Outdated
Comment thread test/unit/build/build_delta_test.cc Outdated
Comment thread test/unit/build/build_delta_test.cc Outdated
Comment thread test/unit/build/build_delta_test.cc Outdated
Comment thread test/unit/build/build_delta_test.cc Outdated
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Index (community)

Details
Benchmark suite Current: e6ec31e Previous: d78e942 Ratio
Add one schema (0 existing) 21 ms 15 ms 1.40
Add one schema (100 existing) 24 ms 20 ms 1.20
Add one schema (1000 existing) 83 ms 60 ms 1.38
Add one schema (10000 existing) 985 ms 552 ms 1.78
Update one schema (1 existing) 18 ms 13 ms 1.38
Update one schema (101 existing) 25 ms 18 ms 1.39
Update one schema (1001 existing) 86 ms 60 ms 1.43
Update one schema (10001 existing) 734 ms 499 ms 1.47
Cached rebuild (1 existing) 9 ms 7 ms 1.29
Cached rebuild (101 existing) 11 ms 8 ms 1.38
Cached rebuild (1001 existing) 35 ms 21 ms 1.67
Cached rebuild (10001 existing) 286 ms 155 ms 1.85
Index 100 schemas 151 ms 104 ms 1.45
Index 1000 schemas 1046 ms 788 ms 1.33
Index 10000 schemas 13427 ms 10721 ms 1.25

This comment was automatically generated by workflow using github-action-benchmark.

Comment thread enterprise/e2e/headless/hurl/documentation.hurl
Comment thread enterprise/e2e/headless/hurl/documentation.hurl
Comment thread enterprise/e2e/html/hurl/documentation.hurl
xpath "count(//div[contains(@class, 'pb-4')]//script)" == 0
xpath "count(//div[contains(@class, 'pb-4')]//iframe)" == 0

# Documentation field does not appear in child entries
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Also test we do not render docs or include the documentation property in other collections

Comment thread src/web/pages/directory.cc Outdated

std::optional<std::string> documentation_content;
if (directory.defines("documentation") && action.dependencies.size() > 1) {
documentation_content = metapack_read_text(action.dependencies.at(1));
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Only attempt to read within the enterprise define

Comment thread src/web/pages/directory.cc Outdated
const auto &directory_path{directory.at("path").to_string()};
const std::filesystem::path relative_path{directory_path.substr(1)};
const auto entry_match{configuration.entries.find(relative_path)};
if (entry_match != configuration.entries.cend() &&
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: I guess we can have a utility in Configuration to check if a given path corresponds to the root of a collection? Plus configuration unit tests

Comment thread src/web/pages/index.cc Outdated
const auto title{configuration.html->name + " Schemas"};
const auto &description{configuration.html->description};

std::optional<std::string> documentation_content;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Delete all of this. The index page NEVER has top level docs

Comment thread src/web/helpers.h Outdated
writer.i().attribute("class", "bi bi-info-circle me-2").close();
writer.text("Add Markdown documentation to this collection. Available in ");
writer.a().attribute(
"href", "https://www.sourcemeta.com/products/one/#commercial-license");
Copy link
Copy Markdown
Member Author

@jviotti jviotti Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Wrong URL. Myst be https://one.sourcemeta.com/commercial/

HTTP 200
Content-Type: application/json
[Asserts]
jsonpath "$.documentation" not exists
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: We must always assert on the ENTIRE output

Comment thread test/e2e/html/hurl/list.hurl
Comment thread src/web/helpers.h Outdated
@jviotti jviotti changed the title [WIP] Support collection-level markdown documentation Support collection-level markdown documentation Apr 23, 2026
@jviotti jviotti marked this pull request as ready for review April 23, 2026 19:34
@augmentcode
Copy link
Copy Markdown

augmentcode Bot commented Apr 23, 2026

🤖 Augment PR Summary

Summary: This PR adds collection-level Markdown documentation support (Enterprise) and a static-file serving mechanism.

Changes:

  • Introduces x-sourcemeta-one:documentation in collection configuration and exposes a documentation field in the directory list API response.
  • Indexes documentation files as metapack “static” artifacts (with a 1 MB limit) and adds a new build action/type to generate them.
  • Adds a new endpoint /self/v1/static/{hash} to serve indexed static files by identifier.
  • Updates build delta logic to detect documentation changes and rebuild affected directory listings/pages accordingly.
  • Renders documentation in the HTML explorer for enterprise builds (Markdown → HTML) and adjusts styling for docs/code blocks.
  • Adds enterprise HTML + headless E2E coverage and CLI/unit tests for documentation behavior and error cases.

Technical Notes: Static file identifiers are derived from the resolved documentation path and are referenced from directory listings as relative URLs.

🤖 Was this summary useful? React with 👍 or 👎

Copy link
Copy Markdown

@augmentcode augmentcode Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 2 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/configuration/read.cc Outdated
const std::filesystem::path documentation_path{documentation_value};
const auto resolved_documentation{std::filesystem::weakly_canonical(
base.parent_path() / documentation_path)};
if (!std::filesystem::exists(resolved_documentation)) {
Copy link
Copy Markdown

@augmentcode augmentcode Bot Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/configuration/read.cc:139 — This only checks std::filesystem::exists(resolved_documentation); if the configured documentation path is a directory or other non-regular/unreadable file, later static-file generation (file_size/ifstream) can throw or fail in less user-friendly ways. Consider validating it’s a regular readable file here so misconfigurations are caught early with a targeted error.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Comment thread src/index/generators.h
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 58 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/configuration/read.cc">

<violation number="1" location="src/configuration/read.cc:139">
P2: Validate documentation paths as regular files, not just existing paths, to prevent build-time filesystem errors when a directory is configured.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread src/configuration/read.cc Outdated
jviotti added 2 commits April 23, 2026 16:10
Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant