Add AI-powered documentation generation (core) by mjwolf · Pull Request #3302 · elastic/elastic-package

mjwolf · 2026-02-24T02:15:30Z

Add the elastic-package update documentation command that uses an LLM (Gemini) to generate package documentation via a section-based generate-validate workflow.

Core components:

docagent: orchestrates documentation generation with per-section validation loops, response analysis, and section normalization
executor: ADK-based LLM executor with tool support
workflow: multi-agent pipeline (generator, critic, validators)
validators: static and LLM-based validators for structure, completeness, accuracy, accessibility, style, placeholders, URLs
parsing: markdown section parser and combiner
prompts: embedded prompt templates and section-specific instructions
tools: package file tools (list_directory, read_file, write_file, get_example, get_service_info)
postprocessor: data stream template and agentless section enforcement

Co-authored-by: Jonathan Molinatto <jonathan.molinatto@gmail.com> Co-authored-by: Quan Nguyen <quan.nguyen@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com>

jrmolin · 2026-02-26T15:37:36Z

internal/llmagent/docagent/agents/critic.go

+## Evaluation Criteria
+Evaluate against these criteria (rate each 1-10):
+
+1. **Voice/Tone**: Friendly, uses "you", contractions, active voice


make sure this coincides with the vale rules

jrmolin · 2026-02-26T15:38:06Z

internal/llmagent/docagent/agents/critic.go

+3. **Grammar**: American English, present tense, Oxford comma, sentence case headings
+   - Sentence case: "### Vendor-specific issues" NOT "### Vendor-Specific Issues"
+
+4. **Structure**: Clear summary, scannable sections, short paragraphs


maybe add the avoid 'via' here?

jrmolin · 2026-02-26T15:42:10Z

internal/llmagent/docagent/agents/registry.go

+	if _, exists := r.agents[name]; !exists {
+		r.order = append(r.order, name)
+	}
+	r.agents[name] = agent


could probably move this inside the if block, too

No, this is right as it is. If the agent already exists with the name, the last line will replace it with the new one. If it was moved in, this couldn't modify the agent if the name already exists.

jrmolin · 2026-02-26T15:55:12Z

internal/llmagent/docagent/prompts/_static/agent_instructions.md

@@ -0,0 +1,142 @@
+# Style guide for Elastic integration documentation


is this still used? i saw a similar prompt in the generator code

It was still needed, it's used as the system prompt when making revisions.

I think it wasn't in a good location though, it's not clear what the use was. I reorganized all the prompts to keep them closer to the location where they're used, and I've added more of the Vale rules to the generator and validator instructions, so a lot of the code around the prompts has been changed and reorganized now.

jrmolin · 2026-02-26T18:33:48Z

internal/llmagent/docagent/docagent.go

+// ConfirmInstructionsUnderstood asks the LLM to confirm it understood the system instructions.
+// This should be called before any documentation workflow to ensure proper adherence.
+func (d *DocumentationAgent) ConfirmInstructionsUnderstood(ctx context.Context) error {
+	fmt.Println("🔍 Verifying LLM understands documentation guidelines...")


have you found this necessary? it seems like it would just waste context and another LLM call.

No, I haven't ever seen a difference without this, and I've never seen the LLM respond that it doesn't understand. I'll take out this prompt.

jrmolin · 2026-02-26T18:49:32Z

internal/llmagent/docagent/docagent.go

+		}
+
+		// Skip AI-generated notice
+		if strings.HasPrefix(line, "> **Note**: This documentation was generated") {


this might need updated

I've updated this to the new prompt

jrmolin · 2026-02-26T18:52:47Z

internal/llmagent/docagent/docagent.go

+// GenerateAllSectionsWithWorkflow generates all sections using the multi-agent workflow.
+// This method uses a configurable pipeline of agents (generator, critic, validator, etc.)
+// to iteratively refine each section. Sections are generated in parallel.
+func (d *DocumentationAgent) GenerateAllSectionsWithWorkflow(ctx context.Context, workflowCfg workflow.Config) ([]Section, error) {


this doesn't seem to be used. is this used somewhere?

No, it wasn't used, I've removed it.

vinit-chauhan · 2026-02-26T22:05:56Z

internal/packages/archetype/_static/package-docs-readme.md.tmpl

 */}}

-## Scaling
+### Vendor resources


Are we keeping links in only 2 places, next to setup steps and at the end. So should we remove this?

Yeah, I removed it. There's almost never troubleshooting specific documentation available, so this section isn't useful.

vinit-chauhan · 2026-02-26T22:13:36Z

cmd/update.go

+	cmd.AddCommand(updateDocumentationCmd)
+	cmd.PersistentFlags().StringP(cobraext.ProfileFlagName, "p", "", fmt.Sprintf(cobraext.ProfileFlagDescription, install.ProfileNameEnvVar))
+
+	return cobraext.NewCommand(cmd, cobraext.ContextGlobal)


I can't think of any scenario where we would run it outside of a Package. Should we change the context to cobraext.ContextPackage?

yeah, I changed this to ContextPackage

vinit-chauhan · 2026-02-27T21:42:40Z

internal/llmagent/docagent/workflow/workflow.go

+	var outputs []string
+
+	// Track input for LLM span
+	inputMessages := []tracing.Message{{Role: "user", Content: truncate(prompt, truncateLen)}}


I think we should not truncate messages used for Tracing. For logs it's okay, but if we do it in traces, it would reduce visibility.

Same comment for L281 ( outputMessagers )

I think we should not truncate messages used for Tracing. For logs it's okay, but if we do it in traces, it would reduce visibility.

Same comment for L281 ( outputMessagers )

I've removed truncate from the tracing, and it's only used for debug logging now.

Add broader inclusive-language checks (disability-defining terms, gendered job titles, execute/abort variants), Git conflict marker detection, and new static style checks (banned words, latinisms, exclamation points, ellipses, version terms, article misuse). Extend style-rule prompts with wordiness, negation, and device-agnostic language guidance. Remove unused parallel generation helpers and trim the readme template. Made-with: Cursor

Move stylerules/ into prompts/ (now a dependency-free leaf package), relocate sectioninstructions.go to docagent/ (its only consumer), drop the _static/ subdirectory, compose AgentInstructions from FullFormattingRules to eliminate duplication, and extract shared validator JSON output suffix into prompts.ValidatorOutputSuffix. Made-with: Cursor

elasticmachine · 2026-03-06T22:23:10Z

💚 Build Succeeded

Buildkite Build
Commit: 9ddc332

History

💔 Build #7395 failed f5dea86
💛 Build #7242 was flaky ead8b9b

cc @mjwolf

Add AI-powered documentation generation (core)

ead8b9b

Co-authored-by: Jonathan Molinatto <jonathan.molinatto@gmail.com> Co-authored-by: Quan Nguyen <quan.nguyen@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com>

mjwolf self-assigned this Feb 24, 2026

mjwolf requested a review from a team as a code owner February 24, 2026 02:15

mjwolf added enhancement New feature or request Team:Ecosystem Label for the Packages Ecosystem team labels Feb 24, 2026

mjwolf mentioned this pull request Feb 24, 2026

[feat/update-documentation] Add latest doc generator changes #3271

Closed

jrmolin reviewed Feb 26, 2026

View reviewed changes

vinit-chauhan reviewed Feb 26, 2026

View reviewed changes

vinit-chauhan reviewed Feb 27, 2026

View reviewed changes

mjwolf added 4 commits March 6, 2026 10:39

Merge remote-tracking branch 'upstream/main' into doc_update_1

dc0c34b

Address review feedback

5a7b34a

mjwolf requested review from jrmolin and vinit-chauhan March 6, 2026 21:42

mjwolf added 2 commits March 6, 2026 13:48

Correct doc template to fix unit tests

b087fbd

Merge remote-tracking branch 'upstream/main' into doc_update_1

9ddc332

		@@ -0,0 +1,142 @@
		# Style guide for Elastic integration documentation

Conversation

mjwolf commented Feb 24, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 6, 2026

💚 Build Succeeded

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants