Skip to content

NLPM audit findings: 1 bug fix + 2 security improvements #976

@xiaolai

Description

@xiaolai

Automated audit: This PR was generated by NLPM, a natural language programming linter, running via claude-code-action. Please evaluate the diff on its merits.

About this audit

Orca was audited by NLPM, a natural language programming linter that scores Claude Code skills, agents, and configuration files on a 100-point scale. The audit scanned 9 NL artifacts and produced an overall score of 96/100 — a strong result that reflects well-written, self-contained skills.

This issue summarises the findings that warranted follow-up. Quality/style issues (vague quantifiers etc.) are listed for information only; the PRs below address only concrete bugs and low/medium security improvements.


Bug fixes

Bug #1 — Broken doc references in skills/orca-cli/SKILL.md (Priority: Medium)

The ## References section (lines 568–570) lists three documentation files that do not exist in the repository:

  • docs/orca-cli-focused-v1-status.md
  • docs/orca-cli-v1-spec.md
  • docs/orca-runtime-layer-design.md

Agents following the skill will hit dead links when trying to resolve ambiguous CLI behavior, silently losing the intended guidance. The skill is otherwise fully self-contained.

PR: #973


Security improvements (Medium / Low only)

The audit also detected one HIGH pattern (the postinstall script in package.json rebuilds native modules — standard Electron practice, not a vulnerability) and two lower-severity items that are safe to address via PR.

Security Fix #1ORCA_ELECTRON_VITE_CLI override lacks an environment guard (Medium)

config/scripts/run-electron-vite-dev.mjs allows the ORCA_ELECTRON_VITE_CLI env var to substitute the electron-vite binary unconditionally. The comment documents this as a test affordance, but without a guard the override is active in dev and CI runs where a compromised environment could inject an arbitrary binary path.

PR: #974

Security Fix #2 — Critical native deps use ^ semver constraints (Low)

node-pty, @parcel/watcher, and electron — all compiled native modules listed in pnpm.onlyBuiltDependencies — use ^ constraints. This allows silent minor/patch upgrades that could ship a different native binary or a vulnerable version. Notably, node-pty already has a patch file targeting 1.1.0 exactly, which a ^ bump could silently skip.

PR: #975


Quality notes (informational, no PRs)

These are style-level findings from the scoring run. No action required — listed here for completeness.

File Issue Score impact
skills/orca-cli/SKILL.md 6 vague quantifiers ("significant checkpoint", "meaningful progress", etc.) -12
.agents/skills/typescript/SKILL.md "complex type definitions", "when applicable" -6
.agents/skills/auto-review-fix/SKILL.md "relevant to that review type" -4
.agents/skills/react-useeffect/SKILL.md "Expensive calculations", "when possible" -4
.agents/skills/auto-pr-merge/SKILL.md "when appropriate" -2
.agents/skills/auto-submit/SKILL.md "fails catastrophically" (no observable threshold) -2
CLAUDE.md "non-obvious constraint" (subjective threshold) -2

Thank you for maintaining a well-structured, high-quality set of NL artifacts. Feel free to close any PR you disagree with — the diff should stand on its own merits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions