Skip to content

Add --remote flag to save command for streaming from GitHub #52

@mihaelamj

Description

@mihaelamj

Summary

Add --remote flag to cupertino save command that builds the search database by streaming documentation directly from GitHub, without saving files to disk.

Motivation

Currently, setting up Cupertino requires crawling Apple's documentation (~20+ hours). This flag provides instant setup by streaming from pre-crawled repos.

Source Repos

  • cupertino-docs - Pre-crawled documentation
    • docs/ - 248 framework folders
    • archive/ - Legacy programming guides
    • packages/ - Package READMEs
    • swift-evolution/ - Swift proposals
    • swift-org/ - Swift.org docs

Command Interface

cupertino save --remote      # Stream from GitHub, build database (~45 min)
cupertino save               # Build from local files (existing behavior)

Note: For instant setup (~30 seconds), use cupertino setup instead (see #65).

Implementation Tasks

Package Creation

  • Create Sources/RemoteSync/ directory
  • Add RemoteSync.swift namespace
  • Add RemoteIndexState.swift - Sendable state model with file-level tracking
  • Add GitHubFetcher.swift - Actor for HTTP with async URLSession (uses Git Tree API)
  • Add AnimatedProgress.swift - Terminal progress UI with spinner
  • Add RemoteIndexer.swift - Main orchestrator actor
  • Update Package.swift with new target

CLI Integration

  • Add --remote flag to SaveCommand.swift
  • Add state persistence path to Constants.swift
  • Add resume prompt when state file exists
  • Handle all phases (docs, evolution, archive, swift-org)

Documentation (per AGENTS.md Feature Checklist)

  • Update README.md - Quick start with save --remote
  • Update CHANGELOG.md - Version entry (0.3.0)
  • Update docs/commands/save/README.md - Add flag section
  • Create docs/commands/save/option (--)/remote/README.md - Flag docs
  • Create docs/commands/save/option (--)/remote/option (--)/base-dir.md
  • Create docs/commands/save/option (--)/remote/option (--)/search-db.md
  • Update CLI help text in SaveCommand.swift

Additional Features Implemented

  • Git Tree API for single-call repo structure fetch (avoids rate limits)
  • GitHub token support via GITHUB_TOKEN environment variable
  • Animated spinner progress display
  • Single-line progress with carriage return updates

Files Created

File Purpose
Sources/RemoteSync/RemoteSync.swift Public namespace
Sources/RemoteSync/RemoteIndexer.swift Main orchestrator
Sources/RemoteSync/GitHubFetcher.swift HTTP client with Git Tree API
Sources/RemoteSync/RemoteIndexState.swift State model (Sendable)
Sources/RemoteSync/AnimatedProgress.swift Terminal progress with spinner

Files Modified

File Changes
Package.swift Added RemoteSync target and product
SaveCommand.swift Added @Flag var remote: Bool, calls RemoteIndexer
Constants.swift Added remoteStateFile path constant

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions