Skip to content

siwenwang0803/RedForge

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿ”ฅ RedForge CLI

Developer-first automated LLM red-team platform

Version Build License Product Hunt Python OWASP

๐Ÿš€ Status: โœ… Sprint S-3 Complete (July 2025) - Product Hunt Ready, Enterprise Deployments Active
๐Ÿ“‹ Reference: Product One-Pager | 12M Roadmap | Landing Page

Problem We Solve

Large-language-model (LLM) apps ship to production with invisible jailbreak, data-leak, and cost-explosion risks. Regulators mandate continuous red-teaming (EU AI Act Art.55, US EO 14110).

RedForge automates OWASP LLM Top 10 testing:

  • โœ… Local execution (keys on-prem)
  • โœ… Audit-ready reports (NIST AI-RMF mapping)
  • โœ… Docker CLI setup in 5 minutes
  • โœ… Coverage tracking for compliance gaps

Example Use Cases

  • Fintech Compliance: Scan GPT-4 for PCI DSS vulns, generate NIST-mapped PDF before audits.
  • Enterprise Dev: Integrate OWASP tests in CI/CD, chaos-test for resilience.
  • Security Research: Dry-run attacks, analyze feedback, customize for bias/PII threats.

Quick Start

pip install redforge
redforge doctor              # Environment self-check
redforge scan gpt-4 --offline --dry-run

๐Ÿš€ Try Cloud Scan - $29/mo | ๐Ÿ“– Full Documentation | ๐Ÿ“Š Threat Model

๐Ÿณ Docker Alternative

docker run --rm siwenwang0803/redforge:latest scan gpt-4 --dry-run

โ˜๏ธ Cloud Starter ($29/month)

  1. Click Get Starter Key โ†’
  2. In CLI:
    redforge signup --email [email protected]
    redforge scan gpt-4 --cloud-api-key <your_key>
  3. Access advanced features: unlimited scans, team collaboration, compliance reports

๐Ÿ“ฆ Advanced Installation

pip install redforge
redforge scan gpt-4 --dry-run

โš“ Helm (Kubernetes)

# Add repository
helm repo add redforge https://siwenwang0803.github.io/RedForge
helm repo update

# Install CLI for job-based scanning
helm install my-cli redforge/redforge-cli \
  --set secrets.openaiApiKey="your-api-key"

# Install sidecar for runtime monitoring
helm install my-sidecar redforge/redforge-sidecar \
  --set secrets.apiKeys.openai="your-api-key"

Core Features

โœจ New in v0.3.1 โ€“ Product Hunt Preview:

  • ๐Ÿ”ฅ Open-core tiering: Free offline mode, Starter ($29/mo), Pro ($99/mo)
  • ๐Ÿ“Š Enhanced reporting: PDF/HTML/JSON with compliance mapping
  • ๐Ÿ’ณ Stripe integration: Seamless checkout + webhook automation
  • ๐Ÿ”ง Improved offline mode: No OPENAI_API_KEY required for dry runs
  • ๐Ÿ› Bug fixes: Typer 0.9 compatibility, CI/CD stability
  • โœ… Production ready: Full E2E workflow validation

๐Ÿš€ Key Capabilities

  • Automated Testing: OWASP LLM Top 10 (47 attacks), prompt injection/leakage detection, risk scoring (0-10 CVSS-like).
  • Reporting: JSON/PDF/HTML/CSV with compliance (NIST, EU AI Act, SOC2, PCI DSS).
  • Compliance & Audit: NIST AI-RMF/EU Act mappings, cryptographic trails.
  • Production Ready: Docker/K8s, rate limiting, chaos testing, lightweight mode.
  • Community-Driven: Feedback collection/analysis, automated roadmaps.
๐Ÿ“„ Detailed Features (Click to Expand)

๐Ÿ”’ Security Testing

  • Complete OWASP LLM Top 10 coverage with 47 pre-built attacks
  • Real-time vulnerability detection with confidence scoring
  • Evidence collection and cryptographic audit trails
  • Custom attack pack support (coming soon)

๐Ÿ“Š Compliance & Reporting

  • Multi-framework support: NIST AI-RMF, EU AI Act, SOC 2, ISO 27001, PCI DSS v4.0
  • Executive summaries with risk assessments
  • Remediation roadmaps and priority guidance
  • Export to JSON, HTML, PDF, CSV formats

๐Ÿš€ Production Features

  • Docker and Kubernetes deployment ready
  • Rate limiting and timeout controls
  • Chaos testing for resilience validation
  • CI/CD integration with GitHub Actions
  • Telemetry and feedback collection

Pro Features & Pilots

  • Free OSS CLI for basics.
  • Paid Pilots: $4-7k one-off pentests with custom PDFs ([email protected]).
  • Upcoming SaaS: $1k/mo for dashboards, monitoring, premium modules (FinOps, Privacy). Waitlist: redforge.com

Attack Packs

๐Ÿ”ด OWASP LLM Top 10 (Default)

Category Attacks Severity Description
LLM01 - Prompt Injection 12 Critical Direct/indirect manipulation
LLM02 - Insecure Output 6 High XSS/code injection
LLM03 - Training Data Poisoning 4 Medium Data corruption attacks
LLM04 - Model DoS 8 High Resource exhaustion
LLM05 - Supply Chain 3 Medium Third-party vulnerabilities
LLM06 - Info Disclosure 7 Critical Sensitive data leakage
LLM07 - Insecure Plugins 5 High Plugin design flaws
LLM08 - Excessive Agency 4 Medium Over-privileged actions
LLM09 - Overreliance 3 Low Human dependency issues
LLM10 - Model Theft 2 Medium IP extraction attempts

Full details: Attack Packs Reference.

๐Ÿ”ฎ Coming Soon

  • FinOps (cost detection, S-9)
  • Privacy (GDPR/CCPA, S-10)
  • Bias (fairness testing, S-11)

Configuration

Sample redforge.yaml:

target:
  endpoint: "https://api.openai.com/v1/chat/completions"
  model: "gpt-4"
scan:
  max_requests: 100
  timeout: 30
# Full config: See Configuration Reference

Roadmap & Status

  • S-1 (โœ… Complete): CLI foundation, OWASP Top 10 coverage, report generation
  • S-2/Pilot-0 (โœ… Complete): K8s sidecar, chaos testing, PCI DSS compliance
  • S-3 (โœ… Complete - v0.3.1): Open-core model, Stripe payments, Product Hunt launch ready
  • S-4 (๐Ÿš€ Next - Aug 2025): SaaS dashboard, team collaboration, advanced analytics

Full: 12M Roadmap

Development & CI/CD

Prerequisites: Python 3.11+, Poetry, Docker.

git clone https://github.com/siwenwang0803/RedForge.git
make install  # Setup
make test     # Run tests

CI/CD examples: CI/CD Guide.

Documentation & Links

Support & Community

Security & License

  • Local exec, no exfil, opt-in telemetry.
  • MIT License: LICENSE.
  • Disclaimer: For authorized testing only.

Contributors

  • AI Team: Claude 4 Sonnet (Dev Lead), ChatGPT o3-pro (Strategy), etc.
  • Partners: 3 confidential enterprises.

Star History

Star History Chart


Ready to secure your LLM? pip install redforge && redforge scan gpt-4 --dry-run

Questions: [email protected]

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

No packages published

Contributors 3

  •  
  •  
  •