Skip to content

Title: Enhancement: Resilient IPFS Multi-Gateway Failover & CID Availability Monitoring #358

@Adithyakp86

Description

@Adithyakp86

Description

The system relies on IPFS (via Pinata) for decentralized evidence storage. However, public IPFS gateways or even dedicated Pinata gateways can experience rate limits, DNS failures, or downtime. In a legal context, failing to retrieve evidence during a court hearing due to a gateway timeout is unacceptable.

We need to build a high-availability IPFS retrieval layer that automatically falls back through a ranked list of public/private gateways, combined with an automated cron-based active monitoring system (as specified in the Automated Verification System roadmap) to ensure pinned CIDs are always resolvable.

Proposed Solution

  1. Gateway Race/Fallback Algorithm: Modify the frontend/backend retrieval logic to ping multiple IPFS gateways concurrently (e.g., Pinata, Cloudflare, IPFS.io, dweb.link). The system should stream from the first gateway that successfully returns a 200 OK header, caching the fastest gateway path.
  2. Availability Cron Job: Utilize node-cron to periodically run a health check against the Supabase database, polling random CIDs from the evidence table to ensure they have not been garbage collected or dropped by the IPFS pinning service.
  3. Admin Alerting: If a CID becomes unresolvable across 3+ gateways, integrate an automated alert via WebSocket/Email to the Evidence Manager dashboard to trigger a re-pin operation.

Technical Stack

  • Node.js (axios, node-cron)
  • IPFS HTTP APIs
  • Supabase (Updating CID health status)

Acceptance Criteria

  • A fallback mechanism implemented that gracefully handles 429 and 504 errors from IPFS gateways.
  • Background cron job polls 50 random CIDs every 6 hours and updates a health_status column in Supabase.
  • If a gateway becomes blacklisted/slow, the system automatically deprioritizes it.
  • Visual indicators on the evidence-manager.html dashboard showing the current health/availability of stored files.

Labels: enhancement, infrastructure, hard, ipfs, reliability

Assign this issue to me under the OSCG

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions