Skip to content

Commit 9e5b3d4

Browse files
ExoMonkExoMonk
committed
docs
1 parent a8c1d72 commit 9e5b3d4

1 file changed

Lines changed: 106 additions & 115 deletions

File tree

‎README.md‎

Lines changed: 106 additions & 115 deletions
Original file line numberDiff line numberDiff line change
@@ -4,105 +4,6 @@ Blazing-fast Rust indexer for Hyperliquid's HyperCore EVM. Ingests block data di
44

55
Built for HIP4 prediction market indexing. Testnet live now -- when HIP4 goes mainnet, one config change enables everything.
66

7-
## Data Model
8-
9-
### Core (EVM blocks from S3)
10-
11-
```
12-
blocks
13-
├── block_number (PK)
14-
├── block_hash, parent_hash
15-
├── timestamp, gas_used, gas_limit, base_fee_per_gas
16-
└── tx_count, system_tx_count
17-
18-
transactions
19-
├── block_number, tx_index (PK)
20-
├── tx_hash ← computed via RLP + keccak256
21-
├── tx_type (Legacy/Eip2930/Eip1559)
22-
├── from, to, value, input
23-
├── gas_limit, gas_used, success
24-
└── FK → blocks
25-
26-
event_logs
27-
├── block_number, tx_index, log_index (PK)
28-
├── address, topic0..topic3, data
29-
└── FK → transactions
30-
31-
system_transfers
32-
├── block_number, tx_index (PK)
33-
├── official_hash, explorer_hash ← dual phantom hashes
34-
├── system_address, asset_type, asset_index
35-
├── recipient, amount_wei
36-
└── FK → blocks
37-
```
38-
39-
### Trade Fills (from S3 `node_fills_by_block`)
40-
41-
```
42-
fills
43-
├── trade_id, user_address (PK)
44-
├── block_number, block_time
45-
├── coin ← "BTC", "ETH", "@230", "#90"
46-
├── price, size, side, direction
47-
├── closed_pnl, fee, fee_token
48-
└── fill_time
49-
50-
hip4_trades ← mirror of fills where coin starts with #
51-
└── same schema as fills
52-
```
53-
54-
### HIP4 Prediction Markets
55-
56-
```
57-
hip4_deposits ← decoded from EVM Deposit events
58-
├── block_number, tx_index, log_index (PK)
59-
├── contest_id, side_id
60-
├── depositor, amount_wei
61-
└── FK → event_logs
62-
63-
hip4_claims ← decoded from EVM Claimed events
64-
├── same structure as hip4_deposits
65-
└── claimer, amount_wei
66-
67-
hip4_contest_creations ← decoded from createContest() calls
68-
├── block_number, tx_index (PK)
69-
└── contest_id, param2
70-
71-
hip4_refunds ← decoded from refund() calls
72-
├── block_number, tx_index (PK)
73-
└── contest_id, side_id, user_address
74-
75-
hip4_sweeps ← decoded from sweepUnclaimed() calls
76-
├── block_number, tx_index (PK)
77-
└── contest_id
78-
79-
hip4_markets ← polled from outcomeMeta API
80-
├── outcome_id (PK)
81-
├── name, description, side_specs (JSON)
82-
└── question_id, question_name
83-
84-
hip4_prices ← polled from allMids API
85-
├── coin, timestamp (PK) ← "#90", "#91", "#11760"
86-
└── mid_price ← 0.0 to 1.0 (implied probability)
87-
88-
indexer_cursor
89-
├── network (PK) ← "mainnet" or "testnet"
90-
└── last_block, updated_at
91-
```
92-
93-
### Data Flow
94-
95-
```
96-
S3 EVM blocks ──→ blocks → transactions → event_logs → system_transfers
97-
└→ hip4_deposits, hip4_claims (decoded)
98-
└→ hip4_contest_creations, hip4_refunds, hip4_sweeps
99-
100-
S3 node_fills ──→ fills ──→ hip4_trades (# coins mirrored)
101-
102-
HyperCore API ──→ hip4_markets (metadata)
103-
└→ hip4_prices (implied probability snapshots)
104-
```
105-
1067
## Installation
1078

1089
### Docker (recommended)
@@ -132,16 +33,18 @@ docker run --rm \
13233
Requires Rust 1.91+:
13334

13435
```bash
135-
# Option 1: install script (builds + generates config)
136-
curl -sSL https://raw.githubusercontent.com/ExoMonk/hypercore-indexer/main/install.sh | bash
137-
138-
# Option 2: manual
13936
git clone https://github.com/ExoMonk/hypercore-indexer.git
14037
cd hypercore-indexer
14138
cargo install --path .
14239
hypercore-indexer init
14340
```
14441

42+
Or use the install script:
43+
44+
```bash
45+
curl -sSL https://raw.githubusercontent.com/ExoMonk/hypercore-indexer/main/install.sh | bash
46+
```
47+
14548
After installation, `hypercore-indexer` is available as a CLI:
14649

14750
```bash
@@ -357,18 +260,6 @@ The indexer is **S3-latency-limited, not compute-limited**. Each block is a sepa
357260

358261
Once caught up, incremental indexing from anywhere keeps pace easily -- Hyperliquid produces ~1 block/sec.
359262

360-
### vs coredrain (TypeScript reference)
361-
362-
| | coredrain | hypercore-indexer |
363-
|---|-----------|-------------------|
364-
| Language | TypeScript (Bun) | Rust (tokio) |
365-
| S3 throughput | ~370 blocks/sec | **450 blocks/sec** |
366-
| Hash computation | JS + viem | Native RLP + keccak256 |
367-
| Storage | MongoDB (row-by-row) | PostgreSQL/ClickHouse (UNNEST batch) |
368-
| Full pipeline | S3 -> match only | S3 -> decode -> hash -> store |
369-
370-
hypercore-indexer is faster while doing strictly more work (full decode + hash computation + storage).
371-
372263
## Networks
373264

374265
| | Mainnet | Testnet |
@@ -408,3 +299,103 @@ Full backfill costs about the price of a coffee. Ongoing indexing is negligible.
408299
- Rust 1.91+
409300
- AWS credentials configured (S3 bucket is requester-pays, region `ap-northeast-1`)
410301
- Docker (optional, for PostgreSQL dev environment via `make dev`)
302+
303+
304+
## Data Model
305+
306+
### Core (EVM blocks from S3)
307+
308+
```
309+
blocks
310+
├── block_number (PK)
311+
├── block_hash, parent_hash
312+
├── timestamp, gas_used, gas_limit, base_fee_per_gas
313+
└── tx_count, system_tx_count
314+
315+
transactions
316+
├── block_number, tx_index (PK)
317+
├── tx_hash ← computed via RLP + keccak256
318+
├── tx_type (Legacy/Eip2930/Eip1559)
319+
├── from, to, value, input
320+
├── gas_limit, gas_used, success
321+
└── FK → blocks
322+
323+
event_logs
324+
├── block_number, tx_index, log_index (PK)
325+
├── address, topic0..topic3, data
326+
└── FK → transactions
327+
328+
system_transfers
329+
├── block_number, tx_index (PK)
330+
├── official_hash, explorer_hash ← dual phantom hashes
331+
├── system_address, asset_type, asset_index
332+
├── recipient, amount_wei
333+
└── FK → blocks
334+
```
335+
336+
### Trade Fills (from S3 `node_fills_by_block`)
337+
338+
```
339+
fills
340+
├── trade_id, user_address (PK)
341+
├── block_number, block_time
342+
├── coin ← "BTC", "ETH", "@230", "#90"
343+
├── price, size, side, direction
344+
├── closed_pnl, fee, fee_token
345+
└── fill_time
346+
347+
hip4_trades ← mirror of fills where coin starts with #
348+
└── same schema as fills
349+
```
350+
351+
### HIP4 Prediction Markets
352+
353+
```
354+
hip4_deposits ← decoded from EVM Deposit events
355+
├── block_number, tx_index, log_index (PK)
356+
├── contest_id, side_id
357+
├── depositor, amount_wei
358+
└── FK → event_logs
359+
360+
hip4_claims ← decoded from EVM Claimed events
361+
├── same structure as hip4_deposits
362+
└── claimer, amount_wei
363+
364+
hip4_contest_creations ← decoded from createContest() calls
365+
├── block_number, tx_index (PK)
366+
└── contest_id, param2
367+
368+
hip4_refunds ← decoded from refund() calls
369+
├── block_number, tx_index (PK)
370+
└── contest_id, side_id, user_address
371+
372+
hip4_sweeps ← decoded from sweepUnclaimed() calls
373+
├── block_number, tx_index (PK)
374+
└── contest_id
375+
376+
hip4_markets ← polled from outcomeMeta API
377+
├── outcome_id (PK)
378+
├── name, description, side_specs (JSON)
379+
└── question_id, question_name
380+
381+
hip4_prices ← polled from allMids API
382+
├── coin, timestamp (PK) ← "#90", "#91", "#11760"
383+
└── mid_price ← 0.0 to 1.0 (implied probability)
384+
385+
indexer_cursor
386+
├── network (PK) ← "mainnet" or "testnet"
387+
└── last_block, updated_at
388+
```
389+
390+
### Data Flow
391+
392+
```
393+
S3 EVM blocks ──→ blocks → transactions → event_logs → system_transfers
394+
└→ hip4_deposits, hip4_claims (decoded)
395+
└→ hip4_contest_creations, hip4_refunds, hip4_sweeps
396+
397+
S3 node_fills ──→ fills ──→ hip4_trades (# coins mirrored)
398+
399+
HyperCore API ──→ hip4_markets (metadata)
400+
└→ hip4_prices (implied probability snapshots)
401+
```

0 commit comments

Comments
 (0)