Commit be2d313
Add fingerprint module for monthly chunk deduplication
New module: src/fingerprint/
- types.ts: MonthlyChunk, DeduplicationPlan, FingerprintConfig
- chunker.ts: generateMonthlyChunks(), createDeduplicationPlan()
- index.ts: Module exports
Key features:
- Groups messages by calendar month for deduplication
- Uses first 10 messages + message count per month for fingerprinting
- Truncates timestamps to minute (floor, not round) to handle WhatsApp ±1s drift
- Per-user isolation (fingerprints not shared across users)
- Includes comprehensive tests for edge cases including real-world drift scenario
See project_docs/CHAT_FINGERPRINTING.md for algorithm details.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>1 parent 71a70fb commit be2d313
File tree
6 files changed
+830
-1
lines changed- src
- fingerprint
6 files changed
+830
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
40 | 45 | | |
41 | 46 | | |
42 | 47 | | |
| |||
47 | 52 | | |
48 | 53 | | |
49 | 54 | | |
50 | | - | |
| 55 | + | |
51 | 56 | | |
52 | 57 | | |
53 | 58 | | |
| |||
0 commit comments