Skip to content

feat: Add V2EX and HuggingFace as free sources (no API keys required)#148

Open
Alan5168 wants to merge 3 commits intomvanhorn:mainfrom
Alan5168:feat/free-mode-v2ex-huggingface
Open

feat: Add V2EX and HuggingFace as free sources (no API keys required)#148
Alan5168 wants to merge 3 commits intomvanhorn:mainfrom
Alan5168:feat/free-mode-v2ex-huggingface

Conversation

@Alan5168
Copy link
Copy Markdown

@Alan5168 Alan5168 commented Apr 3, 2026

Resolves #149

Summary

This PR adds two new free sources to last30days that require no API keys:

  • V2EX: Chinese tech community hot topics (similar to Hacker News for Chinese developers)
  • HuggingFace: Trending AI models with downloads, likes, and tags

Motivation

See detailed rationale in Issue #149:

Key Points:

  1. V2EX fills Chinese tech community gap — Current sources are English-centric (Reddit, HN, X). V2EX covers Chinese developer discussions.
  2. HuggingFace provides AI model trend data — Downloads and likes are real usage signals, not just discussion.
  3. Zero-config, free-first — Both work immediately with no API keys, extending v2.9.6's philosophy.

Changes

File Description
scripts/lib/v2ex.py V2EX hot topics fetcher (public API)
scripts/lib/huggingface.py HuggingFace models fetcher (public API)
scripts/last30days_lite.py Standalone free-mode aggregator
scripts/last30days.py Integration into main search pipeline
docs/free-mode.md Documentation for new sources

Integration

Both sources are integrated into the main search pipeline:

  • Added to VALID_SEARCH_SOURCES: v2ex, huggingface, hf
  • Threaded search functions: _search_v2ex(), _search_huggingface()
  • Normalized output format compatible with synthesis engine
  • Relevance scoring: rank + engagement + topic overlap

Usage

# Free mode standalone
python3 scripts/last30days_lite.py --topic "AI coding"

# Integrated with full search (--sources flag)
python3 scripts/last30days.py --sources v2ex,huggingface,hn --topic "Llama 4"

Output Examples

V2EX:

## V2EX (10 条)
1. [尴尬了,可能招聘了一个不会手写代码的 AI 工程师]
   📍 career | 💬 100 | ⭐ relevance: 0.85

HuggingFace:

## HuggingFace (10 条)
1. [meta-llama/Llama-4-Maverick-17B-128E-Instruct]
   📥 1.2M downloads | ❤️ 3.5K likes | tags: llama, instruct

Testing

Both modules tested locally:

  • ✅ V2EX API returns 10-30 hot topics successfully
  • ✅ HuggingFace API returns trending models with engagement data
  • ✅ Topic filtering works (case-insensitive substring match)
  • ✅ Relevance scoring integrates with existing pipeline
  • ✅ No API keys required — works immediately

Why This PR is Unique (vs other source requests)

PR/Issue Source Coverage Auth Required Unique Value
#134 GitHub Repos, Issues Optional GITHUB_TOKEN Developer workflow
#136 GitHub Same as #134 Optional Duplicate of #134
#140 Tavily Web search TAVILY_API_KEY Paid service
#149 (this) V2EX + HF Chinese tech + AI models None Free + unique coverage

Key differentiators:

  • First Chinese-language source (V2EX)
  • First AI-model-specific source (HuggingFace)
  • Both 100% free — no config step needed

Comparison Table

Feature Free Mode Full Mode
Hacker News
Reddit ✅ RSS ✅ ScrapeCreators
V2EX NEW
HuggingFace NEW
X/Twitter
YouTube
TikTok/Instagram
Polymarket

Expanding last30days coverage to global tech communities and AI model trends — zero API keys required.

Related: #134 (GitHub source), #128 (MiniMax API backend)

Alan Li and others added 2 commits April 3, 2026 14:34
- Add last30days_lite.py for zero-API aggregation
- Add lib/v2ex.py: V2EX Chinese tech community hot topics
- Add lib/huggingface.py: HuggingFace trending AI models
- Add docs/free-mode.md: Documentation for new free sources
- Integrate v2ex/huggingface into main search pipeline

V2EX and HuggingFace require no API keys, expanding coverage to:
- Chinese tech community discussions (V2EX)
- Trending AI models with downloads/likes (HuggingFace)

Co-authored-by: Alan Li <alan5168@github.com>
Improvements based on @Jah-yee's feedback in mvanhorn#149:

1. V2EX: Add 'latest' endpoint for broader coverage
   - New fetch_v2ex_latest() function
   - Combined fetch_v2ex() merges hot + latest
   - Deduplicates by topic ID
   - Hot topics get relevance boost

2. HuggingFace: Add likes-to-downloads quality ratio
   - New _calculate_quality_ratio() function
   - Quality labels: ⭐ excellent (5%+), 👍 good (2%+), ✓ acceptable (0.5%+)
   - Quality boost in relevance scoring
   - Helps identify well-regarded models vs just popular ones

3. Documentation: Add Chinese quickstart guide
   - docs/quickstart-zh.md for V2EX users
   - Explains quality ratio interpretation
   - Usage examples and FAQ

Thanks @Jah-yee for the valuable feedback!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Add V2EX and HuggingFace as free sources (zero API keys required)

1 participant