Skip to content

Commit 3b963b4

Browse files
slarsonclaude
andcommitted
Add Tumblr migration tools and documentation
Automated migration of news.html content to Tumblr blog: - migrate_news_to_tumblr.py: NPF-based migration script with backdating - tumblr_bot.py: CLI tool for manual blog management - TUMBLR_MIGRATION_README.md: Complete setup and usage documentation - tumblr_posts_backup_2026-02-01.json: Backup of migrated posts - .gitignore: Added Python patterns (.env.tumblr, .venv/, etc.) Successfully migrated 19 posts from Sept 2022 - June 2025 OAuth credentials stored in .env.tumblr (gitignored) Blog revived after 6+ years of inactivity Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
1 parent ce00abb commit 3b963b4

File tree

5 files changed

+3330
-0
lines changed

5 files changed

+3330
-0
lines changed

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,9 @@ war/.DS_Store
22
war/img/.DS_Store
33
/.DS_Store
44
/img/.DS_Store
5+
6+
# Tumblr API Tools
7+
.env.tumblr
8+
.venv/
9+
*.pyc
10+
__pycache__/

TUMBLR_MIGRATION_README.md

Lines changed: 290 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,290 @@
1+
# OpenWorm Tumblr Blog Tools
2+
3+
Automated tools for managing the OpenWorm Tumblr blog at https://openworm.tumblr.com
4+
5+
## Overview
6+
7+
These tools enable programmatic posting to the OpenWorm Tumblr blog using the Tumblr API. The primary use case is backfilling the stale blog (last post: July 2020) with current content from `news.html`.
8+
9+
## Files
10+
11+
| File | Purpose |
12+
|------|---------|
13+
| `migrate_news_to_tumblr.py` | Automated migration from news.html to Tumblr |
14+
| `tumblr_bot.py` | CLI tool for manual blog management |
15+
| `.env.tumblr` | OAuth credentials (gitignored) |
16+
| `.venv/` | Python virtual environment (gitignored) |
17+
18+
## Setup
19+
20+
### Prerequisites
21+
22+
- Python 3.x
23+
- pip or pip3
24+
25+
### Installation
26+
27+
```bash
28+
# Create virtual environment
29+
python3 -m venv .venv
30+
31+
# Activate virtual environment
32+
source .venv/bin/activate
33+
34+
# Install dependencies
35+
pip install requests-oauthlib python-dotenv beautifulsoup4
36+
```
37+
38+
### OAuth Credentials
39+
40+
The `.env.tumblr` file contains OAuth credentials for the registered Tumblr app:
41+
42+
```
43+
TUMBLR_CONSUMER_KEY=<key>
44+
TUMBLR_CONSUMER_SECRET=<secret>
45+
TUMBLR_ACCESS_TOKEN=<token>
46+
TUMBLR_ACCESS_TOKEN_SECRET=<token_secret>
47+
```
48+
49+
**⚠️ IMPORTANT:** These credentials are gitignored. Never commit them to version control.
50+
51+
## Migration Script Usage
52+
53+
The `migrate_news_to_tumblr.py` script migrates content from `news.html` to Tumblr with backdating.
54+
55+
### Preview Migration
56+
57+
See what will be migrated without posting:
58+
59+
```bash
60+
source .venv/bin/activate
61+
python migrate_news_to_tumblr.py preview
62+
```
63+
64+
Output:
65+
```
66+
============================================================
67+
MIGRATION PREVIEW - news.html to Tumblr (NPF Format)
68+
============================================================
69+
70+
## June 2025 (backdate: 2025-06-15T12:00:00Z)
71+
----------------------------------------
72+
1. OpenWorm.ai - a C. elegans specific LLM
73+
Blocks: 3 (2 text, 1 images)
74+
2. Updated C. elegans Connectome Toolbox
75+
Blocks: 3 (2 text, 1 images)
76+
77+
...
78+
79+
============================================================
80+
TOTAL: 19 posts would be created
81+
============================================================
82+
```
83+
84+
### Inspect Single Item
85+
86+
View the NPF block conversion for a specific news item:
87+
88+
```bash
89+
python migrate_news_to_tumblr.py inspect June2025 0
90+
```
91+
92+
This shows:
93+
- Original HTML input
94+
- Converted NPF blocks (JSON)
95+
96+
### Create Draft Posts (Recommended)
97+
98+
Create all posts as **drafts** for review:
99+
100+
```bash
101+
python migrate_news_to_tumblr.py draft --confirm
102+
```
103+
104+
After running, review the drafts in the [Tumblr dashboard](https://www.tumblr.com/blog/openworm/drafts) before publishing.
105+
106+
### Publish Directly
107+
108+
**⚠️ WARNING:** This publishes posts immediately with backdating.
109+
110+
```bash
111+
python migrate_news_to_tumblr.py publish --confirm
112+
# Type 'yes' when prompted
113+
```
114+
115+
## Manual Blog Management
116+
117+
The `tumblr_bot.py` tool provides CLI commands for manual blog operations.
118+
119+
### Get Blog Info
120+
121+
```bash
122+
source .venv/bin/activate
123+
python tumblr_bot.py info
124+
```
125+
126+
Output:
127+
```
128+
Blog: OpenWorm
129+
URL: https://openworm.tumblr.com/
130+
Posts: 275
131+
Followers: 163
132+
```
133+
134+
### Get Recent Posts
135+
136+
```bash
137+
python tumblr_bot.py posts 5
138+
```
139+
140+
### Create a Post (Interactive)
141+
142+
```bash
143+
python tumblr_bot.py post
144+
# Follow prompts for title, body, tags
145+
```
146+
147+
### Create a Draft (Interactive)
148+
149+
```bash
150+
python tumblr_bot.py draft
151+
# Follow prompts for title, body, tags
152+
```
153+
154+
## Technical Details
155+
156+
### NPF Format
157+
158+
The migration script uses **NPF (Neue Post Format)** instead of legacy HTML to ensure proper rendering of:
159+
160+
- ✅ Images (with rich preview cards)
161+
- ✅ Italic text (e.g., "*C. elegans*")
162+
- ✅ Bold text
163+
- ✅ Clickable links
164+
- ✅ Headings (H1, H2, H3)
165+
166+
### HTML to NPF Conversion
167+
168+
The script converts HTML elements to NPF blocks:
169+
170+
| HTML | NPF Block |
171+
|------|-----------|
172+
| `<h1>`, `<h2>` | `{"type": "text", "subtype": "heading1"}` |
173+
| `<h3>` | `{"type": "text", "subtype": "heading2"}` |
174+
| `<p>` | `{"type": "text", "formatting": [...]}` |
175+
| `<img>` | `{"type": "image", "media": [{...}]}` |
176+
| `<a>` | Formatting entry: `{"type": "link", "url": "..."}` |
177+
| `<i>`, `<em>` | Formatting entry: `{"type": "italic"}` |
178+
| `<b>`, `<strong>` | Formatting entry: `{"type": "bold"}` |
179+
180+
### URL Fixing
181+
182+
Relative URLs are converted to absolute:
183+
- `img/file.png``https://openworm.org/img/file.png`
184+
- `/assets/file.pdf``https://openworm.org/assets/file.pdf`
185+
186+
### Backdating
187+
188+
Posts are backdated to the middle of their publication month using ISO 8601 format:
189+
190+
```python
191+
MONTH_DATES = {
192+
"June2025": "2025-06-15T12:00:00Z",
193+
"Dec2024": "2024-12-15T12:00:00Z",
194+
"May2024": "2024-05-15T12:00:00Z",
195+
"June2023": "2023-06-15T12:00:00Z",
196+
"September2022": "2022-09-15T12:00:00Z",
197+
}
198+
```
199+
200+
## Migration Summary
201+
202+
| Time Period | Posts | Images | Backdate |
203+
|-------------|-------|--------|----------|
204+
| June 2025 | 2 | 2 | 2025-06-15 |
205+
| December 2024 | 1 | 1 | 2024-12-15 |
206+
| May 2024 | 5 | 1 | 2024-05-15 |
207+
| June 2023 | 4 | 4 | 2023-06-15 |
208+
| September 2022 | 7 | 9 | 2022-09-15 |
209+
| **TOTAL** | **19** | **17** | |
210+
211+
## Recommended Workflow
212+
213+
1. **Preview the migration**
214+
```bash
215+
python migrate_news_to_tumblr.py preview
216+
```
217+
218+
2. **Inspect a sample post** to verify formatting
219+
```bash
220+
python migrate_news_to_tumblr.py inspect June2025 0
221+
```
222+
223+
3. **Create drafts**
224+
```bash
225+
python migrate_news_to_tumblr.py draft --confirm
226+
```
227+
228+
4. **Review in Tumblr dashboard**
229+
- Visit https://www.tumblr.com/blog/openworm/drafts
230+
- Check formatting, images, links
231+
- Use "Mass Post Editor" to publish all at once if satisfied
232+
233+
5. **Publish** (if not done via UI)
234+
```bash
235+
python migrate_news_to_tumblr.py publish --confirm
236+
```
237+
238+
## Future Posting
239+
240+
After the migration, use `tumblr_bot.py` for new posts:
241+
242+
```bash
243+
# Option 1: Interactive
244+
python tumblr_bot.py post
245+
246+
# Option 2: Programmatic (extend tumblr_bot.py)
247+
# Add functions for specific post types or scheduled posting
248+
```
249+
250+
## Troubleshooting
251+
252+
### Authentication Errors
253+
254+
If you get "401 Unauthorized", check:
255+
1. `.env.tumblr` file exists and contains all 4 credentials
256+
2. Virtual environment is activated
257+
3. OAuth tokens haven't expired (regenerate via Tumblr OAuth flow if needed)
258+
259+
### Image Not Rendering
260+
261+
NPF requires absolute URLs. Check that:
262+
1. Image URLs start with `https://openworm.org/`
263+
2. Image files exist at those paths
264+
3. MIME type is correctly detected (PNG, JPEG, GIF)
265+
266+
### Formatting Issues
267+
268+
If text formatting looks wrong:
269+
1. Use `inspect` command to view NPF blocks
270+
2. Check that character offsets in formatting entries are correct
271+
3. Verify nested elements are handled properly
272+
273+
## API Rate Limits
274+
275+
The OpenWorm Blog Bot app has these limits:
276+
- **1,000 requests/hour**
277+
- **5,000 requests/day**
278+
279+
The migration script creates 19 posts = 19 API calls, well within limits.
280+
281+
## References
282+
283+
- [Tumblr API Documentation](https://www.tumblr.com/docs/en/api/v2)
284+
- [NPF Format Specification](https://www.tumblr.com/docs/npf)
285+
- [OAuth 1.0a Documentation](http://oauth.net/)
286+
287+
---
288+
289+
**Last Updated:** January 31, 2026
290+
**Maintainer:** OpenWorm Team

0 commit comments

Comments
 (0)