Analytics and success metrics tracking

[BillChirico]

📊 Analytics and Success Metrics Tracking

Overview

Measure Sobers Buddy effectiveness without compromising privacy — focus on helpfulness, not surveillance.

---

Goals

• Track user-reported helpfulness
• Measure opt-out rates
• Monitor engagement quality
• Inform product improvements
• Never track recovery details

---

Key Metrics

User Experience

Metric	Target	Measurement
Message helpfulness	>70% thumbs up	Per-message feedback
Opt-out rate	<10%	Settings toggle tracking
Response rate	>40%	Message opens/replies
Feature usage	Track adoption	Tool usage rates

Recovery Support

Metric	Target	Measurement
Check-in consistency	+20% vs baseline	Daily check-in rate
Tool engagement	>3 uses/week	Coping tool usage
Meeting finder usage	Track trends	Meeting searches
Slip response helpfulness	>80% positive	Post-slip feedback

Retention

Metric	Target	Measurement
30-day retention	>60%	Active at day 30
60-day retention	>50%	Active at day 60
90-day retention	>40%	Active at day 90

---

Analytics Events

Message Events

• message_sent
• message_opened
• message_responded
• message_thumbs_up
• message_thumbs_down

Feature Events

• buddy_enabled
• buddy_disabled
• preference_changed
• tool_used (anonymized)
• meeting_searched
• pause_activated

Engagement Events

• conversation_started
• conversation_ended
• daily_check_in
• milestone_viewed

---

Privacy-First Approach

Anonymization

• No user IDs in analytics
• Session-based tracking only
• Aggregate data only for reports
• No recovery content in events

Opt-out

• User can disable analytics
• Core functionality unchanged
• Transparent about what's tracked

---

Implementation

interface AnalyticsEvent {
  name: string;
  timestamp: Date;
  sessionId: string;  // Anonymous
  properties: Record<string, any>;  // No PII
}

// Example events
analytics.track('message_thumbs_up', {
  messageType: 'check_in',
  tone: 'warm',
  hasContext: true
});

analytics.track('buddy_disabled', {
  durationDays: 14,
  reason: 'too_many_messages'  // optional
});

---

Dashboard

Real-time Metrics

• Messages sent (last 24h)
• Active conversations
• Feature usage breakdown

Weekly Reports

• Helpfulness trends
• Opt-out analysis
• Engagement by feature
• Retention cohorts

Monthly Deep Dives

• Tone effectiveness
• Pattern learning accuracy
• Slip response outcomes
• User feedback themes

---

Implementation Tasks

• Set up analytics infrastructure (PostHog/Amplitude)
• Define event schema
• Implement event tracking
• Build analytics dashboard
• Create weekly report automation
• Set up retention analysis
• Add feedback collection UI
• Document analytics privacy

---

Acceptance Criteria

• All key metrics trackable
• Dashboard shows real-time data
• Weekly reports auto-generate
• No PII in analytics
• User can opt out
• Retention cohorts accurate
• Feedback integrated into product

---

Related

• Parent: [Epic] Sobers Buddy Implementation #412
• Related: Privacy and data handling implementation #421 (privacy)

---

Part of Sobers v2