A comprehensive platform for music lessons that analyzes student performances and provides AI-powered feedback for musical improvement.
Features | Architecture | Technologies | Installation | Usage | License
- Audio Transcription - Convert student performances to text using NVIDIA Parakeet CTC
- AI Feedback - Intelligent musical analysis powered by LLaMA 3.1
- Semantic Search - Find similar performances using vector embeddings with Qdrant
- Progress Tracking - Monitor improvement over time with detailed metrics
- Multi-Instrument Support - 47+ instruments across categories (strings, wind, percussion, keyboard, voice, electronic)
- Structured Curriculum - Courses with modules, lessons, and materials
- Live & Recorded Lessons - Support for live sessions, recorded content, and tutorials
- XP & Levels - Experience points system for student engagement
- Achievements - Unlock badges and milestones for musical accomplishments
- Weekly Digests - Automated progress reports and activity summaries
- Students - Take lessons, record performances, track progress
- Guardians - Parent accounts with guardian-child relationships
- Teachers - Create courses, provide feedback, manage students
- Coordinators - Manage institutions and teacher assignments
- Admins - Full system access and configuration
- Stripe Integration - Secure payment processing
- Credit Packages - Flexible lesson credits system
- Multiple Plans - Free, Pro, and Enterprise tiers
graph TD
User[User / Admin]
subgraph "Jim Studio Platform"
LB[Phoenix Server]
UI[LiveView UI]
API[REST API]
Workers[Oban Workers]
end
subgraph "Data Layer"
DB[(TimescaleDB)]
Redis[Redis Cache]
Qdrant[Qdrant Vector DB]
end
subgraph "External Services"
R2[Cloudflare R2]
NVIDIA[NVIDIA NIM API]
Stripe[Stripe Payments]
Firebase[Firebase Auth]
end
User -->|HTTPS| LB
LB --> UI
LB --> API
API --> Workers
Workers --> DB
Workers --> Redis
Workers --> Qdrant
Workers --> R2
Workers --> NVIDIA
API --> Stripe
API --> Firebase
sequenceDiagram
participant S as Student
participant A as API
participant R2 as Cloudflare R2
participant T as Transcription Worker
participant N as NVIDIA NIM
participant An as Analysis Worker
participant Q as Qdrant
S->>A: Upload Performance
A->>R2: Store Audio File
A->>T: Queue Transcription Job
T->>N: Parakeet CTC
N-->>T: Transcribed Text
T->>An: Queue Analysis Job
An->>N: LLaMA 3.1 Analysis
N-->>An: Musical Feedback
An->>Q: Store Embeddings
An-->>S: WebSocket Notification
- Elixir 1.15+ - Functional programming for scalable applications
- Phoenix 1.8 - Real-time web framework with LiveView
- Ecto 3.13 - Database wrapper and query generator
- TimescaleDB - Time-series optimized PostgreSQL
- Redis - In-memory caching with Redix
- Qdrant - Vector database for semantic search
- Cloudflare R2 - Object storage for media files
- NVIDIA NIM - AI inference microservices
- Parakeet CTC for audio transcription
- NV-EmbedQA for text embeddings
- LLaMA 3.1 for performance analysis
- Guardian - JWT authentication
- Firebase Auth - Social authentication
- Stripe - Payment processing
- Oban 2.17 - Job processing with persistence
- Phoenix LiveView - Server-rendered real-time UI
- Tailwind CSS 4.0 - Utility-first styling
- Heroicons - Beautiful hand-crafted icons
- Elixir 1.15+
- Docker & Docker Compose
- NVIDIA API Key for AI features
- Clone the repository
git clone https://github.com/gabrielmaialva33/jim-studio.git
cd jim-studio- Start infrastructure
docker-compose up -dThis starts TimescaleDB (5432), Redis (6380), and Qdrant (6335/6336).
- Install dependencies
mix deps.get- Configure environment
cp .env.example .envFill in your credentials:
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/jim_studio_dev
REDIS_URL=redis://localhost:6380
GUARDIAN_SECRET_KEY=your-secret-key-min-32-chars
NVIDIA_API_KEY=your-nvidia-api-key
R2_ACCESS_KEY_ID=your-r2-access-key
R2_SECRET_ACCESS_KEY=your-r2-secret-key
STRIPE_SECRET_KEY=your-stripe-secret-key- Setup database
mix ecto.setup- Start the server
mix phx.server- Open in browser
Navigate to http://localhost:4000
- Browse available Courses and enroll
- Access lesson materials and video content
- Record and upload your Performance
- Receive AI-powered analysis and feedback
- Track your progress and earn Achievements
- Create Courses with structured modules
- Upload lesson materials and recordings
- Review student performances
- Provide personalized feedback
- Student uploads audio recording
- System transcribes audio using NVIDIA Parakeet
- AI analyzes musical elements (timing, pitch, dynamics)
- Detailed feedback delivered via WebSocket
- Performance stored with vector embeddings for comparison
mix deps.get # Install dependencies
mix ecto.setup # Create, migrate, and seed database
mix phx.server # Start development server
mix test # Run test suite
mix test path:line # Run specific test
mix format # Format code
mix credo # Check code quality
mix ecto.reset # Reset database
mix assets.build # Build frontend assets
mix precommit # Run pre-commit checksThis project is under the MIT license. See LICENSE for details.
Contributions are welcome! Please feel free to submit a Pull Request.
- Fork the project
- Create your feature branch (
git checkout -b feature/AmazingFeature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Made with ❤️ by Maia
gabrielmaialva33@gmail.com | @gabrielmaialva33
Jim Studio - Empowering Musical Growth Through AI
© 2017-present Maia