🤖 Flappy Bird AI with Reinforcement Learning

This project implements a complete Flappy Bird game with AI using reinforcement learning. It features both a web-based version with optimized Q-learning and a Python version with advanced Deep Q-Network (DQN) implementation.

🚀 QUICK START - Web Version (Recommended)

Option 1: Direct Launch (Easiest)

1. Double-click launch_training.html to open the launcher
2. Click "🚀 Start Training" button
3. Select "Let AI Play" to watch the AI learn
4. Watch the AI master the game in real-time!

Option 2: Direct Access

1. Open web/index.html in any modern web browser
2. Choose your mode:
   • "Play Yourself": Manual control (Spacebar/Click to jump)
   • "Let AI Play": Watch AI training with live statistics
3. No installation required! Everything runs in your browser

🎯 Web Version Features (Optimized for Fast Learning)

Fixed Pattern Training

• ✅ Consistent pipe pattern: Same sequence every time for faster learning
• ✅ No randomness: Predictable map layout for pattern recognition
• ✅ Enhanced Q-learning: Optimized hyperparameters for rapid convergence
• ✅ Real-time visualization: Live training statistics and progress tracking

AI Training Dashboard

• 📊 Episode counter and training progress
• 🎯 Exploration rate (epsilon) tracking
• 📈 Average & best scores with progress bar
• 🧠 Learning rate and training status
• 💾 Auto-save: Progress saved in browser storage

Expected Training Results

• Episodes 1-25: Learning basic controls (scores 0-3)
• Episodes 25-75: Pattern recognition (scores 3-8)
• Episodes 75-150: Consistent performance (scores 8-15)
• Episodes 150+: Mastery level (scores 15-30+)

🐍 Python Version (Advanced)

Installation

# Install required packages
pip install pygame tensorflow numpy matplotlib

# Or install from requirements file
pip install -r python/requirements.txt

GPU-Accelerated Training

# Ultra-fast GPU training (recommended)
python python/fast_train.py

# Standard training with visualization
python python/train.py --episodes 1000

# Training without rendering (fastest)
python python/train.py --episodes 2000 --no-render

Testing Trained Models

# Watch trained AI play
python python/play.py --mode ai

# Play manually yourself
python python/play.py --mode manual

# Test specific model
python python/fast_train.py test

📁 Project Structure

flappy-bird-ai/
├── 🚀 launch_training.html          # Quick launcher for web version
├── 📁 web/                          # Web-based AI training
│   ├── index.html                   # Main game interface
│   ├── game.js                      # Game engine with fixed patterns
│   ├── ai.js                        # Optimized Q-learning AI
│   ├── main.js                      # Application controller
│   └── style.css                    # Enhanced UI styling
├── 📁 python/                       # Advanced Python implementation
│   ├── flappy_env.py               # PyGame environment
│   ├── dqn_agent.py                # Deep Q-Network with GPU support
│   ├── fast_train.py               # GPU-optimized training
│   ├── train.py                    # Standard training script
│   ├── play.py                     # Testing and play script
│   └── requirements.txt            # Python dependencies
└── 📄 README.md                    # This file

🎮 How to Use

Web Version (Recommended for Beginners)

1. Launch: Open launch_training.html or web/index.html
2. Train: Click "Let AI Play" and watch the learning process
3. Monitor: Observe real-time statistics and progress
4. Play: Switch to "Play Yourself" to try manual control

Python Version (For Advanced Users)

1. Install: Run pip install -r python/requirements.txt
2. Train: Execute python python/fast_train.py for GPU training
3. Monitor: Watch training plots and statistics
4. Test: Run python python/play.py --mode ai to see results

🧠 AI Architecture

Web Version (Q-Learning)

• Algorithm: Enhanced Q-learning with pattern recognition
• State Space: 9 features including bird position, pipe gaps, velocity
• Action Space: Jump (1) or Do Nothing (0)
• Learning Rate: 0.3 (optimized for fixed patterns)
• Memory: 2000 experience replay buffer
• Exploration: Epsilon-greedy with decay (0.8 → 0.05)

Python Version (Deep Q-Network)

• Architecture: Neural network with batch normalization
• GPU Support: TensorFlow GPU acceleration
• State Space: 9 normalized features with lookahead
• Experience Replay: 20,000 experience buffer
• Target Network: Double DQN with periodic updates
• Optimization: Adam optimizer with learning rate scheduling

🏆 Training Performance

Web Version Results

• Training Speed: ~50-100 episodes for basic competency
• Target Performance: Average score >15 in 50 episodes
• Best Scores: 20-40+ points achievable
• Training Time: 5-15 minutes for good performance

Python Version Results

• Training Speed: ~200-500 episodes for mastery
• Target Performance: Average score >25 in 50 episodes
• Best Scores: 50-100+ points achievable
• Training Time: 10-30 minutes (with GPU)

🎯 Reward System

Enhanced Reward Structure

• +10: Successfully passing through a pipe
• +2: Being close to gap center
• +1: Good positioning near pipes
• +0.1: Survival bonus per frame
• -1: Poor positioning (too high/low)
• -100: Collision with pipe or ground

🔧 Customization

Adjusting AI Parameters

// In web/ai.js - modify these values:
this.learningRate = 0.3;        // How fast AI learns
this.epsilon = 0.8;             // Initial exploration rate
this.epsilonDecay = 0.998;      // Exploration decay rate

# In python/dqn_agent.py - modify these values:
self.learning_rate = 0.001      # Neural network learning rate
self.epsilon = 0.95             # Initial exploration rate
self.batch_size = 128           # Training batch size

Changing Game Difficulty

// In web/game.js - modify fixed pattern:
const fixedPattern = [200, 150, 250, 180, 220, 160, 240, 190, 210, 170];
// Change these numbers to adjust pipe heights

🚨 Troubleshooting

Web Version Issues

• AI not learning: Refresh page and try again
• Slow performance: Close other browser tabs
• No statistics: Ensure JavaScript is enabled
• Reset training: Clear browser storage/cookies

Python Version Issues

# Common fixes:
pip install --upgrade tensorflow pygame numpy

# For GPU issues:
pip install tensorflow-gpu

# For memory errors:
# Reduce batch_size in dqn_agent.py

# For slow training:
python python/fast_train.py  # Use optimized version

📊 Monitoring Training

Web Version Indicators

• Progress Bar: Shows mastery level (target: 100%)
• Training Status: Color-coded learning phases
• Episode Counter: Total training episodes completed
• Best Score: Highest score achieved so far

Python Version Outputs

• Real-time plots: Training curves and statistics
• Console logs: Episode scores and performance metrics
• Model checkpoints: Automatic saving of best models
• Performance analysis: Detailed training summaries

🎉 Success Metrics

Beginner Success (Web)

• ✅ AI consistently scores 5+ points
• ✅ Passes through 3+ pipes regularly
• ✅ Training completes in <100 episodes

Advanced Success (Python)

• ✅ AI consistently scores 20+ points
• ✅ Passes through 10+ pipes regularly
• ✅ Achieves mastery in <500 episodes

💡 Tips for Best Results

1. Web Version: Let it run continuously for best learning
2. Python Version: Use GPU acceleration when available
3. Both: Monitor the statistics to track improvement
4. Patience: AI learning takes time - watch the progress!
5. Experimentation: Try adjusting hyperparameters for different results

---

🎯 Ready to Start?

For immediate results: Open launch_training.html and click "Start Training"!

For advanced training: Run python python/fast_train.py after installing dependencies.

Watch your AI evolve from random flapping to masterful gameplay! 🚀

Game Controls

Manual Play

• Web: Spacebar or mouse click to jump
• Python: Spacebar to jump, ESC to quit

AI Training

• The AI learns automatically through trial and error
• Training progress is displayed in real-time
• Models are saved periodically during training

AI Architecture

Web Version (Q-Learning)

• State Space: Bird position, pipe positions, velocity, distance
• Action Space: Do nothing (0) or jump (1)
• Learning: Q-table with discretized states
• Memory: Experience replay buffer (1000 experiences)

Python Version (Deep Q-Network)

• Neural Network: 3 hidden layers (128, 128, 64 neurons)
• State Space: 5 normalized features (bird_y, gap_top, gap_bottom, velocity, distance)
• Experience Replay: 10,000 experience buffer
• Target Network: Updated every 100 training steps
• Optimization: Adam optimizer with learning rate 0.001

Training Progress

The AI typically shows improvement patterns:

1. Episodes 0-100: Random exploration, low scores
2. Episodes 100-500: Learning basic survival, scores 1-10
3. Episodes 500-1000: Consistent pipe passing, scores 10-30
4. Episodes 1000+: Advanced play, scores 30-100+

File Descriptions

Web Files

• index.html: Game interface with mode selection and statistics
• game.js: Core game engine with bird physics, pipe generation, collision detection
• ai.js: Q-learning implementation with epsilon-greedy policy
• main.js: Application controller managing game loop and AI integration
• style.css: Modern responsive design with animations

Python Files

• flappy_env.py: Complete game environment implementing OpenAI Gym interface
• dqn_agent.py: DQN agent with neural network, experience replay, and target network
• train.py: Training script with progress visualization and model saving
• play.py: Testing script for both manual and AI play modes

Customization

Adjusting Difficulty

• Modify updateDifficulty() in game.js or _update_difficulty() in flappy_env.py
• Change pipe speed, gap size, or spawn intervals

AI Hyperparameters

• Web: Edit learning rate, epsilon decay, memory size in ai.js
• Python: Modify network architecture, learning rate, batch size in dqn_agent.py

Reward System

• Adjust rewards in getReward() (web) or _calculate_reward() (python)
• Try different reward structures for faster learning

Troubleshooting

Web Version

• Ensure JavaScript is enabled in your browser
• Use a modern browser (Chrome, Firefox, Safari, Edge)
• Check browser console for any error messages

Python Version

• Install all dependencies: pip install -r python/requirements.txt
• Ensure Python 3.7+ is installed
• For GPU acceleration, install tensorflow-gpu

Common Issues

• Slow training: Use --no-render flag for faster training
• Memory issues: Reduce batch size or memory size in agent
• Poor performance: Try adjusting learning rate or network architecture

Performance Tips

1. Web: The AI learns faster with consistent play sessions
2. Python: Use GPU acceleration for faster neural network training
3. Both: Adjust epsilon decay rate for better exploration/exploitation balance
4. Training: Start with smaller episode counts to test setup

This implementation provides a complete learning environment for understanding reinforcement learning concepts while enjoying the classic Flappy Bird game!