MAGK Excel is an AI-powered desktop application designed to revolutionize how professionals handle repetitive data extraction and processing tasks. By combining natural language processing with deep Excel integration, MAGK transforms complex, multi-step workflows into simple, one-click, reusable automation tools accessible to users with zero programming knowledge.
- Conversational Workflow Builder: Define automation tasks using natural language
- Web & PDF Data Extraction: Extract tabular data from websites and PDF documents
- Excel Integration: Seamless manipulation and generation of Excel files
- Customizable Tool Generation: Create standalone, clickable automation programs
- Zero Programming Knowledge Required: Designed for non-technical professionals
MAGK addresses the critical pain point that professionals spend excessive time on manual, repetitive tasks like:
- Navigating websites to find specific reports and data
- Manually downloading PDFs and Excel files
- Copy-pasting data with high error rates
- Consolidating information across multiple sources
By automating these workflows, MAGK enables users to focus on high-value activities like analysis, modeling, and generating insights. Unlike general-purpose AIs, MAGK creates permanent, customizable automation tools that become reusable assets for recurring tasks.
MAGK is built as a hybrid client-server application with the following architecture:
graph TD
subgraph "User's Desktop"
U[User] --> C["MAGK Desktop Client (.exe)"]
C -- Manages --> WL["Workflow Library (local JSON)"]
C -- File Upload --> AG[API Gateway]
C -- API Call w/ Workflow Config --> AG
C -- Simulates real-time update --> XL[Local Excel File]
end
subgraph "AWS Cloud"
AG --> L[Lambda Function: Workflow Executor]
L -- Reads from --> S3[S3 Bucket]
L --> WEB[(Website)]
L --> PDF[(PDF on S3)]
L -- Writes to --> S3
end
S3 -- Signed URL for Download --> C
sequenceDiagram
participant User
participant Client as Desktop Client
participant API as API Gateway
participant Lambda as Lambda Function
participant S3 as S3 Storage
participant External as External Sources
User->>Client: Describe workflow in chat
Client->>Client: Generate WorkflowConfig
Client->>API: POST /execute-workflow
API->>Lambda: Trigger workflow execution
Lambda->>External: Extract data from web/PDF
External-->>Lambda: Return extracted data
Lambda->>S3: Store processed Excel file
S3-->>Lambda: Return file URL
Lambda-->>API: Return output URL
API-->>Client: Return download link
Client->>S3: Download Excel file
Client->>User: Display results in Excel
graph LR
subgraph "Client Application"
UI[PyQt GUI]
API[API Client]
WF[Workflow Manager]
EX[Excel Integration]
end
subgraph "Backend Services"
AG[API Gateway]
L1[Lambda: Workflow Parser]
L2[Lambda: Data Extractor]
L3[Lambda: Excel Generator]
S3[S3 Storage]
BR[Amazon Bedrock]
end
UI --> API
API --> AG
AG --> L1
L1 --> BR
L1 --> L2
L2 --> L3
L3 --> S3
WF --> EX
| Component | Technology | Purpose |
|---|---|---|
| Client | Python 3.10+ | Desktop application logic |
| GUI | PyQt 6.x | Native desktop interface |
| Backend | AWS Chalice | Serverless API framework |
| AI | Amazon Bedrock | Natural language processing |
| Web Scraping | Selenium 4.x | Dynamic website automation |
| PDF Parsing | PyMuPDF 1.23.x | Document text extraction |
| Excel | openpyxl 3.1.x | Spreadsheet manipulation |
| Packaging | PyInstaller 6.x | Windows executable creation |
Jack Luo
Operating System: Windows 11
Python Version: 3.13.5
Developer: Kian Hong
Operating System: macOS 15.3.1
Python Version: 3.13
Karthik
Operating System: macOS
Python Version: ???
Arman
Status: Not yet onboarded
We welcome contributions! Please see our Contributing Guidelines for details on:
- Code style and standards
- Pull request process
- Issue reporting
- Development setup
- Python 3.10 or higher
- Microsoft Excel (2007 or later, including .xls format support)
- Windows 10/11 (primary target platform)
- AWS Account (for backend services)
-
Clone the repository
git clone https://github.com/your-org/magk-excel.git cd magk-excel -
Set up Python virtual environment
python -m venv venv venv\Scripts\activate
-
Install client dependencies
cd apps/client pip install -r requirements.txt -
Install server dependencies
cd ../server pip install -r requirements.txt -
Configure AWS credentials
aws configure
-
Deploy backend services
chalice deploy
-
Run the application
cd ../client python src/main.py
-
Clone the repository
git clone https://github.com/your-org/magk-excel.git cd magk-excel -
Set up Python virtual environment
python3 -m venv venv source venv/bin/activate -
Install dependencies
cd apps/client pip install -r requirements.txt cd ../server pip install -r requirements.txt
-
Configure AWS credentials
aws configure
-
Deploy and run
chalice deploy cd ../client python src/main.py
-
Clone the repository
git clone https://github.com/your-org/magk-excel.git cd magk-excel -
Set up Python virtual environment
python3 -m venv venv source venv/bin/activate -
Install system dependencies (Ubuntu/Debian)
sudo apt-get update sudo apt-get install python3-dev python3-pip sudo apt-get install libgl1-mesa-glx libglib2.0-0
-
Install Python dependencies
cd apps/client pip install -r requirements.txt cd ../server pip install -r requirements.txt
-
Configure and run
aws configure chalice deploy cd ../client python src/main.py
-
Install development dependencies
pip install -r requirements-dev.txt
-
Set up pre-commit hooks
pre-commit install
-
Run tests
pytest
The project includes helper scripts organized in the scripts/ directory:
scripts/
├── setup/ # Environment setup and configuration
│ ├── check-env.bat # Windows: Pre-setup environment verification
│ ├── setup.bat # Windows: Create venv and install dependencies
│ └── setup.sh # Linux/Mac: Create venv and install dependencies
│
└── testing/ # Testing and validation scripts
├── test.bat # Windows: Run pytest test suite
├── test_python.bat # Windows: Debug Python/venv detection
└── verify_fix.bat # Windows: Verify setup fixes
-
Check your environment (Windows only):
cd apps/server ..\..\scripts\setup\check-env.bat
-
Run setup:
- Windows:
..\..\scripts\setup\setup.bat - Linux/Mac:
../../scripts/setup/setup.sh
- Windows:
From the app directory (e.g., apps/server):
- Windows:
..\..\scripts\testing\test.bat - Linux/Mac:
pytest tests/ -v
Each app maintains its own virtual environment:
- Server:
apps/server/venv/ - Client:
apps/client/venv/
This approach ensures:
- Dependency isolation between apps
- No version conflicts
- Standard Python project structure
- Easy CI/CD integration
- Windows scripts use
.batextension andpycommand - Linux/Mac scripts use
.shextension andpython3command - Virtual environments have different structures:
- Windows:
venv\Scripts\activate.bat - Linux/Mac:
venv/bin/activate
- Windows:
magk-excel/
├── apps/
│ ├── client/ # Desktop client application
│ │ ├── src/
│ │ │ ├── ui/ # PyQt UI components
│ │ │ ├── api/ # Backend API client
│ │ │ ├── workflows/ # Local workflow management
│ │ │ └── main.py # Application entry point
│ │ └── tests/
│ └── server/ # AWS serverless backend
│ ├── app.py # Chalice application
│ ├── chalicelib/ # Backend modules
│ │ ├── excel_integration/
│ │ ├── excel_parsers/
│ │ ├── pdf_parsers/
│ │ └── streaming/
│ └── requirements.txt
├── docs/ # Project documentation
│ ├── brief.md # Project brief
│ ├── architecture.md # Technical architecture
│ ├── prd.md # Product requirements
│ └── ux.md # UX specifications
├── web-bundles/ # AI agent configurations
└── README.md
Create a .env file in the project root:
# AWS Configuration
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
# Application Configuration
MAGK_ENVIRONMENT=development
MAGK_LOG_LEVEL=INFO- Create S3 bucket for file storage
- Set up API Gateway with API keys
- Configure Lambda functions with appropriate IAM roles
- Set up CloudWatch for logging
# Run all tests
pytest
# Run specific test categories
pytest tests/unit/
pytest tests/integration/
pytest tests/e2e/
# Run with coverage
pytest --cov=apps- Unit Tests: Individual component testing
- Integration Tests: API and service integration
- E2E Tests: Full workflow testing
- Performance Tests: Load and stress testing
# Build Windows executable
pyinstaller --onefile --windowed apps/client/src/main.py# Deploy backend services
chalice deploy
# Update environment
chalice deploy --stage production-
PyQt Installation Issues
# On macOS brew install qt pip install PyQt6 -
AWS Credentials
aws configure list aws sts get-caller-identity
-
Excel Integration
- Ensure Microsoft Excel is installed
- Check file permissions
- Verify .xls format compatibility
- Issues: GitHub Issues
- Documentation: Project Wiki
- Discussions: GitHub Discussions
This project is licensed under the MIT License - see the LICENSE file for details.
- BMad Team: For the AI-driven development framework
- AWS Chalice: For the serverless framework
- PyQt: For the desktop GUI framework
- Financial Analysts: For domain expertise and feedback