A. Install Homebrew (package manager)

Open Terminal and run:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Then follow the on-screen "add to PATH" instructions it prints (copy/paste the echo line it gives you), or do:

echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile
eval "$(/opt/homebrew/bin/brew shellenv)"

B. Install Python 3 and git

brew install python git

C. Install Ollama (local LLM runner)

brew install --cask ollama

Launch the Ollama app (from Applications or Spotlight). That starts a local server.

1) Create your project & Python virtual environment

mkdir -p ~/local-rag && cd ~/local-rag
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip

2) Install Python libraries

pip install chromadb fastembed pypdf python-docx gradio requests

3) Pull a local model with Ollama

ollama pull llama3.1:8b-instruct-q4_K_M
# or
ollama pull llama3.2:3b-instruct-q4_K_M

Quick check that Ollama is running:

curl http://localhost:11434/api/tags

If it prints JSON with your model tag, you're good.

4) Put your documents in one folder

Make sure your data is here (PDF/DOCX/TXT/MD):

/Users/nassar/Documents/local_rag/data   (change this to your path)

5) Build the index (once, and whenever files change)

Create build_index.py file:

Run it (first run downloads the small embedding model once):

python build_index.py

You should see something like:

Found X files; prepared Y non-empty chunks.
Computed Y embeddings.
Indexed Y chunks.

6) Start the local chat app (RAG over your index)

Create app.py file:

Run it:

python app.py

Open the local URL it prints (usually http://127.0.0.1:7860)

Re-index when files change

Any time you add/edit files in /Users/nassar/Documents/local_rag/, rebuild:

python build_index.py

(You can keep the chat running; restart if you like.)