feat: do not bundle llama-cpp anymore

[mudler]

Description

This PR fixes #

Notes for Reviewers

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	feecf58
🔍 Latest deploy log	https://app.netlify.com/projects/localai/deploys/687a2abfc9ffd40008172e46
😎 Deploy Preview	https://deploy-preview-5790--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[richiejp]

So a completely separate Dockerfile and Makefile? This will be a major improvement!

[mudler]

So a completely separate Dockerfile and Makefile? This will be a major improvement!

yup! my plan is to isolate everything, one backend at a time. Currently the llama.cpp one is the most heavy, having also lots of specific code in the golang part - ideally I want to get rid of all of the specific llama.cpp code and the binary bundling bits out of the main code.

This is how I'm testing things now with #5816 in:

docker build --build-arg BACKEND=llama-cpp -t llama-cpp-backend -f backend/Dockerfile.llama-cpp .
docker save llama-cpp-backend -o llama-backend.tar
local-ai backends install "ocifile://$PWD/llama-backend.tar"

[mudler]

Any tips on testing this? Does it significantly change the build process for those compiling locally? (So does this require a README update?)

yes actually good point, my plan is to remove all the backends outside so we can build LocalAI in a simpler way by using standard golang tooling. At that point I will re-work documentation, at this stage is not really functional and in a "transient" state. However, for now the steps are the same:

make build # Will build local-ai, with some backends still included in the binary

The difference is in how backends are built. If you want llama-cpp, for example, you can install it from the Backends tab in the webui, or with localai backends install. If you want to build it instead and install you can run:

make docker-build-llama-cpp
make docker-save-llama-cpp
./local-ai backends install ocifile://$PWD/backend-images/llama-cpp.tar

This does the following:

• Builds the backend with docker
• Save the result as a standard container image
• Install it in local-ai (in the default backends folder, next to the binary)