File tree Expand file tree Collapse file tree 1 file changed +2
-7
lines changed
Expand file tree Collapse file tree 1 file changed +2
-7
lines changed Original file line number Diff line number Diff line change @@ -166,19 +166,14 @@ cd llama.cpp
166166make
167167
168168# Download the SmolLM2-1.7B-Instruct-GGUF model
169- curl -L -O https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct.Q4_K_M. gguf
169+ curl -L -o smollm2-1.7b-instruct-q4_k_m.gguf " https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct-q4_k_m. gguf?download=1 "
170170```
171171
172172Then, launch the server (with OpenAI API compatibility):
173173
174174``` sh
175175# Start the server
176- ./server \
177- -m smollm2-1.7b-instruct.Q4_K_M.gguf \
178- --host 0.0.0.0 \
179- --port 8080 \
180- -c 4096 \
181- --n-gpu-layers 0 # Set to a higher number to use GPU
176+ ./build/bin/llama-server -m /mnt/c/Users/yassi/llama.cpp/smollm2-1.7b-instruct-q4_k_m.gguf --host 0.0.0.0 --port 8080 -c 4096 --n-gpu-layers 0
182177```
183178
184179Interact with the server using Hugging Face's InferenceClient:
You can’t perform that action at this time.
0 commit comments