Skip to content

Commit 8e9ff95

Browse files
authored
Update model download and server launch commands
The curl command didn't provide the correct file and since the last version of this course the server location has changed to ./build/bin/llama-server
1 parent 5739145 commit 8e9ff95

File tree

1 file changed

+2
-7
lines changed
  • chapters/en/chapter2

1 file changed

+2
-7
lines changed

chapters/en/chapter2/8.mdx

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -166,19 +166,14 @@ cd llama.cpp
166166
make
167167

168168
# Download the SmolLM2-1.7B-Instruct-GGUF model
169-
curl -L -O https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct.Q4_K_M.gguf
169+
curl -L -o smollm2-1.7b-instruct-q4_k_m.gguf "https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct-q4_k_m.gguf?download=1"
170170
```
171171

172172
Then, launch the server (with OpenAI API compatibility):
173173

174174
```sh
175175
# Start the server
176-
./server \
177-
-m smollm2-1.7b-instruct.Q4_K_M.gguf \
178-
--host 0.0.0.0 \
179-
--port 8080 \
180-
-c 4096 \
181-
--n-gpu-layers 0 # Set to a higher number to use GPU
176+
./build/bin/llama-server -m /mnt/c/Users/yassi/llama.cpp/smollm2-1.7b-instruct-q4_k_m.gguf --host 0.0.0.0 --port 8080 -c 4096 --n-gpu-layers 0
182177
```
183178

184179
Interact with the server using Hugging Face's InferenceClient:

0 commit comments

Comments
 (0)