Error: KoboldCpp HTTP Server is running, but this endpoint does not exist. Please check the URL

[CorentinWicht]

Describe the Issue
Dear Developper,

I have just upgraded to koboldcpp-1.102 because I wanted to test the new llama.cpp UI which may be more adapted to our University setup. But when I try to reach it using the /lcpp endpoint this is what I get:
Error: KoboldCpp HTTP Server is running, but this endpoint does not exist. Please check the URL.

FYI, I am not doing it on localhost but remotely using our https redirection: https://frillm.XXX.ch/lcpp.
What is strange is that I can reach the https://frillm.XXX.ch/l/sdui enpoint perfectly fine.

I went through the --help but couldn't find anything related to an argument that should be added in the execution command.

Additional Information:
Operating System = RHEL 9.6
CPU = AMD Ryzen 9 7950X
GPUs = 2x RTX A600 (48Go VRAM)
KoboldCpp Version = 1.101

content of my /etc/systemd/system/koboldai.service:

[Unit]
Description=Run script at startup after network becomes reachable
After=network.target
[Service]
Type=simple
RemainAfterExit=yes
Restart=always
ExecStart=/usr/bin/python /opt/koboldcpp-latest/koboldcpp.py /home/frillm/Models/Language/Llama-4-Scout-17B-16E-Instruct-Q6_K-00001-of-00002.gguf --port 8008 --multiuser 20 --highpriority --usecublas --websearch --tensor_split 40 58 --gpulayers 47 --nommap --preloadstory /opt/settings.json --flashattention --sdmodel /home/frillm/Models/Images/flux1-schnell-fp8.safetensors --sdquant 2 --sdflashattention --whispermodel /home/frillm/Models/Whisper/whisper-base.en-q5_1.bin
EnvironmentFile=/etc/koboldai-env.conf
TimeoutStartSec=0
 
[Install]
WantedBy=default.target

Best wishes,

C.

[lone-cloud]

You're using the wrong kcpp version. lcpp is only available starting in 1.102 and you're still on 1.101
I was testing lcpp today and it's working fine.

[LostRuins]

No extra commands are needed. I see you have a /l/ subdirectory, is that intentional?

[CorentinWicht]

@lone-cloud & @LostRuins many thanks for your reply and my deepest apologies as I forgot to change my soft link to point to the new 1.102. It works great in 1.102.
The /l/ subdirectory was actually a typo.

I was just wondering if there is a way to load defaut settings for the llama.cpp UI, similar to what can be achieved with -preloadstory /opt/settings.json ?

By the way thanks a lot for your great work !

[LostRuins]

No, llama.cpp ui is a very "basic" ui that will lack many of the features like KoboldAI Lite has. It's more geared towards the ChatGPT assistant crowd and will not support koboldcpp-exclusive features such as multiplayer modes, roleplay, image generation, default saved stories/settings etc.

The copy in KoboldCpp is mostly unmodified from the UI upstream.
For a description of what the UI can do, check out ggml-org#14839

[CorentinWicht]

No, llama.cpp ui is a very "basic" ui that will lack many of the features like KoboldAI Lite has. It's more geared towards the ChatGPT assistant crowd and will not support koboldcpp-exclusive features such as multiplayer modes, roleplay, image generation, default saved stories/settings etc.

The copy in KoboldCpp is mostly unmodified from the UI upstream. For a description of what the UI can do, check out ggml-org#14839

Thanks for your reply, I understood that it's a much simplified UI compared to KoboldAI and I was just wondering whether it would be possible to load default settings that would be applied as baseline for all our users (e.g. forcing the "Dark" theme):

I guess if I just press on "Save Settings" this will only apply to myself and not to others accessing the UI with the same URL.

For KoboldAI Lite we used to do it by passing the -preloadstory /opt/settings.json argument only using 5% of available arguments, we discussed this previously here: #388.

I have checked ggml-org#14839 and could identify that settings are likely saved here (though I am not at all familar with Svelte):
settings.svelte.ts - Handles user preferences, theme management, and configuration persistence with localStorage sync

Do you think you could add an argument that we could pass to the koboldcpp.py so that we can load default settings from a `xxx.json file ?

Best wishes,

C.

[LostRuins]

Yeah because the saving happens locally only in browser I think. I'm not sure what would be required to make it load from a file, since I'm completely unfamiliar with the svelte code myself. But even if I supplied a settings.json, the frontend would still have to interpret and decode the file into its settings which is the harder part.

Technically serving the file is not the issue (in fact the preload story endpoint is just a simple HTTP file storage and serves it, you can access the raw data directly from anywhere using http://localhost:5001/api/extra/preloadstory ), but the Client itself needs to decode the file into something useful. And developing for svelte is beyond the scope for me.

[CorentinWicht]

Yeah because the saving happens locally only in browser I think. I'm not sure what would be required to make it load from a file, since I'm completely unfamiliar with the svelte code myself. But even if I supplied a settings.json, the frontend would still have to interpret and decode the file into its settings which is the harder part.

Technically serving the file is not the issue (in fact the preload story endpoint is just a simple HTTP file storage and serves it, you can access the raw data directly from anywhere using http://localhost:5001/api/extra/preloadstory ), but the Client itself needs to decode the file into something useful. And developing for svelte is beyond the scope for me.

Dear @LostRuins, now I better understand that it's completely out of your project's scope.

No worries and thanks for your help

[CorentinWicht]

Dear @LostRuins,

I just found out by reading the llama.cpp UI discussion that some of the default parameters can be overriden when starting the llama-server, for example:

Wouldn't it be possible to manipulate them in the koboldcpp.py start command to be passed to llama-serve, if that's how you run the llama.cpp UI internally ?

More information here: ggml-org#16515

Best wishes,

C.

[LostRuins]

I don't want to add dedicated flags solely for setting parameters in the lcpp ui though. Do you have a list of specific defaults you want to change? I might be able to make them configurable via the adapter instead.

[CorentinWicht]

I don't want to add dedicated flags solely for setting parameters in the lcpp ui though. Do you have a list of specific defaults you want to change? I might be able to make them configurable via the adapter instead.

I understand that this would add tons of new argument solely to control the the behaviour of the llama.cpp UI, but why not wrap them in a llamacpp-config.json ?

We are solely interested in the "General" tab configuration such as :
o Keep stats visible after generation (off by default)

I misread the "System Message" option because I thought this was some kind of a message that we could display to all users
when they access the service for the first time (including maintenance schedules, etc) but it's not the case (it controls the behaviour of the model instead).

[LostRuins]

I don't actually think the 'keep stats visible' will work as koboldcpp server does not send llama.cpp specific stats, at least when i turned it on I did not notice any difference.

Anyway I will keep this in mind but for now the llama.cpp ui is not really a priority, mostly provided as-is

[CorentinWicht]

I don't actually think the 'keep stats visible' will work as koboldcpp server does not send llama.cpp specific stats, at least when i turned it on I did not notice any difference.

Anyway I will keep this in mind but for now the llama.cpp ui is not really a priority, mostly provided as-is

Very well, thank you !
From their guide, I also understand that it's quite new and that they are actively working to develop it further (like including a web search functionality). Let's maybe wait a bit and see what's coming for next year.

Best,

C.