Improving experience for new users

[Nolendil]

Hello,

After reading #186, I'd like to add to this topic as a Windows user with little space on their C drive.
I've used a few other generative AI tools (e.g. https://github.com/oobabooga/text-generation-webui, https://github.com/lllyasviel/Fooocus/, https://github.com/lllyasviel/stable-diffusion-webui-forge) and expected a similar experience with this one.
I really appreciate the one-click installers and having the whole tool and dependencies being contained in their own folder.

However, the experience was slightly different with this tool:

During the install, pip was downloading and building packages in my temp folder (on C:). I had enough space left but it used about 10 GB and that could have been an issue. I don't think other AI tools did that (but I may have not noticed). Opting to install all the dependencies at once probably amplified the storage required.
If there's a way to tell pip to use a specific working folder and set that to a temp folder inside tts-generation-webui's folder, that would be great.

So far, I've only tried Bark and MAGNet and mostly used the React UI.
I figured that Bark models go in the "voices" folder but I got a bit confused when trying to use MAGNeT. The React UI or the terminal didn't provide any info when I first attempted to generate something with MAGNeT, until it suddenly started to download files (which was only visible on the terminal) and eventually generated what I had requested. I think some feedback was missing there.

Also, while I expected the downloaded files to be put somewhere inside the tts-generation-webui's folder (just like "voices"), I eventually realized they were created on my C: drive and found the HuggingFace cache folder.
This lead me to find #186 and wonder what would be the best way to change the environment variables.
Just now, I've found that the Gradio interface allows setting the environment variables and hope that it will solve my problem. I still have to figure the format to make it a relative path. I didn't find that setting in the React UI.
Even though it can be set later, I think the cache locations is something that should be asked when launching the tool for the first time.

Proposed solutions to improve the first launch experience:

• Make pip's working folder reside in the application folder, if possible.
• Warn that pip may (temporarily) use a lot of space when offering to install all dependencies at once.
• During installation, ask the user where to put the various extra folders (Hugging Face Cache, Torch Hub Cache, others?) and suggest reasonable choices (e.g. either a location to keep the application self-contained, the default location in the home folder or a custom folder)
• Create a proper page to explain the various cache folders (purpose, default location and how to change them) and link it in the Installation section of the ReadMe. The ReadMe currently links to Downloading models and stuff #186 but it's mentioned quite far at the bottom and it's unlikely people would notice it or know that they should read it until they face the problem. Also while Downloading models and stuff #186 provides some information, it's very bare bones and requires more research from the user to find what action they actually need to do. Mentioning the Settings page on the Gradio interface would help.
• Having two different UIs is very confusing for a new user. The Gradio one seems more feature complete, while the React one seems more streamlined. This should be explained in the ReadMe too, I think.
• Explaining how to add models and where. Mentioning what is automatically downloaded and what needs to be manually downloaded.

Anyway, thanks a lot for your work on this project, it's very appreciated.

[rsxdalv]

During the install, pip was downloading and building packages in my temp folder (on C:). I had enough space left but it used about 10 GB and that could have been an issue. I don't think other AI tools did that (but I may have not noticed). Opting to install all the dependencies at once probably amplified the storage required.
If there's a way to tell pip to use a specific working folder and set that to a temp folder inside tts-generation-webui's folder, that would be great.

Actually it is supposed to do that. If conda and pip are working correctly, all of the pip dependencies during install should be within the tts-webui folder, not outside. However, some features always get made that use some 'user' cache, like C:/Users/user/.cache or app_data etc etc and you only realize that by checking the internal code. I.e., the official and original bark uses .cache due to huggingface's 'download' implementation.
I am interested if you could share any more details about the temp folder, since I might have missed that on my own system.

So far, I've only tried Bark and MAGNet and mostly used the React UI.
I figured that Bark models go in the "voices" folder but I got a bit confused when trying to use MAGNeT. The React UI or the terminal didn't provide any info when I first attempted to generate something with MAGNeT, until it suddenly started to download files (which was only visible on the terminal) and eventually generated what I had requested. I think some feedback was missing there.

Thanks, I will make an issue for adding more feedback on gradio/React about what is visible in the console: #488

Also, while I expected the downloaded files to be put somewhere inside the tts-generation-webui's folder (just like "voices"), I eventually realized they were created on my C: drive and found the HuggingFace cache folder.
This lead me to find #186 and wonder what would be the best way to change the environment variables.
Just now, I've found that the Gradio interface allows setting the environment variables and hope that it will solve my problem. I still have to figure the format to make it a relative path. I didn't find that setting in the React UI.
Even though it can be set later, I think the cache locations is something that should be asked when launching the tool for the first time.

Yes, at this point I think I have the capacity to improve the UX for changing the cache location.

One note - for better or worse, most projects such as Audiocraft, Bark, Tortoise etc have adapted this pattern for the last 2+ years. I think as a webui I should explore ways of 'patching' these features.

I will review more later.

[Nolendil]

I am interested if you could share any more details about the temp folder, since I might have missed that on my own system.

I'm not sure what details I can provide. The temp folders were deleted after installation.
I know it used C:\Users\<username>\AppData\Local\Temp but I didn't think to check ~/.cache during the installation.
There were up to 450-500 folders in AppData\Local\Temp with names like pip-build-xxxx and similar (pip-tmp-xxx?). The amount of folders didn't go down at any point except once it was completely done.
Is there any other specific information I could provide?

Yes, at this point I think I have the capacity to improve the UX for changing the cache location.

Sounds great. Thanks!

most projects [...] have adapted this pattern for the last 2+ years. I think as a webui I should explore ways of 'patching' these features.

If changing cache location is made easy, I don't mind it as much. "Patching" all those projects might be very time consuming.

[rsxdalv]

Ok, I'm not sure I will be able to solve that but I will create an issue so that the knowledge about it does not disappear. I might be able to reproduce it. #490

[rsxdalv]

By the way - one note - SD webui does download some transformers to HF Cache last I checked.

[Nolendil]

Checking my .cache folder, its only content is from tts-generation-webui judging by the file dates and file names (plus one file from yt-dlp). It's possible that I haven't used a function of SD webui that triggers using the HF Cache yet (or that I forgot that it happened, I haven't used it in a while).
Thanks for the info, that might help if I encounter similar issues with other applications :)

[rsxdalv]

I recommend using TreeSizeFree (portable) for scanning the .cache and other folders. I have seen many shenanigans such as 24gb sized .txt log file from avast, 4-6gb of NVidia Driver caches, 3gb .npm .pip .yarn .gradle etc caches and so on.
I'd say the average ML/AI project is server grade, meaning 20gb here, 40gb there, 20 system dependencies and everything inbetween as well. But it's pulling into consumer-grade (where devs need to invest a lot more time so that the software installation is devs' problem, not end users problem as it is with Dev Ops whose job it is to install stuff)

[rsxdalv]

I'm trying to see how I can implement the model download warnings and info, but sometimes the model code does not make it easy. For example, the new chatterbox model does the model download internally when you try to use it. Therefore, I have to modify the original model code to rectify this.

[Nolendil]

Is it related to #488 ?
Modifying the original code sounds scary for maintenance, but the alternatives that come to my mind may actually be worse (e.g. parsing the terminal output, checking for the presence of model files...) 😅
Maybe you could display a message when the user launches a problematic models, that says that model files may (or may not) be downloaded? And have it displayed only once per session.

[rsxdalv]

Yes, that's the gist of it. In fact, the default mode (which you can now see with chatterbox) is that the model gets downloaded somewhere (huggingface cache). So I usually try to override as much as I can to keep the models local to tts-webui, and to have control over the checkpoint selection.

I think I will get to a solution, it's just that right now there's a lot of stuff going on. I think soon enough there might be a quieter period, when I'm not working on:

• Quantization
• Training models
• Youtube
• File sharing site
• APIs for other projects
• Upgrading Pytorch again because RTX 50xx needs CUDA 12.8 and PyTorch 2.7.0

[rsxdalv]

During installation, ask the user where to put the various extra folders (Hugging Face Cache, Torch Hub Cache, others?) and suggest reasonable choices (e.g. either a location to keep the application self-contained, the default location in the home folder or a custom folder)

I really like this idea - here's an issue to track it: #489

[rsxdalv]

Having two different UIs is very confusing for a new user. The Gradio one seems more feature complete, while the React one seems more streamlined. This should be explained in the ReadMe too, I think.

I think I had some info on it, but originally I wanted to convert everything to React. React is meant to be the definitive while gradio is meant to be the uncomfortable pro barebones mode.

[Nolendil]

I mostly used the React UI because it is more appealing, so you're on the right track :)
I didn't find any settings on it, though, while Gradio has that.
(As a side note, I really like the HuggingFace Cache and GPU Info tools on React.)

[Nolendil]

Thanks for taking my feedback in account.

One extra thing:
I saw that the default path for the "tts-generation-webui Directory" in the Gradio interface was ./tts-generation-webui/data/models/.
However, the application folder is called tts-generation-webui-main on my drive (because that's its name inside the installation zip file), so I thought I should rename it to tts-generation-webui. Doing so made the application fail to start with the following output in the terminal:

Transcript started, output file is D:\<mypath>\tts-generation-webui\installer_scripts\output.log
"Checking if the Micromamba/Conda environment is already installed."
"Micromamba/Conda environment already installed in D:\<mypath>\tts-generation-webui\installer_files\env, skipping installation"
vswhere is installed, checking for Visual Studio Build Tools installation...
Visual Studio Build Tools is installed, continuing...
The system cannot find the path specified.
'node' is not recognized as an internal or external command,
operable program or batch file.
Failed to init the app, exiting...

The "output.log" file doesn't seem to give more details but I can provide it if needed.
Renaming the folder back solved the issue but it seems there's something (node.js?) that prevents the application from being portable (which would be great).
I assume the path displayed for "tts-generation-webui Directory" is slightly wrong (and I suppose it's hard-coded since it can't be changed) and it should actually display ./data/models/.

[rsxdalv]

It's a conda thing:

conda/conda#1845

Basically, conda installs with the absolute path rather than the relative path. This also tends to be a problem for people who just want to move the folder from say C: to D:
There are some manual fixes for it, the most painful one probably being to delete installer_files and redownload 4-6gb of python packages.

Because it only checks if installer_files/env exists it does not realize that it fails to activate, this is a bug and should be fixed:
#491

Before the first install, you can place the files anywhere with any folder name, but after installing it cannot be moved (unless some day conda stops using absolute paths, which they do not want to yet).

[Nolendil]

I see, thanks for all the information. Not much you can do on your side, indeed, since it's a long-lasting Conda issue.

Otherwise, I relocated all my cache folders (HF, Torch, etc) and it works fine :)

[rsxdalv]

It's been a busy week but I finally got around to implementing one change - info bubbles for when models are being loaded/unloaded.
My current strategy is to try to utilize gr.Info more so that the Gradio UI is as informative as the console.

[Alphaelectric]

I can confirm this bug. Moving my installation broke it and a new installation fails with these two errors:

tts-webui-react@0.4.0 prebuild
node generate-plugin-imports.js

No React UI extensions found

tts-webui-react@0.4.0 build
next build

Creating an optimized production build ...
Failed to compile.

Error: EISDIR: illegal operation on a directory, readlink 'H:\tts-webui\react-ui\src\pages_app.tsx'

[rsxdalv]

Moving it back can fix it if that is an option.

[Git2tony]

This project has all the potential to be something great even for relative n00bs like me, and I really appreciate all the work that has gone into this, but I can't for the life of me get any of it working properly. The main issues are (I'm using docker compose WSL2 inside Windows11 and use the basic docker compose from the repository):

1. Installing the 11Gigs is a problem with a slow internet connection and to be honest not what anyone wants.
2. When finally installed, I expected (and maybe wrongfully so) to see the latest Qwen3-TTS and AceStep1.5 in there...
3. I don't quite understand the difference between the 'normal' and the 'gradio' interface, for me the normal interface did not show the DIA TTS I wanted to try out first so, I went straight to the gradio UI
4. When you install something (like DIA in my case) it throws errors about rights, and it is wiped as soon as you down and up your container.... this for me is not at all pratical
   4a) I used this volume - tts_webui_pip_cache:/app/cache/pip (also adding it to the bottom of the docker compose file) to get rid of the permission errors
5. When trying to actually use the DIA TTS I clicked on the generate audio button and saw the loading model popup, then silence, I did not see any console or what is going on...
6. Nothing.... Apparently the XET is going into forbidden 430 loops when it almost has the whole model downloaded, without ever finishing it (I added this in the environment variables to be able to cache the HF - HUGGINGFACE_HUB_CACHE=/app/cache/hub and this one in the volumes: - ${PATH_TTS_WEBUI_CACHE}:/app/cache )
7. I used an environment flag (- HF_HUB_DISABLE_XET=1) to disable XET so normal http transfers would be used)
8. I finally had a sound generated
9. The sound is a garbled mess of crap
10. I post here, do I uninstall and forget? I really would like this all to work!
11. Ohh and if you down your container and re-up it, nothing is installed, everything is wiped!

What I would like is:

1. Have a separate volume for downloaded models (TTS_WEBUI_MODELS)
2. Have an option to maintain the installed features like DIA, preferably make it a separate volume like (TTS_WEBUI_PACKAGES)
3. Have packages actually working, and or, have a console output telling me what is going on

[rsxdalv]

Thanks for the feedback! I will see what I can go through. The config might break backwards compatibility so I will need some logic around a /config mountable directory that fallbacks on the current way.

As for Ace Step 1.5 - it arrived only recently, and it's only possible to get models in that fast if I'm already at full speed. For ComfyUI etc, they have 3 advantages - lower standard for initial releases, some model authors already include the nodes in the official model release, and a community where someone is always ready to make it. Once I'm back on track I'll do Ace-Step 1.5, then Chatterbox turbo, then Qwen (although the user feedback suggests that Qwen TTS 3 isn't quite so amazing, and it might be better once derived models arrive).

[Git2tony]

Thank you for the quick reply! And I fully understand that it takes time to implement the newest models!

So, for now, I think it would be really helpful if we could have a proper docker compose with the correct volumes and pip caching implemented, like for instance the ./data/models/ mentioned in this thread, is that purely a suggestion? and what is the voices volume for?

And what is difference between the normal UI and the gradio? (since the gradio seems to support more models)

And for the scrambled output I get, is there a way to verify if the downloaded model is corrupt or not? (I checked the DIA dependencies etc. all should be fine, the model does seem to load now from the hf cache hub folder.

Is there anything I can do to help?

Edit:

• I removed all the cache and re-downloaded the model (by just running the generate audio in Dia) and that solved the scrambled output, so not Dia properly generated voices!
• I saw that the cache did not show the safetensors file, so not sure how if we would have a models volume mapping that is going to be working, could we then just drop the safetensors file in there?
• When trying Dia with my own voice I got an error, but clicking on the error did not do/show anything, checking the logs said "RuntimeError: The expanded size of the tensor (1360) must match the existing size (1934) at non-singleton dimension 2. Target sizes: [2, 4, 1360, 128]. Tensor sizes: [2, 4, 1934, 128]"

[rsxdalv]

The model downloads are handled uniquely for each project, it kind of has to be either it works at the end or not. I think a corrupted download that still generates but with bad output is really, really rare, although it depends on your internet.

I saw that the cache did not show the safetensors file, so not sure how if we would have a models volume mapping that is going to be working, could we then just drop the safetensors file in there?

Most models need 5-15 files so it's not quite enough. The stable-diffusion approach basically keeps a copy of the 'typical' files and only the unique weights need to be placed in a folder. But yes, for a given model if you can figure out where it's supposed to go you can just put the weights/modify the files and manage them as you see fit. Most common locations are HuggingFace cache and /data/ folders.

When trying Dia with my own voice I got an error, but clicking on the error did not do/show anything, checking the logs said "RuntimeError: The expanded size of the tensor (1360) must match the existing size (1934) at non-singleton dimension 2. Target sizes: [2, 4, 1360, 128]. Tensor sizes: [2, 4, 1934, 128]"

Might be a torch compilation issue or sample length issue. Was it your first generation after loading or second? Also for Dia here's the repo: https://github.com/rsxdalv/tts_webui_extension.dia

Thank you for the quick reply! And I fully understand that it takes time to implement the newest models!

I've added the initial version of Ace-Step 1.5 https://rsxdalv.github.io/tts-webui-extension-catalog/?browse=true&search=ace&extension=tts_webui_extension.acestep15
it has multiple issues, but it's installable. For better performance I would suggest installing a version that has nano-vllm bundled or to install nano-vllm within your docker manually - I do not have an ETA for when I'll get it working on TTS WebUI.

To install such extensions you need to use the Settings > External extensions (or Extensions > External Extensions on newest TTS WebUI) tab:

[hugalafutro]

As a new user I can only say I spent hours downloading 20Gb docker image, the webui runs and no voice generates complaining about missing extension giving me no information about what an extension is or how to install it. There's no models being downloaded in background as the readme alludes I mean they should be in the 20Gb of stuff I already dl'd no? I'm using linux+docker+rtx3060 (docker is configured to use gpu correctly, kokoro-fastapi works in docker)

Any model I click exhibits #585

[Git2tony]

And what happens when you actually have to down your container? Or want a backup? To my knowledge, you lose everything! Anyway, to each their own.

What I would recommend to add is something like this for the pip cache and the model (hugging face) cache (open to suggestions, but this is how I achieved a somewhat working system):

environment:

  # === Cache Control ===
  - HF_HOME=/app/cache
  - HUGGINGFACE_HUB_CACHE=/app/cache/hub
  # other cache
  - XDG_CACHE_HOME=/app/cache
  # disable the 'fast' but prone to 430 errors HF XET download feature
  - HF_HUB_DISABLE_XET=1
volumes:
  - ${PATH_TTS_WEBUI_CACHE}:/app/cache
  - tts_webui_pip_cache:/app/cache/pip #see also global volumes:

volumes:
#pip cache persistence
tts_webui_pip_cache:

[hugalafutro]

You're right i've not considered stack removal, I assumed the default mounts cover everything. I use docker as somewhat antithesis to what it's supposed to be, I have containers that are years old and have never been removed, basically I use docker to keep the base system clean and for the portability.

[rsxdalv]

also, does installing the extension in gradio ui make it work in the other ui? Why 2 ui anyway :D

Yes if it's visible in the new ui. The main reason was that back in the day gradio had really basic visuals AND there were like 3 models so I could spend time on each of them, now there's a dozen models and gradio isn't terrible for UX anymore.

[rsxdalv]

Yeah, I also missed any idea of what was going on in the background when using the normal UI, but the model (I tried koroko) did download in the background.

Is it possible to share the latest and best volume mappings to use with docker (compose)? I still am unable to have the installed packages (via gradio UI) be persistent between downing and upping containers.

And is there a simple tutorial how to set it up so it automatically starts (the docker container) with the correct model active and usable with OpenWebui via an API?

In the Tools > OpenAI Api have autostart enabled

[rsxdalv]

Oh I wasn't aware the other ui is "better", I used the one that was on the main screenshot of the repo. I got kokoro working as a test in the gradio ui, but I'm confused why it says

ZeroGPU
GPU is usually faster, but has a usage quota

what does that mean? in gpu info it says

VRAM: 5806.3125 MB
Used VRAM: 0.0 MB
Total Used VRAM: 904.0625 MB
Name: NVIDIA GeForce RTX 3060 Laptop GPU
CUDA Capabilities: (8, 6)
Cached VRAM: 0.0 MB
Torch Version: 2.7.0+cu128

which to me looks like it's not using gpu, but then the voices generate almost instantly which they wouldn't on cpu I think? So am I using some online gpu with quota? I'm so confused :D

Kokoro is fast so might be CPU; some of the model UIs are taken directly from, i.e., huggingface so they have wrong details and lack polish. No, none of it is running online gpu, this is local.