How to use memory saver feature?

[prashanthellina]

I used --enable-memory-saver option to enable the feature when I ran the server sglang.launch_server and then I tried to release the memory by doing

$ curl --data '' http://10.0.0.17:8000/release_memory_occupation | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    82  100    82    0     0  30460      0 --:--:-- --:--:-- --:--:-- 41000
{
  "detail": [
    {
      "type": "missing",
      "loc": [
        "body"
      ],
      "msg": "Field required",
      "input": null
    }
  ]
}

However, I get this error above. Should I put something in the POST body?

[kebe7jun]

should run curl -XPOST -H "Content-Type: application/json" -d '{}' http://10.0.0.17:8000/release_memory_occupation | jq .

[prashanthellina]

@kebe7jun Thank you! That worked.

I face a separate issue now where the output of the model is complete gibberish after I doing a release and resume.

[kebe7jun]

maybe need to create an issue for this.

[molocule]

did you end up creating an issue or fixing this issue? is there some workaround?

[prashanthellina]

@molocule I never got around to doing it. I no longer have access to the setup I used for trying out sglang.