Skip to content

Can the stop tokens be retained?  #85

@CSWellesSun

Description

@CSWellesSun

I set the stop token as ., but the output text will not include the stop token, so can the stop tokens be retained in the output?

request:

curl http://localhost:8081/generate \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Once upon a time,",
    "sampling_params": {
      "max_new_tokens": 1024,
      "temperature": 0,
      "stop": ".",
    }
  }'

response:

{"text":" and in a land far, far away, there was a young girl named Lily","meta_info":{"prompt_tokens":6,"completion_tokens":18,"id":"fe8e78ca46ab4f4bb69842202201288c"}}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions