Skip to content

Bug: arun_society() adds completion tokens to prompt counter — completion_token_count is always 0 #595

@marlonbarreto-git

Description

@marlonbarreto-git

Problem

In owl/utils/enhanced_role_playing.py, the async method arun_society() (line ~564) has a copy-paste bug where assistant_response completion tokens are added to overall_prompt_token_count instead of overall_completion_token_count. Additionally, user_response completion tokens are never counted at all.

Code Comparison

run_society (sync, lines ~499-505) — CORRECT:

overall_completion_token_count += assistant_response.info["usage"].get(
    "completion_tokens", 0
) + user_response.info["usage"].get("completion_tokens", 0)

overall_prompt_token_count += assistant_response.info["usage"].get(
    "prompt_tokens", 0
) + user_response.info["usage"].get("prompt_tokens", 0)

arun_society (async, lines ~564-570) — BROKEN:

overall_prompt_token_count += assistant_response.info["usage"].get(
    "completion_tokens", 0          # <-- WRONG: completion tokens go to prompt counter
)

overall_prompt_token_count += assistant_response.info["usage"].get(
    "prompt_tokens", 0
) + user_response.info["usage"].get("prompt_tokens", 0)
# user_response completion tokens are NEVER counted

Impact

  • token_info["completion_token_count"] is always 0 when using the async path
  • token_info["prompt_token_count"] is inflated by the assistant's completion tokens
  • Any cost tracking, billing dashboard, or benchmark reporting that relies on arun_society's token counts is silently wrong
  • The async path is the one used by background processing and web app integrations

Proposed Fix

# arun_society — fixed
overall_completion_token_count += assistant_response.info["usage"].get(
    "completion_tokens", 0
) + user_response.info["usage"].get("completion_tokens", 0)

overall_prompt_token_count += assistant_response.info["usage"].get(
    "prompt_tokens", 0
) + user_response.info["usage"].get("prompt_tokens", 0)

This matches the correct logic already present in run_society.

Happy to submit a PR for this fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions