Tag query fix

[harish876]

Relevant issues

Improves DailyTagSpend write fanout/QPS imbalance by decoupling tag spend flushes from the main spend scheduler and running tag commits at a longer interval.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link: TBD

• CI run for the last commit
  Link: TBD

• Merge / cherry-pick CI run
  Links: TBD

Type

🐛 Bug Fix
🧹 Refactoring
🚄 Infrastructure
✅ Test

Changes

• Split DailyTagSpend flushes into a separate scheduler job instead of running through the main update_spend flow.
• Configured tag spend job interval to run at a longer cadence using DAILY_TAG_SPEND_BATCH_MULTIPLIER, reducing tag-write burst pressure.
• Kept request-path enqueue behavior unchanged, so tag spend data still accumulates per request and is eventually flushed.
• Added and updated tests for:
  • update_daily_tag_spend delegation behavior
  • non-raising/logging behavior on tag flush errors
  • retry-path behavior for tag spend DB writes in the DB writer layer
• Observed improvement target: DailyTagSpend call rate now tracks closer to DailyTeamSpend, improving QPS parity and reducing tag table overrepresentation.
• Docker dev Build for the latest version was failing due to a broken dependency to g++.

The fix reduces the QPS. The below numbers show the most run queries tracked by postgres in a duration of 4-5 minutes under a simulated load test. The validation here is the ratio of database calls between the DailyTagSpend and DailyTeamSpend must nearly 1, which means the same number of QPS is achieved by queries to these tables.

DailyTagSpend Query Performance Analysis

Before Fix

Configuration	avg_qps_since_reset
Without Redis Transaction Buffer	0.31182049273784314215
With Redis Transaction Buffer	0.32670871516733775359

With Adjusted Batch Interval Fix

Configuration	avg_qps_since_reset
Without Redis Transaction Buffer	0.16589514881214770545
With Redis Transaction Buffer	0.14892439673581266593

Summary

• Before Fix: Redis buffer showed ~4.7% increase in QPS (0.327 vs 0.312)
• After Batch Interval Fix: Redis buffer achieved ~10.3% reduction in QPS (0.149 vs 0.166)

---

Before Fix

Full Logs:

litellm=# select
  calls,
  left(query, 50) as query_prefix
from pg_stat_statements
where query ilike 'insert%'
order by calls desc
limit 5;
 calls |                    query_prefix                    
-------+----------------------------------------------------
    52 | INSERT INTO "public"."LiteLLM_DailyTagSpend" ("id"
    25 | INSERT INTO "public"."LiteLLM_DailyTeamSpend" ("id
    25 | INSERT INTO "public"."LiteLLM_DailyUserSpend" ("id
     1 | INSERT INTO "public"."LiteLLM_SpendLogs" ("model_i
     1 | INSERT INTO "public"."LiteLLM_SpendLogs" ("call_ty
(5 rows)


litellm=# SELECT
    left(query, 25) AS query_prefix,
    calls / NULLIF(
        EXTRACT(EPOCH FROM (now() - (
            SELECT stats_reset
            FROM pg_stat_statements_info
        ))),
        0
    ) AS avg_qps_since_reset
FROM pg_stat_statements
WHERE query LIKE 'INSERT INTO "public"."LiteLLM_DailyTagSpend" ("id"%'
ORDER BY calls DESC
LIMIT 10;
       query_prefix        |  avg_qps_since_reset   
---------------------------+------------------------
 INSERT INTO "public"."Lit | 0.31182049273784314215
(1 row)

Using Redis Transaction Buffer

litellm=# select
  calls,                            
  left(query, 50) as query_prefix
from pg_stat_statements
where query ilike 'insert%'
order by calls desc
limit 15;
 calls |                    query_prefix                    
-------+----------------------------------------------------
    50 | INSERT INTO "public"."LiteLLM_DailyTagSpend" ("id"
    25 | INSERT INTO "public"."LiteLLM_DailyUserSpend" ("id
    25 | INSERT INTO "public"."LiteLLM_DailyTeamSpend" ("id
(15 rows)

litellm=# SELECT
    left(query, 25) AS query_prefix,
    calls / NULLIF(
        EXTRACT(EPOCH FROM (now() - (
            SELECT stats_reset
            FROM pg_stat_statements_info
        ))),
        0
    ) AS avg_qps_since_reset
FROM pg_stat_statements
WHERE query LIKE 'INSERT INTO "public"."LiteLLM_DailyTagSpend" ("id"%'
ORDER BY calls DESC
LIMIT 10;
       query_prefix        |  avg_qps_since_reset   
---------------------------+------------------------
 INSERT INTO "public"."Lit | 0.32670871516733775359
(1 row)

---

With Adjusted Batch Interval Fix

Full Logs

litellm=# select
  calls,
  left(query, 50) as query_prefix
from pg_stat_statements
where query ilike 'insert%'
order by calls desc
limit 5;
 calls |                    query_prefix                    
-------+----------------------------------------------------
    53 | INSERT INTO "public"."LiteLLM_DailyUserSpend" ("id
    53 | INSERT INTO "public"."LiteLLM_DailyTeamSpend" ("id
    52 | INSERT INTO "public"."LiteLLM_DailyTagSpend" ("id"
     1 | INSERT INTO "public"."LiteLLM_SpendLogs" ("respons
     1 | INSERT INTO "public"."LiteLLM_SpendLogs" ("mcp_nam
(5 rows)

litellm=# SELECT
    left(query, 25) AS query_prefix,
    calls / NULLIF(
        EXTRACT(EPOCH FROM (now() - (
            SELECT stats_reset
            FROM pg_stat_statements_info
        ))),
        0
    ) AS avg_qps_since_reset
FROM pg_stat_statements
WHERE query LIKE 'INSERT INTO "public"."LiteLLM_DailyTagSpend" ("id"%'
ORDER BY calls DESC
LIMIT 10;
       query_prefix        |  avg_qps_since_reset   
---------------------------+------------------------
 INSERT INTO "public"."Lit | 0.16589514881214770545
(1 row)

Using Redis Transaction Buffer

litellm=# select
  calls,                            
  left(query, 50) as query_prefix
from pg_stat_statements
where query ilike 'insert%'
order by calls desc
limit 15;
 calls |                    query_prefix                    
-------+----------------------------------------------------
    50 | INSERT INTO "public"."LiteLLM_DailyTeamSpend" ("id
    50 | INSERT INTO "public"."LiteLLM_DailyUserSpend" ("id
    50 | INSERT INTO "public"."LiteLLM_DailyTagSpend" ("id"
     1 | INSERT INTO "public"."LiteLLM_SpendLogs" ("end_use

litellm=# SELECT
    left(query, 25) AS query_prefix,
    calls / NULLIF(
        EXTRACT(EPOCH FROM (now() - (
            SELECT stats_reset
            FROM pg_stat_statements_info
        ))),
        0
    ) AS avg_qps_since_reset
FROM pg_stat_statements
WHERE query LIKE 'INSERT INTO "public"."LiteLLM_DailyTagSpend" ("id"%'
ORDER BY calls DESC
LIMIT 10;
       query_prefix        |  avg_qps_since_reset   
---------------------------+------------------------
 INSERT INTO "public"."Lit | 0.14892439673581266593
(1 row)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 4, 2026 6:41am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing harish876:tag-query-fix (e9336a4) with main (a5322c6)}

[greptile-apps]

Greptile Summary

This PR decouples DailyTagSpend flushes from the main spend scheduler by introducing a dedicated APScheduler job (update_daily_tag_spend) that runs at a 2.3× longer interval (DAILY_TAG_SPEND_BATCH_MULTIPLIER). The change removes daily_tag_spend_update_queue from the main store_in_memory_spend_updates_in_redis pipeline and adds two new instance methods — _commit_daily_tag_spend_to_db and _commit_daily_tag_spend_to_db_with_redis — to DBSpendUpdateWriter that are called exclusively by the new scheduler job. A Docker dev build fix (g++ dependency) is included as a minor side-fix.

Key changes:

• db_spend_update_writer.py: Tag spend removed from _commit_spend_updates_to_db_without_redis_buffer; new _commit_daily_tag_spend_to_db[_with_redis] methods added, both correctly guarded and following the existing lock-then-drain pattern.
• redis_update_buffer.py: Tag queue parameter removed from store_in_memory_spend_updates_in_redis; new store_in_memory_daily_tag_spend_updates_in_redis and get_all_daily_tag_spend_update_transactions_from_redis_buffer methods added for the Redis-buffered path.
• proxy_server.py: New scheduler job registered at int(batch_writing_interval * DAILY_TAG_SPEND_BATCH_MULTIPLIER) seconds.
• utils.py: New update_daily_tag_spend coroutine as the scheduler entry point; main update_spend docstring updated to note the tag spend omission.
• Tests added in test_update_daily_tag_spend.py (delegation, error suppression, Redis-path branching, and retry logic) and test_redis_update_buffer.py (pipeline push/drain, early-return, no-Redis sentinel).
• Missing test coverage: store_in_memory_daily_tag_spend_updates_in_redis and get_all_daily_tag_spend_update_transactions_from_redis_buffer — the two new Redis-path tag spend methods in redis_update_buffer.py — have no corresponding tests despite being on a critical data path when use_redis_transaction_buffer=true.

Confidence Score: 4/5

PR is safe to merge; architecture is sound and the performance improvement is well-evidenced, but quality gaps from the previous review cycle remain open.

The core logic — decoupling tag spend into a separate scheduler job with its own Redis lock — is correct and consistent with existing patterns. The non-Redis and Redis paths both follow established idioms. All new code paths have at least some test coverage. The score is held at 4 rather than 5 because (1) the two new Redis-specific tag-spend methods in redis_update_buffer.py have zero test coverage (new finding, distinct from the previously flagged test gap), and (2) the DAILY_TAG_SPEND_BATCH_MULTIPLIER constant remains a plain float literal with no env-var override — previously flagged and still unaddressed. Both are P2 quality issues that don't block correctness but are meaningful enough to warrant a maintainer pass before merge.

tests/test_litellm/proxy/db/db_transaction_queue/test_redis_update_buffer.py — needs tests for store_in_memory_daily_tag_spend_updates_in_redis and get_all_daily_tag_spend_update_transactions_from_redis_buffer

Important Files Changed

Filename	Overview
litellm/proxy/db/db_spend_update_writer.py	Tag spend removed from main spend handler; two new instance methods (_commit_daily_tag_spend_to_db and _commit_daily_tag_spend_to_db_with_redis) correctly implement the decoupled flush path, following the existing lock-then-drain Redis pattern.
litellm/proxy/db/db_transaction_queue/redis_update_buffer.py	Tag queue removed from main pipeline method; two new Redis methods added for the separate tag spend path — no tests cover these new methods.
litellm/proxy/proxy_server.py	New scheduler job registered for tag spend at 2.3× main interval; inline import used to break circular dependency (consistent with existing pattern in same function).
litellm/proxy/utils.py	New update_daily_tag_spend scheduler entry point correctly delegates to the Redis or non-Redis path and suppresses exceptions with error logging.
litellm/constants.py	New DAILY_TAG_SPEND_BATCH_MULTIPLIER=2.3 constant added; DB_DAILY_TAG_SPEND_UPDATE_JOB_NAME added for lock identity — multiplier is a plain float literal with no env-var override.
tests/proxy_unit_tests/test_update_daily_tag_spend.py	Four unit tests added: delegation to correct commit method, error suppression, Redis-path branching, and retry-then-succeed behavior — all use mocks, no real network calls.
tests/test_litellm/proxy/db/db_transaction_queue/test_redis_update_buffer.py	Pipeline push/drain tests updated to reflect removal of tag queue from main method; new Redis pipeline tests added, but store_in_memory_daily_tag_spend_updates_in_redis and get_all_daily_tag_spend_update_transactions_from_redis_buffer lack coverage.
docker/Dockerfile.dev	Adds g++ to build-stage apt-get install to fix dev image build failure.

Sequence Diagram

sequenceDiagram
    participant Req as Incoming Request
    participant Queue as daily_tag_spend_update_queue
    participant MainSched as Main Scheduler<br/>(every N seconds)
    participant TagSched as Tag Scheduler<br/>(every 2.3xN seconds)
    participant Redis as Redis Buffer
    participant DB as PostgreSQL LiteLLM_DailyTagSpend

    Req->>Queue: enqueue tag spend transaction

    Note over MainSched: update_spend() runs
    MainSched->>MainSched: _commit_spend_updates_to_db (skips daily_tag_spend_update_queue)

    Note over TagSched: update_daily_tag_spend() runs
    alt use_redis_transaction_buffer=false
        TagSched->>Queue: flush_and_get_aggregated_daily_spend_update_transactions()
        Queue-->>TagSched: aggregated transactions
        TagSched->>DB: update_daily_tag_spend() with retries
    else use_redis_transaction_buffer=true
        TagSched->>Queue: flush_and_get_aggregated_daily_spend_update_transactions()
        Queue-->>TagSched: aggregated transactions
        TagSched->>Redis: store_in_memory_daily_tag_spend_updates_in_redis()
        TagSched->>TagSched: acquire_lock(DB_DAILY_TAG_SPEND_UPDATE_JOB_NAME)
        alt lock acquired
            TagSched->>Redis: get_all_daily_tag_spend_update_transactions_from_redis_buffer()
            Redis-->>TagSched: aggregated transactions
            TagSched->>DB: update_daily_tag_spend() with retries
        end
    end

Loading

_{Reviews (3): Last reviewed commit: "resolving circular import error flagged ..." | Re-trigger Greptile}

[greptile-apps]

Hardcoded magic number, not env-var configurable — inconsistent with codebase pattern

Every other tuneable constant in this file is read from an environment variable (e.g. DEFAULT_FLUSH_INTERVAL_SECONDS, DEFAULT_BATCH_SIZE), allowing operators to adjust behaviour without a code change. DAILY_TAG_SPEND_BATCH_MULTIPLIER = 2.3 is a plain float literal with no override mechanism.

There is also a typo in the comment: "spcific" → "specific".

Suggested change

	# The number of tag entries are higher than number of user, team entries. This leads to a higher QPS.
	# This will run tag spcific tasks at a later time to smooth QPS
	DAILY_TAG_SPEND_BATCH_MULTIPLIER = 2.3
	# The number of tag entries are higher than number of user, team entries. This leads to a higher QPS.
	# This will run tag-specific tasks at a longer cadence to smooth QPS.
	DAILY_TAG_SPEND_BATCH_MULTIPLIER = float(
	os.getenv("DAILY_TAG_SPEND_BATCH_MULTIPLIER", 2.3)
	)

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

[greptile-apps]

Test exercises update_daily_tag_spend static method directly, but the new scheduler path goes through _commit_daily_tag_spend_to_db

test_daily_tag_spend_retries_then_succeeds calls DBSpendUpdateWriter.update_daily_tag_spend(...) directly with a hand-crafted daily_spend_transactions dict. This tests the underlying write/retry behaviour well, but there is no test that exercises the actual new code path triggered by the scheduler:

update_daily_tag_spend (utils.py)
  └─ _commit_daily_tag_spend_to_db
       └─ daily_tag_spend_update_queue.flush_and_get_aggregated_daily_spend_update_transactions()
       └─ DBSpendUpdateWriter.update_daily_tag_spend(...)

In particular, the flush_and_get_aggregated_daily_spend_update_transactions call is never exercised, so a bug where the queue is empty (e.g. because the main job already drained it to Redis) would never be caught by these tests. Consider adding a test that pre-populates proxy_logging_obj.db_spend_update_writer.daily_tag_spend_update_queue and verifies that the full chain from update_daily_tag_spend (in utils.py) actually writes to the DB.

[ishaan-berri]

lgtm