1202b5a2da
Conversation monitors now allow **10,000,000 calls per UTC month** and
**500,000 calls per UTC day**, per account. Allowances count attempted
provider requests, including preview calls, and no longer depend on
estimated token lengths. Pending evaluations retain their quota holds
when new messages arrive and resume after the applicable reset without
retrying ahead of a provider cooldown.
## What changed
- Enforce both call limits using persisted daily usage under the account
lock, so concurrent workers cannot exceed the remaining allowance. Each
attempted provider batch consumes one call, even when it evaluates
multiple monitors or returns a provider error.
- Remove the daily token reservation and reconciliation, which mixed
JSON request bytes with provider input tokens. Keep request-size limits
and provider token/cost telemetry.
- Preserve daily and monthly quota holds across new conversation input.
Spread retries over up to four hours after the later of the quota reset
and the longest provider cooldown recorded across failed batches.
- Report when the current monthly allowance was reached, ignoring
timestamps from the previous 100,000-call allowance.
- Document the RubyLLM bundle-audit exceptions for the pinned Ruby 3.4.4
runtime, with reassessment required when the runtime or supported
dependency versions change.
## Quota logging
When a committed call reaches the daily or monthly account cap, emit one
structured `conversation_monitor_limit_reached` log event for that
period. It includes `account_id`, `period` (`daily` or `monthly`),
`used`, `limit`, and the UTC reset timestamp `resets_at`. Rolled-back
calls and subsequent blocked retries do not emit events. Reaching both
caps in the same call emits one event for each period.
The event uses the existing Rails log forwarding path. In New Relic,
group events by account and period with:
```sql
FROM Log
WITH aparse(message, '%"account_id":*,%') AS account_id,
aparse(message, '%"period":"*"%') AS period
SELECT count(*)
WHERE message LIKE '%conversation_monitor_limit_reached%'
FACET account_id, period
SINCE 7 days ago
LIMIT MAX
```
## Closes
Fixes https://linear.app/chatwoot/issue/INF-117
## How to reproduce
1. Exhaust an account's daily allowance, then receive additional
conversation messages before the next UTC midnight. Previously, each
message could pull the held work into an early retry and accumulate
another job at the reset. The work now retains its saved retry time
while recording the new input.
2. Have an earlier provider batch return a long `Retry-After`, then
exhaust the daily or monthly allowance before the next batch. The saved
retry now honors both the allowance reset and the longer provider
cooldown before adding jitter.
## Rollout
Existing quota holds retain their saved retry times; the increased
allowances apply when work next reserves a call. Already-scheduled jobs
from before deployment will still run once.
---------
Co-authored-by: Sony Mathew <2040199+sony-mathew@users.noreply.github.com>
1.9 KiB
1.9 KiB