feat(llm): enable Anthropic prompt caching and fix token accounting

repo alibaba/open-code-review · commit 08bb28c · licenseApache-2.0 · modelclaude-haiku-4-5-20251001 · viahttps://api.anthropic.com/v1/messages
Files reviewed
5
of 6 changed · 1 filtered (.md/.json)
Comments
3
2 files surfaced issues
CRITICAL / HIGH
1 / 1
would block merge in CI
Elapsed
1m21s
510,517 tokens · cache: 285,909 read / 90,644 write
review.md rendered from review.json verbatim

ocr review commit 08bb28c · 3 comments

ocr review --commit 08bb28c --format text --audience human --concurrency 2 surfaced three findings across 5 reviewed Go files in 1m21s. One CRITICAL token-accounting inconsistency, one defensive nil-check on a new prompt-caching code path, and one minor comment-style note. Every anchor below is grep-able in diff.patch.

internal/agent/agent.go L315-319 CRITICAL

CRITICAL: Token accounting logic is inconsistent with the usage resolver. In usage_resolver.go (line 78), TotalTokens is computed as prompt + completion + cacheRead + cacheWrite, indicating cache tokens are additional to the base prompt/completion tokens. However, the comment here claims "PromptTokens already includes cache read/write tokens" for Anthropic. This is contradictory. The implementation should either (1) subtract cache tokens if they're included in PromptTokens, or (2) update the comment to match the actual behavior of adding cache tokens separately. As written, the returned total will not match resp.Usage.TotalTokens from the API response.

Existing → suggested
// TotalTokensUsed returns PromptTokens + CompletionTokens across all LLM calls.
-// For Anthropic, PromptTokens already includes cache read/write tokens.
+// This excludes cache tokens which are tracked separately in TotalCacheReadTokens and TotalCacheWriteTokens.
func (a *Agent) TotalTokensUsed() int64 {
return atomic.LoadInt64(&a.totalInputTokens) + atomic.LoadInt64(&a.totalOutputTokens)
}
internal/llm/client.go L618-621 HIGH · NIL DEREF

Potential nil pointer dereference: OfTool field can be nil according to the test checks (see client_test.go lines 134-135), but this code dereferences it without validation. While the current construction guarantees it won't be nil, add a defensive nil check to prevent potential panics if the code is modified in the future or if the SDK allows nil values.

Existing → suggested
if len(tools) > 0 {
- tools[len(tools)-1].OfTool.CacheControl = anthropic.NewCacheControlEphemeralParam()
+ if tools[len(tools)-1].OfTool != nil {
+ tools[len(tools)-1].OfTool.CacheControl = anthropic.NewCacheControlEphemeralParam()
+ }
params.Tools = tools
}
internal/agent/agent.go L161-170 LOW · STYLE

The struct field comment for totalInputTokens should be more precise about atomic access. Since totalCacheReadTokens and totalCacheWriteTokens are also accessed atomically but lack the clarifying comment, add the same comment to all atomic fields for consistency.

Net: cosmetic — no behavior change suggested.


How to consume human · CI

Paste the rendered Markdown above into the PR description, or as inline review comments using each anchor. For CI gating, the structured form lives in review.json — see the Raw JSON tab on the right for the exact schema OCR ships. A minimal merge-blocker, matching this run:

jq '[.comments[]
| select(.content | test("CRITICAL|HIGH"; "i"))] | length' out/review.json
# → 2 (this run) → exit 1 in CI