llm: get costs from server
Calculating costs on the client has the advantage
that it works when not using skaband.
It requires that we maintain multiple sources of truth, though.
And it makes it very challenging to add serverside tools,
such as Anthropic's web tool.
This commit switches sketch to rely on the server for all costs.
If not using skaband, no costs will be calculated, which also
means that budget constraints won't work.
It's unfortunate, but at the moment it seems like the best path.
diff --git a/llm/llm.go b/llm/llm.go
index 2aea24e..9718954 100644
--- a/llm/llm.go
+++ b/llm/llm.go
@@ -6,6 +6,8 @@
"encoding/json"
"fmt"
"log/slog"
+ "net/http"
+ "strconv"
"strings"
"time"
)
@@ -194,6 +196,19 @@
}
}
+func CostUSDFromResponse(headers http.Header) float64 {
+ h := headers.Get("Skaband-Cost-Microcents")
+ if h == "" {
+ return 0
+ }
+ uc, err := strconv.ParseUint(h, 10, 64)
+ if err != nil {
+ slog.Warn("failed to parse cost header", "header", h)
+ return 0
+ }
+ return float64(uc) / 100_000_000
+}
+
// Usage represents the billing and rate-limit usage.
// Most LLM structs do not have JSON tags, to avoid accidental direct use in specific providers.
// However, the front-end uses this struct, and it relies on its JSON serialization.