llm: get costs from server
Calculating costs on the client has the advantage
that it works when not using skaband.
It requires that we maintain multiple sources of truth, though.
And it makes it very challenging to add serverside tools,
such as Anthropic's web tool.
This commit switches sketch to rely on the server for all costs.
If not using skaband, no costs will be calculated, which also
means that budget constraints won't work.
It's unfortunate, but at the moment it seems like the best path.
diff --git a/llm/gem/gemini/gemini.go b/llm/gem/gemini/gemini.go
index ab4788c..26aca0a 100644
--- a/llm/gem/gemini/gemini.go
+++ b/llm/gem/gemini/gemini.go
@@ -21,7 +21,13 @@
// https://ai.google.dev/api/generate-content#response-body
type Response struct {
- Candidates []Candidate `json:"candidates"`
+ Candidates []Candidate `json:"candidates"`
+ headers http.Header // captured HTTP response headers
+}
+
+// Header returns the HTTP response headers.
+func (r *Response) Header() http.Header {
+ return r.headers
}
type Candidate struct {
@@ -162,6 +168,7 @@
if err := json.Unmarshal(body, &res); err != nil {
return nil, fmt.Errorf("GenerateContent: unmarshaling response: %w, %s", err, string(body))
}
+ res.headers = httpResp.Header
return &res, nil
}