)]}' { "log": [ { "commit": "cfd0fe64e379f066b117effe84100a38c48e493f", "tree": "fed8d755bb3a8fffd61f6cd5da706c9267a7c5f3", "parents": [ "c540e8ea6a8dd2a5297345b6f9b885988338f237" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat Jun 21 02:17:41 2025 +0000" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Fri Jun 20 19:21:43 2025 -0700" }, "message": "cleanup: remove unused functions and fix tests\n\nRemove unused Go code identified through systematic analysis:\n\nRemoved Functions:\n- BashRun() in claudetool/bash.go - legacy testing function\n- ContentToString() in claudetool/util.go - unused utility function\n- GetActiveTunnels() in dockerimg/tunnel_manager.go - unused getter method\n\nRemoved Types:\n- baseResponse struct in claudetool/browse/browse.go - unused response type\n\nTest Updates:\n- Replaced BashRun tests with direct Bash.Run calls in bash_test.go\n- Removed ContentToString test from agent_test.go (testing unused function)\n- Updated tunnel manager tests to access internal activeTunnels map directly\n- Fixed all compilation errors caused by removed functions\n\nAll tests now pass and the codebase is cleaner with reduced maintenance burden.\nThe removed functions had no references in production code and were only\nused in tests, which have been updated or removed as appropriate.\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: s2cac4b024f877682k\n" }, { "commit": "fe51d1d1b1f205ce2a417d312991e71d31ce6dc3", "tree": "e45895b2546f9b0a54d6bbd7a635e47f821f36bb", "parents": [ "64f60461b00c474b0b4747a06177d5c7a357bba9" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Mon Jun 16 21:19:44 2025 -0700" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Mon Jun 16 21:22:20 2025 -0700" }, "message": "Further loosen browser timeouts.\n" }, { "commit": "29c481c8d3bbe6124d3b273d2a154081d8472f08", "tree": "616a5f8097fa56da5e315ef2e2948fc049851a01", "parents": [ "c0a4459e3f5c8e68649bb45f2c309f6f01928ab7" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sun Jun 15 21:25:45 2025 -0700" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sun Jun 15 21:25:45 2025 -0700" }, "message": "browser tests: further increasing timeouts\n" }, { "commit": "a35de5f73d31756c8ac972a54bdb370bb26eed5a", "tree": "e86111f53c0e701a38510ed8543403f25665a891", "parents": [ "46be0967c78b0e60b18b4b9ff3c642ebc96c7e4d" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat Jun 14 12:00:48 2025 -0700" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat Jun 14 12:00:48 2025 -0700" }, "message": "Using GH actions provided Chrome for test speedup (#149)\n\nOMG. This had wild behavior. Sometimes you\u0027d log into tmate, and it\nwould work. Something about running Chrome manually would tickle some\ndbus thing into working. The error would sometimes show up, but most\noften it would be swallowed.\n\nThe punchline? We had a context timeout of 5 or 10 seconds in the tests\nthat would trigger silently, and we\u0027d see nothing. Everything else\n(including maybe the dbus stuff) was a red herring: the test executor\nmachine is swamped, and shit\u0027s slow, and the various timeouts needed\ntweaking. I bet there are a few more timeouts to loosen eventually, but\nit seems to be passing now.\n\nError message:\n\tfailed to start browser (please apt get chromium or equivalent): chrome failed to start:\n" }, { "commit": "c013134814facb7da15b59c79399ec1939eb6ef9", "tree": "6072025f9c2e5a87af79a61070fab1f0500c32a5", "parents": [ "c886ac5478194e95f93b166572a7a081eb6cadb8" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Fri Jun 13 21:07:08 2025 -0700" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Fri Jun 13 21:07:08 2025 -0700" }, "message": "Fail to speed up CI by not installing Chrome.\n\nI have no idea why the same \"vintage\" machine I logged into via tmate\nhas this working, but this doesn\u0027t, but the --no-sandbox thing is\ncorrect regardless.\n\nVariants of the following is what SHOULD work but doesn\u0027t yet.\n\n\tPATH\u003d/usr/local/share/chromium/chrome-linux:$PATH GOEXPERIMENT\u003dsynctest stdbuf -oL -eL go test -v -count\u003d1 -run TestBrowserInitialization ./claudetool/browse/...\n" }, { "commit": "542bda3968c6dd5b79392dd63e2955e04520401a", "tree": "ea1a0743849495ca2489c6363d2dc689dd0a56a7", "parents": [ "225e9668aeebc0cae667872dd45222d69ac3cbd8" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Wed Jun 11 18:31:03 2025 -0700" }, "committer": { "name": "Autoformatter", "email": "bot@sketch.dev", "time": "Thu Jun 12 01:31:34 2025 +0000" }, "message": "browser: rename browser_read_image to read_image and auto-send screenshots to LLM\n\nRename browser_read_image tool to read_image and modify browser_take_screenshot\nto automatically send image content to the LLM instead of requiring a separate\nread_image tool call, streamlining the screenshot workflow.\n\nProblem Analysis:\nThe current browser screenshot workflow required two separate tool calls:\n1. browser_take_screenshot - saves screenshot and returns file path\n2. browser_read_image - reads saved screenshot and sends to LLM\n\nThis two-step process was inefficient and created unnecessary round trips.\nAdditionally, browser_read_image was specific to browser automation but\nthe functionality of reading and encoding images is more general purpose.\n\nImplementation Changes:\n\n1. Screenshot Tool Behavior (claudetool/browse/browse.go):\n - Modified browser_take_screenshot to automatically return image content\n - Removed screenshotOutput struct as ID-only response no longer needed\n - Added base64 encoding of screenshot data directly in screenshotRun\n - Returns []llm.Content with both text description and image data\n - Still saves screenshot file for potential future reference\n - Uses same image encoding format as existing read_image tool\n\n2. Tool Rename (claudetool/browse/browse.go):\n - Renamed browser_read_image tool to read_image\n - Updated tool name in NewReadImageTool from \u0027browser_read_image\u0027 to \u0027read_image\u0027\n - Maintained all existing functionality and input/output format\n - Tool description and schema remain unchanged\n\n3. UI Updates (termui/termui.go):\n - Updated template condition from \u0027browser_read_image\u0027 to \u0027read_image\u0027\n - Maintains existing emoji and display format for read_image tool calls\n\n4. WebUI Updates (webui/src/web-components/):\n - Updated sketch-tool-calls.ts to reference \u0027read_image\u0027 instead of \u0027browser_read_image\u0027\n - Renamed sketch-tool-card-browser-read-image.ts to sketch-tool-card-read-image.ts\n - Updated component class name from SketchToolCardBrowserReadImage to SketchToolCardReadImage\n - Updated custom element name from \u0027sketch-tool-card-browser-read-image\u0027 to \u0027sketch-tool-card-read-image\u0027\n - Updated import statement to reference new component file name\n - Removed old component file and updated TypeScript declarations\n\n5. Test Updates (claudetool/browse/browse_test.go):\n - Modified TestGetTools to allow read_image tool without \u0027browser_\u0027 prefix\n - Added special case handling for read_image in tool naming convention check\n - All existing tests continue to pass with updated tool name\n\nTechnical Details:\n- Screenshot auto-send uses same base64 encoding as existing read_image tool\n- Content structure matches browser_read_image output format for consistency\n- File saving still occurs for potential debugging or future reference\n- Error handling preserves existing behavior with proper fallbacks\n- Tool count remains the same (12 tools with screenshots, 10 without)\n\nBenefits:\n- Eliminates need for two-step screenshot workflow\n- Reduces round trips and simplifies user experience\n- More intuitive tool naming (read_image is general purpose)\n- Maintains full backward compatibility for read_image functionality\n- Consistent image encoding across all browser tools\n- Automatic screenshot viewing improves debugging and validation workflows\n\nTesting:\n- All existing browser tool tests pass with updated expectations\n- TestReadImageTool verifies renamed tool functionality\n- Tool naming convention test updated to handle read_image exception\n- TypeScript compilation successful with no type errors\n- Web component functionality preserved across rename\n\nThis enhancement streamlines screenshot workflows while maintaining the\ngeneral-purpose read_image tool for reading arbitrary image files, creating\na more efficient and intuitive browser automation experience.\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: se3e81f997f30f01ek\n" }, { "commit": "bf381a7b6c6df65e7b74796181ab0489566e3795", "tree": "070e419f99682b82892ec85594902d07222dd21a", "parents": [ "b421a24f862d91101b9e5b4738f93693e18672eb" ], "author": { "name": "Josh Bleecher Snyder", "email": "josharian@gmail.com", "time": "Thu May 29 23:45:02 2025 +0000" }, "committer": { "name": "Josh Bleecher Snyder", "email": "josharian@gmail.com", "time": "Fri May 30 08:58:10 2025 -0700" }, "message": "claudetool/browse: block port 80 access in browser tools\n\nPrevents agents from attempting to browse to port 80, which is reserved\nfor the Sketch server itself. This is a common source of confusion.\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: sf481df655a16903ek" }, { "commit": "7fbc8e43ff52b6f5dc69fdd26ad7388bdac70435", "tree": "abc653d8bf0563f00d232ad91edbed38eb5d4abd", "parents": [ "57893c274ca6d7936580b8bd97af9e5a4b4dcd51" ], "author": { "name": "Josh Bleecher Snyder", "email": "josharian@gmail.com", "time": "Thu May 29 19:42:25 2025 +0000" }, "committer": { "name": "Josh Bleecher Snyder", "email": "josharian@gmail.com", "time": "Thu May 29 12:50:41 2025 -0700" }, "message": "browse: set default viewport to 1280x720 (16:9 widescreen)\n\nImprove browser screenshot quality by setting a modern default viewport\nsize during browser initialization. This replaces the previous system-\ndependent default with a consistent 1280x720 resolution that provides:\n\n- Better readability and content coverage in screenshots\n- Modern 16:9 aspect ratio matching typical displays\n- Consistent testing environment across all users\n\nImplementation:\n- Add EmulateViewport call during browser startup in browse.go\n- Set viewport immediately after browser initialization\n- Add comprehensive test (TestDefaultViewportSize) to verify functionality\n\nThe test confirms the viewport is correctly set to 1280x720 on browser\nstartup and validates the JavaScript window dimensions match expectations.\n\nThis addresses screenshot quality issues with small default viewport sizes\nand provides a better default experience for browser automation.\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: s98815cfd1eaca88dk\n" }, { "commit": "9500617d20b037b9ad1f88eb7550068679832a4d", "tree": "15593d668443d3df4f6d223bd753c216f5f0274a", "parents": [ "e89b3080f934a4bc70a0cfa85ffff49ac78d6f2b" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Wed May 28 20:05:46 2025 -0700" }, "committer": { "name": "Philip Zeyliger", "email": "philip.zeyliger@gmail.com", "time": "Wed May 28 20:26:29 2025 -0700" }, "message": "dockerimg: fix Chromium support with multi-stage Docker build\n\nOMG, people, OMG. So, an earlier commit moved us to Ubuntu, and it turns\nout that \"apt-get install chromium-browser\" or whatever just does\nnothing, and tells you to use the snap. Snap requires systemd, and if\nyou\u0027re using containers, you don\u0027t usually have systemd, and ... yeah,\nno screenshots. There are no great stories for where to get Chromium.\nThere\u0027s a dude who publishes the Mint Linux packages in a compatible way\nfor Ubuntu. I chose instead the headless-chrome from a Docker build\nrecommended by the Chromedp library that we use to control Chromium.\n\nI\u0027m a bit snappy about all of this.\n\n...\n\nReplace Ubuntu 24 snap-based Chromium installation with chromedp/headless-shell\nto resolve container compatibility issues where snaps don\u0027t work properly.\n\nChanges include:\n\n1. Multi-stage Dockerfile.base build:\n - Stage 1: Extract headless-shell from docker.io/chromedp/headless-shell:stable\n - Stage 2: Main Ubuntu 24.04 application image with required Chrome dependencies\n - Remove chromium package from apt-get install (replaced with headless-shell)\n - Add required libraries: libglib2.0-0, libnss3, libx11-6, libxcomposite1,\n libxdamage1, libxext6, libxi6, libxrandr2, libgbm1, libgtk-3-0\n - Add headless-shell to PATH so chromedp can find it automatically\n\n2. Updated documentation in browse/README.md:\n - Document Docker multi-stage build approach\n - Clarify requirements for Docker vs local development\n\nBenefits:\n- Resolves Ubuntu 24 snap incompatibility issues in containers\n- Provides self-contained Chrome installation without system dependencies\n- Maintains backward compatibility for local development\n- Uses proven chromedp/headless-shell for reliable browser automation\n- Eliminates need for manual Chrome/Chromium installation in containers\n- No code changes needed in browse.go - chromedp finds headless-shell via PATH\n\nThe headless-shell binary is automatically discovered by chromedp\u0027s default\nexecutable search since it\u0027s added to PATH in the Docker environment.\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: se4808dca7afba802k\n" }, { "commit": "6e35332ceac45166d6dd549d1e3d94937f683320", "tree": "87b4eeea74b9f6d9786d99983a976c1cf0f7ff05", "parents": [ "8d8b7ac6b00b1d0b3a2c6d3a4079e2c2fb76cc8a" ], "author": { "name": "Josh Bleecher Snyder", "email": "josharian@gmail.com", "time": "Wed May 21 18:21:24 2025 +0000" }, "committer": { "name": "Josh Bleecher Snyder", "email": "josharian@gmail.com", "time": "Wed May 21 11:36:32 2025 -0700" }, "message": "all: remove testify\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: s51244611964f2413k" }, { "commit": "74d690e7b4794f2c164a9da30dd106a002b935d9", "tree": "412f1ad777e599ed61904c7f80c53859315babab", "parents": [ "039fc34bd9c8f7cf3e9f3c9aeee2e9677ce28e00" ], "author": { "name": "Josh Bleecher Snyder", "email": "josharian@gmail.com", "time": "Wed May 14 18:16:03 2025 -0700" }, "committer": { "name": "Josh Bleecher Snyder", "email": "josharian@gmail.com", "time": "Thu May 15 12:40:29 2025 -0700" }, "message": "claudetool: remove knowledge base, focus on about sketch\n\nWe should have a more general kb.\nMeanwhile, this is important and standalone.\nMake it all clearer and sharper.\n" }, { "commit": "18e336804bb6f052b29d891f7db54bd45e1de503", "tree": "48a979aa7f5e89975cf8f579722ff9f363768985", "parents": [ "34bb09adf0cfc2f7fa4a2615e72d634aa88a6016" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Tue May 13 16:34:21 2025 -0700" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Tue May 13 16:34:21 2025 -0700" }, "message": "browser: Add console log capture tools\n\nThis was entirely Sketch-coded.\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: sc6bbdde6b1a71d8fk\n" }, { "commit": "05224846741b500d8103bd6930a7717e1211ef01", "tree": "54eaad583c19b5c00719384a268d8dccab0dd13e", "parents": [ "80b488d853e2766b638e65cfe44a5904d7cb24ee" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat May 10 18:26:08 2025 -0700" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat May 10 18:26:08 2025 -0700" }, "message": "browser: add window resize tool with chromedp\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: s68bdc7fb84309a7ck\n" }, { "commit": "80b488d853e2766b638e65cfe44a5904d7cb24ee", "tree": "abd280ec3924efb4dbbd72622129bc4af56e28ca", "parents": [ "6458e7c751856a9f7f4004367cb6274e564a1489" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat May 10 18:21:54 2025 -0700" }, "committer": { "name": "Autoformatter", "email": "bot@sketch.dev", "time": "Sun May 11 01:22:37 2025 +0000" }, "message": "Browser tools: initialize lazily and add timeouts.\n\nAlso rename browser_screenshot to browser_take_screenshot for clarity\\n- Update both Go and UI code to maintain consistency\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: s8a5cabff914f88dfk\n" }, { "commit": "72252cbcb97840d724133be67c4c69cc69ebb2d3", "tree": "a361499dc3fa6b9af2be3e74cfd59fd8ba34690e", "parents": [ "7ce5fb76d8748ebf73c5adf9d6cd8eb67716fba8" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat May 10 17:00:08 2025 -0700" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat May 10 17:00:08 2025 -0700" }, "message": "llm and everything: Update ToolResult to use []Content instead of string for multimodal support\n\nThis was a journey. The sketch-generated summary below is acceptable,\nbut I want to tell you about it in my voice too. The goal was to send\nscreenshots to Claude, so that it could... look at them. Currently\nthe take screenshot and read screenshot tools are different, and they\u0027ll\nneed to be renamed/prompt-engineered a bit, but that\u0027s all fine.\n\nThe miserable part was that we had to change the return value\nof tool from string to Content[], and this crosses several layers:\n - llm.Tool\n - llm.Content\n - ant.Content \u0026 openai and gemini friends\n - AgentMessage [we left this alone]\n\nExtra fun is that Claude\u0027s API for sending images has nested Content\nfields, and empty string and missing needs to be distinguished for the\nText field (because lots of shell commands return the empty string!).\n\nFor the UI, I made us transform the results into a string, dropping\nimages. This would have been yet more churn for not much obvious\nbenefit. Plus, it was going to break skaband\u0027s compatibility, and ...\nyet more work.\n\nOpenAI and Gemini don\u0027t obviously support images in this same way,\nso they just don\u0027t get the tools.\n\n~~~~~~~~~~ Sketch said:\n\nThis architectural change transforms tool results from plain strings to []Content arrays, enabling multimodal interaction in the system. Key changes include:\n\n- Core structural changes:\n - Modified ToolResult type from string to []Content across all packages\n - Added MediaType field to Content struct for MIME type support\n - Created TextContent and ImageContent helper functions\n - Updated all tool.Run implementations to return []Content\n\n- Image handling:\n - Implemented base64 image support in Anthropic adapter\n - Added proper media type detection and content formatting\n - Created browser_read_image tool for displaying screenshots\n - Updated browser_screenshot to provide usable image paths\n\n- Adapter improvements:\n - Updated all LLM adapters (ANT, OAI, GEM) to handle content arrays\n - Added specialized image content handling in the Anthropic adapter\n - Ensured proper JSON serialization/deserialization for all content types\n - Improved test coverage for content arrays\n\n- UI enhancements:\n - Added omitempty tags to reduce JSON response size\n - Updated TypeScript types to handle array content\n - Made field naming consistent (tool_error vs is_error)\n - Preserved backward compatibility for existing consumers\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\nChange-ID: s1a2b3c4d5e6f7g8h\n" }, { "commit": "4962f153fa0d6812543addd690be8dba8c04a406", "tree": "3a90c5736422c77f8ef3dc8d552640e65f1e07a0", "parents": [ "8b43ffbf41d06aee5ad5c734f1661a3b62cfe1a9" ], "author": { "name": "Autoformatter", "email": "bot@sketch.dev", "time": "Tue May 06 17:24:20 2025 +0000" }, "committer": { "name": "Autoformatter", "email": "bot@sketch.dev", "time": "Tue May 06 17:24:20 2025 +0000" }, "message": "all: fix formatting\n" }, { "commit": "33d282f80db786cc60ba521a38ed5166f23239ed", "tree": "9ed1f15c6d3081d5bef7d16b9d72e78a2c7780cf", "parents": [ "a9d87aa69cfefdc91ec7aaa6bc42907749748e76" ], "author": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Sat May 03 04:01:54 2025 +0000" }, "committer": { "name": "Philip Zeyliger", "email": "philip@bold.dev", "time": "Tue May 06 10:23:39 2025 -0700" }, "message": "Add browse tool support.\n\nI reviewed some MCPs (using OpenAI\u0027s deep research to help), and it\nhelped me choose chromedp as the relevant library and helped me come up\nwith an interface. This commit adds chrome to the Docker image which is\nkind of big. (I\u0027ve noticed that it\u0027s smaller on Ubuntu, where it doesn\u0027t\npull in X11.) go-playwright was a library contender as well.\n\nImplement browser automation tooling using chromedp\n\nThis implementation adds browser automation capabilities to the system via the chromedp library,\nenabling Claude to interact with web content effectively.\n\nKey features include:\n\n1. Core browser automation functionality:\n - Created new browsertools package in claudetool/browser\n - Implemented tools for navigating, clicking, typing, waiting for elements,\n getting text, evaluating JavaScript, taking screenshots, and scrolling\n - Added lazy browser initialization that defers until first use\n - Integrated with the agent to expose these tools to Claude\n\n2. Screenshot handling and display:\n - Implemented screenshot storage with UUID-based IDs in /tmp/sketch-screenshots\n - Added endpoint to serve screenshots via /screenshot/{id}\n - Created dedicated UI component for displaying screenshots\n - Ensured proper responsive design with loading states and error handling\n - Fixed URL paths for proper rehomed URL support\n - Modified tool calls component to auto-expand screenshot results\n\n3. Error handling and reliability:\n - Added graceful error handling for browser initialization failures\n - Implemented proper cleanup of browser resources\n\nThe browser automation tools provide a powerful way for Claude to interact with web content,\nmaking it possible to scrape data, test web applications, and automate web-based tasks.\n\nCo-Authored-By: sketch \u003chello@sketch.dev\u003e\n" } ] }