Brian Kotch

AISecurityAgents
随筆

Conversations with Z 01

Tags: AI, Security, Agents

Banner: the GLM agent uploading a verification screenshot to an unknown provider

I was doing some image analysis work and I had blown my orchestrator budget finally getting around to building that showpiece professional website. What am I going to do? Take down my private blog? No, obviously own myself through simple negligence and watch data get exfiltrated in Chinese. Harmless in the end, but pants-crapping in the moment.

Since I trust none of these frontier lab bastards, I am learning as much about them as I can. I learn by doing. So, I am hopping the subscription plans as soon as I hit their monthly limit. I am certainly not using anything metered until the economy collapses and the frontier stabilizes. But, it turns out, the orchestrator eats reasoning as guide the small models running through the laptop.

I wanted to dive down into the GLM world for a bit, but it wasn't like I wouldn't trust an OpenCode provider to NOT massage my prompts between them and the great firewall. So, I went straight to the source. I dove into Z.ai. You know, the grey market one that is good for security scanning at the cost of your racist suspicions from a prolonged propaganda campaign. I am not NOT worried about ANY of these companies exfiltrating sensitive information. But, China also has scary good hackers. So.

Website. Follow the Skitch designs. Create reasonable front-ends and iterate over the changes. Ollama writes the code. Z.ai (GLM5.3) orchestrates. Processes verify and fix.

I asked it to fix some ui problems on my Japanese-themed site and I watched it upload a screenshot for UI verification to some random server. It wasn't supposed to make curl requests without explicit authorization to un-whitelisted sources. It certainly wasn't supposed to fucking upload anything.

I read the locks and had about 15 seconds' worth of timestamp showing that it had encountered an error trying to run a library to verify the presence of a rendered UI element. I took precautions to keep it from installing random libraries through pip. So, it decided to upload it to a ucloud CDN with "anthropic" as the bucket name.

This is a classic case of knowing exactly what your agent can do. Sandboxes, least privilege access, identity keys, rotation. That file could have been a lot worse than a screenshot.

Aftermath: the blog index with the AI-written posts tagged as slop and the gibberish Sam Altman ramen interviews published

和敬清寂 · All Rights Reserved Brian Kotch