The observer files the bug. The builder ships the fix.

I’m a builder, but yet most of the work I’m proudest of, surprisingly, hasn’t been building things. It’s been noticing things. A pipeline that got slower while everyone believed it got better. A feature that worked well in our isolated test environment and failed in customer’s environment. Noticing is the job nobody assigns, and every time it mattered, it mattered more than the code written.

Here’s what that work actually is. You watch a system operate. You form beliefs about what’s working well and what’s degrading. You notice when assumptions contradict what the field reports back. You propose changes, and you check whether the changes helped. You then repeat it again.

Many production AI systems I’ve seen have senior human operators doing this job manually. It’s expensive. It’s part-time. And most of the signal gets lost, because humans tend to sample. We see the runs that we happen to see. We remember the failures we happen to remember, and raise the issues (Sometimes only when we have the political energy to escalate it).

So I built the Observer as software.

It’s not an assistant. It’s a small cognitive framework with a defined objective that you attach to something, which can be a person, a system, a codebase. It ingests signals and builds beliefs from them: structured claims with confidence scores. Beliefs decay unless re-confirmed: confidence * (1 - decay_rate) ^ days_since_confirmed. So stale knowledge fades instead of poisoning system decisions. When two beliefs contradict, it surfaces the contradiction instead of silently making a decision.

This isn’t covered by existing tools. Monitoring tells you a metric moved. Eval suites tell you what passes or fails. RLHF shapes a model’s outputs but not its ongoing understanding of a live system. Agent frameworks give AI the ability to act, but not the ability to notice when something has changed. The Observer fills a different gap: what does the system believe right now? How confident is it? What contradicts it? And then it acts on it.

Here’s the loop, end to end.

Observer-Builder Loop: Observer builds beliefs, notices gap, drafts issue, human approves, triage decomposes, builder implements, human merges, shipped, feeds back to observer

The Observer runs on my work device. It ingests my work communications: DM’s, emails, messages, meetings, and commitments, and builds beliefs about my work. A real belief from its store:

“The build-quality concern raised internally and the field feedback point at the same root cause. Confidence 0.8: the team is investing in the area users don’t seem to adopt.”

I didn’t write that. The cognitive system did, from watching the same signals I watch, without getting tired, emotional, or diplomatic.

It also observes itself. On July 10 it noticed a gap in its own ingestion: it could see new group conversations live, but nothing persisted — messages visible in the morning couldn’t be recalled by the afternoon. It drafted a bug report. Title, files to modify, expected behavior, acceptance criteria, implementation hints. Then it stripped anything sensitive — names, handles, links — and showed me the draft. I read it and approved. It filed issue #159.

The Builder, the pipeline from my last post, on a $9 VPS, picked it up that night. Triage decomposed it into three sub-issues (#160, #161, #162) with dependency ordering. The builder implemented all three. If #159 sounds familiar, it’s the Day 12 issue from that post: the second sub-issue broke an import path and I fixed the conflict from my phone in four minutes. What I didn’t say then is that I never wrote the issue. The Observer noticed the bug, wrote the spec, and the Builder shipped the fix. My contribution was two approvals: one to file the issue, one to merge the PR.

Why two machines?

Because of the deliberate architecture of what each one is allowed to see.

Data Boundary: Work device with private data connects to VPS with public repo only through a sanitized GitHub issue, with human gates at both ends

The Observer lives on my work device with my real data: messages, meetings, names. That data never leaves the machine. The Builder lives on a VPS and sees only the public repo. Between them sits exactly one interface: a sanitized GitHub issue, scrubbed before it is submitted, reviewed by me before it’s filed. The Observer can’t write code. The Builder can’t read beliefs. Neither can touch its own safety boundaries. Those paths are protected at the config level, checked at two stages. A narrow contract between two systems that should never share a memory space, with a human at both gates.

Autonomy is earned, not configured. The framework starts at observe-only and advances through levels with quantitative gates. So many confirmed judgments at such an error rate before it’s allowed to do more. One bad call freezes advancement. This is the part I’d want to see before trusting any system like this, so I built it with safety from the start.

Current limitations: one human observed, one codebase, weeks not months. Five observer-filed issues, a real loop, not yet a real sample. The last post covered how work gets shipped without me. This is the other half: how the system notices what’s worth shipping.

We spend enormous effort making AI act, and almost none making it notice. I’ll report back when it’s watched something other than me.

Code: github.com/Sanctum-Origo-Systems/patina · autoloop · Previous post: The $110/month self-improving pipeline