Stop 1 · the clinic
Filed 200+ faxes with an agent that refuses to guess.
Beacon HealthYC W26
Software Engineer Intern · Aug 2026 to present · Remote
AI agents for primary-care admin work inside the EHR.
- The problem
- Clinic staff filed every incoming fax by hand, and the first screen-clicking agent guessed when it wasn't sure.
- What I did
- Rebuilt it as code-based tools with guardrails that match patients, block duplicates, and refuse to guess.
- The result
- 200+ faxes filed in its first 2 weeks at about 80% lower estimated cost; now runs twice a day, 7 days a week.
- faxes filed, first 2 weeks
- 200+faxes filed, first 2 weeks
- lower est. cost per fax
- ~80%lower est. cost per fax
- checkout success, live sessions
- 33% → 100%checkout success, live sessions
What I did
- Shipped 5 AI agents that ran on live clinic data in 3 EHR systems (athenaOne, PointClickCare, Elation): 50+ patient checkouts with chart transfers from athenaOne to PointClickCare, and 200+ fax filings, for my 2 clinic accounts.
- Rebuilt the fax agent from screen clicking into code-based tools with guardrails that match patients, block duplicates, and refuse to guess. The clinic then moved it from once a weekday to twice a day, 7 days a week, and expanded it from clinical documents to nearly every fax type.
- Filed 200+ faxes into the right patient charts in its first 2 weeks, at about 80% lower estimated cost per fax after moving off a computer-use runner to Stagehand.
- Raised a checkout agent from 33% success over its first 131 live runs to 100% of runs with a live EHR session, by auditing 80+ failed runs against system logs and adding a fix for each failure type. Cut successful run time 31%.
What I built at Beacon
Beacon Health (YC W26) · Software Engineer Intern · Aug 2026 to present
- 5 AI agents that ran on live clinic data across athenaOne, PointClickCare and Elation: patient checkouts with chart transfers, and fax filing.
- The fax agent reads each fax, matches the patient, checks for duplicates, and files it to the chart. It refuses to guess. 200+ faxes filed in its first 2 weeks, now running twice a day, 7 days a week.
- He rebuilt it from screen clicking into code-based tools, which cut the estimated cost per fax about 80%.
- He also found a versioning bug where three live agents were running their first-ever instructions instead of the current ones. An engineer confirmed and fixed it the same day.
What happens when it breaks
Checkout agent · 33% → 100% of live-session runs
- First 131 live runs: 33% succeeded. He pulled every failed run, read its logs, and sorted the failures into 9 causes: visit not found, page didn't load, resident not found, PDF export problems, facility mismatch, and so on.
- One fix per cause. Bounded waits with a retry, a search ladder for finding residents, start-state checks for each stage, and a rule to never type credentials and fail cleanly on a login prompt.
- After that, 100% of runs with a live EHR session succeeded. Successful runs also got 31% faster, because batching clicks cut model turns from 47 to 32.
- He also stopped trusting the agent's own success flag after one 'success' had empty outputs, and wrote a job to reconcile status against the real record.
One real run, traced. Tap a step.
Second Look: search by date of birth alone, re-read the name from the fax, and accept only if exactly one active chart fits.
- strategy
- DOB only, name re-read
- result
- matched: exactly 1 active chart
- if ambiguous
- never filed, left for staff
Case study · the fax agent
From a screen-clicking agent that guessed, to code-based tools that refuse to.
The problem
Clinic staff filed every incoming fax by hand: 10 to 15 a day, 89 waiting in the inbox on one October morning. Each one means reading it, finding the patient, picking the document type and title, checking it is not a duplicate, filing it, deleting the cover page, and sometimes messaging the right staff group.
v1, and why it broke
The first agent took screenshots, let the model decide where to click, and repeated. Two machines ran fixed slices of the inbox. It worked until it did not:
- It guessed a date it could not read (21st or 31st), filed it under July instead of October, and a physician caught it.
- It removed a required result tag because the EHR's description seemed to contradict the finding.
- It could not see earlier runs, so re-sent faxes were filed twice.
- Sessions expired about every 12 hours and runs stopped at the login page. One run crashed at fax 23 of 25 with no log.
- About 12 minutes of run time per filed fax.
The redesign
The planner now calls typed tools that use the EHR's own web requests from inside the signed-in page. Nine tools at first, fourteen now. Reading is cheap and repeatable. Every write needs a permit issued for that run, and every write is read back.
Guardrails, in code, not in the prompt
- Patient match: file only when exactly one active chart matches last name, first name and date of birth. Fallbacks: a second look by DOB alone with the name re-read; name plus printed sex and age when no DOB is printed. Two matches means a human.
- Duplicates: blocked on the same title, same type on the same date, same order number, home health plans within 14 days, or the same study within 7 days. A block lifts only after the run has read the existing report and named it as a different document.
- Dates: the document date must fall within 12 months of the received date. Unclear dates are skipped, never estimated.
- Locks: a page locked by staff is never overridden. Locks the run takes are the only ones it releases.
Coverage, from evidence
I read 2,514 rows of staff activity logs across 623 charts and built a table of each fax type, the document type and title staff actually use, and who gets messaged. The coverage proposal covered 30 of the 42 faxes staff were still filing by hand. The clinic approved it on October 15, moved the schedule to twice a day, seven days a week, and a physician's corrections became code rules the same week.
Cost, honestly
The real tradeoff is a step-heavy planner that re-sends a large prompt every step, plus occasional mistakes. A delete audit found 27 of 30 agent deletes correct. Deleted items stay recoverable, and deletes were turned off until two review findings are fixed. Mistakes get caught by a read-back after every write, a per-fax ledger, a daily audit report, and a side-by-side comparison before any delete.
What's next
- Phase 1 (approved): one call per fax, prompt cut from 46.8k to 9.9k characters. Target under an estimated $0.30 per fax.
- Phase 2 (estimated target $0.05): no planner. Code loops the inbox, the model reads each page once and answers only judgment questions, and it must match a graded set of about 104 decided faxes field by field before it writes anything.
- Agents
- Computer use
- Guardrails
- Evals
- Stagehand
- Claude API
- athenaOne
- PointClickCare
- Elation