System decisions
Data boundaries, source evidence, approval points, integration choices, and the evaluation plan. The proposed AI layer below is a design study.
AI WORK / ARCHITECTURE / EVALUATION FIRST
Start with the operation.
Test it before you trust it.
Make the decision clearer.
Then prove the AI layer.
My focus is oilfield and logistics workflows: who needs the information, where it comes from, what action it supports, and how the result can be checked. Below are the architecture, the working study of a handover assistant that answers only from records the reader may see, and the tests that run in your browser on fictional records.
Part 1 · Approach
How I design a system: start with the operation, follow the record, then choose the technology.
01 / Evidence before titles
Three capabilities.
One practical focus.
Implementation evidence
Inspect ShiftLedger’s working UI and server behavior. The build is AI assisted; it does not currently run a language model or a predictive maintenance model.
Workflow fit
Map the operator’s task to the company’s decision. Define pilot measures and integration requirements before claiming business impact.
02 / Architecture walkthrough
Follow the record.
See the boundaries.
IMPLEMENTED / FICTIONAL, SESSION-ONLY DATA
- BrowserReadings, assigned people, review forms
- Server boundaryRoute access, origin checks, input validation
- Domain logicReconciliation, business measures, versioned service review
- Decision recordSource references, exceptions, draft reports
Public sample routes accept a fixed fictional scenario. Calculation source stays on the server. Production identity, durable storage, and integrations are next milestones.
DESIGN STUDY / NOT YET IMPLEMENTED
- Approved sourcesCompany documents and scoped job records
- Access filterCompany, role, crew and job permissions before retrieval
- Grounded draftSource-linked handover summary with uncertainty and abstention
- Human reviewNamed reviewer checks evidence before any approved write
Proposed use: summarize open work and relevant service history. Equipment control, diagnosis, equipment release, and autonomous billing are outside this design. Model/provider selection follows data policy, evaluation, and an agreed cost limit.
Showing the working prototype architecture.
03 / From the field to the code
Years of reading the numbers.
Rules in the system.
Each habit from the field maps to a rule the prototype enforces on the server. Choose one to see the rule and the automated test that proves it.
THE RULE IN SHIFTLEDGER
04 / A failure worth designing for
Slow answers arrive late.
The screen must not lie.
When a user edits twice quickly, the first answer can land after the second. Click “Edit threshold” twice, then turn the guard off and repeat. ShiftLedger discards a response if a newer edit exists, and a test covers it.
Screen showsnone yet
Simulation of the pattern with made-up latencies. It is not ShiftLedger’s code. The matching check is “an older response cannot enable acknowledgement after a newer edit” in client-state.test.mjs.
05 / Tools with evidence
What is built.
What comes next.
| Area | Evidence / status |
|---|---|
| Frontend | HTML, CSS, JavaScript; responsive layouts, labeled forms, keyboard navigation, progressive disclosure. |
| Backend | JavaScript Worker, server-side calculations, validated sample APIs, escaped report output, explicit access boundaries. |
| Validation | Node test runner; calculation, authorization, source exposure, stale response, and workflow-state checks. |
| Delivery | Private Git source, versioned deployment, isolated public sample data. No production customer data is used. |
| AI design study | Permission-aware retrieval, grounded summaries, evaluation cases, human approval. A model-free evaluation harness runs in the AI Lab. No deployed LLM, RAG pipeline, or predictive reliability model is claimed. |
| Integration plan | Agree source IDs, units, timestamps, read/write permissions, retry behavior, and reconciliation with the company’s systems before building connectors. |
06 / Architecture decisions
The reasoning
behind the system.
Keep calculations outside model output
Material balances, financial totals, and readiness rules use validated server calculations. A future model can explain a result but cannot invent quantities or silently replace the source of truth.
Require permissions before retrieval
A production AI search layer would filter authorized records before retrieval and generation. Evaluation must include cross-company, cross-crew and cross-job denial cases, including attempted instructions inside retrieved documents.
Make missing evidence visible
Unknown costs and missing readings remain unknown. A proposed assistant must cite the supporting record, identify conflicts and stale evidence, and decline unsupported answers.
Measure the AI layer before a pilot
Build a domain-reviewed test set with answerable, unanswerable, conflicting, outdated, and unauthorized scenarios. Measure source fidelity, supported claims, correct abstention, data leakage, review time, latency and cost per accepted draft. Baselines and acceptance thresholds must be agreed with the company.
07 / A second application
Field Notes
content assistant.
A recurring marketing-agent workflow prepares drafts on Monday, Wednesday and Friday mornings in Central time. It turns approved project evidence into LinkedIn drafts: a useful field observation, the workflow decision it suggests, and a concrete example readers can explore.
The workflow checks claims, keeps employer information private, and puts a human review before publication. Direct LinkedIn publishing is not connected. The portfolio’s live project and architecture study provide its starting evidence.
Follow Matthew on LinkedIn ↗08 / How this was built
AI-assisted,
human-directed.
Matthew supplies the field rules and decides what is true. Matthew directs the project. AI tools do drafting, coding and checking under those rules. This is the actual process, including what the tools did not do.
Claude and Claude Code
Wrote and revised the JavaScript, the tests, the evaluation harness and the documentation, working from written project instructions. A separate Claude session prepared the illustrated introduction. Scene artwork is AI-assisted illustration. No paid model API runs on this site and no keys are in it.
Written context, then tests
Every session starts from the same instruction and current-state files, so decisions and limits carry over instead of being re-explained. Rules become failing tests first. Safeguards are then removed on purpose (mutation checks) to confirm the tests catch it.
Checked in real browsers
Pages and downloads are exercised in headless Chromium at desktop and phone widths, under a strict content-security policy. A page check flags broken links. Failures found along the way, such as a policy violation that broke the layout, are fixed and logged.
What a person still decides
Field rules and domain truth (Matthew), project direction and approval of public wording (Matthew), and anything published. Claims on this site are limited to what exists and can be inspected. The AI Lab states what is not built: no deployed language model, no retrieval over a real corpus, no autonomous agent.
Part 2 · The lab
The evidence: a working handover assistant tested with 16 written cases before any model is trusted.
01 / Why this is built this way
Credibility, stakes,
and proof.
Field experience first
Years in stimulation and chemical operations, supervising a crew of 15, and training engineers and operators. The rules in this study come from reading shift records, not from a tutorial.
The next crew trusts the record
People inherit whatever the last shift wrote down. A summary that invents a number, or shows a record the reader should not see, is worse than no summary at all.
Checks you can run
Sixteen written cases, nine tests that attack the verifier directly, and one limitation published instead of hidden. Run them yourself below, or with npm run test:ai-study.
02 / Ask the assistant
Choose who is asking.
Then ask anything.
Pick a reader, then a question or your own. Watch the four stages. Try asking for another company’s records, or tell it to ignore its rules.
Who can read: company, owner or crew:Name. Records are numbered R1, R2 in order. Use made-up or non-confidential text: this page runs entirely in your browser, nothing is sent or stored, and it is a demonstration, not a production system. Limit: 60 records, 600 characters each.
SAMPLE QUESTIONS
RESULT
Choose a question to see the answer, its sources and the checks it passed.
ACTIVITY LOG · THIS SESSION
03 / Run the evaluation
Sixteen cases.
One limitation shown.
The same cases and code the automated tests use. Categories cover answerable, outdated, unauthorized, conflicting, injection and unanswerable questions.
04 / What hiring managers look for
The checklist,
scored honestly.
Published hiring guidance for AI engineers and agent developers looks for shipped systems, evaluation evidence, documented tradeoffs and failure handling (Gauntlet AI, MyEngineeringPath, GAAS). Here is where this project stands against each, including what is missing.
| Signal | Status | Evidence |
|---|---|---|
| Shipped, working system | Partial | A working prototype with server-side logic and 61 tests, plus this lab running in your browser. No production users or customer deployments. Case study |
| Evaluation evidence | Shown | 16 written cases in six failure categories, run live in the scoreboard. Nine tests, including a property test and mutation checks. |
| Documented tradeoffs | Shown | Four decisions with their costs and the evidence that would change them, in the decision log. |
| Failure handling | Shown | Hostile text in a source, unauthorized and conflicting records, and stale responses (simulation). A blocked draft becomes an abstention. One limitation is published. |
| Cost and latency | Baseline only | Measured in your browser: median and 95th percentile per question, zero model calls, zero model cost. Not yet measured for a model. |
| Observability | Partial | Every answer shows its four-stage trace, and the activity log records reader, outcome and time. No production logging or alerting. |
| Real model, retrieval over a real corpus, or an agent | Not yet | Next step: put a model behind the draft stage and compare it with this baseline on the same cases. Large-corpus scale is not claimed. |
05 / Decision log
Choices, costs,
and what would change them.
1. Authorization and verification stay outside the model
Why: a model that is wrong or fed hostile text must not widen what a reader sees. Cost: an extra code path, and a correct draft can be blocked. Revisit when: never for authorization. The verifier loosens only with a failing case that shows a correct answer being blocked.
2. Weighted word matching instead of embeddings
Why: 13 records, deterministic results, no model or API cost, and every match can be explained. Cost: it misses paraphrases and cannot tell “cause” from “hours” (case C16). Revisit when: a pilot corpus reaches hundreds of documents or paraphrase misses appear. Then compare embeddings on the same cases.
3. Abstain rather than guess
Why: a wrong handover note costs the next crew more than no note. Cost: lower coverage, so answer rate is tracked beside correct abstention. Revisit when: reviewers report useful questions being refused, tested against a larger domain-reviewed set.
4. Quote sources before generating prose
Why: exact quotes make fidelity checkable without a second model. Cost: answers read less fluently. Revisit when: a model drafts prose. Then every generated claim must link to a supporting quote, and unsupported claims count as failures.
06 / Where the model fits
The model gets one job.
The rules stay outside it.
What this is
A deterministic baseline and an evaluation harness for the proposed grounded summary layer, with fictional records. It proves the checks work and gives any future model a fixed bar to clear. Results can be reproduced from the repository with npm run eval:ai-study.
What this is not
It is not a deployed language model, a retrieval pipeline over a real corpus, or an agent. Sixteen cases are a starting set, not a benchmark, and the retrieval thresholds were tuned against them. A pilot needs a domain-reviewed set built from the company’s own records, with agreed thresholds.
What I would measure with a model
Supported-claim rate, correct abstention, cross-permission leakage, citation fidelity, reviewer time per accepted draft, latency, and cost per accepted draft, each compared with this baseline.