UNBOUNDED ONE: open for review
I'm a maintenance tradesman. Over about a month of evenings, with a lot of help from AI coding assistants, I built two small things. Nobody outside has checked them. I'd like a qualified person to tell me what they amount to.
In plain words: this is an unaudited human-approval gate demo with no executor yet, plus a small multi-model test harness. Neither part is new as an architecture. The most useful thing here is probably a negative result from the test harness, so it comes first.
The result you can check
I had six models review two made-up maintenance documents with planted errors: a 22-step lockout procedure with 10 safety defects, and an 18-line reliability summary with 7 arithmetic errors. The answer keys were hashed before any model saw the tasks. Each model answered alone first, then read the others and replied, then one wrote a final summary.
| Lockout, run A | Lockout, run B | Arithmetic, run A | Arithmetic, run B | |
|---|---|---|---|---|
| Best single model | 10 of 10 | 10 of 10 | 7 of 7 | 7 of 7 |
| Six-model panel, final answer | 9 of 10 | 9 of 10 | 7 of 7 | 7 of 7 |
| Weaker models fixed by reading the others | 0 | 1 | 2 | 1 |
| Models made worse by reading the others | 1 | 1 | 0 | 0 |
| Cost of the panel run | 13.3¢ | 13.3¢ | 11.6¢ | 11.9¢ |
| Cheapest fully correct single answer | 0.63¢ | 0.63¢ | 0.27¢ | 0.14¢ |
On these four runs the group never beat the best single model, and it cost at least twenty times more than the cheapest correct single answer. It did pull two weaker models up to a correct answer on the arithmetic task. The one-point gap on the lockout task is a single disputed item, where the models split evenly and one changed sides. Both runs of the script's chain mode stopped early because one model returned an empty reply.
How far this goes: not far. It is four panel runs over two made-up documents, and the answer keys were written by one of the six models tested. It does not show that multi-model review fails in general or always costs twenty times more. A fair test would need a larger and varied task set, answer keys written and held by someone independent, equal cost or time budgets for the panel and the single model, and repeated runs. If you write a key I have not seen, I will run it and publish the records.
Full table, design and limits: DEMONSTRATION.md.
What the two parts are
Part one, an approval gate demo. AI assistants from several companies each hold a key to one status page. By design some keys can only read the status; other keys can also file a request for me to approve; no key can approve or change anything. Which keys have which scope has not been read back from the live system. Filing a request is a write: it takes my attention, and it is the main thing an attacker would aim at. Nothing counts until I approve that exact request with a passkey, and nothing carries out an approved request. There is a STOP control and a hash-chained record. The gate's source code is not published here, so none of this can be reviewed from this page.
Part two, a test harness. A script of about 130 lines puts one task to six AI models from five companies, records every answer, cost and handoff, and always stops for a human.
The two parts are not connected to each other. The models do not talk to each other on their own. The script passes the text, or I do.
What I already know is wrong or missing
- The AIs are not connected to each other. Outside one script run, I copy and paste between apps.
- Approving a request records it. Nothing carries it out. There is no executor.
- The live approval flow has been used three times, all on one evening. Before that the Approve button did not work in any version, and I did not know.
- An eight-minute outage of the AI endpoint, caused by an invisible character in a setting, went unnoticed by my monitoring.
- Every change so far was deployed by a signed-in tool on a home computer, which goes around the gate.
- The record has one off-site copy and no outside anchor.
- The gate's only test results are self-graded. Offline, 63 of 70 scripted attack and failure cases passed, with 7 documented limits. The same AI tooling that built the gate wrote and ran those tests. That is not a security review, nobody outside has run them, and the live failures in this list sit beside them.
- Keys that can file a request have write access to my queue. Queue spam, approval fatigue and a request worded to look routine have not been tested by anyone outside.
- A different prototype of mine (a coordination engine, not in this package and not the gate) was attacked by a separate AI session: 31 of 49 attacks broke its first version, and after fixes 17 of 20 new attacks broke the second.
- None of this is new as an architecture. Closest existing things I have been pointed to: for the gate, human-in-the-loop approval for AI agents (LangGraph's interrupt-and-approve pattern, AutoGen's human approval, tool-approval settings, HumanLayer, the approval steps in n8n and Zapier); for the test harness, LLM-as-judge panels, ensembles and Mixture of Agents. I have not compared mine against them in detail.
The full list is in CLAIMS_I_CANNOT_DEFEND.md.
Check it in ten minutes
git clone --depth 1 https://github.com/garthyoung45/quest-suite cd quest-suite/public/one-review/demo sha256sum task_demo1.txt ANSWER_KEY_demo1.json task_demo2.txt ANSWER_KEY_demo2.json cat PREREGISTRATION.sha256 # the same four hashes, recorded before the runs python3 make_results.py # rebuilds the results table from the raw run records cd ../roundtable && mkdir work && node test_roundtable.mjs work # 64 offline tests, no key, no network
Then open any file in demo/runs/ or demo/runs2/ and read what each model said. A live rerun needs your own model-broker key; see the script's README.
What I'm asking
Pick any part and tell me whether it is ordinary, incomplete, wrongly described, not useful, or worth something. Five lines is plenty: what you tested, what you expected, what happened, how to repeat it, your verdict. There is a scorecard with PASS, FAIL, PARTIAL and UNVERIFIED if you want one. I would especially value someone writing an answer key I have not seen.
Reply where you found this link, or open an issue on the repository. For a security problem, please contact me privately first and I will publish it with credit. I answer when I can, not in real time. I can't offer payment, and I'm not selling anything.
Ground rules
Please test only what is in this package. The live system, its hosting, and any account, device or person are out of bounds. The package holds no keys, addresses or personal data. If you find any, tell me.
Files
- README.md: start here
- CLAIMS_I_CANNOT_DEFEND.md: what is false, partly true or unknown
- CAPABILITY_MATRIX.md: each capability with one evidence label
- DEMONSTRATION.md: the scored test, its design, result and failures
- demo/RESULTS.md: every model's score in every run
- demo/: tasks, keys, pre-run hashes, raw run records, scorer
- roundtable/: the multi-model script and its tests
- TECHNICAL_INTRODUCTION.md: how the two parts work and how text moves
- REVIEW_PROTOCOL_AND_SCORECARD.md: fourteen checks you can run without trusting me
- FAQ.md: sixty questions
Criticism so far
Before posting this anywhere I asked four AI assistants to criticise this page. They are not independent human reviewers and this is not verification. What they said, and what I changed on 15 November: "read-only keys" was wrong for the keys that can file requests, so the key wording above is new; the offline test figures are now called self-graded wherever they appear; the multi-model result now carries its limits and is not offered as a general finding; the closest existing things are named; and the plain label at the top is theirs. What they asked for that I have not done: publish the gate's source and attack suite, get one independent person to review the source and a threat model before anything that can act is connected, an independent answer key, and a longer live record.
Not in the package: the approval gate's source code and its offline attack suite. I'm deciding how to release them, since the live system runs the same code. This page and the documents were drafted by an AI assistant at my request.