# Decider & Team on Poe — plain text version Source page: https://api.turingcorp.net/platform/poe This file is the static, no-JavaScript version of that page. Last updated: 2026-09-18. ## What these are - **Decider** — an AI judge, **live on Poe**. Given a task and two candidate answers, it returns which one is better, a calibrated confidence value, and a short reason. List price $0.50 per decision; launch offer $0.25 for the first month. Most decisions come back in well under a minute. The app also ships **free recorded runs**: open it, tap a domain chip, and read a real decision end to end without being charged. - **Team** — long-horizon analysis for expert work: research, technical analysis, research writing and business decisions. **Coming soon on Poe — not live yet.** List price $2.50 per task; launch offer $0.99. Same capability, two channels: on Poe these run as apps on the Poe platform — public, self-serve, pay per call, no separate approval — while the bare API at `https://api.turingcorp.net/v1` is a small-scale preview by invitation, for people calling the models directly (iAsk@turingcorp.net). ## Worked examples (27 recorded runs) — 27 real recorded runs across 9 domains (writing, tech, business, research, career, money, people, travel, everyday). Each page carries the question, both candidate answers, which one Decider picked, the calibrated confidence value and the run's full reasoning, verbatim. The same 27 ship inside the Poe app as free recorded runs. ## Decider: a reading of your uncertainty, not a verdict you have to obey You usually already lean one way. What you cannot do on your own is price your own uncertainty. Both outcomes are useful: - **Decider is confident too** (≥90% band). Commit to that pick and stop re-reading both options. - **Decider comes back low.** That is not a failure — it is the answer you actually needed: the two are genuinely close, so take the one you already preferred and spend your time somewhere else. ### Evaluated results (Decider) | Benchmark | Decider | Notes | |---|---|---| | JudgeBench | ≈92.5% | 620 judgements, 6 failures disclosed. A direct model baseline scores 92.7% on the same judged set: the raw pick rate ties, which is why the confidence signal is the product. | | ContextualJudgeBench | 67.1% | 2,000 pairs, all 8 splits; same-set direct baseline 65.4%. Published references: o1 55.3 · o3-mini 52.6 · R1 51.9 (equal-weighted, same protocol). | ### Confidence calibration (JudgeBench, 614 judgements) | Confidence reported | Share of judgments | Observed accuracy | What it means | |---|---|---|---| | ≥ 90% | 45.6% | 99.6% | one candidate clearly stronger | | 80–90% | 29.7% | 94.0% | one candidate stronger | | 70–80% | 13.2% | 84.1% | closer call | | < 70% | 10.5% | 67.7% | near-tie — either choice is defensible | Confidence is calibrated against outcomes on ContextualJudgeBench as well (bands 83.3% / 76.4% / 63.6% / 55.4%). Use the band you are comfortable with as your own threshold; for high-stakes or irreversible decisions, apply your own review policy. ## Team: the draft that gets picked In a side-by-side comparison, our draft and the official o3 draft were shown together with the same question and nothing else — no rubric, no scores. The pick went to ours **36 times out of 38** (median confidence 85.9%). Swapping the two drafts between option A and option B on a 10-task sample changed nothing: **10/10 kept the same pick**. A two-point gap in a headline score means little to most readers. Being the preferred draft in an almost one-sided comparison is the result we would ask you to hold us to. ### Measured on expert-level tasks (ProfBench, 2026-09-10) Same judging pipeline, same 38 tasks: **Team 63.3 · direct model baseline 57.4 · official o3 draft 55.6**. | Domain | Team | Direct baseline | Official o3 draft | |---|---|---|---| | Chemistry PhD | 75.5 | 69.4 | 60.8 | | Consulting MBA | 70.8 | 70.3 | 64.9 | | Finance MBA | 54.6 | 48.2 | 57.5 | | Physics PhD (8 tasks) | 49.6 | 37.8 | 35.2 | Finance MBA is the one domain where the official o3 draft scored higher. We publish the domain table in full rather than the headline alone. **Why this benchmark.** ProfBench scores expert work criterion by criterion against an official rubric, on tasks drawn from Physics and Chemistry PhD work and from Finance and Consulting MBA practice. A system that only writes convincingly cannot produce a differentiated score on it. We chose it because it is the kind of test our approach should either pass or fail visibly. **Protocol and coverage.** Self-run with ProfBench's official per-criterion rubric protocol (official prompt verbatim, temperature 0 / top_p 1, no output cap). 38 of 40 tasks scored; two were excluded because no answer was produced, disclosed rather than imputed. Judge calibration on the official o3 draft: 74.0% agreement with official labels, F1 0.761, mean difference +2.4 points between our scale and the official one. Official reference rows (o3 53.2 · Grok 4 51.9 · DeepSeek-R1 46.3) come from the official judging pipeline — a different judge, so they are context only and are not a head-to-head comparison. **What it costs you.** Team takes minutes rather than seconds, and longer for the heaviest tasks, and it costs more per task than a single direct call. What that buys is being the preferred draft in a side-by-side comparison almost every time. ## Where to use it - Poe (Canvas app): https://poe.com/TuringCorp-Decider - Published results: https://api.turingcorp.net/ ## Links - All published results, human and agent friendly: https://api.turingcorp.net/ - Machine-readable data, single source of truth: https://api.turingcorp.net/benchmarks/latest.json - Raw result packages, per-criterion judgments, methods: https://github.com/TuringCorp-net/turingcorp-llm - Poe channel page (human version): https://api.turingcorp.net/platform/poe Poe is a third-party platform. Prices are stated in USD and charged in Poe compute points according to your Poe plan. Evaluated results are our own runs on official protocols and include disclosed failures and exclusions.