贵57.1倍的Claude Opus 4.8五项全输，赢它的不是模型，是Harness
Ranking
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - AOE Tech Labs published third-party benchmark results for its Floatboat Harness, claiming that running the cheap DeepSeek-V4-Flash inside its own agent harness beats Claude Opus 4.8 on all five benchmarks at ~1/57 the blended token cost. The pitch is that the non-model half of an agent system (runtime, loop, tools, infra) is a measurable, high-leverage performance lever.
- Single-variable setup: same DeepSeek-V4-Flash 0731 base, compared against DeepSeek's own official harness (not a bare API) in isolated sandboxes. Official harness scores 54.4/73.2/82.7/25.1/70.7; Floatboat lifts DeepSWE 54.4 → 67.25 and hits 87.80 on OpenAI's BrowseComp, above GPT-5.6 Terra (87.5) and Opus 4.8 (84.3).
- Gains scale monotonically with task horizon: 1.9% → 9.6% → 12.6% → 19.9% → 23.6%, attributed to long-horizon loop convergence rather than prompt tricks (typically 1–3 points).
- Proposes HLR (Harness Leverage Ratio) = harness gain ÷ model-upgrade gain; values 0.78× → 3.57×, e.g. DeepSWE's 12.85-point harness gain vs. the 3.6-point Opus 4.8 spread. Self-admittedly a new, non-standard metric sensitive to reference-model choice.
- Architectural claim: self-built Runtime, Agent Loop, Tools, Infra plus FloatSail evolution system are required, since SDK-wrapper products can't patch a loop that drops context at step 20. Note this is vendor-published data, not independent replication.
Sources (1)
贵57.1倍的Claude Opus 4.8五项全输，赢它的不是模型，是Harness
TL;DR - AOE Tech Labs published third-party benchmark results for its Floatboat Harness, claiming that running the cheap DeepSeek-V4-Flash inside its own agent harness beats Claude Opus 4.8 on all five benchmarks at ~1/57 the blended token cost. The pitch is that the non-model half of an agent system (runtime, loop, tools, infra) is a measurable, high-leverage performance lever.
- Single-variable setup: same DeepSeek-V4-Flash 0731 base, compared against DeepSeek's own official harness (not a bare API) in isolated sandboxes. Official harness scores 54.4/73.2/82.7/25.1/70.7; Floatboat lifts DeepSWE 54.4 → 67.25 and hits 87.80 on OpenAI's BrowseComp, above GPT-5.6 Terra (87.5) and Opus 4.8 (84.3).
- Gains scale monotonically with task horizon: 1.9% → 9.6% → 12.6% → 19.9% → 23.6%, attributed to long-horizon loop convergence rather than prompt tricks (typically 1–3 points).
- Proposes HLR (Harness Leverage Ratio) = harness gain ÷ model-upgrade gain; values 0.78× → 3.57×, e.g. DeepSWE's 12.85-point harness gain vs. the 3.6-point Opus 4.8 spread. Self-admittedly a new, non-standard metric sensitive to reference-model choice.
- Architectural claim: self-built Runtime, Agent Loop, Tools, Infra plus FloatSail evolution system are required, since SDK-wrapper products can't patch a loop that drops context at step 20. Note this is vendor-published data, not independent replication.