Meet Collinear at booth 304.
October 6 to 9, Hilton San Francisco Union Square
Come talk hard but fair benchmarks, frontier model failures, and the training environments that turn those failures into measurable capability gains.
Meet us at our 2026 booth 304
Benchmarking AI agents for long-term planning and consistent execution.
An agent runs a simulated startup for a full year, with eight employees, contracts from six clients, 35% of them secretly adversarial, and $200K to start. Final funds are the score. Half of the frontier models we tested finished the year with less than they started.
Ranking models is easy. Finding the next useful learning signal is harder. Our benchmarks are designed to reveal where frontier systems still break, with evaluation that is difficult to game and fair to measure.
Can coding agents find and fix real vulnerabilities?
Held-out audit-and-patch tasks across real codebases, scored with strict programmatic verification. The exploit must stop, and existing tests must still pass.
A clearer test of whether computer-use agents can do real work.
Designed to move past brittle scripts and surface the behaviors that matter when agents navigate interfaces, tools, state, and changing task conditions.
Difficulty is only useful when the score means something. We build environments and verifiers that create real capability pressure without rewarding shortcuts.
Tasks sit where strong models still fail, so every run can reveal the next hill to climb.
Agents operate in runnable tools and persistent environments, not isolated prompts.
Deterministic checks, state inspection, and calibrated rubrics reward success, not plausible-looking output.
The same system can diagnose a gap, create training trajectories, and measure held-out gains.
Collinear is a simulation lab for frontier intelligence. We combine high-value tasks, rigorous evaluation, and off-the-shelf delivery so frontier teams get more useful learning from every model iteration.
Bring your model, harness, and hardest capability gap. We will map the data and evaluation that can move it. We are also hiring researchers and engineers who want their work to shape the frontier.