Tochal is a compiler for agentic AI workloads that transforms sequential agent execution graphs into optimized parallel schedules — the way LLVM optimizes programs, but for LLM agent pipelines.
Public demo — coming soonLangGraph, CrewAI, and similar systems build agents as DAGs, but many implementations still execute nodes one at a time — even when they are fully independent.
Tochal analyzes the execution graph and applies three compiler-style passes before runtime. It extracts parallelism, merges sequential LLM calls that share the same model configuration, and speculatively starts likely branches ahead of time.
For LLM-heavy workflows where each call costs roughly 0.5–5 seconds, that scheduling difference is a major performance lever.
Two run at compile time, rewriting the graph before anything executes. The third runs while your agent is live.
Finds independent nodes at the same DAG level and runs them concurrently with asyncio.gather() instead of serial execution.
Detects sequential chains with the same model config and replaces multiple API calls with a single multi-part prompt.
For condition nodes with high prior probability, pre-starts the likely branch concurrently with evaluation and keeps the fastest path.
The compiler never modifies the original graph; it operates on a deep copy so the agent definition remains intact.
combined simulated speedup across the three optimization passes.
Parallelism extraction reached 2.11×, LLM-call merging reached 1.25×, and speculative branch execution reached 1.33× on representative agent workflows.
Measured on Llama 3.3-70B via Groq, the average real-world speedup was 1.41×, with the merging pass reaching 1.89× on a chain-style pipeline.
| Pass | Scenario | Unoptimised | Optimised | Speedup |
|---|---|---|---|---|
| Loading latest run… | ||||
These figures reflect the project’s simulated and real Groq API benchmark suite. The table is pulled live from the benchmark log when available.
We're finishing real-hardware validation on Groq before opening it up. Tell us about your use case and we'll reach out personally when early access opens.
Coming soon
No spam — just one email, the moment the public demo is ready.