How AI builds a spreadsheet: openpyxl, then 430 MB of LibreOffice
Most AI agents recalculate their spreadsheets with LibreOffice. It misses one in eight Excel test cases, lets broken files through, and takes about 90 times longer than it needs to.
Hjalmar Gislason19 November 2026 · 5 min readIn September, Simon Willison went poking around his disk and found 1.7 GB of runtime inside OpenAI's Codex desktop app, since renamed ChatGPT. Next to Python, Node and git sat a complete headless LibreOffice: 430 MB of office suite, shipped so the agent can work with documents.
This did not surprise us. We have watched this pipeline since the first AI spreadsheet experiments, and it is still how most of the industry builds spreadsheets with AI. It made sense as a stopgap. It doesn't anymore: next to an engine built for the job, it is less accurate, far slower and more than a hundred times bigger.

The recipe
Ask an AI agent for an Excel model and it will almost always write the file with openpyxl, the Python library all over its training data. openpyxl writes formulas but cannot calculate them, so the formula cells hold no values. Anything that reads the stored values sees blanks: the Python data library pandas, file previews, and the agent itself, checking its own work. And the model writes formulas the way a person types them, not the way Excel stores them. Leave out the _xlfn. prefix Excel expects on XLOOKUP inside the file and the cell opens in Excel as #NAME?. Write a LET without its _xlpm. prefixes, or a spill reference like A1#, and Excel greets the whole file with "We found a problem with some content".
So the pipeline needs something to calculate the file, and the answer is nearly always LibreOffice. Anthropic's public xlsx skill runs LibreOffice to recalculate the workbook. Alibaba's QwenPaw agent ships an adapted copy of the same script. Moonshot's Kimi checks its files against LibreOffice's recalculation, according to an extracted copy of its skills. OpenAI bundles the whole suite into its desktop app.
LibreOffice is free, it is already there, and it opens xlsx. None of that means it calculates like Excel. So how close does it get?
Missing one case in eight
Last week we published the GRID Spreadsheet Benchmark: over 100,000 test cases from eight open source test suites, 13 engines, every case checked against Excel itself. LibreOffice matches Excel on 87.4% of them.
That sounds respectable, and LibreOffice Calc is a good spreadsheet app. The test is deliberately hard: it is built from edge cases, not the SUMs that fill most of a real model. But edge cases are where a model goes quietly wrong, matching Excel on them is a high bar, and the last few percent are where the hard work is. 87.4% is 126 misses in every 1,000 cases. The GRID engine gets 1.2 wrong.
LibreOffice isn't sloppy. It is a different spreadsheet app with its own opinions. On its own project's test suite it agrees with Excel only about 84% of the time, because many of those tests expect LibreOffice's answer: it converts units Excel has never heard of, and it finds interest rates where Excel gives up with #NUM!. Of the nearly 500 Excel functions in the benchmark, only 16 pass every test in LibreOffice. In GRID, 439 do.
Then the tool starts dictating the spreadsheet. Anthropic's skill tells the model to never use XLOOKUP, XMATCH, SORT, FILTER, UNIQUE or SEQUENCE, saying the LibreOffice it runs can't evaluate them. LibreOffice has handled them since 2024, but the rule stands: the agent builds an older, worse spreadsheet to suit its calculator.
"We found a problem with some content"
In most of these pipelines, when LibreOffice recalculates a file, it also saves it, and it saves it the way LibreOffice understands it. Usually the file still opens in Excel. Sometimes it doesn't.
Take the sensitivity table, a fixture of every financial model: a what-if data table showing, say, the implied share price under different WACC and terminal growth assumptions. Put a workbook with one through a pipeline that saves through LibreOffice, like QwenPaw's, open it in Excel, and you get "We found a problem with some content". LibreOffice has written the table back as a formula Excel doesn't have. Click Yes, and Excel repairs the file by turning every cell of the table into a fixed number. The numbers are right for today's inputs. Change the WACC and nothing moves. The model still looks finished. It just stopped being a model.

Anthropic's skill was recently changed to recalculate on a copy and write only the values back into the file. That is a sensible patch, and an admission that LibreOffice can't be trusted to save the file. It is still trusted to calculate it, and to tell the agent the file is fine.
That trust is misplaced too, because LibreOffice waves through problems it didn't cause. Ask an agent for a table with years as column headings, 2025 and 2026, as in nearly every financial model. openpyxl writes the headings as numbers, and Excel requires table headings to be text. Name a table Q1 and it is the same story, since Excel reads that as a cell address. LibreOffice opens these files, recalculates them without complaint and saves them. The agent's check passes. Excel opens the file with "We found a problem with some content".
430 MB and a second and a half
Even when it gets the answer right, LibreOffice is the wrong shape for the job. Every check boots a desktop office suite in headless mode, loads the file, calculates, and writes the file back. On nearly 1,800 benchmark workbooks, run on the same machine, that took a median of 1.6 seconds per workbook. The GRID engine took 18 milliseconds, and it doesn't need to save: it holds the workbook in memory and writes the file once, when the agent is done. These are small workbooks, so most of that time is LibreOffice starting up, and today's agent skills pay it on every check. Anthropic's starts it twice. The obvious fix is to keep it running. In a separate test it still took about 80 milliseconds per workbook, and now the client keeps an office suite running in the background whenever a spreadsheet might come up in any thread.
Then there is the weight. ChatGPT's headless LibreOffice is 430 MB. The GRID engine is a 3.5 MB package that runs in process, in Node or in the browser.
The embedded LibreOffice may actually be doing other jobs too, like rendering documents and converting them to PDF. But those jobs have lighter tools as well. SuperDoc builds, edits and renders Word documents in the browser: an engine for DOCX, the way GRID's is for XLSX. A couple of small JavaScript libraries turn DOCX into PDF without a desktop suite. However you slice it, 430 MB of LibreOffice in a client is a kludge.
Calculate it right
AI is about to write more spreadsheets than people ever did. Those files will be opened in Excel, sent to clients and used for decisions, and Excel is where the numbers are checked. If an agent's model shows one figure and the same file shows another in Excel, the user doesn't conclude that LibreOffice and Excel differ on a corner case. They conclude the AI got it wrong. They start rechecking every number, and they stop trusting the next model before they open it. One wrong cell is enough to spend the trust that was supposed to save them the work.
The GRID engine calculates the way Excel does, in milliseconds, without an office suite attached. The benchmark and its harness are open, so check the numbers. If you build agents that write spreadsheets, talk to us.
News
Updates and announcements
10.11.2026
The GRID Spreadsheet Benchmark: how close do spreadsheet engines get to Excel?
Every spreadsheet solution will say it is compatible with Excel. Now we can check those claims. For eight years our job at GRID has been to build a spreadsheet engine that gives the same answer as Excel, because if a model returns different numbers nobody will trust your solution. To make sure we'd earn that trust, we had to build a test rig. Today we are opening that rig up. The GRID Spreadsheet Benchmark measures how closely an engine's formula results match Excel's. It runs thirteen engines today, the results are live on the benchmark page, and everything needed to rerun it is in a public repository. If you are choosing an engine to build on, or building one yourself, this is meant for you.
27.07.2026
An agent with a spreadsheet engine beats one without
Our results are now on the official SpreadsheetBench leaderboard: 91.25% on the verified 400 set, using Claude Sonnet. The rest of this post is how we got here: what SpreadsheetBench is, what six months of running the benchmark taught us - and why an agent with a spreadsheet engine beats an agent without one. We're also releasing @grid-is/agent-tools: the tools our spreadsheet agent uses to read, write, and understand workbooks - complementing our free-for-evaluation GRID engine. What is SpreadsheetBench? SpreadsheetBench is a benchmark for evaluating how well LLMs solve real spreadsheet tasks. It was introduced in a 2024 NeurIPS paper by researchers from Renmin University of China, Tsinghua University, and Zhipu.AI (Zeyao Ma, et al.). The dataset holds 912 tasks scraped from real Excel forum threads. Scoring is done by comparing cells: the cells your agent produced against the cells an expert says should be there. In late 2025 the researchers released a 400-task subset in which every task has been reviewed by experts, the "verified 400", in collaboration with Shortcut (see blog). Their objective was to create a subset of tests with fewer correct interpretations, optimized for automated evaluation. By removing overly ambiguous and simple tasks they created, in our opinion, a subset which gives a much cleaner signal, and it's the set all our published numbers refer to. Why we ran it We build a spreadsheet engine. This is a tricky thing to do, because a modern spreadsheet is a combination of decades of features piled on top of one another, none particularly well documented. Unless you spend your days knee-deep in a workbook, you may not have thought about the engine underneath. You just use Excel, and the calculation engine is invisible. When you've built something invisible, it's hard to tell people what you've done. You can say "We've written a spreadsheet engine, from scratch, that runs the calculations itself, manages the dependency graph, handles the awkward edges of Excel semantics, and exposes itself cleanly to code and to language models", and people will nod politely and then ask you if you have any interesting hobbies. So how exactly should we show off our engine? We have demos, we have case studies, standard stuff. And then we discovered SpreadsheetBench. 912 real-world problems, nabbed from Excel forums, scored by comparing cells to cells. The tasks are the sort of thing a colleague sends you at half past four on a Friday: combine these two sheets by matching on text in two columns, oh and by the way there's a totals row, oh and by the way new rows will be added later, so please make it survive that. A great test for an engine. This would help us test our hypothesis: an agent with a spreadsheet engine is better than an agent without. If you ask an LLM to solve a spreadsheet problem, it will often reach for the OpenPyXL Python library. But OpenPyXL is designed to read and write .xlsx files, not to provide a spreadsheet engine. It can't run calculations, it can't run formulas, it can't pivot tables. Instead, the LLM needs to write Python code to calculate values, and at that point a spreadsheet problem becomes a programming problem. It tries to solve the programming problem and hope the translation was lossless. It usually isn't. Meanwhile our engine, sitting quietly on the other side of the room, can just evaluate the formula. It can solve the spreadsheet problem. We think an engine provides a good foundation for LLMs to solve the sort of problems people have in spreadsheets. There was another reason too. We wanted to see our own engine and harness work on problems we hadn't chosen. It is very easy, when you build something, to end up testing it on the things it happens to be good at. SpreadsheetBench would help us get out of our comfort zone and find out if our spreadsheet engine could stand up against unforeseen problems. How we benefitted Our first tests, in December 2025, ran against the online MCP API we were hosting at the time, and they were surprisingly slow. That turned out to be the token-hungry nature of spreadsheets, not the service. So we quickly built an agent backend with the engine running directly inside it: a fuller exercise in what we believed our customers would want to build, and a faster way to iterate. We'd been developing that engine for more than seven years, but we had only just taught it to save modified .xlsx files. Until then it had powered our first product, Calculator Studio, which turned spreadsheets into visual, interactive calculators that read and calculated across more than 200,000 workbooks but never wrote changes back to a file. Durable editing was brand new. The first tests bore that out. They surfaced plenty of weaknesses in our export path (enough, we reckon, to account for something like 10–15% of the failures) and left us around 75% to start, depending on the tasks, model, and prompt. We started from the prompts and tools we'd been using for our LLM showcase. Our tools at the time reflected that history: mostly read-oriented (detecting data labels, understanding workbook structure), with the write path barely out of the oven. Running hundreds of forum-sourced tasks through it, then round-tripping the results back out to .xlsx for scoring, shook out bugs at a rate no test suite we could have written ourselves would have matched. The benchmark quietly became a development tool. It also changed how we run things. Waiting a few hours for a sequential run of hundreds of tasks to surface one regression is no way to live, so we built our own runner: tasks execute in parallel against the agent and are evaluated while the run is still going. Full 400-task runs are our high-fidelity check, exercising the whole export and roundtrip path, but they aren't cheap in tokens, so we don't run them constantly. Instead we used a handful of full runs to rate each task's difficulty, then picked 100 tasks representative of the overall spread and ran that smaller set far more often. It gives a fast, clear signal while we iterate on tools and prompts, and lets us reserve the full 400 for less frequent, higher-fidelity checks. Once the score settled into a reliable 85–90%, the benchmark's job changed. Since February this year we mostly stopped working on the agent itself: our customers needed pivot tables, charts, and other core engine work more than they needed a better benchmark score, so that's where our efforts were directed. The benchmark stayed on as a regression suite, guarding the score while we evolve engine features, tools, and skills the benchmark doesn't measure at all. The numbers Over the next few months these regression runs accumulated the following results: Model Runs Best Median claude-sonnet-5 7 91.25% 90.75% claude-opus-4-8 2 91.0% 90.5% claude-sonnet-4-6 7 90.0% 89.0% claude-sonnet-4-5 6 86.75% 85.6% The trend is really two stories. The big jump from sonnet-4-5 to sonnet-4-6 was mostly our own doing: those runs carry far more advanced GRID tooling and engine capabilities. The improvement since then is split between smaller tooling refinements and the model itself: sonnet-5 sits at 90% or above in six of its seven runs. It's worth noting that we'd probably need more than 6 or 7 runs for a given configuration of model + GRID tools to understand how well we do. A typical run fails 35–45 of the 400 tasks, but most of that is churn, the run-to-run noise we measure below. Just 14 tasks failed in all seven of the recent claude-sonnet-5 runs; put the other way, 96.5% of the verified 400 passed at least once within that window. Widen the window to every run we've recorded and only three tasks have never passed: 99.25% solved at least once, and the remaining three are arguably defective tasks, not lack of capability. A note on error bars, because leaderboards don't usually draw them. Every eval we'd built before this one was single-turn, and single-turn evals at temperature 0 are fairly stable, across both Claude and GPT models. Nothing had prepared us for how stochastic a 20-plus-turn agentic run is: early on, re-running a day's failed tasks lifted the score from 82-something to 85.7%, and a subsequent full re-run erased those gains. So we measured it properly. We ran the same agent build, configuration and model (claude-sonnet-4-6) five times over a fixed 100-task sample: 12 of the 100 tasks flipped between pass and fail across the runs. An earlier four-run group on claude-sonnet-4-5 showed the same: 14 of 100 flipped. Both groups ran with temperature=0 and top_k=1, sampling pinned to the single most likely token at every step, so the flips survive even with sampling nailed down. The take-away: a single-run score on this benchmark carries several points of noise, and comparisons between agents that differ by a point or two are mostly comparisons of luck. We're not the only ones to notice: the GPT for Work team's write-up of their own run reaches the same conclusion, attributing most of their misses to variance rather than a capability gap, with several failing tasks flipping to pass on re-runs. We standardize on Sonnet because that's the model businesses actually deploy at scale. It can be a real matter of economics: Sonnet is about 40% cheaper, on a workload as token-hungry as spreadsheets. A leaderboard doesn't record what a score costs to serve. In our controlled head-to-head with identical agent build on verified 400 and only the model swapped, Opus 4.8 scored one point higher than Sonnet 5 (91.0% vs 90.0%). A second Opus run came back at 90.0%. The noise shows that the choice has negligible effect on score. Reproducibility We're publishing our leaderboard run in full, every transcript and every output file. It lands in the same ~90% band as the rest of our runs; a single number here means little, which is rather the point. A score without its transcripts is just a claim, and we'd like publishing them to become the norm. Dealglass, whose Tetra sits ahead of us on the leaderboard, led the way here: their spreadsheet-agents dataset includes session logs for their baseline GPT-5.4 and Claude Opus 4.6 runs, alongside the eval results for their own agent. Two tasks from a recent run show what having the engine in the loop looks like, and why it matters: 48643 wants a column of accumulated wealth over 39 years: each year's deposit added to the previous year's total, grown by that year's rate. The calculation is inherently recursive, each row depends on the one above it, so there's no closed-form shortcut. Our agent evaluated the candidate formula against the live workbook first, then wrote it to the top cell and filled it down the column in one call. The engine recalculated the whole 39-row chain, and the agent read the computed values back to confirm the compounding behavior. Without an engine the agent would have to reimplement 39 rows of compounding arithmetic in Python and hope the rounding matches, or write the formulas blind. 48982 is a debugging task: a running-counter formula in B323 displays "000" and Excel complains about a circular reference, and the asker wants to know why. The display is a red herring (the cell's number format is literally 000), and the real bug is in the formula, =IF(A323="E","",MAX(B$7:B377)+1). The MAX range includes row 323 itself, so the cell feeds its own calculation. Our agent found this by reading the formula and its format, narrowed the range to B$7:B322, and evaluated the fixed cell in place to confirm that it now computed a value. A circular reference only exists, and only gets caught, when something is actually running the calculation. What the benchmark is, and isn't The good parts we covered up top: forum-sourced tasks with the underspecified shape of real work, plus expert review on the verified 400. It's the best public signal we've found for spreadsheet agent capability. But there are limitations: The reference harness assumes your agent writes Python. It generates code, runs it against the file, opens the result in Excel to recalculate, and then compares cells. That's a sensible pipeline if OpenPyXL is your only option, but unnecessary if your agent has an engine. Ours reads and edits cells directly and calculates natively. Some tasks have more than one valid solution, so getting them reliably right can mean you've overfit your prompts and tools rather than solved anything general. Only three of the 400 tasks have never passed in any of our runs, on any model. Every other task has passed at least once; most single-run failures are just tasks that fail one run and pass the next. We've shared our notes on some of these. Run-to-run volatility (see above) made us decide it's more valuable as a regression and optimization suite than anything else. That's how we use it: as the thing that tells us when an engine change broke the write path, not as a number to chase. There is also plenty a cell-comparison benchmark can't see: formatting and styles aren't scored at all, and comparing cells, or exact ranges, limits the tasks to those with a single, fixed position where the answer can be checked. SpreadsheetBench V2 addresses both, making it more representative of real spreadsheet work. That makes V2 the version of the benchmark we believe in, and the one our effort is aimed at next. Its leaderboard is young (so far it holds only the researchers' own model baselines, with the best at 34.9%), so nobody has really run at it yet. Try it for yourself The tooling layer the agent uses (the tools for reading, writing, and understanding workbooks that all these runs battle-tested) is now available as @grid-is/agent-tools. The engine itself is already available for non-commercial use, and together with our viewer and editor it's what a number of companies already license to build spreadsheet capability into their own products, increasingly for AI-agent workflows. The newest example: a financial-modeling agent one of our clients released on the engine's latest version just weeks ago. We're also strategically open-sourcing parts of the stack, more on that later. And credit is overdue. SpreadsheetBench has given us far more value than a number to put in a blog post: it has been a working development tool for half a year, and will be for the foreseeable future. Everything here is public: The agent tools: docs.grid.is/agent-tools The engine, free for non-commercial use: grid.is Our leaderboard run, transcripts and outputs: github.com/GRID-is/spreadsheetbench-results Install the tools into Claude Code or Codex: GRID skill marketplace An agent with a spreadsheet engine beats an agent without one. Now you have the tools to check.