Reliable Analysis and Charts

Asking a model to analyse a spreadsheet produces confident output. Whether that output is correct depends entirely on a distinction most interfaces do not make obvious.

The distinction that matters

Generating the answer as text. The model predicts what the result looks like. For anything beyond trivial arithmetic this is unreliable — plausible numbers, wrong values, and no indication of which.

Executing code. The model writes a script, runs it, and reports what it computed. The arithmetic is done by an actual computer.

Only the second is trustworthy for numbers.

Find out which your tool does. Many assistants now execute code for data tasks and say so. Some do not. If you cannot tell, ask it to show the code it ran — if there is none, the number was generated.

A workflow that produces checkable results

One: give it the actual file. Uploading a spreadsheet is far better than describing it or pasting a sample.

Two: ask it to describe the data first. Row count, columns, types, missing values, ranges. This catches import problems before any analysis.

Three: ask for the code, not just the result. Even if you do not read code well, its presence tells you the work was executed.

Four: check one figure by hand. Pick a total or an average you can verify in the spreadsheet. If it matches, the pipeline is working. If not, everything downstream is suspect.

Five: ask what it excluded. Rows dropped for missing values, outliers removed, filters applied. Silent exclusions are the most common source of wrong answers that look right.

Six: ask for the analysis a sceptic would run. Where the result might be wrong, what alternative explanation exists, whether the sample supports the claim.

Charts

Ask for the chart type to suit the question, not the data. Comparison across categories, change over time, distribution, relationship between variables — each has appropriate forms and each has common wrong choices.

The recurring errors: a truncated axis exaggerating differences, a pie chart with a dozen segments, a dual axis implying a relationship, categories ordered alphabetically rather than by value, and colour carrying meaning that is invisible to colour-blind readers.

Ask explicitly for an axis starting at zero unless there is a reason otherwise, and say so when there is.

Check the label against the data. Generated charts frequently have plausible labels that do not match what was plotted.

What models are good at here

Cleaning. Inconsistent formats, dates in five styles, whitespace, duplicate detection. Tedious, mechanical, well suited.

Writing the transformation you can describe but not code.

Explaining a statistical method and when it applies.

Suggesting what to look at in unfamiliar data.

Producing the chart code so you can adjust it.

Drafting the written interpretation once the numbers are verified.

What they are bad at

Judging significance. A model will report a difference without a view on whether it means anything, or will assert significance without justification.

Knowing your domain. Which outliers are errors and which are the interesting cases.

Noticing that the data is wrong. A column mislabelled at source produces a confident analysis of the wrong thing.

Causal claims. Models will describe correlations in causal language readily. This is the most consequential error in the category and it appears in output constantly.

Before anyone acts on the result

Verify one number manually.

Ask what was excluded.

Check the chart against the underlying figures.

Ask what would have to be true for this conclusion to be wrong.

Have someone who knows the domain look at it.

Four checks and a second opinion. That is a fraction of the time the analysis saved and it is the difference between a useful tool and a confident wrong number in a report with your name on it.

The reproducibility habit

Anything you will present should be reproducible without the conversation, and this takes one extra request.

Ask for the script rather than the answer. A file you can run, with the data path as a parameter.

Run it yourself and confirm you get the same numbers.

Save it with the output. Six months later, when someone asks how a figure was derived, you have the answer.

Note the data version. Which extract, which date, which filters.

Record what was excluded as a comment in the script rather than as something you remember.

This converts an analysis that lives in a chat history into one that can be checked, corrected and rerun on new data. It costs a minute and it is the difference between a result and an anecdote.