32 How a Chart Lies
Chapters 25 and 26 were about charts that are badly chosen. This one is about charts that are well made and still untrue.
That distinction matters, because the usual advice on the subject is useless. “Don’t lie with statistics” is not a technique. Nor is the familiar gallery of fraudulent charts with missing axes and invented numbers, because almost nobody in a professional setting produces those, and anybody would catch them.
The charts that mislead in practice have every number correct. The axes are labelled, the data is real, nothing is omitted, and a careful reader checking the figures against the source would find no discrepancy at all. They mislead through choices that are not usually thought of as claims: which axis range, how tall the panel, which groups are pooled, which colour ramp. Each of those is a decision the software will make for you if you do not make it, and each of them can change what the picture says without changing anything a fact-checker would look at.
Almost every case in this chapter was produced without anyone intending to deceive. That is what makes them worth a chapter.
32.1 A Second Axis
The dual-axis chart plots two series with different units against a shared horizontal axis and two different vertical ones, usually with one scale on the left and the other on the right. It is offered by every charting tool and is the single most reliable way to make an audience see a relationship.
The problem is not that the scales differ. It is that the reader is invited to compare the two lines, and no comparison between them is defined.
Three things a reader takes from a dual-axis chart, none of which it can support:
Which line is higher. Where the lines sit relative to each other is set entirely by the two axis ranges. Slide one and the lines swap.
Where they cross. “Costs overtook revenue in the third quarter” is a sentence people say about dual-axis charts. The crossing point is an artifact of scaling and can be placed in any quarter you like, or removed.
Which is moving faster. The apparent steepness of each line is its slope divided by its own axis range. Change the range and the same series becomes a cliff or a plateau.
And the one thing that does not change when you move the axes is the actual statistic. The correlation is invariant to the scaling and the picture is not, which is the gap the chart lives in.
Example
Twenty-four quarters of marketing spend and customer complaints. Spend rises about 3.9 a quarter; complaints fall about 2.0 a quarter; the correlation between them is \(-0.83\).
Now draw it three times, changing only the range of the right-hand axis.
| Right axis runs | What the picture says | Apparent slope ratio | Apparent direction |
|---|---|---|---|
| 0 to 1400 | Complaints are flat and irrelevant | 0.05 | Opposite |
| 250 to 320 | Complaints are falling away as spend climbs | 0.93 | Opposite |
| 320 to 250 | Complaints track spend almost point for point | 0.93 | Same way |
The true ratio of the two slopes is 0.50 in every panel. As drawn it ranges from 0.05 to 0.93, a factor of twenty.
The third row is the one worth sitting with. By putting the top of the right-hand axis below its bottom, which is a single number in a configuration dialog, the two series appear to rise and fall together. A reader would report a strong positive relationship. The correlation is \(-0.83\).
Three honest alternatives, in order of how often they work.
Two panels, stacked, sharing the horizontal axis. Each series gets its own vertical scale, which is exactly what the dual axis was for, and no spurious crossing is created because the lines are never on top of each other. This is Chapter 25’s second channel, position on non-aligned scales, and it costs almost nothing.
Index both to a common base. If the question is really about relative movement, set both series to 100 at the starting period and plot them on one axis. Now the comparison is defined, the units are the same, and “grew faster” is a statement the chart supports.
Plot one against the other. If the question is whether they are related, the scatter plot answers it directly, and it is the chart that actually corresponds to the correlation.
A dual axis is defensible in one case: when the two scales are exactly convertible, such as Celsius and Fahrenheit, or a count and that same count as a percentage of a fixed total. There the second axis is a relabelling of the first and asserts nothing new.
32.2 The Shape of the Box
The aspect ratio of a time series plot is usually set by whatever space is left on the slide. It decides what the reader can see.
The underlying fact is that the eye judges changes of slope well when slopes are near 45 degrees and badly when they are near 0 or 90. A line squashed flat has every segment at a shallow angle and they all look alike; a line stretched tall has every segment near vertical and they all look alike again. In between there is a range where differences in rate are obvious.
The rule that follows is due to Cleveland and is called banking to 45 degrees: choose the aspect ratio so that the mean absolute angle of the line segments is about 45 degrees. For a smooth series that means a tall panel; for a jagged one it can mean a panel many times wider than it is high.
This is not a matter of taste and it is easy to compute. For a series drawn in a panel \(w\) by \(h\), with the data spanning \(\Delta x\) horizontally and \(\Delta y\) vertically, the angle of the segment from \(i\) to \(i+1\) is
\[\theta_i = \arctan\!\left(\frac{(y_{i+1} - y_i)\,h/\Delta y}{w/\Delta x}\right)\]
and the quantity to aim at is the mean of \(|\theta_i|\).
Example
Ninety-eight weeks of demand, following a cycle that builds slowly and drops sharply. Demand rises in 64 of the weeks with an average rise of 4.4, and falls in 33 with an average fall of 8.7. The falls are 1.9 times as steep as the rises, and that asymmetry is the only interesting thing in the series: it says the business loses in three weeks what it took ten to build.
| Panel drawn at | Mean segment angle | What a reader sees |
|---|---|---|
| 13.7 to 1 | 38 degrees | A slow build and a sharp drop |
| 6.4 to 1 | 55 degrees | The same, slightly compressed |
| 2.1 to 1 | 74 degrees | A row of symmetric spikes |
The 1.9 is in the data at every aspect ratio. Whether anybody notices it is decided by the height of the box, which is a layout decision nobody records and nobody reports in the caption.
Note that the failure runs both ways. A panel too tall hides the asymmetry just as effectively as one too short, and the too-tall version is the one that looks more impressive on a slide.
32.3 Which Groups You Pool
Chapter 10 introduced Simpson’s paradox as a fact about conditional probability, and Chapter 21 met it again as a warning about correlation. It has a third life as a way of drawing a chart, and this is the form in which it does the most damage, because the chart looks like raw data rather than an analysis.
Every scatter plot of pooled groups contains a decision: to plot the groups together. That decision is invisible in the result. Nothing on the chart says “these are two populations superimposed”, and the regression line drawn through them describes neither.
Example
One hundred and twenty branches, sixty of each of two store formats. For each, weekly staff hours and average minutes to serve a customer.
| Group | Slope, minutes per staff hour |
|---|---|
| Compact stores | -0.086 |
| Superstores | -0.152 |
| All 120 pooled | +0.144 |
Within either format, more staff hours go with faster service, which is what anybody would predict and what an operations manager would act on. Pooled, the slope is positive and the chart says that adding staff makes service slower.
Nothing was filtered. Every one of the 120 points appears in both versions. The reversal comes entirely from the fact that superstores both employ more people and take longer to serve, for reasons that have nothing to do with the relationship inside either group.
The tell is visible once you know to look: the pooled scatter shows two separate clouds, not one. A relationship fitted across a gap between clusters is describing the gap, not the relationship.
32.4 Colour That Invents Structure
Chapter 25 established that hue carries no order. The consequence in practice is the rainbow colour map, which is still the default in a good deal of scientific and engineering software, and which lies in a specific and measurable way.
Perceived lightness along a colour ramp can be computed. In the CIE Lab system, \(L^*\) is the lightness coordinate, running from 0 to 100, and it is designed so that equal steps look equal. A ramp suitable for a quantity should have \(L^*\) rising monotonically from one end to the other, so that the order of the data is the order of the ink.
Rainbow ramps do not. Their lightness rises and falls several times, so a reader looking at a smooth field sees bands: each local maximum of lightness reads as an edge, and each region where lightness is flat reads as uniform even where the data is changing.
Example
Lightness measured along several ramps, sampled at 64 points, counting how many times the direction of lightness reverses.
| Ramp | Kind | Direction changes | Lightness span |
|---|---|---|---|
R’s rainbow()
|
Rainbow | 5 | 64 |
R’s topo.colors()
|
Rainbow | 3 | 65 |
matplotlib jet
|
Rainbow | 1 | 83 |
matplotlib hsv
|
Rainbow | 5 | 64 |
matplotlib turbo
|
Improved rainbow | 1 | 79 |
viridis |
Sequential | 0 | 76 |
Blues |
Sequential | 0 | 77 |
Greys |
Sequential | 0 | 100 |
Every rainbow-family ramp reverses. Every sequential one does not, and note that they also have a wider usable lightness range, so the supposed gain in vividness costs contrast as well as order.
The code below renders a perfectly smooth field, containing no edges of any kind, in a rainbow ramp and a sequential one. The rainbow version shows crisp concentric rings. There is nothing in the data at those rings.
Rainbow ramps have a second failure that matters more in some settings than the first: they are unreadable in greyscale and unreliable under colour vision deficiency, because two colours can differ in hue while matching in lightness. Print the figure, or hand it to the one reader in twelve who cannot separate red from green, and the structure disappears entirely.
The fix is not a matter of taste and takes one word in the code. Use viridis, cividis, or any perceptually uniform sequential ramp for a quantity; a diverging ramp when there is a meaningful midpoint; and a rainbow never, unless the variable being shown genuinely is hue, such as a wavelength.
32.5 Truncation, Revisited
Chapter 25 derived the rule rather than asserting it: length encodes ratio, so a bar’s origin is part of its claim; position encodes order and difference, so a dot plot’s axis may be cropped as tightly as the data deserves.
That leaves a question Chapter 25 did not answer. When is truncation not merely permitted but required?
Whenever zero is not a possible value, or is so far outside the data that including it destroys the resolution the reader needs. Body temperature charted from zero is an absurdity: the interesting range is 36 to 40 degrees and a patient at zero is not a clinical concern. Share of a two-party vote, blood pressure, pH, credit scores, and any index defined to sit near 100 are all in the same position.
The general statement is that the axis should cover the range over which the quantity is meaningfully interpretable, which is a question about the subject matter and not about charting. Zero earns its place on the axis when zero is a real and relevant possibility, and not otherwise.
So the honest version of the rule is not “always start at zero” but:
| If the mark is a | It encodes | Then the axis |
|---|---|---|
| Bar, column, filled area | Ratio | Must include zero |
| Dot, point, line, tick | Order and difference | May start anywhere the subject justifies |
And the accompanying obligation, which applies to both rows: if the axis does not start at zero, make that unmistakable. Do not rely on the reader inspecting the tick labels, because most will not.
32.6 Charts With No Uncertainty On Them
Six modules of this book have been about the fact that an estimate is not a value: it comes with a standard error, an interval, and a sampling distribution. Then it gets drawn as a bar, and all of that disappears.
A bar chart of group means is the commonest chart in business and one of the most misleading things in this chapter, because a bar has an edge and the edge looks like a fact. Two bars of visibly different height read as two different numbers. Whether the difference would survive another sample is not on the chart at all, and Chapter 17 established that it very often would not.
The remedy is not merely to add error bars. Error bars are themselves ambiguous, because three different quantities are drawn identically:
| Bar shows | Meaning | Roughly |
|---|---|---|
| Standard deviation | Spread of the observations | Does not shrink with \(n\) |
| Standard error | Spread of the sample mean | Shrinks as \(1/\sqrt{n}\) |
| Confidence interval | Plausible range for the true mean | About 2 standard errors at 95% |
These differ by a factor of \(\sqrt{n}\) and then by about 2, so at \(n = 100\) the standard deviation bar is roughly twenty times the length of the standard error bar over the same data. An error bar with no caption saying which of the three it is conveys nothing, and a reader is entitled to assume the most flattering interpretation was chosen.
Two habits fix most of it. Say in the caption exactly what the bars are. And where \(n\) is small enough, draw the observations rather than a bar with a whisker, because twelve points with a mean line tell the reader everything the summary would have, plus the shape.
32.7 A Checklist for Reading One
The cases in this chapter have a family resemblance, and most of them can be caught by asking six questions of any chart, including your own.
Where does each axis start and end, and who chose that? If an axis is truncated, is the mark a length or a position?
Is there a second vertical axis? If so, nothing about the relative position, crossing or steepness of the two series means anything.
What is the aspect ratio doing? Does the finding survive redrawing it twice as tall and half as tall?
What groups are pooled, and would splitting them change the sign? A scatter showing two clusters is a warning, not a dataset.
Does the colour ramp have an order, and is that order in the data?
Where is the uncertainty? If the chart shows estimates with no indication of how firm they are, the differences on it may not be differences.
None of these requires access to the underlying data, which is the point: they are all answerable from the picture. Two of them, the aspect ratio and the pooling, are the ones most often missed, because neither looks like a decision.
One closing observation, which is uncomfortable and worth stating anyway.
Every example in this chapter was produced by choosing a default. Nobody deleted a data point, invented a number or mislabelled an axis. The dual axis came free with the chart type; the aspect ratio came from the size of the slide; the pooling came from having one spreadsheet rather than two; the rainbow came from the software’s default palette, and for about fifteen years that default was jet.
The most common way to mislead with a chart is to accept what the tool gives you. The corresponding responsibility is not to be more honest than other people. It is to treat the defaults as claims, and check them.
Three outputs, three ways of being accurate and untrue. A correlation of \(-0.83\) drawn so that the two series appear to rise together, achieved by typing a larger number in the box marked “axis minimum”. An asymmetry of 1.9 to 1 made invisible by the height of a panel. And crisp rings appearing in a field that is smooth everywhere, because the default palette’s lightness reverses five times.
Looking Ahead
Every chart in this chapter is accurate. No number was altered, no observation removed, no axis left unlabelled, and a reader checking the figures against the source would find nothing wrong. What each of them does is invite an inference the data does not support, and in every case the invitation came from a setting somebody accepted rather than chose. A second vertical axis makes the relative height, the crossing point and the relative steepness of two series into artifacts, so that a correlation of \(-0.83\) can be drawn as two lines rising together. The height of a panel decides whether a 1.9 to 1 asymmetry is the obvious feature of a series or invisible in it. Pooling two store formats reverses the sign of a relationship that is negative inside both of them. A rainbow ramp puts sharp rings into a field that is smooth everywhere, because its lightness reverses five times on the way from one end to the other. And a bar chart with no uncertainty on it turns estimates into facts by giving them edges. The defence is not to be more scrupulous than other people. It is the six questions above, asked of your own work before anybody else asks them, and the habit of treating a default as a claim. What remains is the other half of the job. This chapter is about not misleading a reader; it says nothing about actually reaching one. A chart can be scrupulously honest and still fail, because the reader cannot tell which line is theirs, cannot find the point in it, or never works out what they were supposed to conclude. The last chapter of this module is about the reader who will never get to ask you what you meant: direct labelling instead of legends, the title as the finding rather than the subject, ordering by value, colour that survives a photocopier, and showing uncertainty without burying the estimate.
Summary
| Concept | Description |
|---|---|
| The Problem | |
| Accurate and Untrue | Every number correct, every label right, and the inference invited is wrong |
| The Deception Is in the Defaults | Nearly every case here came from accepting a setting rather than choosing it |
| A Second Axis | |
| The Dual Axis | Two series, two vertical scales, and the most reliable way to suggest a link |
| No Defined Comparison | The reader compares the two lines, and no comparison between them exists |
| Which Line Is Higher | Set entirely by the two axis ranges; slide one and they swap |
| Where They Cross | An artifact that can be placed in any period you like, or removed |
| Which Is Moving Faster | Apparent steepness is the slope divided by that series' own axis range |
| The Correlation Is Invariant | It does not depend on the axes, so the picture can move while it does not |
| Reversing an Axis | Putting the top below the bottom makes a negative relation look positive |
| Two Stacked Panels | Same horizontal axis, separate vertical ones, no spurious crossing |
| Indexing to a Base | Both series to 100 at the start, one axis, a comparison that is defined |
| When a Second Axis Is Honest | Only when the two scales are exact conversions, such as Celsius and Fahrenheit |
| The Shape of the Box | |
| Aspect Ratio Is a Decision | Usually made by the space left on the slide, and it decides what is visible |
| Banking to 45 Degrees | Choose the aspect so the mean absolute segment angle is about 45 degrees |
| The Angle Formula | Arctangent of the vertical step times h over dy, divided by w over dx |
| Too Tall Hides It Too | A panel too tall flattens the differences as surely as one too short |
| Pooling | |
| Pooling as a Choice | Plotting groups together is a decision that leaves no trace on the chart |
| Simpson's Paradox in a Scatter | Slope negative inside both store formats, positive across all 120 branches |
| Two Clouds, One Line | A regression fitted across a gap between clusters describes the gap |
| The Tell | The pooled scatter shows two separate clouds rather than one |
| Colour | |
| Lightness and CIE L star | The lightness coordinate, 0 to 100, built so equal steps look equal |
| Monotone Lightness | What a ramp for a quantity needs, so the order of data is the order of ink |
| Rainbow Ramps Reverse | Every rainbow-family ramp tested changes lightness direction at least once |
| Bands That Are Not in the Data | Each local maximum of lightness reads as an edge in a smooth field |
| Sequential Ramps Have More Range | Viridis, Blues and Greys all span more lightness than the rainbows do |
| Greyscale and Colour Blindness | Two colours can differ in hue and match in lightness, so structure vanishes |
| Truncation | |
| When Zero Is Required | When the mark is a length, because length encodes a ratio |
| When Zero Is Absurd | Body temperature, pH, credit scores, any index defined to sit near 100 |
| The Honest Version of the Rule | Bars must include zero; dots and lines may start where the subject justifies |
| Say So Unmistakably | Do not rely on the reader inspecting the tick labels, because most will not |
| Uncertainty | |
| Bars Give Estimates Edges | A bar has an edge and an edge looks like a fact, whatever the standard error |
| Three Kinds of Error Bar | Standard deviation, standard error and confidence interval, drawn identically |
| A Factor of Root n | At n of 100 the SD bar is roughly twenty times the standard error bar |
| Draw the Observations | At small n, points with a mean line beat a bar with a whisker |
| Reading One | |
| The Six Questions | Axes, second axis, aspect, pooling, colour order, uncertainty |
| Treat a Default as a Claim | The corresponding responsibility, and all six are answerable from the picture |