pdfmarks

Read the data back out of a vector figure in a PDF.

A scatter plot in a paper looks like a picture of the data, and if you want the numbers the usual answer is to email the authors or trace the dots by hand. But a vector PDF does not contain a picture. It contains a list of instructions for placing marks, and the coordinate in each instruction is where the measurement was. The numbers never left. They are sitting in the file at full precision.

pdfmarks walks a page's drawing instructions and hands you every mark on it.

pip install pdfmarks

from pdfmarks import marks, affine

ms = marks('paper.pdf', 4)                       # 0-based page index

# a series is a colour and a marker shape. matplotlib draws a circle with 26
# path points, MATLAB with 15; a triangle is 5, a square or diamond is 4.
series = [m for m in ms
          if m['fill'] == (0.121569, 0.466667, 0.705882) and m['npts'] == 26]
series.sort(key=lambda m: m['seq'])              # drawing order = data order

# calibrate from two marks whose data values you know -- axis tick stubs are
# ideal, they are short two-point strokes on the frame.
fx, fy = affine((225.28, 292.21), (0.0, 0.0),
                (410.30, 427.67), (1.0, 1.0))
data = [(fx(m['x']), fy(m['y'])) for m in series]

Each mark is a dict: x, y (centre, PDF user space, points, y upwards), w, h (bounding box), fill, stroke, npts, op (the paint operator, or Do for a placed image), seq (position in drawing order), and pts (every vertex, so a line series comes back whole).

There is a command line too:

pdfmarks paper.pdf 5 --colours          # what series are on page 5
pdfmarks paper.pdf 5 --long 40          # paths long enough to be a line series
pdfmarks paper.pdf 5 --path 27          # every vertex of that path
pdfmarks paper.pdf 5 --csv marks.csv --max-size 12

How exact is it

Exact. It reads numbers, not pixels.

The test suite downloads two real papers and checks two things whose answers exist independently of this code. In one (a MATLAB figure) two panels carry copies of a third panel's markers; matched in drawing order the recovered coordinates agree to 0.0012 pt. In the other (matplotlib) three series are plotted against a parameter running 0 to 1, and calibrated from nothing but the tick stubs the recovered values land on k/24 — the authors' linspace, which the reader is never told about — to 3.3 × 10⁻⁵.

What it does not do

It does not find your axes. Every plotting library writes them differently, and a wrong guess produces numbers that are confidently wrong to four decimal places, which is worse than no numbers. You supply two reference points and affine does the rest. Tick stubs are easy to spot: two-point strokes, a few points long, sitting on the frame.

It also cannot help with a raster figure. If the plot was saved as PNG and pasted in, there are no instructions to read and you want a tracing tool instead.

Three things worth knowing

Series separate by shape as well as colour. Two panels that reuse one colour still come apart if their markers differ, because npts differs. Run --colours and check the counts against what the figure should contain; if a series is short, something is merging.

Drawing order is usually data order, and often time order. Trajectories come out in the order they were measured. It is also how you tell a legend swatch from data: the swatch is the same marker in the same colour, drawn in a separate run.

A filled marker is often drawn twice, once filled and once as a stroked outline at the same centre. De-duplicate by adjacency in the stream, never by proximity in the plane: in a dense trajectory, neighbouring real data points sit closer together than a marker is wide, and a proximity rule quietly eats them.

Why it exists

It came out of wanting the numbers behind one figure — twelve lab earthquakes published as a scatter plot with no data file. Reading them back out took an afternoon and turned three of the paper's qualitative sentences into measurements. It also caught something invisible: the copies of one panel's data laid into the other two had been translated rather than replotted, so they sat 1.4 % too far right. Harmless, and the sort of thing only the instructions know.

MIT licensed. Written by Iris.