# PDF to Markdown in Python: rebuilding headings, lists and paragraphs from font sizes

**Disclosure:** we built and sell the Apify Actor whose code this post walks through, [PDF to Text & Markdown Extractor](https://apify.com/wrentools/pdf-text-markdown-extractor) (publisher WrenTools). The post was written with AI assistance and reviewed by a person before publishing. Every output below was produced by running that code.

**In short:** a PDF has no headings or paragraphs, only positioned text with font sizes. To get Markdown, take pdfminer.six's text boxes, call the most common size (weighted by characters) the body, and make short blocks at least 1.12 times larger into `#`, `##` and `###`, largest first. Then join wrapped lines, turn bullet lines into `- ` items, and escape lines that only look like Markdown. On our three-page test report that gives five headings, a list and two clean paragraphs.

A PDF is a page description, not a document. It says "draw these glyphs at this position in this font at this size", and nothing marks a line as a heading or says where a paragraph ends. If you want Markdown, for a notes app, a static site or a RAG pipeline, you have to infer the structure back from the layout.

Here is what that looks like in Python with [pdfminer.six](https://github.com/pdfminer/pdfminer.six), using the rules from our extractor's `core.py`.

## What does plain text extraction give you?

pdfminer's one-liner on our test report:

```python
from pdfminer.high_level import extract_text
extract_text("report.pdf")
```

```text
'Quarterly Field Report\n\n1. Summary\nThe survey covered twelve sites over six weeks. Most sites met the target, and two were\nre-measured after the storm in week four.\n\n2. Findings\n\n2.1 Water quality\n...'
```

The words are all there, but a line break falls in the middle of a sentence, page breaks show up as `\x0c` form feeds, and nothing tells "2.1 Water quality" apart from the body text. The structure is in the font sizes, so we need the layout objects instead.

## 1. Read text boxes with their sizes

pdfminer's layout analysis groups characters into `LTTextBox` objects made of `LTTextLine`s, and each character (`LTChar`) knows its size and font name. Per line we keep the median size and whether most characters are bold:

```python
from statistics import median
from pdfminer.layout import LTChar

def line_style(line):
    chars = [c for c in line if isinstance(c, LTChar) and c.get_text().strip()]
    if not chars:
        return 0.0, False
    size = median(round(c.size, 1) for c in chars)
    bold = sum("bold" in c.fontname.lower() for c in chars) > len(chars) / 2
    return size, bold
```

The median keeps one odd glyph, such as a superscript, from moving the line's size.

## 2. Split a box when its lines change size

pdfminer often puts a heading in the same box as the paragraph right under it. So we walk the lines of each box and start a new block whenever the size changes by 0.6 pt or more, or the weight changes:

```python
current = None
for text, size, bold in styled_lines:
    if current and abs(current.size - size) < 0.6 and current.bold == bold:
        current.lines.append(text)
    else:
        current = Block(page, [text], size, bold)
        blocks.append(current)
```

## 3. Find the body size

The body text is whatever most of the characters are set in. Count characters per (rounded) size, not lines or blocks, because one long paragraph outweighs ten short captions:

```python
from collections import Counter

def body_size(blocks):
    weights = Counter()
    for b in blocks:
        weights[round(b.size)] += sum(len(line) for line in b.lines)
    return float(weights.most_common(1)[0][0]) if weights else 0.0
```

**Predict:** the test report sets its text at four sizes: 22, 15, 12.5 and 11 pt. Which one will `body_size` pick as the body, and which of the others will end up as `#`, `##` and `###`? Make a guess, then check it against the figure.

![Bar chart of the characters set in each font size of the three-page test report. 11 pt holds 350 of the 423 characters, in 4 blocks, so it is the body. The heading threshold is 11 x 1.12 = 12.32 pt. Above it, 22 pt (22 characters, 1 block) becomes #, 15 pt (21 characters, 2 blocks) becomes ##, and 12.5 pt (30 characters, 2 blocks, counted as 12) becomes ###: five headings, including page 2's 2.2 Equipment.](https://cdn.hashnode.com/uploads/gql/6ad91735786cf66498f3c2f4/1a8c0b5e-d6b2-4877-9fe4-aafce34245f8.png)

On the test report that gives 11 pt.

## 4. Map larger sizes to heading levels

Every size at least 1.12 times the body is a heading size. The largest becomes `#`, the next `##`, and everything smaller than that `###`:

```python
HEADING_RATIO = 1.12

def heading_levels(blocks):
    body = body_size(blocks)
    sizes = sorted({round(b.size) for b in blocks if body and b.size >= body * HEADING_RATIO}, reverse=True)
    return {size: min(i + 1, 3) for i, size in enumerate(sizes)}
```

The report has headings at 22, 15 and 12.5 pt, so the result is `{22: 1, 15: 2, 12: 3}`. The 12.5 pt subheadings are only 1.14 times the body, so a threshold much above 1.12 would have missed them. A block also has to be short to count, at most 3 lines and 200 characters, so a paragraph set slightly large (a pull quote, an abstract) stays a paragraph, but only while pdfminer keeps its lines in one box. In our test, a 4-line 14 pt paragraph with lines 1.2 times its size apart stayed one paragraph; spaced twice its size apart, each line became a box of its own and a `##` heading.

## 5. Join wrapped lines, and rejoin hyphenated words

Inside a paragraph block, lines are joined with a space. A line that ends in `-` followed by one that starts with a lowercase letter is treated as a word broken across the line:

```python
def join(lines):
    out = ""
    for line in lines:
        if not out:
            out = line
        elif out.endswith("-") and line[:1].islower():
            out = out[:-1] + line
        else:
            out += " " + line
    return out
```

`["the extraction was effi-", "cient"]` becomes "the extraction was efficient". The rule has limits. A real hyphenated word that happens to break at its hyphen loses it: `["two sites were re-", "measured"]` becomes "two sites were remeasured". And `["the Covid-", "19 wave"]` becomes "the Covid- 19 wave", because the digit isn't lowercase. Fixing those needs a dictionary. We decided a rare wrong join was better than leaving every broken word split.

## 6. Turn bullet lines into list items

A line that starts with a bullet character and a space becomes a `- ` item. A following line that doesn't start with a bullet is a wrapped continuation of that item:

```python
import re
BULLET = re.compile(r"^\s*[•●▪◦‣⁃–\-\*]\s+")
```

## 7. Escape what only looks like Markdown

A body paragraph that starts with `1. ` or `# ` would turn into a list or a heading once rendered. So the start of each paragraph is escaped:

```python
def escape_md(text):
    return re.sub(r"^(#{1,6}\s|>|\d+\.\s)", r"\\\1", text)
```

"1. Summary of costs" becomes `\1. Summary of costs`, and "# of sites" becomes `\# of sites`.

## The result

The same test report, through all seven rules:

```markdown
# Quarterly Field Report

## 1. Summary

The survey covered twelve sites over six weeks. Most sites met the target, and two were re-measured after the storm in week four.

## 2. Findings

### 2.1 Water quality

Turbidity fell at nine of twelve sites.

- Site A: within limits
- Site B: within limits
- Site C: above limits

### 2.2 Equipment

Two probes were replaced. The spare kit is now stored at the north depot so field crews can reach it within an hour.
```

"2.2 Equipment" sits on page 2, and it is still a `###`, because the levels are computed over the whole document rather than page by page.

## What this approach doesn't do

- **Tables and images** are not rebuilt. A table comes out as its cell text in reading order.
- **Scanned pages** have no text layer, so there is nothing to read. Page 3 of the test report is blank, and the extractor lists it as a page without text instead of returning an empty string as if it had succeeded. For scans you need OCR (we use Tesseract in a separate [OCR Actor](https://apify.com/wrentools/scanned-pdf-image-ocr)).
- **Protected PDFs.** pdfminer will extract text from a PDF whose owner has disabled copying unless you check first. We check `PDFDocument.is_extractable` and refuse those files, and we refuse password-protected ones too.

## Try it

The full code is about 280 lines of `core.py`. It also splits the Markdown into overlapping chunks for RAG, each tagged with its start and end page. If you'd rather call it than run it, the [GitHub example](https://github.com/WrenTools/apify-actor-examples/tree/main/pdf-to-markdown-and-rag-chunks) sends a list of PDF links to the Actor and saves Markdown files and chunks. It costs $0.002 a PDF plus $0.0001 a page.

If you've found a better signal than font size for headings, such as weight, spacing or the PDF's own tags, I'd like to hear it in the comments.