PDF to Markdown in Python: rebuilding headings, lists and paragraphs from font sizes

Disclosure: we built and sell the Apify Actor whose code this post walks through, PDF to Text & Markdown Extractor (publisher WrenTools). The post was written with AI assistance and reviewed by a person before publishing. Every output below was produced by running that code.

In short: a PDF has no headings or paragraphs, only positioned text with font sizes. To get Markdown, take pdfminer.six's text boxes, call the most common size (weighted by characters) the body, and make short blocks at least 1.12 times larger into #, ## and ###, largest first. Then join wrapped lines, turn bullet lines into - items, and escape lines that only look like Markdown. On our three-page test report that gives five headings, a list and two clean paragraphs.

A PDF is a page description, not a document. It says "draw these glyphs at this position in this font at this size", and nothing marks a line as a heading or says where a paragraph ends. If you want Markdown, for a notes app, a static site or a RAG pipeline, you have to infer the structure back from the layout.

Here is what that looks like in Python with pdfminer.six, using the rules from our extractor's core.py.

What does plain text extraction give you?

pdfminer's one-liner on our test report:

from pdfminer.high_level import extract_text
extract_text("report.pdf")

'Quarterly Field Report\n\n1. Summary\nThe survey covered twelve sites over six weeks. Most sites met the target, and two were\nre-measured after the storm in week four.\n\n2. Findings\n\n2.1 Water quality\n...'

The words are all there, but a line break falls in the middle of a sentence, page breaks show up as \x0c form feeds, and nothing tells "2.1 Water quality" apart from the body text. The structure is in the font sizes, so we need the layout objects instead.

1. Read text boxes with their sizes

pdfminer's layout analysis groups characters into LTTextBox objects made of LTTextLines, and each character (LTChar) knows its size and font name. Per line we keep the median size and whether most characters are bold:

from statistics import median
from pdfminer.layout import LTChar

def line_style(line):
    chars = [c for c in line if isinstance(c, LTChar) and c.get_text().strip()]
    if not chars:
        return 0.0, False
    size = median(round(c.size, 1) for c in chars)
    bold = sum("bold" in c.fontname.lower() for c in chars) > len(chars) / 2
    return size, bold

The median keeps one odd glyph, such as a superscript, from moving the line's size.

2. Split a box when its lines change size

pdfminer often puts a heading in the same box as the paragraph right under it. So we walk the lines of each box and start a new block whenever the size changes by 0.6 pt or more, or the weight changes:

current = None
for text, size, bold in styled_lines:
    if current and abs(current.size - size) < 0.6 and current.bold == bold:
        current.lines.append(text)
    else:
        current = Block(page, [text], size, bold)
        blocks.append(current)

3. Find the body size

The body text is whatever most of the characters are set in. Count characters per (rounded) size, not lines or blocks, because one long paragraph outweighs ten short captions:

from collections import Counter

def body_size(blocks):
    weights = Counter()
    for b in blocks:
        weights[round(b.size)] += sum(len(line) for line in b.lines)
    return float(weights.most_common(1)[0][0]) if weights else 0.0

On the test report that gives 11 pt.

4. Map larger sizes to heading levels

Every size at least 1.12 times the body is a heading size. The largest becomes #, the next ##, and everything smaller than that ###:

HEADING_RATIO = 1.12

def heading_levels(blocks):
    body = body_size(blocks)
    sizes = sorted({round(b.size) for b in blocks if body and b.size >= body * HEADING_RATIO}, reverse=True)
    return {size: min(i + 1, 3) for i, size in enumerate(sizes)}

The report has headings at 22, 15 and 12.5 pt, so the result is {22: 1, 15: 2, 12: 3}. The 12.5 pt subheadings are only 1.14 times the body, so a threshold much above 1.12 would have missed them. A block also has to be short to count, at most 3 lines and 200 characters, so a paragraph set slightly large (a pull quote, an abstract) stays a paragraph.

5. Join wrapped lines, and rejoin hyphenated words

Inside a paragraph block, lines are joined with a space. A line that ends in - followed by one that starts with a lowercase letter is treated as a word broken across the line:

def join(lines):
    out = ""
    for line in lines:
        if not out:
            out = line
        elif out.endswith("-") and line[:1].islower():
            out = out[:-1] + line
        else:
            out += " " + line
    return out

["the extraction was effi-", "cient"] becomes "the extraction was efficient". The rule has limits. A real hyphenated word that happens to break at its hyphen loses it: ["two sites were re-", "measured"] becomes "two sites were remeasured". And ["the Covid-", "19 wave"] becomes "the Covid- 19 wave", because the digit isn't lowercase. Fixing those needs a dictionary. We decided a rare wrong join was better than leaving every broken word split.

6. Turn bullet lines into list items

A line that starts with a bullet character and a space becomes a - item. A following line that doesn't start with a bullet is a wrapped continuation of that item:

import re
BULLET = re.compile(r"^\s*[•●▪◦‣⁃–\-\*]\s+")

7. Escape what only looks like Markdown

A body paragraph that starts with 1. or # would turn into a list or a heading once rendered. So the start of each paragraph is escaped:

def escape_md(text):
    return re.sub(r"^(#{1,6}\s|>|\d+\.\s)", r"\\\1", text)

"1. Summary of costs" becomes \1. Summary of costs, and "# of sites" becomes \# of sites.

The result

The same test report, through all seven rules:

# Quarterly Field Report

## 1. Summary

The survey covered twelve sites over six weeks. Most sites met the target, and two were re-measured after the storm in week four.

## 2. Findings

### 2.1 Water quality

Turbidity fell at nine of twelve sites.

- Site A: within limits
- Site B: within limits
- Site C: above limits

### 2.2 Equipment

Two probes were replaced. The spare kit is now stored at the north depot so field crews can reach it within an hour.

"2.2 Equipment" sits on page 2, and it is still a ###, because the levels are computed over the whole document rather than page by page.

What this approach doesn't do

• Tables and images are not rebuilt. A table comes out as its cell text in reading order.
• Scanned pages have no text layer, so there is nothing to read. Page 3 of the test report is blank, and the extractor lists it as a page without text instead of returning an empty string as if it had succeeded. For scans you need OCR (we use Tesseract in a separate OCR Actor).
• Protected PDFs. pdfminer will extract text from a PDF whose owner has disabled copying unless you check first. We check PDFDocument.is_extractable and refuse those files, and we refuse password-protected ones too.

Try it

The full code is about 280 lines of core.py. It also splits the Markdown into overlapping chunks for RAG, each tagged with its start and end page. If you'd rather call it than run it, the GitHub example sends a list of PDF links to the Actor and saves Markdown files and chunks. It costs $0.002 a PDF plus $0.0001 a page.

If you've found a better signal than font size for headings, such as weight, spacing or the PDF's own tags, I'd like to hear it in the comments.