pdf-image-compress

Compress "image-only" PDFs by re-encoding each page image as JPEG and rebuilding the PDF, and export such PDFs to per-page image folders. Designed for scanned documents, comics, art books, photo collections where each page is a single embedded RGB image.

When to use

• Source PDF has no text layer (or text can be discarded) and pages are dominated by 1 large embedded image
• Original encoding is FlateDecode / unencoded (single stream per page, 5-15 MB per page)
• User wants the file under a hard size limit (typical: 30 MB / 100 MB email attachment)

If the PDF has searchable text, vector content, or mixed scans, this skill will degrade those features. It is not for OCR'd searchable PDFs.

Layered / stacked pages (different beast)

Some PDFs (typical: exported from Illustrator/InDesign) stack multiple images per
page — e.g. a 72 DPI RGB preview underneath a 300 DPI CMYK original, plus
detail crops like stamp close-ups. Detect it early:

for im in page.get_images(full=True):
    for r in page.get_image_rects(im[0]):
        print(im[0], im[2], 'x', im[3], '->', r, 'eff DPI =', im[2]/(r.width/72))

If a page shows several images at very different effective DPIs, do not naively
re-encode each image (you would keep a useless 72 DPI duplicate) — and do not
assume one image per page. Instead flatten: render the composite at the page's
true available resolution and rebuild:

python scripts/flatten_layered_pdf.py SRC OUT [CAP_DPI] [Q]

Per page it takes min(CAP_DPI, max effective DPI across layers) — pages with a
300 DPI layer get 250, pages that only carry a 72 DPI base stay at 72 (upscaling
adds zero detail, just bytes). Guard rail: the script refuses to run if any page
has text, vector paths, or fonts — flattening would rasterize them.

Illustrator files are often not what they claim: check get_drawings() and
get_fonts() before treating one as vector. A "designed" PDF whose pages are
scanned contracts will show zero paths and zero fonts.

Real run: 6-page Illustrator-exported scanned contract, 4 pages carrying a
75-megapixel 4-channel duplicate layer — 361 MB → 16.8 MB (4.6%), stamps and
small print visually identical.

Task routing

Merge PDFs

Use scripts/merge_pdfs.py:

python scripts/merge_pdfs.py OUT.pdf SRC1.pdf SRC2.pdf ...   # 按命令行顺序
python scripts/merge_pdfs.py --auto OUT.pdf SRC_DIR          # 按自然页码顺序

insert_pdf() copies object streams verbatim — no re-encoding, no quality loss.
Verify this actually happened: output size should be within ~3% of the sum of
inputs. If it comes out much smaller, something re-encoded the images and you lost
a generation of quality. The script prints this ratio and warns below 97%.

--auto strips a -压缩 suffix and sorts by the first number in the filename, so
19-42 lands before 114-137, and unnunnumbered names like 封面-18 sort first.

Good defaults:
- Add a bookmark per source file (set_toc) so readers can jump between volumes
- Preserve original page dimensions — do not normalise page boxes, or pages
from different scans will get letterboxed

Real run: 9 books × 24 pages → 216 pages, 241.19 MB in → 241.18 MB out (100.0%),
9 bookmarks, 0 s.

Export pages to images

Use scripts/export_images.py:

python scripts/export_images.py SRC_DIR [OUT_DIR_NAME]
#   OUT_DIR_NAME 默认 "图片"
# 输出: <SRC_DIR>/图片/<书名>/001.jpg ...

Always prefer lossless extraction over rendering. If a page embeds a JPEG,
doc.extract_image(xref)['image'] returns the original JPEG bytes — writing them
straight to disk is bit-identical and costs zero quality. Rendering at dpi=72
and re-encoding at q95 is strictly worse: it adds a generation of loss and
makes the files bigger.

imgs = page.get_images(full=True)
if len(imgs) == 1 and not page.get_text().strip():
    info = doc.extract_image(imgs[0][0])
    if info['ext'].lower() in ('jpg', 'jpeg'):   # 注意是 'jpeg' 不是 'jpg'
        open(dst, 'wb').write(info['image'])     # 无损

Fall back to page.get_pixmap(dpi=72) only when the embedded image is not JPEG
or the page carries other content.

Conventions that worked well:
- Folder name = PDF basename with any -压缩 suffix stripped (封面-18, 19-42)
- Files named 001.jpg, 002.jpg … zero-padded so they sort by page order
- Report per-book page count vs. image count, and how many came out lossless

Real run: 9 books × 24 pages → 9 folders / 216 images / 241 MB in 1 s, all 24/24
lossless per book.

Workflow

1. Inspect

Use PyMuPDF to enumerate pages and check:

import pymupdf
d = pymupdf.open(SRC)
for i in range(d.page_count):
    p = d[i]
    imgs = p.get_images(full=True)
    print(i, p.rect, len(imgs), imgs[0][2:], p.get_text()[:40])

Capture per-page:
- page.rect (preserved verbatim on rebuild)
- xref, image width/height, Filter (FlateDecode/CCITTFax/etc.), ColorSpace
- text length — if 0 across the board, re-encoding is safe

If pages use CCITT Fax / JBIG2 already, the file is likely well-compressed and you should decline or only do minimal re-encoding. This skill is for uncompressed scans.

2. Decide color strategy

Sample pixel chroma to decide per-page:

def color_frac(pix, thr=30):
    s = pix.samples; n = pix.n
    step = n * 97
    cnt = sum(1 for off in range(0, len(s)-n, step)
              if max(s[off],s[off+1],s[off+2]) - min(s[off],s[off+1],s[off+2]) > thr)
    return cnt / (len(s) // step)

• color_frac < 0.01 → store as grayscale JPEG (halves bytes for free, edges crisp)
• otherwise → keep RGB JPEG (PyMuPDF tobytes("jpg") defaults to 4:2:0 chroma; good enough)

For full-color documents with mostly-color pages, the savings from per-page grayscale detection are modest; just keep everything RGB and skip the per-page decision.

3. Pilot quality on a few pages

Run 3-5 representative pages through Pixmap.tobytes("jpg", jpg_quality=q) for q in (60, 70, 78, 85, 90, 93). Compute bytes/px and extrapolate to total page count. Pick the highest quality whose projected total stays ≤ 80% of the size budget (leave container overhead margin).

Empirical sweet spots for a ~90 MP / 24-page scanned comic to fit 30 MB:

For purely-photo (no line art) content, q80-q85 is plenty.

4. Build

Use scripts/compress_pdf.py. Key construction:

out = pymupdf.open()
for i in range(src.page_count):
    xref = src[i].get_images(full=True)[0][0]
    pix = pymupdf.Pixmap(src, xref)
    if pix.alpha:
        pix = pymupdf.Pixmap(pymupdf.csRGB, pix)   # drop alpha before JPEG
    jpg = pix.tobytes("jpg", jpg_quality=Q)
    page = out.new_page(width=src[i].rect.width,
                        height=src[i].rect.height)
    page.insert_image(page.rect, stream=jpg)       # embeds as DCTDecode
out.set_metadata(src.metadata)
out.save(OUT, deflate=True, garbage=4, clean=True)

PyMuPDF embeds the JPEG stream verbatim (Filter becomes /DCTDecode), so no re-compression happens during save.

For a whole folder of scans, use scripts/batch_compress.py instead of looping manually — it auto-selects quality per book and verifies each output:

python scripts/batch_compress.py SRC_DIR [BUDGET_MB] [Q_START]
#   BUDGET_MB 默认 29.5（对 30MB 预算留 0.5MB 余量）
#   Q_START   默认 92
# 输出：原名 + "-压缩.pdf"，与源文件同目录，原始文件不动

It tries Q_START, and on overflow steps quality down by 2 until it fits (floor 70). Real run on 9 scanned books of 24 pages / 85-99 MP each: 7 books passed at q92 on the first try, 2 needed a drop to q90; total 1136 MB → 241 MB (21.2%) in 57 s.

5. Verify

a, b = pymupdf.open(SRC), pymupdf.open(OUT)
assert a.page_count == b.page_count
for i in range(b.page_count):
    ra, rb = a[i].rect, b[i].rect
    assert abs(ra.width-rb.width) < 1 and abs(ra.height-rb.height) < 1
    assert 'DCTDecode' in str(b.xref_get_key(
        b[i].get_images(full=True)[0][0], 'Filter'))

Then render a 100% crop (dpi=72 for 1px/pt images) of one color page and one line-art page side-by-side with the same crop of the source. Tiny text + thin lines are where JPEG artifacts show up first; check those before declaring success.

Pitfalls

• Don't trust xref_stream for size estimates. For DCTDecode-filtered streams PyMuPDF returns the decoded raw bytes, not the on-disk size. Use os.path.getsize(OUT) as the ground truth.
• Sandbox blocks overwriting outside the workspace. pymupdf.save() first removes the destination; if it errors with Operation not permitted, write to /tmp then cp to the target.
• Don't downscale unless you have to. Losing 50% of pixels at fixed quality saves 50%, but losing 50% of pixels at +1 quality step often saves more with sharper edges. Keep native pixels and only drop quality.
• PyMuPDF's JPEG encoder has no 4:4:4 option in the public API (as of 1.28). For pure B&W line art, JPEG is the wrong tool — consider CCITT G4 via Ghostscript instead, but that's a different workflow.
• Don't re-encode PDFs that already use JBIG2 / CCITT — they're well-compressed already.

Resources

scripts/

• compress_pdf.py — single-file builder. CLI: compress_pdf.py SRC OUT Q
• Q = JPEG quality 1-100
• rebuilds the PDF preserving page dimensions and metadata
• prints per-page and total size with pass/fail against 30 MB
• batch_compress.py — folder-level batch runner with adaptive quality: starts at
  Q_START and steps down until each book fits the budget. CLI:
  batch_compress.py SRC_DIR [BUDGET_MB] [Q_START]. Verifies page count, page
  dimensions, and DCTDecode encoding for every output, and prints a per-book
  summary. Prefer this over the single-file script when there is more than one PDF.
• export_images.py — exports every PDF in a folder to per-page JPGs, one
  subfolder per PDF. CLI: export_images.py SRC_DIR [OUT_DIR_NAME].
  Prefers lossless extraction of embedded JPEGs; falls back to rendering.
  Prints per-book page count, image count, and how many were lossless.
• merge_pdfs.py — losslessly merges multiple image-only PDFs into one, adding a
  bookmark per source. CLI: merge_pdfs.py OUT.pdf SRC1.pdf ... or
  merge_pdfs.py --auto OUT.pdf SRC_DIR. Warns if the output is smaller than
  the sum of inputs (a sign of unwanted re-encoding).
• flatten_layered_pdf.py — for pages that stack several images at different
  resolutions. CLI: flatten_layered_pdf.py SRC OUT [CAP_DPI] [Q]. Renders each
  page's composite at its true available DPI and rebuilds; refuses to run on pages
  with text/vector content.

references/

• notes.md — engineering notes on quantization choices and pitfalls observed in real runs