vocab-pdf-extract
Extract a vocabulary table from a word-list PDF and produce a styled Excel
workbook whose 课次 (lesson) column is derived from the matching textbook PDF.
Purpose
Given:
- a vocabulary PDF (the word list: word / phonetic / pos / meaning, often
grouped by unit, sometimes with the textbook page where each word first appears), and
- a textbook PDF (the actual lessons, used only to map printed pages → lessons),
produce an .xlsx with the canonical 7 columns:
单词 | 音标 | 词性 | 中文意思 | 单元 | 页号 | 课次
The skill is format-tolerant: it does not assume one fixed layout. It detects
whether the PDF is text-based or scanned, adapts column detection, and reports any
deviation from the baseline profile instead of failing.
When to use
- "从这个单词表 PDF 提取单词，导出 Excel"
- "把词汇表生成词表，并加上课次"
- "单词表格式和之前不一样，也能识别吗" → yes; follow §Generic handling
- Any task to build/extend a textbook vocabulary spreadsheet with a lesson column.
Core principles (read references/rules.md)
- Transcribe exactly — never correct typos in the source (note them instead).
- Standard IPA in
/.../; keep(r),(pl. …)parentheticals. - POS uses the book's abbreviations (
n.v.adj.adv.pron.conj.prep.aux.). - One part of speech per row; repeat word/phonetic/meaning across rows.
Unit Nheader opens unit N; entries before it belong to the previous unit.page= the word's first printed page in the textbook (integer, no "p.").- Culture/reading sections (e.g. Understanding cultures) have no lesson number;
their课次is the section name, not "Lesson N".
Workflow
Step 1 — Render & classify the vocabulary PDF
python scripts/render_pdf.py --pdf vocab.pdf --out vocab_pages/ --dpi 200
Inspect the JSON: "scanned": true/false.
Step 2a — Text-based PDF: auto-extract
python scripts/extract_text_vocab.py --pdf vocab.pdf --out work/
Writes work/vocab.tsv + work/format_profile.json (adaptive column detection).
Step 2b — Scanned PDF: vision transcription
Read each vocab_pages/page_NNN.png and transcribe using the per-page prompt
template in references/format_signals.md §B. Write the result to work/vocab.tsv
with header word\tphonetic\tpos\tmeaning\tunit\tpage. Record the detected column
structure for each page.
Step 2c — Image input (photos / screenshots / scans)
When the source is one or more images rather than a PDF, skip PDF rendering and
prep the images directly:
python scripts/extract_image_vocab.py --images a.png b.jpg --out work/ --split-cols auto
python scripts/extract_image_vocab.py --dir photos/ --ext png,jpg --out work/
This normalizes each image (EXIF rotation, upscale for legibility), optionally splits
a wide two-column / two-page spread into left and right halves, writes prepared PNGs
to work/img_prepared/page_NNN.png, and emits work/image_jobs.json with a
ready-to-use transcription prompt per image. Then read each prepared image in order
and transcribe it into work/vocab.tsv using the same template in
references/image_input.md. The downstream steps (lesson map, xlsx, verify) are
identical to the PDF paths.
Step 3 — Build the lesson (课次) map from the textbook PDF
The textbook's structural convention turns a printed page into a lesson.
# default spec = 冀教版九上 (16pp/unit, 2pp/lesson, culture unnumbered):
python scripts/build_lesson_map.py --mode spec --out work/ \
--pages 2,4,6,8,10,12,18,19,20,... # the printed pages that appear in vocab.tsv
# scanned textbook with a bespoke layout:
python scripts/build_lesson_map.py --mode vision --map lesson_map.tsv --out work/
# different digital textbook:
python scripts/build_lesson_map.py --mode spec --spec my_spec.json --out work/
This writes work/lesson_map.json = {printed_page: {unit, lesson}}.
Step 4 — Assemble the Excel
python scripts/build_xlsx.py --vocab work/vocab.tsv \
--lesson-map work/lesson_map.json \
--out result.xlsx --title "英语（冀教版）九年级上册单词表"
Adds the 课次 column by looking up each row's page in the lesson map, writes a
styled workbook (merged title, blue header, zebra rows, borders, AIGC marker).
Step 5 — Verify & report format deviations
python scripts/verify_vocab.py --vocab work/vocab.tsv \
--xlsx result.xlsx --baseline references/baseline_profile.json
Fix any errors; review warnings and deviations, then confirm with the user.
Generic handling (format differs from baseline)
- Run Step 1. If
scanned:true, use §2b vision path; the prompt template tells the
model to adapt to whatever columns/order it sees and to record the detected schema. extract_text_vocab.py(text mode) locates columns by fuzzy header matching, not
fixed coordinates, so reordered/renamed columns still parse.verify_vocab.pydiffs the detected structure againstreferences/baseline_profile.json
and lists every deviation (missing page column, missing unit headers, extra columns,
different unit set, etc.). Surface these to the user instead of guessing.- If the textbook layout is unusual, supply a custom
--specJSON or a--mode vision
lesson_map.tsv; do not hard-code a single layout.
Bundled resources
scripts/render_pdf.py— PDF → per-page PNG + scanned/text classification.scripts/extract_text_vocab.py— text-based PDF → normalized TSV (adaptive).scripts/extract_image_vocab.py— images → prepared PNGs + transcription job manifest.scripts/build_lesson_map.py— printed page → lesson, via spec or vision map.scripts/build_xlsx.py— TSV + lesson map → styled 7-column xlsx (+ AIGC marker).scripts/verify_vocab.py— data integrity + baseline structural diff.references/rules.md— transcription rules (IPA, POS, one-pos-per-row, no-correction).references/format_signals.md— format-variation detection + per-page vision prompt.references/image_input.md— image-input preprocessing + per-image transcription template.references/baseline_profile.json— known-good structure for deviation reporting.
Notes / limitations
- No OCR is bundled. Scanned PDFs and image inputs both rely on the model reading the
rendered / prepared images (Steps 2b and 2c). If an OCR engine (tesseract) is later
installed,extract_text_vocab.py/extract_image_vocab.pycan be extended to call
it as an optional accelerator, with vision transcription kept as the fallback. - The default lesson spec matches 冀教版九上. Other textbooks need a spec or vision map.
