PDF Translation (e.g. EN -> PT) with bilingual PDF output

Trigger: user provides a PDF (product brochure, user manual, datasheet, spec sheet) and asks to translate it and return a PDF. Proven on a 76-page EV-charging product brochure and a 51-page charger user manual (both EN -> Brazilian/European Portuguese), and a 34-page ZH controller technical manual (ZH -> EN) — the in-place pipeline is language-agnostic.

Direction extender: The in-place redaction pipeline works WHATEVER the source language is. For a Chinese-source PDF, line keys are the exact CJK strings from page.get_text("dict"); replace_inplace.py needs NO code changes (proven: 34-page 微耀 manual, 1,177 ZH lines / 880 unique keys → EN, 0 CJK leftover). Before replacing, ALWAYS run the coverage gate scripts/check_coverage.py in.pdf map.json — hand-built maps silently miss dotted-leader TOC keys (e.g. 附录D ...34) whose exact string differs from what you'd type; fix missing keys programmatically by re-reading the PDF (startswith prefix rules) rather than re-typing. After replacement, image counts INCREASE (redaction splits images intersecting rects — p19 went 78→117); that is NORMAL, verify visuals via pixel diff not image count. Full session notes: references/zh-en-inplace-notes.md.

Environment facts (this Windows host)

• No system Python (python/python3 absent or WindowsApps stubs). Do not plan around Python.
• Hermes Desktop bundles Node.js v22 + npm at:
  D:\HermesAgentCN\Hermes Agent CN Desktop\data\versions\<version>\node\node.exe (and npm.cmd)
• Scratch dir + deps:
  New-Item -ItemType Directory -Force -Path "C:\Users\<user>\pdf_translate" $env:PATH = "D:\HermesAgentCN\Hermes Agent CN Desktop\data\versions\<version>\node;" + $env:PATH npm install pdf-parse pdf-lib
• PowerShell & gotcha: a command whose first token is & ... is rejected as "backgrounding" by the terminal tool. Never start commands with &; put the tool dir on PATH and call npm/node bare. When invoking a script located outside the project dir, set $env:NODE_PATH = "<scratch>\node_modules" so require('pdf-parse') resolves.
• Embeddable Python runs in ISOLATED mode (a python312._pth file exists, even after appending import site): python script.py does NOT put the script's directory on sys.path, and PYTHONPATH is ignored → sibling-module imports fail with ModuleNotFoundError. Bootstrap explicitly at the top of the script: sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))). Embedded python also has no venv module → use pip install --target <proj>\pylib -r requirements.txt for self-contained deps that don't pollute other environments (this is how the file_translator tool ships, see below).

In-place replacement pipeline (PyMuPDF redaction — proven end-to-end on 51-page manual)

1. Get Python + PyMuPDF (no system Python on this host — use the embeddable distribution):
   curl.exe -s -L -o python-embed.zip https://www.python.org/ftp/python/3.12.8/python-3.12.8-embed-amd64.zip Expand-Archive python-embed.zip py Add-Content py\python312._pth "`nimport site" # REQUIRED or pip can't install curl.exe -s -o py\get-pip.py https://bootstrap.pypa.io/get-pip.py cmd /c "py\python.exe py\get-pip.py" cmd /c "py\python.exe -m pip install pymupdf"
   Import as import pymupdf (the fitz name is deprecated in 1.28+). Run scripts via cmd /c "py\python.exe script.py" (bare & in PowerShell is rejected by the terminal tool as "backgrounding").
2. Analyze line structure: page.get_text("dict") → blocks → lines → spans. Per line: join s["text"], compute line bbox (min/max of span bboxes), take size and font from the first span. Dump lines that need translation (letters only, dedupe) to JSON.
3. Build a per-LINE translation map (not per-page): key = exact English line text, value = PT. This is the bulk of the work — for a 51-page manual ~950 lines / ~900 unique texts. Rules: keep model numbers, standards codes, OCPP/vendor error codes (ConnectorLockFailure etc. stay English — they're protocol constants), URLs/emails, numeric specs; translate everything else.
4. Replace per line with redaction (scripts/replace_inplace.py): for each line whose text is in the map, page.add_redact_annot(rect, text=pt, fontname="helv", fontsize=size, fill=(1,1,1), text_color=(0,0,0), align=0), then page.apply_redactions() per page. Auto-shrink font when PT is wider: new_size = size * (rect.width*0.96 / text_w), floor ~4pt.
5. Save without recompression: doc.save(out, garbage=0, deflate=False).
6. Verify (scripts/verify_inplace.py): page count + page size equal source, per-page line count identical (layout 1:1), PT section markers present, no EN header leftovers, technical tokens + OCPP codes preserved, no U+FFFD, no span bbox beyond page bounds, no fonts < 3pt.

In-place pitfalls (all hit in practice — do not re-discover)

• add_redact_annot signature: (self, quad, text=None, fontname=None, fontsize=11, align=0, fill=None, text_color=None, cross_out=True). The white-fill param is fill, NOT fill_color — passing fill_color= raises TypeError: got an unexpected keyword argument.
• Do NOT locate lines with page.search_for() — it silently misses lines whose text spans are split (e.g. a ● bullet is its own span, so search_for on the joined text returns nothing → line left untranslated while the bullet gets erased). Use the line bbox captured during the dict traversal instead; it is exact and 100% reliable.
• Alignment: keep align=0 (left). A "detect centering" heuristic mis-fires on full-width lines and shifts them ~50pt sideways.
• WinAnsi applies to helv too: redaction text is drawn with built-in Helvetica, so the same sanitize() needed for pdf-lib applies (≤≥℃×–—•●○□→²Ω smart quotes). See scripts/generate_bilingual_pdf.js sanitize().
• Line-based translation is mandatory: translate one EN line → one PT line (never merge/split lines), or the layout breaks. Over-long PT is handled by font shrink, not reflow.
• File size grows slightly (redaction rewrites content streams) even with garbage=0 — that is expected and matches "don't compress"; never use garbage=4 (object dedup/compression) on these files.
• The document-translation skill overlaps this one (same domain, same toolchain pitfalls) — prefer pdf-translation for the full pipeline + scripts.
• PyMuPDF 1.28: doc.page(n) is REMOVED — a Document is directly indexable; use doc[n] (and doc[0].rect for page size). doc.page(0) raises AttributeError. (Hit while writing the ZH→EN verifier.)
• PowerShell cmd /c with space-containing arguments breaks (cmd /c "py\python.exe ... D:\HermesAgentCN\Hermes Agent CN Desktop\..." → "can't open file" because the path is split). Correct pattern: prepend the python dir to $env:PATH, then call python "script" "in" "out" "map" with quoted args as a bare command (also satisfies the &-gotcha rule).

ZH -> EN in-place translation (Chinese-source PDFs) - proven 2026-09 on 34-page controller manual

The same replace_inplace.py pipeline works for Chinese-source PDFs translated to English; only the map-building and verification change. Proven on the Weiyao charging-controller technical manual (微耀充电桩控制器方案说明书.pdf, 34 pages, 1177 CJK lines / 880 unique → 880-entry map, output _EN.pdf).

1. Extract lines + filter CJK: standard dict traversal; keep only lines matching [\u4e00-\u9fff] (ASCII-only lines like DC 12V +/-10%, J1, RS485 stay untouched automatically since they're not in the map). Dump unique CJK lines to a text file for translation and build the map as {exact_cjk_line: english}.
2. Cheap single-call lines are normal: Chinese table cells/paragraphs break into fragments ((含, + 反馈, /日, 第). Translate each fragment literally even without full context — the fragments rejoin visually in the layout.
3. Coverage check BEFORE replacing (skip-the-replace is too late): re-walk the PDF, count every CJK line not in map, assert MISSING_UNIQUE=0. Most misses are TOC dot-leader lines (附录B 160kW ... 34) where the dot count differs from what you typed; fix by taking the exact line text from the PDF as the key (prefix-match on the section title, assign the translation), never hand-typing the dots.
4. Replace: identical to EN->PT (add_redact_annot(rect, text=..., fontname="helv", fontsize=size, fill=(1,1,1), text_color=(0,0,0), align=0) + apply_redactions() + doc.save(out, garbage=0, deflate=False)). Sanitize still applies (Chinese source uses <, >=, degC etc). EN usually renders SHORTER than CJK, so font-shrink almost never triggers — layout risk is lower than EN->PT.
5. Verify with CJK regex: text-level checks = page count/page size equal, sum(t.count(c) for c in chars 4e00-9fff) == 0, U+FFFD == 0, EN section markers present, technical tokens preserved (OCPP, CCS, CHAdeMO, GB/T 27930, ISO 15118, J12, RS485, BMS, Plug & Charge).
6. Image-count increase is EXPECTED, not damage: get_images(full=True) counts rise after redaction (e.g. 18 -> 29 on a page) because PyMuPDF clips any image intersecting a redaction rect into sub-images; visual content is intact. Verify with pixel-diff + connected-components instead of image counts: render src/out pages at dpi=80, mask abs diff > 40, assert the largest diff blobs are thin text bars (w ~100-230, h ~13) and never big rectangles (w>25% page, h>15% page) — if a big region shows out_mean near 255 while src_mean is dark, an image was erased.

Multi-format in-place translation — use the shipped tool first

For Word (.docx), Excel (.xlsx), PPT (.pptx), text files, and images (OCR in-place redraw) — or any mixed batch — use the ready tool at D:\Willdigits\file_translator\ (built 2026-08, docx/pdf/xlsx/jpg/pptx/txt E2E-tested): double-click 启动.bat → browser at http://127.0.0.1:5000 → upload → pick target language (17 supported) → download the same-extension file. It also does PDF in-place the same way (redaction + font subsetting). Engines: auto (Google→MyMemory fallback), MyMemory, Baidu (key), OpenAI-compatible LLM (recommended for quality). Large documents are batched (whole pages per translation request — line-by-line calls were slow + rate-limited); if the service is unresponsive, check /api/health and server.log first (see "Server diagnostics" in the reference). Full architecture, per-format processor techniques, engine details, font/OCR asset sources, and extension guide: references/multiformat-inplace-tool.md. If the user reports 启动.bat "flashes and exits", the fix (GBK+CRLF encoding, chcp 936, errorlevel-in-block trap) is in that reference's "启动.bat maintenance" section.

PyMuPDF 1.28 pitfalls (newer installs — affects the in-place scripts above)

• Rect.adjust() removed → construct the rect manually.
• add_redact_annot() no longer takes images/graphics/text kwargs — they moved to apply_redactions(images=, graphics=, text=) (constants renumbered in 1.28; do not hard-code assumptions).
• apply_redactions() wipes the page's font resources — re-insert_font() before insert_text() with a custom font, else "need font file or buffer".
• A custom embedded font embeds IN FULL (17 MB NotoSansSC → 18 MB PDF!) → call doc.subset_fonts() before save (~700 KB result), then save garbage=0, deflate=False.
• Measure custom-font text with fitz.Font(fontname, fontfile=...).text_length(text, fontsize=).
• Details + verified working order: references/pymupdf-128-pitfalls.md.

Bilingual side-by-side pipeline (only when explicitly requested)

1. Pick the right file. When several candidate PDFs exist, extract a few pages of text from each and compare (per-page char counts also reveal blank pages). scripts/extract_render.js does extraction + rendering in one shot.
2. Extract text + render pages. pdf-parse v2 API (the old callable pdf(buffer) is GONE):
   js const { PDFParse } = require('pdf-parse'); const parser = new PDFParse({ data: new Uint8Array(fs.readFileSync(f)), verbosity: 0 }); const res = await parser.getText({}); // res.text, res.pages[{num,text}] const shot = await parser.getScreenshot({ scale: 1.2, imageBuffer: true, imageDataUrl: false }); // shot.pages[i].data = raw PNG Buffer
3. Translate page-by-page into JSON keyed by page number (write in batches of ~15-20 pages, then merge). Rules:
   - Keep model numbers (WY11, WY-OCU1), standard codes (CCS1/CCS2, CHAdeMO, GB/T, NACS, OCPP, IP54, IK10, IEC 61851), units and numeric specs verbatim.
   - Official certificates (patents, software copyrights) stay as-is; add a bracket note summarizing them.
   - A blank source page -> empty translation string (correct by design, not an error).
4. Merge + verify coverage (every page 1..N present; known blank pages excluded).
5. Generate the bilingual PDF with pdf-lib (scripts/generate_bilingual_pdf.js): A3 landscape 1190x842pt, original page image left (~520pt wide) with border, translation right, cover page, auto page-break for overflow.
6. Verify the deliverable (scripts/verify_translated_pdf.js): file in target dir, page coverage, images rendered, PDF parses with >= N+1 pages, target-language markers present (case-insensitive), no English spec-label leftovers, no U+FFFD.

Pitfalls (all hit in practice — do not re-discover)

• pdf-parse v2 API change: require('pdf-parse') returns an object exposing a PDFParse class; the old function-style pdf(buffer) throws "pdf is not a function". Constructor requires { data: Uint8Array, verbosity } (verbosity must not be undefined).
• WinAnsi-only standard fonts in pdf-lib: StandardFonts.Helvetica cannot encode code points outside CP1252 — throws WinAnsi cannot encode "≤" (0x2264). Sanitize BEFORE widthOfTextAtSize (word-wrap measures text) AND before drawText. See sanitize() in scripts/generate_bilingual_pdf.js (handles <=, >=, °C, x, bullets, [ ], ->, ², Ohm, quotes/dashes) plus a fallback regex stripping any remaining non-WinAnsi char. Diagnose offenders with scripts/find_bad_chars.js.
• Verification script from %TEMP%: require('pdf-parse') fails there — run with NODE_PATH set to the scratch node_modules (or require via absolute path).
• Long translations overflow: the generator must create continuation pages automatically (labeled "…(continuação)"), otherwise text silently drops off the page.
• Screenshot options: imageBuffer: true + imageDataUrl: false returns raw Buffers (avoids huge base64). scale 1.2 -> ~714x969 px PNGs for A4 source.
• Wording case-sensitivity in verification: check language markers case-insensitively (translations may capitalize terms like "Fator de potência").

User preference (Willdigits docs — apply by default)

• Domain: Willdigits EV-charging equipment docs under D:\Willdigits\充电桩产品说明书\通用说明书\<model>\ (brochures + per-model user manuals).
• Target language: professional Portuguese (Brazilian-style charging terminology — see references/ev-charging-pt-glossary.md).
• DELIVERY FORMAT — user explicitly corrected this (2026-08): IN-PLACE REPLACEMENT, NOT bilingual side-by-side. The user said: "不，这不是我想要的效果。我不想要双语对照版，我给你的是英语版，你返回给我的是葡萄牙语版。内容排版都不变，整个文件只更改了文字。文件大小也不要压缩。" → For Willdigits docs, default to in-place text replacement (same page count, same layout, same line positions — only the words change to PT), saved with garbage=0, deflate=False (no recompression). Do NOT produce the bilingual side-by-side layout unless the user explicitly asks for it.
• Save INTO the source folder with _PT suffix (e.g. 3Guns-DC Fast Charging Station 60-240kW_PT.pdf) — this part of the original preference stands.
• If the user does not state a language, continue the language of the previous task; check the folder for hints (e.g. a 俄语/ subfolder may mean Russian versions are also produced) and confirm.

Two delivery modes

1. In-place replacement (DEFAULT for Willdigits) — PyMuPDF redaction: erase each English text line at its exact coordinates and write the PT translation in the same spot, auto-shrinking font size to fit the original line width. Layout/line positions preserved 1:1. Use the dedicated pipeline + scripts below (scripts/replace_inplace.py).
2. Bilingual side-by-side (only on explicit request) — pdf-lib A3 landscape, original page image left, translation right + cover page. Original pipeline in the next section.

Support files

• scripts/replace_inplace.py — in-place EN→PT replacement via PyMuPDF redaction (generic, argv-driven; line-bbox based, auto font-shrink, no compression)
• scripts/check_coverage.py — pre-replace coverage gate: reports target-language lines missing from your map (default regex = CJK; works for any language); exit 1 if anything missing. Run BEFORE replace_inplace.py.
• scripts/verify_inplace.py — verify the in-place replaced deliverable (page/line-count preservation, PT markers, no EN leftovers, no overflow)
• scripts/extract_render.js — per-page text extraction + PNG rendering (generic, argv-driven)
• scripts/find_bad_chars.js — enumerate non-WinAnsi chars in a translations JSON
• scripts/generate_bilingual_pdf.js — build the side-by-side translated PDF (generic, argv-driven)
• scripts/verify_translated_pdf.js — verify the translated deliverable (generic, argv-driven)
• references/ev-charging-pt-glossary.md — EN->PT EV-charging industry terminology
• references/multiformat-inplace-tool.md — the D:\Willdigits\file_translator tool: architecture, per-format processors, translation engines, font/OCR sources, user prefs, 启动.bat maintenance (GBK/CRLF encoding, chcp 936, errorlevel trap, port guard)
• references/pymupdf-128-pitfalls.md — PyMuPDF 1.28 API changes + verified redact-redraw order + subset_fonts sizing
• references/zh-en-inplace-notes.md — ZH→EN in-place proven workflow (34-page manual): real run numbers, dotted-TOC key fix, verification markers/token list, image-count-vs-pixel-diff pitfall, cmd /c space-path fix