Image to Excel
Convert data-bearing images into clean, editable .xlsx workbooks: the multimodal vision model does the reading, openpyxl does the writing.
When to use
Trigger on any request pairing an image (png/jpg/webp/pdf page screenshot) with a spreadsheet outcome: table screenshots, screen photos, scanned documents, invoices, receipts, statistical charts, dashboards, log screenshots.
Pipeline
- Locate & inspect — resolve image paths to absolute, confirm readable.
- Read the image — call
Readwith the image path. The tool renders images visually, so data can be read directly. Do NOT reach for OCR libraries. - Tile if dense — crop overlapping tiles and read each one, then merge (see Step 2b).
- Transcribe to JSON — describe the workbook: sheets / headers / rows / formats. Contract in
references/schema.md. - Build — run
scripts/build_xlsx.py. - Verify —
--verifyprints dimensions and first rows; reconcile against the image. - Deliver —
present_filesthe.xlsx.
Step 1 — Locate images
Images arrive as workspace-relative paths, absolute Windows paths, or conversation attachments. Resolve to absolute before reading. For multiple images decide whether to merge into one sheet or produce one sheet per image.
Step 2 — Read the image
Call Read with the image path; extract data from the visual rendering.
Accuracy rules
- Dense, wide, or small-text images must be tiled:
bash python scripts/crop_image.py in.png --mode thirds --outdir tiles/ python scripts/crop_image.py in.png --mode grid --rows 2 --cols 3 --overlap 0.15 --outdir tiles/
Three vertical strips beat one full-width read for wide tables. - Re-read uncertain regions and reconcile against the first pass.
- Never invent values. Illegible →
null, and record it innotes.
Step 2b — Cropping modes
--mode |
Behaviour | Use for |
|---|---|---|
thirds |
3 vertical strips with 10% overlap | wide tables, spreadsheet screenshots |
halves |
2 vertical strips | moderately wide tables |
quads |
2×2 grid | full-page documents, invoices |
grid |
--rows × --cols with --overlap |
custom layouts |
Step 3 — Transcribe structure
Full field-by-field contract: references/schema.md. Shape:
{ "sheets": [ { "name": "Sheet1", "headers": [...], "rows": [[...]] } ] }
Classification:
| Image looks like | Structure |
|---|---|
| Grid with header row | headers = header row, rows = data |
| Grid without header | synthesise descriptive headers, flag in notes |
| Chart (bar/line/pie) | one sheet per series: category column + value column; legend mapping in notes |
| Invoice / receipt | sheet 明细 for line items, sheet 汇总 for totals and key-value pairs (field/value rows) |
| Dashboard | sheet KPI for card values, one sheet per embedded table |
| Log / console | columns 时间, 内容 unless an obvious delimiter exists |
Typing rules: pure numbers stay numbers — no thousands separators, no %, no currency symbol; put units in the header (金额(元)) and let the script apply display formats. IDs with leading zeros (00123) stay strings.
Step 4 — Build
python scripts/build_xlsx.py data.json -o out.xlsx
Flags: --freeze (default A2), --no-styles, --verify.
Step 5 — Verify
python scripts/build_xlsx.py data.json -o out.xlsx --verify
A 12-row table must yield 12 data rows. Mismatch → re-read the image, fix the JSON, rebuild.
Step 6 — Deliver
present_files the .xlsx. In the reply state sheet names, row counts, and any cells read with low confidence.
Bundled resources
scripts/build_xlsx.py— JSON → styled xlsx, with--verify.scripts/crop_image.py— tile images for higher-accuracy reading.references/schema.md— full JSON contract.references/recipes.md— worked examples per image category.references/troubleshooting.md— failure modes and fixes.
Environment
Use the managed venv that already has openpyxl:
PY="C:/Users/90002269/.workbuddy/binaries/python/envs/default/Scripts/python.exe"
If it is missing, create it:
"C:/Users/90002269/.workbuddy/binaries/python/versions/3.13.12/python.exe" -m venv "C:/Users/90002269/.workbuddy/binaries/python/envs/default"
"$PY" -m pip install openpyxl pillow
pillow is only required by crop_image.py.
