Smart OCR: AI-Assisted Image-to-Text Recognition

A three-layer OCR pipeline that combines Tesseract multi-strategy recognition with DeepSeek AI validation and correction for high-accuracy text extraction from images.

When to Invoke

• User uploads a .doc / .docx / .pdf file containing mostly images (scanned pages)
• User asks to "extract text from images," "OCR these pictures," or "convert image to text"
• User needs accurate Chinese+English bilingual OCR from document images
• User complains about garbled OCR results and needs better quality

Architecture

┌──────────────────────────────────────────────────────────┐
│  Layer 0: Mosaic Detection                               │
│  → Detect images with dense mosaic/blurred regions        │
│  → If valid Chinese char density < 5%, skip OCR entirely  │
│  → Mark image as "(含马赛克区域，已跳过识别)"              │
├──────────────────────────────────────────────────────────┤
│  Layer 1: Multi-Strategy Tesseract OCR                   │
│  (Original / Gray / CLAHE / 2× Upscale / Sharp)          │
│  → Auto-select best result by Chinese char count          │
│  → Apply common OCR error correction dictionary           │
├──────────────────────────────────────────────────────────┤
│  Layer 2: DeepSeek AI Validation                         │
│  → Judge if OCR text is coherent (score 0-10)            │
│  → Route based on confidence level:                       │
│    ≥ 7/10: Accept as-is, minor typo fix only             │
│    4-6/10: Request DeepSeek to correct garbled chars      │
│    < 4/10: Check if mosaic; if yes, discard; if not, raw │
├──────────────────────────────────────────────────────────┤
│  Layer 3: Results Assembly                                │
│  → Combine per-image results into structured output       │
│  → Skip mosaic-only images (output note instead)          │
│  → Generate .docx with continuous paragraphs              │
└──────────────────────────────────────────────────────────┘

Prerequisites

1. API Key Setup

This skill requires a DeepSeek API key for the AI validation layer. Each user must provide their own key.

How to get a key: Register at platform.deepseek.com and create an API key.

How to provide the key: Set it as an environment variable before running the pipeline:

$env:DEEPSEEK_API_KEY = "sk-your-own-api-key-here"

The scripts read the key at runtime from $env:DEEPSEEK_API_KEY. No key is ever hardcoded in the skill files.

• API Endpoint: https://api.deepseek.com/v1/chat/completions
• Model: deepseek-chat

> Note: If no API key is set, the pipeline still runs — it simply falls back to Tesseract-only OCR (Layer 1 + error dictionary), skipping the DeepSeek validation step.

2. Dependencies (Pre-installed)

Workflow (Step-by-Step)

Step 1: Prepare Images

# Pseudo-code — see scripts/ocr_pipeline.py for full implementation
1. Convert .doc → .docx (via LibreOffice: soffice --headless --convert-to docx)
2. Unpack .docx to extract /word/media/* images
3. Sort images by index (image1.png, image2.png, ...)

Step 0 (Pre-filter): Mosaic Detection

Before running OCR, detect images that contain mostly mosaic/blurred/redacted regions:

When an image is classified as mosaic-containing:
- Skip the full OCR + DeepSeek pipeline for that image
- Output （含马赛克区域，已跳过识别） in the final document
- The image's entry is still listed with its filename but no text content

### Step 2: Multi-Strategy OCR

For each image, run Tesseract with **5 preprocessing strategies** and auto-select the best:

| Strategy | Code | Best For |
|----------|------|----------|
| **Original** | Raw image → OCR | Clean photos, good lighting |
| **Gray** | `cv2.COLOR_BGR2GRAY` | Standard scanned pages |
| **CLAHE** | `cv2.createCLAHE(clipLimit=2.0)` | Uneven lighting, faded text |
| **2× Upscale** | `cv2.resize(..., INTER_CUBIC)` | Small text, low DPI |
| **Sharp** | `cv2.addWeighted(sharp)` | Slightly blurry edges |

**Selection criterion**: Run each strategy with `--psm 6 --oem 1` and pick the result with the highest Chinese character count.

### Step 3: Apply OCR Error Dictionary + Space Removal

After Tesseraut, apply a correction map for common Tesseract errors found in testing, then remove inter-CJK spaces:

```python
# Common Chinese char substitutions from Tesseract
ocr_corrections = {
    "睿": "特",    # 睿(ruì) vs 特(tè) — very common
    "蔼": "勒",    # 形近字
    # See full list in scripts/ocr_errors.json
}

# Remove spaces between Chinese characters (Tesseract artifact)
# "等 、 博 爱 而 庆 祝" → "等、博爱而庆祝"
# English words and numbers are kept intact

Step 4: DeepSeek Validation & Correction

For each image's OCR text, call the DeepSeek API.

Validation Prompt — checks coherence:

你是一位专业的OCR校对专家。请评估以下OCR识别出的文字的通顺程度。
评分标准（0-10分）：
- 8-10分：意思通顺，基本无错别字
- 5-7分：基本可读但有个别错别字或乱码
- 2-4分：部分可读但存在大量乱码
- 0-1分：几乎完全不可读

注意：这是书籍/文章内容，是一段连续的论述文字。
请直接输出分数，不需要解释。

文字内容：
{ocr_text}

Correction Prompt — for score 4-6:

你是一位专业的OCR文字校对专家。请对以下OCR识别出的文字进行修正。
修正原则：
1. 只修改明显错误的字词（形近字、音近字）
2. 不要改变原意和句式结构
3. 保持原文的标点和段落
4. 如果遇到无法判断的乱码，保持原样不要猜测
5. 专有名词（人名、地名、书名）请根据上下文合理修正

只输出修正后的文字，不要添加任何解释。

待修正的文字：
{ocr_text}

Step 5: Assemble Output

Generate a .docx file with:
- Each image → labeled section (▎图片 N (filename.png))
- OCR text as a single continuous paragraph (no internal line breaks)
- Separator between images

Scripts Reference

The skill includes these scripts in scripts/:

Error Handling

Known Error Patterns (from testing)

```