PROJECT 03
OCR Pipeline Upgrade
Making document understanding more practical for real-world workflows.
- OCR
- Qwen2.5-VL
- AWS SageMaker
- Ollama
- Python
01 Context
Document processing relied on legacy Tesseract-based OCR, which struggles with complex layouts. I worked on improving OCR performance and on deploying a vision-language model for document understanding.
02 Challenges
- Large, dense pages that exceed what small document models handle well.
- Keeping a self-hosted model server reliably available.
03 My role
Optimised the SmolDocling pipeline, deployed Qwen2.5-VL, wrote the availability tooling, and proposed a VLM-based alternative to the legacy pipeline.
04 Decisions
Patching with overlap
Pages are split into overlapping patches so the model sees manageable regions without losing text at the boundaries. MSER helps locate text regions.
Ollama on SageMaker
Qwen2.5-VL is served through Ollama on AWS SageMaker, with a Bash script that keeps Ollama running and available.
05 Architecture
- Input document → region detection (MSER) → overlapping patches.
- SmolDocling for structured document conversion.
- Qwen2.5-VL served via Ollama on AWS SageMaker.
- Bash watchdog to maintain Ollama availability.
06 Results
- Improved OCR handling through patching, overlapping and MSER-based optimisation.
PROPOSED — NOT YET IMPLEMENTED
- A VLM-powered pipeline as an alternative to legacy Tesseract-based processing (proposed, not yet replacing it).
NEXT PROJECT · 01
Complaint Analytics Platform