PDF → Markdown: about 90% restoration rate

01 Background

Graduate students, our core users, mostly work with papers distributed as PDFs. I tried open-source tools that convert PDFs to Markdown so they could be used as knowledge, but table row/column relationships and equations came out broken.

02 Experiments and improvements

1. Selective repair and missed errors

   Problem Open-source conversion broke table row/column relationships and equations, so the key figures in PDF papers came out in a form hard to bring into the Wiki. I needed a way to repair the damaged parts.

   Experiment and result Docling extracted the structure, and only damaged tables and equations were repaired with local Qwen sLLMs. Tables, where row/column positions matter, were handled mainly by the vision model (qwen2.5vl:7b); equations were drafted by the text model (qwen2.5:7b) and checked by the vision model only when needed. This restored more than applying both models to everything and cut sLLM processing time by about 42%. Keeping OCR and prompts unchanged and only re-cropping images with Docling's strict bboxes fixed two previously failing equations. Evaluating repaired and untouched regions separately, 85 of the 96 sLLM-repaired regions passed, but 107 errors or omissions remained in untouched content. A separate experiment adding OCR comparison raised the pass rate from 80.00% to 97.53%, but also selected 50 correct regions for repair; processing took about 12 minutes 35 seconds.

   Decision Selecting repair targets missed errors, and extra checks to catch them added cost. I decided to test whether having the vision sLLM read every original page could avoid this dependence on damage detection.

2. Table/equation errors and cost in full-page sLLM conversion

   Problem Letting AI decide what to fix missed errors, and adding OCR checks also flagged correct content, increasing check time and cost. I needed a way to pick and fix only the damaged parts.

   Experiment and result Full conversion of 25 research PDFs with the vision sLLM (qwen2.5vl:7b), which can check the original page directly, (415 pages) achieved a 91.92% content-preservation pass rate. However, only 58.84% of tables and equations fully matched in values and structure, and processing every page imposed a heavy processing load.

   Decision I judged that sending all content through the model the same way was the problem. Body text should use AI only where needed, and tables and equations should get inputs that preserve their structure. So I split the paths for body, tables and equations, and looked for fast body extraction with AI fixes only where needed.

3. Comparing text extraction, adopting AnyDoc, and moving to commercial APIs

   Problem Full-page sLLM processing still left table/equation errors, and sending all body text to the model was a heavy load. To cut that load and improve speed and accuracy, I needed separate repair for the tables and equations that usually break, alongside fast body extraction.

   Experiment and result I found AnyDoc on Threads and, after seeing its GitHub comparison with Docling, tested it on the same 4 PDFs (30 pages). The recorded Docling run took 134.19s, while AnyDoc text extraction took 1.68s, about 80 times shorter. Docling’s time also included OCR and table analysis. One AnyDoc document had garbled characters because the PDF lacked mappings from character codes to actual characters.

   Decision The processing scopes differed, so this was not an 80-fold improvement in complete conversion. Still, AnyDoc was useful for fast text extraction. I kept separate table/equation repair, switched body extraction to AnyDoc, used AI on garbled or misordered pages, and preserved original figures. With no early users yet, running GPU servers for an sLLM would cost more than it saved, so from this point I switched the repair model to commercial LLM APIs.

03 Improved processing flow

flowchart TD
 A[Original PDF] --> B[Separate text, tables, equations and figures]
 B --> C[Preserve text context and position]
 B --> D[AI reads original table and equation regions]
 B --> E[Preserve original figures]
 C --> F[Reassemble in original order]
 D --> F
 E --> F
 F --> G{Check markers and missing content}
 G -->|Pass| H[Converted document]
 G -->|Mismatch| I[Keep original content for that region]

flowchart TD
 A[Original PDF] --> B[Separate text, tables, equations and figures]
 B --> C[Preserve text context and position]
 B --> D[AI reads original table and equation regions]
 B --> E[Preserve original figures]
 C --> F[Reassemble in original order]
 D --> F
 E --> F
 F --> G{Check markers and missing content}
 G -->|Pass| H[Converted document]
 G -->|Mismatch| I[Keep original content for that region]

04 Results and lessons

Pass rate against 30 original pages: 45.17% → 89.89%

I evaluated the existing Docling and the improved pipeline by splitting the same papers into 445 evaluation units covering body text, tables, equations and other content, and comparing them with the originals. The practical pass rate rose from 201 (45.17%) to 400 (89.89%), and the strict rate, which requires values and symbols to match exactly, rose from 193 (43.37%) to 393 (88.31%).