@jerryjliu0
Introducing the Document Parsing Protocol (DPP) There are hundreds of different OCR models and tools, but they might all parse a document in different ways. Some errors are very clear-cut (e.g. hallucinating text, or completely misaligning a table), but many errors are much more ambiguous (e.g. how to represent split tables sharing a header). It's an issue that plagues most parse benchmarks too, because they inherently reward a certain style even if other styles might be semantically correct. We introduce a protocol that proposes a standard way of parsing these documents into a correct format. We've curated a list of 140 rules that cover literally every aspect of doc parsing, for instance: - tables: borderless whitespace tables, header-only tables, horizontal slicing, side-by-side mini tables - forms: key-value regions, checkboxes, checklists, ids and labels, fields outside the form, - charts: standardized chart output, multiple charts, gantt charts, line charts You can use DPP as an evaluation rubric for LLMs when determining the quality of parsing a document. It's not a full evaluation benchmark (yet), but an initial evaluation guide, and meant to encourage a discussion on what are the "best" ways to parse a document. Check it out: https://t.co/HsLh6ma9bB Blog: https://t.co/0md2O2M2Mf
