by Tamir Hassan
Abstract:
A number of methods for evaluating table structure recognition systems have been proposed in the literature, which have been used successfully for automatic and manual optimization of their respective algorithms. Unfortunately, the lack of standard, ground-truthed datasets coupled with the ambiguous nature of how humans interpret tabular data has made it difficult to compare the obtained results between different systems developed by different research groups. With reference to these approaches, we describe our experiences in comparing our algorithm for table detection and structure recognition with another recently published system using a freely available dataset of 75 PDF documents. Based on examples from this dataset, we define several classes of errors and propose how they can be treated consistently to eliminate ambiguities and ensure the repeatability of the results and their comparability between different systems from different research groups.
Reference:
Towards a Common Evaluation Strategy for Table Structure Recognition Algorithms (Tamir Hassan), Technical report, PRIP, TU Wien, 2010.
Bibtex Entry:
@TechReport{TR125,
author = "Tamir Hassan",
title = "Towards a Common Evaluation Strategy for Table Structure Recognition Algorithms",
institution = "PRIP, TU Wien",
number = "PRIP-TR-125",
year = "2010",
url = "https://www.prip.tuwien.ac.at/pripfiles/trs/tr125.pdf",
abstract = "A number of methods for evaluating table structure recognition systems have been proposed
in the literature, which have been used successfully for automatic and manual optimization
of their respective algorithms. Unfortunately, the lack of standard, ground-truthed datasets
coupled with the ambiguous nature of how humans interpret tabular data has made it
difficult to compare the obtained results between different systems developed by different
research groups.
With reference to these approaches, we describe our experiences in comparing our algorithm
for table detection and structure recognition with another recently published system using
a freely available dataset of 75 PDF documents. Based on examples from this dataset,
we define several classes of errors and propose how they can be treated consistently to
eliminate ambiguities and ensure the repeatability of the results and their comparability
between different systems from different research groups.",
}