When humans write documents they insert visual elements resp. an underlying visual structure for other humans to understand them easier and faster. However, conventional computer programs do not recognize this visual structures. One can imagine it as if the computer would only read from top left to bottom right and does not understand the visual information. This can easily lead to errors. A prominent example is regional text blocks that are not identified, e.g.

The computer program would interpret this as follows:

Strasse
Rechnungsnummer: 8006 Zürich

20128

Another prominent example is key value pairs in non-trivial arrangements, such as

The computer program would again fail to interpret the correct invoice number and invoice date. Hence the computer should somehow understand the visual structure and the text.

Technologically speaking documents need to be available in structured representations, as this is a crucial requirement in order to analyse their content automatically. However, such analyses are prevented by most file formats that are prevalent today due to being rendered without structural information. A prominent example are PDF documents: this file format benefits from portability and immutability, yet it is flat in the sense that it stores all content as isolated elements (e.g. combinations of characters and positions) but without hierarchical information. As such, the structure behind figures and especially tables is discarded and thus no longer available for computerized analyses. In contrast, file formats such as XML or JSON naturally encode hierarchical structures among textual elements. Hence, techniques are required in order to convert document renderings into structured, textual representations, so that the content can be analysed while considering the actual structure.

At BLP Digital we have built an automatic document processing solution, which extracts all relevant information from business relevant documents, recognizing the complete document structure using a unique combination of deep learning algorithms for vision and natural language processing. To make it sound simple, our algorithm does the following: It looks at the document like a human would do it and identifies regions of elements that belong together, e.g. a letter head, a table, etc. Then it interprets the data within those regions and extracts relevant information. The origins of our solution lie in our ETH-background and we are still closely connected to the relevant research, ensuring our solution stays highly innovative and constantly further improves.