The need for and consequently the number of solutions for reading hand-written forms in an automated manner has been on a rise for as long as one could remember. Almost all businesses to varying degrees utilize paper based forms that are filled by customers by hand. Most if not all of these businesses convert this handwritten information into the digital format. Depending on the technological sophistication or the size of the business this digitization might be done manually by one or more data entry specialists or through an automated solution

It’s easy to see how the manual route may not be an ideal solution for a medium or large sized business. Some of the apparent drawbacks are:

a. The cost of having data entry specialists quickly adds up as more documents need to be digitized necessitating adding more resources

b. Manual data entry is a slow process

c. Manual data entry is error prone and requires a quality inspection which is costly and not fail proof

Many businesses have realized this and have transitioned to some form of a partially or fully automated solution to this problem. However, it’s not all rosy for these businesses either. The problems these businesses face is primarily related to the accuracy of the current solutions in the market

The industry average for ICR accuracy at character level is about 70% and it will drop significantly if measured at word level which is what matters at the end. Such automation may allow for reducing the number of data entry personnel but with such a low level of accuracy, there will be a need for increased quality check resources which are more expensive than data entry resources, hence diluting the cost benefit of automation. Moreover since quality check is a slower process than data entry, this kind of automation doesn’t even address the speed problem

a. Poor form design

b. User input not in line with format

c. Noisy images

d. Misaligned documents

e. Low quality scanning of documents

f. Spelling mistakes by user

g. Overwriting/corrections by user

The DocParser solution in Flowmagic provides an intuitive user interface where data can be extracted from any standard form in three easy steps:

step1: The user annotate the form (this is a one-time exercise for each new form) using an easy and intuitive UI. During annotation each input field can optionally be labelled as mandatory. The user can specify the datatype for each field as alphabets, numeric or check box and also set the context for the field e.g. Name, PAN, City, Car Make, Date etc. Once done, the saved template can be used repeatedly for reading forms of same type as long as there are no changes in the form design. In case of a change, the saved template can be easily modified

step2: The user uploads one or more forms and chooses the corresponding template (from previous annotations). The system automatically extracts data from the forms

step3: The system exports the output in csv, xml or json as desired by the user. If any field was marked as mandatory during annotation, the system also outputs a list of all mandatory fields that are blank

• The standard form being annotated can be any number of pages. The input form need not have the same number of pages. If there is a mismatch between the pages in the input form and the template, the system does a matching and runs the data extraction on matching pages only. This also means that the input form need not be sorted correctly

• The system can read handwritten as well as printed forms

• The system corrects for minor misalignments during scanning of documents or documents scanned in wrong orientation

• The system has inbuilt dictionaries for various contexts such as Name, Cities, States, Countries, PAN, Profession, Marital Status, Relationship, Amount, Car Make, Date, Gender

• The various datatypes supported by the system are alphabets, numeric, alphanumeric, checkboxes and special characters

• The system corrects user errors or scanning issues by performing datatype and dictionary checks (see examples below)

• The system checks for mandatory fields to make sure the form is completely filled

• Flexibility – you can annotate a wide variety of forms with complex inputs and data formats using the multiple data types and contexts built into the system

• Speed – Both annotation and data extraction are very user friendly and fast. The system can extract data from a five page form in under 30 seconds

• Scalability – The system is highly extensible and once setup for one type of form can easily be scaled for multiple forms or to process documents in bulk of the same format

• Accuracy - The character level accuracy of our model is over 90%. Word level accuracy depends on the form design and quality but in general varies between 75% and 85%

1. Have all instructions in bold at the top of the form

2. Instruct the user to write clearly in block letters as the form will be processed by a machine

3. Provide examples of how to enter data wherever there is a scope for confusion

4. Instead of providing a free form space for data entry provide a clearly marked space with a specific location to enter each character

5. The overall space should be large enough to contain the requisite data to avoid the user writing outside of this space

6. Have enough separation between the space for two fields to avoid overlap