Process Complex Tables with AI Builder and Form Processing

Introduction

In today's ever-evolving technological landscape, Microsoft has made significant strides in democratizing the use of artificial intelligence (AI) through tools like AI Builder. One of the remarkable capabilities of AI Builder is its ability to automatically extract information from forms, effectively making data entry a relic of the past.

Recently, Microsoft released a new feature within AI Builder that allows users to tag complex tables through the forms processor model. Previously, if the forms processor model was unable to detect a table, users had little recourse to customize the detection. However, with this new preview feature, users can now custom tag tables, training the forms processing model to recognize table structures specific to their needs. This innovation not only enhances flexibility but also increases efficiency in processing forms with complex tables.

Demonstration of the New Feature

To illustrate how this feature works, let's consider an example of a form used for a food sign-up by Contoso. It includes several entries detailing names, food items, and quantities. Traditionally, users could not extract data from complex tables like this; however, with the newly introduced feature, we can now train the forms processing model to accurately capture such information.

Step-by-Step Process

1. Create a Form Processing Model: Start by navigating to the AI Builder service and creating a new form processing model. After naming the model (e.g., Contoso Food Sign-Up Form), specify the fields you wish to extract. You can tag the event date as a standalone field and then specify the structure of the table that includes names, food items, and quantities.

2. Upload Training Data: Ensure you upload at least five forms to provide a solid training set for the model.

3. Analyze and Tag Documents: After uploading, you analyze the documents and tag the necessary fields, including the event date and table structure. For each row in the table, select and tag data in the defined categories.

4. Train the Model: Once all documents are tagged, train the model to recognize and process the data accordingly.

5. Publish the Model: After training, publish the model so it can be used in applications such as Power Automate or PowerApps.

6. Utilize in Power Automate: Create a flow in Power Automate that triggers upon an input (e.g., a file). Add a step to process the submitted document with the AI Builder model to automate data retrieval and send results via email.

Limitations and Considerations

While this feature is groundbreaking, it does come with limitations. It requires a footer below the custom table for it to function correctly. Additionally, if you expect a table to span multiple pages, you must define multiple tables for each anticipated page and later merge the results.

The training set used in the demonstration will be available on GitHub, allowing users to replicate the example seamlessly.

Keywords

• AI Builder
• Form Processing
• Complex Tables
• Data Extraction
• Microsoft
• Automation
• Power Automate
• Model Training

FAQ

1. What is AI Builder?
AI Builder is a Microsoft tool that enables users to integrate artificial intelligence capabilities into their applications, promoting automated processes and data extraction.

2. How can I tag complex tables using AI Builder?
Users can tag complex tables in AI Builder by creating a form processing model, uploading training forms, and specifying fields and table structures that the model should recognize.

3. What are the limitations of the new feature for processing complex tables?
One limitation is that there needs to be a footer below the custom table for the process to work correctly. Additionally, if tables span multiple pages, users must create separate tables for each expected page.

4. Can I use this data in Power Automate?
Yes, once the model is trained and published, it can be used in Power Automate to automate workflows that include processing forms and extracting data.

5. Where can I find the training sets used in the demonstration?
The training sets showcased in the demonstration will be available on GitHub for users to download and replicate the example.