Real-Life Use case using AI Builder Form Processing with Power Automate Desktop

Introduction

Hey Makers, this is Druven, and I welcome you all to the fourth episode of my new segment called "Talk with Daves." This segment is where I invite industry experts from around the globe to discuss various topics, leverage their expertise, and gain knowledge. Today’s session is particularly interesting as we are going to talk about AI Builder Form Processing integrated with Power Automate RPA.

To present this session, we have our industry expert Mr. Steve Winward, who has 14+ years in the industry and has been working at Microsoft for over 10 years. Steve is a Technical Director for Business Applications at Microsoft and an active YouTuber who publishes content about various Power Platform tools such as PowerApps, Power BI, and Power Automate.

Introduction

Steve, a technophile, loves cooking in his spare time. His expertise and passion for Power Platform tools enables him to deliver these knowledge-expanding sessions.

Session Overview

In this session, Steve demonstrates a powerful solution combining AI Builder Form Processing with RPA to automate data extraction from forms and input them into a web application.

What We Will Cover:

AI Builder for form processing
Power Automate Desktop for RPA
Bringing it all together with Power Automate Cloud Flows

Detailed Walkthrough

The Solution Overview

The problem is the manual entry of information from scanned forms into a legacy web application. Steve's solution automates this process using AI Builder and Power Automate integrations.

Steps to Create the Solution:

AI Builder:
- Create a form processing model.
- Train the model with at least five sample forms.
- Validate the model’s accuracy.
Power Automate Desktop:
- Record the manual process of entering data into a web application.
- Create variables and replace hardcoded values with input variables.
- Test the recorded process and refine it with necessary adjustments.
Power Automate Cloud Flow:
- Integrate AI Builder’s trained model to extract data from forms.
- Implement a Cloud Flow to call the RPA process, providing extracted data as input.

Execution and Debugging:

Steve runs the Cloud Flow and showcases how AI Builder extracts the handwritten text accurately, and how Power Automate Desktop handles the legacy web application interaction seamlessly. He further demonstrates debugging and monitoring capabilities within Power Automate.

Available Resources

Steve shares his GitHub repository containing:

Word Document of the sign-up form
Sample forms (digitally filled and handwritten)
Solution file for easy setup

How To Get Started:

Download the solution from Steve’s GitHub.
Set up the environment using trial licenses for Power Automate, AI Builder, and Office 365 if you do not have an existing subscription.

Conclusion

This integration showcases the potential of combining AI with RPA to solve real-world problems efficiently. Steve’s demo is a perfect example of how technology can simplify tedious manual tasks.

Keywords

AI Builder
Power Automate Desktop
AI
RPA
Form Processing
Legacy Systems
Automation
Power Platform

FAQ

Q1: What is the use case presented in the session? A1: The session demonstrates automating the extraction and entry of data from handwritten forms into a legacy web application using AI Builder and Power Automate Desktop.

Q2: What tools are required to achieve this solution? A2: You need AI Builder for form processing, Power Automate Desktop for RPA tasks, and Power Automate Cloud Flows to integrate and orchestrate the process.

Q3: How can I access the resources used in the demo? A3: All resources, including sample forms and solution files, are available on Steve Winward’s GitHub repository.

Q4: Can I try this solution without a paid subscription? A4: Yes, you can sign up for trial licenses for Office 365, Power Automate, and AI Builder to explore and implement the solution.

Q5: How do AI Builder and Power Automate Desktop work together in this solution? A5: AI Builder extracts data from forms, and this data is fed into a Power Automate Desktop flow that automates entering the data into a web application.

Q6: What are the benefits of using AI Builder for form processing? A6: AI Builder simplifies data extraction, handles both digital and handwritten text, and reduces manual data entry errors, thereby streamlining the overall process.