How To Build a $5,000 AI Voice Assistant For FREE With Vapi

Introduction

Today, I'm going to show you how we can build an AI Voice Assistant from start to finish that will blow your mind. Right now, AI voice assistants are the hot new AI topic and they are honestly scary good. They can do anything from onboarding people, answering questions, to booking meetings. I'm going to show you my favorite way of how I build these and how you can get started.

Hello guys, I'm Mike and I'm here to help you make more money and save more time with all these AI tools. Right now, my agency focuses on making voice assistants much like the one you're going to see in this video. So if you want a fully custom voice assistant to do anything you can dream of, make sure to book a time with me down in the description below and we can talk about it there. Before I get into it, let me show you just a sneak peek little demo of what we're going to be building in today's video.

Sneak Peek Demo

User: Welcome to Mike's Diner, this is Mary. How can I help you today?

Customer: Hey Mary, do you guys accept reservations at Mike's Diner?

Mary: We don't accept reservations due to the casual nature of the diner. However, our friendly staff will do their best to accommodate your party as quickly as possible. Is there anything else I can help you with today?

Customer: Um, yeah, what time you guys open?

Mary: Mike’s Diner is open seven days a week. Monday to Friday from 6:00 a.m. to 9:00 p.m. and Saturday and Sunday from 7:00 a.m. to 10:00 p.m. Is there anything else I can help you with today?

Customer: Um, no, that'll be all.

Mary: Okay, thanks for choosing Mike's Diner. Have a good day.

Awesome, so enough talking, let's get into building.

Getting Started with Vapi

The tool that we're using to build all of these crazy cool AI phone bots is called Vapi and Vapi, in my opinion, is the state-of-the-art, the best way to go about building phone bots right now as of May 2nd, 2024. They have a super clean dashboard and a really flexible building structure to allow you to build these voice bots to do whatever you want.

You can give it a demo just by heading on over to the Vapi website and clicking on the demo button. They also mention on their website that they can handle a wide variety of tasks. One of the big things that I commend Vapi for working on is the latency. If you've ever seen those demos online where it sounds like you're talking to a real person, that's because the latency of the phone bot from when it gets transcribed by the AI is just so low. Vapi’s number one priority is to get this latency time down as much as possible so it just sounds like a natural person when you're talking to it. And what's amazing is you can get started for free. I'll have a link down in the description below to where you can follow along with this tutorial.

Let's head on over to the dashboard and get started with Vapi.

Vapi Dashboard Overview

Once you're inside the Vapi dashboard, all we're going to want to do is head over to the "Create New Assistant" button. We can pick from some of the predefined template assistants they have for us. I'm going to go with this appointment setter here and click on continue. We can name her if we would like—I'll keep it on Mary and click on create Mary. This will set up a very quick dummy demo bot to kind of show you what these phone bots can do.

To get started talking with Mary, all we have to do is click on "Talk with Mary". Allow Vapi to use your microphone in your browser.

User: Hello, this is Mary from Mary's Dental. How can I assist you today?

User: Hey Mary, what time are you guys open?

Mary: Hey there, we're open from 8:00 a.m. to 5:00 p.m., Monday through Saturday. Closed on Sundays though because even dental offices need a day off, am I right?

User: Alright Mary, that's great.

User ends call.

As you can see, there's some fine-tuning we need to do to make this a little bit better, but you get the idea. Here’s how this works.

Tabs Overview

Model Tab
- Prompt and AI Framework: This is where we're going to have our prompt and our kind of AI framework of how this is all going to work.
- First Message: This message will be said exactly as it is spelled out every time someone connects with our phone bot.
- System Prompt: All the guts and juicy bits of our AI phone bot which will explain how it functions, responds to people, and the user interacts with the bot.
Choosing Models
- You can select from various models for our bot, but I found OpenAI to be the most reliable.
- You can select between GPT-3.5 or GPT-4, I prefer GPT-4 for better performance.
- Cost per minute updates according to the chosen model.
- Breakdown of various costs such as Vapi fix cost, speech to text model, large language model processing, and text to speech.
Latency
- This measures the time in milliseconds it takes for the bot to respond from the user’s last word. Adjusting different models affects the latency.
Transcriber Tab
- I like to keep it on Deepgram for speech-to-text as it’s cost-effective.
Voice Tab
- Uses 11 Labs, considered state-of-the-art for text-to-speech models.
- You can select voices, add your custom voice ID, adjust audio settings like background noise and change model parameters.
Functions Tab
- Add API webhooks inside your phone bot for extra functionalities like sending texts or booking meetings.
- Predefined functions like ending the call, forwarding phone number, etc.
Advanced Dashboard
- Contains options such as HIPAA compliance, enable recording, silence timeout, response delay, and more.
- Configure server URL for messaging, set voicemail message for outbound calls, and end-call messages.

Building a Custom Bot

Adding a Phone Number

You can either buy a number from Vapi or import a number from platforms like Twilio or Vonage.

Creating a Custom Voice

Sync your 11 Labs account to Vapi and use your custom voice's ID by entering it manually into your assistant’s settings.

Review Call Logs

Every call is recorded along with audio, the transcript, summary, cost breakdown, and other relevant metadata for in-depth analysis.

Creating a Restaurant Assistant Example

Let’s make a restaurant assistant for Mike’s Diner:

Create a New Assistant: Name it "Mike’s Diner Assistant".

First Message:

Welcome to Mike's Diner!!.., This is Mary. How can I help you today?

System Prompt:

You're Mary, a helpful assistant for Mike's Diner. Be polite, sweet, and straightforward. Answer any questions using the information below. If you cannot answer, say "I'm sorry but I do not have enough information to answer that. Is there anything else I can help you with today?"

Q&A:

- Where is Mike’s Diner located?
  - Mike’s Diner is located at 123 Main Street in Millersville, NY.

- What are Mike’s Diner's hours?
  - Monday to Friday: 6:00 a.m. – 9:00 p.m.
  - Saturday to Sunday: 7:00 a.m. – 10:00 p.m.

Publishing and Testing: Ensure the bot responses are appropriate and functions like call ending are working.

Final Testing

User: Welcome to Mike's Diner, this is Mary. How can I help you today?

Customer: Hey Mary, what time you guys open?

Mary: Mike's Diner is open seven days a week. Monday to Friday from 6:00 a.m. to 9:00 p.m. and Saturday and Sunday from 7:00 a.m. to 10:00 p.m. Is there anything else I can help you with today?

Customer: No, that's all.