ad
ad

my local, AI Voice Assistant (I replaced Alexa!!)

Science & Technology

Introduction

I’m done with cloud-based voice assistants! After years of using Alexa, I’ve decided to build my very own fully local AI-powered voice assistant. I was inspired by my frustrations with Alexa's limitations and the constant need for internet connection. Thus, I set out to create an assistant that respects my privacy and operates entirely on local hardware.

The Vision

The dream is to connect my voice assistant to my AI server, affectionately named Terry. I want Terry to be able to understand commands and respond in his own unique voice. The good news? With a bit of effort, this isn’t too hard to implement.

Parts List

Level One: Basic and Easy:

  • Raspberry Pi
  • Microphone (e.g., Blue Yeti)
  • Speaker
  • Phillips Hue Lights

Advanced Level:

  • LLM compatible with local deployment (e.g., Llama 3)
  • Additional hardware as needed for advanced functionalities.

Initiating the Project

In my previous video, I discussed installing Home Assistant on a Raspberry Pi. Home Assistant is a fully local home automation software that I found essential for this project. We can integrate voice capabilities into Home Assistant using a new open-source tool called Raspy.

Raspy enables various voice assistant services to work offline, including:

Setting Up Voice Assistant

I installed Whisper and Piper as add-ons in Home Assistant. I also configured a microphone add-on to capture voice input. Given the simplicity of the initial setup, I could switch from cloud-based services to a fully local assistant environment.

Configuration Steps

  1. Install the Whisper add-on in Home Assistant.
  2. Set up Piper for TTS.
  3. Install Open Wake Word to wake the assistant.
  4. Configure the assistant to respond using the entity names of my connected devices.

Once everything was set up, it was gratifying to see the voice assistant respond to commands and interact with my home automation system.

Expanding Capabilities with Wyoming Protocol

To extend the voice assistant capabilities, I wanted to utilize the Wyoming protocol. This allows me to connect multiple Raspberry Pi(s) or other devices as remote voice assistants. I also wanted to serve the audio input/output conveniently through a microphone hat.

Setting Up Wyoming Protocol

  1. Install the necessary libraries and clone the Wyoming protocol repository.
  2. Set up a service for the Wyoming satellite on each Raspberry Pi, ensuring continuous functionality.
  3. Configure the Home Assistant to recognize these Wyoming satellites.

Once configured, I verified its functionality—turning lights on and off through voice commands. Great, but I desired a smarter interaction.

Level Up to Local LLM and Offloading Processing

To enhance the intelligence of my assistant, I integrated an LLM model, Llama 3. After trying it on my main machine (my laptop), I realized I could offload Whisper and Piper as Docker containers, allowing heavier processing and improved speeds.

  1. Set up Llama 3 to run on my local server.
  2. Configure Home Assistant to process all commands through this enhanced model.
  3. Integrate Whisper and Piper on beefier hardware, enabling my voice assistant to read texts and respond intelligently.

By creating automations in Home Assistant, I expanded the assistant’s functionality, continuously improving its ability to interface seamlessly with my environment.

Customizing Terry

I realized that naming my assistant “Terry” and giving it a customized wake word was essential. Additionally, I found ways to create a more personalized voice for Terry but had some challenges implementing this. While I could get it to respond to custom voice commands, the initialization of voice parameters took a bit longer than expected.

Future Enhancements

In the next phases of this project, I plan to:

  • Finalize Terry’s unique customized voice.
  • Optimize and speed up processing using advanced hardware components.

Building a fully local AI voice assistant has proven exhilarating and certainly a worthwhile endeavor. It’s liberating to control my environment without reliance on cloud services, and I look forward to sharing more updates on this journey!

Keywords

  • Local AI Voice Assistant
  • Home Assistant
  • Raspberry Pi
  • Whisper
  • Piper
  • Llama 3
  • Wyoming Protocol
  • Voice Recognition
  • Automation

FAQ

Q1: Why did you decide to create a local voice assistant?

A1: I found cloud-based assistants like Alexa to be limited and intrusive, leading me to want a voice assistant that works entirely offline, respecting my privacy.

Q2: What hardware do I need to build a local assistant?

A2: At a basic level, you'll need a Raspberry Pi, a microphone, and speakers. Advanced setups can incorporate LLMs for more intelligent interactions.

Q3: How does the Wyoming Protocol work?

A3: The Wyoming Protocol allows multiple devices to communicate seamlessly, acting as remote voice assistants that connect back to a central server, such as Home Assistant.

Q4: How do you manage voice recognition for your assistant?

A4: I utilize tools like Whisper for speech-to-text recognition and Piper for text-to-speech, both running locally on hardware without external internet dependencies.

Q5: What future enhancements do you plan for your assistant?

A5: I plan to improve Terry's voice recognition capabilities, finalize personalized voice settings, and optimize the overall performance of the assistant.