Close Menu

    Deploy a full stack voice AI agent with Amazon Nova Sonic

    AI-powered speech solutions are transforming contact centers by enabling natural conversations between customers and AI agents, shortening wait times, and dramatically reducing operational costs—all without sacrificing the human-like interaction customers expect. With the recent launch of Amazon Nova Sonic in Amazon Bedrock, you can now build sophisticated conversational AI agents that communicate naturally through voice, without the need for separate speech recognition and text-to-speech components. Amazon Nova Sonic is a speech-to-speech model in Amazon Bedrock that enables real-time, human-like voice conversations.

    Whereas many early Amazon Nova Sonic implementations focused on local development, this solution provides a complete cloud-deployed architecture that you can use as a foundation for building real proof of concept applications. This asset is deployable through the AWS Cloud Development Kit (AWS CDK) and provides a foundation for building further Amazon Nova use cases using preconfigured infrastructure components, while allowing you to customize the architecture to address your specific business requirements.

    In this post, we show how to create an AI-powered call center agent for a fictional company called AnyTelco. The agent, named Telly, can handle customer inquiries about plans and services while accessing real-time customer data using custom tools implemented with the Model Context Protocol (MCP) framework.

    Solution overview

    The following diagram provides an overview of the deployable solution.

    The solution is composed of the following layers:

    • Frontend layer – The frontend layer of this system is built with scalability and performance in mind:
    • Communication layer – The communication layer facilitates seamless real-time interactions:
      • Network Load Balancer manages WebSocket connections. WebSockets enable two-way interactive communication sessions between a user’s browser and the server, which is essential for real-time audio streaming applications.
      • Amazon Cognito provides user authentication and JSON web token (JWT) validation. Amazon Cognito provides user authentication, authorization, and user management for web and mobile applications, alleviating the need to build and maintain your own identity systems.
    • Processing layer – The processing layer forms the computational backbone of the system:
      • Amazon Elastic Container Service (Amazon ECS) runs the containerized backend service.
      • AWS Fargate provides the serverless compute backend. Orchestration is provided by the Amazon ECS engine.
      • The Python backend processes audio streams and manages Amazon Nova Sonic interactions.
    • Intelligence layer – The intelligence layer uses AI and data technologies to power the core functionalities:
      • The Amazon Nova Sonic model in Amazon Bedrock handles speech processing.
      • Amazon DynamoDB stores customer information.
      • Amazon Bedrock Knowledge Bases connects foundation models (FMs) with your organization’s data sources, allowing AI applications to reference accurate, up-to-date information specific to your business.

    The following sequence diagram highlights the flow when a user initiates conversation. The user only signs in one time, but authentication Steps 3 and 4 happen every time the user starts a new session. The conversational loop in Steps 6–12 is repeated throughout the conversational interaction. Steps a–c only happen when the Amazon Nova Sonic agent decides to use a tool. In scenarios without tool use, the flow goes directly from Step 9 to Step 10.

    Prerequisites

    Before getting started, verify that you have the following:

    Deploy the solution

    You can find the solution and full deployment instructions on the GitHub repository. The solution uses the AWS CDK to automate infrastructure deployment. Use the following code terminal commands to get started in your AWS Command Line Interface (AWS CLI) environment:

    git clone https://github.com/aws-samples/sample-sonic-cdk-agent.git 
cd nova-s2s-call-center 

# Configure environment variables
cp template.env .env

# Edit .env with your settings

# Deploy the solution 
./deploy.sh

    The deployment creates two AWS CloudFormation stacks:

    • Network stack for virtual private cloud (VPC) and networking components
    • Stack for application resources

    The output of the second stack gives you a CloudFront distribution link, which takes you to the login page.

    You can create an Amazon Cognito admin user with the following AWS CLI command:

    aws cognito-idp admin-create-user 
  --user-pool-id YOUR_USER_POOL_ID 
  --username USERNAME 
  --user-attributes Name=email,Value=USER_EMAIL 
  --temporary-password TEMPORARY_PASSWORD 
  --region YOUR_AWS_REGION

    The preceding command uses the following parameters:

    • YOUR_USER_POOL_ID: The ID of your Amazon Cognito user pool
    • USERNAME: The desired user name for the user
    • USER_EMAIL: The email address of the user
    • TEMPORARY_PASSWORD: A temporary password for the user
    • YOUR_AWS_REGION: Your AWS Region (for example, us-east-1)

    Log in with your temporary password from the CloudFront distribution link, and you will be asked to set a new password.

    You can choose Start Session to start a conversation with your assistant. Experiment with prompts and different tools for your use case.

    Customizing the application

    A key feature of this solution is its flexibility—you can tailor the AI agent’s capabilities to your specific use case. The sample implementation demonstrates this extensibility through custom tools and knowledge integration:

    • Customer information lookup – Retrieves customer profile data from DynamoDB using phone numbers as keys
    • Knowledge base search – Queries an Amazon Bedrock knowledge base for company information, plan details, and pricing

    These features showcase how to enhance the functionality of Amazon Nova Sonic with external data sources and domain-specific knowledge. The architecture is designed for seamless customization in several key areas.

    Modifying the system prompt

    The solution includes a UI in which you can adjust the AI agent’s behavior by modifying its system prompt. This enables rapid iteration on the agent’s personality, knowledge base, and conversation style without redeploying the entire application.

    Adding new tools

    You can also extend the AI agent’s capabilities by implementing additional tools using the MCP framework. The process involves:

    • Implementing the tool logic, typically as a new Python module
    • Registering the tool with the MCP server by using the @mcp_server.tool custom decorator and defining the tool specification, including its name, description, and input schema in /backend/tools/mcp_tool_registry.py

    For example, the following code illustrates how to add a knowledge base lookup tool:

    @mcp_server.tool(
    name="lookup",
    description="Runs query against a knowledge base to retrieve information."
)
async def lookup_tool(
    query: Annotated[str, Field(description="the query to search")]
) -> dict:
    """Look up information in the knowledge base"""
    results = knowledge_base_lookup.main(query)
    return results

    The decorator handles registration with the MCP server, and the function body contains your tool’s implementation logic.

    Expanding the knowledge base

    The solution uses Amazon Bedrock Knowledge Bases to provide the AI agent with company-specific information. You can update this knowledge base with:

    • Frequently asked questions and their answers
    • Product catalogs and specifications
    • Company policies and procedures

    Clean up

    You can remove the stacks with the following command:

    # move to the cdk folder, assuming you are in the project root folder
cd cdk
# Removes both stacks sequentially
npx cdk destroy --all

    Conclusion

    AI agents are transforming how organizations approach customer service, with solutions offering the ability to handle multiple conversations simultaneously, provide consistent service around the clock, and scale instantly while maintaining quality and reducing operational costs. This solution makes those benefits accessible by providing a deployable foundation for Amazon Nova Sonic applications on AWS. The solution demonstrates how AI agents can effectively handle customer inquiries, access real-time data, and provide personalized service—all while maintaining the natural conversational flow that customers expect.

    By combining the Amazon Nova Sonic model with a robust cloud architecture, secure authentication, and flexible tool integration, organizations can quickly move from concept to proof of concept. This solution is not just helping build voice AI applications, it’s helping companies drive better customer satisfaction and productivity across a range of industries.

    To learn more, refer to the following resources:

    About the authors

    Reilly Manton is a Solutions Architect in AWS Telecoms Prototyping. He combines visionary thinking and technical expertise to build innovative solutions. Focusing on generative AI and machine learning, he empowers telco customers to enhance their technological capabilities.

    Shuto Araki is a Software Development Engineer at AWS. He works with customers in telecom industry focusing on AI security and networks. Outside of work, he enjoys cycling throughout the Netherlands.

    Ratan Kumar is a Principal Solutions Architect at Amazon Web Services.A trusted technology advisor with over 20 years of experience working across a range of industry domains, Ratan’s passion lies in empowering enterprise customers innovate and transform their business by unlocking the potential of AWS cloud.

    Chad Hendren is a Principal Solutions Architect at Amazon Web Services. His passion is AI/ML and Generative AI applied to Customer Experience. He is a published author and inventor with 30 years of telecommunications experience.

    Source: Read More 

    Related Posts

    Leave A Reply

    For security, use of Google's reCAPTCHA service is required which is subject to the Google Privacy Policy and Terms of Use.