Build a business visualization solution for Amazon Quick

0 1 6 minutes read

Build a business visualization solution for Amazon Quick

When hundreds to thousands of users are onboard an enterprise AI platform, business leaders and platform owners need visibility into who is using the platform, whether users are satisfied with the responses they're getting, and which capabilities are driving the most engagement. Without a centralized visibility solution, this data is distributed across multiple AWS services and is difficult to analyze at scale.

Amazon Quick is an AI-powered productivity platform that combines Spaces, Chat agents, Flows, Automate, Research, and Amazon Quick Sight's business intelligence capabilities in one place. As organizations scale their Amazon Quick shipments, they need a reliable way to track receipts, measure satisfaction, monitor costs, and manage accounts in a single window of glass.

In this post, we show you how to export a solution that integrates Amazon Quick performance data from Amazon CloudWatch transaction logs and AWS CloudTrail events into a secure data pool in Amazon Simple Storage Service (Amazon S3) that can be queried using Amazon Athena, the Quick Sight dashboard, and a custom Quick Sight agent.

Solution overview

Amazon Quick publishes usage and interaction data through sales logs to bring chat conversations, user feedback, agent usage/research hours, and inventory usage to Amazon Quick. Amazon Quick is integrated with AWS CloudTrail, which provides a record of actions taken by a user, role, or AWS service in Amazon Quick.

Figure 1: Amazon Quick Enterprise observability solution architecture

The workflow consists of the following steps:

Business users interact with Amazon Quick.
Amazon Quick publishes transaction logs to Amazon CloudWatch transaction logs. You can protect these logs with data protection policies to hide sensitive data, such as credentials (private keys, private AWS access keys), financial information, personally identifiable information, protected health information, and device identifiers.
CloudWatch subscription filters forward log events to Amazon Data Firehose delivery streams. The Firehose delivery stream transforms data using an AWS Lambda function and writes it to a data pool in Amazon S3.
An Amazon EventBridge rule routes Amazon Quick API calls from AWS CloudTrail and sends them to a dedicated Firehose delivery stream. The Firehose delivery stream transforms the data using an AWS Lambda function and writes it to the data pool.
The AWS Glue data catalog stores data pool metadata for Amazon Athena external tables and analytic views.
Administrators can use Amazon Athena to query data. AWS Lake Formation provides clean database permissions at the table and column level.
Business leaders and stakeholders can visualize data on the Quick Sight dashboard to collaboratively monitor acquisition, satisfaction, cost, and management data. They can also use a fast custom chat agent with natural language questions to get instant visual answers.

The solution encrypts data at rest using a customer key managed by AWS Key Management System (AWS KMS) with automatic key rotation. The solution encrypts Amazon CloudWatch Log Groups, Amazon Data Firehose delivery streams, AWS Lambda workspace variables, and Amazon S3 data pool. This provides a unified encryption strategy for all pipelines.

What is required

To use this solution, you need:

Submit the solution

The deployment is organized into steps, each building on the previous one. You can stop after any step and have a solution that works at that level. Settings such as the AWS CLI profile, resource prefix, database name, and workgroup name are saved locally after each step, so the next steps fill in automatically.

Remove the storage area

Close the GitHub repository and navigate to the project directory:

git clone 

cd sample-quick-observability-platform

Set up timber for sale

Deploy Amazon CloudWatch Logs infrastructure:

The script automatically detects your Instant subscription location, creates an AWS KMS key, and configures delivery of sales logs for chat, response, agent hours, and index usage data.

The deployment instructs CloudWatch log groups to be created (/aws/vendedlogs/quick/chat, /aws/vendedlogs/quick/feedback, /aws/vendedlogs/quick/agent-hours, /aws/vendedlogs/quick/index-usage). It also displays the prefix (quickobserve) to create other AWS services.

Chat message content (user_message and system_text_message) may contain sensitive or controlled data from connected enterprise sources such as databases, Amazon S3 buckets, or third-party integrations. Before enabling message content logging, review your organization's data privacy, compliance, and data retention policies. Chat message content is automatically omitted so that no user chat data reaches CloudWatch Logs. Posting prompts you if you want to log the content of the chat message.

Verify CloudWatch commercial log groups in the AWS console:

Use a data pipeline

Use the following command to run the pipe:

python3 deploy.py --pipeline

This uses an Amazon S3 data pool, Amazon CloudWatch Logs subscription filters, Amazon Data Firehose delivery streams, AWS Lambda functions and Amazon EventBridge rules.

You can see log data in the Amazon S3 data pool (quickobserve-pipeline-datalake-).

Set up a data catalog

Use the following command to run the data catalog setup:

python3 deploy.py --datacatalog

The script asks for the name of the database (quickobserve_db) and ensures that it is not already available in the AWS Glue Data Catalog, preventing erroneous changes to tables for other operations. It then prompts you to choose how access to the data pool is controlled:

Pool Creation (default) – Registers the data pool environment and grants fine-grained permissions to the Amazon Instant Service role at the table and column level. When message content logging is enabled, column-level exclusion prevents message content from flowing into the Quick Sight dashboard and header.
IAM Policies – Skips AWS Lake Formation setup and relies on IAM policies to control access. Use this if your account does not use Lake Formation.

The script creates an AWS Glue Data Catalog database, Athena tables and views of CloudWatch transaction logs and CloudTrail events. You can see the data catalog in AWS Glue:

Verify that the data is flowing by running the following queries in the Amazon Athena query editor:

SELECT * FROM "quickobserve_db"."agent_hours_logs" ;

SELECT * FROM "quickobserve_db"."chat_logs" ;

SELECT * FROM "quickobserve_db"."cloudtrail_events" ;

SELECT * FROM "quickobserve_db"."feedback_logs" ;

SELECT * FROM "quickobserve_db"."index_usage_logs" ;

SELECT * FROM "quickobserve_db"."cloudtrail_events" ;

Use the Quick View dashboard

Use the Quick Sight dashboard:

python3 deploy.py --dashboard

This uses Instant Insights resources: a custom theme, data source, datasets with a daily refresh schedule, analytics, and a dashboard to view Amazon Instant Insights metrics.

You can see visibility metrics in the Quick Sight dashboard:

Sign in to the Amazon Quick console.
In the left navigation menu, select Dashboardsand select Quick View Dashboard.

Each sheet in the dashboard includes date range parameter controls and a table of details below. Selecting any chart, pie slice, or KPI filters the detail table to display matching records.

Create a Quick View topic

Use the following command to create a Quick View topic:

python3 scripts/create_topic.py

The script verifies that each dataset contains data from a successful import, then creates a Quick View topic with custom commands that route queries to the appropriate dataset. You can see the Quick View Article in the Amazon Quick console.

Sign in to the Amazon Quick console.
In the left navigation menu, select Articlesand select Quick View Topic.

Create a custom instant chat agent

This step is done through the Amazon Quick console.

In the left navigation menu, select Vacanciesand select Create space.
On the space creation page that opens, enter ua name again definition for your space.
Select Add information to start adding content to your space.
From the menu, select Articles.
In Add a titlechoose Quick View Topic.

Create a custom instant chat agent:

In the left navigation menu, select Negotiation agentsand select Create a chat agent.
On the create agent page that opens, enter name again definition for your agent.
Underneath Instructionspaste the command from the GitHub repository.
Underneath Sources of informationchoose Connect the Gaps and select Quick Observability Space.
Select Launch the chat agent to publish an agent to the chat agent library and use it in a chat.

Business leaders can ask questions such as “What are the most frequently used Amazon features in the last 30 days?”

They'll get instant visual feedback through metrics, charts, and actionable recommendations.

Clean up

To clean up your used resources, use the cleanup script:

python3 cleanup.py

The conclusion

In this post, we've shown how to implement a visibility solution that integrates Amazon Quick performance data into a secure data pool. The solution makes engagement metrics, user feedback, agent hourly usage, storage index usage, and management events accessible through Amazon Athena, the Amazon Quick Sight dashboard, and the Amazon Quick custom chat agent. tools.

To get started, you can link to a GitHub repository and run the solution.