Why QuickSight increases CloudTrail cost so much?

Does QuickSight increase CloudTrail cost? In short, it does. But it could be either minor or major. It really depends on your use case.

As mentioned in another post AWS Cost Anomaly Detection, we noticed a dramatic CloudTrail cost in one of our AWS accounts.

After some diggings, it turns out it was caused by the use of QuickSight. Here is what I found in this account:

• CloudTrail has been configured to keep track of both the management and data events (S3).
• Some QuickSight datasets’ query mode are configured as Direct Query.

In case you are not familiar with AWS QuickSight. Simply speaking, it is a Business Intelligence tool which allow users to visualise the data and get some insights from it. Users can create Analyses against the Datasets, and share the Analyses as Dashboards.

The Dataset can be created from many different data sources. For example, files(.csv, .json…), databases, (mysql, postgresql, sql server…), jira, github, Athena and more. And there are two types of query modes:

• Direct query – To run the SELECT statement directly against the database
• SPICE – To run the SELECT statement against data that was previously stored in memory

In our case, the dataset uses Athena as the data source, which is an interactive query service that analyses data in Amazon S3 using standard SQL, and stores the database in S3.

By now, it all makes sense, right? Whenever users refresh the QuickSight dashboard, QuickSight kicks off the Athena query directly against the database. As the Athena database is stored in the S3, there are many S3 API calls (e.g GetObject) need to made behind the scene. Also those S3 API calls are tracked by CloudTrail as data events which are way more than the QuickSight management events that are recorded by the CloudTrail (more events mean more cost).

What is the solution to reduce the CloudTrail data events cost? As Direct Query mode is not really necessary in most cases. The common approach is to use SPICE mode which queries against the data that was previously stored in memory. And creating a schedule to periodically refresh the data in memory to keep it up to date, or use the event-driven refresh solution.