Azur Games migrates all game analytics data to ClickHouse Cloud on AWS

Azur Games , a mobile game publisher and developer, recently achieved over 8 billion game installs across their portfolio of hypercasual games. In the beginning of 2024, Azur Games migrated all 120 TB of their active game telemetry data to ClickHouse Cloud on Amazon Web Services (AWS), a fully managed service that allows companies to easily operate and scale their high-performance analytical workloads without the burden of managing the underlying infrastructure.

Screenshot of Kingdom Clash, published by CASUAL AZUR GAMES.

A hypercasual game is a simple, easy-to-play mobile game with low barriers to entry. Typical hypercasual gameplay involves straightforward, one-thumb interactions that are quick to learn and play in short bursts. See the hit game Stack Ball 3D published by Azur Games. It is a mobile game where players smash through colorful rotating platforms to reach the bottom, all while avoiding black segments that cause the ball to shatter. Its simple yet addictive gameplay, combined with vibrant graphics and easy controls, makes it a quintessential example of the hypercasual genre.

Such fast-paced, simple games generate massive amounts of telemetry data from user interactions, which is crucial for understanding player behavior, optimizing monetization, experimenting, and improving the overall gaming experience.

Azur Games needed robust data infrastructure to store, process, and analyze the constant influx of data, informing their rapid product iterations and marketing strategies. For this task, several years ago, they chose the self-managed, on-premises installation of ClickHouse, a trusted high-performance real-time data warehouse.

But the existing ClickHouse database self-hosted on-premises faced several challenges. One of them was the ever-present reliability risk of consecutive disk failures on the 20+ underlying bare-metal servers. Another was limited data redundancy and risk of permanent data loss. Initially, Azur Games compromised on best practice redundancy and business continuity to mitigate total cost of ownership. Third, self-managed infrastructure with no auto-scaling caused performance bottlenecks during peak loads. In addition, maintenance, management and scaling of this on-premises solution, including hardware and software, was very complicated and time-consuming.

“We had 20 powerful ClickHouse DB-servers, 3 ZooKeeper-servers, 11 AirFlow-machines, 6 nodes hosting storage-emulator MinIO for AirFlow-cache, 4 MySQL DB-servers, and 4 BI backend servers and other auxiliary nodes. It was 7 major architectural units manually supported by two AirFlow-engineers and one of DevOps-engineers full-time. Software versions were 2+ years old, as we couldn’t easily keep them up-to-date. At some point we understood that it just takes too much time and effort. We had other parts of our infrastructure residing in AWS, so we decided to consider migration of our data to a cloud-based managed service,” says Vladimir Rudev, Lead Solution Architect, Azur Games.

Transition to ClickHouse Cloud on AWS

ClickHouse Cloud on AWS, a managed service by ClickHouse Inc., is available in AWS Marketplace and hosted on AWS, is an alternative to the self-hosted on-premises ClickHouse implementation. It provides a reliable and flexible platform for storing and processing massive amounts of data, with features like automatic data tiering, on-demand scaling, and enterprise-grade security. For companies like Azur Games, which operate huge databases of over 120TB, ClickHouse Cloud offers several key benefits. It abstracts away the complexities of infrastructure management, allowing the engineering team to focus on deriving insights and business value from the data. The service also provides high availability, data durability, and the ability to quickly adapt to changing data processing needs, all while optimizing costs based on actual usage. This empowers data-driven organizations to rapidly innovate and make decisions based on the most up-to-date information.

“As we already had parts of our infrastructure residing on AWS, and we had experience with ClickHouse, it was just logical for us to consider ClickHouse Cloud on AWS as the alternative. And after just three months of proof-of-concept, pilot production use, and some fine-tuning, we ended up with all our mission-critical data living in the cloud,” says Vladimir Rudev.

To accomplish this transition without downtime, Azur Games adopted a “full duplication, verification and atomic switchover” approach. The migration process involved carefully replicating the entire database to the new cloud-based infrastructure, thoroughly testing the new setup, and then seamlessly switching over to the ClickHouse Cloud solution. This meticulous planning and execution allowed Azur Games to migrate their massive data volumes to the managed cloud service with no interruption to their critical analytical capabilities. Auxiliary databases were migrated to Amazon RDS managed service.

Benefits of the cloud-based solution

The cloud-based solution provided several key benefits to Azur Games, including improved reliability and data durability, enhanced flexibility and scalability, faster ETL and data processing, as well as reduced operational overhead and maintenance burden.

“The migration to the cloud-based solution provided us several key benefits,” said Vladimir Rudev. “First, the overall system became simpler, faster, and more reliable. Second, the administrative layer between the code and the outcome became thinner—our team gained more control and capabilities that they can safely use without involving administrators. They can now add the required software in Python and scale up and down without relying on admins, reducing the intermediary steps in routine operations. Third, we gained access to performant and reliable cloud resources from AWS. This is particularly evident with our new Airflow, for which fast Amazon Simple Storage Service (Amazon S3) storage, network, and disks have significantly accelerated our ETLs. Faster ETLs mean faster access to information for decision-making by our product teams and marketing, which is a critical skill in the highly competitive mobile gaming market.”

Screenshot of Railroad Empire, published by Azur Interactive Games Limited.

By using ClickHouse Cloud, Azur Games gained access to the latest version of ClickHouse, which includes a wealth of new features and optimizations. This allows them to take advantage of the ongoing improvements in ClickHouse, such as enhanced analytical functions, storage optimizations, and ETL enhancements, without the burden of complex and risky software upgrades.

Also, with ClickHouse Cloud, Azur Games no longer fears experimenting with their data infrastructure. The ability to quickly scale resources up and down has eliminated the previous constraints. Now, the team can easily test new configurations and optimizations without the risk of overloading the system. This has opened up opportunities for continuous improvement of their data processing capabilities.

“The modern game company is a data company,” said Arno van Driel, VP EMEA at ClickHouse. “We see a massive shift from on-premise database deployments to the usage of real-time data warehouses in cloud providers. The ability to blend high-performance analytics—across a variety of datasets—with player data lets companies like Azur Games focus on their speciality…the user experience. We can focus on ours…delivering the fastest real-time data warehouse via the most adopted cloud provider.”

Cost and resource optimization

The overall cost of operating the ClickHouse Cloud solution remained comparable to the previous self-hosted setup. However, the way Azur Games manages the costs has changed significantly.

In the cloud-based model, Azur Games now pays for the exact resources they consume, such as data transfer, processing power, and storage. This “pay-as-you-go” approach can be more cost-effective, as the company can scale resources up and down as needed, without the fixed overhead of maintaining their own on-premise infrastructure.

Additionally, Azur Games can leverage cost optimization strategies in the cloud, such as AWS Savings Plans or Reserved Instances, to further optimize their operating cost. This flexibility and granular control over how they use their resources has enabled them to keep the overall costs at a similar level to the previous setup, while gaining significant benefits in terms of reliability and scalability (as described above), and in reduced operational burden.

“The key difference is that in the cloud, we need to be more mindful of our resource consumption and implement efficient algorithms and architectures to maximize the cost-effectiveness of our data processing workloads. This new “cloud reality” has required us to re-evaluate our approach, but the benefits in terms of agility and productivity have offset the need for this additional cost optimization effort big time,” says Vladimir Rudev.

Business impact

The migration to ClickHouse Cloud on AWS has delivered substantial time savings for the engineering teams in Azur Games, allowing them to focus on more strategic initiatives, as well as faster delivery of new features and business insights from improved data processing capabilities. Additionally, the team can now easily experiment and optimize the data infrastructure without the constraints they had previously.

“The main benefit is the time savings for our employees,” says Vladimir Rudev. “One of our administrators now has 60% of their time freed up for new projects, and our ETL engineer saves 40% of their time. This allows us to redirect our focus to more interesting and strategic tasks, rather than constantly maintaining complex on-premises infrastructure.”

The AWS for Games Solutions Library includes curated AWS and Partner Solutions to support all aspects of game development, operation and publishing, and our team of specialists is happy to help any game developers build on AWS.