Game Developer’s Guide to Amazon DocumentDB Part 1: An Overview

Introduction

Document database is one of the fastest growing categories of nonrelational database that makes it easier for game developers to store and query data as JSON-like documents. The flexible, semi-structured, and hierarchical nature of document database works well with game use cases such as player profiles and in-game items, and allows game developers to iterate faster to meet players’ demands. As a document database, Amazon DocumentDB (with MongoDB compatibility) is fast, scalable and highly available. Its flexible schema, extensive query capabilities, durability, scalability, and ease of use makes it a great database engine for supporting mission-critical game workloads at scale.

We launched Amazon DocumentDB in 2019. Since launch, we have continued to work backwards from our customers’ needs and have delivered an additional 80+ capabilities, including MongoDB 4.0 and 5.0 compatibility, support for ACID transactions, and more recently, support for sharding with Elastic Cluster. While Amazon DocumentDB supports a vast majority of the MongoDB APIs that customers actually use, it does not support every MongoDB API. Our focus has been to deliver the capabilities that customer actually use and need.

DMM GAMES is a Japanese publisher and developer of a wide variety of popular online games and here is their testimony on the values that Amazon DocumentDB has delivered for them. “DMM GAMES uses Amazon DocumentDB (with MongoDB compatibility) to store user data, friend data and battle data for two of our popular game titles. Both games experience heavy traffic which can be as high as tens of millions of requests/day. And to match our growing user base of over 200,000+ users per month, we need a database that can scale quickly and efficiently. Amazon DocumentDB is a fully managed database, so rather than manage the underlying infrastructure, our engineers can focus on developing better games. In addition, the service scales quickly and easily, which gives us the flexibility to respond to critical events when we expect to see high traffic. As a result, we see better performance and lower cost with very little downtime.”

In this blog post, we’re going to highlight some key features of Amazon DocumentDB that you should know as a game developer and help you get started quickly if you decide to adopt Amazon DocumentDB as the database engine for your next project in games.

What’s Amazon DocumentDB

Amazon DocumentDB was built from the ground up with a cloud-native database architecture to solve common database challenges around reliability, scalability, performance and manageability. We want to focus on building capabilities to remove undifferentiated heavy lifting and reduce costs.

Amazon DocumentDB is MongoDB compatible allowing game developers to use the same MongoDB application code, drivers, and tools as they do today to run, manage, and scale workloads on Amazon DocumentDB. Learn more DocumentDB Supported MongoDB APIs, Operations, and Data Types. You can use AWS Database Migration Service (DMS) to migrate your data to Amazon DocumentDB with virtually no downtime. The DMS free tier provides 750 hours of Amazon DMS Single-AZ dms.t2.micro instance usage for your migration.

Amazon DocumentDB was built from the ground up with a cloud-native database architecture. Its unique architecture separates storage and compute so that each layer can scale independently. Amazon DocumentDB uses a purpose-built, distributed, fault-tolerant, self-healing storage system that is highly available and durable by replicating data six ways across three AWS Availability Zones (AZs).

With Amazon DocumentDB, you don’t need to worry about database administrative tasks such as hardware provisioning, patching, or backups. Amazon DocumentDB continuously monitors database health and automatically fails over to a read replica in the event of a failure, typically in less than 30 seconds. Amazon DocumentDB continually backs up data to Amazon S3 and provides up to 35 days of point-in-time recovery with no downtime or performance degradation.

DocumentDB provides multiple levels of security for your database, including network isolation using Amazon VPC, encryption at rest using AWS Key Management Service (KMS), and encryption-in-transit using TLS. DocumentDB provides a variety of CloudWatch metrics that you can monitor to determine the health and performance of your clusters and instances.

Game use cases well-suited for Amazon DocumentDB

Why should you consider using Amazon DocumentDB for your gaming workloads? As a document database, Amazon DocumentDB makes it easy to store, query, and index JSON data. Amazon DocumentDB offers a very flexible and simplified data model which means you don’t have to know upfront how your schema is going to evolve over time. This allows you to quickly iterate on your game and continuously add new features or new items to keep your players engaged. With Amazon DocumentDB, you also have the ability to perform ad hoc queries and create flexible indexes, for example, DocumentDB supports geospatial, compound, multi-key, sparse and TTL indexes. In addition to performing operational workloads of CRUD operations, there are many powerful built-in aggregation pipeline capabilities for processing and transforming data.

One of the top game use case for Amazon DocumentDB is player profile. DocumentDB’s flexible document data model allows you to continuously adding new features such as achievements, in-game items and gears. Another use case is to store information on inventory and players’ entitlements, for which you needs low-latency reads and writes at scale and oftentimes strong consistency as well. Delays in those operations can cause frustration for players and potentially lead to churns. Matchmaking is a very critical function in multiplayer online games and players need to be matched reliably and in a timely manner for the purpose of delivering an optimal experience. Amazon DocumentDB offers powerful aggregation pipeline framework that allows you to perform real-time analytics on player metrics and put the right players in a session. Amazon DocumentDB can also be used to capture telemetry data such as game events and perform real-time analytics, allowing game developers to track player behavior, performance and engagement and dynamically optimize player experiences.

Additionally, Amazon DocumentDB can be used to support other parts of game ecosystem. For example, building leaderboards, managing user-generated content, messaging for real-time interactions, driving marketing campaign, and creating merchandise catalog. These are just some more common use cases and there are many other opportunities that Amazon DocumentDB can help unlock.

Architecture overview

Amazon DocumentDB is designed for the cloud and to avoid the limitations of traditional database architectures. In the following picture you can see the compute and storage layers are decoupled, and can be scaled independently.

Because of this architecture Amazon DocumentDB supports both vertical and horizontal scaling. You can scale vertically by increasing the size of your instances. You can scale horizontally by adding up to 15 read replicas, supporting millions of requests per second. The primary instance and read replicas share the same storage, and read replicas can be added in a few minutes with minimal impact on database availability.

Amazon DocumentDB instances are available in various sizes, starting 2 vCPUs and 4-GiB RAM, to the 96 vCPUs and 768-GiB RAM. You choose an appropriate instance type based on the RAM, vCPU, and network throughput required. Compute scaling operations typically complete in a few minutes irrespective of the size of your data. Scaling does not require any copying of data because the storage and compute layers are decoupled in Amazon DocumentDB. Scaling up is useful if you want to scale your write capacity or to provision a larger read replica instance for running read-only analytics workloads.

With Amazon DocumentDB, unlike traditional databases, you do not have to provision storage space explicitly while creating the database. Amazon DocumentDB data is stored in an SSD-backed virtual volume (cluster volume) that automatically grows as the amount of data in the database increases. The volume grows in increments of 10 GiB up to a maximum of 128 TiB. This process is transparent to your application, without any impact on application availability or performance.

The cluster volume spans three Availability Zones in a single AWS Region, and each Availability Zone contains two copies of the cluster volume data. This functionality means that Amazon DocumentDB can transparently handle the loss of up to two data copies or an Availability Zone failure without losing write availability, or the loss of up to three data copies without losing read availability.

This architecture also provides near instant crash recovery. Unlike other databases, Amazon DocumentDB does not need to replay redo logs after a crash before making the database available for operations. The storage is organized in many small segments and Amazon DocumentDB can perform crash recovery asynchronously on parallel threads. This approach reduces database restart times to less than 30 seconds in most cases.

How to migrate to Amazon DocumentDB

When choosing Amazon DocumentDB, you have various options to migrate existing workloads over. Existing data sources do not need to be MongoDB compatible in order to migrate over to DocumentDB through AWS Database Migration Service (DMS). If you do have a MongoDB compatible data source, you can migrate to DocumentDB without the use of DMS. Below we have laid out two main options in order to migrate your data sources to DocumentDB.

From MongoDB compatible environment to Amazon DocumentDB.

When migrating from an existing MongoDB compatible databases to DocumentDB one could leverage DMS, but command line utilities are also available with Mongo compatible data sources. Such commands are mongodump, mongorestore, mongoexport, and mongoimport. Mongodump and mongorestore are typically used as more performant options while as mongoexport and mongoimport are used when you need more human-readability of the data during the migration process and tends to be slower.

From Non MongoDB compatible environment to Amazon DocumentDB.

Sometimes we need to start from a non MongoDB compatible data source. AWS offers Database Migration Services (DMS) to help map existing data source to new ones such as DocumentDB. DMS can be used as a one time service or it can be configured as a replication service until you are ready to cut ties with original data source. DMS makes it possible to migrate from relational databases, data warehouses, NoSQL databases, and other types of data stores.

NOTE: For either DMS or command line options, its is recommend that you first create indexes in Amazon DocumentDB before beginning your migration as it can reduce the overall time and increase the speed of the migration. Learn more Amazon DocumentDB Index Tool.

When migrating with either path (Non MongoDB compatible or MongoDB compatible) it’s important to plan out your steps. Outlined in this developer guide walks through the following in great detail:

1. Migration tool/method selection – Usage of DMS and/or command line.
2. Discovery work – Identifying architectural details and operational characteristics that lead to migration approach and cluster sizing.
3. Planning for availability, performance, and recovery
4. Migration approaches: Offline, Online, Hybrid
5. Testing – Performance and Fail-over Testing

For a real-world example of customer’s production workload migration, read Venmo’s process of migrating to Amazon DocumentDB (with MongoDB compatibility)

How pricing works

As a managed service, you only pay for what you use, and there are no upfront costs. Amazon DocumentDB pricing has four dimensions:

1. On-demand instances: The amount of compute instances for a cluster (pricing per second with a 10-minute minimum). On-demand instances let you pay per second, with no long-term commitments or upfront fees.
2. Database I/O: The amount of I/O used when reading and writing data to your cluster’s storage volume (pricing per million I/Os). I/Os are input/output operations performed by the Amazon DocumentDB engine when performing reads and writes against cluster’s storage volume.
3. Database storage: The amount of data stored in your cluster’s storage volume (pricing per GB/month). Billable storage includes your data, indexes, and change stream data.
4. Backup storage: The amount of backup storage used in excess of your cluster’s database storage usage (pricing per GB/month). Increasing your backup retention period or taking manual cluster snapshots increases the backup storage consumed.

See Amazon DocumentDB (with MongoDB compatibility) pricing for more information.

How to get started with Amazon DocumentDB

To better understand how DocumentDB works along with discovering various case studies, faq, pricing, etc follow AWS getting starting link for more details. Furthermore, there is a free-tier available. As long as your organization hasn’t created a DocumentDB cluster, you are eligible for a free trial. The free DocumentDB trial includes 750 hours per month of db.t3.medium instance usage, 30 million IOs, 5 GB of storage, and 5 GB of backup storage for free for 30 days. After the 30 day free trial expires or if you exceed the allowance usage; you can chose to shut down your cluster to avoid any charges or you may keep it running out our standard on-demand rates.

NOTE: The DocumentDB free trial is not available in the in the AWS GovCloud (US) regions or the China (Ningxia) region at this time.

If you are having an issue within your account and need someone to examine the issues within your account, the first action would be to open a support ticket with AWS Support.

If you are wanting further guidance with your DocumentDB journey, feel free to explore these other links below. We wanted to consolidate as many guides and links as possible to reduce your searching.

• Amazon DocumentDB launches, blogs, and videos
  Find the latest Amazon DocumentDB launches, blogs, and webinars.
• Amazon DocumentDB Documentation
  We are constantly updating our technical documentation and best practices based on your feedback.
• MongoDB API compatibility and functional difference
  You can find the most up-to-date list of supported MongoDB APIs and functional differences. We are constantly listening and working backwards from our customers to deliver the capabilities that they want the most
• Ask a question on the Amazon DocumentDB forum
  Use the Amazon DocumentDB AWS Forums to learn, share knowledge, and get answers from a community of developers and the service team.
• Ask questions and seek answers on repost
• Getting Started Guide
  Detailed guild that walks you step by step on how to get started.
• Free trial available
  Earlier we mentioned there is a free tier available, this link gives you all the details you should need for the free trial.
• Another blog post walking through some first steps
  A great post that walks through some first steps in setting up DocumentDB.
• Another top 10 Dev Guide
  Consolidated top 10 developers guide of posts and other links to topics (such as connecting to DocumentDB and many others).
• Workshop that you can follow through
  Do you want to get your hands on DocumentDB today but would like a walkthrough process? Let this workshop be your guide!

Summary

We hope this blog has provided you with fundamental knowledge about Amazon DocumentDB and helped you understand where in your game development workflow it can be utilized. You can follow this video for a console walkthrough of Amazon DocumentDB key features. In Part 2 of this blog series, we’re going to introduce some best practices for using Amazon DocumentDB to build gaming applications. Stay tuned!

About the Authors

	Jackie Jiang, Principal Solutions Architect In her role as Solutions Architect at AWS, Jackie works with enterprise gaming customers to support their innovation in the cloud. She leverages her 20+ years of cross-industry experience to help her customers bring their visions to life in a reliable, scalable, and cost-effective manner. Outside of work, Jackie enjoys living a simple, active and healthy lifestyle, traveling around the world with her family to explore nature and culture, and taking advantage of the unlimited outdoor adventures that Pacific Northwest offers.
	Douglas Bonser, Sr. DocumentDB Specialist SA Douglas Bonser is a Senior DocumentDB Specialist Solutions Architect based out of the Dallas/Ft. Worth area in Texas. He enjoys helping customers solve problems and modernize their applications using NoSQL database technology. Prior to joining AWS he has worked extensively with NoSQL and relational database technologies for over 30 years.
	Matthew Nimmo, Solutions Architect for Game Tech Matthew Nimmo is a life long gamer, developer, and architect. At the young age of 5 he started his gaming passion through his first game, Mega Man, on NES. Each step in his career he used gaming as a vehicle to stay on top of technological changes by experimentation and prototyping new technologies as they came out. In 2016 Matthew established his own gaming studio in which he has designed, developed, and published 3 games to date. Before joining AWS he served as a CA for Stifel Financial for nearly 9 years. Today he’s able to pour his passion for gaming into gaming customers.