How to Make GraphQL and DynamoDB Play Nicely Together

Serverless architectures, GraphQL APIs, and DynamoDB databases form a powerful combination for website development. While serverless and GraphQL are widely popular, DynamoDB is often misunderstood or avoided, sometimes wrongly dismissed as only beneficial at massive scales.

Initially, I shared this misconception, preferring SQL databases for my serverless projects. However, after gaining experience with DynamoDB, I've discovered its advantages across various project sizes. To illustrate, let's build a sample API from scratch, bypassing complex ORMs or GraphQL frameworks to reveal the underlying mechanics. This hands-on approach might change your perspective on DynamoDB's value.

Addressing Common Concerns

The primary obstacle to DynamoDB adoption is its steep learning curve, although its capabilities are rarely disputed. While the learning curve is significant, SQL databases aren't ideal for serverless applications. The challenges of deploying and managing SQL database connections within a serverless environment are substantial. DynamoDB, being inherently serverless-friendly, offers a long-term advantage by mitigating future scaling complexities. The initial investment in learning DynamoDB ultimately saves considerable future headaches.

The suitability of pairing GraphQL with DynamoDB is more complex. Much of the existing GraphQL documentation, tutorials, and examples assume relational databases. Even Alex Debrie, a DynamoDB expert and author of "The DynamoDB Book," advises against this combination, primarily due to the common practice of writing GraphQL resolvers as sequential, independent database calls, leading to excessive reads.

Another potential issue is DynamoDB's preference for pre-defined access patterns. GraphQL's strength lies in its ability to handle arbitrary queries more easily than REST. This becomes more problematic with public APIs where users can craft unpredictable queries. However, GraphQL is frequently used for private APIs, where control over both client and server allows for managing and controlling query patterns. Without careful design, GraphQL queries can easily overload any database.

A Simple Data Model

This example API will model an organization with teams, users, and certifications. The relationships are depicted in the entity-relationship diagram below. Each team has multiple users, and each user can possess multiple certifications.

Relational Database Representation

While our target is a DynamoDB model, a SQL database representation would resemble the following diagram: (Diagram would be here, showing tables and relationships)

To handle the many-to-many relationship between users and certifications, an intermediary table called "Credential" is introduced. The expiration date serves as the unique attribute in this table. For simplicity, other attributes are omitted.

Access Patterns

Effective DynamoDB data modeling hinges on understanding access patterns. Relational databases begin with normalized data and employ joins for data retrieval. DynamoDB lacks joins; therefore, the data model must align with intended access methods. This is an iterative process. Prioritize frequent access patterns. Many will directly map to GraphQL queries, while others might be used internally for authentication or authorization. Infrequent operations (e.g., weekly administrative checks) don't require optimized design; inefficient methods (like table scans) can suffice.

High-Frequency Access:

• User by ID or name
• Team by ID or name
• Certification by ID or name

Frequent Access:

• All users within a team (by team ID)
• All certifications for a user
• All teams
• All certifications

Infrequent Access:

• All certifications of users within a team
• All users possessing a specific certification
• All users with a certification within a team

DynamoDB Single-Table Design

DynamoDB's lack of joins necessitates querying based on the primary key or pre-defined indexes. The database doesn't enforce a schema, allowing diverse item types within a single table. The best practice is to store all items in a single table for efficient co-located data access. The schema below reflects this approach, aligning with the access patterns identified earlier. (Schema details would be here, showing primary key, sort key, and indexes)

The primary key is a composite of the partition key (pk) and sort key (sk). Retrieving items requires specifying the partition key and either a single sort key value or a range. Indexes (gsi1pk, gsi1sk, etc.) are used for flexible access to different item types. The "#" symbol acts as a placeholder for empty sort keys.

Database Schema Implementation (Code Snippet)

The database schema is enforced within the application. DynamoDB's API is powerful but complex. Many developers use ORMs for simplification. Here, we'll directly access the database using helper functions to define the Team item schema. (Code snippet for DB_MAP, including get, put, and parse functions for Team, would be here)

To add a new team, you'd call: DB_MAP.TEAM.put({teamId:"t_01",teamName:"North Team"})

This generates the index and key values for the database API. The parse method converts database items back to the application model.

GraphQL Schema (Code Snippet)

(GraphQL schema definition for types Team, User, Certification, Credential, and Query would be here.)

Connecting GraphQL and DynamoDB with Resolvers

Resolvers execute GraphQL queries. Resolvers are essential for building our API. Each query in the GraphQL schema has a corresponding root resolver (only team resolvers are shown here). These resolvers return either a promise or an object containing partial query results.

If a query returns a Team type, execution proceeds to the Team type resolver. This resolver has a function for each Team attribute. If a resolver is missing (e.g., for id), it checks if the root resolver provided the value.

A query takes four arguments: root (parent object), args (query arguments), context (application data, here including the database reference), and info (query details, unused here).

The resolvers below use ctx.db.singletable to access the DynamoDB table. The get and query methods interact directly with the database, and DB_MAP.TEAM... translates the schema using helper functions. The parse method converts data to the GraphQL schema format. (Code snippet for resolverMap, including resolvers for Query.team, Query.teamByName, Query.allTeams, Team.name, Team.members, User.name, User.credentials would be here)

Let's trace the execution of the following query: The team root resolver reads the team by ID, returning ID and name. The Team type resolver then retrieves all team members. The User type resolver fetches credentials and certifications for each user. With five members and five credentials each, this results in seven database reads. While this might seem excessive compared to a SQL database (potentially four calls), the seven DynamoDB reads might be faster and cheaper depending on various factors.

query { team( id:"t_01" ){
  id
  name
  members{
    id
    name
    credentials{
      id
      certification{
        id
        name
      }
    }
  }
}}

Over-fetching and the N 1 Problem

Optimizing a GraphQL API involves numerous trade-offs. Two key considerations when choosing between DynamoDB and SQL are over-fetching and the N 1 problem. These are often opposing forces. Over-fetching occurs when a resolver requests more data than needed. This happens when a root or type resolver (e.g., members in the Team resolver) attempts to retrieve excessive data in a single database call. If the query didn't request the name attribute, retrieving it is wasteful.

The N 1 problem is the opposite. Pushing all reads to the lowest-level resolver means minimal or no database requests at higher levels. Instead of one call to get all five members, five separate reads are made. This can lead to many more reads. In practice, tools like DataLoader batch these requests, reducing the number of database calls. These smaller, atomic requests are crucial for efficient batching.

For SQL, small, low-level resolvers with DataLoader are usually best. For DynamoDB, "smarter" higher-level resolvers that align with the single-table design are more effective. Over-fetching is often the lesser evil in this context.

Deployment (Summary)

This example can be deployed quickly using Architect, an open-source tool for building serverless applications on AWS. The GitHub repository provides the code. After cloning and running npm install, the application (including a local database) can be launched locally. Deployment to production on AWS (including DynamoDB) is also straightforward.

The above is the detailed content of How to Make GraphQL and DynamoDB Play Nicely Together. For more information, please follow other related articles on the PHP Chinese website!