Skip to main content

Paging

nestjs-query supports multiple paging strategies that each have their own pros and cons. This documentation will cover the different paging strategies and their applicable use cases.

The following examples are based on the following TodoItemDTO

todo-item.dto.ts
import { FilterableField, IDField } from '@ptc-org/nestjs-query-graphql';
import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql';

@ObjectType('TodoItem')
export class TodoItemDTO {
  @IDField(() => ID)
  id!: string;

  @FilterableField()
  title!: string;

  @FilterableField()
  completed!: boolean;

  @FilterableField(() => GraphQLISODateTime)
  created!: Date;

  @FilterableField(() => GraphQLISODateTime)
  updated!: Date;
}

Cursor Based Paging​

By default nestjs-query will expose all query many endpoints as cursor based connections that you can use to page through results.

note

When using cursor based connections you are not tied to any particular implementation described below, because of the opaque nature of cursors you can start out with the default and switch to key set based cursors later on without changing your clients.

All cursor connections, regardless of paging strategy, expose the following schema

scalar ConnectionCursor

type PageInfo {
  hasNextPage: Boolean
  hasPreviousPage: Boolean
  startCursor: ConnectionCursor
  endCursor: ConnectionCursor
}

type TodoItemConnection {
  pageInfo: PageInfo!
  edges: [TodoItemEdge!]!
}

type TodoItemEdge {
  node: TodoItem!
  cursor: ConnectionCursor!
}

type TodoItem {
  id: ID!
  title: String!
  description: String
  completed: Boolean!
  created: DateTime!
  updated: DateTime!
}

Offset Based Cursor​

By default all cursors will use a form of offset based paging to back cursors.

ProsCons
Allow for recursive relation pagingPages and nodes may change between queries
Straight forward to implementInconsistent Sorting - natural sorting is used by default
Queries grow more inefficient as you go through more pages

Key Set Based Cursor​

You have the option to specify a key set on your DTO which will replace the offset with a where clause. A keyset is a set of fields that uniquely identify a record (e.g. id).

ProsCons
Consistent Results - All cursors uniquely identify a record allowing for deterministic before/after pagesUnable to do recursive paging for relations
Stable Sorting - All results will be deterministically sortedA unique key set must be specified
Consistent performance with correct indexes in placeUnable to jump to arbitrary page
Suitable for datasets of any size

To enable key set based paging all you need to do is decorate your DTO with the @KeySet decorator.

import { FilterableField, IDField, KeySet } from '@ptc-org/nestjs-query-graphql';
import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql';

@ObjectType('TodoItem')
@KeySet(['id'])
export class TodoItemDTO {
  @IDField(() => ID)
  id!: number;

  @FilterableField()
  title!: string;

  @FilterableField({ nullable: true })
  description?: string;

  @FilterableField()
  completed!: boolean;

  @FilterableField(() => GraphQLISODateTime)
  created!: Date;

  @FilterableField(() => GraphQLISODateTime)
  updated!: Date;
}

Sorting and Key set cursors​

When using key set based cursors we must take into account any client provided sorting in order to uniquely identify a record within the sorted set of nodes

For example assume we're using the same DTO as above. If we added a sort by completed a comparision on id would no longer be sufficient

{
  todoItems(sorting: [{ field: completed, direction: ASC }]) {
    edges {
      cursor
      node {
        id
        title
        completed
      }
    }
  }
}

If we only compared on the keyset [id] our pages would no longer be sorted properly, if we only compared on completed you would not be able to page.

We solve this problem by encoding information about the each field in the sort as well as the key set fields into the cursor so we can page properly.

In the above example the filter from the cursor (when paging forward) would be something like

WHERE (completed > ? OR (completed = ? AND id > ?))

Relation Connections​

Key set paging will not apply to relations because they are recursive by nature.

For example if you query for multiple TodoItems and their subTasks if key set paging was used for the subTasks connection the cursor from one todoItems subTasks may not be applicable to all todoItems

For example, assume you have the following todo items and subTasks

[
  {
    "id": 1,
    "title": "Todo 1"
    "subTasks": [
      {"id": 1, "title": "Todo 1 - Sub Tasks 1"}, // cursor: "abc"
      {"id": 2, "title": "Todo 1 - Sub Tasks 2"}  // cursor: "def"
    ]
  },
  {
    "id": 2,
    "title": "Todo 2"
    "subTasks": [
      {"id": 3, "title": "Todo 2 - Sub Tasks 1"},  // cursor: "ghi"
      {"id": 4, "title": "Todo 2 - Sub Tasks 2"}  // cursor: "jkl"
    ]
  }
]

If you ran the following graphql query

{
  todoItems {
    edges {
      node {
        title
        subTasks(paging: {first: 2, after: "ghi"}){
          title
        }
      }
    }
  }
}

The resulting query would look for all subTasks with an id > 3 breaking paging for Todo 1, for this reason the @KeySet decorator is used for all relations.

Paging​

To page with cursors it works the same way for all strategies.

In this example we'll fetch the first 2 records.

{
  todoItems(paging: {first: 2}) {
    pageInfo{
      hasNextPage
      hasPreviousPage
      startCursor
      endCursor
    }
    edges{
      node{
        id
        title
        completed
        created
        updated
      }
      cursor
    }
  }
}

Lets take a look the pageInfo from the previous example

{
  "pageInfo": {
    "hasNextPage": true,
    "hasPreviousPage": false,
    "startCursor": "YXJyYXljb25uZWN0aW9uOjA=",
    "endCursor": "YXJyYXljb25uZWN0aW9uOjE="
  },
}

Notice how hasNextPage is true and there is an endCursor that can be used to fetch the next page.

{
  todoItems(paging: { after: "YXJyYXljb25uZWN0aW9uOjE=", first: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
      startCursor
      endCursor
    }
    edges {
      node {
        id
        title
        completed
        created
        updated
      }
      cursor
    }
  }
}

We can also page backward by using before and last. In the following example we'll use the startCursor from the previous example and set last to 2.

{
  todoItems(paging: { before: "YXJyYXljb25uZWN0aW9uOjI=", last: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
      startCursor
      endCursor
    }
    edges {
      node {
        id
        title
        completed
        created
        updated
      }
      cursor
    }
  }
}

Offset Based Paging​

An alternative to cursor based querying is to use the OFFSET pagingStrategy. When using the OFFSET strategy queries that return multiple records will return an OffsetConnection that looks like the following.

type OffsetPageInfo {
  hasNextPage: Boolean
  hasPreviousPage: Boolean
}

type TodoItemConnection {
  pageInfo: OffsetPageInfo!
  nodes: [TodoItem!]!
}

type TodoItem {
  id: ID!
  title: String!
  description: String
  completed: Boolean!
  created: DateTime!
  updated: DateTime!
}
ProsCons
Easy to understandunstable - pages and nodes may change between queries
Able to jump to arbitrary pagesInconsistent Sorting - natural sorting is used in the strategy
Queries grow more inefficient as you go through more pages
State must be maintained in the client to know last limit and offset in order to page

To enable OFFSET based paging all you need to do is set the pagingStrategy option using the @QueryOptions decorator.

todo-item.dto.ts
import { FilterableField, IDField, QueryOptions } from '@ptc-org/nestjs-query-graphql';
import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql';

@ObjectType('TodoItem')
@QueryOptions({ pagingStrategy: PagingStrategies.OFFSET })
export class TodoItemDTO {
  @IDField(() => ID)
  id!: number;

  @FilterableField()
  title!: string;

  @FilterableField({ nullable: true })
  description?: string;

  @FilterableField()
  completed!: boolean;

  @FilterableField(() => GraphQLISODateTime)
  created!: Date;

  @FilterableField(() => GraphQLISODateTime)
  updated!: Date;
}

In this example we'll fetch the first 2 records.

{
  todoItems(paging: { limit: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
    }
    nodes {
      id
      title
      completed
      created
      updated
    }
  }
}

In this example we'll also pass in an offset to fetch the next 2 records.

{
  todoItems(paging: { limit: 2, offset: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
    }
    nodes {
      id
      title
      completed
      created
      updated
    }
  }
}


Disabling Paging​

caution

This strategy is only recommended if you are sure your dataset is small.

When using the NONE paging strategy the paging argument is removed and all methods will return an ArrayConnection.

ProsCons
All data is returned at onceNot suitable for large datasets

To disable paging all you can set the pagingStrategy option using the @QueryOptions decorator to PagingStrategies.NONE.

todo-item.dto.ts
import { FilterableField, IDField, QueryOptions, PagingStrategies } from '@ptc-org/nestjs-query-graphql';
import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql';

@ObjectType('TodoItem')
@QueryOptions({ pagingStrategy: PagingStrategies.NONE })
export class TodoItemDTO {
  @IDField(() => ID)
  id!: number;

  @FilterableField()
  title!: string;

  @FilterableField({ nullable: true })
  description?: string;

  @FilterableField()
  completed!: boolean;

  @FilterableField(() => GraphQLISODateTime)
  created!: Date;

  @FilterableField(() => GraphQLISODateTime)
  updated!: Date;
}

When using the strategy queries that return multiple records will return an array instead of a connection.

{
  todoItems {
    id
    title
    completed
    created
    updated
  }
}