One bottleneck I started feeling when using Maffin was how long it took to load the detail page of an Account or adding/updating/deleting transactions. This happened for Accounts that had many Transactions because we were loading all of them at once.

The good thing about that is that once loaded, it is blazing fast to navigate through transactions but I knew this wouldn't scale because the most used accounts will grow on amount of transactions over the years.

Note that in Maffin, we have everything frontend side, what was costly is the computation of retrieving all the transactions and creating an object for each.

📄 Typeorm pagination

The solution when you have this problem is to add pagination. Basically, instead of retrieving all objects, you only retrieve a subset of them (usually the ones that are visible to the user) and if the user changes page, retrieve the next subset and so on. This reduces memory footprint and computation.

Typeorm has a section on how to use pagination. Basically, in SQL terms you use the OFFSET and LIMIT clauses. An example would look like:

const pageSize = 10;
const pageIndex = 2;

Split.find({
 where: { accountId },
 take: pageSize,
 skip: pageSize * pageIndex,
});

With the code above, we would be skipping the first 20 Split entities and retrieving 20-30. Page size and page index is something you want to keep as a state in the component where you display this information.

# In our case, we display the information in a table
import { PaginationState } from '@tanstack/react-table';

const [{ pageIndex, pageSize }, setPagination] = React.useState<PaginationState>({
  pageIndex: 0,
  pageSize: 10,
});

📖 How do I use this in my component

We want our component to display this data to the user. Due to the data being asynchronous and dynamic by nature, we need a React hook. Something like:

const { data: splits } = useSplitsPagination(account.guid, { pageIndex, pageSize });

In Maffin, we use react-query to manage state and dynamic data so the example above can return many other things like isPending, error, etc. The hook would look like this:

export function useSplitsPagination(
  account: string,
  pagination: { pageSize: number, pageIndex: number },
): UseQueryResult<Split[]> {
  const queryKey = [...Split.CACHE_KEY, account, 'page', pagination];
  const result = useQuery({
    queryKey,
    queryFn:
      async () => 
        Split.find({
          where: { accountId },
          take: pageSize,
         skip: pageSize * pageIndex,
        }),
      queryKey,
    ),
  });

  return result;
}

If you prefer, you can use the useQuery hook directly in the component. I like to group them in generic hooks so I can re-use them easily in other components and encapsulate logic when needed.

When loading the data in the component, the key ["api","splits","f16959f1-dd0b-42bc-b0e8-6b82c9c","page",{"pageIndex":0,"pageSize":10}] appears, if you change page then it would be ["api","splits","f16959f1-dd0b-42bc-b0e8-6b82c9c","page",{"pageIndex":1,"pageSize":10}].

🪑 Bonus: Linking with React table

So now that we have a way to load the data dynamically, we just need to display it. As mentioned above, we are using react-table.

export default function TransactionsTable({
  account,
}: TransactionsTableProps): JSX.Element {
  const [{ pageIndex, pageSize }, setPagination] = React.useState<PaginationState>({
    pageIndex: 0,
    pageSize: 10,
  });
  const { data: splitsCount } = useSplitsCount(account.guid);
  const { data: splits } = useSplitsPagination(account.guid, { pageIndex, pageSize });

  const pagination = React.useMemo(
    () => ({
      pageIndex,
      pageSize,
    }),
    [pageIndex, pageSize],
  );

  return (
    <Table<Split>
      id="transactions-table"
      columns={columns}
      data={splits || []}
      onPaginationChange={setPagination}
      pageCount={Math.ceil((splitsCount || 0) / pageSize)}
      state={{
        pagination,
      }}
      manualPagination
    />
  );

See line 8 where we have an extra hook. For knowing how many pages we have, we need to know the total amount of Transactions we have for that split.

This code is missing dependencies and some others, you can always check Maffin source code to see a full example.

💚 Conclusion

You can see this in action in our demo site!.

After this change, the loading time of the detail page has reduced and the increase in time it's not going to be so drastic as when loading all of the entities at once. Note that the queries for pagination also depend on the amount of Transactions but not as heavily.

You can check the whole change for implementing this in Maffin here. The PR is quite big because we changed other related things like how we load extra Transaction information for each row and others.