Using paginated requests with Workspace ONE UEM REST APIs

Welcome to day 17 of Let’s Git Commit(ted) to </Dev> Resources!

Have you encountered any of the Workspace ONE UEM REST APIs that make use of the paginated requests? Have you experienced issues trying to query all records for large data sets, such as devices, organization groups, or administrator accounts? Have you seen REST APIs that use the page and pagesize query parameters and wondered what function these serve?

Today’s discussion will clarify these questions and guide you through using these paginated requests in the Workspace ONE UEM REST APIs.

What are Paginated Requests?

Paginated requests occur for datasets where the requested data cannot be returned in a single request and must be queried and returned over multiple pages. Pagination is a common practice to prevent performance-intensive queries, to ensure that responses remain under a certain payload size, and to prevent noisy neighbors in SaaS environments. 

The Basics of Paginated Requests

There are two important parameters that you must provide when working with paginated request:

• Page: An integer, starting at zero, that must be incremented on each subsequent request to determine the subset of records that will be returned.
• Pagesize: The number of records that are requested per page.

When you make a request to a Workspace ONE UEM API that supports pagination, the response will also include a Total property, indicating the total number of records that exist for the data set (such as devices, administrators, users). You must use the total property to determine how many pages you need to request of a given pagesize to query all your records.

Paginated Requests Example: Querying Devices

One example of a paginated request in Workspace ONE UEM is searching for device records. There are several Device Search APIs, but for this example, we will focus on GET /mdm/devices/extensivesearch in the MDM V1 APIs (see accessing the API documentation for full details on this API).

Let’s assume you have 5,000 device records in your environment, and you wish to query all of the records.

Step 1: Determine pagesize

The first step is to determine the appropriate pagesize. If the pagesize parameter is not provided, the default value of 500 is used. The maximum pagesize you can provide for a paginated request is also 500. While you can specify a lower pagesize, remember that you have a daily API quota in Workspace ONE UEM, so it is recommended to use the default 500 pagesize.

For this example, I will provide the pagesize parameter, but still use the default value of 500 so you can see how to format the URL should you wish to change the pagesize parameter.

Step 2: Form the First Paginated Request

The first page parameter will be 0 and the pagesize will be 500. This means that we will query the first 500 device records in our environment. We should not assume that we know the total device record count in our environment prior to making the first paginated request, as it may have changed since we last viewed the data in the Workspace ONE UEM administration console.

This request will return the following response on a 200 OK:

• Devices: The devices array has been omitted for simplicity here, but you will see an array of records containing information about the devices, such as device identifiers, organization group details, user details, and more.
• Page: The page value that was provided in the request, which is 0 in this case, as it is our first request.
• PageSize: The number of records that were requested as specified by the pagesize, which was 500.
• Total: The total number of all device records that exist in our environment for the parameters we specified, which is 5,000 in this example. Since we did not specify other scope-limiting parameters, such as organizationgroupid, platform, startdatetime, or enddatetime, we are querying all device records that are available to administrator account specified in the Authorization header with the provided API key specified in the aw-tenant-code header.

You can now calculate how many pages of pagesize 500 you will need to request in order to retrieve all devices by dividing the total (5,000) by the pagesize (500).  5,000 / 500 = 10. Note: Don’t forget that page starts at 0 and needs to be incremented by one each time. This means the final page would be 9.

Step 3: Form the Remaining Paginated Requests

Now that you have calculated how many pages you must request (10) in order to retrieve the total (5,000) for your given pagesize (500), you can form and request the remaining device record pages.

Each one of the ten requests you must make to retrieve all the device records will return the same information as the first request, except the Devices array in the response will contain the devices specified by the page and pagesize properties.

The second response:

The third response:

And so on until all ten pages have been requested. You would store the device records returned in the Devices array at each request in a table or array of your own if you need to interact with all 5,000 device records at once.

Additional Logic for Handling Total, Page, and Pagesize

A common question is how to handle the page and pagesize parameters when the Total property isn’t evenly divisible by the page size. For instance, what if our example had 5,1000 devices using a pagesize of 500. The last request would be ?page=10&pagesize=500 to retrieve records 5,000 – 5,500, but we only have 5,100 records. How do we form our paginated responses to capture all the device records?

Pagesize and Total Interactions

You do not need to change your pagesize to match the returned Total property. When you provide the pagesize, you are specifying the maximum number of records you can receive, but if the remaining number of records are lower than the pagesize, then only the remaining records will be returned.

Let’s pick up from where our previous example left off and assume that we now have 5,100 device records rather than 5,000.

GET /api/mdm/devices/extensivesearch?page=9&pagesize=500 would return device records 4,500 – 5,000, but we still have 100 records left to retrieve! As with the previous examples, you can simply increment page by 1 and retrieve the remaining 100 device records.

The response will only contain 100 records in the Devices array because that is all that remains in your data set:

This simplifies the pagination responses so that you do not need to worry about modifying the pagesize to match the appropriate total of your final page.

Calculating the Final Page

Speaking of the final page, the formula we used before (total / pagesize = number of pages) doesn’t work well when our total isn’t evenly divisible by pagesize, does it? We can use a revised formula to determine the final page, or if scripting the result, you can use a logical if to determine if the requested page was the final page.

Formula:

1. Ceiling is a math function that always rounds a number up to the next largest integer.
2. Continuing the 5,100 total records and 500 pagesize example, this would produce Ceiling(5,100 / 500) = Ceiling(10.2) = 11.
3. Since the page count starts at 0, we need to subtract 1 from the ceiling result to indicate which page={number} will be our final page. In this case, 11 – 1 = 10, so the final page request would be ?page=10&pagesize=500.

Logical If

The above formula can be applied to a logical if when scripting to loop through all the paginated requests to retrieve all records in a data set. A no-code outline of this process would be:

1. Start by requesting the first page, such as GET /mdm/devices/extensivesearch?page=0&pagesize=500.  I recommend storing the page value in a variable, like curPage, that you can increment for ease.
2. When the 200 OK response is received, store the records array (devices, in this case) and use the final page formula above to check if your current page is the final page (finalPage = Ceiling(response.total / pagesize) – 1).
3. If the current page is the final page, all records are queried, and no further API calls are needed. If the current page is less than the final page, increment your page variable and call the next page of records, starting back at step #1.

Check out the PaginationRecursion.ps1 sample in euc-samples to see this in action, and feel free to leverage this as a starting point when scripting for your own pagination requests! You can also create this recursive logic using API tools such as the Collection Runner for Postman. 

Conclusion

When working with API requests that require the page and pagesize parameters, remember to use the provided logic to calculate your final page of records and then iteratively requests each page until all records are retrieved. Leverage the provided PowerShell sample as a starting point and utilize API tools like Postman to help automate the recursive calls if scripting isn’t desirable.

If you enjoyed this API overview and want to see more examples or a deeper dive – let us know!

Agenda

Make sure to check out the other blog posts in our 28-day series:

• Day 1: Let's Git Commit(ted) to Dev Resources
• Day 2: Getting Started with the Workspace ONE UEM REST APIs
• Day 3: Getting Started with the Workspace ONE Access APIs
• Day 4: Getting Started with the VMware Workspace ONE Intelligence APIs
• Day 5: Getting Started with the VMware Horizon REST APIs and VMware PowerCLI
• Day 6: Getting Started with Automating the Unified Access Gateway Deployment
• Day 7: Podcast: Day 0 Onboarding Automation with Scot Curry
• Day 8: Video: Anatomy of the Workspace ONE UEM API
• Day 9: Introduction to using Postman - Part 1
• Day 10: Introduction to using Postman - Part 2
• Day 11: Pro Tips and Tricks - How to be an API Boss
• Day 12: What is OAuth - Learning the Basics
• Day 13: Getting Started with Intelligent Hub Notifications
• Day 14: Git Basics: Getting Git Going
• Day 15: Podcast: Git Commit(ted) to Resources: Customer Spotlight with The Home Depot
• Day 16: Git VMware {code} Samples and Flings
• Day 17: Using paginated requests with Workspace ONE UEM REST APIs
• Day 18: Event Notifications
• Day 19: Overview of Script Samples using PowerCLI for Horizon
• Day 20: Uploading Windows apps using REST APIs 
• Day 21: Uploading macOS apps using REST APIs and Admin Assistant 
• Day 22: API-based user lifecycle and SCIM 
• Day 23: Video: Community Expert Roundtable on Leveraging APIs and Scripting
• Day 24: Video: Exploring the Workspace ONE GitHub Samples Repository 
• Day 25: Featured Fling: Forklift for Workspace ONE UEM
• Day 26: Featured VMware {code} Samples for Horizon
• Day 27: Featured Flings for VMware Horizon
• Day 28: Continuing to Focus on </Dev> Resources Page