pub async fn list_objects_parallel_pages(
client: &Client,
bucket: &str,
prefix: &str,
max_pages: usize,
) -> Result<Vec<ListPage>, AppError>Expand description
Fetch up to max_pages pages of hierarchical object listing starting
at the given prefix.
ยงDesign choice: sequential pre-fetch
AWS continuation tokens are derived from the last key of each page, so
the token for page N is only available after fetching page N-1. True
parallel pagination would require guessing alphabetic split points, which
is fragile and adds little value for the typical page sizes used here.
This function therefore fetches pages sequentially in a loop. The speed
benefit over calling list_objects in a loop comes from batching the
results and returning them in one allocation.
max_pages defaults to 4. Pass 0 to get a single page.