Skip to main content

Folder.get_items() limit param seems not working / slow response times when there are a too many files in a folder

  • March 6, 2024
  • 1 reply
  • 237 views
Folder.get_items() limit param seems not working / slow response times when there are a too many files in a folder

I am using box-python-sdk ver3.7.2.



I referred to the issue below. Is it the same with version 3.7.2 or later?



There seems to be an issue with slow response times when there are a too many files in a folder.

It seems that there are a large number of files(about 12,000 files) under folders in the Box environment that the customer is using.

I think the response would be faster if get_items() could limit the number of records retrieved, but I think it would be difficult because it retrieves all records.

Is there a source code workaround for this problem other than reducing the number of files in the folder?

Did this topic help you find an answer to your question?

1 reply

  • Participating Frequently
  • 15 replies
  • March 12, 2024

Hi,

you can limit retrieved items by stopping the iteration. They are downloaded just in time by the iterator when needed, so:


counter = 0

for item in folder.get_items():

    print(item.id)

    counter += 1

    if counter == 20:

        break


You can use this or similar snippet to only get first 20 items without getting all of them. You can also use offset parameter to start iteration not from the first item in a folder.

Please also note that MarkerPagination is faster than LimitOffsetPagination. You can enable by setting use_marker param to True. Also LimitOffsetPagination allows to iterate over 11.000 items at maximum, when MarkerPagination allows to get infinite numer of items.

Best,

Lukasz


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings