Avoiding parallel folders created by concurrent uploads via sdk

Highlighted
Occasional Contributor

Avoiding parallel folders created by concurrent uploads via sdk

We use the Box node sdk to upload files whenever a file arrives in one of our S3 buckets. As part of the upload, we first create a desired folder structure per upload to Box by creating any folders that don't exist (according to folder name) in the desired folderpath. I recently discovered that when there are concurrent uploads whose desired folderpaths are similar, there's a race condition and as a consequence, sibling folders with the same name are created.

 

I'd prefer not to serialize my requests to Box, unless there's some support in the sdk for queuing. I searched these forums for any advice about post-processing to merge such folders, or avoid creating duplicates, but found none.

 

Can you recommend a best practice?

2 REPLIES 2
Highlighted
Box Employee

Re: Avoiding parallel folders created by concurrent uploads via sdk

Hi @davidswalkabout, are you saying the folders have exactly the same name? From what I know about are API that's not the intended behaviour. 


---
Developer Advocate @ Box
Tags (1)
Highlighted
Occasional Contributor

Re: Avoiding parallel folders created by concurrent uploads via sdk

Yes, some names high in our folderpaths refer to entities that persist over
months. We deliver updates about those entities over months via files.
Sometimes the updates occur within a few seconds of each other, leading to
a race condition about whether a folder has been created yet or not.

I believe AWS SQS will solve this by linearizing the concurrent updates.
I'm no longer looking for a solution in the Box sdks.