How can I “remove download” from a folder if only a few items are downloaded and the rest are online?

I have a folder with over 1,000 items, but only 30–40 were downloaded for offline use.

I’d like to make the entire folder online-only, but when I right-click on it there’s no “remove download” option. It seems I have to manually find those 30–40 items and remove the download from each one. Is there an easier way? *without signing out..