fileDatastore request to AWS S3 limited to 1000 files

4 vues (au cours des 30 derniers jours)
MaKe
MaKe le 29 Jan 2020
Commenté : Jhon Wine le 22 Avr 2021
I have a bucket on an AWS S3 server which I can reach without any problems.
The issue is: The answer from the server is limited to 1000 files. How can I increase this number? My target is to get just the list of filenames.
I also tried different calls of fileDatastore.
imds = fileDatastore(['s3://',<bucket>,','/'],'FileExtensions','.csv','ReadFcn',@load,'IncludeSubfolders',true)
  4 commentaires
Sameer Gummuluru
Sameer Gummuluru le 1 Déc 2020
I was able to figure out a temporary solution to this, today. On older versions of MATLAB, I was able to use filedatastore to access more than 1000 files from S3. (I tried on MATLAB R2019a)
You can give it a try if working on older MATLAB versions is fine by you.
Jhon Wine
Jhon Wine le 18 Avr 2021
I have the same issue. When using Matlab R2020b I cannot get more than 1,000 files.
However while using Matlab R2019b, fileDataStore seems to return all the files I need.
Using AWS CLI seems to work fine, but is not as elegent of a solution.

Connectez-vous pour commenter.

Réponses (1)

Div Tiwari
Div Tiwari le 19 Avr 2021
Hi all, this issue was resolved in MATLAB R2021a and R2020b Update 5: https://www.mathworks.com/support/bugreports/details/2407120.
If you're using R2020b, you can install updates by clicking on the notification icon towards the top-right of the MATLAB window.
  1 commentaire
Jhon Wine
Jhon Wine le 22 Avr 2021
I just installed R2021a, problem persists.
In fact I have two versions of Matlab on the same computer. 2019a shows 2000 files, 2021a shows 1000 files...

Connectez-vous pour commenter.

Catégories

En savoir plus sur Introduction to Installation and Licensing dans Help Center et File Exchange

Produits


Version

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by