Downloading and saving Excel files using Matlab

I would like to download this file: https://www.nbp.pl/en/statystyka/m3/podaz_bilansowa_en.xlsx and save it in the folder, all using Matlab. Is there any way I can do this? I red about 'urlread' and 'urlwrite' however do not know how exactly use it for my purpose.

[Mathieu NOE]

hello

so why not dowload directly ?

[Rik]

What release are you using? And what have you tried? urlwrite has two inputs: URL and filename. What is your issue?

[Joanna Przeworska]

I am using Matlab R2019b. I've tried the code below and had an error, however it might be due to security lock on my computer. Am I right? Could you please try this code on your computers to prove it works? My goal is to download this Excel file from website and save it in folder on my computer.

fullURL = ['https://www.nbp.pl/en/statystyka/m3/'];
filename = 'podaz_bilansowa_en.xlsx';
urlwrite(fullURL,filename);

Error using urlreadwrite (line 98)
Error downloading URL. Your network connection may be down or your proxy settings improperly configured.

Error in urlwrite (line 52)
[f,status] = urlreadwrite(mfilename,catchErrors,url,filename,varargin{:});

[Rik]

And what if you try websave?

websave(filename,fullURL);

[Joanna Przeworska]

Unfortunately it returns error as well...

Error using matlab.internal.webservices.HTTPConnector/copyContentToFile (line 412)
The server returned the status 403 with message "Forbidden" in response to the request to URL
https://www.nbp.pl/en/statystyka/m3/.

Error in websave (line 107)
copyContentToFile(connection, filename);

Answer 1

[Rik]

It seems this website blocks requests when it detects non-human requests.

You might be able to circumvent this by use the Wayback Machine. I have found that on some occasions that will work.

If you want an easy way to interact with the Wayback Machine from Matlab, you can get my WBM function from the FEX.

WBM(filename,fullURL);

[Joanna Przeworska]

I downloaded this function and it doesn't return an error, but what should be the output? and where I should look for it?

[Rik]

It should download the file to whereever you told it. What is the content of filename?

[Joanna Przeworska]

I used the following code (below). The variable 'out' is empty.

fullURL = ['https://www.nbp.pl/en/statystyka/m3/'];
filename = 'podaz_bilansowa_en.xlsx';
out = WBM(filename,fullURL);

[Rik]

The name fullURL promises a full URL to the file. It doesn't fulfill that promise. The actual full URL is the concatenation of the two.

Try this:

filename = 'podaz_bilansowa_en.xlsx';
fullURL = ['https://www.nbp.pl/en/statystyka/m3/' filename];
out = WBM(filename,fullURL);

If that fails: I have an update almost ready, so I could attach it here for you.

[Joanna Przeworska]

Unfortunately it also returns an empty 'out'... Of course I may test updated version as well.

[Rik]

For me it works as expected. Note that sometimes the Wayback Machine needs some time to properly store the capture. If you retry now, it will probably work.

[Joanna Przeworska]

I tried today the same code and this time had another type of error:

filename = 'podaz_bilansowa_en.xlsx';
fullURL = ['https://www.nbp.pl/en/statystyka/m3/' filename];
out = WBM(filename,fullURL);

Warning: File 'podaz_bilansowa_en.xlsx' not found. 
> In WBM (line 355) 

[Joanna Przeworska]

I found podaz_bilansowa_en.xlsx on my computer, however, is empty.

[Rik]

That is not podaz_bilansowa_en.xlsx, but podaz_bilansowa_en.xlsx.html. This can happen when there are errors. The update should automatically delete this file.

The update should now be available on the FileExchange.

[Joanna Przeworska]

Unfortunately, still an error..

filename = 'podaz_bilansowa_en.xlsx';
fullURL = ['https://www.nbp.pl/en/statystyka/m3/' filename];
out = WBM(filename,fullURL);

Error 403 tries(4,4,4) (download of podaz_bilansowa_en.xlsx)
Error 403 tries(4,3,4) (download of podaz_bilansowa_en.xlsx)
Error 403 tries(4,2,4) (download of podaz_bilansowa_en.xlsx)
Warning: File 'podaz_bilansowa_en.xlsx' not found. 
> In WBM (line 423) 

[Rik]

That line should never trigger a warning, as it will only attempt deletion if the file exists, so your warning is very odd. It is also a bit strange that you're getting 403 errors.

filename = 'podaz_bilansowa_en.xlsx';
fullURL = ['https://www.nbp.pl/en/statystyka/m3/' filename];
out = WBM(filename,fullURL)

Building with 'gcc'.
MEX completed successfully.

Warning: No date found in file, unable to check date, assuming it is correct.

out = '/users/mss.system.0ZBHG6/podaz_bilansowa_en.xlsx'

As you can see, this should work as expected. You should get a warning about a missing capture date, but the warning you're getting should never happen at all. This should not be different for R2019b.

Can you open web.archive.org normally in your browser?

[Joanna Przeworska]

No, I can't. The connction is blocked and it is due to the security policy introduced on my computer. I'm afraid there's no way around it.

[Rik]

If your IT department policy even disables reaching out to archive.org, you should ask them for suggestions.

If you have such a strict environment, why do you want to dynamically load a file from the internet anyway?

[Joanna Przeworska]

To be honest I didn't know it was that strict. I was hoping there was a solution. Anyway, I'm really grateful for the time you took to help me.