How to access streaming data and store it?

Hello, I'm a newbie in MATLAB and I want to retrieve financial streaming data and using it with MATLAB.

I know how to get this data using curl:

C:\>curl -k -H "Authorization: Bearer 137529d301a65re3a32b00ee2c94b8f3-72ffdf45bb72dacy72w40fc67b6b9fbfa" "https://stream-fxpractice.oanda.com/v1/prices?accountId=1234567&instruments=AUD_CAD%2CAUD_CHF"

And the data looks like this:

{"tick":{"instrument":"AUD_CAD","time":"2016-05-20T08:31:03.770867Z","bid":0.94756,"ask":0.94784}}
{"tick":{"instrument":"AUD_CHF","time":"2016-05-20T08:30:58.241559Z","bid":0.71726,"ask":0.71753}}
{"heartbeat":{"time":"2016-05-20T08:31:05.298930Z"}}
{"tick":{"instrument":"AUD_CAD","time":"2016-05-20T08:31:07.084095Z","bid":0.94756,"ask":0.94783}}
{"tick":{"instrument":"AUD_CAD","time":"2016-05-20T08:31:07.595119Z","bid":0.94756,"ask":0.94781}}
{"heartbeat":{"time":"2016-05-20T08:31:07.770028Z"}}
{"tick":{"instrument":"AUD_CAD","time":"2016-05-20T08:31:08.326162Z","bid":0.94756,"ask":0.9478}}
{"tick":{"instrument":"AUD_CAD","time":"2016-05-20T08:31:09.211011Z","bid":0.94756,"ask":0.94781}}
{"tick":{"instrument":"AUD_CAD","time":"2016-05-20T08:31:09.580424Z","bid":0.94756,"ask":0.94784}}
{"heartbeat":{"time":"2016-05-20T08:31:10.258639Z"}}
{"heartbeat":{"time":"2016-05-20T08:31:12.770908Z"}}

JSON Response Fields:

tick:
 instrument: Name of the instrument.
 time: Time in a valid datetime format.
 bid: Bid price
 ask: Ask price

heartbeat:
 time: Time in a valid datetime format.

How can I retrieve this streaming data using MATLAB?

Cheers.

Francisco

Answer 1

[Guillaume]

The equivalent would probably be:

queryoptions = weboptions('KeyName', 'Authorization', ...
                      'KeyValue', 'Bearer 137529d301a65re3a32b00ee2c94b8f3-72ffdf45bb72dacy72w40fc67b6b9fbfa', ...
                      'ContentType', 'json');
jsondata = webread('https://stream-fxpractice.oanda.com/v1/prices', ...
                   'accountId', 1234567, ...
                   'instruments', 'AUD_CAD''CAUD_CHF#', ...
                   queryoptions)

However, I don't think there's a way to tell matlab to ignore certificate errors (your -k option in curl), so if the website certificate is not valid (why?) then you may need to install that invalid certificate on your machine or degrade to http connection.

[Francisco]

Thanks for your help!

As the incoming data is a continuous stream I don't know how to use the data because the process using webread stays busy forever...

I don't find examples using webread for continuous stream subscription, please can anyone help me with that?

Cheers

Francisco

[Guillaume]

Unfortunately, I don't think there's anything in matlab that can deal with continuous streams. webread wants the response to be complete before it returns the data.

The only way I can see to make this work in matlab would be to delegate most of the stream processing to either Java or .Net, both of which can be called directly from matlab. So for example, using .Net you would create an HttpWebRequest object directly from matlab, obtain its Response, get the ResponseStream and either use blocking IO (synchronous) to get the stream piecewise or use asynchronous IO with callbacks. Unfortunately, I don't have enough experience with that to tell you how to do it, I can only offer you a start.

One thing for sure, there's no easy way to do this in matlab.

[Francisco]

Thanks a lot!

[pedro galan feliubadalo]

Hi Francisco,

How did you manage to make it work?