webread can't read the table of a website

Hi, I want to read the table in the left of this website:

https://booleangenenetworks.math.iastate.edu/

I'm using

a=webread("https://booleangenenetworks.math.iastate.edu/")

but the table is hiden inside the source code (line 153) so I can't get the content of the table.

"<table id="rowSelectTable0" onmousedown="toggleSelecting(true)" onmouseup="toggleSelecting(false)"></table>"

How should I get the table?

Thanks!

Answer 1

[Rik]

The reason is that this data is loaded separately from the rest of the web page.

The data seems to be loaded from these two pages:

1. https://booleangenenetworks.math.iastate.edu/static/data/pidToTitle.txt
2. https://booleangenenetworks.math.iastate.edu/static/data/stratification.csv

I found these using the network monitor of my browser. This extra work is an artifact of the modern web, which allows the page itself to be loaded quickly, but also allows editing/updating the data without having to rewrite all pages using this data for display.

You should be able to read these files with webread and do the processing required.

[Wisteria]

Thanks for your answer! However, it seems that these 2 links don't contain all the information of the table. Specifically, I want the "size" and "rules" columns which cannot be found in the 2 links (for "rules", I want the hyperlink to the "view" of each line). Can you help me find where these information is hiden, or can you tell me how you find those two links so that I can try to explore myself? Thanks!

Answer 2

[Steven Lord]

Are you trying to read in the data sets? If so I'd look at the "Download Dataset" link in the upper-right corner. Once you have that ZIP file the unzip command will let you access the files archived inside it.

Alternately, you may want to contact the authors and/or maintainers and ask them to help you access that information, if it's not in the ZIP file.