Why do so few fonts work on plots?

If I use "listfonts" it shows a lot of available fonts, and "uisetfont" shows many of them working, but if I actually try to use most of them in a plot (e.g. text command), very few actually do anything. Is there anything I can do to remedy this? I have a grand total of about 10 that actually do anything. I'm on linux, 2016b.