Having played with other options of plotting graphs, most recently I’ve settled with Python, and in particular matplotlib and pyplot. Pyplot is a part of Matplotlib, which in turn is Python’s answer to Matlab. Really neat and easy to use. I recently
generated about 120 graphs using an app I wrote, which is called the Disruptive Set Analyzer and can be downloaded here. The readme and the code should be self explanatory. The thresholds are determined empirically, and to summarize, this app can be used to generate plots like this for any scenario where you want to compare two systems, where each system is producing some kind of a numerical score for a bunch of same instances. The instances themselves are divided into different sets, which are also produced by the app in the form of text files. I am currently planning on making this available through the Web, where the user uploads a CSV file and a pair of systems they are interested in, along with the threshold values, and the backend generates this plot. Next, the user can hover the mouse on the individual plot points and the actual instance [it might be a string or anything] will then be revealed. This might be accompanied with a legend of sorts at the bottom of the plot. However, these are just an added layer of user interface, so it is at lower priority for me right now.
PS: in the code you see that I’ve identified by comments the part which makes it possible to produce graphs like this and save them in PNG format even if you only have a non-X-based shell access to the machine on which you are running the code and producing the graphs.