Release of ctioga2 version 0.14

The day has finally come again to release a new version of my plotting program, ctioga2. Version 0.14 features, among others:

• a new binner, useful to make histograms;
• a command to hide elements already presents, that can be very useful to make animations;
• a whole set of functionalities to make it easier to draw complex grids, including easy ways to style grid elements and a way to switch to the next grid element (picture);
• a whole series of command-file functions to obtain informations about loaded datasets;
• a way to draw legends separately by hand;
• and more, including bugfixes, better error reporting, such as when some of the numbers are infinite, and much much faster debug output.

As usual, the best way to update the package is running:

~ gem update ctioga2

An updated Debian package should find its way to the archive later on today. Have fun !

QSoas tips and tricks: better baselines using the "save points" feature

During data processing in our team, we often subtract plain polynomial baselines to our data, drawn by hand using the b command. Note that this is legitimate because we know that the signal we are looking for quickly drops to 0 outside a given range. The usual workflow is to just load the data, use b to draw a baseline and subtract it, and then have further fun with the baseline-subtracted data. This was the only possibility in the old SOAS. However, I don't like so much this kind of workflow, because such a baseline is arbitrary, and once it's subtracted, it's hard to reassess the baseline afterwards.

This is why I've designed a better way in QSoas: when you are done creating a baseline, push p: this exits pushing only the interpolation nodes:

Above, the baseline subtraction interface, and below, the resulting nodes:

The nodes can be reloaded by hitting uppercase L from within b and giving a buffer number (admittedly, the interface as of now isn't that good, I'll have to work on it). Alternatively, you can greenerate the baseline by using interpolate, and then further subtract it using S:

QSoas> interpolate 1 0

where 1 is the original buffer (in fact, only its X values will be used, so you don't have to use the same buffer) and 0 is the buffer containing the interpolation nodes. Saving only the interpolation nodes makes it possible to reedit the baseline easily, and regenerate the baseline-subtracted data using a script. This is my favorite approach, because I like to be able to trace every single step from the raw data to processed data. Another neat feature is that, from within b, you can reload only the X values of nodes using lowercase l (and giving buffer number): the Y values are computed as if you had clicked at the corresponding positions. This is very useful is you have a whole series of similar baselines to make: most often, you won't even need manual adjustment to get a good baseline. I hope you'll find this tip useful !

Making oprofile work again with recent kernels

I've been using oprofile for profiling programs for a while now (and especially QSoas, because it doesn't require specific compilation options, and doesn't make your program run much more slowly (like valgrind does, which can also be used to some extent for profiling). It's a pity the Debian package was dropped long ago, but the ubuntu packages work out of the box on Debian. But, today, while trying to see what takes so long in some fits I'm running, here's what I get:

~ operf QSoas
Unexpected error running operf: Permission denied

Looking further using strace, I could see that what was not working was the first call to perf_event_open.
It took me quite a long time to understand why it stopped working and how to get it working again, so here's for those of you who googled the error and couldn't find any answer (including me, who will probably have forgotten the anwser in a couple of months). The reason behing the change is that, for security reason, non-privileged users do not have the necessary privileges since Debian kernel 4.1.3-1; here's the relevant bit from the changelog:

  * security: Apply and enable GRKERNSEC_PERF_HARDEN feature from Grsecurity,
    disabling use of perf_event_open() by unprivileged users by default
    (sysctl: kernel.perf_event_paranoid)

The solution is simple, just run as root:

~ sysctl kernel.perf_event_paranoid=1

(the default value seems to be 3, for now). Hope it helps !