Thursday, August 1, 2013

Importing lots of files into git-annex

If, like me, you find git-annex very appealing, but have been moving files without it for a while, here is a little script that may help you through the transition. I've just converted my MP3 library to git-annex, but, now I must convert all my copies of the library on various computers to track one (or more) of them. The trick is to teach git-annex about the already existing files, as re-downloading a 20GB library through a 80 KB/s upload DSL line doesn't sound that funny. Here is how I did it.

First, convert one repository to using git-annex. Then, switch to another computer, move away the directory containing the files to a backup directory, and git clone the first repository. Then, from the directory created, run:


~ ./import path/to/old/backup/copy

Where import is the following script:


#! /bin/bash

src="$1"

IFS=$'\n' 
for f in $(find -type l); do 
    if stat -L "$f" >/dev/null 2>/dev/null; then
        echo "File $f ok"
    else
        tg=$(readlink "$f" | sed 's/.*git/.git/' )
        dir=$(dirname "$tg")
        if [ -r "$src/$f" ]; then
            mkdir -p "$dir"
            cp -avl "$src/$f" "$tg"
        fi
    fi
done

Once this is done, you'll need to run

~ git annex fsck

Of course, there is no warranty !! It saved me lots of download time, but it could irretrievably damage your data, put your dog in danger or set fire to your house, so use with care !

Thursday, January 10, 2013

Introducing dvdcopy, an error-resistant command-line DVD copier

My wife likes watching DVDs a lot, which is why she borrows many from the public library. The problem is that many of those were handled by people who have no clue whatsoever about the proper handling of DVDs, which means that they are in a state ranging from moderately scratched in the best cases to properly scarred in the worst ones... xine or VLC fail miserably to read such DVDs. What's worse is that failures happen right in the middle of DVDs, which is quite frustrating !

That's why I've started quite some time ago to write a small C wrapper around libdvdread to first copy the DVD files onto the disk and then read them from there. It was also an easy solution to copy files to a driveless netbook to watch the DVD there. With time, this project evolved into a neat C++ command-line utility that's able to keep track of badly read sectors and come back to them later.

Today is the 0.0 release of dvdcopy. You can get the tarball there. It follows the standard autoconf dance:

~ ./configure
~ make
~ make install

Using it is as simple as running:

~ dvdcopy /dev/dvd MyMovieDirectory

If everything went fine, you're set, and you can open the directory created with vlc or xine:

~ xine dvd://`pwd`/MyMovieDirectory

(yes, xine needs the full path, I haven't checked with VLC, but from what I remember, you can open DVD directories directly from a menu).

If you're not that lucky and dvdcopy found some bad sectors, try reading again using

~ dvdcopy --second-pass /dev/dvd MyMovieDirectory

I found that in many cases, one or two second passes get all the sectors back. While that wasn't really the intended purpose, the fact that most copy protection schemes (in addition to CSS) are implemented using bad sectors make dvdcopy quite resistant to them.

I have no plan of making a Debian package for it at the time being, but if there is demand for that, I may reconsider. Enjoy !

Wednesday, January 9, 2013

Lots of hard drive accesses with MacOS

Sorry, a post about MacOS again, but this is a problem my boss has been faced with, and judging by what google searches gave us as clues to go on, I thought it would be useful for others to know about this.

So here it is: for a couple of weeks, my boss's Mac had been slowing down very painfully, to the point that he was just sitting in front of it waiting for web pages to show up or applications to start... We tried a whole bunch of things, such as disabling spotlight, dropbox, and even the time machine, but nothing worked. Finally, he had a look at Disk Utility that said that the disk was gone beyond repair. Fortunately, a new colleague suggested to restore from the Time Machine, holding Command-R during startup. It took ages (over a slow network), but now the machine is up again, without any slowing down.

Thursday, December 8, 2011

Graphical connection to a Mac from Linux

For my lab work, I have developed an (almost) cross-platform program based on Qt4 that help us analyze our data. One of the motivations to use Qt was the ability to port the program to other platforms, and in particular MacOSX (half of my team uses them). After installing the required development tools using MacPorts (mind you, the MacOS version of the development tools, not the X11 one), I was able to compile and run the program without a single code change on my bosses' Mac. But here come problems: several things do not work the same way in Mac and in Linux (whereas it should, I think), so additional tweaking and debugging will be necessary.

This is where the problem comes in: I don't own a Mac and don't intend to, so I need to connect to my bosses' Mac in order to perform testing. How can I connect to a Mac display ? X11 connection won't work, since my application is a Mac application and not a X11 one. I had him turn on screen sharing (the most basic tick box), which sets up a VNC server on the Mac. Unfortunately, when I connect using the command-line VNC clients available for debian, ie vncviewer (from the packages xvnc4viewer or xtightvncviewer), I get the following problems:

~ vncviewer chrismac

VNC Viewer Free Edition 4.1.1 for X - built Mar 10 2010 22:31:05
Copyright (C) 2002-2005 RealVNC Ltd.
See http://www.realvnc.com for information on VNC.

Wed Dec  7 10:40:41 2011
 CConn:       connected to host chrismac port 5900
 CConnection: Server supports RFB protocol version 3.889
 CConnection: Using RFB protocol version 3.8
 CConnection: No matching security types
 main:        No matching security types

The good news is that remmina is able to connect graphically, although for me it required that I set the display depth to 24 bits (else it seems to connect, but the connection dies immediately). Great, I'll be able to debug, then...

Edit: after a while, I found out that remmina had some drawbacks when connecting to a Mac, such as, at least in my case, a weird problem with Shift: once I hit Shift, I'm never able again to type in lowercase characters, which is quite painful. I tried also Vinagre, who handled that better (but had some quite painful freezing moment), so I must say that I finally found a client I'm happy with: gtkvncviewer.

Tuesday, June 7, 2011

Record sound currently playing to file using ALSA

I've been wondering for a long time about how to record sound which is currently being played on the computer. I know the ALSA framework is powerful enough to do that, and today just proved me so (at least to some extent). It turns out that it's pretty trivial using the file PCM plugin; all you need is a stanza like this in your $HOME/.asoundrc file:

pcm.save {
    type file
    format wav
    slave.pcm front 
    # my main PCM is called front
    file output.wav
}

Afterwards, you can use this save PCM with the tools that understand it, such as aplay or mplayer:

~ aplay -Dsave stuff.wav
~ mplayer -ao alsa:device=save stuff.wav

(for xine users, you may want to have a look at the audio.device.alsa_front_device configuration element).

Funnier, though, is the possibility to use it with programs that don't provide such an easy way to change that, (say, the flash player in a browser ?) via the use of the default PCM specification in $HOME/.asoundrc:

pcm.!default save

After that, all programs using the default PCM will save the sound they play as a save.wav file in their current directory - but nothing stops you from specifiying an absolute directory. Better yet, you may apparently use pipes in stead of the file name, à la popen.

Simply impressive...

... that's what I think of the work of the FTP Team. Three uploads to NEW in one day, all of which processed in less than half a day. Truly impressive !

Many thanks to them !

Tuesday, April 26, 2011

Newer xorg, missing mouse features and xinput

Recent versions of xorg have seen changes in defaults for the behaviour of mice/touchpads, such as third button emulation or edge scrolling suddenly not working anymore. What is worse now is that the corresponding entries in the xorg.conf are now ignored... Wait, wait, don't go flaming the Xorg maintainers, but keep on looking rather. This is where I found out about xinput, the new way to customize/parametrize input devices for the X server. And, lets face it, it beats any customization that could have been done in xorg.conf ! Here's how to use it.

First, you'll have to install it from its own package (xinput), as it doesn't come bundled with standard X clients. Second, you can find out about the devices connected to your X server using:

~ xinput --list
⎡ Virtual core pointer                          id=2    [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
⎜   ↳ SynPS/2 Synaptics TouchPad                id=12   [slave  pointer  (2)]

It is easy to find the device you're interested in (here, say the touchpad entry). Then, you list the properties you can set/get using:

~ xinput --list-props "SynPS/2 Synaptics TouchPad"
[...]
        Synaptics Edge Scrolling (269): 0, 0, 0
 

The list is quite impressive, so I have only shown the one I'm interested in. You see here everything is 0, which probably explains why edge scrolling is disable (I don't have a clue what the actual numbers mean). You can finally change the properties this way:

xinput --set-prop "SynPS/2 Synaptics TouchPad" "Synaptics Edge Scrolling"  6 6 6

This gave me back edge scrolling. I used a similar approach to enable again the third button emulation... What is neat here is that the property names are readable enough, and there's a lot of them to play with. Great !