WEBlog -- Wouter's Eclectic Blog

Wed, 04 Aug 2010

Something's very wrong...

I maintain NBD

I have an irrational emotional dislike for python. (I know, I know).

I needed to build nbd in a clean environment. So I set up the environment, install openssh-client (to get the sources across) and then do apt-get build-dep nbd.

That pulls in python.

Something is wrong with this world.

/en/computer/debian

PermaLink

Mon, 26 Jul 2010

Maintainer stupidity

As a buildd admin, you get to see the myriad of ways in which packages can fail to build. Sometimes this is due to interesting technical reasons about the architecture in question; sometimes... not so much.

cc1: unknown option: -mmmx

For reference, I maintain powerpc and m68k buildd hosts.

configure: error: pkg-config: command not found

Checking build-dependencies is hard. Right?

ccache: failed to create /home/buildd/.ccache (No such file or directory)

No, I don't manually install ccache inside the buildd chroot, which must mean some build-dependency pulled it in. Why one would want to do that, I dunno—it's not like a build that takes longer would produce a different package, so it clearly is not required for the build.

(BTW, the invalidity of $HOME is on purpose -- packages are not allowed to write outside their build directory during builds, which includes home directories)

... and more. Sometimes, I wonder how people get the right to upload. But, well.

/en/computer/debian/buildd

PermaLink

Fri, 04 Jun 2010

Re: beid

Sometimes help is just around the corner, and all you have to do is ask.

Not so long ago, I asked for help on the support software for the Belgian electronic ID card. I'd been packaging that since it came out back in 2004, but recently I've had some issues with getting the most recent version of this software to work properly. Since Debian 6.0 "squeeze" is fairly near, and since I don't want a horribly outdated version of beid to be part of that release, I realized that something needed to happen, and rather soon.

My request got quite some response; much more than I'd expected. There have been answers from people with various amounts of experience in Debian packaging; one of the people to reply to my request was Laurent Bigonville, who's also a Debian Developer. So we've now set up a mailinglist, a git repository, and work seems to be proceeding nicely.

I've also asked for the removal of the old belpic source package. However, due to a mistake on my end, the beid source package rather than the belpic one was removed, which means that it'll have to be re-added to the archive now. That should happen sometime during the next few days, after the worst problems have been fixed. However, since I now asked for belpic to be removed, too, this means that for the time being, there is no eID software in unstable. We'll fix that in due time; stay tuned.

/en/computer/debian

PermaLink

Sat, 22 May 2010

Help!

Yes, there is something like an 'RFH', or 'Request For Help', in the WNPP in Debian, but I thought I'd go through this blog, first, since I believe many more people who will feel inclined to help here will read this blog.

I've been maintaining the beid packages in Debian since, eh, quite a while now; and while it's not always been the package that I have spent the most time on, it's not a package that I want to give up.

However, in recent times, I've had issues with the code. I've not really been able to get the 3.5.2 release to compile properly as a package, which means that for the moment, squeeze is still stuck with a 2.x release. This is a problem, since that old release isn't supported anymore upstream, and what's worse, it does not support newer-issued eID cards. So if your card is less than about two or three years old, you can't even use it anymore. For this reason, the beid packages really, really should be updated before squeeze releases.

I've been trying to spend some time doing so, but real life kept intervening. And I haven't even been able to start looking for the 3.5.3 code that I'd heard rumours would exist.

So, this is a request for help. If you want a more recent beid package to appear in Debian (and, by extension, Ubuntu) any time soon, and you can spare some extra time on maintaining a somewhat convoluted package, you'd be an ideal candidate for comaintainer of this package. You don't need to have any experience working on Debian, but you do need experience with build systems and Makefiles. If you don't have any experience in working on Debian, but would like to learn, this might be an interesting opportunity. If you do have experience working on Debian and are as interested as me in getting this to be up-to-date in the next Debian release, your help would be very much appreciated. Even if you only have time for a little while and would like me to take over once everything is going fine again, that's helpful.

At any rate, if you want to help, contact me at wouter@debian.org. We'll work out the specifics then.

Thanks in advance,

/en/computer/debian

PermaLink

Wed, 17 Feb 2010

Local kernels

LC_ALL=C debian/rules debian/control
md5sum --check debian/control.md5sum --status || \
		/usr/bin/make -f debian/rules debian/control-real
make[1]: Entering directory `/home/wouter/debian/other-peoples-source/linux-2.6-2.6.32'
chmod +x debian/bin/gencontrol.py
debian/bin/gencontrol.py
Traceback (most recent call last):
  File "debian/bin/gencontrol.py", line 331, in 
    Gencontrol()()
  File "debian/bin/gencontrol.py", line 14, in __init__
    self.process_changelog()
  File "debian/bin/gencontrol.py", line 305, in process_changelog
    (distribution, version))
RuntimeError: Can't upload to unstable with a version of 2.6.32-8~local1
make[1]: *** [debian/control-real] Error 1
make[1]: Leaving directory `/home/wouter/debian/other-peoples-source/linux-2.6-2.6.32'
make: *** [debian/control] Error 2

The issue at hand was that I'd created a version of '2.6.32-8~local1', to document that I'd locally branched version 2.6.32-7 with that one config option turned on, but that my version should not be deemed larger than 2.6.32-8 (signalled by the ~ and whatever follows that). However, something somewhere in the Debian kernel build system (I was unable to figure out what, exactly) disliked that version number and told me I could not upload to unstable with that version. So it borked, and told me I had to fix my version.

Well, no, I don't want to do that. Instead, I wanted to disable that check somehow. Turns out that isn't too hard; gencontrol.py not only checks the version number, it also checks the target distribution. So if you don't want to upload to unstable, you just need to tell the tool that:

-linux-2.6 (2.6.32-8~local1) unstable; urgency=low
+linux-2.6 (2.6.32-8~local1) local; urgency=low

Simple, but you'd need to know.

/en/computer/debian

PermaLink

Tue, 28 Jul 2009

Extramadura: gnuLinEx and NBD

So obviously I already knew that the region of Extramadura uses a version of Debian they call gnuLinEx, but I didn't know the specifics. As such, it was nice and interesting that they offered us the option of going to a local school, where we could see an installation of gnuLinEx in action. Obviously I went there.

This was an interesting experience, for sure. When I arrived in Cáceres, I learned from Vagrant Cascadian that the school installations make extensive use of LTSP. This, in turn, uses NBD. Since they run this on 80.000 computers, it's quite likely that they're the largest NBD installation in the world. I had no clue.

So, today, when noticing that nbd-client was used on the local machines, I had a short little chat with José, one of the guys from gnuLinEx who's a Debian Developer, about the fact that I've been wanting to do some work on NBD's performance (mostly profiling runs etc), but that I don't have the setup to do this efficiently. To keep a long story short: I now have a test lab the size of which is several times the country I live in. Whee.

I had my camera with me, and took some pictures. I'll upload them soon, but have some other, more urgent, matters to attend to now (such as 'eat').

Later.

/en/computer/debian

PermaLink

Sat, 25 Jul 2009

RC NMUs

A few days ago, during DebCamp9, someone NMU'ed belpic to close #525593, a 'failed to build from source' bug that was filed against it on april 25th, 2009.

Since such bugs (for good reason) are deemed release critical, my package was facing the prospect of being thrown out of testing, and this person (who shall rename nameless in this blog post, since it is not about fingerpointing; but if you must, check the bugreport to find out who) did an upload to prevent that from happening.

In and of itself, this is a good thing. I don't like it when my packages have bugs, and generally try to keep the number as low as possible. There used to be a time when my maintainer bug listing had zero open bugs for most of the time, and though I long ago gave up trying to keep it that way, it is still a goal I would like to reach at some undefined point in the future. Yet, with this particular upload, I was quite unhappy, even going so far as to cancel it, preventing it from going into the archive; and when, over lunch, I discussed the situation with Adeodato Simó, he seemed to feel that I improperly blocked this upload; that I instead should have allowed it to proceed. I felt as if he was almost hostile to my notion that the uploader should have coordinated more with me than had happened. I tried to explain why I felt the uploader had done something wrong, but I do not think I convinced him.

After having thought about it for a few days now, I still don't feel I was wrong in my actions; and I don't like the idea of possibly being a bad maintainer. So here's my position on the whole thing:

First of all, the bug was indeed open for a long time. The reason is that I didn't have much spare time in May to look at it. I also had no clue as to what the problem was, which makes it kinda hard to fix it. I did have some spare time in June, but since Belgian citizens have to file their tax reports in that month, and since you may need the software in these packages to be able to do so, I did not think it was proper to indeed do an upload before the deadline of June 30th; I did not want to have to scramble to fix a botched upload at exactly the wrong time.

On the 11th of this month, a patch was sent to the bugreport that would fix it. This was the weekend before I would leave for DebCamp/DebConf, so I postponed working on it until I would arrive here in Cáceres.

Secondly, what I'm most upset about is the fact that the upload wasn't tested. It couldn't have been; the only way you can test this package is by using a smartcard reader and a Belgian ID card; since the uploader does not have the Belgian nationality, I doubt he has such a card. I don't personally do an NMU that often; but when I do, I consider it my duty to perform extensive testing, even more so than with my own packages. In this particular case, doing such tests is quite important, as I've had issues with the software in the past, where new builds wouldn't work properly for reasons that I haven't fully been able to pin down.

Of course the uploader couldn't know all that, but this is exactly why he should have talked to me before doing the upload. Of course, in theory, I could have documented all my knowledge about the package; but in practice, it's pretty hard to do that well (i.e., many of the things I know about my packages involve 'feeling', 'instinct', and 'experience', which does not easily translate to 'words').

For clarity: I'm not upset about the fact that I've been NMU'ed. That's happened in the past, even for this particular package, and that's been perfectly fine. What I am unhappy about, is that the NMU happened to DELAYED/1 (which gives me very little time to intervene) and that it happened without prior coordination with me.

So, was I really wrong in preventing the upload?

note: since the new upstream release that was available was eventually uploaded to experimental rather than unstable, I just uploaded 2.6.0-7 which also fixes the bug in question.

/en/computer/debian

PermaLink

Fri, 24 Jul 2009

DebCamp 9: stuff

This has, by far, been the most productive DebConf ever, for me.

Not that this means all that much—mostly that previous DebCamps haven't been productive at all—but I still got a few things done.

There've basically been three things that I worked on: d-i support for the Intel SS4000-E; a belpic/beid upstream update; and a minor incremental NBD update.

The latter was simple, and was basically the first thing I did. I had a chat with Vagrant Cascadian, who does a lot of LTSP stuff in Debian, and added some stuff to the package that would make his life a bit easier. Not a lot of work; as I'm also upstream for NBD, and as it's been one of those packages that I've maintained since an eternity, I know the code pretty well. Half a day later, all the code was there.

D-i support for the SS4000-E wasn't that hard (most of the hard parts had already been done by Martin Michlmayr), but unfortunately some bits are not yet completely in order—mostly having to do with the fact that the original firmware has a kernel command line embedded in the kernel. As such, for now, you'll have to connect to the serial line in order to fix the redboot config; maybe we'll come up with a sane way to fix that in the future, but as long as we don't, that does mean you need a serial null modem cable.

Not that you need to solder anything (the main board has a connector for a regular serial port; you just need to plug the right cable to the right connector, so it's not that bad.

The final thing was horrible. A piece of software that presumably works well, but initially wouldn't even compile on my laptop because of pointer/int confusion; a build system made on shell scripts and qmake; and other similar things.

Eventually, I just gave up and uploaded what I had to experimental. It works, to some extent, but should be improved over the next few weeks. That's not for today, however.

/en/computer/debian

PermaLink

Wed, 22 Jul 2009

Buildd maintenance

In Debian, I've been a buildd maintainer since 2001; most of that time was for the m68k port (I still am active there, though not as much as I used to be), but there's also been a short stint with armeb, and since a while I'm now also a PowerPC buildd maintainer. I used to do just one powerpc host at first, but now I maintain both malo and voltaire, with Philipp Kern doing praetorius.

This probably makes me one of the more experienced buildd maintainers in Debian today, together with the likes of LaMont Jones and Ryan Murray. I did do a talk about how this is supposed to work at FOSDEM 2004, but that's now five years ago, and some things have changed since. Also, a not-videotaped talk isn't very helpful if you weren't there.

So I'd thought I'd write up what it means to be a buildd maintainer. There's of course the documentation on the Debian.org website, but that only explains how the system works in theory; it does not explain what us buildd maintainers tend to do on a daily basis.

So let's have a look at that, shall we?

Basically, the work of a buildd maintainer is pretty monotonuous, and an experienced buildd maintainer will usually have a set of scripts to help them. Their work can be categorized into three main categories. In order of frequency, these are:

1. Log handling;
2. State handling;
3. Host and chroot maintenance

Log handling

The first is the most obvious one. Every time the buildd builds a package, it will send a full log of the build to buildd.debian.org and to myself. The successful ones are signed with a simple script:

#!/bin/bash
tmpfile=$(mktemp)
sed -i -e '1,/\.changes:$/d;/^[[:space:]]*$/,$d' $1 | tr -d "\200-\377" > $tmpfile
cat $tmpfile > $1
rm $tmpfile

Easy: use a sed command to fish out the embedded .changes file, and write that to the original file. I use a folder-hook to set this script as my 'editor' in mutt when I'm in my buildd mail directory; thus the result is thereby mailed off to the buildd. In that same folder-hook, mutt is also configured to send the reply gpg-signed in the 'traditional' format, without confirmation, and with just one keystroke, so that (after I have entered my GPG key passphrase) I can send off all the signed changes files in one go. A possible improvement could be to change the macro so that it would work with mutt's 'tag' feature (it doesn't, currently), but that's not a big issue (currently, doing 100 mails takes a few seconds and some careful counting).

Note the 'tr'; this is to avoid 8bit characters from appearing in the mail, which might otherwise be converted to their quoted-printable version in transit to the buildd; and since buildd-mail (the part that receives that mail) does not understand MIME, this would corrupt the GPG signature. This way, we do lose a few characters from the changelog, but that doesn't really matter -- the source still contains the unmodified changelog entry.

With this script, I often handle my 'success' folder several times a day. It's no effort, anyway.

The somewhat harder but much more fun part of log handling, is the handling of failure mails. Since there are loads and loads of possible failures, the scripts to handle these are somewhat more involved. I did receive a script from LaMont at some point, a few years ago, which I then built on so as to improve it. It's not perfect, but it does handle a few common cases with no extra input from me. Some of the others are not so easy, however.

One of the more common cases that cannot easily be automated is the case of the buildd failing to install a certain package, because 'foo depends on bar, but it will not be installed'. This is apt's way of telling you that bar depends on foobar which depends on quux (>= 1:2.3.4-5) which depends on libfrobnitz2, but that has now been replaced by libfrobnitz3. Or some such. The only way to figure out what the hell the problem is, is to walk the dependency tree and figure out stuff from there.

There is an 'edos-debcheck' that reportedly can help with this; personally, I wrote a set of perl scripts that will cache a Packages file into DBM files, and then allow you to walk over them to help you figure out what's wrong. They're not perfect, but if you use the '-v' option to check-dep-waits and verify the output when it tells you about missing libraries, it should be able to figure out the whole dependency tree I described above, and will allow me to write a proper dep-wait response, allowing the buildd host to automatically retry the package when the missing dependency is available.

Also somewhat common and routine are things like transient network failures (in which case we use either 'retry' or 'give-back' if the buildd hasn't figured that out by itself and done the latter), the maintainer uploading a new version of the package while the previous version is building (resulting in wanna-build firing off an email to the buildd host, which in turn results in buildd killing the build by removing the build directory; this is not always easily distinguishable from a regular failure, so I commonly respond to that mail with a failure message; if it did indeed fail because of a newer version, then buildd will notice that and ignore my mail), the incoming.d.o Packages file (which is only available to buildd hosts, so don't ask) being out of sync with reality (which happens 4 times a day for about an hour. In this case build-deps will fail to install, requiring a retry or give-back), and similar things.

Other things are less common; but because of that, they are not routine and require an in-depth investigation. Sometimes the fix is to just file a bug report and/or to mark the package as 'failed' (and let the maintainer or a porter handle the problem); sometimes the failures are due a maintainer script in a package being utterly broken, resulting in either some build-deps being uninstallable or (worse) the buildd chroot being fucked up. Sometimes a build is interrupted halfway through, leaving the chroot in an unclean state (sbuild is not pbuilder, and does not remove and recreate its chroot between builds). This would push us to category 3 of our work.

Basically, however, figuring out which is which takes some experience. Not all compilers are based on gcc (there are some really weird languages in Debian), and thus not all of their error output is the same; learning their different error modes can help quite a lot. Additionally, by continually compiling 10G worth of software, you'll be stress-testing your toolchain. If you've never seen an 'Internal Compiler Error' before, you will once you become a buildd maintainer, and it helps if you know what they are and how to deal with them (even if there isn't much one can do beyond filing bugs).

Obviously, handling failures takes some more time than does handling success mails, and it's not something I do quite as often. The exact time between both varies, but it's usually somewhere between a few days and one or two weeks—unless I suddenly stop receiving success mails from one of my buildd hosts, in which case I know something is utterly wrong and will usually investigate immediately.

State handling

With 'state handling', I mean managing the state of a package in the wanna-build database. There's help about this from the people on the debian-wb-team mailinglist; call me oldfashioned, but I still do consider this to be the final responsibility of the buildd maintainers. After all, the routine state changes are a result of decisions that I make; as such, if I fuck up, it should be me who fixes the fuckup. Also, if I mark a package as 'failed' because I believe the maintainer fucked up, then the debian-wb-team people may not know about my reasoning there, and might give the package back to another failure (although I would consider the latter pretty rare).

These requests are pretty common. Quite often, they're unnecessary—many maintainers are unaware of the intricacies of the wanna-build system, and may misunderstand that when a build is in dep-wait state, it will automatically migrate to needs-build once dependencies are available. About as often, however, they are very much necessary, and, since regular Debian package maintainers do not have access to the wanna-build database, require someone who does have access to said database to update it for them.

Having said that, there are cases where I will preemptively edit the wanna-build database. Usually this is to do something useful with packages in 'Building' state that have been in that state for far too long; either upload the package if its signature mail got lost (which happens once in a blue moon), or give the package back if its build was not attempted even though it is marked as such (this should not happen, but the system is not perfect and it does). Sometimes this is because I figured out that some common build-dependency (say, the GTK or Qt libraries) are in a transitional state and currently not installable; and rather than having a build daemon try a bunch of packages and failing them all, I may want to note in the wanna-build database that they should not bother attempting these 75 packages before the GTK package was done. This isn't done as often on the official Debian machines (since the release managers will do it for me there), but in m68k we do need to do this ourselves.

These kind of requests happen once every few days up to once every few weeks, and take little time to deal with.

Host and chroot maintenance

This is the hardest and least fun part of buildd maintenance, but it is just as necessary. Luckily, it is not as often needed.

Because Debian Unstable is a system that's in a constant state of flux, often things will break. This is even more of a problem on a buildd chroot, since it builds out of incoming; a maintainer may upload a package with a fucked postinst script, have its build succeed, but then fail spectacularly to install. This maintainer may notice that, and may upload a new package half an hour later. As such, the broken package will not end up on the system of a user or Debian Unstable, but between the time of the upload of the broken package and that of the new package, the old package will be available to buildd hosts, who may use it to completely and utterly destroy their build chroot. The joys of having a high turnaround time.

Luckily, Debian package maintainers are not stupid, and this kind of fuckup does not happen every other day. It does happen, however, and when it does, this often means manual work for the buildd maintainer. In the best case, it's a matter of syntactically fixing a postinst script and calling 'apt-get -f install' or 'dpkg --configure -a'. In the worst case (which is almost, but not quite entirely, totally unheard of), it's a matter of rebuilding the buildd chroot. In addition to that, a machine which runs 24/7 for the sole purpose of building packages tends to generate quite a lot of disk activity, which in turn tends to be detrimental to the disk in the long turn. If not looked after properly, disks will die, taking the entire buildd chroot with them. That requires rebuilding them. Obviously, this last issue is dealt with by the Debian System Administration team in the case of official Debian hosts, but the same is not true for the m68k port.

A somewhat more common thing that needs to be taken care of is the fact that buildd does not in all cases clean up after itself. For instance, when a new version of a package is uploaded to the archive between the time that the buildd host built it and the time the buildd maintainer sent the signed .changes file back, then buildd will say "I haven't got that package taken as Building" and refuse to upload it. This makes sense (you can't upload an old version of a package, since there wouldn't be any source for it, and dak would refuse the upload), but it does mean that the packages aren't cleaned out. Arguably a bug in buildd-mail, over time it will result in the disk filling up with outdated packages, and those require manual work from the admin. I recently (as in, a few hours ago now) finished a script to check each .changes file in the "build result" directory against the wanna-build database, and list those that are no longer necessary. I already had a script that, given a list of .changes files, would remove every .deb file listed in the given .changes files, and then proceed to remove the .changes files themselves. Combined, these do make that kind of work somewhat less of a burden.

As said, this kind of work does not need to be done all that often; for instance, I just cleaned the build result directory on voltaire and malo, my two powerpc buildd hosts, and found old files from late 2008...

And that's it, I guess. It may seem to be quite much, but in reality it isn't; the thing I've always liked about buildd maintenance is the fact that you do something little for Debian every day, but that it ends up being something big and helpful after a while.

Of course, the little things are the cherry on the cake. By looking at a lot of build logs, one eventually learns a thing or two about build systems, which is valuable knowledge. Getting build logs from the whole of Debian allows one to learn things about the archive that many people don't know about—for instance, did you know that we had a package called trousers? I didn't, until I signed the buildd log...

Update: changed the URL of this post to be under the buildd/ directory, rather than having it conflict with that and thus killing its permalink and making it impossible to comment on this post. Oops.

/en/computer/debian/buildd

PermaLink

Sat, 16 May 2009

Belpic 3.5.2 has been released.

... and this time, with the source. Finally; it's been about a year since 3.5.0 came out without any source.

That's not to say it's going to be in the archive tomorrow; they changed the build system (again), and the source has been overhauled to such an extent that it basically isn't even the same software anymore. I'll have to re-learn everything, make sure it builds properly, and hope to do so by the time squeeze releases.

Yes, I realize that squeeze is still far away from release. Sigh.

Perhaps this is a good project for debcamp...

/en/computer/debian/belpic

PermaLink

Mon, 20 Apr 2009

On Xorg and HAL

Recently there's been a bit of a fuzz about the fact that the xorg folks have introduced HAL as a requirement for X. I understand this is to make things easier.

So instead of configuring stuff through /etc/X11/xorg.conf, you now have to configure things through /etc/X11/xorg.conf, /etc/hal/whatever, and /etc/default/console-setup. In three completely and utterly different syntaxes, no less (xorg.conf format, XML, and shell script). How exactly this is 'easier' escapes me.

I presume this makes life easier for those multiseatcomputer folks; but for those of us with a normal computer, it doesn't, really.

Oh well.

/en/computer/debian

PermaLink

Sat, 18 Apr 2009

Using NBD to get rid of an LVM installation.

LVM can be a virtue. However, it can also be a pain at times; for instance, there's loss of diskspace due to LVM's metadata that needs to be stored as well. This amounts to approximately 5% of the diskspace (if I'm not mistaken), which can be quite a bit. Also, this is an extra abstraction layer, which requires memory and CPU time to be processed. This will be negligable on most modern computers, but sometimes it isn't.

I had a system that had quite a few things on LVM, and wanted to get rid of them. Unfortunately, to remove the LVM physical volumes would mean that I'd have to get rid of the data, which would mean storing it somewhere else, which in turn would probably mean that I wouldn't be able to use the system for quite a long time. That wasn't acceptable.

So, instead, what I did was to export a whole hard disk (one with enough space to hold the logical volumes on that machine) using NBD, and migrate stuff that way:

• First, set up the NBD export. The config file should look something like this:

[generic]
	user = nbd
	group = disk
[export]
	exportname = /dev/hde
	port = 1234

This will export all of /dev/hde on port 1234. Obviously you wouldn't want to do this on an untrusted network, but that's always true with NBD (in my case, the link was a 1M crossed cable -- quite safe).

• Next, we'll connect the client:

nbd-client <server> 1234 /dev/nbd0

... assuming /dev/nbd0 is not in use yet. Provided the client machine is running kernel 2.6.26 or newer and nbd-client 2.9.10 or later (e.g., Debian Lenny or later), and you're running udev, this will create a number of /dev/nbd0pX device nodes, with X starting at 1, one for each partition on the disk. Of course, one can also export a specific partition rather than the whole disk, or just one file on a filesystem, or create a physical volume of the entire disk rather than just a partition -- your choice.

• Now we need to add the networked block device to the LVM volume group. Assuming the second partition is the one we want to use:

pvcreate /dev/nbd0p2
vgextend /dev/<vg> /dev/nbd0p2

If you now run 'vgdisplay', you'll see that the size of the LVM volume has increased considerably.

• Next, we'll want to move data over the network. Assuming the 'local' LVM volume is on /dev/sda7:

pvmove -v /dev/sda7

This will show progress information every 15 seconds while the data is being moved from the local hard disk to the NBD device. You'll be able to continue using the system as usual--although of course the system will gradually get slower and slower as more and more data is only available through a comparatively slow network connection.

• Once everything's moved to the other sytem, you're all set to change stuff locally:

vgreduce -a /dev/<vg>
cfdisk /dev/sda
...

• Finally, you'll need to move the data from the NBD device back to the local hard disk. If you're still using LVM, that can be done using another pvmove operation. If not, you'll have to do something else, like cp -a. Obviously in that case, there will be some time in which you won't be able to use the system properly.

This final step might not be as simple as it sounds; e.g., if your /home filesystem is on LVM currently and you want to move that out of LVM, you'll have to make sure only root is logged on; If /var is on LVM, too, you may have to go to single-user; etc. Figuring out the details for this one is left as an exercise to the reader.

However, I will note that since somewhere between etch and lenny, doing root-on-NBD is possible; and if you're running unstable, then doing root-on-LVM-on-NBD should be possible, too, although I would suggest a little caution before trying that.

In any case, using NBD in this way just saved me a whole lot of no-computer-time here. Which is great.

/en/computer/debian

PermaLink

Mon, 13 Apr 2009

Ugly fonts

This week seems like ugly fonts week.

First, KDE4 was uploaded to unstable. Now I'm not a KDE user, but I do cherry-pick a few KDE applications which I find useful or interesting, and use them in my IceWM environment; these are mainly korganizer (with konsolekalendar in my .bashrc), kdebluetooth, and digikam.

First thing I noticed is that something went horribly bad with the fonts -- see this screenshot for an example of what the problem is; the korganizer window in the background has ugly artifacts, while the kbluetooth menu in the foreground is still a KDE3 window which does what I was expecting.

Second, someone thought it good to add console-setup as a dependency of Xorg 7.4. Now by itself, there's nothing wrong with that, except for the fact that console-setup by default changes the console font to something else. I had, in fact, tried out this setup a while back, but eventually (after a few weeks), decided that I did not like this font, because it's a) ugly and b) unreadable on a high-resolution 12" display.

So I changed /etc/default/console-setup to get the VGA font back, ran the console-setup initscript, and all was well.

Until my laptop was resumed from suspend-to-RAM. Apparently the console font is changed at that point in time, too, with some other configuration, and I'm not sure how to fix it.

Oh well.

/en/computer/debian

PermaLink

Thu, 19 Mar 2009

Dealing with apt's GPG signing stuff -- the right way.

Philippe blogs about how one can import a GPG key into apt's GPG keyring so that it will stop complaining about unknown keys. While his method will work, it has a major flaw:

Importing random keys without checking them first makes secure apt totally useless, since it allows an attacker to replace an apt repository with another one that he signed with his own key and you won't even notice because you blindly import keys anyway.

So what's the right way? Depends:

• If you have a GPG key yourself, and you have built a local web of trust (with gpg's --update-trustdb command—beware, this is an interactive process that can take a long time if you have a lot of keys in your local gpg keystore), you can verify that the key really is what it claims to be:

  wouter@celtic:~$ gpg --edit-key 6070D3A1
  gpg (GnuPG) 1.4.9; Copyright (C) 2008 Free Software Foundation, Inc.
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.
  
  
  pub  1024D/6070D3A1  created: 2006-11-20  expires: 2009-07-01  usage: SC  
                       trust: never         validity: full
  [  full  ] (1). Debian Archive Automatic Signing Key (4.0/etch) <ftpmaster@debian.org>
  
  Command> quit
  

  Why --edit-key? Because --list-keys, --list-sigs, or even --check-sigs does not show you the validity information. The --list-sigs --with-colons output does, but that is meant for computer consumption, and isn't very useful for human beings. But --edit-key contains the information too, so that works.

  What you need to look at is the information that immediately follows 'validity:'. In my case, it says 'validity: full', meaning, "enough" people who I have indicated (using --update-trustdb) to trust that they can sign keys the proper way have signed this particular key. "Enough" is configurable, but for now, let's ignore that (there's plenty of data about that on the web if you want it).

  If your output says 'validity: full', then go ahead and add it to apt's trusted keyring:

  gpg --export 6070D3A1 | apt-key add -
  

  Note: go ahead and ignore the 'trust: never' data in the output. This represents the fact that I told gpg not to trust the key when doing gpg --update-trustdb. This doesn't mean I won't trust data that's signed with this particular key; however, it does mean I won't trust key signatures signed with this particular key. Since this is not a personal key, but a key only used to sign data with, this is the right thing to do for this particular key.

• Alternatively, if you don't have a keyring, don't want to maintain your local web of trust, don't understand what I just said, or any combination of the preceding, just check what the maintainers of the particular repository to which this key corresponds tell you. If they're good maintainers, they should've given you a fingerprint right beside the information that contained the apt repository. Copy-and-paste that information on two command lines:

  gpg --keyserver subkeys.pgp.net --recv-key <fingerprint>
  gpg --export <fingerprint> | apt-key add -
  

  If they didn't give you that information, then send a giant cluebat mail to them. They really should've done that.

• Finally, if you don't care about all this gpg stuff and don't think anyone would be interested to break into your system, you'd be wrong—scriptkiddies don't look at who's using a particular system, they just like to be stupid. If you insist, however, you should note that you can actually disable secure apt, by adding the following to a file under /etc/apt/apt.conf.d:

  APT::Get::AllowUnauthenticated "yes";
  

  Not that I'd recommend it, but if you insist...

/en/computer/debian

PermaLink

DV from webcams

DVswitch is a neat piece of software, but it only works with DV streams. If you want to use something else (like, say, a webcam), you'll need to do something 'special':

wouter@celtic:~$ mkfifo dvsource-fifo
wouter@celtic:~$ ffmpeg -f video4linux2 -r 25 -s vga -i /dev/video0 -target pal-dv - > dvsource-fifo
wouter@celtic:~$ dvsource-file dvsource-fifo

And there you are, a DV stream from my webcam. Not that this is very useful, probably—but who cares, right?

/en/computer/debian

PermaLink

Wed, 11 Mar 2009

Samba upgraded to lenny

Samba.grep.be, my main server (which occasionally also hosts a domU for p2) was upgraded to lenny this monday evening. I wish I could say there were no issues, but unfortunately that's not true.

At first, everything seemed to run smoothly; of course I did have to recompile a few locally written programs to run against some SONAME changes (programs that are too trivial to package, really, and too ugly too), but other than that, everything was fine.

It went wrong sometime during the night. Apparently something didn't like the new kernel or the new xen, because everything started to lock up. Since I only noticed this when I was at a customer the next day, I couldn't fix it until I got home again in the evening (meaning, downtime for a full day); and unfortunately, the exact failure mode meant that not much was written to the log -- certainly not enough to figure out what went wrong:

Mar 10 04:04:41 samba kernel: [ 7033.011523] INFO: task cron:21006 blocked for more than 120 seconds.
Mar 10 06:43:46 samba kernel: [ 7033.011614] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 10 17:35:04 samba kernel: [ 7033.011701] cron          D 7fffffffffffffff     0 21006   4147

After that, nothing before the reboot, which happened at around 21:00. Take good note of the timings on those messages, and the apparent discrepancies between the syslog timestamp and the kernel timestamp. 7033 seconds after boot was around 3:30 AM...

When I arrived at the physical server, this kind of message was rolling over the console several times a second, with just the name and PID of the blocking process changed. And with several more lines following the message, of course. As a result, the system was not responsive at all.

I didn't file a bug since, frankly, there's so little information that I wouldn't even know what to file a bug against (the kernel? Xen? Some rogue process that runs as root and changes settings which cause a driver to block? I wouldn't know). Instead, all I could do was to reboot the server and hope for the best.

Not that I like that kind of situation. Fortunately, the problem didn't reproduce itself today; here's for hoping that it never will.

/en/computer/debian

PermaLink

Sun, 08 Feb 2009

Tartan theme

Debian so needs a Debian Tartan theme.

For reference, my .Xresources file contains these xtartan configuration items:

XTartan.colorCode.EB: rgb:09FF/21FF/48FF
XTartan.colorCode.DR: rgb:57FF/0000/0AFF
XTartan.colorCode.DY: rgb:E4FF/A8FF/48FF
XTartan.Mine.nextTartan: Debian
XTartan.Debian.sett: %a EB16 DY2 EB16 DR16 BK2 DR16 W6 BK2 W2 BK2 W2 DR6 W2 DR6 W6 BK2 W2 BK2 W2 BK2 W2 DR6 W2 BK2 W2 DR6 W2 BK2 W6 DR6 W6 BK2 W2 DR16 BK2 DR16

Which (when calling xtartan -r -t Debian) provides for a nice Tartan background; but it's not a theme. I'd do it myself, but I'm not a graphical guy; but I would so love such a theme.

Someone with graphical skills please make it!

/en/computer/debian

PermaLink

Mon, 19 Jan 2009

Re: emulated buildds

Aurelien, your claim is wrong. About a year ago (IIRC; might've been longer), the Debian/m68k team decided that it wanted to do emulated buildd hosts, since that would allow us to more easily keep up with unstable. We discussed it in the team, we discussed it with ftp-masters, and we all decided to go for it. We've had emulated m68k builds go to the archive for quite a long time, with full knowledge and agreement of ftp-masters. We realized that emulated builds could be problematic, but we evaluated the issues and decided to take that risk, as a team.

The reason your key was rejected for uploads of arm binaries was because you started doing those emulated builds without discussing it with the arm buildd maintainers, and without discussing it with the arm porters. You just decided it might help, so it must be good, right?

Finding the difference is left as an exercise to the reader.

/en/computer/debian/buildd

PermaLink

Thu, 08 Jan 2009

On DebGem

I feel I should comment on the whole DebGem thing.

About a week ago, the guys from that project mailed me, stating that they were about to go public, and if I would please have a look at it before they would do so and perhaps give some constructive criticism.

So I did. I must say I wasn't very impressed, initially. What you get there is what the name suggests: a Ruby Gem, but then just repackaged as a Debian package. As in, files laid out exactly the same way. As in, still not FHS compliant. As in, many other problems.

I came up with a list of problems that was quite long, and they've fixed most of them; but the fundamental problem—that RubyGems are peculiar beasts, making it hard, if not impossible, for Debian people to package them so that they adhere to Debian's strict quality requirements—remains, and this service does not (or cannot) solve that.

To be sure, it's a step in the right direction, which will ease the maintenance of Ruby software on many a Debian system, and for that they deserve praise. However, it is not a solution; much more work needs to be done—and it needs to be done within the Ruby community.

/en/computer/debian

PermaLink

Fri, 02 Jan 2009

Dependency graphs

Andrew blogs about this nice Debian Dependency graph generator over at gnowledge.org.

It nicely shows why I'm not entirely fond of the belgian electronic ID software: just have a look at the dependency graph for libbeidlibopensc2. Seriously.

/en/computer/debian

PermaLink

Thu, 18 Dec 2008

Manoj Srivastava resigning

I went on IRC to ask a simple technical question earlier today. I wasn't going to ask it in the #debian-devel channel, since it was a samba question rather than a debian-related one; but since my IRC client was still running, and had that channel active, I couldn't help but notice some discussion about "what-ifs" when there isn't a secretary, and who gets to do what then.

Surely this was idle speculation, I thought? But that can sometimes be fun, so let's look at the backlog, and see...

That's when I found out that this wasn't, in fact, idle speculation.

Oh dear.

Well, that's the end of an era, I guess. Not that it wasn't totally unexpected; when I learned that Neil McGovern had become "assistant secretary", I already suspected that Manoj was going to resign eventually, and he does confirm that suspicion in his resignation email. However, the suddenness of it certainly was unexpected.

Here's a tip of the hat for everything Manoj has done for Debian so far. I can only wish to be at your level.

What does distress me, however, is a part of Manoj's email, in which he mentions some people had been trying to get him kicked from the project:

As to the people who emailed me that they are putting together a petition for the DAM to have me removed from the project, I hear you too. I am going to spend the next few days evaluating how important the project is to me, and whether I should save you the bother or an expulsion process.

Well, Manoj, if my voice means anything to you: ignore those people, and stay a happy Debian Developer. You became project secretary not too long after I became Debian Developer, and over the years I've learned to respect you as a person with great integrity for your work, who would always do what you considered to be "the right thing". Even when I disagreed with you on what "the right thing" is (and there have been cases of this happening, including this particular vote, although I wasn't very active in the discussions), I have never been able to pinpoint our disagreement on you abusing your position as secretary in order to win the argument, or some such. You have been consistent and (where necessary) predictable in performing your duties as project secretary, and that is a great compliment. I can only hope we'll see those same virtues in our next secretary.

As for that expulsion process, please do not tell us who those people were, for I do not like seeing people made utter fools of in public.

/en/computer/debian

PermaLink

Mon, 15 Dec 2008

On fixing firmware blobs

Ingo blogs about the current vote, and wonders whether we did not have enough time in two years to do something about the fact that there were firmware blobs in the kernel.

Well, we did, and we did. It's just that in two years, several new kernel releases came out, and that also resulted in some new firmware blobs entering the kernel. This, in a way, is a relatively new issue.

Having said that, I do agree with his sentiment; even if they're new issues, they could and should have been fixed ages ago. As I said in my 'no more hitting' post, too.

Speaking of which, Jorg did not understand why that post did not show up, either. So I've just moved it into /en/computer/debian (where it belongs, really); perhaps that will fix it. Having been the maintainer of Planet Grep for a few years, I know now that this kind of strange behaviour would not be beyond planetplanet...

/en/computer/debian

PermaLink

Planet Sucks

That is, the python software which runs a blog planet; not a particular planet instance.

Case at hand: I wrote a rather extensive piece on the current Debian vote, but it's not showing up on Planet Debian for one reason or another that I can't immediately find out.

I wouldn't even have know if not for Kurt, who noticed that it did show up on Planet Grep, but not on Planet Debian.

I contacted the planet people, hopefully this will get resolved. Perhaps it's related to the fact that I semi-accidentally also committed that blog post about gnodbify-gtk, which I still had to proofread. Oh well...

/en/computer/debian

PermaLink

Sun, 14 Dec 2008

The pragmatist pledge

I had a revelation a few days ago.

Years ago, I had a tendency to hit my computer screen when the computer would not do what I wanted it to do. I stopped that at some point; like I said, it was years ago. That's not the big revelation, though.

The big revelation was the fact that the day of me stopping to hit the screen happened to coincide with the day of me reformatting my Windows partition to free up space for /home.

I guess that, more than anything, is the main reason why I've never looked back. Though I subscribe more to the Free Software philosophy than I do to the Open Source philosophy (that is, "software should be free" rather than "open source improves software quality"), I do not think Free Software is the end solution to all problems relating to software. In that regard, I am a pragmatist: if there is a free alternative, I'll prefer to use that, but I do not shirk away from using something which is non-free—or not entirely free—when it will solve a problem.

In that context, I have recently been somewhat depressed and demotivated because of some things that have been happening of late in Debian. Without wanting to comment on the case at hand, I must say that after having been a Debian Developer for almost eight years now, I am starting to see a pattern:

1. Debian releases
2. Developers are happy, and start working on all sorts of new code to put in Debian
3. We work a lot, and interesting things happen all over our code base.
4. Somebody (presumably someone from the release team) calls a freeze.
5. Because new features are a no-go now, suddenly loads and loads of people have too much time on their hands.
6. Loads and loads of people spend time looking at other people's bugs, and see things they do not like.
7. Suddenly, it becomes immensely important to resolve that issue now. Never mind that we're about to release, that can wait. Never mind that some people actually depend and/or wait on the next Debian release. That doesn't matter. All that matters is that this issue must be resolved now, because it's not free!!!1!!.

As a pragmatist, I'm extremely unhappy about that. Yes, I agree that getting Debian rid of non-free software is a worthy goal, but not at every cost. Yes, I agree that releasing Debian with non-free software is a bad thing, but did people really have to wait until we were about to release to bring that issue up?

The problems were out there for a very long time. Ignorance is not an excuse; if you cared about these things so much, you could have looked for them earlier, too.

I'm not about to say what people should and should not do during the freeze. But as a pragmatist, I do not think that bringing up such issues when we're trying to release tings is a good idea. So I'm putting forward this pledge, which I call "the pragmatic DD's pledge":

I will not, deliberately or otherwise, delay the release of the next Debian stable, by proposing or seconding a GR that will have an impact on the release schedule when the freeze is in progress. I will instead make sure I detect such issues before the freeze has started, or will deliberately wait until after the release has happened before bringing these issues up.

As for the current Lenny vote, here's my ballot:

[ 4 ] Choice 1: Reaffirm the Social Contract
[ 2 ] Choice 2: Allow Lenny to release [...]
[ 2 ] Choice 3: Allow Lenny to release [...]
[ 2 ] Choice 4: Empower the relase team to decide about allowing DFSG violations
[ 2 ] Choice 5: Assume blobs comply with GPL unless proven otherwise
[ 1 ] Choice 6: Exclude source requirements for firmware (defined)
[ 3 ] Choice 7: Further discussion

My only wish is that I would now also be able to find some more time to fix some RC bugs. Sigh...

(as an exception to my policy, comments on this item will be silently dropped. UYOB)

update: Only after I posted this blog entry did I see that many other people had issues with this insanity; and I see some problems with some of the voting strategies that people propose. Let me make something clear: if you do not want choice 1 to win, then you must not vote any of the other options below "Further Discussion". The reason is that since those need a 3:1 supermajority, and since supermajority is decided by whether an option on the ballot beats Further Discussion in our voting system, this means that if more than 25% of our voters vote everything below "Further Discussion", Choice 1 will win by virtue of being the only option reaching its required majority.

If you think, like I do, that "releasing Debian" is more important than "being able to tell people that vrms isn't lying to you", then you should vote Choice 1 below "Further Discussion", and vote all the other options above it, even those which you think are not the best option; because if your preference does not win, then at least you allow the other options that you can live with to stand a chance of reaching the required supermajority, and will allow them to win.

Also note that if you've already voted, you can still change your vote!

/en/computer/debian

PermaLink

Tue, 09 Dec 2008

"Going to FOSDEM"

Planet Debian is currently plastered with this "I'm going to FOSDEM" banner that the FOSDEM people have created. That's great; it means I'll be seeing lots of people there the first weekend of February. I'd put it here, too, but I'm lazy, and people know I'll be there—I'm doing th Debian devroom. So just assume a blue banner here.

What I'm not seeing, however, is talk submissions. So far 5 people have expressed interest in giving a talk, and two have made a definite commitment. That's not enough! I'll probably be sending out second a Call for Talks if I don't get enough submissions by next weekend, but for now, let me give people a list of talk titles I'd like to see in the DevRoom but that haven't been submitted yet:

• The Lenny release cycle: the release team's view
• New stuff from the Debian-CD team. (Not only would that be interesting, I also owe them that much after my messup two years ago.)
• Booting Debian everywhere: the live-CD tools.
• Doing Another Katie: the dak rewrite.
• Debian on the EeePC
• ...or, well, anything people can come up with, really.

People, above all, please remember that FOSDEM is less than two months from now. Usually I can (and do) send the CfT about four months before the actual event, but circumstances have forced us in this position now. If you want to have an interesting Devroom, I need talks, and I need them sooner rather than later. See my CfT for details.

/en/computer/debian

PermaLink

Fri, 14 Nov 2008

Oldest bug ever

I've had my share of old bugs, but #234448 holds the distinct record of being the oldest bug that I ever found a fix for. It's so old, I don't even know how old exactly.

Don't be betrayed by the number 234448. While the Debian bug was filed in 2004, the bug itself is older than that. Take a look at this FreeBSD commit, for example; the original import of the 4.4BSDLite code of the 'trek' program already has the bug. And that was over 14 years ago. The first NetBSD commit is even older: 15 years, 7 months ago.

BSD itself, of course, is much older than that, and so is trek. The copyright header contains 'Copyright(c) 1980, 1993', which suggests that this bug might be as much as 28 years old. That's almost as old as I am...

Like I said, a record.

/en/computer/debian

PermaLink

Sat, 06 Sep 2008

Bughunting is fun

Two reasons:

1. 'foo FTBFS on m68k' is not RC. 'foo FTBFS on m68k because it segfaults during the build since it does bar and baz, which just happens to result in a segfault on m68k at startup but might randomly result in a segfault on every architecture, really' in most cases is.
2. It, well, is just fun to debug someone else's code. On m68k, it's easy to do that, too, if you know a bit of assembly language.

Today's (and last week's) bug: #497262. Xemacs21 segfaults at startup on m68k (and FTBFSes because of that), which is RC for m68k. Reason: confusion between a pointer and its value. In other words: the sanity check might succeed, or it might not, depending on your luck. In fact, the first time the code flows through this buggy path, it does not bomb. Once we do get through this bit, it's also very likely that regular error handling will catch it—the only reason why xemacs segfaults here is because we're still in initialization, and the error object in the lisp stuff hasn't been initialized yet, so we get a nice null pointer dereference...

That alone isn't enough to make it RC everywhere, but it's good enough to make it important and fixable for the maintainer. Like I said, fun.

Sorry for those of you who want to release Lenny ASAP, but I think I'll be making your life a bit harder :->

Next week's bug: a similar issue in emacs, or the build failure of ghc6 on m68k, depending on time and complexity.

/en/computer/debian/m68k

PermaLink

Mon, 25 Aug 2008

Why apt sucks

From a buildd log:

The following packages have unmet dependencies:
  apache2-threaded-dev: Depends: apache2.2-common (= 2.2.9-7) but it is not going to be installed
                        Depends: libaprutil1-dev but it is not going to be installed
E: Broken packages

Investigating with my set of perl scripts yields:

wouter@country:~/code/perl$ ./check-dep-waits
apache2-threaded-dev
dep-wait libmysqlclient15off (>> 5.0.51a)

Note the complete lack of any reference to mysql in the apt output. Also, in case you thought any of the above is bogus: apache2-threaded-dev depends on apache2.2-common which depends on libaprutil1 which depends on libmysqlclient15off. So there.

I've long been hoping for someone to finally fix #325786, but that never happened. Now that I wrote that perl script that I linked to above, I don't think I still need it -- particularly because my script works on my laptop, too, rather than only on m68k machines. Which is good.

/en/computer/debian

PermaLink

Thu, 21 Aug 2008

Solving sudoku puzzles in package dependency relations

Daniel, you are evil. But yeah, it does look to be quite some fun.

/en/computer/debian

PermaLink

Wed, 20 Aug 2008

Hardware test: followup

I have gotten quite some response to my blog post about the hardware test proposal thingy from Bdale.

It would seem I haven't been entirely clear

Someone referred me to a page by Kenshi Muto that parses 'lspci' output into a hardware compatibility list. This doesn't help. It ignores x.org stuff (which is important, too) and the page itself clearly says that it cannot guarantee whether a piece of hardware will actually work.

For clarity: when I said 'Apparently all the other vendors have it, too', I really meant 'Apparently all the other GNU/Linux vendors have it, too'. People have suggested some cooperation between kernel/xorg upstream, other vendors, and perhaps something like freedesktop.org; and while that may be a good idea in and of itself, in the end Debian will have to provide something which it will call 'official' and which will tell a vendor whether or not Debian is supported on their hardware. This thing may not be fully automated or polished; it may need interactivity; but it must be something which will give a result that a product manager may want to lose some sleep over if it's not good enough.

Holger also suggested we try implementing this with Debian Live, rather than debian-installer. This may be a good suggestion; a debian-live image will have a full Debian system available rather than the somewhat limited d-i environment, so writing a test should be easier. Also, I don't even want to think how testing X.org drivers would work out from within d-i.

So what's left is a way to figure out how to do such a live system. I'll be posting some suggestion to the debian-devel mailinglist 'soon'.

/en/computer/debian

PermaLink

Sun, 17 Aug 2008

Hardware test

So something Bdale came up with this morning at breakfast (and during his talk too, apparently, which I didn't attend) was this idea of a "hardware compatibility test"): a bootable image that hardware vendors could run to see whether their hardware would run Debian. Apparently all the other vendors have it, too, and the lack of it may be one of the main reasons why Debian isn't currently supported by a whole lot of hardware vendors yet.

Such a test wouldn't have to do all that much; just boot the machine (if it can) with the kernel that would be used for the installer and the system that is eventually installed; then run through a check of the available hardware, and finally come up with some kind of score that tells the vendor whether their hardware is supported at all, or if not, what they could do to improve the score.

It seemed to me (and to Mark Hymers, who was seated to my left) that this is something that could be done fairly easily with a slightly modified version of debian-installer. It would be okay if there was a different version for every Debian release we do; and I tend to think it's not even going to be a problem if the first time we don't make the release, but release such a test slighty after the release of Debian.

Having such a test would certainly give hardware vendors an incentive to improve their Debian support, especially if it's a simple thing that they can have some summer student do over all their hardware who'd then store the results in a database of sorts. Or so.

Additionally, if we do this right, we could diversify between 'a wireless driver that will probably work if you load ndiswrapper or something similar' (which would get a score that tells them 'yes, it will run Debian', but no perfect score) and 'a wireless driver that works with free drivers and no additional firmware required' (which would get a perfect score if there's nothing more). By doing this, we would put Debian's collective driving force behind a move to better and more Linux-friendly hardware, which can only be a good thing. Bdale seems to thing this could be an industry-changing thing, and I can't think of a reason why it wouldn't work.

Except for one: I'm not sure I'll have the time to work on this myself; and even if I would be sure of that, it's not going to be something that I can do all by myself -- other people would have to run the test on their hardware and communicate the results.

So here's a request: is anyone else interested in this kind of thing? It doesn't sound like something too complicated; and given my business, it surely is something I have a personal interest in, so I will try to make time for it. But I can't do it alone...

/en/computer/debian

PermaLink

Wed, 13 Aug 2008

Panther on Debian/m68k.

Some crazy fellow tried to run Mac OS X Panther (that's 10.3 if I'm not mistaken) on a Centris 650 (a 25Mhz m68k). Procedure: run PearPC on an athlon or another insanely fast box to install OS X in an image that PearPC can boot from. Then copy the image to the Centris, run PearPC on that box, and, uh... wait a week for the stuff to boot.

Cute.

/en/computer/debian/m68k

PermaLink

Sat, 09 Aug 2008

Scattergraphs

Debcamp is nice. Talking to people in person rather than having to do IRC to get anything done is a breeze, really.

One thing I haven't been particularly happy about is the fact that there aren't any good statistics on what the build capacity of an architecture is; i.e., you don't know you're low on build capacity until you suddenly start backlogging and it's too late. So, since Jörg Jaspert, Mark Hymers, and Steve Gran were there, I asked them whether it'd be possible to add some data to the projectb database about installing a binary. One additional column with a default now() option later, and we now have interesting data that I can make statistics of. Of course, since that column was only added four days ago, during one of which the link to ftp-master.debian.org was down (meaning, no uploads), there's not much data there yet; but I can start making some graphs now.

Of course, the hard part is trying to figure out how to present the data in a manner that one can actually get useful conclusions from, which is harder than it seems. Anyhow, I'm trying. For now, on my public space on merkel, you can find a (mostly empty) per-architecture scattergraph relating the size of a binary package against the time between the dinstall of a source package and that of its binary for that particular architecture. If the time between a binary upload and a source upload is "often" more than a day for small packages, then it's obvious that the architecture in question is having problems keeping up.

It's not very useful (yet), but the graphs will be updated on a daily basis, so that hopefully one or two months from now, they will contain useful data. Note that the scales are fixed to go from 0.1 day to 1000 days, and from 1k packages to 512M packages, so as to make sure one can actually compare graphs.

I'm also trying to come up with a useful line-based graph, so that it is possible to compare architectures over time against eachother. This will probably involve something with averages and standard deviations or some such, I guess. Not sure yet.

/en/computer/debian

PermaLink

Tue, 29 Jul 2008

Debian testing

Martijn isn't too sure about Debian testing, though he doesn't go into much detail about the "not sure" bits. Well, Martijn, even though I could really use some more detail, let me make an attempt:

• Debian Testing isn't the perfect thing to use at all times. Especially right after a release, when people start uploading all kinds of crap to Debian unstable, it's probably better to stick with Debian Stable for a while. Right before a release (such as the time we're in right now), however, using testing on a laptop is probably a much better choice if you want reasonable up-to-dateness of the software on your system.
• In case you didn't know, testing does get security updates these days (since quite a few years already, actually).
• In the pre-sarge days, installing Debian almost required some magician to do it for you. These days, however, installing Debian is much easier, so if you're afraid of that, don't be. Also, the default "desktop" installation can compete with Ubuntu in many areas already, although there surely is some more work to be done.
• Debian is a bit more conservative with updates than is Ubuntu, even in the development distributions ('testing' and 'unstable'). This is generally a good thing if you don't want your system to break in all kinds of "interesting" ways right when you don't have the time (although that might still happen, though more so with unstable than with testing).
• Most of the bits that are available in Ubuntu are also available in Debian; and in Debian, in most cases they're done more thorough, because of the difference between Debian's and Ubuntu's approach to package maintenance: in Debian, every package has its own maintainer who is solely responsible for the package; in Ubuntu, maintainers are responsible for a whole lot more packages, which improves their view on the distribution as a whole, but makes it more difficult for them to follow up on the state of individual packages.

In all, I still prefer Debian over Ubuntu, mostly because of the higher attention to detail you'll find in Debian; I find that ubuntu developers usually don't have the time to pay too much attention to pesky details. This makes the system work slightly better. But, of course, YMMV, and if you decide to stay with Ubuntu because you think it works better in your particular use cases, more power to you!

/en/computer/debian

PermaLink

Wed, 02 Jul 2008

No more OpenCT

I just uploaded belpic 2.6.0-4, with a few minor changes, and one major one.

The minor changes are, first, a fix to beid-register-pkcs11.html so that it tries to install libbeidpkcs11.so.2, rather than just the .so file, into iceweasel; otherwise you have to install the -dev package (or manually install the library, or change the .html file) to be able to use beid in your browser. Second, there were some typo fixes of which a bugreport had been open embarrasingly long. Oops.

The major change, however, is the fact that the debian packages now no longer have support for OpenCT as an alternative to PC/SC Lite. This isn't actually supported by the belpic upstream; and while the Debian packages did include support for such cardreaders, I do have a feeling that it was causing some problems.

So why did I include it in the first place, you ask? Well, because my very own cardreader wasn't supported by PC/SC Lite. You see, when I originally packaged the eID software back in 2004, most people didn't have an eID yet, let alone a cardreader; and the fairly common ACR38U-based smartcard readers that are thrown to people's heads these days were nowhere to be found yet. So, I googled for a smartcard manufacturer, found Gemplus who claimed Linux support for (at least some of) their models, found the distributor for Belgium, and ordered a smartcardreader. It did work with Linux allright—but OpenCT, only. Only later on did I find that belpic expects PC/SC Lite.

In itself, this was no problem; the first versions of belpic (up to and including 2.3.13) did have the OpenCT and other reader drivers still compiled in, anyway; only with 2.5 and later did Zetes choose to remove them. That's when I started patching, and that's when I discovered that problems began...

So, anyway. No more mr nice guy OpenCT. I needed to do my tax declaration, and since it'd been a while since I last tried to do some belpic stuff (since before the move, actually), I discovered that I couldn't find my cardreader anymore. So, I bought me a proper ACR38U-based one, and did my tax declarations before doing the upload. How's that for testing?

In any case, if anyone beside me was using the OpenCT bits in belpic, they're now out of luck. Go get yourself an ACR38U-based reader—they're only €15ish, anyway...

/en/computer/debian

PermaLink

Sun, 08 Jun 2008

Whee!

 158 Ns+ Jun 04 Debian Bug Trac ( 13K) Bug#250202 closed by Russ Allbery 
That only took, eh, four years. The report started out as a "I'm sick
of all these confusing and different ways to package a package, we need
a solution", suggesting a few extra targets to debian/rules, but then
quickly changed into a proposal to add a README.source file to the
debian/ directory. At some point, early on, I started to prefer the
README.source solution way more than the extra debian/rules targets
thing, but the proposal reached a stalemate as some people did prefer
the extra targets thing. A solution was only reached when Russ Allbery
wrote a compromise proposal that managed to satisfy everyone, which he
first sent out about half a year ago.
Now, finally, the update is in. Which makes me really,
really happy.
View comments on this entry (0 comments)...

/en/computer/debian
PermaLink
        
Mon, 05 May 2008
Microsoft on installing PHP and Perl

On their "get the
facts" site (as if), Microsoft recently posted a set of screencast
demonstrating how much "easier" it is to install Perl and PHP on IIS
than it is on Linux.
Being bored, I had a look at these screencasts. The first covered
installing stuff on Windows; in a 5 minute screencast, they first
explain how you install ActivePerl, and have it handle perl scripts for
a webserver through a CGI interface. This involves way too many steps to
remember for me (including clicking through a wizard, going to the IIS
management interface and enabling it, creating a directory and enabling
that in the IIS interface as a CGI location, and more).
Next, they do the same for PHP. Since they install PHP as a CGI
binary rather than an ISAPI module, the setup isn't as
complex—they conveniently reused many of the settings they'd done
for perl—but it still required quite some steps.
After having watched this screencast, I was wondering how they'd make
it look harder on a Linux system. My guess was that they'd use Slackware
and would compile apache, perl, and php from scratch or some such.
That didn't turn out to be the case, however. They used ubuntu.
To make the process look complicated, they refrained from
using synaptic, instead opening an (almost full-screen) shell window,
in which they use apt-get to install the
libapache2-mod-php5 package. They conveniently also first
installed the wrong version of apache, so that it would do all kinds of
magical and confusing things. Really, however, after having installed
this package, everything works—and they show that.
The perl step should have been ignored. After installing apache on
Debian or any of its derivatives, such as Ubuntu, support for Perl CGI
scripts has already been installed and configured. Configuring
that requires exactly zero steps. However, they still install
libapache2-mod-perl, just to make it look a bit more confusing. It's not
a very bad idea to do this, actually; installing mod_perl makes serving
Perl scripts go quite a bit faster. But if you just want to have apache
serve perl through CGI scripts, as they'd done on the IIS setup, all you
need to do is install ubuntu, and make sure there's an apache installed.
Period.
But that's not all. They cheat. The installation of the perl and php
modules on ubuntu includes the process of downloading the binaries and
their dependencies; on the Windows screencast, the downloads have
already been performed, and are placed on the desktop. This nicely
avoids showing how you need to go to the ActivePerl and PHP websites,
track down installers, download them, wait for your download to
complete, and then manually start the installation process.
To make things even more laughable, they install Perl and PHP as CGI
handlers on Windows, but perform an installation of the same software
through apache's C API. In other words, they told you how to install a
fast and useful Linux system, and how to install a slow and less useful
Windows installation (since a PHP setup that uses CGI will not support
sessions or persistent database connections, amongst other things).
And even with all that cheating and blundering, the Windows
screencast still takes 5 minutes, as compared to the four for
Ubuntu.
I think Mahatma Gandhi forgot one line in his famous quote. It
should have this:
Then they make complete and utter fools of themselves in a
vain attempt to still look relevant
... right before the "then you win".
View comments on this entry (7 comments)...

/en/computer/debian
PermaLink
        
Fri, 18 Apr 2008
Dep-wait parser

One thing that's been bothering me since quite a while is the fact
that apt and friends are not very verbose when things go wrong. Usually
that's just fine; if a package can't be installed, there isn't much you
can do about it anyway.
However, for those of us who have to maintain buildd machines, a
message like the following is not very helpful:
ome packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
  kdelibs4-dev: Depends: kdelibs4c2a (= 4:3.5.9.dfsg.1-2) but it is not going tobe installed
                Depends: libarts1-dev (>= 1.5.0) but it is not going to be installed
                Depends: libavahi-qt3-dev (>= 0.4) but it is not going to be installed
                Depends: libopenexr-dev (>= 1.2.2-3) but it is not going to be installed
                Depends: libqt3-mt-dev (>= 3:3.3.5) but it is not going to be installed
E: Broken packages

If you're not convinced, let me tell you that the culprit is
libopenexr2ldbl. If you've noticed that libopenexr2 is not
mentioned anywhere in the above output, you understand why I hate this.
Usually, figuring out this stuff requires you to manually walk the
dependency tree, which is silly. So today, I finally spent some time to
sit down and write a couple of perl scripts to help me with that:
wouter@country:~/code/perl$ ./cache-packages Packages 
Reading data... done (20492 packages). Closing cache... ready. Have a nice day!
wouter@country:~/code/perl$ ./check-dep-waits 
kdelibs4-dev
checking dependencies of kdelibs4-dev
 checking dependencies of kdelibs4c2a (= 4:3.5.9.dfsg.1-2)
   checking dependencies of kdelibs-data (>> 4:3.5.9.dfsg.1)

[... copious output snipped ...]
  checking dependencies of  libopenexr2ldbl (>= 1.2.2)
libopenexr2ldbl is not available in any version

So, there. Obviously the "Packages" file in the first command line
above came from a Debian mirror. The "cache-packages" script writes a
file "~/.packages-dbm", which contains an MLDBM tiehash of all the
packages, indexed on package name. If you have no clue what that means,
just remember that it parses the Packages file into something my laptop
can load in a split-second rather than requiring 45 seconds to open and
parse the Packages file. Once you've ran that, you can call
'check-dep-waits' as many times as you want, that is, until the Packages
file is outdated and you need to download a new one.
The second command will just sit and wait. On the first line, you
give it a piece of Depends: information, and it will parse that and see
whether the dependency can be resolved. There's a lot of output; but
given how I want to verify dependencies, I thought I'd rather have too
much than too little information. If there are version numbers involved,
this script will call dpkg --compare-versions to do the actual
verification, so I don't have to watch policy for any possible changes,
which is helpful; but it does mean that it'll only work on a Debian
machine.
The code is available for download at http://comp.uter.be/perl-Packages,
FTWCA buildds.
Update: While on the train, I realized that I'd
totally forgotten about virtual packages. That's now been fixed; if you
downloaded the code before, you may want to re-fetch it...
View comments on this entry (2 comments)...

/en/computer/debian
PermaLink
        
Tue, 11 Mar 2008
Whereami or guessnet?

Before my hard
disk adventure, I used to use whereami to manage my
laptop network setup. Whereami is pretty good. It allows me to detect
whether a cable is connected to my NIC, and not even bother wasting
several minutes in trying to get a DHCP lease. It can run tests on
several interfaces, and decide that I am "home" if it finds a known
network on either the wireless or the wired interface. Most importantly,
it allows me to build a script based on where I actually am, so that I
can have my laptop modify several configuration settings that depend on
the actual network I'm on -- say, whether or not to use a proxy, or
whether or not to use an SMTP smarthost.
I was having issues with whereami lately, however, and thought to
look for something else. After all, it's true what they say; whereami
does not properly integrate with ifupdown. If you use whereami,
modifying the network configuration involves "/etc/init.d/whereami
start" or some such; just "ifup eth0" will break things horribly.
Additionally, I was having issues with my WPA setup; there is a testwpa,
but it does not seem to work for me for some reason—and I can't
seem to figure out what's going on in order to file a bug.
Apart from the two issues above, I was having some minor annoyances
with my setup; enough so that after reinstalling my system, I thought to
try something else. The guessnet package is
designed to properly integrate with ifupdown, so I thought I'd
give it a try.
I don't think it is what I'm looking for, though.
#  route_data = ${lookup{
#			${readfile{/etc/whereami/topmost_location}}
#		}lsearch{/etc/exim4/smarthosts}}
  route_data = ${lookup{
  			${extract{eth0}{
				${readfile{/etc/network/run/ifsftate}}
			}}
		}lsearch{/etc/exim4/smarthosts}}
		${lookup{
			${extract{wlan0}{
				${readfile{/etc/network/run/ifstate}}
			}}
		}lsearch{/etc/exim4/smarthosts}}

The upper version is what I used with whereami (but not active, see
the comment marks :). My script would write something to
/etc/whereami/topmost_location, and several other things
(including my exim configuration, as shown above) would use that to
automagically modify their configuration. Which is pretty cool.
The lower version is the equivalent of the upper version with
guessnet. How ugly. And then I didn't even see a way to avoid
ifup from trying to 'up' eth0 if the MII tells me there is no
cable in the NIC. Oh my.
View comments on this entry (0 comments)...

/en/computer/debian
PermaLink
        
Sun, 02 Mar 2008
eIDconfig-belgium

Someone over at Novell wrote an
application to enable eID in various applications with a simple
click: you can enable web authentication in firefox, and email
signing in Thunderbird and Evolution. It also does stuff which I didn't
even know was possible under Linux—enabling eID card use under
OpenOffice.org.
So I'm now a bit in dubio as to what I should do with this. I have an open bug report against
libbeidlibopensc2 that claims the mozilla/firefox plugin should be
automatically registered when you install the package, rather than
having to go through a bit of javascript in some HTML file, and I kindof
agree with that. I could analyze the C# code to see how the Novell
people do it, translate that to C (since C# doesn't work on every
architecture Debian supports, and besides I don't want to depend on
yet another huge list of dependencies after wxWidgets
and Qt), and call the relevant code from postinst to enable the
relevant plugins system-wide. OTOH, allowing every user to make the
choice for themselves, could be a good idea as well. Then again, that's
not really the Debian way (if installed, it should just work).
Then again, I don't think that enabling these plugins system-wide allows
one to still disable it on a per-user basis.
Guess I'll have to give it some thought—other people's insights
are appreciated.
View comments on this entry (2 comments)...

/en/computer/debian/belpic
PermaLink