My mutt said this last night:
894 + Jan 15 Archive Adminis (0,4K) ipcfg_0.1_amd64.changes ACCEPTED
This obviously means that if you wish to use it, you no longer need to go through git; you can just add experimental to sources.list and run 'apt-get install ipcfg'. A few notes, though:
And in case you wonder why the hell I went from 0.1 to 0.3:
ipcfg (0.2) experimental; urgency=low * Rebuild without .git directory. D'oh. -- Wouter VerhelstTue, 12 Jan 2010 17:43:09 +0100
srsly
Yesterday, I had some time to debug ipcfg some more. The main blocker for me to upload it to unstable was the fact that I could not get WPA security to work; therefore, I could not use it myself. In the interest of "eat your own dogfood", I did not feel that uploading some experimental code to Debian that I'm not using myself is a good idea.
That problem is now fixed, albeit through something of a hack, one that I hope will not be necessary forever: I decided to write a plugin to run ifupdown extension scripts (found in /etc/network/if-*.d). It does require some set-up, and there are still some severe issues; but as of now, I am using ipcfg rather than ifupdown on my laptop.
Those interested in trying it out can either wait for ftp-master to ack the upload and then install from unstable, or just fetch it from the git repository.
Since about a week, I have a Nokia N900. They're not actually on sale in Belgium yet, but I managed to get hold of one through a grey market shop.
The device is quite nice. I do have a few irks with the interface here and there, but Nokia has a public bugzilla that I'll surely use to file bugs whenever and wherever appropriate. And after using it for about a week, I thought of migrating this small application which I once wrote on half an afternoon using perl, postgres, and Gtk2, to the device. It's mainly a cataloguing system that I store my DVD collection in (which has gotten large enough that this is necessary), nothing earthshattering. The ideal candidate to get my feet wet in developing for the N900, so to speak.
That effort got stopped dead in its tracks pretty quickly.
Nokia-N900-42-11:~# perl use Gtk2; Can't locate Gtk2.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.8.3 /usr/local/share/perl/5.8.3 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl .) at - line 1. BEGIN failed--compilation aborted at - line 1. Nokia-N900-42-11:~# apt-cache search perl perl - Larry Wall's Practical Extraction and Report Language. libpcrecpp0 - Perl 5 Compatible Regular Expression Library - C++ runtime files po4a - tools for helping translation of documentation libpcre3 - Perl 5 Compatible Regular Expression Library - runtime files libgdbm3 - GNU dbm database routines (runtime version) perl-base - The Pathologically Eclectic Rubbish Lister. liblocale-gettext-perl - Using libc functions for internationalization in Perl
Or, in other words: they only ship the perl bits needed to make dpkg run.
Hrmpf. This would make stuff slightly more... involved.
Oh well.
I received a number of comments on my "Grrr" post, all of which missed the point:
Yes, I am aware that there are many more ways to fix this issue beyond a memcpy. However, the example code is legal and would not crash the application, if not for the fact that libc thinks I am doing something wrong. On top of that, this kind of overflow "protection" only kicks in when the code is compiled with -O2 rather than with -g -O0. While I am not sure whether the difference is due to the absense of debugger symbols, or rather due to the different optimization levels, fact is that software which runs fine in debugging should also run fine in production.
There are good arguments for compiling all C code with -Wall -Werror, and I do that as a matter of course for all C software that I write. However, sometimes automated tools are just wrong in their compile- or run-time bug detection, in which case such it should be possible to disable that detection. This is one such case, and my blog post was more about ranting about the inability to do so, rather than about the fact that I had to memcpy when in fact there were other options available.
But yeah, perhaps I should have been clearer about that. Forgive me for not being clear after having fought with compilers for far too long.
struct {
char str[4];
char sc1;
char str2[3];
char sc2;
} foo __attribute__((packed));
snprintf(foo.str, 5, "%04X", data);
foo.sc1 = ';';
snprintf(foo.str2, 4, "%03X", otherdata);
foo.sc2 = ';';
Yes, I know that both snprintf() calls in the above snippet will overflow their immediate buffer. Yet this code is safe; the network protocol for which this code is written does not actually need nor expect NUL bytes; instead, it wants semicolons. I could of course use a "char foo[9]" rather than a struct as above, but I find this to be slightly more convenient than to count offsets.
However, this code does not work with glibc, because the buffer overflow detection kicks in.
Solution:
char buf[5]; snprintf(buf, 5, "%04X", data); memcpy(foo.str, buf, 4);
In other words: add a stupid and useless memcpy, because someone thinks they're smarter than me. Stupid morons.
I was really happy a few years back, when Martin announced he'd start working on netconf. Not just because I agreed with his assessment that ifupdown needed replacement; also because I had been thinking about a good way to implement such a thing, and had been planning to start writing 'soon'. With Martin's announcement, I made a few suggestions to give him some input, and then put my plans away—I had more important things to take care of, anyway.
Unfortunately, netconf did not manage to reach the ambitious goals that Martin set out, mainly due to lack of contributors. I wanted to help out; I really did. But Martin's choice of python to implement a 'prototype' which then had to be reimplemented in C or C++ didn't get me very thrilled. Not just because I have a severe dislike of python; also because implementing something twice is, in my humble opinion, not quite the best way to do something like this.
But hey, who am I, if I don't put my code out there.
Never the less, I tried helping. I really did. During debconf8 at Mar del Plata, I approached Martin with the suggestion that I start reimplementing those bits that were unlikely to change anymore in C, as a first step toward something that could be in the base system. Unfortunately, that didn't happen. After trying for about two weeks, I just gave up. I thought I could read python code and reimplement that so that it would work the same way, but I couldn't. I thought I could implement python modules in C without knowing python, but as it turns out that's a laughable idea.
For several months then, I didn't think about it anymore. In March of this year, however, I got a little bored. And what does one do when they get bored? Right, you find something to do. I my case, that was 'figure out how bison works'. I'd of course known about parser generators for quite a while, but just never had the time to dive in deep and figure out how they work. When I needed to do a config file for nbd, I instead hand-wrote my parser and used a lexer from libglib. But I wanted to learn how to do things properly, so I sat down.
Long story short, when I had to choose something somewhat more complex to implement, I thought 'how about a hypothetical config file for a network configuration utility', and I started implementing that. It turned out to be fun, so I didn't stop after I knew enough about bison. And just last week, for the first time I successfully used it to bring up my wired interface using DHCP -- and bring it down again. It can also already manage static interfaces through netlink, though it isn't quite able to the equivalent of 'ip addr flush' yet.
Fun.
The system is quite flexible. It has a plugin interface (which needed some refinement that I did using some help on IRC just today), which should allow developers to implement extra functionality by just dropping a shared object in the right directory. In fact, I'm working on the wireless support (hence my blog post of last week) as a plugin.
Obviously the code isn't remotely ready yet. The wireless code has a long way to go. The firewall module needs to be started. It will probably crash and burn if put to the test. But even so, it already has one feature that ifupdown, in its 10 years of existence, never acquired: it will not try to DHCP off an interface if it finds out that there is no cable connected to it—unless you wrote a configuration that asks you to do so.
I wanted things to be as simple as possible, and therefore the minimal valid configuration file that will do something useful is the following:
That's right, it's empty. This will cause the system to bring up the 'lo' interface at bootup, and do nothing else. If you say 'ifup eth0', where eth0 is a valid and existing interface, it will first check whether there is a link, and if so, DHCP off of it.
There's much to do still, but I have now reached a point where I feel the system is ready to meet all challenges it should be able to meet (or so I hope), and where it might be nice if interested people check it out.
The code is in a git repository that's mirrored on git.debian.org, github.com, and my own git.grep.be. The latter is my --mirror repository, but is not available for cloning to anyone else. If you're interested in checking it out, you may want to start with doc/tech.txt, and/or examples/ipcfg.cfg. I updated the former just yesterday, so it's quite up-to-date. On the other hand, while the latter shows the direction I would like the system to take, the code is a far cry from implementing everything shown there, and it has been a while since I last checked that file out, so the comments in there are, in some cases, somewhat outdated. But that's good, because I want to keep the file as a reference of the goals that I originally had, before I'd written this whole corpus of code, so that I can go back at some undefined point in the future and fix things.
Comments are (more than) welcome.
The Bison parser expects to report the error by calling an error reporting function named "yyerror", which you must supply. It is called by "yyparse" whenever a syntax error is found, and it receives one argument. For a syntax error, the string is normally "syntax error".
If you invoke the directive "%error-verbose" in the Bison declarations section, then Bison provides a more verbose and specific error message string instead of just plain "syntax error".
Sounds good, right?
Well, no, not entirely.
syntax error, unexpected $undefined
Well, goody. Now I know what's going on.
note: yes, I do know that there are other ways to debug a Bison parser than just to use the parser error string. It's just that this could have been more useful, like, say, provide the line on which the error is found? The file I'm trying to parse here is pretty large, thank you very much.
Ever since I read about the ATA TRIM command, I've been thinking that something like that would make a lot of sense for things like NBD.
Currently, it's possible to serve a sparse file using NBD. I often do this, for example, testing things like the fix for Bug #513568 (nbd-server choking on a >3 TB RAID device) which would otherwise require me to spend significant amounts of money so that I get enough disks to test. Obviously that's not going to happen.
If we're using sparse files, that means we claim to export more than we actually do. This is obviously useful for testing, but there are other cases where this kind of behaviour could be useful; for instance, when exporting swapdevices to several diskless clients, it would be a rare case indeed where all those clients need all of their available swapspace; it might be safe to slightly overcommit the disk space on your hard disk, on the assumption that none of your clients is going to use all that space at the same time.
Similarly, when a file is removed on a filesystem on an NBD device, the kernel might use the generic TRIM layer described by Ted in the article above to relay that removal to the nbd-server process, which could then make sure the diskspace is deallocated.
However, unfortunately, there currently is no way for an nbd-server to tell the kernel to create a new hole in a sparse file. Creating sparse files is done at file creation time; if you later want to add more holes than you did when creating the file, you need to recreate the whole file from scratch in order to do so.
Additionally, when we're talking about a large amount of data that needs to be removed, it's usually much faster to say 'remove this block of data' rather than 'overwrite several megabytes with zeroes here'—not just in API, also in kernel-side implementation, provided the filesystem supports extents.
I think it might be nice if there were a public (userspace) API to tell the kernel that particular bits of a file or a block device are no longer required, and that they may thus be removed. But then, I'm not very experienced with kernel code, and it might be that I'm just plainly missing something here. Anyone?
So you do computer stuff for work, and what do you do in the weekend? Right, more computer stuff.
Not that I dislike it, obviously. But, well.
The computer stuff that I did this weekend was situated mainly around dvswitch. During their talk at the Debian devroom at FOSDEM, Ben and Holger mentioned that one of the things they'd still like to see implemented in DVswitch is the ability to do tally lights. Such lights are useful, not only for the benefit of those in front of the camera, but also for the benefit of those behind the camera—allowing them to know when they're free to roam about in order to find something interesting to show, or when they should keep the camera somewhat stead because they're live.
Since I was planning on using dvswitch to record the concert (dutch only) of the choir group that I'm part of, and since a concert has different requirements than does a talk, I set about implementing tally light functionality. For now, the actual tally light is just a laptop screen—which turns red when you're live, or green when you're not—but there's nothing stopping anyone from changing that into, say, a USB-controlled light or some such.
In any case, it works now. It's not entirely pretty (involving two processes working with the same socket) and not entirely finished yet, either (it only works for primary sources, not secondary sources), but it gets the job done.
It did require me to refresh knowledge that I hadn't used in literally 10 years, however. I had some C++ at school, but haven't used it since my final exam on that course, and while the code is still readable, it did require me to refresh my memory at some levels. Oh my...
Anyway, that was yesterday, for the most part. Today, a friend came over with a DV camera (since I don't have any), and we ran some tests. Most of it worked immediately, but there were a few bugs, which I ironed out today. Right now, it works correctly—the parts that are implemented, at least.
This video stuff is rather fun to play with.
Some guy from New Zealand blogged about a vulnerability in .desktop files as used by Gnome and KDE, and claims you can write a virus with them for Linux. He actually meant a Trojan rather than a virus, but still.
I'm not saying the vulnerability does not exist. It does, and the trojan he describes should work in theory.
However, in practice, I believe that the real reason why there aren't any Linux viruses is not the fact that Linux is somehow 'safer', but really the fact that Linux isn't a monoculture like Windows is.
If you want to write a Windows virus, you have to deal with, at most, four to five different versions of windows that are currently still in active use with most users. Target that, and your virus will live like there's no tomorrow.
If you want to write a Linux virus, the range of target system is much, much larger. In his very blog entry that describes the vulnerability, 'foobar' describes already two special cases he has to consider: the fact that KDE and Gnome write .desktop entries that get executed on startup to different locations, and the fact that, while most distributions ship either curl or wget, none of them has standardized on either of those, requiring the virus writer to account for that, too.
While his blog entry stops at that, there's way, way more he has to deal with. If our supposed trojan writer wants his trojan to proliferate, he will have to deal with distributions that come with or without SE Linux; distributions that ship with a /tmp mounted noexec; distributions that ship with python 2.3, 2.4, 2.5, or 3.0; users who use Thunderbird, Evolution, Kmail, mutt, pine, or another mail client; and possibly much, much more (this only describes what's required as described in the referenced blog post). All these differences would make writing some malware that would exploit the described vulnerability tedious, at best, and rather impractical in most cases.
Again, I'm not saying the vulnerability does not exist; it does. But I think this lack of monoculture under Linux, not the (perceived) strength of the platform, is what helps users defend against malware on Linux.
I received quite some feedback on my dlhack() post. Most of it negative.
For clarity: no, I do not consider such practices to be the best of ideas. I called it a 'hack' for a reason—I fully expect this mechanism to be replaced by the time I eventually release the code.
It was just fun to do it this way, that's all.
Having said that,
There's one comment from a xine developer that deserves a followup:
I'm sure I have seen a few different software that can simulate keypresses from the keyboard through the user event interface of the kernel with a joystick, and thus you should just need to wire it down with the original xine-ui. Or use gxine and the gxine_client program to control it.
While this will work, there are several problems with this approach:
While all of the above is not a problem if I write a xine plugin, and indeed I might eventually convert the code to a one, the problem with that is that I don't know the xine plugin API yet; and writing a standalone application makes for easier prototyping. For the time being, at least.
Oh, and in case you wonder why I don't just buy a LIRC-capable device and use a regular remote control: I have my gamepad. I do not have such a LIRC-capable device. My budget is not unlimited. 'Nuf said.
In my living room, there's a TV, with a computer connected to it.
In that computer, there's a DVD-ROM player, a hard disk, and a card with a TV output.
On that hard disk, there's Debian with xine to view DVDs
When I originally put it there, I would pick up the keyboard to do some user input. This works, but a keyboard is klunky. So, instead, I've been working on jsxine, a little application that does not much more than open a window, and tell xine to play a DVD there, but use the joystick for UI[1]. My 'joystick' is a Logitech Wingman Gamepad Extreme, which has a cable long enough for me to be able to use it comfortably from the couch, so I can use it as a sort of remote control.
The current version has quite some things hardcoded: button 0 is 'select', button 5 is 'eject', button 6 'previous chapter', button 7 'next chapter', and button 8 is 'start or stop playing'. The primary two axes move around in the DVD menu; and other stuff is not yet implemented. This layout, of course, makes a lot of sense on this particular gamepad, but I doubt it does on other joystick devices. And since I want this to eventually be useful for other people, too, I recognized that I would need some way to configure things.
My initial stab at configuration would create a hash table of available actions, read out a configuration file, and then find the appropriate function pointer in the hash table. While this works, it has one rather important downside: adding a configurable action would requrie another item in the hash table; this would eventually make the hash table horribly long. So I thought, why not use the configuration value to ask the dynamic linker to get me a function pointer?
With a little help from my friends, that's exactly what I did. Proof-of-concept code:
#include <dlfcn.h>
#include <stdio.h>
int function() {
printf("In function!\n");
}
int main(int argc, char**argv) {
int (*funcptr)(void);
void* handle;
printf("foo\n");
handle = dlopen(NULL, RTLD_NOW);
funcptr = dlsym(handle, "function");
funcptr();
return 0;
}
Compile with cc -ldl -rdynamic. Et voila, that gets you an application which can do a lookup of function names at run time.
Long live C.
Of course, there's a bit of a danger in the above example. A user could just write a config file that has things like 'open' and/or 'write' as arguments, or the like. One should make sure that it can be obvious from the function name that it does indeed define an action; for example, by giving them a common prefix, and prepending that prefix to the function name before handing the name off to dlsym().
But other than that, this seems to work. It apparently also is documented in the FreeBSD documentation on dlopen(), so maybe it's not as much a hack as I thought...
Oh well.
Note that on GNU systems, you can also put a #define _GNU_SOURCE before the #include <dlfcn.h>, you can drop the dlopen() call and go straight to dlsym(), with RTLD_DEFAULT as the handle.
[1] Yes, I could have done a Xine plugin. I didn't want to. This is as much a project to produce something useful as it is a project to hack and have fun.
A while back, I wrote about this GObject-based Object/Relational mapper that I wrote, which is called 'GNoDbify'. It isn't complete yet, but parts of the library work; I use it for a project that I'm supposed to commercially support. For those who don't know what that is: an O/R mapper builds objects out of database tuples; so I can run a 'lookup' function on the object class and get one (or more) objects representing data from the database, then perform some modifications on that object, and store it back in the database; my O/R mapper does all the SQL magic.
Last week, I've been working on another object that takes an array of GNoDbify objects, and presents them using the GtkTreeModel interface, for easy viewing in a GtkTreeView:
my @customers = Vpbs::Db::Customer->get_all();
my $model = GNoDbify::ListStore->new_from_array(@customers);
my $view = Gtk2::TreeView->new_with_model($model);
my $renderer = Gtk2::CellRendererText->new();
my $column = Gtk2::TreeViewColumn->new_with_attributes("Name", $renderer, "text", $model->get_column("name"));
$view->append_column($column);
And there you have it: a treeview showing all the names of all the customers in the database. On the classical approach, one would have to use DBI or some such to fetch data from the database, create a GtkListStore or something similar, and then append the data one row at a time to that. Of course that still happens here, in a manner of speaking, but it all happens behind the scenes. It also means that in an application which wants to show a load of data, I don't have to do the same thing all the time.
The magic happens in the second line, and in the second-to-last line. GNoDbify::ListStore->new_from_array is pretty much self-explanatory, really. It takes an array of GNoDbify objects, and returns a GNoDbify::ListStore.
The second-to-last line shows how to add a column to a GtkTreeView. With GtkListStore and GtkTreeStore you basically define yourself what column gets what number; however, with GNoDbify, that isn't as easy (it depends on the implementation of the GNoDbify subclass), so the ->get_column returns the number of a column as represented by the model, and wants the name of the field that you want the column of.
Everything else is really the same.
It won't stop here, though. The original plan for GNoDbify was that it could detect changes in the database, and update the tree model accordingly. Most of that has already been implemented in GNoDbify (though not yet in the ListStore thing), but just needs some debugging. The fun thing is, the above six lines of code would not have to change at all for that.
Of course all this juicyness comes at a price. In order to generalize this much, I had to do a lot of extra stuff; so there's some performance loss. Especially in perl; most of GNoDbify is written in C, and converting perl arrays to and from C arrays is a process that can only be done by looping over the perl array (or the C array) and creating the other side of the equation. Obviously, that's a costly approach, performance-wise; an obvious optimization of the above example could be to have a subroutine that returns Customer objects already in a GtkTreeView in one API call. We'll see.
Autotools can be nice, but they can be pretty ugly too, sometimes. Especially when you're trying to do something that the autotools weren't originally meant to do.
I was trying to generate scripts using autotools, in such a way that they'd be able to use variables that I've AC_SUBST'ed. At first, that did not seem possible, unless I wouldn't mind doing what autoconf.info suggests:
edit = sed \ -e 's,@datadir\@,$(pkgdatadir),g' \ -e 's,@prefix\@,$(prefix),g' autoconf: Makefile $(srcdir)/autoconf.in rm -f autoconf autoconf.tmp $(edit) $(srcdir)/autoconf.in >autoconf.tmp chmod +x autoconf.tmp mv autoconf.tmp autoconf autoheader: Makefile $(srcdir)/autoheader.in rm -f autoheader autoheader.tmp $(edit) $(srcdir)/autoconf.in >autoheader.tmp chmod +x autoheader.tmp mv autoheader.tmp autoheader
(sic, including the "autoconf.in" in the autoheader stub. Dunno whether that's a bug or whether autotools is really that ugly)
This would work, but it has one downside: it can replace instances of "@datadir@" with the likes of "${prefix}/lib/${PACKAGE}". Or, if the user is really sadistic, "${my_stupid_variable_specified_on_configure_command_line}/lib/${PACKAGE}". Personally, I do not like that approach.
So I had to find something else. With C programs, if you want to AC_SUBST something that could end up being something like the above, you'd just make sure you'd have something like -DFOO="@FOO@" in your AM_CFLAGS (or your program_CFLAGS); this would then evaluate your AC_SUBSTed variable, making it available (the proper way) to your C program. Occasionally, this is why it can sometimes be a good idea not to have a config.h...
Anyway, the solution to my little problem was to be found that way. First, I have a little sed program:
#!/bin/sh
sed -e '/@config_sh@/{
rconfig.sh
d
}'
Or: 'replace all occurrences of @config_sh@ in the input with the contents of the file config.sh'. Simple. All we need now is to generate a config.sh file:
config.sh: Makefile env -i FOO=@FOO@ BAR=@BAR@ > config.sh %: %.shin config.sh $(srcdir)/makesh < $< > $@
The first generates our config.sh file; the second uses the sed program above to turn any '.shin' file into a processed file. And now, my shell scripts only need to say '@config_sh@' somewhere appropriate to get access to all my AC_SUBSTed variables. Whee.
(Side note: I could probably run the sed inside the Makefile itself, but apparently my sed- and make-fu is not string enough -- whenever I try that, sed errors out one way or another. Oh well.)
It was recently brought to my attention that the free newspaper "Metro" has an online version, where you can download the entire newspaper in PDF form; or use it in an online version, which consists of a series of images of each page, where if you click on an article, the usemap of that image is a link to the relevant article in HTML form.
Of course my hanlin doesn't have a webbrowser (it has a somewhat limited HTML parser, but that does not understand even hyperlinks, let alone usemaps), so the PDF version is what I need. Unfortunately, the only option which "Metro" provides is a set of one-page PDF files. They are somewhat readable on my hanlin (someone with worse eyes than my own would probably have trouble reading the small letters, but for me it's not a problem); however, the fact that rather than just using the 'page turn' buttons on my hanlin I have to close the current file, open the next, and reconfigure the zoom all over, means that this multi-PDF thing isn't exactly great, and that I'd prefer just getting one PDF file instead.
So I got to work, and tried out a few things.
There doesn't appear to be anything to merge multiple PDF files into one. There is 'psmerge', but that only does PostScript files, not PDF ones. But that's not a problem, because PDF files can easily be converted to postscript and back, right?
Well, no.
The first pdf-to-postscript converter that I tried was 'pdf2ps', a wrapper script around ghostscript. Unfortunately, using that results in some ugliness:
As you can see in the second image, using ps2pdf results in some quality loss. Ghostscript apparently doesn't understand the letter 'A' very well, and (more importantly) loses the anti-aliasing that is part of the pdf file. Converting this file back to PDF and storing it on the hanlin results in an unreadable text.
But it gets worse. Once I had converted all those PDF files to postscript using pdf2ps and merged them with psmerge, the output was gibberish. Or Klingon, take your pick:
In case you were wondering, yes, that is the exact same fragment (counting lines sucks) at the exact same zoom level. Clearly this was a dead end.
I found that there was a second pdf-to-ps converter, called 'pdftops', which uses the xpdf code base to do its thing. This converter produced clearly better PDF output; I could not see any difference between the Xpdf rendering of the original file, and the gv rendering of the pdftops output. Also, the psmerge output does not garble things as badly:
Sadly, it still loses anti-aliasing, and thereby the readability of the document on my ebookreader device. Moreover, the pdftops output seems to confuse psmerge, to the extent that it hangs on some pages.
Another approach I tried was to import the pdftops output into scribus, and create a multi-page PDF file with that program. Unfortunately, however, scribus does not seem to like the pdftops output.
Xpdf also has a 'pdftoppm' converter. With that, I was able to create a multi-page .pdf file that was not garbled, and did still contain anti-aliasing. Unfortunately, since the PPM format is a raster image format, the PDF file that is created in this way does not scale very well, resulting in artifacts and, again, an unreadable PDF file on the hanlin device.
So it appears that for now I'll be stuck with storing multiple files on my device. Sigh. I wish that wouldn't be necessary...
Update: So I needed pdftk.
A common way to debug a network server is to use 'telnet' or 'nc' to connect to the server and issue some commands in the protocol to verify whether everything is working correctly. That obviously only works for ASCII protocols (as opposed to binary protocols), and it obviously also only works if you're not using any encryption.
But that doesn't mean you can't test an encrypted protocol in a similar way, thanks to openssl's s_client:
wouter@country:~$ openssl s_client -host samba.grep.be -port 443
CONNECTED(00000003)
depth=0 /C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=svn.grep.be/emailAddress=wouter@grep.be
verify error:num=18:self signed certificate
verify return:1
depth=0 /C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=svn.grep.be/emailAddress=wouter@grep.be
verify return:1
---
Certificate chain
0 s:/C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=svn.grep.be/emailAddress=wouter@grep.be
i:/C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=svn.grep.be/emailAddress=wouter@grep.be
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDXDCCAsWgAwIBAgIJAITRhiXp+37JMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNV
BAYTAkJFMRAwDgYDVQQIEwdBbnR3ZXJwMREwDwYDVQQHEwhNZWNoZWxlbjEUMBIG
A1UEChMLTml4U3lzIEJWQkExFDASBgNVBAMTC3N2bi5ncmVwLmJlMR0wGwYJKoZI
hvcNAQkBFg53b3V0ZXJAZ3JlcC5iZTAeFw0wNTA1MjEwOTMwMDFaFw0xNTA1MTkw
OTMwMDFaMH0xCzAJBgNVBAYTAkJFMRAwDgYDVQQIEwdBbnR3ZXJwMREwDwYDVQQH
EwhNZWNoZWxlbjEUMBIGA1UEChMLTml4U3lzIEJWQkExFDASBgNVBAMTC3N2bi5n
cmVwLmJlMR0wGwYJKoZIhvcNAQkBFg53b3V0ZXJAZ3JlcC5iZTCBnzANBgkqhkiG
9w0BAQEFAAOBjQAwgYkCgYEAsGTECq0VXyw09Zcg/OBijP1LALMh9InyU0Ebe2HH
NEQ605mfyjAENG8rKxrjOQyZzD25K5Oh56/F+clMNtKAfs6OuA2NygD1/y4w7Gcq
1kXhsM1MOIOBdtXAFi9s9i5ZATAgmDRIzuKZ6c2YJxJfyVbU+Pthr6L1SFftEdfb
L7MCAwEAAaOB4zCB4DAdBgNVHQ4EFgQUtUK7aapBDaCoSFRWTf1wRauCmdowgbAG
A1UdIwSBqDCBpYAUtUK7aapBDaCoSFRWTf1wRauCmdqhgYGkfzB9MQswCQYDVQQG
EwJCRTEQMA4GA1UECBMHQW50d2VycDERMA8GA1UEBxMITWVjaGVsZW4xFDASBgNV
BAoTC05peFN5cyBCVkJBMRQwEgYDVQQDEwtzdm4uZ3JlcC5iZTEdMBsGCSqGSIb3
DQEJARYOd291dGVyQGdyZXAuYmWCCQCE0YYl6ft+yTAMBgNVHRMEBTADAQH/MA0G
CSqGSIb3DQEBBQUAA4GBADGkLc+CWWbfpBpY2+Pmknsz01CK8P5qCX3XBt4OtZLZ
NYKdrqleYq7r7H8PHJbTTiGOv9L56B84QPGwAzGxw/GzblrqR67iIo8e5reGbvXl
s1TFqKyvoXy9LJoGecMwjznAEulw9cYcFz+VuV5xnYPyJMLWk4Bo9WCVKGuAqVdw
-----END CERTIFICATE-----
subject=/C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=svn.grep.be/emailAddress=wouter@grep.be
issuer=/C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=svn.grep.be/emailAddress=wouter@grep.be
---
No client certificate CA names sent
---
SSL handshake has read 1428 bytes and written 316 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : DHE-RSA-AES256-SHA
Session-ID: 65E69139622D06B9D284AEDFBFC1969FE14E826FAD01FB45E51F1020B4CEA42C
Session-ID-ctx:
Master-Key: 606553D558AF15491FEF6FD1A523E16D2E40A8A005A358DF9A756A21FC05DFAF2C9985ABE109DCD29DD5D77BE6BC5C4F
Key-Arg : None
Start Time: 1222001082
Timeout : 300 (sec)
Verify return code: 18 (self signed certificate)
---
HEAD / HTTP/1.1
Host: svn.grep.be
User-Agent: openssl s_client
Connection: close
HTTP/1.1 404 Not Found
Date: Sun, 21 Sep 2008 12:44:55 GMT
Server: Apache/2.2.3 (Debian) mod_auth_kerb/5.3 DAV/2 SVN/1.4.2 PHP/5.2.0-8+etch11 mod_ssl/2.2.3 OpenSSL/0.9.8c
Connection: close
Content-Type: text/html; charset=iso-8859-1
closed
wouter@country:~$
As you can see, we connect to an HTTPS server, get to see what the server's certificate looks like, issue some commands, and the server responds properly. It also works for (some) protocols who work in a STARTTLS kind of way:
wouter@country:~$ openssl s_client -host samba.grep.be -port 587 -starttls smtp
CONNECTED(00000003)
depth=0 /C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=samba.grep.be
verify error:num=18:self signed certificate
verify return:1
depth=0 /C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=samba.grep.be
verify return:1
---
Certificate chain
0 s:/C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=samba.grep.be
i:/C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=samba.grep.be
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDBDCCAm2gAwIBAgIJAK53w+1YhWocMA0GCSqGSIb3DQEBBQUAMGAxCzAJBgNV
BAYTAkJFMRAwDgYDVQQIEwdBbnR3ZXJwMREwDwYDVQQHEwhNZWNoZWxlbjEUMBIG
A1UEChMLTml4U3lzIEJWQkExFjAUBgNVBAMTDXNhbWJhLmdyZXAuYmUwHhcNMDgw
OTIwMTYyMjI3WhcNMDkwOTIwMTYyMjI3WjBgMQswCQYDVQQGEwJCRTEQMA4GA1UE
CBMHQW50d2VycDERMA8GA1UEBxMITWVjaGVsZW4xFDASBgNVBAoTC05peFN5cyBC
VkJBMRYwFAYDVQQDEw1zYW1iYS5ncmVwLmJlMIGfMA0GCSqGSIb3DQEBAQUAA4GN
ADCBiQKBgQCee+Ibci3atTgoJqUU7cK13oD/E1IV2lKcvdviJBtr4rd1aRWfxcvD
PS00jRXGJ9AAM+EO2iuZv0Z5NFQkcF3Yia0yj6hvjQvlev1OWxaWuvWhRRLV/013
JL8cIrKYrlHqgHow60cgUt7kfSxq9kjkMTWLsGdqlE+Q7eelMN94tQIDAQABo4HF
MIHCMB0GA1UdDgQWBBT9N54b/zoiUNl2GnWYbDf6YeixgTCBkgYDVR0jBIGKMIGH
gBT9N54b/zoiUNl2GnWYbDf6YeixgaFkpGIwYDELMAkGA1UEBhMCQkUxEDAOBgNV
BAgTB0FudHdlcnAxETAPBgNVBAcTCE1lY2hlbGVuMRQwEgYDVQQKEwtOaXhTeXMg
QlZCQTEWMBQGA1UEAxMNc2FtYmEuZ3JlcC5iZYIJAK53w+1YhWocMAwGA1UdEwQF
MAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAAnMdbAgLRJ3xWOBlqNjLDzGWAEzOJUHo
5R9ljMFPwt1WdjRy7L96ETdc0AquQsW31AJsDJDf+Ls4zka+++DrVWk4kCOC0FOO
40ar0WUfdOtuusdIFLDfHJgbzp0mBu125VBZ651Db99IX+0BuJLdtb8fz2LOOe8b
eN7obSZTguM=
-----END CERTIFICATE-----
subject=/C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=samba.grep.be
issuer=/C=BE/ST=Antwerp/L=Mechelen/O=NixSys BVBA/CN=samba.grep.be
---
No client certificate CA names sent
---
SSL handshake has read 1707 bytes and written 351 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : DHE-RSA-AES256-SHA
Session-ID: 6D28368494A3879054143C7C6B926C9BDCDBA20F1E099BF4BA7E76FCF357FD55
Session-ID-ctx:
Master-Key: B246EA50357EAA6C335B50B67AE8CE41635EBCA6EFF7EFCE082225C4EFF5CFBB2E50C07D8320E0EFCBFABDCDF8A9A851
Key-Arg : None
Start Time: 1222000892
Timeout : 300 (sec)
Verify return code: 18 (self signed certificate)
---
250 HELP
quit
221 samba.grep.be closing connection
closed
wouter@country:~$
OpenSSL here connects to the server, issues a proper EHLO command, does STARTTLS, and then gives me the same data as it did for the HTTPS connection.
Isn't that nice.
Bruno
De Wolf blogs about programming languages. His statement: "all
programming languages suck". Not sure whether I agree. But's let look at
his arguments. In his words, a good programming language, among others,
is readable by a guy who didn't write the original program and who
doesn't know all the language details or libraries
.
Personally, I don't agree. Code readability, above all, depends on the programmer. Programming languages that impose structure can probably help, but it's possible to create unreadable code in any language—that doesn't need goto.
Since I went to the 'karel de grote' institute of higher education, I learned quite a number of programming languages. Hairy code is possible with all of them. Unreadable code is easy with some of them, and some of those I do consider good programming languages. I guess my criteria are different:
That's it, I guess. In my book, a programming language is a tool to get a job done. A good tool is flexible, not too unwieldly, and somewhat intuitive. Like a hammer. If you want to build a table, you probably need a hammer, among other things. The better the hammer, the more likely it is that you'll get a nice table. But even the best hammer won't get you a wonderfully crafted table if your carpenter sucks at his craft.
And so it is with programming languages: it might help if you use a proper language, but any good programmer can write readable code in any programming language.
Valgrind rocks my pants off.
That is all.
Julien Blache has been writing something called "pomme", to do some stuff with backlights for apple laptops--which explaines the name of the thing, since Julien is French.
One thing I've been wondering about, however, is why he chose to write something entirely new rather than submitting patches for pbbuttons, which does exactly the same.
His blog doesn't allow for comments currently (boo!), so I made this a blog post of my own. So, Julien, please convince me: what does pomme do that pbbuttons doesn't?
For a project at work, I needed to write some application that would talk to sockets and store some state.
use MLDBM, so I was planning to use that.
So, I was facing a dilemma here: either use the cumbersome C API to libdb, or try to figure out a way to be able to talk to many sockets all at once from perl while still being able to use buffered PerlIO.
I chickened out.
wouter@country:~$ perldoc perlembed
Fast forward half a day, and I now have an application that will use perl to talk to libdb, and that will use C to talk to a number of sockets.
I feel dirty now. But it seems to work...
About a year and a half ago (has it been that long already?), I blogged about wanting to write some gobjects. People did give me pointers to some documentation then, but for various reasons I never looked into it in detail afterwards.
Since yesterday, though, that's changed. For the last two days, I've been writing on something which I'll eventually release to the world as libgjs:
wouter@country:~/code/c/libgjs/src$ ./gjstest There are 1 joysticks on this system Joystick 0 is a Logitech Inc. WingMan Gamepad Extreme, with 10 buttons and 4 axes. wouter@country:~/code/c/libgjs/src$
Wonderful, isn't it? Obviously the above doesn't require the 306 LOC that I've written now (which isn't much, but that's mostly because I had to read a lot of documentation in between); but when finished, the API of that library will be approximately something like this:
if(gjs_joystick_get_max>=0) {
GjsJoystick *joystick = gjs_joystick_new(0);
if(joystick->buttons >= 1) {
g_signal_connect(joystick, "button-pressed",
start_firing, NULL);
g_signal_connect(joystick, "button-released",
stop_firing, NULL);
g_signal_connect(joystick, "axis-moved",
move_crosshair, NULL);
}
gjs_joystick_set_interval(1);
}
where start_firing, stop_firing and move_crosshair are, obviously, callback functions (they get the button or axis number as a parameter). The set_interval function sets the time between two polls of the kernel device file; this will use the glib mainloop, but it'll also be possible to use a different event loop with gjs_joystick_poll and/or gjs_joystick_poll_all.
As of now, it's almost finished; only the signals seem to be something I'm having issues with still. But that's only a matter of not having used gdb enough—I'm sure I'll get there.
What gave me the sudden idea to write this library? Something silly, really: for a project I'm currently negotiating, it might be nice if I knew how GObject works; and what better way to find that out than by writing something fun for it? Exactly.
Update: Bugs squashed, and the library (along with a small test/demo application) is available. Please try it out and let me know what you think.
Via Bruce Schneier:
Michael Sutton apparently did a simple test wherein he found that more than 10% of the sites in his sample were vulnerable to SQL injection.
Interesting.
In 2003, I needed to write a PHP site that would need to be internationalized, and decided to use gettext() for this purpose. While PHP has proper support for gettext, it unfortunately does not have any builtin way to easily convert the data in the Accept-Language and Accept-Charset HTTP headers into something that can be understood by gettext.
So I sat down, and over the course of an afternoon, I wrote 83 lines of PHP code (excluding comments) that would do just that: parse the above two headers, and return a gettext string which represents the best match out of a set of strings specified. It's pretty easy in use, and was written quite generically, so I put a GPL tag on top of it, put it online, added a comment to the documentation of the gettext stuff in the PHP manual advertising the URL, and stopped thinking of it. The project that I wrote this file originally for eventually wasn't even finished; today, I don't even recall what it was.
In early 2005, I received a mail from Matthew Palmer thanking me for the code, and telling me that he'd incorporated it in IRM of which he's part of the development team. Which reminded me of its existance. For a short while.
Today, I was cleaning out my INBOX, and stumbled upon this old email again. Which got me curious, so I started googling for those 83 lines of code that I'd entrusted to the Free Software world three years ago. I must say I'm pleasantly surprised. My file accept-to-gettext.inc seems to be used in a variety of free software projects, including of course IRM, but also the GnuCash website, or in this french educational thing called SLIS. It seems to be quoted in its entirety on some turkish PHP-related forum. It has been adapted to support different translation systems. And so on... the list is rather long.
Can't say I'm not proud. And yet, the code contains a rather silly bug: the assumption that character set encodings are somehow linked to translations. They are not; gettext is perfectly capable of transparently transcribing one character set to another.
Oh well...
#!/bin/bash modprobe cpufreq_userspace echo userspace > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor while true do if expr $(uptime|cut -d',' -f3|cut -d':' -f2) '>' 0.75 > /dev/null then echo 1333333 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed else if expr $(uptime | cut -d',' -f4) '<' 0.75 > /dev/null then echo 666666 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed fi fi sleep 10 done
I tried using some of the standard cpufreq stuff that's in the archive, but none of them seem to work for me; they all make my laptop work at full speed, all the time. Which is silly.
Also, the kernel ondemand and conservative governors don't seem to work—when I write "ondemand" to /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor, it comes back with write error: invalid argument, or some such.
So, I hacked up the above script. It sets the speed at full (1.3Ghz powerpc here) when the load rises above 0.75 for the last minute, and sets it at the low speed again when the load drops below 0.75 for both one and five minutes.
Using the system load average as a metric to decide what speed you run your processor at isn't terribly state of the art, but it Works For Me(tm); my processor isn't put to full speed when some short cpu-intensive job comes up for three seconds, but if something decides to use it for a bit longer, it does turn it up after approximately 30 seconds. Once it reaches that, it stays there until (on average) a few minutes after the cpu-intensive job has finished.
I like it that way.
Update: I received no less than four replies to this post that I need to make sure that cpufreq_ondemand is loaded, which I can check by reading /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors. Yes, I know. I did that. It's there. Yet, it still doesn't work, not even with echo -n.
Which is why I'm so confused. It's probably just a kernel bug, though.
I don't believe it. It finally happened: SourceForge finally offers the ability to get subversion repositories.
The NBD tools are in the queue for migration right now... after I did some commits to the CVS repo. Oh well.
Dear lazyweb, please tell me:
wouter@country:~/debian/eID/belpic-2.3.13.full$ LC_ALL=C make Making all in winscarp make[1]: Entering directory `/home/wouter/debian/eID/belpic-2.3.13.full/winscarp' [...] Making all in belpcscd make[1]: Entering directory `/home/wouter/debian/eID/belpic-2.3.13.full/belpcscd' if g++ -DPACKAGE_NAME=\"belpcscd\" -DPACKAGE_TARNAME=\"belpcscd\" \ -DPACKAGE_VERSION=\"2.3.13\" -DPACKAGE_STRING=\"belpcscd\ 2.3.13\" \ -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"belpcscd\" -DVERSION=\"2.3.13\" \ -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 \ -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 \ -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_FCNTL_H=1 \ -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_SYS_TIMEB_H=1 -DHAVE__BOOL=1 \ -DHAVE_STDBOOL_H=1 -DHAVE_UNISTD_H=1 -DHAVE_FORK=1 -DHAVE_VFORK=1 \ -DHAVE_WORKING_VFORK=1 -DHAVE_WORKING_FORK=1 -DHAVE_STDLIB_H=1 \ -DHAVE_MALLOC=1 -DHAVE_STDLIB_H=1 -DHAVE_REALLOC=1 -DHAVE_SYS_SELECT_H=1 \ -DHAVE_SYS_SOCKET_H=1 -DSELECT_TYPE_ARG1=int \ -DSELECT_TYPE_ARG234=\(fd_set\ \*\) -DSELECT_TYPE_ARG5=\(struct\ \ timeval\ \*\) -DRETSIGTYPE=void -DHAVE_FTIME=1 -DHAVE_INET_NTOA=1 \ -DHAVE_MEMSET=1 -DHAVE_SELECT=1 -DHAVE_STRCASECMP=1 -DHAVE_STRCHR=1 \ -DHAVE_STRSTR=1 -I. -I. -Wall -Dlinux -I. \ -I../ObjectsMultiPlatform/Include -I/usr/include/PCSC `wx-config \ --cppflags` -g -O2 -MT belpcscd-wxmainapp.o -MD -MP -MF \ ".deps/belpcscd-wxmainapp.Tpo" -c -o belpcscd-wxmainapp.o `test -f \ 'wxmainapp.cpp' || echo './'`wxmainapp.cpp; \ then mv -f ".deps/belpcscd-wxmainapp.Tpo" ".deps/belpcscd-wxmainapp.Po"; else rm -f ".deps/belpcscd-wxmainapp.Tpo"; exit 1; fi wxmainapp.h:88: warning: 'class MyApp' has virtual functions but non-virtual destructor wxmainapp.cpp: In function 'int main(int, char**)': wxmainapp.cpp:239: error: 'wxInitializer' was not declared in this scope wxmainapp.cpp:239: error: expected `;' before 'initializer' wxmainapp.cpp:240: error: 'initializer' was not declared in this scope make[1]: *** [belpcscd-wxmainapp.o] Error 1 make[1]: Leaving directory `/home/wouter/debian/eID/belpic-2.3.13.full/belpcscd' make: *** [all-recursive] Error 1 wouter@country:~/debian/eID/belpic-2.3.13.full$
(line feeds added for readability)
The offending bit of wxmainapp.cpp looks like:
int main(int argc, char **argv)
{
wxApp::CheckBuildOptions(wxBuildOptions());
wxInitializer initializer;
if ( !initializer )
{
fprintf(stderr, "Failed to initialize the wxWindows library, aborting.");
return -1;
}
MyApp *pTheApp= new MyApp();
pTheApp->OnInit();
while(1) sleep(1); /* run */
}
which doesn't look incorrect to me, but I could be wrong. class wxInitializer is defined in <wx/app.h>, which (according to gcc -E output) seems to be included (even if only indirectly), and its use is to initialize and cleanup the library automatically, so it would seem that upstream used it properly in the above bit of code.
People with more C++/wx knowledge than me would make me very happy if they'd point out what's wrong. Even if they first need more info to do so.
Thanks.
Update: Seems I need wxbase-2.4-config, rather than wx-config. As the upstream makefile (which I'm ignoring for the time being) showed me. Really need to learn to trust other people's judgement some time...
Daniel Silverstone asks about ternary operators, and whether people like to use them.
Well, I know I do. In fact, it's the single operator I like to use a lot. Too much, in fact -- sometimes I abuse it in horrible ways, such as nesting the thing three times, or so. Which is stuff that gets removed after a first reading, of course.
Most of the Microsoft languages have IIF, a function that does basically the same thing (as in "iif(condition, trueval, falseval)"). I find that a bit more intuitive than the ?: construct, but the difference isn't that important since it's semantically the same thing (and, more importantly, the ?: operator requires less characters, so is nicer).
I'm not sure about most other languages. I have learned a lot of programming languages at school, but it's been way too long, and I don't use most of them anymore. I'm quite positive that COBOL does not support it, so you need to type it out in a full 'if' structure. But that's okay, because writing COBOL feels like writing a novel anyway.