2008-01-04

User configuration

Posted in Computers, PlanetDebian at 21:47 UTC (+0000) by sven

Anthony Towns wrote about user configuration (i.e. ~/.foo) and the XDG spec/proposal versus his own (and according to him) simpler version. The main goal of all this is to get rid of all the ~/.foo directories and files in a users home directory. Apart from technically minor differences, I don’t see why AJs proposal would be simpler than the XDG approach. All it does is hardcoding the values which are adjustable in XDG via XDG_*_DIRS.

There is one notable difference though: AJ proposes fallback values which are compatible with existing application paths (i.e. HOME_ETC/XDG_CONFIG_HOME should default to ~/ – resulting in ~/.foo files/dirs). This is a good proposal in a way: If you patch an existing application to strictly follow the proposal made by AJ, it still finds its old configuration (and other files) if the new environment variables are not set. No need to move anything around. However, I think the XDG approach is better since it makes sure that compliant applications don’t clutter ~/ anymore. They do need some mechanism to move existing configurations and data around though if the XDG-compliant stuff isn’t already there, but an old style config exists.

All in all, I think it is worth following the XDG spec, even if it is slightly more complex than AJs proposal. For one, it already exists for a while and I think that some applications already started following that spec. It also makes slightly more sense to me to have the system fully configurable as to where the apps store data as opposed to the hardcoded fallback/default values AJ proposes (even if we would change AJs proposal to use “$HOME-cleaning” default values).

2007-10-17

CPU feature flags and their meanings

Posted in Computers, Personal, PlanetDebian at 15:03 UTC (+0000) by sven

Since I never really found a nice overview of which CPU flags (see /proc/cpuinfo) mean what, so I gathered some information using the web, with the most notable sources being the BOINC FAQ entry on CPU Register Acronyms at [1] and the output of the nice little (though seemingly mostly unmaintained) cpuid utility. See my results at [2]. Any suggestions for enhancements and completions are highly welcome, just leave a comment to this post.

[1]: href=”http://boincfaq.mundayweb.com/index.php?language=1&view=176
[2]: http://blog.incase.de/index.php/cpu-feature-flags-and-their-meanings/

2007-09-18

Brane Dump — The Thoughts of Matt Palmer

Posted in Computers, PlanetDebian at 14:20 UTC (+0000) by sven

In “Documentation – the chicken and the egg” [1], Matt Palmer wrote about the problem that noone writes documentation because noone reads it and that noone reads documentation because the few documentation in existence usually isn’t very good. So he, like me hs the habit of searching for help on the net instead of in a projects documentation.

However I disagree with him in the consequences a bit. Because I noticed that on several projects with good documentation (subversion is an example that immediately comes to my mind, but postfix isn’t too bad either), the internet search returns a reference to the docs more often than not. Of course, this means that the documentation needs to be searchable for those webcrawlers. So if you have the task to write documentation, I strongly suggest to make it searchable in some way. For company-internal projects, this obviously means that the documentation must be reachable via an internal search engine. In a project a few years back, we implemented an internal search engine which included company internal information (even with user based access rules so that each user only got those results he could actually access) as well as an external search engines results. It was closed source, but a pretty nice idea. It didn’t, however, index any locally (user’s desktop) stored documents, only what was on some company web page (but including .doc, .rtf, .pdf and the like which where retrievable via http).

[1] http://www.hezmatt.org/~mpalmer/blog/general/documentation_the_chicken_and_the_egg.html

2007-08-13

Migrating aptitudes knowledge about auto-installed packages

Posted in Computers, PlanetDebian at 14:09 UTC (+0000) by blog

Jonathan McDowell wonders how to take aptitude’s knowledge about auto-installed packages from one computer to the other. Well, I looked into the issue for a few minutes and I found a solution though it doesn’t look nice.

for i in
    # grep installed package names
    `COLUMNS=200 dpkg -l | grep -E '^ii'| awk '{print $2};' `
do
    # find package in /var/lib/aptitude/pkgstates
    # check for the right state (1 seems to mean
    # auto-installed, 3 manually installed)
    if
        grep-available -s Package,State \
                       -F Package \
                       -X $i /var/lib/aptitude/pkgstates  \
        | grep -q 'State: 1'
    then
            echo $i
    fi
done

Results in a list of auto-installed packages AFAICT. If you change the “State: 1″ into “State: 3″, it seems you get only manually installed packages. So if you take the latter and feed it to “aptitude install”, your database should be right. If you take the former, you feed the list to “aptitude markauto”. A cleaner solution (which works even if /var/lib/aptitude/pkgstates changes formats) would involve checking “aptitude show” output, evaluating the “Automatically installed:” field. However, simply feeding “aptitude show” output to grep-dctrl (or any equivalent) resultes in the rather irritating error message “grep-dctrl: -:14: expected a colon.”, with the 14 changing to a different line in the packages description (usually a line directly following an empty line). So I’m just giving you an idea here.

See also No related posts
Permalink 7 Comments

2007-07-26

Re: voting on sponsorship

Posted in Computers, PlanetDebian at 18:56 UTC (+0000) by sven

Joey, I see a huge difference between voting on the current DM proposal and your virtual (sorry, I can’t remember a better word for it) voting on sponsoring uploads. Debian Developers were always allowed to upload any package they saw as fitting into Debian (though there were no guarantees for these packages to actually being included). Also, NMUs are OK if the maintainer agreed. So all they (seemingly) did was putting a different maintainer address into the control file. However, at least in theory, the sponsors were still responsible for their upload if it turned out fishy.

The DM proposal (don’t get me wrong there, I generally support the idea, I just don’t like the way it is implemented in the proposal) is shifting this responsibility from the sponsor onto the shoulders of the DM, which is a good thing. However I think it doubles some of the work the NM queue already needs done, so my proposal would be not to have it as a completely seperate path for a contributor, but instead see it integrated into the NM process. Wether it would mean adding a new maintainer to the DM keyring after he had an advocate (which is pretty near to the current proposal) or after an AM was assigned and checked some basic things (which I would prefer) doesn’t make a lot of difference to me.

The DM GR

Posted in Computers, Personal, PlanetDebian at 16:03 UTC (+0000) by sven

Following up to Myon’s post, I also wanted to blog my honest opinion about the GR. Given my current understanding of the GR, I hope to see it fail (and will send my vote shortly after this blog entry I think). The reason is not that I think it shouldn’t be made easier to contribute to Debian (on the contrary). But the GR micro-manages too much and on the other hand doesn’t make the current NM process any easier for DM’s than for any non-DM applicant to the NM process. Also, like some others said before, I seriously doubt that a big part of the current DD-applicants (i.e. those in the NM queue at some level) would prefer DM status over DD status. And the same is true (IMHO) for other people wanting to contribute to Debian. This is a simple thing: If I contribute to something regularly in my spare time (like most contributors in the FL/OSS world do), I also would like to have some influence over its directions. And a DM is missing this influence even more than a DD is.

So what should be done instead in my opinion?
I think that a DM like status is fine as part of a (possibly re-designed) NM process. Given the current NM process, I would like people been given a DM like status after they finished the T&S (tasks^IIRC and skills) part if their AM found their skills and knowledge of the policy rules sufficient to give them that status. If that is all they wanted, they can drop from the NM process at that point and stay DMs. If they want to become DDs later on, they can resume the NM process at that very step (or continue right ahead).

This would do several things:

introduce the DM status as an alternative to full DD status
integrate the DM status closer with the existing NM process
take some load of the friendly sponsors who upload non-DD packages now, giving them more time to help more new contributors
give people waiting for DD status some intermediate level at which they already can do more than before their joining of the NM queue

So I really think “more discussion” is the only valid option for me in this GR. If you want to change my mind: Please leave some feedback in my blog. Or give a precise (but brief) description why the current proposal is better than integration with the current NM process in one way or the other.

Abbreviations:

DD = Debian Developer (a full Debian project member, includes voting rights)
DM = Debian Maintainer (planned to be a somewhat restricted DD, among others without voting rights)
NM = New Maintainer (should actually be: New Developer)
GR = General Resolution (a way in Debian to create new rules or to ask the project leader or others in the project to do or refrain from doing something)

Update:
The GR even seems to contradict itself a bit. If the upload rules are applied in an AND fashion, one of these rules is superfluous (the first one), at least if I’m not mistaken:

the Maintainer: field of the uploaded .changes file corresponds with the owner of the key used (ie, non-developer maintainers may not sponsor uploads)
the most recent version of the package uploaded to unstable or experimental includes the field “DM-Upload-Allowed: yes” in the source section of its control file
the most recent version of the package uploaded to unstable or experimental lists the uploader in the Maintainer: or Uploaders: fields (ie, non-developer maintainers cannot NMU or hijack packages)

The first rules also means that no DM could ever do an upload for packages team-maintained in the fashion cyrus-imapd-2.2 is maintained (Maintainer address is the teams mailinglist address, all uploading team members are in the Uploaders field). This is IMHO a serious flaw in the proposal. If the first rule was dumped, all would be well for team maintainers.

2007-07-02

Improving graphs

Posted in Computers, Personal, PlanetDebian at 13:02 UTC (+0000) by sven

In Improving simple charts, Dirk Eddelbuettel wrote a nice little summary how to produce “nicer” charts using R. However, in my opinion, the charts he provided as alternate options to the original, purely linear graph are less readable for the average person. If anything, using a logarithmic scale on the original Y axis (number of packages not built after a specific date) would be of use, but the other graphs only hide the original point of the post Dirk replied to (How old are our (Debian’s) packages?). The point was that 94% of all packages had been rebuilt since the release of Sarge or, if you want to put it the other way around, 1264 (6%) were built before Sarge was released. This relationship is hidden from immediate recognition by the reader of the graphs. I admit though, that Dirk’s second graph (Number of packages against age in days with the age being on y logarithmic scale) is also insightful. The other two graphs he presents might make sense in a scientific publication if some point in the “more detailed” areas of the graph needs proving, but for this specific problem, they really don’t make sense to me. Actually, I would question their use even in scientific publications, since they distort the original data a lot. Scientists are pretty much used to logarithmic scales these days, but I couldn’t work out how the scales work on those two graphs. And another note on the second graph (logarithmic age): The way the X axis is labeled (10, 20, 50, 100,..) is also pretty non-intuitive to me. I’m much more used to something like 5, 10, 20, 40, 80, 160 and so on, which gives the same distance between each label and seems a lot more usable to me..

(edit: Fix spelling of Dirk’s surname)

2007-04-17

init script generators

Posted in Computers, PlanetDebian at 20:34 UTC (+0000) by sven

Gunnar is following up on the init script blog posts by Erich and me. First a note to Gunnar: The idea of the init scrip snippets isn’t exactly my own idea, but rather taken from a comment to one of Erich’s posts (IIRC), I don’t know the original author anymore.

Anyway, I really think that some init script generator should work. It should even be possible to fetch variables from /etc/default/<daemon-name> if needed. I’m just doing a braindump of what I have in mind for the start and stop of a daemon. I would expect a configuration file (currently I think one for each operation might be best, but INI-lookalike files might also be OK). So for the start operation, I would expect these options:

simple daemon, uses a default config with a hardcoded path and needs no options or only static options passed:
```
[SIMPLE]
/path/to/daemon
option 1
option 2
option 3
```
Where I chose to name one option per line so whitespace and controll characters can be part of the options without the need to quote anything.
a bit more complicated, needs a few variables from /etc/default/daemon:
```
[TEMPLATED]
/path/to/daemon
option 1
%PARAMETER_FROM_ETC_DEFAULT%
--parameter=%ANOTHER_ONE_FROM_ETC_DEFAULT%
```
Where the script generator would have to make sure that the parameters from /etc/default are filled in at runtime. I would expect POSIX-Shell-Scripts in /etc/default for this.
Really complex: A custom script snippet is needed which has to do some specificc tasks, like cleaning possible stable locks which had been left behind.
```
[SCRIPT]
rm /var/lock/daemon.lck
/path/to/daemon --background --and --other --options
```
Where the script generator would simply take this script and use it where it would otherwise generate its own code.

As for stopping, I generally see only three options for the sysv-like background processes: Either a script is needed (see above for how that would look like) or a process needs to be killed according to a PID-file or the process name (with the later being rather bad). So this would result in:

[SCRIPT] like for starting the daemon
killing by PID-file:
```
[PIDFILE]
/path/to/pid-file
```
killing by process name/executable:
```
[PROCESS]
binary
```
or
```
[PROCESS]
/path/to/binary
```
where the generator would either only look at the name of the process (first version, no full pathname) or for a process with the given executable (from /proc/ /exec, I would expect).

Alright, this would leave the cases of running (instead of backgrounding) the server and stopping such a running server (the later should be fairly simple unless the stopping of the server requires additional actions other than simply killing it) as well as the reload/restart/status stuff. But these should also be relatively simple actually.

What would be interesting for me:
Does any init system require information that is not part of the following list? I would plan with supplying information how to:

start server in background
start server in foreground (without the need for a terminal)
determine wether a server runs (status)
determine wether a server is ready to serve requests (also part of status)
determine wether the server is installed (and able to be started)
stop a background server
stop a foreground server (unless specified, a simple SIGKILL to the process is assumed)
restart foreground server (unless specified, a simple stop-start cycle is assumed)
restart background server (unless specified, a simple stop-start cycle is assumed)
reload config (unless specified, it is assumed that reloading doesn’t work and a restart is needed)
reopen logs (unless specified, the “reload config” operation is assumed to do this)
some meta information: short description, needs writeable filesystem(s) mounted, needs a certain daemon to be running already (or even: able to serve requests), enhances other daemon X and should be started before it (don’t care if X isn’t installed/configured), is enhanced by daemon Y and should be started after it (don’t care if Y isn’t installed/configured), replaces daemon Z, should only run once at startup, …

Finally, even with all that information available inside a Debian package, this would still leave one question unanswered: Who triggers init script generation? If a daemon package does so in postinst: How do I switch to a different init system? If the init system does it in postinst: What happens to additional daemons installed later on? If they get regenerated during system start: How much penalty does that cause? Currently I think the route to go is to provide a command to daemon packages’ postinst that (re)generates only the needed scripts for that package and have a command used in an init system’s postinst which regenerates all of them. That should do. Regenerating during boot would most likely take too much time and has other problems, too, like the availability of a writeable filesystem to store the generated scripts.
Alright, if someone would like to see this post (or rather parts of it) on the Wiki page about an Init HackFest, please transfer it there yourself. I’m currently not able to log in, for whatever reason.

2007-04-13

Init HackFest / New init systems in Debian

Posted in Computers, PlanetDebian at 13:53 UTC (+0000) by sven

In his “Init followup” post, Erich posted a reference to his Debian-Wiki post about a proposed Debian HackFest after Etch was released (which would be about 5 days ago<smile />). Anyway, since that page at the wiki is protected, which means I can’t edit it, I’m posting my thoughts here.

I agree with Erich and some other commentors at Gunnar Wolf’s blog post that it doesn’t make sense to add X init-script-only-packages for any daemon to support X init systems. Instead, each package containing one or more daemons should contain the meta information needed to create all init scripts. Basically, this would probably include script snippets which do the following operations:

Pre-check: Check wether the package actually is installed rather than in removed/configured status.
start: Start the daemon in background mode.
run: Start a daemon in foreground mode (so that an exiting daemon can immediately be recognized by the init system).
stop: Stop a daemon running in background mode.
stop-running: Stop a daemon running in foreground mode. This should only be needed if the daemon doesn’t shut down nicely on a SIGTERM but instead needs some sort of message/semaphore to do so.
status: Return current running status.
reload: Reload configs, reopen logs (if supported by daemon).

Additionally, some control information would be needed including, but probably not limited to:

Dependencies: Should allow for alternatives, lists daemons which need to be running before this daemon is started.
Provides: Similar to package-provides, this should be used to allow things like replacing inetd with xinetd or something else providing similar functionality.
Short Description: For init systems that are capable of showing a short description next to the name of configured daemons
Run-Sets: Not strictly like sysvinit-runlevels, but similar, should allow the admin/user to configure different sets of daemons to start, selecting at boot by a kernel cmdline-parameter or at runtime through some command (“init 1″) which set should be running.

The script snippets listed above should allow basically any operation on a daemon, including the force-reload and try-restart commands not yet implemented by all current sysv-initscripts.

See also init script generators
See also live-cd-on-removable-disk
See also About
Permalink 4 Comments

2007-03-13

Wishlist for lenny… or why debian packaging is considered hard

Posted in Computers, PlanetDebian at 14:45 UTC (+0000) by sven

Eddy, while I agree that ebuilds look a lot cleaner, shorter or easier than typical debian/rules + debian/control, you should consider a few things:

There is cdbs in Debian, which provides a lot of help in packaging “easy” packages. Note though that I don’t really like the hidden magic it does.
ebuilds don’t have to handle architecture independent data, since Gentoo doesn’t have such packages. If Debian wouldn’t have architecture independent data, debian/rules would look easier immediately.
Updating an easy package (i.e. one without Debian specific patches) to a new upstream usually is a no-brainer in Debian as much as in Gentoo.
With ebuild classes, you still have to know/learn which class matches your package best, so you still need to know what each class actually does. That isn’t too different to understanding dh_* stuff, if you ask me.

So all in all, I’m quite certain that the amount of learning needed to create a correct ebuild is lower than that needed to create a Debian source package, but I don’t think that packaging for Debian is hard either.

And finally, Erich is quite right that a central repository for all Debian packaging scripts would be a nice thing. However it would be quite hard to determine the right VCS to use for that. Most de-central VCSes I glanced over need manual intervention on the “server” side to include patches from local developer branches. And of course there is the problem of several packages having a non-clean upstream tarball they need to repack for each release as well as several packages having a tarball which includes more than one single upstream tarball. You would need to find a way to handle those as well.

Svoccblog

2008-01-04

User configuration

2007-10-17

CPU feature flags and their meanings

2007-09-18

Brane Dump — The Thoughts of Matt Palmer

2007-08-13

Migrating aptitudes knowledge about auto-installed packages

2007-07-26

Re: voting on sponsorship

The DM GR

2007-07-02

Improving graphs

2007-04-17

init script generators

2007-04-13

Init HackFest / New init systems in Debian

2007-03-13

Wishlist for lenny… or why debian packaging is considered hard

Pages

Categories

Recent Posts

Recent Comments

Commercial links

Debian related

Archives

Planet Debian

Meta