Continuous integration

Requirements:

Test Plash on old and latest distribution releases: from Ubuntu edgy to gutsy
Build Debian packages and make them available for download -- package repositories are listed on AutoBuildPackages
Test that Plash builds against CVS HEAD of glibc
Test that applications work with Plash
Test on various architectures, beyond i386 and amd64
- Joey Hess is running Debian under emulation on several architectures for testing; see his blog post and the debian-installer test lab
- qemubuilder is a variant of pbuilder which knows how to set up Debian/Ubuntu under QEMU for different architectures and communicate with the VM via a serial device
- Cross-compile PlashGlibc to various architectures
Other data that could be generated: code coverage, performance statistics
Run tests from other projects. See Thomas Leonard's blog posting
Build the upstream non-Plash glibc, so that we can check that problems with PlashGlibc do not occur upstream
Test against multiple kernel versions (perhaps including latest development kernel) using a VM. Things like the ELF chainloader (Story16) could be broken by kernel changes.

When I have a separate machine to run this on, it should also do:

Tasks

Add logging to chroot_build script
- Make the log format structurally the same as XML so that we can load it into elementtree and do xpath queries on it
Rearrange directory structure:
- ubuntu-gutsy
  - chroot
  - stamps
  - logs
  - plash -> chroot/work/plash
Add error recovery so that we can run multiple targets in a batch OK
Write log formatter/summariser. Columns for targets. Time is a vertical axis, like buildbot.
- Fix so that it can handle partially-written logs without "end_time" attributes
Upload results to Web server:
- Upload build logs and summary
  - Build logs could get quite big, though. Prune them down before uploading because there is limited space on the account.
- Upload built Debian packages
  - Make sure that version numbers for generated packages are suitable, and indicate a snapshot build. Prepend an entry to debian/changelog. What format should the version number be? Perhaps "<last-release>.YYYY-MM-DD.svn<revision>", e.g. "1.18.2007-01-02.svn678". Including the release number means that newer releases override old snapshots. The date is included besides the SVN revision number because builds can be affected by factors outside the code in SVN.
Remove dependencies on the setup on my machine.
- Debian package repositories, including Debian/Ubuntu mirrors
- User name
- The chroot_build script should copy template SVN/Git working directories from inside the "master" directory

Buildbot likes to have the build slave (buildbot code) running inside the environment we're building in. This assumes the environment has been set up by hand and so is not good for testing the environment-building code.
Buildbot is not suitable for keeping logs long-term, because it doesn't store them efficiently, i.e. with delta compression. It doesn't record other build products. It has support for copying files from the build slave, but this is a separate feature from logging, and the destination files are overwritten rather than appending the data by writing the files to a fresh location.