Continuous integration
Requirements:
Test Plash on old and latest distribution releases: from Ubuntu edgy to gutsy
Build Debian packages and make them available for download
-- package repositories are listed on AutoBuildPackages - Test that Plash builds against CVS HEAD of glibc
- Test that applications work with Plash
- Test on various architectures, beyond i386 and amd64
Joey Hess is running Debian under emulation on several architectures for testing; see his blog post and the debian-installer test lab
- qemubuilder is a variant of pbuilder which knows how to set up Debian/Ubuntu under QEMU for different architectures and communicate with the VM via a serial device
Cross-compile PlashGlibc to various architectures
- Other data that could be generated: code coverage, performance statistics
Run tests from other projects. See Thomas Leonard's blog posting
Build the upstream non-Plash glibc, so that we can check that problems with PlashGlibc do not occur upstream
Test against multiple kernel versions (perhaps including latest development kernel) using a VM. Things like the ELF chainloader (Story16) could be broken by kernel changes.
When I have a separate machine to run this on, it should also do:
- backups of the SVN repository
- backups of the Wiki
See BuildTools.
Tasks
Add logging to chroot_build script
Make the log format structurally the same as XML so that we can load it into elementtree and do xpath queries on it
Rearrange directory structure:
- ubuntu-gutsy
- chroot
- stamps
- logs
plash -> chroot/work/plash
- ubuntu-gutsy
Add error recovery so that we can run multiple targets in a batch OK
Write log formatter/summariser. Columns for targets. Time is a vertical axis, like buildbot.
Fix so that it can handle partially-written logs without "end_time" attributes
- Upload results to Web server:
- Upload build logs and summary
- Build logs could get quite big, though. Prune them down before uploading because there is limited space on the account.
- Upload built Debian packages
Make sure that version numbers for generated packages are suitable, and indicate a snapshot build. Prepend an entry to debian/changelog. What format should the version number be? Perhaps "<last-release>.YYYY-MM-DD.svn<revision>", e.g. "1.18.2007-01-02.svn678". Including the release number means that newer releases override old snapshots. The date is included besides the SVN revision number because builds can be affected by factors outside the code in SVN.
- Upload build logs and summary
Remove dependencies on the setup on my machine.
- Debian package repositories, including Debian/Ubuntu mirrors
- User name
- The chroot_build script should copy template SVN/Git working directories from inside the "master" directory
Is Buildbot suitable?
- Buildbot likes to have the build slave (buildbot code) running inside the environment we're building in. This assumes the environment has been set up by hand and so is not good for testing the environment-building code.
- Buildbot is not suitable for keeping logs long-term, because it doesn't store them efficiently, i.e. with delta compression. It doesn't record other build products. It has support for copying files from the build slave, but this is a separate feature from logging, and the destination files are overwritten rather than appending the data by writing the files to a fresh location.
