Integrate PlashGlibc with glibc's build process
Status: done
Currently, the build process for PlashGlibc runs glibc's build process but is separate from it. Having built glibc, it links together the *.os files that were generated as a by-product, omitting some object files and adding its own. This should be changed so that PlashGlibc is just a patch to glibc.
Benefits:
Make PlashGlibc closer to the real glibc and hence more reliable. It will eliminate a class of issues that arise when PlashGlibc's build scripts do not match glibc's build system.
Make it easier to build a libc6 deb package containing PlashGlibc. We can simply use the existing deb packaging scripts.
Make it easier to build PlashGlibc for architectures other than i386.
How to build
$ svn co svn://svn.gna.org/svn/plash/trunk plash $ cd plash $ git-clone http://plash.beasts.org/git/glibc.git/ glibc-source $ autoconf $ ./configure GLIBC_BUILD_TYPE=integrated GLIBC_VERSION=9999 $ ./make.sh $ ./build-glibc-integrated.sh link_source $ ./build-glibc-integrated.sh configure_glibc $ ./build-glibc-integrated.sh make_glibc
Tasks
Refactoring to do in preparation for this change:
- Split the libc-*.c files into smaller .c files, one per function, to match glibc's organisation. -- Not done.
- Currently, the Plash libc code avoids using stdio functions if IN_RTLD is defined. This should be inverted so that it only uses stdio if some DEBUG macro is defined.
Create unit tests for glibc functions. The existing functional tests are too hard to debug. glibc's own test cases are not very manageable, and tend to be regression tests.
- Add test cases for: connect(), getsockname(), getsockopt(), execve(), truncate(), getuid(), xmknod(), inotify_add_watch()
- Add test cases for ld.so
- Add test to check that libc.so exports all the symbols that it should
Convert the stat() tests in tests/run-tests.pl to Python
Where the code wraps a library call, ensure that it calls, for example, kernel_close() instead of close().
Main changes:
- Change the export() macros to use glibc's macros for defining aliases and creating versioned symbols
Add function prototypes to the libc-*.c files because glibc's macros require that function types are declared in advance
Rename the entry for the "stat" method in the vtable struct to "fsobj_stat". "stat" conflicts with a macro that glibc defines in an internal header file.
- For consistency, "type", "utimes" and "chmod" have been renamed as well
Work out how the glibc build system will be able to run programs linked with the newly-built libc.so
Use wrapper program, python/scripts/simple_wrapper.py, to set up Plash environment. This means the Plash server code must be built before the glibc build can be run all the way through.
Ensure that extra symbols get exported from libc.so: plash_libc_duplicate_connection, plash_libc_kernel_execve, plash_libc_reset_connection.
These are no longer imported by executables as weak symbols. The executables use dlsym() to access these instead. This change was done under PlashIssues/NestedSandboxesNotWorking.
Split src directory into subdirectories: library code, libc code, front-ends (shell and pola-run.c) -- Not done.
Advance from building glibc 2.5 to glibc 2.7
Fix truncate64(): this should have been wrapped before but was not
Add scripts to the Plash module for building glibc using the new method -- have added build-glibc-integrated.sh
Rules for building using the new method are encoded in scratch/build-tools/chroot_build.py, but this is not a convenient place to have them. They should be rewritten for use within the Plash trunk tree.
- Scripts may assume that patched glibc source has already been placed in the Plash build directory, which is done either by git-clone or by unpacking and applying a patch.
Build on x86-64
Ensure all tests pass
Sort out Debian source packages
Continue using glibc-source-X.XX packages, but include a patch to glibc in the plash source package. The patch will be extracted from Git at the point that the plash source package is created. We must use a single big patch -- can't extract a series of patches from Git because some of the commits will be merges. This is done by BuildTools.
Questions
How do we store the changes to glibc?
Ideally no changes are required to existing glibc files and PlashGlibc functions as an add-on as Linuxthreads does. However this is probably not possible in practice.
Could store a patch file in SVNRepository, but it wouldn't be practical to edit directly or review the changes to it.
Could import the whole glibc source tree into the SVNRepository. However, it is very big compared with the existing code. Also, SVN's support for merging is not very good.
Chosen method: Import the glibc source into a Git repository. It would be relatively easy to re-apply our changes to newer versions of glibc by merging. It would be easy to generate patches from Git automatically. Can produce regular snapshots. These patches would be distributed as part of the Plash source package.
The Plash library code that is built into PlashGlibc would continue to be stored in SVNRepository. This is similar to how glibc does not contain the Mach libraries for when being built for GNU Hurd. The C files would be symlinked into the glibc source tree for the glibc build process to compile. Unit tests for the Plash C library stay in SVN.
How do we synchronise changes to Plash with changes to the PlashGlibc tree in Git? Cannot commit or tag across repositories. This is a problem that Xorg also faces with its repository-per-module arragement.
- The Git repository is here (see SVNRepository):
git-clone http://plash.beasts.org/git/glibc.git/
Given the Git arrangement, how does building Plash work? There would be a circular dependency between the SVN and Git modules. It still makes sense for the two to be in separate modules, because the glibc version can be changed independently of Plash.
In the longer term we may well have branches of other components such as Gtk.