A few days ago, Google as announced the results for GSOC 2020. And guess what! I have been selected this year to work at gentoo project. I have already applied at GSOC once before in summer 2016 at radareorg. It was my first open-source contribution and engagement with a community in my life. I can still remember how happy I was back then. And this time is no different. I am already Gentoo user, and I liked the OS (I tried all sorts of Linux kernel flavours: Ubuntu, Debian, Fedora, Arch, Slackware, Mint, Mandriva and every distro of them got its downside). So I decided to try helping with developing Gentoo at least for a little while. It is the least one can do in gratitude. I thought that GSOC would be a kind of motivation to commitment for at least until GSOC end.
So what is it all about?
My project is about porting relibc into Gentoo. Relic is a relatively new C standard library, (almost) completely written in rust. As for Gentoo, there is already support for multiple libc implementations including Glibc, Musl, uClibc. Gentoo provides stage-3 builds for each target libc. Stage 3 builds are simply a chroot environment that contains all the needed binaries, libraries and configuration files needed to compile a whole Gentoo OS that supports our libC of choice. So my ultimate goal is creating Relibc Stage 3 for Gentoo.
But the problem with Relibc is that it is fairly new. It doesn’t have compiler support, no Binutils support. And there isn’t readily available cross-compiling toolchain for Relibc. and we don’t need “a” compiler we actually need 2 of them: one is the GCC compilers and the other one is rust compiler. The reason is that without rust compiler we can never build Relibc based system. Another issue is that relibc is not yet production tested. So there might be many regressions that need to be fixed.
The biggest and scariest issue is that having two libc implementations in one system isn’t exactly the best idea, one wrong shell command and your system is done for.
Since the project is “my type”, and I am interested in it anyways, and I had free time during the semester break. I realized there is nothing stopping me from starting to work on the project early. Actually, if I recall correctly, I started way early, even before GSOC announced which organizations got accepted and which one did not. While this may not be the best strategy for someone whose final goal is just joining GSOC, I was really hyped by the project and at this point, I nearly forgot about the whole GSOC program and kept just working on the task that sometimes my mentor kept reminding me to send my proposal.
Setting up the environment
What is the easiest way to isolate a development environment that is completely separated from your day to day system? Of course a virtual machine! But every once in a while, there will be a scenario where virtual machines are the worst way to do things. And that is exactly the case here. What we want to do is utilize the whole CPU power for compiling, save as many memory for linking while having a completely isolated environment. So that calls for I don’t know, maybe docker? Except for I don’t like docker, and I sucks at docker commands. So I ended up setting chroot environment for Gentoo. The original chrooted environment is x64 Glibc based. If anything goes wrong I can always discard the whole folder and create a new one. Next, I (naively) created an ebuild repository for relibc for Gentoo and hoped that that will be mainly what is needed. I was thinking what could possibly be missing for this to work. And the answer is basically everything. Relibc conflicted with the default Glibc installation. So we first need to get rid of the old Glibc and everything built with Glibc and instead install Relibc. But to do that we need a Relibc based compiler installed somewhere. And this is exactly where the journey begins.
So it won’t be possible to have a relibc-based stage3 without cross-toolchain. But building cross-toolchain for an unsupported target is considered tricky business. For that, I started by following LFS guide, while looking at the same time at code snippets that are Musl specific in gcc / Binutils .. etc. ٍSince the process started to get complicated, and I had to add custom command line arguments here and there. I decided to keep track of what I did in a separate repository. It is basically short notes and small scripts to guide me through the process should I ever destroy my chroot environment.
During the process, I came to realize that relibc’s linker is missing many features like being transparent for example or loading weak symbols. So down the path, I kept implementing those missing bits until things worked.
Native toolchain and debugger support
At this point, the next natural step is using the cross-toolchain to build stage3. But that will be overwhelming especially that it will be more than likely that I will find bugs in relibc. So instead I decided to use the cross-toolchain to build native toolchain. I started with building libstdC++. Well, I got segmentation faults everywhere.
That was when I realized that gdb cannot debug binaries that are dynamically linked against relibc because relibc’s dynamic loader doesn’t’ cooperate with gdb on finding symbols. So I had to implement the RTLD debugging protocol which took much much longer than anticipated. I mean you can always fix bugs by using a debugger. But then you might be in a situation where you are trying to fix bugs in a debugger failing to debug another process by debugging the debugger while failing. If that sounds like lots of “debug” prefixed words, that is exactly the point. It is a complete mess, but someone has to do it.
Once it was implemented, I quickly got all ld.so missing features implemented and
Hello C++ was working. Next, I
started building Binutils against relibc. During each step, there were regressions that were uncovered, debugged, fixed
and in some cases added to relibc test suite.
What is next?
Next thing to do is to have a native gcc compiler and rust compiler. If these are done then I can say it will finally be possible to create relibc based Gentoo stage3.