Offload compilation through SSH?

Question

I have a hacked up Chromebook on which I'm running Gentoo. When I try to compile anything, CPU usage spikes up to 100%, the temperature increases by ~10 degrees C, battery usage spikes (4.X W -> 10 W), and it's a slow process. But I also have an Arch Linux computer running, and I can connect to it over SSH. They are both x86_64 CPUs. Is there any way I could offload the compilation of stuff (Linux kernel, everyday packages, etc.) onto the Arch Linux machine over SSH? I haven't done anything like this before. Might cross-compilation be necessary?

Answer 1

No, you don't have to cross-compile (that would be necessary if you targeted another architecture.) There are two ways that I can think of that you could set up your systems to do this:

1. Use distcc. The Gentoo and Arch wikis do a good job of describing how to install and configure the program, so I won't copy the entire thing here. Briefly, you need to have the following set up in order for it to work:

   • Your CFLAGS in /etc/portage/make.conf must not use march=native or mtune=native, because the remote computer will use its idea of "native" CPU, not the local computer's. If you're using "native", find out which flags to use by running:

     $ gcc -v -E -x c -march=native -mtune=native - < /dev/null 2>&1 | grep cc1 | perl -pe 's/^.* - //g;'
     

   • Both computers need the same compiler and binutils versions.

   • Both computers need distcc installed, configured and running.

2. Use a chroot environment on your Arch system with a copy of your Chromebook filesystem (treat this like you're doing an installation of Gentoo, so copy resolv.conf from your Arch installation, and mount the appropriate filesystems inside per the Gentoo installation manual, keeping in mind the warning about /dev/shm if Arch's version is a symlink.) It needs to be as close as possible to your Chromebook environment, or else you'll end up with possibly incorrect binaries; if you do a copy, you'll have to rebuild less packages. Inside of this environment:

   • Add FEATURES="buildpkg" to /etc/portage/make.conf.
   • The generated packages will then be in /usr/portage/packages. You can also compile the kernel in this way and simply copy the generated kernel and appropriate /lib/modules directory to the Chromebook. (Remember that these directory locations are relative to the chroot!) The wiki recommends having an NFS mount or other server so that you don't have to copy files manually: this can be set up on the Arch system proper. I like setting up rsyncd for this purpose, but use whatever method you prefer for file access.

   On your Chromebook:

   • Make sure to add FEATURES="getbinpkg" to /etc/portage/make.conf if you want to prevent it from compiling locally.
   • If you're using remote file access, add PORTAGE_BINHOST="protocol://path/to/your/chroot/usr/portage/packages" to /etc/portage/make.conf.

   Refer to the Binary package guide in the Gentoo wiki for more information.

I have done both of these methods in the past, and they both work pretty well. My observations on the two methods:

1. distcc is finicky to get working, even if you have identical setups on both sides. Keeping gcc and binutils versions the same will be your biggest challenge. Once you get it going, however, it's pretty fast, and if you have extra computers that are fast enough you can add them.

2. The chroot environment is less finicky, but if you make changes to any part of the portage environment (CFLAGS, USE flags, masks, profiles, etc.) you have to make sure that both sides stay consistent, or else you can end up with packages that have the wrong dependencies. Gentoo is pretty good about making sure the USE flags match, but it doesn't track compiler options in binary packages. One advantage is that you're not limited by the (lack of) disk space and memory on the Chromebook for compilation.

If you're going to use the chroot method, I would make a script to do all the uninteresting work required in setting it up (replace /mnt/gentoo with your chroot location):

cp -L /etc/resolv.conf /mnt/gentoo/etc
mount -t proc proc /mnt/gentoo/proc
mount --rbind /sys /mnt/gentoo/sys
mount --make-rslave /mnt/gentoo/sys
mount --rbind /dev /mnt/gentoo/dev
mount --make-rslave /mnt/gentoo/dev
chroot /mnt/gentoo /bin/bash
umount -R /mnt/gentoo/dev
umount -R /mnt/gentoo/sys
umount /mnt/gentoo/proc

Answer 2

Inspired by the accepted answer, but too much for a comment:

On the chromebook, to isolate the root filsystem from system mount points /dev, /sys, /run etc.:

mount -o bind / /mnt/gentoo
# If neccesary
mount -o bind /boot /mnt/gentoo/boot
mount -o bind /usr /mnt/gentoo/usr

In /etc/exports: /mnt/gentoo 192.168.0.2(rw,sync,nohide,no_root_squash)

Make sure there is kernel support for exporting over NFS. And start the relevant services. (nfsd)

On the Arch linux box, /etc/fstab:

# You might need to tune mount options for performance.
192.168.0.3:/mnt/gentoo   /mnt/gentoo   nfs   defaults 0 0

Make sure the correct service is started on the Arch side (netmount or nfs). Check if the remote filesystem has successfully mounted.

Gracefully stolen from ErikF (don't copy resolv.conf, since the running system should already have it):

mount -t proc proc /mnt/gentoo/proc
mount --rbind /sys /mnt/gentoo/sys
mount --make-rslave /mnt/gentoo/sys
mount --rbind /dev /mnt/gentoo/dev
mount --make-rslave /mnt/gentoo/dev
chroot /mnt/gentoo /bin/bash
umount -R /mnt/gentoo/dev
umount -R /mnt/gentoo/sys
umount /mnt/gentoo/proc

Make sure /etc/portage/make.conf has a correct -march setting, not native!

Now, run emerge to your liking and everything is installed directly into the rootfs of the chromebook.

Important note: I've added this answer as a possibility that seems fun. It has not been tested. Also, exporting the root filesystem (rw, no_root_squash) will give the creeps to any guy that respects IT security. So only do this on a very trusted network!