TIP: Compressing portage using squashfs: initscript method

I have left gentoo and have not updated this script for a while, though it probably still works, mv has taken over and proposed new features and improvements, aufs is one good thing among others. And his script can be used on any directory (I would recommend /usr/src/linux and the docs) You should probably have a look at the thread and check mv's version, now. Last version is available here --- I was sure I had left a notice like this one here but only realize now I must have forgotten...

Abstract The portage tree takes typically 600MB on /usr/portage, some people have chosen to have it on a separate partition with a small block size to

decrease its global size
decrease fragmentation
increase the speed during searches and update

However that also means that the free space on that partition is lost. Another, better, solution is to use a stack file containing the filesystem with small block size to avoid the shortcomings of the separate partition. Following the discussion in this post, the best solution found is to compress the portage tree using the compressed, read-only squashfs filesystem and mount a read-write partition on top of it using unionfs. Second, for convenience, the portage tree should be mountable on boot-up and updated on shutdown, hence I have written an initscript to automate this process. I will publish and update this program here.

Assumptions I will assume that you have

read the original idea as it details what you need to get started and that you are familiar with the bootup/shutdown sequence of your computer,
read the Gentoo doc on initscripting.
DISTDIR should be out of the tree, etc.

Please just read the original post as everything is well documented there and come back once you have done that, unless you do understand what you are doing!

Introduction In brief, you will need to load the relevant kernel modules and emerge the userland utilities

Code:

emerge -avt sys-fs/unionfs sys-fs/squashfs-tools

, unmask if needed (I am now successfully using sys-fs/unionfs-1.2 on sys-kernel/suspend2-sources-2.6.16-r8 and sys-fs/squashfs-tools-3.0) 

Code:

modprobe loop squashfs unionfs

add the modules to /etc/modules.autoload.d/kernel-2.6.

the script

/etc/conf.d/squash_portage:

# /etc/conf.d/squash_portage # SQFS_DIRNAME points to the directory that will contain the sqfs # images, recommended value is /var/tmp SQFS_DIRNAME="/var/tmp" # Leave PORTAGE_RW empty for use with tmpfs, a ram-based filesystem, # This is recommended unless you are short of RAM PORTAGE_RW=""

and 

/etc/init.d/squash_portage:

#!/sbin/runscript # Copyright 1999-2006 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 # $Header: $ # # /etc/init.d/squash_portage allows efficient compression of # Gentoo portage arborescence # # It requires support for the loop device and squashfs enabled in the kernel, # module autoloading is also *highly* recommended. # sys-fs/squashfs and sys-fs/unionfs are necessary for read-write support. # # Author: Mathias Laurin <mathias_laurin @ users.sourceforge.net> # 2006-11-28, v.0.1.5(4) source /etc/make.conf SQFS_CUR="$SQFS_DIRNAME/portage.sqfs" SQFS_NEW="$SQFS_DIRNAME/portage-current.sqfs" SQFS_OLD="$SQFS_DIRNAME/portage-old.sqfs" DEF_RW="/dev/shm/.portage-rw" depend() {    need localmount } start() {    ebegin "Mounting read-only squashfs image"    mount -rt squashfs -o loop,nodev,noexec $SQFS_CUR $PORTDIR    retval=$?    eend $retval    [ $retval -ne 0 ] && return $retval    ebegin "Mounting read-write with unionfs"    if [ ! $PORTAGE_RW ]    then       einfo "  mounted in tmpfs (RAM)"       PORTAGE_RW="${DEF_RW}"    fi    [ -d $PORTAGE_RW ] || mkdir -p $PORTAGE_RW    chmod 0750 $PORTAGE_RW    chown portage:portage $PORTAGE_RW    mount -t unionfs -o nodev,noexec,dirs=$PORTAGE_RW=rw:$PORTDIR=ro unionfs $PORTDIR    eend $? } stop() {    ebegin "Updating portage tree"    [ ! $PORTAGE_RW ] && PORTAGE_RW="${DEF_RW}"    if [ ! -z `ls -A $PORTAGE_RW | head -n1` ]    then       einfo "  Syncing the tree"       mv -f $SQFS_NEW $SQFS_OLD       mksquashfs $PORTDIR $SQFS_NEW -no-duplicates 2>/dev/null       retval=$?       ln -sf $SQFS_NEW $SQFS_CUR    else       einfo "  Nothing to do"       retval=0    fi    eend $retval    ebegin "Unmounting the tree"    umount -t unionfs  $PORTDIR    umount -t squashfs $PORTDIR    rm -rf $PORTAGE_RW    eend 0 }

That obviously is bash as any initscript. Make it executable and

Code:

/etc/init.d/squash_portage start rc-update add squash_portage default

The script assumes that you already have an sqfs image on your disk ready to be mounted and called $SQFS_DIRNAME/portage.sqfs it will save the new image if you sync'ed the tree the rest is pretty obvious and the rw partition is mounted on /dev/shm by default for speed and simplicity. But beware that it means you will lose your update in case of power failure or if you are not caring. I would recommend to restart the script upon hibernation if you use it and do not always reboot into your ram image.