How to install Ubuntu with both disk encryption AND SSD caching

Question

I'm using Ubuntu in a corporate environment, and our security policy states that we have to use full disk encryption.

I've also got a laptop with a 32GB mSATA SSD and 750GB of spinning rust. My current installation uses bcache to leverage this, installed using this procedure. This provides a very welcome performance boost without me having to worry about filling up the SSD.

This will be a bountied question. The bounty will be awarded for :

• A clear, reliable method of performing a fresh install of Ubuntu
  • Any release is acceptable but 15.04 (Vivid) will be fine
• The entire filesystem will be encrypted
  • The preference here is to use the relevant checkbox in the default Ubiquity installer program (dm-crypt encryption)
• The filesystem will be cached on an SSD
  • For preference, the kernel dm-cache / lvmcache method see here for method to do this with Debian Jessie
  • The cache must also be secured (ie encrypted)
  • There must be a clear explanation as to why the cache is also encrypted

Have already tried the method for Debian Jessie above, but it refuses to boot for me. Have not so far tried the method described in the comments here.

The posted solutions will be tested on a VirtualBox VM with two blank virtual disks and a release copy of 15.04 desktop (amd64 release). Bounty goes to the first solution that I adopt to reinstall my actual hardware.

Please write your solution as if it were going into the community wiki.

---

I've awarded the bounty - I think there is still potential for a "LUKS-on-LVM" solution that combines the ease of the approved answer in only having one password, with only using device-mapper components.

Answer 1

LVM on LUKS on bcache

Here the russian doll game is a little deeper with 3 stacks/layers...

My initial idea about this question was to use a default Ubuntu install with LVM on LUKS and convert it into a bcache backing device with blocks but it did not work for me on my test with LVM.

Moreover, the ubuntu installer (ubiquity) is too limited to install inside a bcache device prepared in advance (at least with LUKS on LVM), so we fallback to a method of doing things manually.

Boot into the live CD/USB and choose "Try Ubuntu" and open up a terminal

Pre-install

sudo -i
# Define some variable to avoid confusion and error
luks_part=/dev/sda3
boot=/dev/sda2                    # boot partition
caching_bcache=/dev/sdb           # SSD or partition in SSD

# Do secure erase of encrypted backing and caching device (see Notes [1])
dd if=/dev/urandom of=$luks_part || dd if=/dev/urandom of=$caching_bcache
# Go and grab some coffe, this will take a while...

apt-get install bcache-tools
# Setup bcache caching and backing devices
make-bcache -C $caching_bcache -B $luks_part
# (Optional) Tweak bcache
echo writeback > /sys/block/bcache0/bcache/cache_mode

# Below we now create manually what ubiquity should have done for us
# Setup LUKS device on bcache device
cryptsetup --key-size 512 luksFormat /dev/bcache0
cryptsetup luksOpen /dev/bcache0 crypted

# Setup LVM on LUKS
# You can skip that part if you don't want to use a swap
# or don't want to use multiple partition. Use /dev/mapper/crypted
# as you root latter on
pvcreate  /dev/mapper/crypted
vgcreate vg /dev/mapper/crypted
lvcreate -L 1G vg -n swap
lvcreate -l 100%FREE vg -n root

Installation

Keep the terminal opened and now run the installation. Choose "Something else" when partitioning and specify

• your boot partition (/dev/sda2)
• your root partition (/dev/mapper/vg-root)
• your swap (/dev/mapper/vg-swap)

and check the checkbox to format your partitions

At the end of the installation, don't reboot but just click "Continue trying ubuntu"

Post-install

In our opened terminal

# Install bcache-tools to add bcache module to initramfs
mount /dev/mapper/vg-root /mnt
mount $boot /mnt/boot
mount -o bind /sys /mnt/sys
mount -o bind /proc /mnt/proc
mount -o bind /dev /mnt/dev
chroot /mnt
# To get apt-get running in the chroot
echo 'nameserver 8.8.8.8' > /run/resolvconf/resolv.conf
apt-get install bcache-tools

# Create /etc/crypttab to add crypted bcached partition
echo "crypted UUID=`blkid -o value /dev/bcache0|head -1` none luks" > /etc/crypttab

exit
sync
umount /mnt/sys
umount /mnt/proc
umount /mnt/dev
umount /mnt/boot
umount /mnt
vgchange -an /dev/mapper/crypted
cryptsetup luksClose crypted
sync

# Reboot & enjoy

There is a known Ubuntu 15.04 reboot bug from Live CD/USB so you might have to force reboot/shutdown

Check

Once booted, you can check that /dev/bcache0 is in fact a LUKS partition with

if sudo cryptsetup isLuks /dev/bcache0; then \
    echo "crypted";\
    else echo "unencrypted";\
fi

This is because it is the cache of your LUKS partition, and you now access your data via the device /dev/bcache0 and never from the original backing device (/dev/sda3 here)

References

http://bcache.evilpiepirate.org/

https://wiki.archlinux.org/index.php/Bcache

https://wiki.archlinux.org/index.php/Dm-crypt

bcache-status is not officially merged into bcache-tools, yet. You can have it here: https://gist.github.com/djwong/6343451

[1] There might be better ways to do this wiping

Answer 2

LVM on LUKS + LUKS/dm-cache

The Ubuntu installer uses the LVM on LUKS configuration for it's full disk encryption.

If you want to also use dm-cache / lvmcache to boost performance, you will need to place your cache pool into an encrypted volume to maintain the security of your data.

The steps are

• Create a LUKS volume on the target block device
• Extend the default volume group with the new encrypted LUKS volume
• Create the cache metadata and data volumes in the new LUKS volume
• Bind them together as a cache pool
• Bind this cache pool to the root volume
• Make sure the new encrypted volume can be mounted at boot by adding it to /etc/crypttab
• Make sure that your boot environment supports dm-cache

---

The script below provides an example, and will add an encrypted cache pool to an existing root filesystem. It was designed for systems that have used the default disk encryption option in the Ubuntu installer - ie ; whole disk partitioned and encrypted, no custom partitions etc.

Please note that there is very little validation or defensive programming in this script. If it destroys your working system, that is your responsibility.

Call thus :

# 1   2          3           4     5    6
sudo bash lvmcryptocache /dev/sdb 32M 1968M

1. Needs root to work
2. run script in bash
3. the script name
4. the block device you want to use (only tested with whole disk)
5. the metadata size
6. the cache data size

The size parameters are by default in MB : you will need a ratio of 1:1000 metadata space to cache space (e.g. if your cache disk is 180GB, you need 180MB of metadata space and 179820MB of data space - you may want to round the metadata up a little to be cautious. There is a lower limit for the metadata of 8M.)

You will be prompted for a password for your cache volume - you will be prompted for the passwords for BOTH of your disks during boot.

References

• Using LVMs New Cache Feature (Richard Jones)
• Encrypting an entire system (dm-crypt wiki)
• Comments on Launchpad Bug #1423796
• Booting Debian Jessie from an lvmcache

---

#!/bin/bash
#
# lvmcryptocache
#
# Add an LVM cache pool on and attach it to the root volume
# Including LUKS encryption
# Assumes you are using an "all on root" setup
# If you are not, adapt it if you like
#
# Script licensed GPL3 or later
# © Adrian Wilkins May 2015
#
# Pass the name of the disk device you are using as a cache
# This should ideally be totally blank, so run
#
# dd if=/dev/zero of=/dev/${DISK}
#
# over it for a short while to nuke the partition table

CACHE_DISK=$1
META_SIZE=$2
DATA_SIZE=$3

DISK_NAME=$(basename $CACHE_DISK)

CRYPT_VOLUME=${DISK_NAME}_crypt
CACHE_PV=/dev/mapper/${CRYPT_VOLUME}

# Create LUKS volume in raw disk

cryptsetup luksFormat $CACHE_DISK
cryptsetup open --type luks $CACHE_DISK $CRYPT_VOLUME

# Started to try and work out disk size stuff but it's complex
# Have a go if you like, I kept running out of extents
#
# DISK_SIZE=$(fdisk -l | grep "Disk ${CACHE_DISK}" | awk ' { print $5 } ')
# 
# META_SIZE=$(( DISK_SIZE / 1000 ))
# META_SIZE=$(( META_SIZE + 512 ))
# MOD=$(( META_SIZE % 512 ))
# MOD_OFFSET=$((512 - MOD))
# META_SIZE=$((META_SIZE + 512)) 
# META_SIZE=$((META_SIZE + MOD_OFFSET))
# 
# DATA_SIZE=$(( DISK_SIZE - META_SIZE))
# 

# Create new PV inside encrypted volume

pvcreate $CACHE_PV
vgextend ubuntu-vg $CACHE_PV
lvcreate -L ${META_SIZE} -n cachemeta ubuntu-vg $CACHE_PV
lvcreate -L ${DATA_SIZE} -n cachedata ubuntu-vg $CACHE_PV
lvconvert --type cache-pool --poolmetadata ubuntu-vg/cachemeta --cachemode writethrough ubuntu-vg/cachedata --yes
lvconvert --type cache --cachepool ubuntu-vg/cachedata ubuntu-vg/root

# Now add the UUID of the cache pool PHYSICAL DRIVE (/dev/sdb) to /etc/crypttab
DISK_UUID=$(ls -al /dev/disk/by-uuid/ | grep $DISK_NAME | awk '{ print $9 }')
echo "${CRYPT_VOLUME} UUID=${DISK_UUID} none luks,discard" >> /etc/crypttab

apt-get install --yes thin-provisioning-tools

HOOK=$(tempfile)
# Add a hook script to initramfs to add the right tools and modules

echo "#!/bin/sh" > $HOOK
echo "PREREQ="lvm2"" >> $HOOK
echo "prereqs()" >> $HOOK
echo "{" >> $HOOK
echo "  echo \"$PREREQ\"" >> $HOOK
echo "}" >> $HOOK
echo "case $1 in" >> $HOOK
echo "prereqs)" >> $HOOK
echo "  prereqs" >> $HOOK
echo "  exit 0" >> $HOOK
echo "  ;;" >> $HOOK
echo "esac" >> $HOOK
echo "if [ ! -x /usr/sbin/cache_check ]; then" >> $HOOK
echo "  exit 0" >> $HOOK
echo "fi" >> $HOOK
echo ". /usr/share/initramfs-tools/hook-functions" >> $HOOK
echo "copy_exec /usr/sbin/cache_check" >> $HOOK
echo "manual_add_modules dm_cache dm_cache_mq dm_persistent_data dm_bufio" >> $HOOK

cp $HOOK /etc/initramfs-tools/hooks/lvmcache
chmod +x /etc/initramfs-tools/hooks/lvmcache

echo "dm_cache" >> /etc/initramfs-tools/modules
echo "dm_cache_mq" >> /etc/initramfs-tools/modules
echo "dm_persistent_data" >> /etc/initramfs-tools/modules
echo "dm_bufio" >> /etc/initramfs-tools/modules

# Update initramfs

update-initramfs -u

echo Now reboot!