Thread Tools Display Modes
Old 06-23-2012, 09:17 PM
kingneutron kingneutron is offline
Join Date: May 2005
Posts: 53
Thanks: 6
Thanked 0 Times in 0 Posts
Talking HOWTO: P2V of non-mirrored ZFS root disk (or Bare-metal recovery)
# See for scripts:

-Hi there It's been a while, so I wanted to contribute something to the community again. (Hopefully it helps somebody, cuz it took me like 3 hours to write up!!)

I got to thinking it would be a good idea to be prepared in case I ever had to do a bare-metal recovery of my PCBSD 9/Freebsd 9 server, in case the non-mirrored Root disk died, and Vmware was a natural choice. (Plus I wanted to see if my idea worked.)

I've seen some really complicated recovery howtos out there, with custom scripts and ZFS SEND and the whole 9 yards, but they didn't seem to fit my needs, so I just went ahead and did it My Way (TM.)

In short: I have a tar.gz backup of my PCBSD 9 Root ZFS filesystem on another machine, and want to restore it with a minimum of fuss. To accomplish this, you will need the following:

o First, Obtain the PCBSD 9 " Live DVD 64-bit ", which will act as a recovery boot environment:

o You will need tar and netcat on the PC that has your backup file on it (in my case, Linux Mint.)

o If you want to use my ' nclisten ' script on your Backup PC, you should either install " pv " (Shell pipeline element to meter data passing through) which gives you a progress report, or you can delete the entire " |pv " line in the script, it will still work.

o Both the backup PC and the VM (or your FREEBSD box) need to be connected to the same network, since we'll be transferring data with netcat. (In my case, Gigabit DHCP 192.168.x.x)


This requires a little bit of preparation, so it's always good to run tests like this *BEFORE* the system fails!

** Document the sizes of your partitions on the original machine. I have a backup script that I use (don't forget to chmod +x it. See vmware page for script.) This documents most of what you need to know for a Restore in " /root/df-h.txt ", which I have included as an example. If you're in a bind and need to know how that original box was configured, it can be easily restored from your tar.gz backup file.

For example purposes, in my case I kept the slightly-tweaked defaults for the PCBSD install (I allocated only 2GB of swap space

On an 80-gig root filesystem disk:

zfs tank0 " / " == 54 GB (used: 322MB) # These are important if you're restoring to a smaller disk!
zfs tank0/usr == 69 GB (used: 15GB)
zfs tank0/var == 54GB (used: 371MB)

This is also important during the restore later, because we'll be using Bind mounts for the various ZFS mountpoints.

Also note: In this case, I restored an 80GB ZFS disk to a smaller 20GB disk. You will probably want a slightly bigger vdisk, like 25-30GB or so, if you have a similar setup.


o Boot your new VM (or PC) with either the PCBSD 64-bit install DVD, or the live DVD.

PROTIP: Learn from my mistake and boot your VM with at least 4GB of RAM. I reduced mine from 6GB to 3GB to save on bootup time, and it froze at the end of the restore. (It booted OK tho!)

PROTIP: The live DVD will ask you which desktop environment to start; since we're doing a Restore and want speed and light resources, I would go with LXDE.

Note: If you're using the PCBSD 9 Live DVD, the installer link is broken on the desktop as of this writing. To run it, Click on the Start menu (lower left) \ Accessories \ Terminal, and run
' pc-sysinstaller '.

o Install PCBSD, making sure you replicate the ZFS mount points the same as the original machine ( in this case, " / " , /usr , /var ) with enough space for everything to fit and some leftover free space.

PROTIP: When selecting the disk for system installation, I used GPT, Entire disk, ZFS - and then went into the Advanced tab and selected " Use default layout ", then Edited the Swap and brought the slider down to 2GB; then Edited /usr and used the slider to fill up the rest of the free space. Whatever you do, if you used GPT on the original box then you should use GPT on the VM or Restore-PC.

PROTIP: When installing the Desktop, you don't need to do the full KDE/GNOME thing, or Ports. Most of this temporary install will be Deleted later on to prep for the Restore, so just install ICEWM. (I actually rebooted into the temporary-install environment to test it, but you don't really have to.)

Note: when you get to the "Users" part of the install, if you're not restoring to bare-metal you might want to make sure you change the Hostname here.

Let the install finish, then Shutdown the VM.


OK - after your nice new PCBSD install is finished in your new VM, Reboot into the Live DVD - and we're going to delete most of it for the Restore

o Boot VM with PCBSD live DVD

# Reference - I picked up some tips from: (Thanx!)

Start menu \ Accessories \ Terminal
# So you have a command line to work with ;-)

o Issue the following commands:

mkdir /mnt/tank0
zpool import
# This does not actually do any importing yet, it just gives you a list.

zpool import -R /mnt/tank0 tank0
# this grabs the ZFS root and mounts -almost- everything OK (/usr and /var, but not " / ") - which we will be fixing shortly...


zfs list
# since /tank0 mountpoint is LEGACY, it needs to be manually mounted somewhere

# Here we are creating a temporary place to mount the VM's ZFS root so we can have a place to Restore to

mkdir /mnt/tank0/root
mount -t zfs tank0 /mnt/tank0/root
# best compromise - but unless we do union mounts, it's gonna make restoring from tar a pain

# this fixes it right - we have to unmount the Imported /usr and /var and remount them under the temporary root
# for all the files from the tarball to go to the right places automagically:

umount tank0/usr
# otherwise we get busy error
mount -o union -t zfs tank0/usr /mnt/tank0/root/usr

umount tank0/var
mount -o union -t zfs tank0/var /mnt/tank0/root/var

# Explanation: If you don't unmount and UNION those mount points, /mnt/tank0/root/usr ( and var ) are BLANK directories!

# As always, repeat for any other ZFS mount points that are under ' zfs list '.

# Okay - at this point, everything should be in place for the restore, you can check with ' df '

# /mnt/tank0/root == " / "
# /mnt/tank0/root/usr == " /usr "
# /mnt/tank0/root/usr == " /var "

# Midnight Commander - thankfully included on the Live DVD FTW!
cd /mnt/tank0/root

# Fix weird immutable files 1st: (for linux folks, this is equivalent to ' chattr ')
# Otherwhise when we try to delete the dirs, mc will complain that certain files can't be deleted
chflags noschg bin/rcp
chflags noschg lib/*
chflags noschg libexec/*
chflags noschg libexec/resolvconf/*
chflags noschg sbin/*
chflags noschg usr/bin/*
chflags noschg usr/lib/*
chflags noschg usr/lib32/*

# Delete the following dirs: (select multiple with Insert, Delete with F8)

# Enter into your ZFS mountpoint dirs, and delete the CONTENTS of:
# To select: (with Numlock on, hit the numeric keypad "+" key, select "*" and delete with F8)

# Leaving behind in the root dir:
~home # symlink


/ Truncated for 10,000 char post limit!
Reply With Quote
Old 06-23-2012, 09:18 PM
kingneutron kingneutron is offline
Join Date: May 2005
Posts: 53
Thanks: 6
Thanked 0 Times in 0 Posts
Angry The rest of it:
Now we're ready for the RESTORE: for some WEIRD REASON, have to do the gunzip on the Receiving side, or tar bitches about an unknown compression protocol...

# Exit out of 'mc' with F10
# Thankfully Netcat is also included on the Live DVD -- Thanks, PCBSD!!

cd /mnt/tank0/root
# make a note of your IP address

time nc -l 32100 |gzip -cd |gtar xpf -
# NOTE: This assumes that the software Firewall is turned off, or allowing that particular port.

# on the PC that has the backup tarfile (type everything after the $, w/o the quotes
bkps-local/p2400quad $ ' cat bkp-p2400quad--pcbsd-zfs--root--80gig-20120621--running-ok--ftr-major-upgrades.tar1.gz |nc -w 10 32100 '

# Use the IP address from the VM here, and the port

PROTIP: From another Terminal in the VM/Restore PC, you can ' renice -5 9999 ' (use the PID of gzip here, find it with ' ps ax|grep gzip ') to give the gzip decompression a little CPU boost and speed up the restore.

! VM froze up after restore completed (possibly due to lack of RAM for ZFS), but booted OK


OK! Once the restore is done, shutdown the VM and remove the PCBSD install ISO from being mounted at boot.


If you're like me, you're prolly running an Nvidia driver - and X won't start in the VM after the reboot. We need to drop the Nvidia driver by doing this:

o Login as root on the text console.

Xorg -configure
cd /etc/X11
cp xorg.conf xorg.conf-nvidia
cp ~/ xorg.conf

o We also need to edit /etc/rc.conf for your new Ethernet adapter and IP configuration. In my case, I have both DHCP and static on the same adapter, so:

# make note of the adapter, in my case " em0 "

nano /etc/rc.conf

# Comment out your old entries 1st, then:
ifconfig_em0_alias0=" netmask"


At this point you should probably reboot. X should come up without the Nvidia driver, and your new networking settings should be active.

Installing vmware tools now is a good idea, but you may need to install compat6x-amd64 1st.

' pkg_add -r compat6x-amd64 '

I had some trouble getting the tools CD to mount, but firing up ' dolphin ' and selecting the CD worked OK. I copied the tarball to /root/dnld and extracted it with ' tar xzf ', and ran the tools installer.

After the tools are installed, Logout of X and you should see the new screen resolution.

--If indeed everything is working as expected at this point, and you took a snapshot, remember to Delete it. ;-)

If you need to enable ZFS compression on /usr:

' zfs set compression=gzip-1 tank0/usr '

# To save some space right away:
' portsnap fetch update '


--I leave as an exercise for the reader to find which files were Immutable and put them back that way. (Feel free to post your solution here!) This was my P2V Freebsd ZFS root-disk proof-of-concept, and I'm happy with it.


Boot into PCBSD 9 Live environment in the VM/Restore PC

Install PCBSD with the same ZFS mountpoints as the original box (can install to smaller disk if needed)

Import your ZFS root and Delete almost everything


Reboot and fix X / Nvidia and networking


Note: I used Workstation 8.02--64 Linux for this. Thanks, Vmware and PCBSD!!

This was my 1st crack at the procedure, so feel free to post optimizations
Reply With Quote

p2v zfs root bare metal

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT. The time now is 07:21 PM.

Powered by vBulletin® Version 3.8.5
Copyright ©2000 - 2015, Jelsoft Enterprises Ltd.

Copyright 2005-2015, The PC-BSD Project. PC-BSD and the PC-BSD logo are registered trademarks of iXsystems.
All other content is freely available for sharing under the terms of the Creative Commons Attribution License.