There is something seriously wrong with my server, I can only write over nfs at 5 MB/s. Local writes rarely break 35 MB/s. I've tried disabling sync but to no avail. So… I'm afraid I'm going to have to dump Solaris for Linux and part of that is re-arranging my drives.
As a stop-gap measure I want to run Solaris in a virtual machine, and I want my LSI 2008 controller to be pci passed through, so I had to move my rpool from one controller to the LSI.
Like everything Solaris, this was harder than it should have been. Solaris 11 does not have failsafe mode so all of the awesome howto's only apply to Solaris 10.
# boot to single user mode via grub kernel$ ... -s # can't boot, can't find rpool # boot with solaris cd, thankfuly my motherboard is awesome and I # can redirect an iso into a virtual cdrom via ipmi zpool import rpool # error, previously mounted on other host (or something like that) zpool import -f rpool zpool status # very different disk ids reboot # success!