Proxmox iscsi over zfs

FreeNAS

Download & Install

• https://www.freenas.org/download-freenas-release/

Web Access

• https://dhcp_ip_of_freenas

Network Settings

• Network -> Interfaces -> Select your Interface -> press the ">" symbol at the right side -> EDIT -> change config as required

Prepare disks

Create Pool

• Storage -> Pools -> ADD
  • Create new pool
  • Select the disks and Raid level you wanna use

• • Create -> tick confirm and hit CREATE POOL

Create dataset within the pool

• Storage -> Pool -> go to the pool you like and press the 3 dots -> Add Dataset -> change as required

Enable iSCSI

• Services -> iSCSI -> start and enable iSCSI

Create Dataset

• Storage -> Pool -> Select Pool ->

Prepare iSCSI

Create Portal if it doesn't already exist

• Sharing -> Block Shares (iSCSI) -> Portals -> ADD

• Save

Create Target

• Sharing -> Block Shares (iSCSI) -> Targets -> ADD

• Save

Proxmox

Lets create the SSH keys on the proxmox boxes. (The IP must match your iSCSI Portal IP)

You only need to create the keys on one node if they are clustered as the keys will replicate to the other nodes.

• portal_ip=192.168.1.1
• mkdir /etc/pve/priv/zfs
• ssh-keygen -f /etc/pve/priv/zfs/$portal_ip_id_rsa
• ssh-copy-id -i /etc/pve/priv/zfs/$portal_ip_id_rsa.pub root@$portal_ip

Make an SSH connection from every node to the iSCSI Portal IP

• ssh -i /etc/pve/priv/zfs/$portal_ip_id_rsa root@$portal_ip

Install the REST client on every node

• apt-get install librest-client-perl git

Download the patches on every proxmox node

• git clone https://github.com/TheGrandWazoo/freenas-proxmox

Install the patches on every proxmox node

• cd freenas-proxmox
• patch -b /usr/share/pve-manager/js/pvemanagerlib.js < pve-manager/js/pvemanagerlib.js.patch
• patch -b /usr/share/perl5/PVE/Storage/ZFSPlugin.pm < perl5/PVE/Storage/ZFSPlugin.pm.patch
• patch -b /usr/share/pve-docs/api-viewer/apidoc.js < pve-docs/api-viewer/apidoc.js.patch
• cp perl5/PVE/Storage/LunCmd/FreeNAS.pm /usr/share/perl5/PVE/Storage/LunCmd/FreeNAS.pm

Restart the PVE services

• systemctl restart pvedaemon
• systemctl restart pveproxy
• systemctl restart pvestatd

Reload the PVE webgui. Now FreeNAS-API should we available as a iSCSI provider.

Create a iSCSI target on the FreeNAS box.

You dont need to create any extents as the FreeNAS-API plugin will do this automatically when the drive is created on the VM.

NOTE: If you followed this guide you already created your target

Setup ZFS Over iSCSI i Proxmox GUI. choose FreeNAS-API as provider.

ID: Whatever you want
Portal: iSCSI portal IP on the freenas box
Pool: Select your pool (eg: dagobert/VirtualMachines )
ZFS Block Size: 4k
Target: IQN on the FreeNAS box and target ID (eg: "qn.2005-10.org.freenas.ctl:proxmox")
API use SSL: Unchecked
API Username: root
API IPv4 Host: iSCSI portal IP on the freenas box
API Password: root password on freenas box
Thin provision and Write cache is optional

API after update

Note: A pve-manager upgrade will replace the patched files so i suggest you create a bash script and run it on every node after an upgrade.

I have a script that looks like this

#!/bin/bash

cd /root/freenas-proxmox 

patch -b /usr/share/pve-manager/js/pvemanagerlib.js < pve-manager/js/pvemanagerlib.js.patch
patch -b /usr/share/perl5/PVE/Storage/ZFSPlugin.pm < perl5/PVE/Storage/ZFSPlugin.pm.patch
patch -b /usr/share/pve-docs/api-viewer/apidoc.js < pve-docs/api-viewer/apidoc.js.patch

cp perl5/PVE/Storage/LunCmd/FreeNAS.pm /usr/share/perl5/PVE/Storage/LunCmd/FreeNAS.pm

systemctl restart pvedaemon
systemctl restart pveproxy
systemctl restart pvestatd