Back to the main page

Cobbler iPXE boot and install on iSCSI LUN

Intro

The doc explains how to use iPXE ( open source network boot firmware) to boot and start installation on iSCSI LUN. It applies for both BIOS/UEFI systems.
Cobbler supports gPXE, which is older implementation and is not developed any more. The iPXE has started as fork of gPXE, hence Cobbler uses ipxe and gpxe interchangeably.

Create iSCSI LUN

Have LUN created on ZFS appliance. This is how.

iPXE boot and install

Cobbler setup

Have sub-dir "gpxe" under /tftpboot and place files snponly.efi and undionly.kpxe there.

Additionally, the system needs to be added as Cobbler client, with "net boot" enabled. Select profile (this is distro + kickstart) to install.
Kernel option looks like: ks=https://[cobbler-server]/cblr/svc/op/ks/system/[system] inst.nonibftiscsiboo
Note the option "inst.nonibftiscsiboo", see this doc for more info:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/installation_guide/appe-iscsi-disks#sect-iscsi-disks-anaconda

DHCP setup

The DHCP cfg for boot and installation is: 
 

host [system] {
     hardware ethernet [MAC] ;
     fixed-address [system IP];
     option routers [router ip];
     next-server [cobbler server ip] ;
     #
     if exists user-class and option user-class = "iPXE" {
        filename "http://[cobbler- server]/cblr/svc/op/gpxe/system/[system]" ;
     } elsif option arch != 00:00 {
       filename "gpxe/snponly.efi" ;
     } else {
      filename "gpxe/undionly.kpxe" ;
     }
  }
Note iPXE boot file is http://[cobbler-server]/cblr/svc/op/gpxe/system/[system]
It is created by Cobbler and it reads OS to be installed (OL8.0 in this example), kernel options (including inst.nonibftiscsiboot), ks file, etc.
It's created from templete /etc/cobbler/pxe/gpxe_system_linux.template, "initrd=$initrd_name" doesn't come with Cobbler, so add it in. Without this, UEFI systems will not be installed successfully. The template reads: 
 

#!gpxe
kernel http://$server:$http_port/cobbler/images/$distro/$kernel_name
imgargs $kernel_name $append_line initrd=$initrd_name inst.nonibftiscsiboot inst.sshd
initrd http://$server:$http_port/cobbler/images/$distro/$initrd_name
boot

 

#!gpxe
kernel http://[cobbler-server]:80/cobbler/images/+OracleLinux-R8-U0-dvd-x86_64/vmlinuz
imgargs vmlinuz  ksdevice=bootif lang=  inst.nonibftiscsiboot ks=https://[cobbler-server]/cblr/svc/op/ks/system/[system] text initrd=initrd.img
initrd http://[cobbler-server]:80/cobbler/images/+OracleLinux-R8-U0-dvd-x86_64/initrd.img
boot
(From OL7.6 Release Notes) Booting from an iSCSI device that is not configured by using iBFT now supported. The installer now includes a new boot option, inst.nonibftiscsiboot. This boot option can be used to install the boot loader onto an iSCSI LUN that has not been configured in the iSCSI Boot Firmware Table (iBFT).

Kickstart file

An iSCSI LUN can't be added via text installation, hence vcn will be used, example of kickstart file is: 
 

install
vnc
url --url=http://[cobbler-server]/cobbler/links/+OracleLinux-R8-U0-dvd-x86_64
network --device [mac] --bootproto static --ip [system ip] --netmask 255.255.248.0 --gateway [router ip] --nameserver [dns] --hostname [system]
lang en_US.UTF-8
keyboard us
rootpw some-root-passwd
timezone --utc America/Los_Angeles
bootloader --location=mbr
reboot

Installation

Reboot the server, select network boot and continue with installation.

Netboot

Disable Netboot in Cobbler (you can do it right after installation starts), otherwise endless pxe boot/install will continue.
After Netboot is disabled, the file http://[cobbler-server]/cblr/svc/op/gpxe/system/[system] reads (file content is changed by Cobbler): 
 

#!gpxe
# Boot from local hard disk
iseq ${smbios/manufacturer} HP && exit ||
sanboot --no-describe --drive 0x80

VNC

Since the installation starts VNC server, connect to it via VNC client. As reminder, adding iSCSI LUN is not possible in the text mode.

Install destination

In this example, the system doesn't have any local disk, hence select iSCSI LUN as installation destination. With OL8 installation, select Specialized & Network Disks - Add a disk - Add iSCSI Target.

The iSCSI target has to be IP address, and Initiator Name is provided by installation.



Select "Bind target to NIC" - Start Discovery.



Select IQN and Login In. If login is successful, expect to see the disk.



Verify Capacity (30G) and Model (ZFS storage 7330), like on the picture above. For UEFI systems, you may need to go for Custom Storage Configuration, if Automatic gives an error. Select Done.



If all looks good, like on the picture above, select Done.



Select Begin Installation.

OS boot from iSCSI LUN

After installation, in order to boot OS from iSCSI LUN, the "filename" in DHCP configuration has to be different. 
 

host [system] {
     hardware ethernet [mac] ;
     fixed-address [ip] ;
     option routers [route ip] ;
     next-server [cobbler ip] ;
     #
     if exists user-class and option user-class = "iPXE" {
        filename "http://[cobbler-srv]/tftpboot/gpxe/clients/[system]" ; 
     } elsif option arch != 00:00 {
       filename "gpxe/snponly.efi" ;
     } else {
      filename "gpxe/undionly.kpxe" ;
     }
  }
The content of file http://[cobbler-srv]/tftpboot/gpxe/clients/[system] reads (commands like echo, show, ifstat can be omitted). This file is not Cobbler generated, hence needs to be created manually or via some tool (iSCSI server IP and IQN have to be known) 
 

#!gpxe
#http://ipxe.org/cmd
dhcp || echo "Cannot configure interface via DHCP"
echo "Starting server, Boot filename is ${filename}"
show net0/ip
ifstat
sanboot iscsi:<zfs-app-ip>::::iqn.1986-03.com.sun:02:<some-numbers> || \
echo "Command sanboot fails"
boot || echo "Cannot download and boot an executable image"

Login to the system

If all is well, your system should be running and you can login to it. Some of verification steps can be:

Problems

It can happen that OS cannot boot completely and goes into dracut shell. The dracut creates temporary file system (initrd.img) used by kernel for loading block device modules (like iSCSI) which are needed to access the root filesystem. It's under investigation why this happens for certain distros, the iSCSI modules fail to load and root can't be mounted from iSCSI LUN.

Back to the main page