Install CentOS 7 with kickstart over proxy

In the previous post, I've installed CentOS 7 with kickstart and pxeboot. Unfortunatelly I was doing this in work and of course I'm behind proxy server. It is not as easy as I might think.

TOC

• Overview
• Attach console to guest
• Setup libvirt network
• Prepare ks file and publish it to web
• Sources

Overview

I need to slightly edit kickstart file because of proxy. I've added --proxy to url line:

url --url="http://merlin.fit.vutbr.cz/mirrors/centos/7.0.1406/os/x86_64" --proxy="http
://name:password!@proxy.somewhere.com:8080"

Normal installation (without kickstart) over proxy went OK, but with kickstart it was not so easy. It failed to download .treeinfo and stage2 image:

[    8.390807] localhost dracut-initqueue[557]: anaconda fetching installer 
from http://merlin.fit.vutbr.cz/mirrors/centos/7.0.1406/os/x86_64
[    8.398284] localhost dracut-initqueue[557]: % Total    % Received % Xferd
  Average Speed   Time    Time     Time  Current
[    8.399503] localhost dracut-initqueue[557]: Dload  Upload   Total   Spent
    Left  Speed
[    8.568479] localhost dracut-initqueue[557]: 0     0    0     0    0     0
      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed connect to merlin.
fit.vutbr.cz:80; Connection refused
[    8.573426] localhost dracut-initqueue[557]: Warning: Downloading 'http://
merlin.fit.vutbr.cz/mirrors/centos/7.0.1406/os/x86_64/.treeinfo' failed!
[    8.575191] localhost dracut-initqueue[557]: Warning: can't find installer
 mainimage path in .treeinfo
[    8.586934] localhost dracut-initqueue[557]: % Total    % Received % Xferd
  Average Speed   Time    Time     Time  Current
[    8.588771] localhost dracut-initqueue[557]: Dload  Upload   Total   Spent
    Left  Speed
[    8.756254] localhost dracut-initqueue[557]: 0     0    0     0    0     0
      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed connect to merlin.
fit.vutbr.cz:80; Connection refused
[    8.761437] localhost dracut-initqueue[557]: Warning: Downloading 'http://
merlin.fit.vutbr.cz/mirrors/centos/7.0.1406/os/x86_64/LiveOS/squashfs.img' failed!
[  193.758814] localhost dracut-initqueue[557]: Warning: Could not boot.
[  193.775091] localhost dracut-initqueue[557]: Warning: /dev/root does not exist
[  193.783319] localhost systemd[1]: Starting Dracut Emergency Shell...

Attach console to guest

It is not convenient for me to use vnc as console to see what is happening during boot. Hence I decided to access guest console with virsh console:

[root@kra ~]$ virsh console centos-pxe
Connected to domain centos-pxe
Escape character is ^]
error: internal error: cannot find character device (null)

Of course it didn't work because I had no serial device nor console defined in centos-pxe domain. Definition follows:

<devices>
 .
 .
 . 
 <serial type='pty'>
   <target port='0'/>
 </serial>
 <console type='pty'>
   <target type='serial' port='0'/>
 </console>
 .
 .
 .
</devices>

If I reboot guest and try virsh console it will hang. That's because I didn't tell the kernel what device to use as console. Therefore I append console=ttyS0,115200 to kernel parameters. Meaning of this is: Use serial device on port 0 with speed 115200 b/s as console. Syslinux menu now looks like this:

[root@kra ~]# cat /var/lib/tftp/pxelinux.cfg/default 
default menu.c32
prompt 0
timeout 300
ONTIMEOUT local

MENU TITLE PXE Menu

LABEL CentOS 7 x86_64 pxe
        MENU LABEL CentOS 7 pxe
        KERNEL images/centos/7/x86_64/vmlinuz
        APPEND ks=http://192.168.122.1/ks.cfg initrd=images/centos/7/x86_64/initrd.img console=ttyS0,115200 

Start guest and attach console to it. Console still hangs and it will hang until I choose to boot an item from syslinux menu (or until timeout expires).

Inside initial ramdisk

Let's get back to the problem - proxy is not working. Although I can download required files manually from dracut emergency shell:

dracut:/# curl \
> --output treeinfo \
> --progress-bar \
> --proxy 'http://user:password@proxy.somewhere.com:8080' \
> http://merlin.fit.vutbr.cz/mirrors/centos/7.0.1406/os/x86_64/.treeinfo
######################################################################## 100.0%

After while I've found out what's the problem - python script (/lib/dracut/modules.d/80anaconda/parse-kickstart from initrd) responsible for parsing kickstart file simply ignores --proxy parameter. I've reported it as bug. Anyway, solution is quite simple. At first I need to get inside initrd, so how to extract it:

[root@kra ~]# mkdir -p /tmp/test
[root@kra ~]# cd /tmp/test/
[root@kra test]# cp /var/lib/tftp/images/centos/7/x86_64/initrd.img .
[root@kra test]# file initrd.img 
initrd.img: XZ compressed data
[root@kra test]# mv initrd.img initrd.img.xz
[root@kra test]# unxz initrd.img.xz 
[root@kra test]# mkdir initrd
[root@kra test]# mv initrd.img initrd/
[root@kra test]# cd initrd/
[root@kra initrd]# cpio -id < initrd.img 
247025 blocks
[root@kra initrd]# rm initrd.img 
[root@kra initrd]# ls
bin  dev  etc  init  lib  lib64  proc  root  run  sbin  shutdown  sys  sysroot  tmp  usr  var

I've opened file /tmp/test/initrd/lib/dracut/modules.d/80anaconda/parse-kickstart and replaced:

class URL(commands.url.F18_Url):
    def dracut_args(self, args, lineno, obj):
        # FIXME: self.proxy, self.noverifyssl
        return "inst.repo=%s" % self.url

with:

class URL(commands.url.F18_Url):
    def dracut_args(self, args, lineno, obj):
        result = "inst.repo=%s" % self.url
        if self.proxy:
                result += "\nproxy=%s" % self.proxy

        return result

Next step is to get ramdisk back to original compressed state:

[root@kra initrd]# pwd
/tmp/test/initrd
[root@kra initrd]# find . | cpio -o -H newc > /tmp/test/initrd.img
247025 blocks
[root@kra initrd]# cd ..
[root@kra test]# ls
initrd  initrd.img
[root@kra test]# xz -C crc32 initrd.img 
[root@kra test]# ls
initrd  initrd.img.xz

I've replaced original ramdisk with the new one:

[root@kra test]# cd /var/lib/tftp/images/centos/7/x86_64/
[root@kra x86_64]# cp initrd.img initrd.img.backup
[root@kra x86_64]# cp /tmp/test/initrd.img.xz initrd.img
cp: overwrite ‘initrd.img’? y

Workaround for MS TMG proxy

At this point I thought it would work, but I was wrong:

dracut-initqueue[561]: curl: (22) The requested URL returned error: 407
dracut-initqueue[561]: Warning: Downloading 'http://merlin.fit.vutbr.cz/mirrors/centos/7.0.1406/os/x86_64/.treeinfo' failed!
dracut-initqueue[561]: Warning: can't find installer mainimage path in .treeinfo
dracut-initqueue[561]: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dracut-initqueue[561]: Dload  Upload   Total   Spent    Left  Speed
  0  4124    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0:--:-- --:--:--     0
dracut-initqueue[561]: curl: (22) The requested URL returned error: 407
dracut-initqueue[561]: Warning: Downloading 'http://merlin.fit.vutbr.cz/mirrors/centos/7.0.1406/os/x86_64/LiveOS/squashfs.img' failed!

I dont't know what's going on there. I try to download a file again and this time I will look inside the file. I'm surprised what I've found there:

.
.
.
Error Code: 407 Proxy Authentication Required. Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. (12209)
.
.
.

I believe that my solution will work for normal proxy. Unfortunatelly I'm behind MS TMG proxy which requires NTLM autentication. Now I need to tell dracut (or whoever calls the curl) that it should use --proxy-ntlm option. So I've extracted ramdisk again, edited /lib/url-lib.sh and compressed it back. Original url-lib.sh code:

.
.
.
### HTTP, HTTPS, FTP #################################################

export CURL_HOME="/run/initramfs/url-lib"
mkdir -p $CURL_HOME
curl_args="--globoff --location --retry 3 --fail --show-error"
getargbool 0 rd.noverifyssl && curl_args="$curl_args --insecure"

proxy=$(getarg proxy=)
[ -n "$proxy" ] && curl_args="$curl_args --proxy $proxy"
.
.
.

New code:

.
.
.
### HTTP, HTTPS, FTP #################################################

export CURL_HOME="/run/initramfs/url-lib"
mkdir -p $CURL_HOME
curl_args="--globoff --location --retry 3 --fail --show-error"
getargbool 0 rd.noverifyssl && curl_args="$curl_args --insecure"

proxy=$(getarg proxy=)
[ -n "$proxy" ] && curl_args="$curl_args --proxy $proxy --proxy-ntlm"
.
.
.

Finally it is working as expected.

Sources

• Setup console for guest domain
• Extract and compress initrd
• SYSLINUX information