Yearly Archives: 2016

10 posts

Quickly create a (swap)file on linux

The traditional way for creating a linux swapfile would be using dd to create an empty file e.g.

dd if=/dev/zero of=/swapfile1 bs=1M count=2048

A faster way is to use “fallocate” e.g. (( ))

fallocate -l 2G /swapfile1

Don’t forget the usually procedure for swapfiles:

chmod 0600 /swapfile1
chown root:root /swapfile1
mkswap /swapfile1
swapon /swapfile1
# edit /etc/fstab to add the swapfile during boot

Migration of OpenVZ Container to KVM Guest

This is a short tutorial how to migrate an OpenVZ container to a KVM Guest. Some ideas have been taken from other tutorials(( )) (( )), the other half has been extracted from grml-debootstrap which can generate a KVM guest by using debootstrap.

 # form where to where migrate
 # don't forget the trailing / on the source!
 export SOURCE=/srv/vz/private/xxxxxxx/
 # an empty LVM volume!
 export DEST=/dev/mapper/vg0-myfirstvm

 # create some magic for grub/partion table
 echo 4 66 | /usr/share/grml-debootstrap/bootgrub.mksh -A | sudo dd of=${DEST} conv=notrunc
 sudo dd if=/dev/zero bs=1 conv=notrunc count=64 seek=446 of=${DEST}

 # this is a partition with a partion table on on its own
 sudo kpartx -av ${DEST}
 # create partition
 sudo parted -s "${DEST}" 'mkpart primary ext4 2M -1'
 # create filesystem
 sudo mkfs.ext4 "${DEST}p1"

 # mount it
 sudo mount "${DEST}p1" /mnt

 # not it's time to rsync your files to the destination folder (/mnt)
 sudo rsync -av --numeric-ids --stats --progress ${SOURCE}/ /mnt

 # mount some needed thins inside the KVM guest
 sudo mount -t proc none /mnt/proc
 sudo mount -t sysfs none /mnt/sys
 sudo mount --bind /dev /mnt/dev 

 # fix fstab
 echo `sudo blkid -o export  "${DEST}p1" |grep UUID `  /  ext4   defaults,noatime   0 0   | sudo tee -a /mnt/etc/fstab

 # install kernel + grub + acpi
 # IMPORTANT: during install you are asked to install grub - DO NOT install on any disks/partitions!
 sudo chroot /mnt aptitude install linux-image-amd64 linux-headers-amd64 busybox firmware-linux-free firmware-linux grub-pc acpid acpi-support-base
 # the last command installed and started acpid, so we need to stop it again
 sudo chroot /mnt service acpid stop
 # configure grub
 sudo chroot /mnt/ grub-mkimage -O i386-pc -p '(hd0,msdos1)/boot/grub' -o /tmp/core.img biosdisk part_msdos ext2
 # ATTEENTION: wheezy uses an old gurb and you need to copy it to boot/grub directly!
 sudo cp -rp /mnt/usr/lib/grub/i386-pc /mnt/boot/grub
 sudo dd if=/mnt/tmp/core.img of=$DEST conv=notrunc seek=4
 sudo chroot /mnt  update-grub

 # cleanup
 sudo umount /mnt/proc
 sudo umount /mnt/sys
 sudo umount /mnt/dev
 sudo umount /mnt
 sudo kpartx -d "${DEST}"

 # you may need to remount and rerun 'update-grub'
 # you should check /boot/grub/gurb.cfg within the virtual server that the linux kernel command arguments contain the correct root parameter (with a UUID)!
 # create the KVM VM with virt-install (see my previous post about KVM)

Update 2016/09/20: with some minor fixes and one missing command (copy grub/i386-pc directory)
Update 2016/10/04: I’ve found one more interesting post by ch:

Moving to KVM

Since our beloved OpenVZ virutalisation technology is not moving in a direction we are comfortable with, we are currently evaluation several virtualisation technologies. One the the possible options is KVM. This is not a full tutorial about KVM, there are many good tutorials already, e.g. (( or (( or (( or (( or ((, this is just a collection of some notes which I collected during the evaluation.

General Documentation

RedHat has some good KVM virtualisation documentation available at:
Debian also has some nice documentation:

Creating reproducable VMs for KVM

The easierst way is to use grml-debootstrap to create a new virtual maschine. We already have working netboot environment so we already have added some tuning to grml-debootstrap which makes it even easier.

sudo grml-debootstrap --hostname myfirstvm --vm --target /dev/mapper/vg0-myfirstvm
sudo virt-install --virt-type kvm --name=myfirstmv --vcpu=4 --ram=8192 \
--disk path=/dev/vg0/myfirstvm \
--os-variant=debianwheezy --cpuset=auto --network bridge=br0 --boot hd --vnc

virt-install has many more options(( ))
You still need to setup the /etc/network/interfaces file inside your VM!

Examine VM configuration

virsh dumpxml client1

Serial Console for VM

You can use ‘virsh console clientvm’ to connect to the serial console of the virtual machine. In order to make use of it it, you need to activate the serial console in the VM((

in order to see startup/shtudwon messages:

GRUB_CMDLINE_LINUX='console=tty0 console=ttyS0,19200n8'
GRUB_SERIAL_COMMAND="serial --speed=19200 --unit=0 --word=8 --parity=no --stop=1"

in order to be able to login: (for Debian Jessie with systemd)

systemctl start serial-getty@ttyS0.service
systemctl enable serial-getty@ttyS0.service

Mounting the virutal maschine disk

There are two options available: either with guestfish (install libguestfs-tools ((
)) or with virsh virt-edit (install libguestfs-tools ((

guestfish --rw --add /dev/vg0/myfistvm

virt-edit NameOfGuest /boot/grub/grub.conf

VM Remote Access with VNC

You need to specify a password in order to make it work with MacOSX buildin VNC client, otherwise the client won’t connect! You can add the password by editing the configuration (( ))

 <graphics type='vnc' port='-1' autoport='yes' passwd='mysuperduperpassword'/>

Support VM Shutdown

In order to support restart/shutdown from outside the KVM client, you need to install the following packages((

– acpid
– acpi-support-base

Change VM parameters

Most parameters (e.g. RAM, CPU) cannot be changed during runtime. You can configure a maximal amount and a (lower) current amount. During runtime of a VM you can only allocate until the defined maximal amount. To increase the maximal value you need to shutdown the VM and change the configuration (( )).

Lessions learned: CPU Placement

We have one VM which requires a lot of CPU usage (more cores than on one physical CPU). By default KVM seems to to limit one VM to one physical CPU, we need to adjust the settings directly in the XML to use all cpus(( )) (( )) (( )) (( )) (( )):

<vcpu placement='static' cpuset='0-7'>16</vcpu>
<vcpu placement='static' cpuset='0-15'>16</vcpu>


There are still many things to discover, rethink and consider for moving from OpenVZ to KVM, e.g. better resources planing as resources cannot be changed as easily as in OpenVZ. So there might be some updates to this post in the future. Stay tuned!

Running “backticks” commands on remote servers

Sometimes it’s necessary to run a complex command on a remote server witch also includes some “backticks”. Usually these commands are interpreted by the local shell so you need to use a little trick to force execution on the remote server:

ssh '( echo `echo "This Command is run on the remote server" ` )'

You need to use single quotes combined with brackets to use the backtick on the remote server.

Resetting Supermicor IPMI system

In case the IPMI system on a Supermicro system is unresponsive, but you are still able to log into the main server, you can issue the following command(s) to reset the IPMI:

# load the necessary modules (optional)
sudo modprobe ipmi_si
sudo modprobe ipmi_devintf
# reset the IPMI
sudo ipmitool mc reset cold
# remove all the modules
sudo rmmod ipmi_devintf
sudo rmmod ipmi_si

MacOSX: Manually restoring TimeMaschine Backup

In case you are a CLI junkie as myself and want to restore some files from a time maschine backup manually with the CLI (or using the finder), you will notice that the restored files cannot be changed. The restored files are copied with an ACL on the time machine backup witch prevents changes to those files. You need to remove the ACL from the restored files:
chmod -R -N restored-files/

Cloudflare and Haproxy Lodbalancer

We are currently trying out the cloudflare service to protect one of our company service. In front of this service we are using haproxy as SSL endpoint and loadbalancer. Cloudflare adds a number of custom headers((

 _SERVER["HTTP_CF_RAY"]            17da8155355b0520-SEA
 _SERVER["HTTP_CF_VISITOR"]        {"scheme":"http"}

In order to extract the original client IP in the X_FORWARDD_FOR header, you need to use the following configuration(( in haproxy:

  acl  FROM_CLOUDFLARE src -f /etc/haproxy/cf-ips-v4
  reqidel  ^X-Forwarded-For:.* if ! LOCALHOST
  reqirep  ^CF-Connecting-IP:(.*)$ X-Forwarded-For:\1 if FROM_CLOUDFLARE
  option  forwardfor if-none

Additionally you need to have the cloudlare IPs in the file /etc/haproxy/cf-ips-v4. You can retrieve their IP ranges from:

Some additional links:


Java SSL Certificate Verification Error

If you come across the situation, that your java programs are not able to connect to ssl encrypted services, it might be most likely that the java cacerts keystore is empty or not uptodate. This might also be due to a bug in the java (or ca-certificate-java) package (( In order to fix the issue, you can run:

sudo /var/lib/dpkg/info/ca-certificates-java.postinst configure