Installing Linux on a Lenovo EMC px6-300d

px6_case Today I received my new storage solution / NAS / file server for my living room: a Lenovo EMC px6-300d.

After booting it the first time, it comes up with a web interface. The interface looks nice but is sprinkled with non-free third-party apps like backup solutions, cloud storage and CCTV management.

The tech spec shows, that the main board is a Intel Atom on and the running OS is Linux-Based. Therefore I opened the case (just 3 normal screws) and removed the main board (4 normal screws).

The system is shipped with 2GB of RAM. I changed these with 2x 4GB DDR3 modules.

px6_serial_connector The main board has a proprietary VGA output somewhere on the board. I’ve look around for an adapter but can’t find one. But, while searching for the main boards model number (which I still can’t find) I found four pins labeled SERIAL_B. Using my multimeter I found a pin providing stable 5V (power), a pin providing round about 3V (TX), a pin which is not connected (RX) and a pin connected to ground (GND). I’ve connected a TTL-to-USB-adapter to the pins and opened a terminal. By trying I found the baud rate to be 115200-N-1 and I can see a linux login prompt.

px6_bios As I can’t find the password, I tried to reboot the storage and I can see the BIOS screen and the GRUB screen flashing by. Rebooting again while pressing the DEL-Key (through the serial connection) works well and I can change BIOS settings. The BIOS shows, that the board has a build-in USB hard disk (flash disk) and allows to change the boot order.

Connecting a USB-Stick with ArchLinux installation to the board and choosing to boot from the USB-Stick was the next thing to try. As syslinux comes with a serial configuration of 38400-N-1 I have changed the BIOS settings for the serial console to the same settings. I had to change the redirection of the console to Boot Loader to avoid double redirection. Now, rebooting and pressing F11 during boot allows to choose between the build in disk and the USB-Stick.

px6_usb_stick To keep the serial console alive during installation, the boot record must be edited by pressing the TAB-Key. The following string must be appended:

console=ttyS0,38400n8

Now the arch installer is booting.

Before installing a new Linux to the internal disk (1 GB size), a backup of this disk should be done:

dd if=/dev/sda | gzip > /backup.gz
scp /backup.gz user@1.2.3.4:/target/backup.gz

Now we can install arch linux using the standard installation manual with the following differences:

During the installation of GRUB, the following config located in /etc/default/grub has to be extended:

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0,38400n8"
GRUB_TERMINAL_INPUT="console serial"
GRUB_TERMINAL_OUTPUT="console serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=38400 --word=8 --parity=no --stop=1"

This will configure GRUB to enable serial output (also with 38400n8) for the boot menu and the started kernel.

px6_booting_arch Here we go! A full customizable linux running on the px6. The next step is to control the LCD panel…

References:

Using a sparse file for portage tree

As I’m running gentoo on my notebook I have to do a sync of the portage tree from time to time. The sync process uses rsync and touches a lot of small files, which leads to some performance impacts as I’m using btrfs on this system.

A way to deal with this is using a sparse file containing a filesystem which can handle small files better than btrfs.

At first, the sparse file must be created:

truncate -s 10G /usr/portage.img

This file can now be formated using ext2 with a reduced block size:

mke2fs  -b 1024 -i 2048 -m 0 -O "dir_index" -F /usr/portage.img

To disable file system check complains during boot, the following command can be used:

tune2fs -c 0 -i 0 /usr/portage.img

btrfs would do all its COW magic on the sparse file, too. As this would also impact the performance, the behavior could be disable for the sparse file:

chattr +C /usr/portage.img

Now, it’s time to mount the sparse file and move the data from the existing portage tree to the sparse file (this can be skipped the sparse file is created and mounted before the first sync during installation):

# Move away the existing portage
mv /usr/portage{,.old}
 
# And recreate it
mkdir /usr/portage
 
# Mount the sparse file
mount -o loop,noatime,nodev /usr/portage.img /usr/portage
 
# Copy the data
cp -vR /usr/portage.old/* /usr/portage
 
# Remove the old portage tree
rm -fR /usr/portage.old

To make the portage tree available after boot, the following line must be added to /etc/fstab:

/usr/portage.img    /usr/portage    ext2    noatime,nodev,loop    0 0

That’s it.

Encrypted root on Raspberry Pi

I’m a paranoid guy. Therefore all my disks, connections and mails are encrypted as much as possible.

While building a file storage and backup machine using one of my Raspberry Pis I was playing around with encrypted USB hard disks. While they are working very well on the Pi, I started thinking about encryption of the root partition.

On all my machines the root partition is encrypted and contains a folder with all the key required to decrypt all other hard disks. To do so, I install my system into the encryption container and create a initrd which is able to prompt for the password for the root partition and do the encryption before the kernel is loaded.

As the arch linux image for the Raspberry Pi comes pre-installed and ready to run, the process to create an encrypted root partition for the Pi is a little more complicated.

First step is to download and transfer the latest arch linux image from the interweb:

pv /path/to/archlinux-hf-*img > /dev/mmcblk0; sync

The SD card can be put into the Raspberry Pi the thing can be booted for the first time. As the current image does not use a initrd, the tools to create one are missing and must be installed:

pacman -Syu && pacman -S binutils cryptsetup gzip mkinitcpio

The config file for creating a initrd, /etc/mkinitcpio.conf must be changed to include support for crypted root:

MODULES=""
BINARIES=""
FILES=""
HOOKS="base udev autodetect modconf block filesystems keyboard fsck encrypt"
COMPRESSION="gzip"

Finally, the initrd can be created:

mkinitcpio -g /boot/initrd -v

The next two step is about modifying the boot configuration to use the created initrd while booting and specify the decryption parameters. Therefore the following line must be appended to the file /boot/config.txt in order to load the initrd into the RAM while booting:

initamfs initrd 0x00f00000

The file /boot/cmdline.txt must be changed to use the loaded initrd and to encrypt the root partition during boot:

ipv6.disable=1 selinux=0 plymouth.enable=0 smsc95xx.turbo_mode=Y dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 cryptdevice=/d
ev/mmcblk0p2:root:allow-discards root=/dev/mapper/root rootfstype=ext4 elevator=noop initrd=0x00f00000 rw rootwait

As our kernel and booting system is now able to handle encrypted root partitions, it is time to encrypt the root partition. Therefor the SD card must be removed from the Raspberry Pi and placed back in a PC. There we can create a backup of the data, repartition the SD card, encrypt the root partition and restore the backup:

pv /dev/mmcblk0p5 > rpi.root.img
 
parted /dev/mmcblk0 rm 5
parted /dev/mmcblk0 rm 2
parted /dev/mmcblk0 mkpart primary 95.4M 100%
 
cryptsetup luksFormat /dev/mmcblk0p2
cryptsetup luksOpen /dev/mmcblk0p2 cryptedpi
 
pv rpi.root.img > /dev/mapper/cryptedpi; sync
 
cryptsetup luksClose cryptedpi; sync

Now the SD card can be put back in the Raspberry Pi. While booting it asks for the Password to decrypt the root partition.

Mointing partitions from disk images

disk_image For analysis and repairing it is sometimes required to mount disk images from visualization systems like KVM on a host system.
These disk images contain multiple partitions and there own partition table like a real disk – therefore the can not be mounted directly.

fdisk can be used to list the partitions in the disk image and shows the beginning of each partition:

fdisk -l /path/to/disk.img

giving the following output:

Disk disk.img: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00024628

   Device Boot      Start         End      Blocks   Id  System
disk.img1   *          63    16048934     8024436   83  Linux
disk.img2        16048935    16771859      361462+   5  Extended
disk.img5        16048998    16771859      361431   82  Linux swap / Solaris

The partition start sector, multiplied by the sector size in bytes (512 in the example), can be used as the value for the offset parameter for mount to specify which partition to mount.

The following command can be used to mount the first partition, starting at sector 63:

mount -o loop,offset=$((63 * 512)) disk.img /mnt/disk

Creating a customized Ubuntu Live CD for OpenNMS trainings

ulf1 For the OpenNMS training at Unix User Group Frankfurt, we have the idea to create a custom Ubuntu Live-CD with OpenNMS preinstalled.

Such a CD would allow a quick start of the training and can be used from a CD, an USB stick and in a virtual machine.

The Ubuntu Customization Kit allows easy creation of remastered Ubuntu Live-CDs and is available from the Ubuntu Repository.

At first we have to install uck:

sudo apt-get install uck

As the remastering process requires a existing Ubuntu Live-CD, the current Live-CD must be downloaded from the ubuntu homepage.

We have chosen the classical x86 version, because we cannot guarantee, that all used notebooks are amd64 ones.

After downloading the base Live-CD image, we can start the remastering process.

The process will run in the ~/tmp directory. Ensure that there are no files in this directory, you will miss afterwards.

The following commands will unpack the base and create a chroot environmant, which can be modified. All changes done in this environment are available on our custom Live-CD afterwards:

sudo uck-remaster-clean
sudo uck-remaster-unpack-iso Downloads/ubuntu-12.10-desktop-i386.iso
sudo uck-remaster-unpack-rootfs
sudo uck-remaster-chroot-rootfs

At first, some required tools can should be installed:

apt-get install vim

To install OpenNMS on our Live-CD, The source list must be edited: /etc/apt/sources.list:

deb http://archive.ubuntu.com/ubuntu/ quantal main universe multiverse restricted
deb http://security.ubuntu.com/ubuntu/ quantal-security main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ quantal-updates main universe multiverse restricted

The OpenNMS repository must be added to the source list: /etc/apt/sources.list.d/opennms.list:

deb http://debian.opennms.org stable main
deb-src http://debian.opennms.org stable main

To use the OpenNMS repository, the repositories PGP key must be downloaded and installed:

wget -O - http://debian.opennms.org/OPENNMS-GPG-KEY | sudo apt-key add -

The system should be upgraded completely:

apt-get update
apt-get upgrade

The last requirement for OpenNMS is a running PostgreSQL server. It can be installed from the repository:

apt-get install postgresql

As the OpenNMS installation requires full access to the database. The most security concerns can be ignored, as this CD should be used for trainings only.

The file /etc/postgresql/9.1/main/pg_hba.conf must be edited to make PostgreSQL to trust all access:

 local   all             all                                     trust
 host    all             all             127.0.0.1/32            trust
 host    all             all             ::1/128                 trust

Now, the PostgreSQL server must be restarted for the OpenNMS installation:

service postgresql restart

As PostgreSQL in the default installation comes with the wrong encoding for the OpenNMS database, the encoding must be changed by recreating the template database:

su postgres -c psql
    UPDATE "pg_database" SET "datistemplate" = 'false' WHERE "datname" = 'template1';
    DROP DATABASE "template1";
    CREATE DATABASE "template1" TEMPLATE "template0" ENCODING 'UTF8';
    UPDATE "pg_database" SET "datistemplate" = 'true' WHERE "datname" = 'template1';

Now OpenNMS can be installed:

apt-get install opennms

Some additional installation steps must be done to make OpenNMS running:

/usr/share/opennms/bin/runjava -S /usr/bin/java
/usr/share/opennms/bin/install -dis

To make the Live-CD somewhat more comfortable for the training, some defaults should be changed. At first, a custom wallpaper can be downloaded:

wget http://files.opennms-edu.net/free-software-ulf.jpg -O /usr/share/backgrounds/free-software-ulf.jpg

To make the wallpaper available as default wallpaper, the file /usr/share/gnome-background-properties/ubuntu-wallpapers.xml must be edited:

...
/usr/share/backgrounds/free-software-ulf.jpg
...

And the wallpaper must be specified as default by editing the file /usr/share/glib-2.0/schemas/30_gnome-desktop.gschema.override:

[org.gnome.desktop.background]
picture-uri='file:///usr/share/backgrounds/free-software-ulf.jpg'

To make the OpenNMS web interface, the welcome page of firefox, the file /etc/xul-ext/-homepage.properties must be created:

browser.startup.homepage=http://localhost:8980/opennms

To make the changes effective, the file /etc/xul-ext/ubufox.js must be edited:

...
pref("browser.startup.homepage", "file:/etc/xul-ext/homepage.properties

And the example folder should be used for the user:

rm /etc/skel/examples.desktop

To clean up the list of applications in the quick start bar, the file /usr/share/glib-2.0/schemas/10_gnome-shell.gschema.override mus be edited:

 ...
 [org.gnome.shell]
 favorite-apps=[ 'firefox.desktop', 'evolution.desktop', 'gnome-terminal.desktop', 'nautilus.desktop', 'yelp.desktop' ]

After a schema override has changed, the changes must be compiled using the following command:

glib-compile-schemas /usr/share/glib-2.0/schemas/
<pre>
 
As the Live-CD should not be used to install <em>Ubuntu</em>, the installer should be removed:
<pre lang="bash">
apt-get remove ubiquity

Most users prefer the gnome desktop. This should be installed and the unity desktop should be removed:

apt-get install ubuntu-gnome-desktop
apt-get remove ubuntu-desktop ubuntu-settings
 
/usr/lib/lightdm/lightdm-set-defaults -s gnome

The timezone used on the Live-CD should be configured for the target audience:

dpkg-reconfigure tzdata

To change the default keyboard layout, the file /etc/default/keyboard can be edited:

...
XKBLAYOUT="de"
...

As last step, some cleanup must be down to shrink the size of the resulting image:

apt-get autoremove
apt-get autoclean
 
rm /etc/hosts
rm /etc/resolv.conf
 
rm /var/lib/dbus/machine-id
 
rm /sbin/initctl
dpkg-divert --rename --remove /sbin/initctl
 
rm -rf /tmp/* ~/.bash_history

Now we can exit the chroot environment:

exit

To make a Live-CD from our changed chroot environment, the image must be packed using the following commands:

sudo uck-remaster-pack-rootfs
sudo uck-remaster-pack-iso opennms-training.iso

Now, the Live-CD is finished and ready to boot. The image can be found under ~/tmp/remaster-new-files/opennms-training.isoOne easy way to test the image is qemu:

qemu --cdrom ~/tmp/remaster-new-files/opennms-training.iso

To make additional changes to the chroot environment, the following command must be called again:

sudo uck-remaster-chroot-rootfs

After changing the chroot environment, the image must be recreated using the following commands:

sudo uck-remaster-pack-rootfs
sudo uck-remaster-pack-iso opennms-training.iso

Have a lot of fun!

Raspberry Pi and the TV grabber card

The Raspberry Pi has two ports to get a picture from it. There is a HDMI port which is able to transport a 1080p image to a monitor or a beamer and there is a composite port for the good old TVs. The problem with the HDMI port is HDMI itself – HDMI does not allow any conversion into another signal format. The con for the composite is its bad signal and the low resolution.

Currently I have no monitor which supports HDMI and there is no adapter from HDMI to something else. Therefor I have used an old TV grabber card as a workaround and attached the Pi using the composite port to the composite in of the TV card.

To get the output from the Pi on linux I use mplayer. It allows me to grab the signal from the TV card and display it at a window using the following command:

mplayer tv:// -tv driver=v4l2:device=/dev/video0:input=1:normid=3:fps=29 -nosound

Using a 3 meter cable placed between a whole bunch of other cables, the resulting image is somewhat bad. The white text on a console is displayed in rainbow colors and I can see some flickering and some scan lines all the time.

Restoring a killed partition table

While playing around with my Raspberry PI, I have managed to destroy my partition table of my notebook – ouch!

What happened: I was building a new boot image for my PI on my desktop box as it has much more power than my notebook. To copy the image to the SD card, I want to use my notebook, because my workstation has no SD card slot. As both boxes are using linux, I was going to try nbd – the Network Block Device, which allows to provide a block device over the network.

While I was fighting with the poorly documented configuration syntax, I needed a test device. As it was 4 am in the morning I useded /dev/sda instead of something like /dev/zero – what a bad idea.

After I got the config working and the NBD server running I started to copy the image – without changing the exported device from /dev/sda to the SD card.

Around a trillionth second after I started the copy process, I realized what I have done – I just killed my partition table.
Now I had put my system in a state which will destroy itself on the next reboot. But as long as the system was not shut down it was still usable and I had some time to do backups and try to rescue my system.

The good thing is, my system was still running and nothing had triggered a reload of the partition table. As /proc/partitions shows, the kernel still knew the old partition table. The sys filesystem had some additional details about the loaded partition table:

grep . /sys/block/sda/sda*/{start,size}

That’s all I had needed to restore the partition table. I have used parted as it allows me to enter the complete partition definition at once.
As the command above only showed the start sector and the size, I had to calculate the end sector required by parted:

end = start + size - 1

As I had a extended partition, I had to recalculate the boundaries for that partition, too. The problem with the extended partitions was, the kernel had no clue about these partitions. The solution for this was to use the start of the first and the end of the last logical partition as boundary for the extended one.

After I had calculated all the values needed, I used parted to recreate the partitions:

mkpart primary ${START}s ${END}s

whereas ${START} and ${END} must be replaced with the calculated values. The suffix s tells parted to use sectors as unit.

The last step was to recreate the boot loader – grub in my case:

grub-install /dev/sda

The backup has finished and the partition table was restored (hopefully) correctly – it was time to reboot.

Unexpectedly my system booted without any problems. The filesystem checks completed without any error and any thing worked as expected.

Sorting IP Addresses in BASH

I was working on a little script to test if a bunch of hosts is reachable by ping.

The first version of the script was ready in a minute, but sending pings to hundred hosts in sequence takes some time. Especially if the hosts are down the ping takes a full second until timeout.

To call the pings in parallel is just wrapping the ping command in a ( ... ) &. But this will change the order of the output and the parent script ends before the child scripts do.

To get back the synchronization and having the output sorted I piped the whole output to sort.

As sort uses the alphabetical order to sort the IP Addresses the order is not as expected. But sort has a flag -V to sort versions – which are technically the same as IP addresses: dot separated numbers.

Here comes the example:

for i in $(seq 1 10); do
  (
    ip=$(printf "10.0.0.%s" "$i")
 
    if ping -W 1 -c 1 $ip > /dev/null; then
      printf "%s	e[32m%se[0m
" "$ip" "up"
    else
      printf "%s	e[31m%se[0m
" "$ip" "down"
    fi
  ) &
done | sort -V

Btw.: The watch command supports a --color flag which allows to run the script in a loop.