1. $Conf{Ping6Path} is not set

This was a surprise to me, as I use IPv4 for running backups. Since natty, not setting $Conf{Ping6Path} causes BackupPC to report “no ping response” for most hosts. Strangely, this error is irregular. Pings to some hosts always succeed, while others only succeed sometimes. No hosts consistently failed.

The solution is easy. Set this in /etc/backuppc/config:

$Conf{Ping6Path} = ‘/bin/ping6′;

2. Use of qw(…) as parentheses is deprecated

Since Perl 5.14 in Ubuntu 11.10 oneiric, BackupPC reports these errors at startup, in logs, and during command-line operations:

Use of qw(...) as parentheses is deprecated at /usr/share/backuppc/lib/BackupPC/Storage/Text.pm line 302.
Use of qw(...) as parentheses is deprecated at /usr/share/backuppc/lib/BackupPC/Lib.pm line 1425.

To resolve this, download this patch, and apply it:

cd /usr/share/backuppc/
patch -p0 < /tmp/BackupPC-3.2.1-CVE-2011-170886.diff

3. BackupPC displays bogus "Last email sent" messages

BackupPC will send error message emails to the administrator or user of a host (backup target) if it cannot reach the host for a set period of time. The user interface always displays this last message, even if it doesn't exist:

Last email sent to user@example.com was at 2012-05-01 01:23, subject "".

Last year I wrote a patch to resolve this. If no email has ever been sent (last email subject is undefined), it simply doesn't output the "Last email sent" line. My patch has never made it upstream, so I patch it manually after upgrades.

To resolve this, download this patch, and apply it:

patch /usr/share/backuppc/lib/BackupPC/CGI/HostInfo.pm < /tmp/BackupPC_noemail_HostInfo.pm_.patch

Version 1.0 of this guide had a good run. Very little changed between VirtualBox 2.0 and VirtualBox 4.0, or between Ubuntu 8.04 and Ubuntu 11.10, which made it obsolete. However, new screenshots are nice, and changes in Ubuntu 12.04 necessitates a change in dnsmasq.

This updated guide assumes you are using VirtualBox 4.1.14 from a Linux host running Ubuntu 12.04 “Precise Pangolin”. Similar commands can be used on any Debian or Ubuntu release since 2008. You’ll have to adapt some things to use it on RPM- or source-based Linux distributions. Assume all commands are run as root (directly or with sudo).

This guide enables IPv4 networking between VMs and the host OS. IPv6 networking is similar, but dnsmasq does not support DHCP on IPv6. If I write a guide to IPv6 auto-configuration and router advertisements, I will integrate it with these instructions.

Step 1: Create a bridge interface

First, we must create a bridge interface for the VMs. VirtualBox creates a “vboxnet0″ interface by default, but this is not a bridge. Install the bridge utilities:

apt-get install bridge-utils

Now make the bridge start on boot. Add the following to /etc/network/interfaces:

# VirtualBox NAT bridge
auto vnet0
iface vnet0 inet static
        address 172.16.0.1
        netmask 255.255.255.0
        bridge_ports none
        bridge_maxwait 0
        bridge_fd 1
        up iptables -t nat -I POSTROUTING -s 172.16.0.0/24 -j MASQUERADE
        down iptables -t nat -D POSTROUTING -s 172.16.0.0/24 -j MASQUERADE

Either reboot or start it manually:
ifup vnet0

We now have a bridge interface to which VirtualBox can attach virtual machines. That traffic will be NATed to your host’s IP address when the guest OS accesses the Internet. However, the traffic won’t yet route.

Note: if you are already using a firewall such as iptables, shorewall, or ufw, you should remove the two iptables lines above and add equivalent commands to your firewall configuration. Otherwise NAT will probably not function.

Step 2: Enable IP forwarding

Now you must tell the kernel to route traffic. Find the ‘net.ipv4.ip_forward’ line in /etc/sysctl.conf, and uncomment it:

# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1

And load it:
sysctl -p

Step 3: Setup DHCP and DNS for clients

OK, now you can forward and NAT traffic from client VMs. But you still have to configure static IPs in each guest’s OS. Here is where DNSMasq shines. It provides an all-in-one DHCP/DNS server in a small footprint. Install it:
apt-get install dnsmasq

And edit /etc/dnsmasq.conf to include:

interface=vnet0
domain-needed
dhcp-range=172.16.0.2,172.16.0.254,1h

As of Ubuntu 12.04, network manager runs a local dnsmasq resolver on localhost. This will conflict with our dnsmasq, which (despite an interface argument) also tries to run on localhost. Therefore we need to add:

bind-interfaces
except-interface=lo

It is safe to use these options on older releases or if you have disabled the network manager dnsmasq instance.

That’s all you need, but you may want to explicitly define DNS servers and domains for the guests, or set static assignments. Examples:

dhcp-option=option:dns-server,172.16.0.1,8.8.8.8,8.8.4.4
dhcp-option=option:domain-name,example.com
dhcp-host=08:00:27:00:00:02,windows,172.16.0.2
dhcp-host=08:00:27:00:00:03,lucid,172.16.0.3

This defines the host OS and the OpenDNS servers as the DNS servers (instead of passing on whatever your host OS uses), tells all guests they are part of the domain example.com, and defines two static assignments by MAC address.

My final dnsmasq.conf looks like this:

interface=vnet0
bind-interfaces
except-interface=lo
domain-needed
dhcp-range=172.16.0.2,172.16.0.254,1h
dhcp-option=option:dns-server,172.16.0.1,8.8.8.8,8.8.4.4
dhcp-option=option:domain-name,example.com
dhcp-host=08:00:27:00:00:02,windows,172.16.0.2
dhcp-host=08:00:27:00:00:03,lucid,172.16.0.3

Step 4: Set up the virtual machine

1. Start VirtualBox Manager, highlight a VM, and select “Settings” from the toolbar or right-click menu.
2. Choose “Network” on the left.
3. Choose an adaptor, and check “Enable Network Adaptor”.
4. For “Attached to:”, select “Bridged Adaptor”.
5. For “Name:”, select the bridge you created in step 1, vnet0.
6. If you assigned a static DHCP assignment above, expand the advanced settings and set the same MAC address.

Example:

Your virtual machines will now automatically receive an IP address in the 172.16.0.0/24 network, will resolve DNS, will NAT to your host’s external IP address, and can directly address each other.

1. handbrake-gtk and handbrake-cli, taken directly from the official PPA. The oneiric package runs in precise so I haven’t rebuilt it.
2. mysql-query-browser, mysql-admin, and their dependencies, taken directly from oneiric. These have been replaced by MySQL Workbench and dropped from precise. However, Workbench doesn’t support MySQL 4, so if you are still forced to work with that ancient release, you still need mysql-query-browser.
3. loco, from the lucid repo. I like to colourise logs whilst debugging.
4. system-profile, a bash script with cron job that I wrote to take a summary of all installed hardware and running settings which aren’t easily found in the filesystem. Sure, you can run “cat /proc/cpuinfo” while the server is running, but what about when it’s dead? system-profile gives you the same information, provided you made a backup of the filesystem. You do have backups, right?

Highlights:

1. maverick: fceux 2.1.5, backported from precise
2. maverick: get-iplayer 2.8.0, taken directly from precise
3. lucid: logcheck 1.3.14, backported from precise to correct a problem with wildcard globs in the lucid release

After the upgrade, one key on my Logitech diNovo Mini keyboard stopped working. Previously, I had configured the “Media” key in the lower right corner to switch between full screen and windowed mode, so I could access the normal Ubuntu desktop. However, this now launches the Music Library. They keymap for this key changed from <launch_media_center> in Dharma to <launch_media_select> Eden.

This file corrects it.

What you’ll need:

• Calibre, to manage your eBook library. Calibre is awesome. It supports all the formats you are likely to care about and can convert between them, and stores and manages your books in a way you can easily access with a file browser.
• Calibre2opds, which generates OPDS and HTML catalogs from the metadata of Calibre. OPDS is an interchangeable format for describing and sharing books – such as in a personal library.
• A personal server with an HTTP server, such as Apache. You can use a real server or a virtual machine as long as it has a public IP address you can access. Or you could use your desktop PC, accessible only within your private LAN when you’re at home. If you don’t want to run your own server, you could even use Dropbox.
• eBook software with OPDS support, such as FBReader. I read primarily on my Android devices, a Nook Color and an HTC Desire HD, both running community-built ROMs. You can get FBReader on the Market.

How to do it:

Install Calibre

Calibre is easy to install (apt-get install calibre on Ubuntu), and easy to use. I won’t cover it here. Install it, add some eBooks to your library, and close the application.

Install and configure Calibre2opds

Calibre2opds supports both a GUI mode, which I use only to configure it, and a command-line mode, which is handy for updating your library from a script. Unfortunately, the CLI mode has been broken since 2.4-beta4, and isn’t expected to be fixed until the next major release. So I continue to use that release. The developers no longer provide 2.4-beta4, so download it here.

Unpack the zip file somewhere, and open a terminal. Run it for the first time:

sh ./rungui.sh

Configure the following settings on the “Main options” tab:

1. Set “Database folder” to the root of your Calibre library directory.
2. Leave “Destination folder” blank.
3. Uncheck “Copy Catalog to database folder”.

Now press “Save”, then “Generate catalogs”. This can take several minutes to complete if you have a large library.

If you have run Calibre2opds once and saved your settings, you can run it without the GUI using:

sh ./run.sh

Upload to the server

Now upload your library and catalog to your server. If you run your own server you probably know how to do this. If not, see the Dropbox method above.

I use a simple script with rsync to upload ~/Library to a private area of my web server. Here is a very simple script that you can run after you add new books to Calibre.

#!/bin/sh

# if Calibre metadata is newer than the OPDS index, update the catalog
if [ "$HOME/Library/metadata.db" -nt "$HOME/Library/index.xml" ] ; then
   cd ~/calibre2opds/
   sh ./run.sh
else
   echo "Calibre has not been run since last update, skipping calibre2opds."
fi      

rsync -av --delete $HOME/Library/ webuser@example.com:~/public_html/my_personal_library/

Once this is done, you will have a HTML catalog you can browse:

Install and configure FBReader

1. Install FBReader from Android Market and open it.
2. Press the Menu key, then select “Network library
3. Press the Menu key, then select “Add catalog”
4. Select “Enter URL manually”
5. Enter the URL to your library, appended with “_catalog/index.xml”. For example, “http://www.example.com/my_personal_library/_catalog/index.xml”.
6. Press OK twice. Your catalog will now appear in the list.

You can now browse your catalog within FBReader, select any book, and download it, and read it without navigating out again.

If you are an iPhone user, consider MegaReader, available free in the App Store. Configuration is similar.

Updating the library

Once you have all this set up, updating your library is easy. Just add the book in Calibre, run calibre2opds, and sync the files to your server. It’ll be available in FBReader as soon as your access the network library.

I cleared it with the bank, but after that I could no longer buy applications through the Android Market application. When I attempted to do so, the Market required me to check “I agree to the Payments for Android
Market Terms and Conditions”, then failed with the message “Error while retrieving information from server.”

If you don’t care to read the grimy details, perhaps you should skip to the solution.

I reported the issue to Android Market technical support, who replied to me the following day requesting steps to reproduce. I gave them a full bug report:

1. Open Market.
2. Navigate to any app. Press the amount button, such as “£0.10″, button on the upper right.
3. The confirmation page loads, listing my credit card, gmail account, and total. “Accept & Buy” is grey, and the statement “I agree to the Payments for Android Market Terms and Conditions” is displayed, unchecked.
   
4. Check next to “I agree …”. “Accept & Buy” becomes blue.
   
5. Press “Accept & Buy”.
6. The error message “Error while retrieving information from server” appears.
   

What immediately stood out is that I had never previously seen “I agree to the Payments for Android Market Terms and Conditions” appear during a purchase. The Market normally prompts you to accept this the first time you run it after factory reset. My wife’s phone did not have this message.

Also of interest, I could purchase Android apps through the Android Market website, and send them to the device (or install them via the Market). The problem was only with purchases made via the Market app.

I provided all these details, including the fraud problem to Android Market technical support. Unfortunately, I also identified my devices as running community-build Android ROMs. My HTC Desire HD phone runs MIUI, and my Nook Color tablet runs CyanogenMod. That caused another day of delay while technical support insisted that they could not help me because “unauthorized operating systems (custom ROMs) may have limited functionality due to the compatibility of the device.”

I asked technical support to check that my Market account is functioning normally, and they responded with “I’ve taken another look at your account, and can find no reason you shouldn’t be able to access Android Market. Please remember that Android Market may not work well with devices in a modified state as far as operating system or user access privileges. I can only recommend using Android Market with your device in its factory default state.”

I did not expect technical support to support a modified ROM. And before anyone accuses them of refusing to help for that reason, they did not blame the ROM outright and they did try to help. However, they did not solve the problem.

So I spent another day testing. I purged Market settings from /data/data/com.android.vending. I compared to friends’ devices. I arranged to borrow a stock, unmodified phone from a friend and test the following day. And then I realised something – my wife’s phone showed a different credit card number in the purchase screen. We have the same joint account and credit card number, which means that she was able to make purchases with a different credit card.

I logged in to Google Wallet and deleted my credit card, and added it again. The problem was immediately solved. The Market no longer displayed “I agree to the Payments for Android Market Terms and Conditions”, and I’ve been able to buy apps ever since.

Conclusions:

1. The Android Market error message “Error while retrieving information from server” apparently means “Your credit card with Google Wallet once refused a charge”. Or at least indicates a problem with Google Wallet. This error message is totally unhelpful and should be corrected.
2. If you get this error message, try logging in to Google Wallet, remove your credit card, and add it again.
3. If you are running a custom ROM, Android Market technical support may pause because the ROM could be at fault. I appreciate that they did try, and they were polite and courteous the whole time. But it still sucks that they believe the CyanogenMod team is any less professional than HTC. And it especially sucks when you provide professional-quality steps-to-reproduce which clearly indicate the ROM is not the problem.
4. Android Market technical support does not know about this error message or issue with Google Wallet. This is especially disconcerting, as it probably happens all the time and they are exactly the people who should know about it.

I found only a handful of references to this error via Google searches, none with solutions. So I’m writing this all up here, in hopes that it will help other people.

(In case you miss it the first time, try reloading or clicking the button.)

If you’d like to know more about our awesome card, see a previous year’s post. Hey Mark, have you stolen my idea yet? You’re becoming rather tardy, mate.

$happy $holiday, everyone!

This time the upgrade preserved most of my settings. I still had to reinstall my usual software, and fix a few things. Use the following to install my original list of useful software. This should be handy when rc7 or, haha, the “final” 10.03.1 is released.

opkg update
opkg install kmod-fs-btrfs kmod-fs-ext2 kmod-fs-ext3 kmod-fs-ext4 kmod-fs-isofs kmod-fs-reiserfs kmod-fs-vfat kmod-fs-xfs
opkg install kmod-nls-cp1250 kmod-nls-cp1251 kmod-nls-cp437 kmod-nls-cp775 kmod-nls-cp850 kmod-nls-cp852 kmod-nls-cp866 kmod-nls-iso8859-1 kmod-nls-iso8859-13 kmod-nls-iso8859-15 kmod-nls-iso8859-2 kmod-nls-koi8r kmod-nls-utf8
opkg install kmod-usb2 kmod-usb-storage kmod-usb-storage-extras block-hotplug block-mount hotplug2
opkg install --force-overwrite avahi-daemon bwm conntrack-tools fdisk fping iftop ip kmod-ipv6 libnl lft lsof luci-app-ntpc luci-app-openvpn luci-app-upnp luci-ssl miniupnpd mtr net-tools-hostname ngrep nmap ntpclient openssh-client openvpn rsync screen snmpd sshfs tcpdump vim
mkdir /mnt/usbstorage -p

This list no longer includes iptables-utils, which has been merged into iptables and is now simply a dummy package.

After that, I also had to:

1. Restore my OpenVPN certificates to /lib/uci/upload/
2. Delete duplicate configurations in “LED Configuration”
3. Go to System -> Startup and enable the newly-installed daemons

Warning: do not restore to /overlay, even if that is what your BackupPC server is archiving. Restore to the root directory.

I tested wireless throughput on rc4 and rc6 on my Buffalo WZR-HP-G300NH, and also on a Linksys WRT-54-GL with DD-WRT v24-sp1 for comparison. Since rc5 was unstable (except in 802.11b-only mode), I didn’t bother with full tests. All wireless networks had an identical configuration, from ESSID to channel to encryption settings.

I used iperf to transmit 10-second UDP streams from my laptop (on wireless) to zuul (on gigabit Ethernet). When testing between two directly-connected nodes, iperf will scale its burst rate until it finds the maximum link speed. Example: iperf -u -c zuul -b 35M -t 10. The results, averaged over 5 tests each:

Linksys WRT-54-GL with DD-WRT v24-sp1 – 28.5 mbit
Buffalo WZR-HP-G300NH with 10.03.1-rc4 – 23.9 mbit
Buffalo WZR-HP-G300NH with 10.03.1-rc6 – 26.8 mbit

I’m glad to see improved throughput since rc4, but it still doesn’t quite reach the WRT-54-GL, which is surprising. The difference is small enough to ignore. So far, I’ve had no issues with drops in wireless. I’ll update this post after I’ve had time to observe wireless stability.