The HyRax Macrocosm » Apt

HowTo: Migrate an Apt-Mirror-generated Ubuntu archive to another mirror source or merge a foreign Apt-Mirror archive into yours

HyRax — Wed, 09 Dec 2009 11:39:11 +0000

So, you’ve gone and created your very own local Ubuntu mirror using Apt-Mirror, and you’ve come across a situation similar to:

You’ve decided to change where you update your Apt-Mirror archive from (eg: you’ve changed ISP’s or feel that another source is more reliable than your current one to update from)
You’re adding another large repository to your Apt-Mirror archive (such as the next version of Ubuntu) and don’t have the quota to download it, so you’re getting a friend to download it for you from their free server using Apt-Mirror (eg: iiNet and Internode customers can access their respective Ubuntu mirrors for free), so you need to be able to merge it with your own Apt-Mirror archive and have it update from your preferred source afterwards.

So how do you do this? Read on.

Migrating your Apt-Mirror archive to update from a new source

This one is really easy. Let’s say you are updating your Ubuntu mirror from Internode, but now want to get your updates from iiNet. To make this happen you need to change the following files:

Your /etc/apt/mirror.list file needs to be updated to point to the new source, and
the Apt-Mirror’s record of downloaded files needs to be updated so that it doesn’t waste time trying to re-download the entire mirror again not realising that it’s already got 99% of all the files already, because Apt-Mirror tracks the files it has downloaded by the source URL and filename, not just the filenames themselves.

So let’s go through this.

Open a terminal load your /etc/apt/mirror.list file into your favourite text editor. In this case I will use the Nano text editor:
```
$ sudo nano /etc/apt/mirror.list
```

In your mirror.list file, The lines for updating Ubuntu 32 and 64-bit versions plus source code from Internode can look similar to this:

# Ubuntu 9.10 Karmic Koala 32-bit
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic main restricted universe multiverse
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-updates main restricted universe multiverse
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-backports main restricted universe multiverse
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-security main restricted universe multiverse
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-proposed main restricted universe multiverse

# Ubuntu 9.10 Karmic Koala 64-bit
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic main restricted universe multiverse
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-updates main restricted universe multiverse
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-backports main restricted universe multiverse
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-security main restricted universe multiverse
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-proposed main restricted universe multiverse

# Ubuntu 9.10 Karmic Koala Source
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-updates main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-backports main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-security main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-proposed main restricted universe multiverse

We need to change the Internode URL to the iiNet URL, so bring up Nano’s search and replace function by pressing CTRL+Backslash (“\”).
.

Now type in the text to replace, in this case:

http://mirror.internode.on.net/pub/ubuntu/ubuntu

Press Enter and you’ll be prompted for the text to replace this with. In this case it’s:
```
http://ftp.iinet.net.au/pub/ubuntu/
```
Press Enter and Nano will find the first occurrence of the Internode text string and highlight it for you. If the selection is correct, press “A” on the keyboard to automatically replace “all” occurrences.
.
Once the update is done, manually go back and visually verify that all the entries were changed correctly.
.
When you’re happy, save your changes by pressing CTRL+X, then “Y” and then Enter.
.
Now we need to update the Apt-Mirror record of downloaded files. First, let’s take a backup of the index in case you stuff up. Type in:
```
$ sudo cp /var/spool/apt-mirror/var/ALL /var/spool/apt-mirror/var/ALL_Backup
```
NOTE: the filename “ALL” must be in uppercase
.
Now let’s bring up the original file into the Nano text editor.
```
$ sudo nano /var/spool/apt-mirror/var/ALL
```
Depending how large your index file is, there may be a brief delay while Nano opens it up. Once it appears, do the same search and replace as you did in steps 3-6 again. Note: If the editor comes up blank, then you have not opened up the index file – check your path spelling in Step 9 and try again.
.
Save your changes by pressing CTRL+X, then “Y” and then Enter.
.
Finally, we need to modify the Apt-Mirror’s cache of downloaded files so that its directory structure matches that of the new source. In the case of iiNet, you’ll notice it’s URL has one less ubuntu word in it compared to Internode’s URL, so we’ll need to move some directories to eliminate the extra ubuntu directory. At the terminal, move the dists and pool directories of the mirrored files one directory back using the commands:
```
$ sudo mv /var/spool/apt-mirror/mirror/mirror.internode.on.net/pub/ubuntu/ubuntu/dists /var/spool/apt-mirror/mirror/mirror.internode.on.net/pub/ubuntu
$ sudo mv /var/spool/apt-mirror/mirror/mirror.internode.on.net/pub/ubuntu/ubuntu/pool /var/spool/apt-mirror/mirror/mirror.internode.on.net/pub/ubuntu
```

Now rename the mirror.internode.on.net directory to become the name of the iiNet server:

$ sudo mv /var/spool/apt-mirror/mirror/mirror.internode.on.net /var/spool/apt-mirror/mirror/ftp.iinet.net.au

The directory structure now matches iiNet’s server and your ALL file is up to date, so now we can test your changes by launching Apt-Mirror. Launch it manually with:
```
$ apt-mirror
```
Watch the output. First Apt-Mirror will download all the repository indexes from the new location and will compare the files presented in those indexes to your local index of downloaded files (the modified ALL file). It will skip all files already listed as being present and will only download new files not listed in your local mirror. You should find Apt-Mirror advises only a small subset of data to download, perhaps only a few megabytes or no more than a gigabyte or two since your last update under the old setup. If you see that Apt-Mirror wants to download some 30GB or more, then you have made an error in changing the URL in the ALL index file or you incorrectly renamed the mirror directories. Press CTRL+C to stop Apt-Mirror, and go check your configuration from Step 10.
```
$ apt-mirror
Downloading 1080 index files using 5 threads...
Begin time: Wed Dec  9 15:59:23 2009
[5]... [4]... [3]... [2]... [1]... [0]...
End time: Wed Dec  9 16:00:45 2009

Proceed indexes: [SSSSSSSSSSPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP]

1.7 GiB will be downloaded into archive.
Downloading 998 archive files using 5 threads...
Begin time: Wed Dec  9 16:02:31 2009
[5]... [4]... [3]... [2]... [1]... [0]...
End time: Wed Dec  9 16:54:15 2009

207.4 MiB in 256 files and 1 directories can be freed.
Run /var/spool/apt-mirror/var/clean.sh for this purpose.
$
```
You’re done! Pat yourself on the back.

Inserting a foreign Apt-Mirror archive into your own archive

This one is slightly more involved, but is not difficult. In the case of a full Ubuntu Mirror, let’s say you were adding an Ubuntu Karmic mirror archive taken from iiNet’s mirror servers into your own local Apt-Mirror archive that featured only Intrepid and Jaunty, both of which you are updating from Internode’s mirror servers. There are some obstacles we need to overcome such as:

Continuing to perform future updates for the Karmic repository from Internode rather than iiNet.
The foreign iiNet Karmic archive contains lots of files that you already have in your own archive – files that are common between all releases of Ubuntu. How do you filter those ones out and only copy the new files?
Finally, how do you update the Apt-Mirror index file with the potentially thousands of new entries from the foreign archive? How do you avoid duplicate lines potentially confusing Apt-Mirror?

Follow these steps:

First ensure that you have the full copy of the foreign Apt-Mirror archive supplied on a suitable storage medium. Aside from the mirror directory itself (usually under /var/spool/apt-mirror/mirror), you must have a copy of its /var/spool/apt-mirror/var/ALL file. It does not matter if the foreign mirror is not completely up to date, as Apt-Mirror will catch up with what is missing when you run the next update.
.
Let’s prepare your local Apt-Mirror installation for grabbing Ubuntu Karmic from our preferred source first. We need to load up the /etc/apt/mirror.list file into your favourite text editor and add the entries relevant to our new repository that we are mirroring. I will use the Nano text editor for this, but you can use any text editor you like:
```
$ sudo nano /etc/apt/mirror.list
```

Now we add the entries relevant to Ubuntu Karmic for Apt-Mirror to use. In this case, I am going to update Ubuntu Karmic from Internode and I will be grabbing both the 32-bit and 64-bit versions plus the source code (reflecting what is already included in the foreign archive on my storage medium, or Apt-Mirror will be doing a LOT of downloading the next time you run it), so I need to add the following entries:

# Ubuntu 9.10 Karmic Koala 32-bit
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic main restricted universe multiverse
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-updates main restricted universe multiverse
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-backports main restricted universe multiverse
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-security main restricted universe multiverse
deb-i386 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-proposed main restricted universe multiverse

# Ubuntu 9.10 Karmic Koala 64-bit
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic main restricted universe multiverse
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-updates main restricted universe multiverse
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-backports main restricted universe multiverse
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-security main restricted universe multiverse
deb-amd64 http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-proposed main restricted universe multiverse

# Ubuntu 9.10 Karmic Koala Source
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-updates main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-backports main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-security main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu karmic-proposed main restricted universe multiverse

Save your changes and exit the editor using CTRL+X, then “Y” and then Enter.
.
Make a backup copy of the foreign mirror’s /var/spool/apt-mirror/var/ALL file, so you can revert to it if you make a mistake. Call the copy something like ALL_Backup.
.
Now open the foreign mirror’s original /var/spool/apt-mirror/var/ALL file into your favourite text editor.
.
Use your text editor’s search and replace function (in Nano, press CTRL + Backslash “\”) to replace the URL of each entry in the foreign mirror’s ALL file to the URL of the mirror you will be performing your future updates from. In the case of changing iiNet URLs to Internode URLs, you would replace any occurrence of the text string:
```
http://ftp.iinet.net.au/pub/ubuntu
```
…with…
```
http://mirror.internode.on.net/pub/ubuntu/ubuntu
```
Once updated, save your changes and close your text editor.
.
Now we need to merge the modified foreign ALL file into the ALL file from your local Apt-Mirror setup. First up, rename the modified foreign ALL file so we don’t confuse it. For this tutorial, I will assume your foreign mirror is supplied on an external USB hard-drive called “myhdd” and is simply a copy of the foreign system’s /var directory in its entirety. The following will rename the file from ALL to ALL_modified in a terminal:
```
$ mv /media/myhdd/var/spool/apt-mirror/var/ALL /media/myhdd/var/spool/apt-mirror/var/ALL_modified
```
Now concatenate the original ALL file and the modified foreign mirror’s ALL_modified file into one new file called ALL_new in your local Apt-Mirror’s var directory. Concatenating alone will result in duplicate lines and we need to sort the file so that any duplicate lines in both the local and foreign ALL files are brought together. We can sort the content of the concatenated files and remove duplicate lines in one hit with:
```
$ sudo cat /var/spool/apt-mirror/var/ALL /media/myhdd/var/spool/apt-mirror/var/ALL_modified | sort | uniq > /var/spool/apt-mirror/var/ALL_new
```
The cat part of the command simply joins the content of /var/spool/apt-mirror/var/ALL and /media/myhdd/var/spool/apt-mirror/var/ALL_modified into one big file, but before it’s written to a physical file, the concatenated data is “piped” using the pipe symbol “|” into the sort command, which sorts the concatenated data into alphabetical order which will group duplicate lines together. But before that resultant output is written anywhere, the sorted data is then piped again into the uniq command which automagically removes all duplicate lines, leaving one unique copy of each line. Finally, we direct the output from uniq using the “>” character into our physical destination file at /var/spool/apt-mirror/var/ALL_new at the end. The sudo command at the start is used simply because only the root and the apt-mirror users can actually write to the /var/spool/apt-mirror/var directory.

Alternatively, we can replace the “| sort | uniq” part with “| sort -u” which does the exact same thing, since the sort command does have it’s own “unique” functionality as well. I’ll leave it up to you which way you’d like to go.
Check your new /var/spool/apt-mirror/var/ALL_new file and you will find it now contains all your local and foreign mirror’s entries in alphabetical order and with no duplicate lines. If you’d like to see how this worked, re-work Step 10 without the sort and uniq commands or the pipe characters and see how it affects the output file. Try adding just the sort or just the uniq command too.
.
Now rename your local mirror’s original ALL file because we’re about to replace it with the new one:
```
$ sudo mv /var/spool/apt-mirror/var/ALL /var/spool/apt-mirror/var/ALL_old
```

Now rename the new ALL_new file to take the place of the old one:

$ sudo mv /var/spool/apt-mirror/var/ALL_new /var/spool/apt-mirror/var/ALL

Right, that’s the index taken care of. We’re nearly done! Now we only have to merge the foreign mirror’s actual files into your local mirror. Once again, for the purposes of this tutorial I’m going to assume you have them stored on an external USB hard-drive called “myhdd” and is a copy of the foreign system’s entire /var directory, so the path to the foreign mirror’s files will be /media/myhdd/var/spool/apt-mirror/mirror – got that? Let’s change to that directory now in a terminal to save us having to type so much:
```
$ cd /media/myhdd/var/spool/apt-mirror/mirror
```

Now, the observant of you may have noticed that Apt-Mirror stores its mirrored files using a directory structure that follows the path of the URL the data is obtained from, so in the case of a mirror from iiNet, there is a directory here called ftp.iinet.net.au. You can see it by using the ls command to list the directory contents:

$ ls -l
-rw-r--r--  1 apt-mirror apt-mirror   198599 2009-12-09 10:19 access.log
-rw-r--r--  1 apt-mirror apt-mirror   544373 2009-12-01 06:45 access.log.1
-rw-r--r--  1 apt-mirror apt-mirror  1863467 2009-11-03 06:44 access.log.2
-rw-r--r--  1 apt-mirror apt-mirror  1865334 2009-10-01 06:28 access.log.3
-rw-r--r--  1 apt-mirror apt-mirror 18152891 2009-09-01 06:42 access.log.4
-rw-r--r--  1 apt-mirror apt-mirror     6135 2009-12-09 06:46 error.log
-rw-r--r--  1 apt-mirror apt-mirror    33898 2009-12-01 06:45 error.log.1
-rw-r--r--  1 apt-mirror apt-mirror   124512 2009-11-03 06:44 error.log.2
-rw-r--r--  1 apt-mirror apt-mirror   554851 2009-10-01 06:28 error.log.3
-rw-r--r--  1 apt-mirror apt-mirror   831227 2009-09-01 06:42 error.log.4
drwxr-xr-x  3 apt-mirror apt-mirror     4096 2008-09-11 02:00 ftp.iinet.net.au
$

We need to modify the foreign directory names and structure to exactly match that of the URL path your local mirror updates from. Starting with the obvious, we need to rename the ftp.iinet.net.au directory to be mirror.internode.on.net with:
```
$ sudo mv ftp.iinet.net.au mirror.internode.on.net
```
Next we need to create an extra subdirectory called “ubuntu” because Internode’s URL path is mirror.internode.on.net/pub/ubuntu/ubuntu/ and iiNet’s path is ftp.iinet.net.au/pub/ubuntu/ only:
```
$ sudo mkdir mirror.internode.on.net/pub/ubuntu/ubuntu
```

Now we need to move the “dists” and “pool” directories under the first “ubuntu” directory to be under the second “ubuntu” directory:

$ sudo mv mirror.internode.on.net/pub/ubuntu/dists mirror.internode.on.net/pub/ubuntu/ubuntu
$ sudo mv mirror.internode.on.net/pub/ubuntu/pool mirror.internode.on.net/pub/ubuntu/ubuntu

With the directory structure and directory names all amended, we are now ready to merge the foreign mirror’s files into your local mirror. We will do this using RSync. This tool traditionally is used to make backups and is indeed used to keep the official worldwide Ubuntu mirrors up to date 1:1 with the master archive, but in our case we are using it to add the “missing” files in the local mirror with the files from the foreign mirror whilst skipping the files that are already present, which means instead of copying around about 60GB worth of data from the foreign mirror, we’ll only copy a percentage of that instead, saving us time and drive space:
```
$ sudo rsync -avz --progress /media/myhdd/var/spool/apt-mirror/mirror/mirror.internode.on.net /var/spool/apt-mirror/mirror/
```
The “–progress” parameter allows you to see which file is being copied over. You may see a large number of directory names whizz past because those directories don’t have any files that are different between your current Ubuntu Intrepid and Jaunty mirror and the Karmic mirror you are merging. Unfortunately rsync does not provide an all-over progress. It only provides a progress of the file it is currently working on. This procress can take several hours to complete depending on how much data needs to be copied and the speed of your storage medium containing the foreign mirror (which if on a USB HDD can take a looooong time).
.
Once RSync has finished, it will give a summary of what was copied. If you were to run the rsync command in Step 16 again, you will see it finish rather quickly because there is no data that has changed or is missing anymore.
.
Now we just quickly ensure that all the merged foreign files belong to the Apt-Mirror user with:
```
$ sudo chown apt-mirror:apt-mirror -R /var/spool/apt-mirror
```
And now we are ready to try a manual update to see if it all worked. If you now execute the Apt-Mirror application manually, you should now see that it reads in the new repository entries you added into your /etc/apt/mirror.list file in Step 3 and will compare the files presented in those indexes to your local index of downloaded files (the newly modified ALL file). It will skip all files already present and will only download new files not present in your local mirror. You should find Apt-Mirror advises only a small subset of data to download, perhaps only a few megabytes or a gigabyte or two since your last update under the old setup and depending on how old the foreign archive was. If you see that Apt-Mirror wants to download about 30GB or more, then you have made an error in changing the URL in the ALL index file or the renaming of mirror directories. Press CTRL+C to stop Apt-Mirror, and go check your configuration from Step 5.
```
$ apt-mirror
Downloading 1080 index files using 5 threads...
Begin time: Wed Dec  9 15:59:23 2009
[5]... [4]... [3]... [2]... [1]... [0]...
End time: Wed Dec  9 16:00:45 2009

Proceed indexes: [SSSSSSSSSSPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP]

1.7 GiB will be downloaded into archive.
Downloading 998 archive files using 5 threads...
Begin time: Wed Dec  9 16:02:31 2009
[5]... [4]... [3]... [2]... [1]... [0]...
End time: Wed Dec  9 16:54:15 2009

207.4 MiB in 256 files and 1 directories can be freed.
Run /var/spool/apt-mirror/var/clean.sh for this purpose.
$
```
If all is good, then pat yourself on the back. You’ve successfully merged the foreign repository and it will now update from your preferred ISP’s mirror from now on.

HowTo: Restore the Windows Master Boot Record (without using a Windows CD) using Ubuntu Karmic.

HyRax — Thu, 29 Oct 2009 12:41:05 +0000

You know how it is – you take a client’s Windows based machine, do a dual-boot installation of Ubuntu (which replaces the Windows Master Boot Record, or MBR, with GRUB and sets up an option to boot Ubuntu or Windows) so the client can evaluate Ubuntu, but then later on for whatever reason, Ubuntu is no longer wanted. It’s removed and you need to restore the system’s ability to natively boot Windows directly without a GRUB menu.

You’re probably thinking “why the hell would anyone want to do that?!”… well, the fact of the matter is you sometimes come across a client who is just too mind-set and refuses to use anything but Windows, so yes – sometimes you need to restore the Windows MBR, but how do you do that when you don’t have a Windows CD handy?

Well, here’s how to do it using nothing but an Ubuntu 9.10 Karmic LiveCD.

It’s a little known fact that the Windows bootloader is nothing special. In fact it contains nothing proprietary to Windows at all. All the Windows bootloader does is simply look for the partition marked as “bootable” or “active” and transfer control of the boot process to it.

And would you know it? The Ubuntu LiveCD has a binary image of a generic open source bootloader that does just that!

Boot your soon-to-be-Windows-only machine using the Ubuntu 9.10 Karmic LiveCD. Doesn’t matter if it’s the 32-bit or 64-bit version.
.
Once booted on the LiveCD, open a terminal by going to the Applications menu and then choose Accessories and then Terminal.
.
Find out what the designation of the Windows drive is (generally it will be the first drive, eg: /dev/sda or /dev/hda). If you are not sure, issue the command:
```
$ sudo fdisk -l
```
…and review the output, looking for your NTFS Windows partition. Make note of the drive that partition resides on (not the partition itself), eg: “/dev/sda”, not “/dev/sda1″.
Now type in the following (remembering to substitute the correct drive device name for your setup in place of “/dev/sda”):
```
$ sudo dd if=/usr/lib/syslinux/mbr.bin of=/dev/sda
```
…which will write the image of a standard MBR contained in the /usr/lib/syslinux directory of the LiveCD environment to the first hard-drive, overwriting GRUB. WARNING: Do NOT use a partition designation, eg: “sda1″ or “sda2″, etc. This will overwrite the start of that partition which will effectively destroy data. The MBR exists at the start of the drive only, so only specify “sda” with no number on the end.
Shutdown and reboot. Windows should now start “natively” without GRUB appearing at all.
.
Normally I’d say “pat yourself on the back” here, but it’s Windows…

HowTo: Make use of Ubuntu PPA repositories

HyRax — Sat, 04 Jul 2009 04:58:07 +0000

What is a PPA repository?

A PPA is a Personal Package Archive hosted by the ubuntu.com servers that contains binaries and/or source related to a project. The project can be anything from a new application to a backport of an existing one. A good example is the easy availability of OpenOffice.org 3.0.1 to Intrepid users before Jaunty came out rather than having to deal with the mess of packages from Sun’s own website.

PPA’s can be wholly personal to you or may be open to the public. In particular it is very useful for providing Ubuntu packaged versions of a given application instead of dealing with tarballs or converting RPM packages.

So how does one make use of a pre-defined PPA and are there any things to be wary about? Read on.

PPA’s are great for getting the latest versions of a given piece of software instead of waiting for the official Ubuntu repository versions to be updated, though you will of course open yourself to any potential bugs in that software. Classic examples include getting Pidgin’s 2.5.8 update with the included new authentication method for Yahoo Messenger, or getting the latest version of the Deluge BitTorrent client/daemon.

In this HowTo, we are going to grab the latest version of Deluge to install on an Ubuntu Jaunty 9.04 system using the unofficial Ubuntu PPA, but this guide should apply to just about any version of Ubuntu or other Debian-based distribution.

NOTE: Ubuntu has now further simplified the PPA process by introducing a new way of adding PPA repositories and the GPG key in one hit from Ubuntu Karmic 9.10 onwards. While you can still use the process outlined below, please see the note at the end of this article for the simplified way.

First up, we need to create a sources.list file for the PPA repository we want to add to our system. In the case of Deluge, the PPA is at https://launchpad.net/~deluge-team/+archive/ppa so go there first.

Under the “Install packages” section is a box with two lines in it:

deb http://ppa.launchpad.net/deluge-team/ppa/ubuntu jaunty main
deb-src http://ppa.launchpad.net/deluge-team/ppa/ubuntu jaunty main

Highlight and copy these two lines to your clipboard.
Now open a terminal and create a new file in your favourite text editor (in this case, GEdit) by typing in:
```
$ sudo gedit /etc/apt/sources.list.d/deluge.list
```
This creates a new file under /etc/apt/sources.list.d called deluge.list. You are presented with a blank page. Paste the content of the clipboard down so you have the two lines you copied earlier.
Save and close the file.
Now, in the terminal, type in:
```
$ sudo apt-get update
```

When the update completes, you should see a warning error at the end similar to the following:

W: GPG error: http://ppa.launchpad.net jaunty Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY C5E6A5ED249AD24C
W: You may want to run apt-get update to correct these problems

What this means is that apt-get has processed all your package lists and found that for the newly added Deluge source list, it does not have a GPG key to authenticate any of the files from it. This doesn’t stop you from installing Deluge or other files from it, but it does prevent Ubuntu from proving whether or not these files are untampered with, so it will pester you with warnings until you can provide it that GPG public key. Providing it is simple. Make note of the hexadecimal value provided after NO_PUBKEY and then type in the following:
```
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C5E6A5ED249AD24C
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv-keys C5E6A5ED249AD24C
gpg: requesting key 249AD24C from hkp server keyserver.ubuntu.com
gpg: key 249AD24C: public key "Launchpad PPA for Deluge Team" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
$
```
This fetches the GPG public key from the Keyserver at ubuntu.com and adds it to your GPG keyring. Now if you run:
```
$ sudo apt-get update
```
…again, you will see no errors output this time, which means you can safely install applications from it now without Apt warning you about being unable to authenticate them.
To prove this, let’s try and install Deluge now:
```
$ sudo apt-get install deluge
```
…and it should install like any other ordinary Ubuntu applications with no fuss, no worries and no error messages.
Pat yourself on the back.

I get an “HTTP fetch” error when I try to import a GPG public key!

If at step 9, you get the following error:

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C5E6A5ED249AD24C
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv-keys C5E6A5ED249AD24C
gpg: requesting key 249AD24C from hkp server keyserver.ubuntu.com
gpgkeys: HTTP fetch error 7: couldn't connect to host
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
$

It is because your firewall is blocking access to the Keyserver. Keyservers use port 11371 to communicate, not port 80 which is the normal HTTP port, so open 11371 as an outbound port on your firewall and re-run the command and it will work fine.

The simplified way of adding PPA’s using Ubuntu Karmic 9.10 or later.

Ubuntu 9.10 introduced a new, simpler way to add PPA’s to your system. Using the above Deluge example, you now only have to type in:

$ sudo add-apt-repository ppa:deluge-team/ppa

…and that’s it. This will do the whole sources.list creation and GPG key for you in one hit. That now simply leaves you to update your package lists with:

$ sudo apt-get update

…and then you can install Deluge with:

$ sudo apt-get install deluge

As you can see, it’s a far simpler method. You can, of course, still use the original method if you prefer.

Hey these PPA things are cool – can I create one of my own?

You certainly can. Refer to the Personal Package Archives for Ubuntu Help Page for everything you need to know, however please do not use a PPA as your own personal off-site backup for personal data. It is intended to help individuals and small groups who develop new software and do not have the resources to host their software for easy distribution by providing them a place where the masses can gain access to their project. To help curb the potential for abuse, PPA’s are limited to 1GB of storage and you are bound by the Ubuntu Community Code of Conduct.

HowTo: Upgrade Ubuntu to the next version without burning a physical CD.

HyRax — Mon, 16 Mar 2009 06:31:46 +0000

As you are probably aware, Ubuntu has a regular six-month release cycle, and if you are someone like me, then you are probably regularly downloading all the Alpha and Beta versions as well as the final releases. The problem with such regularity is that this would use up an awful lot of blank CD’s that you will probably only use a few times before discarding just to install these upgrades.

Fortunately, you don’t have to burn Ubuntu CD ISO images to a physical disc to upgrade your installation to the next version. Here’s how to do it using just the ISO image, or wholly via the Internet with no ISO image or CD at all.

There are three ways to upgrade Ubuntu from one version to the next:

Wholly on-line via the graphical Update Manager application, or
Wholly on-line via command line Apt application, or
The quicker and more easily repeatable method of using an upgrade CD, especially if installing multiple machines (this process optionally will also retrieve additional updates from the Internet released since the CD’s initial release). I will outline all three methods here.

NOTE: You cannot upgrade one CPU architecture to another, ie: you cannot upgrade Ubuntu 8.04 x86 (32-bit Intel/AMD compatible) to Ubuntu 8.10 AMD64 (64-bit Intel/AMD compatible), etc. If you previously used the 32-bit version of Ubuntu and wish to upgrade to the 64-bit version, you will need to do a complete fresh installation from scratch.

Upgrading via the CD ISO image

First up, you will need to download the Alternate Installer CD ISO image of the new version of Ubuntu you want as opposed to the Desktop Live CD version, which cannot do upgrades. All Ubuntu Alternate discs are named in the format of ubuntu-x.xx-alternate-yyyy.iso where x.xx is the release version, eg: 8.10 or 9.04, and yyyy is the target architecture, eg: amd64 for 64-bit Intel-compatible platforms and i386 for 32-bit Intel-compatible platforms. Thus the Ubuntu Intrepid 8.10 64-bit version is called ubuntu-810-alternate-amd64.iso and when it’s eventually released, the Ubuntu Jaunty 9.04 64-bit version will be called ubuntu-9.04-alternate-amd64.iso accordingly.

Once you have the CD image downloaded, make a note of where you downloaded it to, eg: /home/jbloggs/downloads/ubuntu-8.10-alternate-amd64.iso and then jump into a terminal window.

Make sure that you do not have any actual physical CD’s inserted into your CD-ROM drive.

At the $ prompt type in:

$ sudo mount -o loop /home/jbloggs/downloads/ubuntu-8.10-alternate-amd64.iso /media/cdrom

…and hit Enter. If all is well, it should look as though nothing happened, however you will notice a CD icon appear on your desktop simply called “cdrom”.

Now start the upgrade process with the following command:

$ gksu /media/cdrom/cdromupgrade

…and then just follow the prompts. When you reboot after the upgrade, the CD image will no longer be mounted and you can use real CD’s again without any issue.

Upgrading via the Internet only – GUI installation

This method is possibly more convenient, but generally takes longer to do and is not the ideal way to upgrade multiple machines. Using this method, you don’t need the ISO image at all. Everything is handled by the Update Manager.

Open a terminal (or press ALT + F2 to bring up the Run Application dialog) and type in the following:

gksu "update-manager -d"

…and hit Enter. This will launch the familiar Update Manager. If you are not immediately advised of a newer version of Ubuntu to upgrade to, as illustrated below, then click on the “Check” button to update your system’s cache of available updates, then click on the “Upgrade” button that appears at the top.

Before Update Manager is aware of the next distribution version:

And after Update Manager is made aware of the next distribution version by clicking on the “Check” button (note that the Upgrade option will NOT appear if you did not start Update Manager with the “-d” parameter):

Follow the prompts.

Upgrading via the Internet only – Command Line installation

This method is identical to the GUI Update Manager installation, except that it is done wholly via the command line using Apt.

First of all, make sure that Apt is fully up to speed with current updates by jumping into a terminal and type in at the $ prompt:

$ sudo apt-get update && sudo apt-get upgrade

When that has finished, commence the upgrade to the next version of Ubuntu with:

$ sudo do-release-upgrade

Follow the prompts.

That’s it!

HowTo: Uninstall software that makes Ubuntu’s boot process fail

HyRax — Tue, 24 Feb 2009 03:34:16 +0000

Whilst rebuilding a friend’s Medion laptop, in my attempt to get the Wireless LAN adapter working, I set about trying to use ndiswrapper and the Windows drivers. Unfortunately upon rebooting, the system failed to boot, always locking up when the boot process tried to load the Windows driver. It was so bad that I wasn’t even able to boot to a recovery prompt because it still attempts to load the hardware drivers before dropping you into a root shell.

The solution was simple – get rid of ndiswrapper and that will prevent the offending Windows driver loading which I can then delete afterwards, but how do you do this when you can’t even boot to a terminal?

With the assistance of an Ubuntu LiveCD (on USB stick in this case), I was able to remove ndiswrapper without needing to do a complete re-install of the system. Here’s how to do it.

Shutdown your system, insert your Ubuntu LiveCD and turn on your system.
.
Allow the system to boot to the LiveCD desktop and then open a terminal.
.
Determine what your system’s root partition is called by getting a list of available drives and partitions with:
```
$ sudo fdisk -l
```
In my case, this Medion lappy had Windows installed and a couple of other partitions too. The root Linux partition ultimately resided on sda8, so I need to mount this somewhere. First I need a mountpoint:
```
$ mkdir /dev/shm/medion
```
…this creates a directory in the RAM disk called “medion” (but you can call it whatever you want).
.
Now mount the partition to the mountpoint with:
```
$ sudo mount /dev/sda8 /dev/shm/medion
```
Check that we can access the mounted drive with:
```
$ ls -l /dev/shm/medion
```
If it contains the root filesystem of your lappy’s Ubuntu install, then you’ve done well so far. Now we need to change the system’s root filesystem from the LiveCD over to the hard-drive’s root filesystem so we can work on it. Type in:
```
$ sudo chroot /dev/shm/medion
```
This will temporarily make the current session’s root filesystem the one that is on your hard-drive, as though we’d actually booted from it. From here, it’s now a simple case of removing the ndiswrapper application that was causing all my problems (notice that we are at a root prompt designated by the hash symbol):
```
# apt-get remove ndiswrapper-common
```
A few warning messages popped up advising that it couldn’t find the log to write to, but that’s OK – the software still gets removed, and thus will prevent the Windows driver from killing the boot process. Once Apt had finished doing its thing, just type in:
```
# exit
```
…and the root filesystem will revert back to the LiveCD’s root filesystem.
.
Shutdown and reboot as normal without the LiveCD, and viola – the system boots without hanging this time!

As you can see, the ability to change root filesystem can be very useful in trouble-shooting a non-booting installation but also has other uses too, one of the most common being creating custom LiveCD’s by modifying the squashfs filesystem on the LiveCD and burning to a new disc – thus allowing you to add or remove components from the default Ubuntu LiveCD environment.

Unlike

HowTo: Make and simplify sources.list files

HyRax — Fri, 31 Oct 2008 01:01:38 +0000

With Penguin-fans everywhere madly hitting Ubuntu mirrors worldwide today grabbing the Final Release of Ubuntu 8.10 Intrepid Ibex (including myself), people are discovering that using the official major country mirrors can be a bit slow. In the case of Australia, the official mirror is au.archive.ubuntu.com but it’s much slower than many of the other official and unofficial Australian mirrors, especially if your local ISP keeps their own mirror.

It makes sense then, to change your setup to point to a faster mirror rather than rely on the primary.

There are a couple of ways to change your sources, including simply going to System->Administraion->Software Sources and changing the “Download from:” dropbox to another official mirror closer to you. One of my favourite speedy Australian mirrors is Internode. They are registered as an official Australian mirror and automatically appear in this drop-down list, but if your local ISP has their own unofficial mirror, or you have your own mirror at home, then you won’t be able to select it from this list, so you need to do some manual work instead.

Jump into a terminal and type in the following:

$ sudo gedit /etc/apt/sources.list

This will bring up the current sources.list into your text editor. Go through every line that doesn’t have a leading hash and add one, so that every single line is commented out, eg:

This line is not commented
#This line IS commented out because of the leading hash

Do that for all un-commented lines. This will totally disable your sources.list file. “Woah”, you say “Don’t I need that to update my system?” Yes you do, however there is method to my madness.

Save and exit your sources.list file and then type in the following:

$ sudo gedit /etc/apt/sources.list.d/mynewsources.list

This will create a new empty file in your text editor. Now type in the following:

# Ubuntu Intrepid Binaries source.list
deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid main restricted universe multiverse
deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid-updates main restricted universe multiverse
deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid-security main restricted universe multiverse
#deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid-backports main restricted universe multiverse
#deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid-proposed main restricted universe multiverse

# Ubuntu Intrepid Source source.list
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid-updates main restricted universe multiverse
deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid-security main restricted universe multiverse
#deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid-backports main restricted universe multiverse
#deb-src http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid-proposed main restricted universe multiverse

(The Intrepid-Backports and Intrepid-Proposed lines should remain commented out with the leading hash unless you really, really need them)

Save and exit.

Now type in:

$ sudo apt-get update

All of a sudden you see that your machine is pulling all its package lists from Internode’s mirror instead of your old repository! How does this work? And why is this source list so much shorter than the one that Ubuntu already provides?

Apt-get can handle multiple source.list files

By default Apt reads the /etc/apt/sources.list file and normally you would have this setup just for Ubuntu only.

The /etc/apt/sources.list.d/ directory (note the .d on the end of the directory name) can contain any number of other source list files. They can be named anything you like as long as they end in “.list”, so “foobar.list” and “banana.list” are perfectly valid names – Apt will look at all of them. The idea behind being able to make multiple sources lists is that you can setup multiple sources for various software packages, eg; you might have a separate source list to get Virtualbox from Sun’s website, Wine from the WineHQ website, Medibuntu from the Medibuntu website, etc.

Do you have to have separate source lists? Not at all – you can have multiple deb and deb-src lines going to different sites in the one sources.list file if you like. Having multiple separate lists just allows you to keep things clean and tidy and easily allows you to setup other workstations with various repository sources by simply copying a file over instead of editing the sources.list file everytime.

Shortening your sources.list

Now in the above example, all we ultimately did was setup Internode’s Ubuntu repository in a separate source list file but we could have just as easily modified the /etc/apt/sources.list file and replaced all occurences of the official AU mirror’s address with Internode’s address instead, but by showing you how to create your own source list, I’ve also shown you how to simplify the content of the Ubuntu source list too, so you can compare them side by side.

As you know, the deb line tells Apt which address to go to followed by the variant of Ubuntu you are using (in this case “Intrepid”) and then you specify the repository section you are interested in. in the case of Ubuntu, there are four sections in each repository group called “Main”, “Restricted”, “Universe” and “Multiverse”. This could be expressed in your source list like:

deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid main
deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid restricted
deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid universe
deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid multiverse

But this is a lot of unnecessary text. You can cover the lot with one line by specifying the sections together on one line:

deb http://mirror.internode.on.net/pub/ubuntu/ubuntu intrepid main restricted universe multiverse

Repeat this for each repository, eg: intrepid-updates, intrepid-security etc and you end up with something much shorter and easier to read than the default sources.list file.

The main reason the default Ubuntu etc/apt/sources.list file is so big is to explain to the new user what the file is about and how to configure it. A lot of it is just comment lines.

So, now you can apply this to setting up your favourite Ubuntu mirror yourself be it your local ISP’s mirror or even your own mirror at home. Just replace the URL in the source list with the new one.

Unlike