Backporting with apt-src

So often with Linux distributions, the choice is between running a bleeding-edge system, or sticking with stable (and sometimes stale) software.  Most of us settle in to a distro that balances both to our liking, but there are times when you just have to have a little newer version of a package than the default repositories offer.  While it’s great to find a backport repo or PPA that offers newer stuff, sometimes that’s not possible.

So for times like that, I’m going to describe a method by which Debian or Ubuntu users can backport their own software using handy little tool called “apt-src”.

Compiling?!?

“Wait a minute…”, I hear you saying, “This ‘src’ doesn’t stand for ‘source’ does it?  Backporting… wait, that means I’ll have to… ”

Oh teh noes!!

Well… yes… and no.  Technically, the computer is going to be compiling.  All you have to do is enter some commands that, ostensibly, are only slightly more complicated than a standard “apt-get install”.  Relax; you don’t need software developer skills here, just follow along and see how simple it is…

Getting set up for backporting

To get your system set up for this, there are a couple things you need to do.  You only need to do this the first time, and after that it’s simple.

Add a source repository

The first step is to add a source repository.  You want to add a repository for the next release of your distro, not the one you’re currently using.  So, for example, if you’re using Debian stable, you want to add a source repository for Debian testing.  If you’re using Ubuntu 12.04, you want to add a repository for Ubuntu 12.10 (quantal).

Bear in mind we’re adding source repositories, not binary repositories.  This is an important difference!

To add the repository, do this:

• Go to /etc/apt/sources.list.d.  Created a file called “source_packages.list” (You’ll need to be root or using sudo to do this).
• If you’re using Debian stable, put this into the file (replace “ftp.debian.org” with your preferred mirror, if desired):

  deb-src http://ftp.debian.org/debian/ testing main

• If you’re using Ubuntu (let’s say 12.04), put this into the file (again, using your preferred mirror):

  deb-src http://us.archive.ubuntu.com/ubuntu quantal main universe multiverse restricted

• Save the file and test it with an apt-get update. If you get an error, double check the contents of the file!

Install the software

Installing apt-src is as simple as:

apt-get install apt-src

Naturally, you can use whatever package manager you prefer, and you’ll need to be root or using sudo to run this.  The apt-src package will pull in all necessary dependencies like the compiler and essential development libraries.

Make a place for your sources

Unlike apt-get, apt-src is going to be run as a regular user and will download source code to the current working directory whenever you run it.  So to keep things tidy, I recommend having a special directory just for your downloaded sources.  This will have some added benefits as we can see later.

I typically use ~/src as my sourcecode directory, so unless you have another preference, create ~/src on your computer.

You should now be set up and ready to backport a package, so let’s see how that’s done.

Backporting a package

The first time you try this, I recommend starting with a small simple package, probably a standalone utility or game.  Backporting something like your desktop environment or office suite is likely to be a hairy mess, so start small and innocuous.  For this example, we’ll backport a little game called “freealchemist”.

Update sources

First, update your sources with the command “apt-src update”.  This is just like apt-get update, and may require root privileges.  It will make sure your list of available sources is fully up-to-date.

Install the source

In a (non-root) terminal, change to your source directory (“cd ~/src”, if you followed my example).  Then we download the source with this command:

apt-src install freealchemist

apt-src will do several things here.  First, it downloads the sourcecode, verifies its signature, and unpacks it to a directory.  Next, it checks for compile dependencies, and if any are needed, it installs them (you may be prompted for root/sudo password at this point).

Compile the source

Once we have all the source and dependencies, it’s time to compile them.  Still in your ~/src directory, type this command:

apt-src build freealchemist

apt-src will configure and build the sourcecode, and package it into a .deb file. If all went well, you should now see a .deb file for freealchemist in your current directory.

Install the package

Now that we have the package, we can install it just like any other .deb file.  From the terminal, you can do this:

sudo dpkg -i freealchemist_*.deb

Assuming your dependencies are met, freealchemist is installed, and everything is good!  (see below if you get an error).

Doing it in one fell swoop

Of course, if you really want to do it all in one command, you can lump download, build, and install in one command line, like so:

apt-src install -b -i freealchemist

Potential difficulties

Of course, it’s not quite all this rosy in some cases.  Software is complicated business, and any given package is usually dependent on many other packages (often specific versions of other packages), and of course there are sometimes good reasons why this software is in the “testing” or “beta” repositories.  So here are some things that can go wrong, and how to deal with them:

Unmet build dependencies

Sometimes building a package requires development libraries that are either unavailable in your release, or unavailable in the required version.  This will show up as an “unmet build dependency” error during the “apt-src install” phase of the process.  There are two things you can do when this happens:

• Give up
• Backport the dependency!

Yes, using the exact same method you’re using the backport the application, you can backport its dependencies.  Keep in mind, this can send you down quite a rabbit hole, because the dependencies themselves may have unmet dependencies, and you may find yourself following a trail deeper and deeper into the bowels of your OS.  All I can say is, when you feel uncertain about what you’re doing, stop, and give it up.  There are times when packages are just to big and deeply-rooted to backport.

Failure to compile

During the “build” part of the process, you may get an error that the compile process failed for some reason.  This can happen for a variety of reasons, sometimes fixable and sometimes not.  The best you can do is check back over the compile output for something that looks like an error (it’s usually 3-6 lines back from the end of the output) and drop it into your favorite search engine.  It may be as simple as a missing dependency that wasn’t listed in the package, or it may require you to alter the source.

As usual, all I can say is dig into it until you don’t feel comfortable messing with it any more.

Can’t install due to missing dependencies

If you’ve successfully compiled and build your .deb file, it still may not install correctly due to missing run-time dependencies (which are distinct from compile-time dependencies).  This can almost always be fixed with the command:

sudo apt-get -f install

…which will attempt to fix any broken packages by installing or removing packages as necessary.  If it just ends up removing your newly-minted package, then chances are that resolving the dependencies were harder than apt-get could handle.  In this case, what you need to do is try to install the missing dependencies manually (they’re listed in the error output from dpkg).

For example, when trying to install my freealchemist package on Debian Squeeze, I got this error:

dpkg: dependency problems prevent configuration of freealchemist:
 freealchemist depends on python-pygame; however:
 Package python-pygame is not installed.

Obviously, in this case, I need to install python-pygame.

In the rare event that you can’t satisfy the dependencies in their correct version, the final option is to (you guessed it) BACKPORT THEM!  The same caveats from build dependencies apply here as well — the dependency chain can get quite long and dig deep into the system, so be cautious and know when to accept the futility of moving further.

Questions and objections

Why not just add a binary repository?

You might well ask, why not just add in a testing/beta binary repository and install the already-compiled binaries?  The reason is because of dependencies.  The binaries in those repositories were compiled against the toolchains and library versions found in the testing/beta releases, and will likely pull in updated dependencies when installed.

Ideally, your goal in backporting is to get a newer version of the application with minimal impact on your underlying system and other applications.  While we discussed the possibility that you might have to backport some dependencies for your application, this will only happen when strictly necessary from a sourcecode standpoint, and won’t happen without explicit action on your part.

One should also reasonably expect that software compiled on your system with your libraries  is going to be more stable than software compiled and tested against a completely newer system.

Why not just download upstream source?

If you’re a little more savvy, maybe you’re asking why not just grab the source from the developer, and configure/make/make install.  Well, there are several advantages to using apt-src:

• It’s been “debianized” — in other words, it’s been patched, configured, and tweaked so that it’s exactly what you’d get if you installed the package from apt-src.  This helps it play a little nicer with other software on your Debian/Ubuntu system.
• No hunting down compile dependencies — unless you’re pretty experienced, hunting down compile dependencies can be a pain when doing a generic compile, especially if the problem is version requirements.
• You get complete integration with your package manager, meaning that you won’t get weird issues doing a release upgrade because things are named correctly.  It also means that it’s easier to roll back, and you won’t have to dig around in /opt or /usr/local to figure out where “make install” chucked all the files.

Does this work with testing/sid/experimental?

This method should theoretically work fine to backport (or, I suppose, “forward-port”) packages between any releases of Debian or Ubuntu (or their derivatives).  The only caveat is that the farther removed you get from your running release, the more likely it is that you’ll run into build or install dependency issues.  For instance, backporting from testing to squeeze is usually reasonable, but trying to backport from experimental to stable, or between Ubuntu LTS releases, is almost always problematic.

If there’s an apt-src install, is there an apt-src remove?  apt-src upgrade?

Yes, and yes.  You could just delete the sources, of course, but using apt-src remove will alert apt-src that the sources have been removed (it keeps track of these things).  Also, apt-src upgrade will check all your installed sources and make sure they’re up-to-date.  You can combine this with the -b and -i switches to go ahead and build/install the downloaded sources.

apt-src also has a “list” function to show a list of installed sources, and a “location” function to show where sources are installed.  More information can be found in the very fine apt-src man page.

What do you think?

How does this method work out for you?  Feel free to comment or ask questions, or suggest a better approach!