Fast Parallel Downloading/Upgrading for Ubuntu using aptitude/apt-get/aria2c

I recently moved to Ho Chi Minh City (HCMC), Vietnam from the US of A. There’s a lot different between the two countries – the people, the culture. I am throughly enjoying my stay in this city, the social life is rocking, the food is amazing and the people are very nice. However, there are a few things I do miss about the US of A and one of those things being the blazing fast Internet speeds.

A couple days back, I decided to give KDE 4.4 a go. I knew it won’t be available in the standard Ubuntu repositories anytime soon, and hence googled for a PPA which I could download it from. I came across this tutorial which had step-by-step instructions for upgrading to KDE 4.4 on Ubuntu Karmic. I followed the tutorial and added the Kubuntu Backports PPA and issued the standard ‘apt-get update; apt-get dist-upgrade’ command to start the upgrade. The apt-get command let me know that there are 170 packages to be upgraded. I gave it the go-ahead and I could see the packages being downloaded one by one; I couldn’t wait to try out the new version of KDE, I read so many rave reviews about. To my dismay, even after half an hour only 30 odd packages were done and it was already 1 a.m.. Finally at about 1:30 a.m., I gave up and went off to sleep, letting the download take place through the night and made a mental note that I needed to find a more efficient way of upgrading.

Today, I was googling around to see if I could find a solution to my internet/upgrade woes and came across this blog post. The solution is to dump the URIs of all the packages that need to be upgraded ( apt-get -y –print-uris install $x | egrep -o -e “http://[^\']+”) and then download them 3 at a time using wget (xargs -l3 -P5 wget -nv) and finally issue the upgrade command. That definitely did speed up the upgrade process. However I remembered that Archlinux has a tool called PowerPill which uses aria2c to speed up downloads by fragmenting them. I wondered if I could use aria2c instead of wget to speed up my downloads even more.

I downloaded aria2 and fired the aria2 man page to figure out what the wget equivalent command would be.

sudo aptitude install aria2

Its usage turned out to be very simple and I used the follow command to download my updates:

sudo apt-get -y --print-uris upgrade | egrep -o -e "http://[^\']+" | sudo aria2c -c -d /var/cache/apt/archives -i -; sudo aptitude safe-upgrade

aria2c downloads concurrently from 5 URLs and uses 5 fragments per file at a time by default. The settings can be tweaked using -j for the no of concurrent files to be downloaded and -s for the no of fragments.

I noticed a significant increase in my download speeds by using aria2c instead of wget.

Please leave your comments to let me know if you know of better ways to boost upgrade speeds.