Author Archives: Rcart

Packaging apport hooks

Standard

Apport is the software that handles bug crashes in Ubuntu. Whenever you want to report a bug in Ubuntu, apport collects data about that package, attach it to the bug report and send it to Launchpad.

Hooks are scripts written in python that help apport to collect specific data about a package. In this post, I’ll show how to write one, specifically an apport hook for the MPD package which includes the user configuration file. There are some links that you have to read to get a better understanding of what hooks are, and how to write them, those links will be at the end of this post.

If you want to write a hook for a package that doesn’t have a hook but need it, then you can report a bug similar to this one: https://bugs.launchpad.net/bugs/947551

First off, we need to get the source of the package:

bzr branch ubuntu:mpd ; bzr branch mpd fix-missing-hook ; cd fix-missing-hook

And we’re ready to start. In this example, th eMPD user configuration file is stored at ~/.mpdconf. Everytime we write a hook, we *must* ensure that we keep the user data safe. That means that if the configuration file contains passwords, they *must* kept away from the public. Fortunately for this example I found a snippet that replaces passwords strings in files (link at the end). I just had to modified it to fit my needs and add some little things to it. Here’s the apport hook to include MPD user configuration file:

”’
apport package hook for Music Player Deamon

Author: Ronny Cardona

This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 3 of the License, or (at your
option) any later version. See http://www.gnu.org/copyleft/gpl.html for
the full text of the license.
”’

from apport.hookutils import *
import os

# Reference for this function: http://pastebin.ubuntu.com/263396/
def _my_files(report, filename, keyname):
if not (os.path.exists(filename)):
return
key = keyname
report[key] = “”
for line in read_file(filename).split(‘\n’):
try:
if ‘password’ in line.split(‘”‘)[0]:
line = “%s \”@@APPORTREPLACED@@\” ” % (line.split(‘”‘)[0])
report[key] += line + ‘\n’
except IndexError:
continue

def add_info(report):
_my_files(report, os.path.expanduser(‘~/.mpdconf’), ‘UserMpdConf’)

As you can see, the script was initially released under GPL3+, BUT! MPD uses GPL2+ which means that it’s incompatible with the software license. The best solution is to always release the hooks under the same licence as the software the hook was written for, in this example GPL2+.

After the script is ready and copied to /usr/share/apport/package-hooks it *must* be tested to check that it works like expected:

ubuntu-bug mpd

This will call apport which will call then the hooks for the specific package, if you can see in attached files the user configuration file (or whatever file you wanted to include), then the script works like it should do.

So, the script is ready and working, just left to package it. Here comes the interesting part. Apport hooks *must* be included in the debian/ directory, named source_package.py. That means that MPD hook must be named as source_mpd.py and then put it inside the debian/ directory.

Now, we can edit mpd.install file to point out where our script will be allocated after installation. That destination is at /usr/share/apport/package-hooks/. So, we append a line like this one:

debian/source_mpd.py usr/share/apport/package-hooks

That is all we need to do when writing a simple apport hook. We haven’t change code outside debian/ dir, so we’re ready to write the changelog entry:

dch -i

And here we can specify our changes:

mpd (0.16.5-1ubuntu4) precise; urgency=low

* debian/source_mpd.py
– Added apport hook to include user configuration file. (LP: #947551)

– Ronny Cardona Wed, 07 Mar 2012 18:03:12 -0600

Then, commit the change:

bzr commit

And build the package:

bzr builddeb -S
pbuilder-dist precise build ../*.dsc

We install the package and test it. And check the hook destination directory to verify that it was installed in it’s respective directory. We also verify (one more time) if the hook is called when apport is launched =)

Push the branch to LP:

bzr push lp:~rcart/ubuntu/precise/mpd/fix-947551
bzr lp-open

Then propose for merge

That’s all.

Links:
https://wiki.ubuntu.com/Apport
https://wiki.ubuntu.com/Apport/DeveloperHowTo
https://wiki.ubuntu.com/MeetingLogs/devweek0909/ApportPkgHooks
http://pastebin.ubuntu.com/263396/

pbuilder: Building i386 packages on x64 hosts

Standard

A wonderful thing when creating packages with pbuilder would be to have the freedom to build packages for i386 archs on x64 hosts. Well, thats easy using recent versions of pbuilder (I don’t know since when this is possible). The first step is to have those architectures created with pbuilder previously. let’s do it:

pbuilder-dist precise i386 create

When it’s done, create the x64 sistem base (since I’ve a x64 machine, isn’t necessary to specify the architecture to pbuilder, ’cause this creates a system base according to the host arch):

pbuilder-dist precise create

So, when we want to build a package for i386, just do:

pbuilder-dist precise i386 *.dsc

For x64 packages we do it as for any package:

pbuilder-dist precise *.dsc

This is because we have a x64 host, and pbuilder with no arch specified, will build packages for the host’s arch. A really nice tip.

[Pdf] An introduction to Debian packaging

Standard

This is really good text that focus in the basics of packaging for Debian (also applies to Ubuntu). I really would recommend it to get an overview of the process.

Like the author says:

About this tutorial
Goal: tell you what you really need to know about Debian packaging
Tries to keep a reasonable size
No attempt to be complete
You might need to read more documentation

Download at: www.webupd8.org

Patching BitTornado

Standard

Hello,

In this occasion, we’ll patch the bittornado package. (More info about BitTornado here)

Launchpad bug report
Debian bug report

So, here’s what we’ll do:

  1. Creating the patch from scratch.
  2. Apply upstream’s (debian) patch.

Creating the patch from scratch

First of all, we need to understand the bug report; what’s the problem and what’s the solution.

So, as we can read in the bug report, the problem it’s about deprecated sha module. The solution is to replace it with hashlib module. (More information here)

To start, we need to get BitTornado’s branch. But before that, we need to set a working directory and then we’ll be ready to work:
$ bzr branch lp:ubuntu/bittornado

Then, we need to create a branch starting from the original one:
$ bzr branch bittornado bittornado.fix-420387

And now it’s time to get down to work. What we firstly need, is to create a quilt patch (I hope that you know this patches system, BUT, if someone (if there’s someone over here xD) ask for a quick tutorial we’ll gladly give it to you ^^).

It’s recommended that we take a look at the patches naming under debian/patches directory, thus we can get an idea of what name use for our patch. So, the bug report patch’s name looks pretty good, and we’ll use it (following the numbering):
$ quilt new 32_use_hashlib_for_sha.patch

Then, find the python scripts that use the deprecated sha module and add them to our new patch. This can be achieved with the find and grep commands:
$ find . -path ./.pc -prune -o -name '*.py' | xargs grep 'from sha import'

In the above expression, we search in all the python scripts, skipping the files under .pc (applied patches), for the string “from sha import” and we’ll get an output like this:

./btshowmetainfo.py:from sha import *
./btrename.py:from sha import *
./btdownloadgui.py:from sha import sha
./btdownloadheadless.py:from sha import sha
./BitTornado/__init__.py:from sha import sha
./BitTornado/download_bt1.py:from sha import sha
./BitTornado/parsedir.py:from sha import sha
./BitTornado/BT1/Rerequester.py:from sha import sha
./BitTornado/BT1/makemetafile.py:from sha import sha
./BitTornado/BT1/track.py:from sha import sha
./BitTornado/BT1/StorageWrapper.py:from sha import sha
./BitTornado/BTcrypto.py:from sha import sha
./btdownloadcurses.py:from sha import sha

So, we need to extract from the result the files name, and then, add them to the patch, and we can do that with this:
$ find . -path ./.pc -prune -o -name '*.py' | xargs grep 'from sha import' | awk -F ":" '{print $1}' | xargs quilt add

From now on, quilt will be tracking our files for changes, and when we write the changes, it’ll generate the patch.

What we need now, is to edit the files and correct the bug. We have two choices:

  1. Go for the files one by one and correct them
  2. Files have same bug, use an expression for all of them

Well, the easiest way is to use an expression, and to achieve this we’ll use find, grep, awk and sed. I’ve built an expression like this:
$ find . -path ./.pc -prune -o -name '*.py' | xargs grep 'from sha import' | awk -F ":" '{print $1}' | xargs sed -i 's/from sha import sha/from hashlib import sha1 as sha/'

With that, we’ll edit all the bug files, but not the two files that had ‘*’ because they do not fit the sed arguments. So, is not difficult edit them and we will do it manually by changing:

from sha import sha *

To

from hashlib import sha1 as sha

Now, we’re ready to write the changes and get the patch:
$ quilt refresh

After that, we can visualize in the debian/patches directory our patch. That’s all about the patch, it’s time to edit the changelog:
$ dch -i

Here, we’ll increase the ubuntu revision and change the distribution version to natty (the actual development version), also create and entry specifying our changes. This is my changelog entry (thanks to ari-tczew for correcting it):

* debian/patches/32_use_hashlib_for_sha.patch:
– Updated use of deprecated sha module to hashlib. (LP: #420387)

— Ronny Cardona (Rcart) Mon, 24 Jan 2011 17:27:47 -0600

After save the changes, we need to add the quilt patch that we have created to our local branch, clean up the local branch by removing all applied patches, and then just miss commit the changes.
$ bzr add debian/patches/32_use_hashlib_for_sha.patch
$ quilt pop -a
$ bzr commit

We can leave as the commit message, the changelog entry.

Well, until here we have our patch ready to be applied, now we need to create the package’s source code, build the package and test it to confirm that our patch works correctly.

Create the package source code (for a gpg signed package):
$ bzr bd -- -S

And then, depending on our pbuilder configuration, build the package:
$ cd ../ ; pbuilder-dist natty build *.dsc

The package is finished, tested and the changes are correctly confirmed. So, let’s upload it to Launchpad, and afterward, we will request to be Proposed for Merge
$ bzr push lp:~rcart/ubuntu/natty/bittornado/fix-420387
$ bzr lp-open

In firefox (or whatever web browser that we have) click on “Propose for Merge

And we can now wait for a reviewer member to sponsor our branch.

Apply Upstream bug report patch

This patch differs with the above, in that this uses exception handling and it’s safer.

The process is almost the same, difference is that we’ll not edit the files directly, instead we’ll use patch utility to edit them with an existing patch.

Get the branch, create a branch starting from the original one and enter to it.

Then, copy and save the patch code in a file (ending with .patch extension) to the parent working directory (../). Create a new quilt patch (the name of the patch from scratch is okay), find the bug files, add them to quilt patch, and then run:
$ patch -p1 -i ../patchname.patch

The above will correct the bug files by inserting the Debian patch. Now, write the patch:
quilt refresh

Update changelog file:
$ dch -i

Add the new patch to the local branch:
$ bzr add debian/patches/32_use_hashlib_for_sha.patch

Pop out the applied patches:
quilt pop -a

Commit the changes (use the changelog entry):
$ bzr commit

Now, create the source package, build it with pbuilder, test it and confirm that patch works. If all with building was fine, then upload it to LP:
$ bzr push lp:~rcart/ubuntu/natty/bittornado/fix-420387

Open LP and propose it for merge ;)
$ bzr lp-open

That’s all!

DEP3 Patch Tag

Standard

Hello,

DEP3 is a Debian proposal to set embedded information in patches. It’ll help us to get information while reviewing patches; about the patch itself, author(s), origin, upstream related stuff, etc.

This information is used by some tools like Debian’s Patch Tracking System to display it.

The official site and information resource: http://dep.debian.net/deps/dep3/

DEP3 Tag requires some key fields, but not all of them are normally needed (all depends of the patch itself, the bug status and, somebody told me, the project size). Also, we must follow some rules in those key fields.

I’ll show you a simple dep3 tag patch, and I’ll explain the used fields. But first, let’s explain the patch and bug status:

  • There’s a Launchpad Bug report with a patch
  • There’s an upstream bug report with it’s respective patch
  • We will use the upstream patch

So, keep in mind that in an after post, we’ll patch the above bug report package. But this is just a DEP3 Tag patch example:

From: Ronny Cardona (Rcart) <rcart1019@gmail.com>
Description: Updated use of deprecated sha module to hashlib.
Origin: http://bugs.debian.org/593653#17
Bug-Debian: http://bugs.debian.org/593653

Here starts the diff code
…..

From (or Author): The patch author
Description: A non-so long Description of what the patch does, ending with a dot, and using dot to split two paragraph. We can use the Subject field for a briefer description.
Origin: This patch it’s from upstream, so, i must explicitly signalize from where I’ve found it.
Bug-Debian: Like the bug it’s reported in upstream, here should be the Debian bug report address. Otherwise, (using Bug-Ubuntu) should be LP bug report address (a little bit doubtful about this, please confirm it in the website).

This tag must be placed in the patch head, and these are some rules that must be follow:

 

  • Description

field must end with a dot (.)

  • If the patch is not yours, you must specify where comes from, in the Origin tag
  • If it’s possible, put a shorter address
  • There must be an empty new line between the patch tag and the diff code

 

 

So, with these little indications added to our patches, we can distribute them as Debian suggests.

Related links:

Hello World!

Standard

Hi! I’m Rcart, an Ubuntu user and Open Source Software believer.

I’ll document here my personal MOTU journey (too), in a, hopefully, understandable English.

In company with chilicuil, we’ll try to make 2 or 3 post every week, to registry our activities related with packaging and bugs in Ubuntu. We hope that these registries help others with the same intention, Become A Master Of The Universe.