subversion

bzr-svn has always been using the standard Python bindings that were provided with Subversion itself. Unfortunately, I had to fix some issues in these bindings since they were incomplete or broken and thus bzr-svn has always depended on a development snapshot of Subversion.

As of today, bzr-svn is using its own Python bindings for Subversion.

There were several reasons for switching to our own bindings:

• There are no requirements for backwards compatibility within bzr-svn. This means the API can be made sane without worrying about the mess it was in the past and users who still rely on that.
• Deployment. It took 2 years for my fixes to the Subversion Python bindings to be part of a release. It'll be even longer before Subversion 1.5 makes it into most available distributions. That makes it very hard to just download and install bzr-svn.
• They're in plain C, not SWIG. SWIG has a big advantage for the Subversion folks since it can generate python, ruby, java or tcl bindings all at once without a lot of overhead per language. However, it has issues as well that make it a bad choice for bzr-svn.
  • It generates inefficient code - it generates proxy classes that add more layers in the stack
  • Bindings tend to be very much like the C API rather than "Pythonic". To make them more Pythonic, you need tons of typemaps. For example, the Python bindings in bzr-svn provide an iterator when browsing the revision history rather than a callback as C and the SWIG bindings do.
  • Hard to write - personally at least, I write bindings in C faster than in SWIG
  • Adds an extra dependency to the build process. Several people had trouble building Subversion on their Mac machines because they didn't have the right version of SWIG available.

Since all of the patches that bzr-svn depended on previously were in the Python bindings for Subversion, it is now possible to use bzr-svn with any version of Subversion newer than 1.4.0. Of course, you do need to have the development headers installed as well.

cp: Kathleen Edwards - Independent Thief

One of the things that I've always missed in DVCS is the ability to refuse commits in a branch that's shared by multiple people based on a test suite run. Sure, it's possible to have a pre-commit hook - but that would mean that you'd have to wait for the full test suite to run until the commit finishes.
With the time it takes to run the Samba testsuite, this is not really an option.

One of the things that would work is to have everybody work in a separate branch and then have some sort of tool that merges those revisions from everybody's personal branches that worked ok. However, to my knowledge, there is no such tool for Subversion.

Bazaar uses a tool called PQM (Patch Queue Manager). PQM usually controls the main branch (for example for Bazaar, it controls bzr.dev), and waits for GPG-signed requests to merge a specific revision into that main branch. Before accepting such a revision, it will try to run the testsuite to make sure it passes. This guarantees that the main branch never contains broken code (as far as can be indicated by the testsuite).

Now that bzr-svn supports true push, it is possible to actually use a PQM with a Subversion branch. I've tried it on a smaller branch last week, and am now looking into using this for my Samba work.

I'm currently doing a bit of sightseeing in London, after attending the
Bazaar sprint at the Canonical office. It was a good sprint, and a bit
different from the previous ones in that there was only a limit amount of
actual coding involved. The view from the Canonical office is magnificent, so we were even able to do some sightseeing while working; (-:

Bazaars' focus has previously mainly been on correctness and features. The
first has always been one of our strengths, and we're in pretty good shape
regarding the second. Performance has been one of the main complaints from
users about Bazaar and so we've recently tried to get better in that area.

Since 0.12, we have already tried to optimise some of the common code paths and
some people have been working on a high performance smart server (to speed up
remote operations).

During the first two-and-a-half days of the sprint, we have analysed 20
of the most common use cases with Bazaar and determined what complexity they
should ideally require to be able to work. After this analysis, we looked at
ways to change our data structures to reach these goals.

I've been mainly a spectator during the latter parts of these discussion, but
they were interesting to follow.

One of the things I worked on were support for true push in bzr-svn. This was
one of the bugs that bit a lot of users of bzr-svn. The upcoming bzr-svn 0.4
now supports true push as well as commits in heavyweight checkouts. I hope to
release 0.4 after adding nested tree and ignores support so that I don't
have to upgrade the mappings again.

Ah well, time for some more sightseeing and getting back to the reason I'm working on all this: Samba!

As of 0.15, Bazaar will have initial support for nested trees. Nested trees basically allow you to add a 'magic' directory to your Bazaar branch that in itself is another Bazaar branch. A very good use case of this is
if you have a project for which you need to include some library (maybe popt?). Nested trees allow you to add a reference to the upstream popt branch, avoiding the need to synchronize every time upstream fixes a bug.

Subversion has similar functionality, in their terminology called 'externals'. Externals are very easy to use - all you need to do is set the svn:externals property.

I've done some early work on supporting externals in bzr-svn and mapping them to nested trees. At the moment, bzr-svn is able to track nested trees that are being added. Removes and changes to existing nested tree locations are still on my todo list, but will definitely be supported in bzr-svn 0.4.0.

The Bazaar PQM has just merged my first Bazaar commits. Hopefully there are more to follow in the near future. I also managed to fixed some more annoying bzr-svn bugs today, 0.2 is nearer than ever now.

Currently playing: Transatlantic - Bridges Across Forever - Duel With The Devil

I discovered Last.fm's Recommended Events feature a few days ago. It lists all events of artists you've listened to, within a specified range from where you live. No more searching through long lists of concerts or being late for tickets. The only remaining thing I'd like to see is an iCal feed.

Thanks to one of the maintainers of Subversion in Debian, Peter Samuelson, for backporting some of my fixes to the Python bindings in 1.4 to Debian's Python Subversion package. This means bzr-svn will hopefully be usable in Debian etch.

In reaction to the Novell<->Microsoft deal that creates first- and second-class citizens in the Free Software world, the Samba team has requested Novell to reconsider.

In somewhat more personal news, I found out yesterday I'm allergic to house dust mite and have a mild form of hayfever. Apparently I have to look forward to spending more time cleaning :-/

At last, I've started to use bzr-svn in production. I'm now working on Samba in bzr, pulling from and pushing to Subversion branches as if they were native Bazaar branches.

It seems to work quite well. Bazaar can be slow dealing with large history, but that appears to be a temporary issue rather than a design flaw.

I've also published conversions of the two main Samba branches on the web at http://people.samba.org/bzr/jelmer/samba/.

Now, time to get the Subversion Python fixes into all the distro packages! :-)

The example below demonstrates just how well foreign branch support in Bazaar is progressing. Here's how you can actually push Mercurial branches in Subversion:

charis:~/bzr/hg-bzr-svn% svnadmin create svn-repos
charis:~/bzr/hg-bzr-svn% mkdir hg-branch; cd hg-branch
charis:~/bzr/hg-bzr-svn/hg-branch% hg init; echo data > afile; hg add afile
charis:~/bzr/hg-bzr-svn/hg-branch% hg ci -m "Add afile"
charis:~/bzr/hg-bzr-svn/hg-branch% cd ..
charis:~/bzr/hg-bzr-svn% bzr branch svn+file://`pwd`/svn-repos bzr-co
Branched 0 revision(s).
charis:~/bzr/hg-bzr-svn% cd bzr-co
charis:~/bzr/hg-bzr-svn/bzr-co% bzr pull ../hg-branch
All changes applied successfully.
1 revision(s) pulled.
charis:~/bzr/hg-bzr-svn/bzr-co% bzr push svn+file://`pwd`/../svn-repos
This transport does not update the working tree of: 
svn+file:///home/jelmer/bzr/hg-bzr-svn/bzr-co/../svn-repos
1 revision(s) pushed.
charis:~/bzr/hg-bzr-svn/bzr-co% cd ..
charis:~/bzr/hg-bzr-svn% svn co file://`pwd`/svn-repos svn-co
A    svn-co/afile
 U   svn-co
Checked out revision 1.
charis:~/bzr/hg-bzr-svn% cd svn-co
charis:~/bzr/hg-bzr-svn/svn-co% svn pl
Properties on '.':
  bzr:merge
charis:~/bzr/hg-bzr-svn/svn-co% svn pg bzr:merge
hg:ef40d1bed1d1a1739457191eb979f10bfec793a2
charis:~/bzr/hg-bzr-svn/svn-co% svn log -v
------------------------------------------------------------------------
r1 | jelmer | 2006-07-10 21:33:49 +0200 (Mon, 10 Jul 2006) | 1 line
Changed paths:
   M /
   A /afile

Add afile
------------------------------------------------------------------------

charis:~% bzr branch svn+ssh://svn.samba.org/home/svn/smb-build/trunk smb-build
Branched 50 revision(s).
charis:~% cd smb-build
charis:~/smb-build% ls
README        autogen.sh  bla    cflags.map    config.sub    configure.list  include
acinclude.m4  bin         build  config.guess  configure.in  foobar          install-sh
charis:~/smb-build% echo This line was added from Bazaar >> README
charis:~/smb-build% bzr ci -m "Update README"
modified README

You need a passphrase to unlock the secret key for
user: "Jelmer Vernooij <jelmer@samba.org>"
1024-bit DSA key, ID 1EEF5276, created 2001-10-30

Committed revision 51.
charis:~/smb-build% bzr push svn+ssh://svn.samba.org/home/svn/smb-build/trunk
This transport does not update the working tree of: 
svn+ssh://svn.samba.org/home/svn/smb-build/trunk
1 revision(s) pushed.
charis:~/smb-build% bzr pull
Using saved location: svn+ssh://svn.samba.org/home/svn/smb-build/trunk
All changes applied successfully.
0 revision(s) pulled.
charis:~/smb-build% svn log -v -r HEAD svn://svn.samba.org/smb-build/trunk
------------------------------------------------------------------------
r61 | jelmer | 2006-07-04 21:37:52 +0200 (Tue, 04 Jul 2006) | 1 line
Changed paths:
   M /trunk
   M /trunk/README

Update README
------------------------------------------------------------------------
charis:~/smb-build% svn diff -r 60:61 svn://svn.samba.org/smb-build/trunk/README
Index: README
===================================================================
--- README      (revision 60)
+++ README      (revision 61)
@@ -1 +1,2 @@
 Bazaar rocks
+This line was added from Bazaar

Or view the screenshot on

Left to do before public announcement:
* Fix nasty bug when branching off Samba (revno 11098 gives an error because it can't find a file parent...)
* Get workingtree fixes integrated into bzr.dev
* Get Ubuntu bug #51304 fixed (integrate Subversion fixes into package).

Now that commits, merges, renames and working trees work in bzrsvn, I'm looking at the next items on the list:

* push into Svn repositories
* creating working copies from Bazaar
* performance improvements

For the plugin, I've had to make quite a few changes to the Python bindings to Subversion. I recently became partial committer to Subversion.
My patches are listed by the contribulyzer and commits in the archives. It's been fun working with the Subversion developers.

After a few days hard work, I finally got commits working! Mandatory screenshots are up:

Time to get some sleep, it's been a long night.

Spent most of this week working on version control systems. I now have (partial) commit access to Subversions Subversion repository.

As more people on samba.org are using bzr now, Jerry has worked on setting up a public place for publishing Bazaar branches. Mine are in http://people.samba.org/bzr/jelmer/