Stationary Traveller - soc

Samba Summer of Code

nospam@example.com (Jelmer Vernooij) — Thu, 10 Jun 2010 16:56:15 +0200

As I have done in previous years, I am again participating in the Google Summer of Code as mentor for the Samba project.

Last year I Andrew and I co-mentored three students with mixed results. In the end we had to drop one of our students but the other two did well. I've only taken on one student this year for various reasons.

The amount of time required to mentor a student varies wildly depending on the student and is hard to predict based on their application. Some students seem to require quite a lot of mentoring while others are self-motivated and self-learning. This has not just been my experience, I've heard similar stories from fellow mentors on other projects.

Last summer Ricardo worked on SWAT for Samba 4 and he is still actively working on the project, even after the Summer of Code has finished. I hope to find the time to package SWAT in time for Debian Squeeze. At the moment SWAT just supports managing shares but Ricardo is working on user management.

In 2009 Calin worked on the GTK+ frontends for Samba, in particular changing them to be Python-based rather than C-based. This year his work is going to be continued by Sergio, hopefully with the some user-ready tools as the end result.

cp: Gazpacho - 117

US: Observations

nospam@example.com (Jelmer Vernooij) — Fri, 30 Oct 2009 17:54:00 +0100

These past few days in the US were a bit of a rollercoaster. Some random observations:

The mentor summit was very nice and well organized (or rather: well disorganized). Lots of awesome people around from a wide variety of projects and nationalities.
"Next Generation VCS" seems to be an alias for git these days in the minds of most people.
I didn't write a single line of code in almost a week, something that is very rare.
Driving an automatic gives you two spare limbs to use for other things. What those other things are, I have yet to figure out.
Is the fact that your kid was student of the month or the fact that you own two cats and a dog really something that belongs on a bumper sticker?
Gas is cheap (compared to Europe). I drove 300 miles on a $30 tank.
The malls in the Bay Area are some of the biggest I've ever seen, but strangely enough they seem to lack both book- and cd-stores.
Visiting Fry's continues to have a significant effect on the contents of my wallet.
It is legal to turn right on a red traffic sign in California unless otherwise indicated. It took me a while to realize this until people repeatedly started honking behind me...
The waiver I had to sign to be able to skydive in California was scary. I can cope with my operating system coming without even the implied warranty of merchantability or fitness for a particular purpose, but my parachute?
I stopped pretending to have any regularity in my sleeping habits. 6 AM flights? It seemed like a good idea at the time.

Summer of Code 2009

nospam@example.com (Jelmer Vernooij) — Fri, 11 Sep 2009 16:22:06 +0200

For this years (the fifth?) Summer of Code, I participated once again as a mentor for the Samba and OpenChange projects.

Samba was assigned four slots this year: one was a CIFSFS project mentored by Steve French and the other three were Python projects related to Samba 4, co-mentored by Andrew and me. Our students did very well this year, although we unfortunately had to drop one after the mid-term evaluations due to lack of effort. Nonetheless, we're very happy with the results of the other two projects:

Calin Crisan (France) converted the rest of the applications in SambaGtk to Python, and worked on a GTK+ user manager for Samba and Windows. With his improvements, it is now possible to edit registries, manage users, inspect the endpoint mapper, plan tasks and manage services on a remote Windows machine using a GTK+ application on a Linux workstation.

Ricardo Velhote (Portugal) designed and implemented a new version of SWAT - the Samba Web Administration Tool. Unlike the old SWAT, his implementation is more than just a simple web-based editor for smb.conf. As we were expecting at the start of the Summer of Code, not all of the functionality could be implemented properly in a couple of months, not while getting the design and infrastructure right. With a basic version working, we now hope the remaining subsystems can be contributed with help from the community.

I'm planning to merge Calin's improvements to Samba-Gtk into the mainline in the next month or so. SWAT is a standalone application and will continue to live as a separate project, while being a part of the Samba ecosystem. Congratulations to both Calin and Ricardo on their achievements!

New job

nospam@example.com (Jelmer Vernooij) — Sat, 02 Sep 2006 22:14:34 +0200

Google's Summer of Code was again a pleasure to participate in! I'm quite pleased with the overall results and have enjoyed working on the CUPS tree. Mike, main author of CUPS, has merged my Kerberos branch into trunk a couple of days ago. It should be part of the upcoming 1.3 series. My aim is to stick around and maybe work on SPNEGO/NTLMSSP support.

As of recently I have started working part-time for SerNet. I'll be working on random Samba bits for them, all free software. The first thing I'm working on is backporting the excellent libndr (one of the core parts of DCE/RPC) subsystem from Samba 4 to Samba 3.

The university year has also started, so my activity in free software will probably decrease a bit in a few days.

Back to work

nospam@example.com (Jelmer Vernooij) — Wed, 16 Aug 2006 02:12:22 +0200

After cycling and playing cards in the Vosges (France) I'm back working. After browsing through 10 days of mail, I'm now making significant progress on CUPS-krb5 again.

Currently Playing: Rory Gallagher - Bad Penny

Applications for Summer of Code and Debian

nospam@example.com (Jelmer Vernooij) — Mon, 29 May 2006 21:43:22 +0200

I'll be participating in the Google Summer of Code again this year. My project this time is Kerberos integration in CUPS.

My Debian application also finally got moving, after a little less than half a year. Hopefully I can get through this quickly.

Currently Playing: Counting Crows - Goodnight Elizabeth

SoC Deadline

nospam@example.com (Jelmer Vernooij) — Wed, 24 May 2006 01:27:23 +0200

It's almost 2 AM here, in other words, 17:00 PM in California - just a few more minutes before the list of SoC applicants will apparently be published. Hopefully one of my applications will be accepted; the quality of applications appears to be better overall than last year, so it'll be tough...

Currently Playing: Stevie Ray Vaughn - Crossfire

Results published by Google

nospam@example.com (Jelmer Vernooij) — Tue, 25 Oct 2005 21:25:03 +0200

Google has posted a summary of the results of the Summer of Code on the Company Blog. There's also a link in there to a cool map with all the participants and mentors.

[Update 23:13] Slashdot now also has a story.

SoC Deadline

nospam@example.com (Jelmer Vernooij) — Thu, 01 Sep 2005 23:49:44 +0200

So, just a few more minutes until the deadline of the Google Summer of Code, 2005! I got pretty much all that I wanted in there done, except for the ldb_map module, which is still a bit dodgy.

It sure was nice and interesting to do participate! The other participants that were working on Samba for SoC have produced some great code and it looks like they're going to stick around. Especially Brad's work on DSSYNC looks promising. It'd certainly recommend anyone to participate if Google is holding another SoC again sometime in the future!

Looking back, I guess it could've been interesting to work on a different project then Samba such as mono (I do actually actively follow mono development, I just don't contribute at the moment), althought what I did now was probably more productive and allowed me to spend some time on other Samba things as well :-)

I just committed the following file 'STATUS' to the tree:

--- Samba3 -> Samba4 Upgrade ---
(C) 2005 Jelmer Vernooij <jelmer@samba.org>
Published under the GNU GPL

Sponsored by the Google Summer of Code program (http://code.google.com/summerofcode.html)
Mentored by Andrew Bartlett <abartlet@samba.org>
Thanks!

Done:
- Reading wins.dat
- Reading registry.tdb
- Reading passdb.tdb
- Reading account_policy.tdb
- Reading group_mappings.tdb
- Reading winbindd_idmap.tdb
- Reading share_info.tdb
- Reading secrets.tdb
- Reading smbpasswd
- Reading + writing (generic) smb.conf files
- Testsuite for read support mentioned above
- Console utility for dumping Samba information
- Import user accounts in Samba4
- Import groups in Samba4
- Import secrets in Samba4
- Import WINS data in Samba4
- Dump idmap data to LDB
- Import registry keys/values in Samba4
- Import account policies in Samba4
- Testsuite for upgrade
- Console utility from upgrading from Samba3 -> Samba4
- SWAT (Web interface) support for upgrading from Samba3 -> Samba4
- LDB generic mapping module
- (Experimental) Samba4 LDB <-> Samba3 LDAP mapping module based on LDB generic mapping module
- Testsuite for Samba4 LDB <-> Samba3 LDAP mapping module

Source files:
source/lib/ldb/modules/ldb_map.c
source/lib/ldb/modules/ldb_map.h
source/lib/samba3/group.c
source/lib/samba3/idmap.c
source/lib/samba3/policy.c
source/lib/samba3/registry.c
source/lib/samba3/samba3.c
source/lib/samba3/secrets.c
source/lib/samba3/share_info.c
source/lib/samba3/smbpasswd.c
source/lib/samba3/tdbsam.c
source/lib/samba3/winsdb.c
source/lib/samba3/samba3.h
source/scripting/libjs/upgrade.js
source/scripting/ejs/smbcalls_param.c
source/scripting/ejs/smbcalls_samba3.c
source/param/generic.c
source/param/generic.h
testdata/samba3/verify
testprogs/ejs/samba3sam
source/setup/upgrade
source/scripting/bin/samba3dump
source/dsdb/samdb/ldb_modules/samba3sam.c
source/script/tests/test_s3upgrade.sh
swat/install/samba3.esp

Known remaining issues:
- [upgrade] Conversion from the smbpasswd/TDB passwords to ntPwdHash / lmPwdHash is broken. Couldn't find out why.
- [ldb_map] Conversion of attribute names in DN's is still a bit dodgy
- [ldb_map] mapped objectClass names may be mentioned multiple times in returned records
- [ldb_map] add/modify support not tested very well with LDAP yet (only LDB+TDB)
- [ldb_map] group membership is not yet mapped (only primaryGroupID / sambaPrimaryGroupSID)

The alternative approach

nospam@example.com (Jelmer Vernooij) — Tue, 30 Aug 2005 00:04:29 +0200

Looking more closely at one of the other approaches I could've taken for migrating from Samba3. We will probably end up supporting this method later on anyway, once we expand the vampire code. I do think the path I've taken for SoC is the right one though - it allows the upgrade with the least hassle and user input.

This approach could be splitted up into the various parts:
- samba3 pidl backend
- Samba4 vampire + server side samsync support in Samba3
- unixinfo (\unixinfo) (exposes idmap)
- in Samba4 (client side)
- in Samba3 (server side)
- winsrepl (thru seperate pipe?)
- enum/add shares (\srvsvc)
- enum/add registry (\winreg)
- enum/add printers (\spoolss(?))
- convert smb.conf (using Jerry's registry hack)

Advantages of this approach:
- Can be used to 'clone' Samba4 boxes as well
- Relatively few extra loc necessary (a lot shared with vampire)
- 'Clean' code - simply enumerates and adds

Disadvantages:
- Needs running Samba3 daemon for upgrade (!)
- Needs (complex?) Samba3 code changes
- Needs latest version of Samba3

Partially working upgrade paths

nospam@example.com (Jelmer Vernooij) — Mon, 29 Aug 2005 21:59:53 +0200

The one-way upgrade from Samba3 to Samba4 is now working, at least partially! There are both a web-frontend and a console utility that can do an upgrade, given a libdir (usually /var/lib/samba) and a configuration file (usually in /etc/samba/smb.conf).

Last night, I rewrote most of the conversion code to be in EJS rather then in C. One of the positive things about this is that I can use embedded LDIF inside of the JavaScript code and that I can build on the existing Samba4 provisioning code (which is in EJS as well). Writing EJS wrappers for my existing C code was easier then I had thought.

One of the other things I did today is add a fallback mechanism for ldb_map where it can store the data that could not be mapped. There are now two types of records in ldb_map: those that are partially mapped (mapped, but might have one or two attributes stored in the fallback database) and those that are completely stored in the fallback database.

Below are some screenshots of the upgrade procedure in SWAT:

LDIF output working

nospam@example.com (Jelmer Vernooij) — Sun, 28 Aug 2005 17:04:12 +0200

The sam.ldif and winbind.ldif generation code now works as well. I'll be working on generating the new smb.conf next.

One of the things that worries me is the fact that I can generate so few of the Samba4 'user' and 'group' attributes. Probably need to borrow some code from the current provisioning script to fix that up.

I'm wondering whether I should do some of the upgrade using EJS, so it can be integrated into
SWAT easily...

ldb_map somewhat working

nospam@example.com (Jelmer Vernooij) — Sun, 28 Aug 2005 02:43:00 +0200

ldb_map and the samba3sam ldb module that is based on it work nicely now and appear to be memory-leak free. I have so far been testing with a TDB backend loaded with an LDIF from Samba3. Tomorrow will be the big day, testing against an OpenLDAP server.

Since the LDAP schema for Samba3 contains information that the Samba4 SAM database doesn't store and vice versa, it will be interesting to find out whether the current mapping will be sufficient. If it is not, I might have to fall back to storing the Samba4 attributes I can't store in the Samba3 LDAP database in a seperate database instead.

Another thing I did today is clean up the parameter list in Samba4 - I removed all unused parameters. This should hopefully make it easier to do the Samba3->Samba4 smb.conf conversion code.

TODO is down to:
- smb.conf conversion
- Finish sam.ldif / winbind.ldif from the TDB files
- Test samba3sam module against OpenLDAP

First LDIF working

nospam@example.com (Jelmer Vernooij) — Fri, 26 Aug 2005 01:10:47 +0200

Today was a productive day. I just got the LDIF output for samba3dump working for wins.ldif and the various registry hives. TODO is down to:

- sam.ldif
- winbind.ldif
- smb.conf
- test and finish ldb_map and ldb_samba3

I hope to get the first three done tomorrow.

Starting on LDIF output

nospam@example.com (Jelmer Vernooij) — Thu, 25 Aug 2005 17:50:44 +0200

Next on my list is the generation of the LDIF's for the one-way upgrade path. These will be generated by internally building ldb_ldif structs and then dumping them using the LDB helper functions. This would easily allow us to later on directly write to LDB database.

I'll need to generate the following LDIF files:

- hklm / hkcu / etc.ldif: from samba3_regdb
- wins.ldif: from samba3_winsdb
- sam.ldif: from samba3_secrets, samba3_policy, samba3_privileges, samba3_samdb
- winbind.ldif: from samba3_winbind

I'll also make a start on the smb.conf output.