RFI Keeping Open Source Dependencies Up To Date

We just deployed the first phase of a new project that uses a large number of open source components including Castle (Windsor & MonoRail), NHibernate, and Rhino Commons to name the big players.  As with any project as much as I know about programming I learned even more when the rubber met the road.  Many right decisions were made, some wrong decisions were made, and the code has only been deployed for a month and I'm already making a long list of how I can do it better next time. 

One particular area that I want to get better at is staying current with dependencies.  Most OSS projects are constantly being improved and still remain stable.  Here is how my project began...I downloaded a debug build of Castle RC3, picked out the assemblies I needed, and added them to my /trunk/lib folder (What I look back on as my first mistake).  I built the latest trunk revision of Rhino Commons, and added those assemblies to my lib folder (at least I was consistent).  I then happily worked on my application, writing code, unit tests, a build script, life was good.  As we progressed in development there was a few things we needed to do in MonoRail that just didn't jive with MR conventions.  So, I synced to the RC3 branch, changed two lines of code, rebuilt, and updated my libs (second BIG mistake).  Life was good again, but then ran into a small snag with Rhino Commons.  Ok, I was able to tweak MR just a bit, so I can do the same with Rhino (see consistent, but I shoulda learned :).   Hmmm...the AI Sourcing machine has been busy and Rhino Commons has changed quite a bit since I grabbed the source last.  So I had to do some digging and diffing to find out what rev I had built from previously.  Finally I get back to the right rev (I right it down this time) make my changes, rebuild, and update my libs.  Coding progresses, but eventually I run into the same situation again, this time I actually commit a copy of the relevant (read only the projects that I need to tweak) code into our source repo (which isn't SVN to make things even more interesting) and begin to diligently comment explicitly the lines I change.  Just call me Sofa King Screwed :)

I'm a half way decent coder but I make dumb decisions sometimes.  I acknowledge that, and I want to get better.  At some point I will need to re-up my dependencies or be forever locked into an out dated version.  What I'm looking for is a better way.  What do you do with your OSS dependencies?  I've read posts where devs refer to a "private branch" which I presume to be a more organized approach to what I ended up doing.  Do you merge with the projects trunk?  How often do you merge with the projects trunk?  Are your dependencies built with your application or do you build them separately and only reference the assemblies?  Dan's post on How to Branch Properly got me thinking that may be the way to go.  Grab a trunk copy of the dependencies to be used, add them into my application's solution/build process, and merge with the trunk on a scheduled basis.  If appropriate the changes I make to my private branch can be refactored into patches to be submitted.  Is this an approach any of you have used?  Does it work?  Have you automated the trunk merge process?  We don't use SVN, but our source control solution is touted as king of merging, so I presume it would be fairly straightforward.  The other issue is when my dependencies depend on each other.  Oren makes it look easy.  This leads me to believe that the right approach is to stick with the build/reference as lib approach.  I'm willing to earn my paycheck, but if somebody already has a working solution, I'd love to hear about it.

posted @ Thursday, April 03, 2008 9:44 AM


Print

Comments on this entry:

# re: RFI Keeping Open Source Dependencies Up To Date

Left by Willie Tilton at 4/3/2008 4:09 PM
Gravatar

What I typically do, is similar to yours but just stick with the initial version of OSS that I started with. I think developing side-by-side with another project that has nothing to do with you and you have no control of would be a constant nightmare. If the OSS does what you need it to do, why upgrade?

I think if there is a relevant enough reason to upgrade, like performance or a huge update then you're just going to do what you've done before. Sorry, but I wouldn't want to content with Oren, he's a coding freak of nature - I'm also guessing he's more in tune with the OSS projects that he's using then you are.

Just to reiterate, you have to look at it with some business sense, there's always going to be something better, greater, and faster on the horizon or just released. The question you need to ask though is (and I've been asking myself) how can I get this application into a solid state the quickest way possible.

# re: RFI Keeping Open Source Dependencies Up To Date

Left by Jeff Brown at 4/4/2008 12:44 AM
Gravatar

Try to minimize how often you sync up unless there's really something important that you need from the latest versions that you don't have.

Another good choice is to keep a local version controlled copy of the original source tree that you are forking. We have a "vendor" branch for this that contains original versions. That makes it easier for us to merge in new editions because we can diff against what we started with.

An even better choice is to contribute your changes back to the mainline project. Chances are someone else will find them useful.

In the short run, contributing back may require more work because you'll have to write tests that are up to the standards of the OSS project. It's also possible that the project maintainers will object to your changes or your approach to solving your particular problem. They may have something more general in mind or just different.

Of course the advantage is that if your contributions are accepted (or something equivalent that meets your needs) you don't need your private branch anymore. Problem solved!

# re: RFI Keeping Open Source Dependencies Up To Date

Left by Bill Pierce at 4/4/2008 8:21 AM
Gravatar

@Willie:
"If the OSS does what you need it to do, why upgrade?" I agree completely. Where I run into trouble is when the OSS doesn't do exactly what I need, the latest version doesn't do it, and I need it to do it now.

# re: RFI Keeping Open Source Dependencies Up To Date

Left by Chris Ortman at 4/6/2008 10:38 AM
Gravatar

I asked a similar question a little while ago. My solution is here http://codeprairie.net/blogs/chrisortman/archive/2008/04/06/follow-up-how-do-you-manage-your-open-source-software.aspx

Your comment:



 (will not be displayed)


 
 
 
Please add 4 and 1 and type the answer here:
 

Live Comment Preview:

 
«July»
SunMonTueWedThuFriSat
293012345
6789101112
13141516171819
20212223242526
272829303112
3456789