Common libraries in a large team

Question

Assume you have five products, and all of them use one or more of the company\'s internal libraries, written by individual developers.

It sounds simple but in practice,

Answer 1

I would like to point a problem in the solutions suggested above: treating the library as an internal project with its own versioning scheme.

The problem

If your company has more than one product (lets say two teams - two product: A, B), than each product has its own release schedule. Let's give an example: Team A is working on product A v1.6. Their release schedule is two weeks from now (suppose Oct 30th). Team B is working on product B v2.4. Their release schedule is 1.5 months from now - Nov 30th. Lets assume both are working on acme-commons-1.2-SNAPSHOT. Both are adding changes to acme-commons, as they need it. Couple of days before Oct 30th, team B introduce a change which is buggy, to acme-commons-1.2-SNAPSHOT. Team A is getting into stress mode since they discover the bug 1 day prior to code freeze.

This scenario shows that treating a common library as a third party library is almost impossible. The trivial, but problematic, solution is for each team to have their own copy of the version they are about to change. For example, product A v1.2 will create a branch (and version) for acme-commons named "1.2-A-1.6". Team B will also create a branch in acme-commons called "1.2-B-2.4". Their development will never collide and they will be stress free once they tested their product.

Of course, someone will have to merge their changes back to the original branch (lets say master or 1.2).

The problems I found with this solution is:

1. Branch inflation - the tree structure will be very puffy and it will be harder to understand the flow of changes/merges.
2. Merges back to 1.2 will probably never happen - Unless a team/developer is dedicated to this library, the chances that Team A or Team B merges their code back to 1.2 branch is slim. They will always stay focused on their tasks, thus creating and using their own branch space. Allocation of a developer/team is expensive, thus not always a viable solution.

I'm still trying to figure this one out, so any thoughts of this matter are welcome

Answer 2

I agree - this is difficult. In our small team (consulting .. not a product company - which made it harder), we had one common component that stood out from the others. In this case the recipe for success was:

• Make a good developer responsible for developing the component
• Make a good developer the gatekeeper for maintaining the component
• Make sure all upgrades (there were several) are backward compatible
• Make sure there is some basic documentation (or a simple reference application) explaining how the component is to be used
• Make sure all developers know that the component exists (!) and where they can find it (along with the code, if they wish to review it)

Give developers the ability to review the code and suggest better implementations or refectoring, but have the final mods go through an experienced gatekeeper. When the component were upgraded, older apps did not have to upgrade. If we did a new release, we evaluated if we wanted to upgrade, and if we did, all we needed to do was swap the libraries - no code needed to change, unless we wanted to use some new features available through the upgrade. Resistance is inevitable, but sometimes it is a good sort of resistance when it comes from good developers who have better ideas for a new generation or refactored component.

Answer 3

Create an Anti-corruption (DDD) layer for the existing library... this is nothing but a facade.. and then write unit-test for this anti-corruption layer... Now even if someone upgrade/update the library you would know if something is broken by running the unit tests...

These tests could also serve as documentation of contract... and not every project that need to use the library has to write this anti -corruption layer, if they are using the same exact functionality..

Answer 4

Duplication is the root of all evil

I would argue that unchecked government is the root of all evil :)

I do get a lot of flack for even suggesting that duplication should be an option. I understand why, but let me complicate this a bit.

Say you have a fairly large library that doesn't actually do anything in particular - it's just a collection of utilities. There are NO tests for this library - at all. You need only one function from it. Say, something that parses out a file's extension.

Pop quiz: do you just write something as small as this in your own project, or you bite the bullet and use the free-for-all untested set of utilities, which WILL break your application if someone breaks the function?

Also, imagine you are in environment where writing tests is not part of the culture, since most projects are very intense and have a very short development span.

Duplicating large systems - such as client registration - would be dumb beyond belief, of course. However, aren't there any cases where it is safer to duplicate something fairly small in your project if the alternative is not safe enough (no system for maintaining common code).

Think of it this way - and this happens all the time - multiple contractors working on different projects, for the same company. They don't even know about each other.

My argument is this:

If a team cannot dedicate to maintaining a solid common codebase, or if the environment does not give them enough time to, it's best to let them work as separate "contractors".

You will STILL need to use large existing systems that simply cannot be duplicated.

Answer 5

I don't agree with this:

Duplicate untested code is better than common untested code (you break only one project).

If you are all equally likely to create bugs by implementing the same thing, then you'll all have to fix potentially different bugs in each instance of the "duplicate" library.

It also seems that it'd be much faster/cheaper to write the library once and, instead of having multiple other teams write the same thing, have some resources allocated to testing.

---

Now to solve your actual problem: I'd mimic what we do with real third-party libraries. We use a particular version until we're ready, or compelled to upgrade. I don't upgrade everything just because I can--there has to be a reason.

Once I see that reason (bug fix, new feature, etc.), then I upgrade with the risk that the new library may have new bugs or breaking changes.

So, you're library project would continue development as necessary, without impacting individual teams until they were ready to "upgrade".

You could publish releases or peg/branches/tag svn to help with all this.

If all teams have access to the bug tracker, they could easily see what known issues exist in the upgrade-candidate before they upgrade, too. Or, you could maintain that list yourself.

@Brian Clapper provides some excellent guidelines for how to run your library as a project in his answer.