Archive for the 'development' Category

15
Dec

The open source question

I’m tired. Tired of hearing yet another big bozo speech about how we will move to open source and how that will magically reduce our costs and make our software better.
Gentlemen (and those few ladies who end up reading this), behold, a couple of reality checks

1. Open source is not free
Contrary to popular belief, open source comes at a cost. Unlike commercial packages (full software, or libraries) the cost here is not tied to the initial cost of acquisition (either as a license fee with or without a support contract). The true cost of open source hides at a deeper level. The place where it still costs money is a cost of integration. Even though the idea of open source is that there are a whole slew of people working on it and that the community will be the best support group ever, the reality is that a lot of open source projects are actively maintained by only a handful of people per project. Which generally means that you (the developer who needs to integrate it) will spend a lot of time reverse engineering the code.
The second point where there is a form of (rather indirect) cost attached is in the area of strategy and maintenance. Since this code is created and maintained on a voluntary basis it doesn’t really carry any form of guarantee that it will be maintained and/or evolve, and if it does evolve in a direction that might not be what you wish/expect, there is no sales/account/technical staff to help you out in the migration or to voice your concerns to.

2 The licensing aspect
There is a vast number of licenses out there, making the distinction between all of them is beside the point here. Suffice to say that, in general, there are a number of common themes.
Not everything can be used for commercial purposes, some might require that you submit any modification back to the community under the same license, some allow you to use it as if it were closed source, … . Best to have your legal people look at the license if you are unfamiliar with the matter. Just a point of attention…

In conclusion, yes I believe open source code can make your software better but there is an amount of sanity checking that needs to be done (just as we do with closed source). The questions to be asked are not just a matter of financial or legal cover-your-ass but also of strategy (do I like where this thing is heading?, do I want this vital piece of my software to be open source?). There is no one-size-fits-all answer to these questions, so let us please answer them and do the due diligence.

14
Aug

Embracing failure

The most burning question a business generally has for engineers is “when will it be finished”.  They are looking for a specific date and often a specific plan for how you’ll get there. They would most likely be overjoyed if you could tell them exactly at which point in time you will write which line of code. As engineers we have a love/hate relationship with that question. Somehow there is an understanding of the importance, but we are dealing with so much unknowns that we find it hard to commit. It’s in our nature to look at all possibilities and account for a large number complications, even though they may never occur. And then there’s off course always the “give us a clear requirement” catch-phrase we throw back at the business. (truth to be told, we often don’t do a good job in helping the business find that clear requirement, but that’s a different discussion altogether)

To many engineers the request for a date seems unreasonable and shows a clear lack of understanding and appreciation for engineering from the business side. However, if you look at where those people come from it’s not so hard to understand.
When you have to plan a production-type operation you are dealing with a known process. You know every step of the way, probably in great detail. So it’s rather easy to plan. If you know that turning a screw takes 2 seconds, it’s a no-brainer to see that in one minute you can turn 30 screws. It’s very predictable and controlled. And the more detail you get in your planning, generally the more control you’ll have.

Unfortunately, all that is not so much true for engineering. Crafting software is a creative process. It’s about figuring stuff out. Stuff ranging from a vague requirement to the logic that goes into an algorithm. It’s dealing with unclarity and vagueness. Software is often more of a trial and error thing. And the unknown is feared by many as it means letting go of that tight control they had for decades.
The key to engineering is not fearing but embracing failure, and often to try and fail as soon as possible. Many innovations have come after numerous iterations of failure and software is no exception to that rule. Engineers need room to experiment, and you can’t plan for that. You can’t plan (in the typical production way): “let’s fail five times and get it right the sixth”.

All that doesn’t mean that engineers shouldn’t do planning but there’s a different angle to it. Engineers might not readily admit it, but generally if you give them a problem they can make a reasonable estimate. It won’t be accurate per se, but they will have a gut feeling for what it will take. They understand their job and even though they can’t predict every possible failure they encounter they generally can outline an approach in their head and take a shot at telling where the painpoints will be. The key here is that in the end they are still only estimates. They are not the exact science to what traditional managers are used to, only an indication of magnitude of work.
Sometimes though, that simply isn’t possible. The problem might be to vague, the problem space not enough understood, … . In that case I am a huge fan of timeboxing. Reserve a specific set of time to do some preliminary analysis, come up with questions, possibilities, … and present that back to the original requester.

As I said many times before, the job of a good project manager is twofold. First there is the making of a plan, which accounts for both the operational tasks and the more freeform creativity needed for engineers. Second, and more important, is dealing with changes in that plan.

15
Apr

Technical Debt

Technical debt is a concept which I never fully realized how accurately it is named. (I have to thank some colleagues for listening and discussing this subject on several occasions)

For those not ‘in the know’…technical debt refers to dirty pieces of code/db design/UI clutter/… that build up over the time of a project. They can be due to changing requirements, time pressure, developer oversights, … . Essentially it’s the stuff you don’t want but somehow hardly ever get round to really cleaning up, as they don’t necessarily generate bugs or functional problems. And rest assured, every piece of software (of a significant size) has it. As all developers know, there is no such thing as perfect code so there’s always an area to refactor, improve or otherwise clean out.

Back to the name though…Technical debt really is a debt. Extremely comparable to monetary debt. In a typical situation monetary debt comes from you buying something that costs more than you can afford at that time. So what you do is: you get an instance to give you what you want and you work out a plan to repay them. That plan entails a couple of things. It looks at what you can afford to pay them based on your current paycheck, it looks at the length of time it will take before you are without debt again and they will charge you an extra fee on top of the amount you borrow for their service. But above all, based on these conditions they will set a limit to the amount of debt you can accumulate.

Oddly enough, technical debt isn’t all that different. You create that same amount of debt when building software, for reasons I mentioned before. However the big difference is that there is no governing body that puts a limit on the debt you can accumulate. It doesn’t take a rocket scientist to figure out this is indeed a baaaaad situation. If you keep on building your debt it means that maintenance becomes harder (i.e. you *pay* more for doing a small change than in a clean codebase), new features become “just another hack” in the software, there are exception routines everywhere and this spiral keeps spinning down. Eventually this will drag down your development velocity and ultimately will grind your project to a halt.

This is where, for me, good product ownership (/project management) comes in. The product owner can help here in several ways…

  1. Understand how technical debt gets created (i.e. understand the implications of selecting a specific implementation)
  2. Allow developers time to gradually work away that debt. This also means standing up to the business and actively refusing new features* (just like you wouldn’t buy that Porsche if you know that you can’t pay off your house anymore if you did)

*actively refusing is obviously more than saying no. It means working out a plan on where features can fit in and/or look at the scope. The essence is really to work away technical debt as part of the entire planning routine.

19
Feb

Product ownership

I’ve been catching up on agile development lately, more specific, on the scrum methodology.

After sorting through various books, course material and tons of useful talks to colleagues, peers and agile people (not necessarily claiming they are agile but definitely adopting the concepts). First of all, I wish to thank all of these people for the time they took to educate me. I feel at a point now where I have a grasp on how it works and I’m past the steep hill in the learning curve. A good time to look back.

Starting with a statement: "Having seen the waterfall system in action one to many times, I am sure there is a better way". This, combined with my current employer wanting to adopt scrum, is what drove me down this road. Being on the operational side of the building, I have been experiencing product ownership from up close and here’s what I think are important qualities for becoming a good product owner.

  1. A deep understanding of the problem at hand
  2. A good enough technical background
  3. Great communication and negotiation skills
  4. A "business sense" and a vision
  5. No other job

Unfortunately, many companies (including mine), horribly fail at most (or even all) of the above, especially the last one.

In the coming articles, I’ll be elaborating more on those 5 points (= nice hook to have some more articles on my blog :) )

14
Jan

Why we will not have senior developers anymore

I have the pleasure of working with some people who live and breath software development. Coming from the old days where assembly was new and cool and above all something that made you think about the inner workings of a computer.

Now what makes a software developer senior?

These days it seems like becoming "senior" developer means having a few years of experience under your belt. You can easily have senior guys who never did anything else than php or .NET development. (nothing against php or .NET if used under the right circumstances, don’t get me wrong).
To many businesses this would qualify as someone who would be labeled ’senior’.
I think there’s a bigger issue here. Those new senior developers don’t have that deep understanding that many of the current senior developers have. They never had to be bothered with allocating memory or worrying about limited hardware resources.

The current trend seems to evolve to "if we have an issue with resources or performance, we’ll throw some extra hardware at it". I understand Ruby relies heavily on this principle, after all, development time is more expensive than cpu time.  A fact, just like looking good is more important to Paris Hilton than having something useful to say.

I personally doubt this level of seniority somewhat. They might be very good developers, but in their own language. When it comes to high performance apps and scalability, often it requires an intimate understanding of the inner workings of a compiler, down to bit level.

The "old" senior developers know for every line of code they write what bits move around, how that relates to the heap and stack. They intimately understand Pascal and C strings and the (dis)advantages of both, or why certain algorithms are more scalable than others. Simply because they grew up having to care about every bit in that machine.

Our future senior developers…

These days people often don’t get down to that level of computing. They have fancy things like objects, garbage collectors, excessive amounts of disk and memory available, … . And they often start out with a higher language like Java or C# which makes them unaware of all these intriguing little details that make or break large scale applications. I don’t blame the persons, those who really want will still find all the information necessary to get that understanding of the machine.

I think we should start reconsidering our school system (for Belgium for sure) and start teaching our computer graduates the basics first before letting them have a go at a high level language which takes away a lot of the machine related problems. In their first years, give them an 8086 or an emulator that lets them run assembly. Make them aware of address registers, stack and heapspace, allow them to fool around with 0 terminated strings and the up and downsides. Make them feel the pain, they will even more so appreciate the higher level languages, but above all, they will learn why some things simply require a lower level language.

This is a call to all deciding people on school boards. Don’t try to have a sexy education with fancy new languages, just get a good old hardcore education that teaches what they need to provide the world with senior developers.