Advertisements
Posted by: jsonmez | July 3, 2011

A Drawback of Agile

I want to preface this post by saying that I am a pretty avid practitioner and believer in Agile development methodologies.  If we divide the world into Waterfall and Agile, I’m going to fall on the Agile side every time.

I have written in the past about some of the shortcomings of specific agile methodologies.  I’ve even suggested a new one myself.

The truth of the matter is that agile is pretty vague.

The point of this post is not to attack Agile, but rather to honestly reflect on one of the shortcomings that I have come to realize seems to be prevalent in all of the various flavors of Agile methodologies.

Driven instead of steered

I spent a good deal of time trying to think about what seemed to be missing from software development ever since I started really embarking on that Agile journey.

Ever since I’ve been wading around in Agile-land, from Scrum to XP to Kanban, there seems to have always been something missing.  Something that was there around the beginning of my career, but now seems to be somewhat lost.

I’ve noticed it in others as well.  I have noticed more and more software developers becoming more passionate about their side work than their main job, even when they are working on some pretty interesting stuff.

I don’t think it is a coincidence that so many open source projects have sprung up along with the rise in Agile development methodologies.

So what am I getting at here?  What do I believe is… missing?

I think it comes down to the difference between being driven and being steered.

Most Agile projects, regardless of which specific methodology they are adhering to, focus on the idea of having prioritized backlogs that are worked on in a specific period of time and measured.

The good thing about this kind of flow is that the customer or the business is setting the priority on the work being done.  The bad thing about this kind of flow is that the developer is being driven down a specific path of software development.

What I mean by this is that software developers in this kind of system do not have much control over their own priorities and what they work on in their day.

Backlog work is measured, non-backlog work is not measured and even frowned upon for the most part.

A certain amount of freedom is given up in exchange for efficiency.

My business mind says this is good.  My developer passion feels like it’s been stifled just a bit.

Now don’t get me wrong here.  I am not exactly complaining.  And I am not trying to derail Agile development or say we should stop doing it. 

What I am trying to do is point out something I think many people ignore in the name of progress.

I think sometimes the whole Agile mindset has a tendency to kill the parts of developers that got them interested in software development in the first place.  The idea of exploration and creativity and feeling they have the freedom to explore areas of code and technologies and projects that interest them the most.

This is why I believe so many developers have taken up side projects which do not even offer up a paycheck.

Think about it… Open source developers must be getting something they are missing in their 9 to 5 jobs or why would they be so passionate about it and be willing to expend so many hours for no monetary compensation?

Agile puts a gun to your head

Did I grab your attention with that one?

GunToHead

I mean it part in jest of course, but there is certainly some truth to it as well.

When I am working on an Agile project, I know that time is ticking away on the backlog I am working on and that it is going to need to be done in a relatively short timeframe.

I certainly have the tendency to look at some area of code that I would perhaps like to delve deeper into and refactor, but instead pass it by in the interest of completing the work at hand.  This can be argued as a good or bad thing, but I’m not interested in debating it at this point.

What I am interested in saying is that there is certainly an amount of pressure that is applied just by having work broken down and measured.  It is a constant pressure that doesn’t let up week after week. 

If you complete all the items in a sprint, you’ll have new ones the next sprint.

If you complete a backlog, you’ll need to grab the next one on the pile.

Before Agile came strong onto the scene, many software projects were still doing iterative development of sorts, but not really calling it Agile.

You couldn’t really avoid it, since most work is maintenance work, and it is pretty hard to not iteratively do maintenance work.

The key difference is that we were a little less efficient, but a little more free.

We tended to be steered into areas of work, but given the freedom to work on things as we saw fit.  Sure there were deadlines and certain goals that needed to be accomplished in certain timeframes, but day to day work was more about what do I feel like getting done today rather than what must I get done today.

I don’t want to paint this picture by indicating that we should get to our desks in the morning and work on whatever we want to work on each day, (I think that would be the other extreme of what I am suggesting,) but I do want to point out that there is a distinct difference in being driven and being steered and that Agile methodologies tend to drive developers down a specific path versus steering them in a general direction.

Why this problem is important

Now I could be going out on a limb here with this post.  Maybe I am the only one who thinks this problem exists.

Maybe many of the newer, younger developers don’t remember the time when you came to your desk and each day was a brand new path to explore instead of having the path laid out for you ahead of time.

But, I suspect that deep down many developers, like Neo in the Matrix, feel that there is just something that doesn’t seem right and that this might be it.

If that is the case, I think solving this problem is very important, not just to software developers, but to the companies they work for.

If we can solve this problem I think we can achieve benefits like these:

  • Higher retention rates. I have been on many Agile projects in the last decade and the retention rates have been pretty bad.  I don’t have any data to back this up, but I suspect they run much higher than non-Agile projects.
  • Improved architectures.  One related problem I have noticed on Agile projects is that architectures tend to get messy, because no one ever has time to focus on the architecture.  There are many things that developers would do that would improve architecture of projects that just would never make it onto a backlog.
  • Improved efficiency through synergy.  I strongly believe there is a strong synergy component between the work that a person wants to work on and how much they accomplish.  Isn’t it amazing how fast skunk works projects end up being built?  Again I don’t have statistics to back this up, but in my experience skunk works code gets written 5-10 times faster than regular project code.  I think it has to do with this synergistic relationship between passion and work output.
  • Increased adoption of Agile.  Some “old timer” run software development shops really resist the idea of doing Agile development.  Probably a large amount of this resistance comes from not wanting to give up their freedom.  Agile as it is now tends to trade freedom for dependability.
  • Decreased half-ass Agile adoptions.  So many software development shops claim to be practicing Scrum or another Agile methodology, but they are not really doing it.  I suspect some of the reason for this is related to the constant pressure they know really following an Agile methodology might impose on all members of their organization.

I think I have pretty clearly highlighted the problem in this post, and in my next post, I hope to come up with some possible solutions to address this problem.

What do you think?  Am I totally off-base here?  Does what I say ring strangely familiar?

As always, you can subscribe to this RSS feed to follow my posts on Making the Complex Simple.  Feel free to check out ElegantCode.comwhere I post about the topic of writing elegant code about once a week.  Also, you can follow me on twitter here.
Advertisements

Responses

  1. I’m with you John. We introduced scrum 6 months ago, and I estimate I am now 15% less efficient than I was beforehand, and all the interesting portions of the project have been stripped away in order to achieve “Done” status on the task list.

    I see the perceived benefit for management to have the task list on the go all the time, but I see that sometimes we are running very quickly in the wrong direction because the focus is on the short term goals, not the bigger picture.

    In the same manner that you can’t run a kilometre in 10 x 100m sprints, you face vocational burnout by trying to get 50 hours of work done in a 40 hour period, week after week.

  2. Hey dude 🙂 Long time follower, first time commenter 🙂

    I’m not so sure. In my 6 years in the field so far, I have had what I call the privilege of working in an agile dev shop for only a mere 8 months, as a subcontractor.

    Those 8 months were THE. ONLY. 8 months I’ve worked in my whole life where the extent of my personal stress level was essentially, “OK, what is the bus schedule this week?” If Agile does indeed have flaws even when orchestrated correctly, I’d call that a “sanity tax.”

    I’ve only worked at two other places, and their retention rates are worse than , and for about a million good reasons.

    Those reasons can basically be summed up as, “Oh, you program computers? GREAT! That means you obviously also have the skills of an IT support specialist, a trained sales person, a system administrator, a project manager, and a client liaison…right?”

    The answer to that anecdotal question is, of course, a resounding “Wait…so who’s doin’ what now?”

    Agile, when done correctly, is a godsend to a programmer. There’s a certain level of comfort and security in knowing that, from the second you walk in the door to the second you leave in the evening -> you know exactly what is expected of you, and you don’t have to fly by the seat of your pants, running around like a chicken with its head cut off at every turn.

    Being “driven” in that sense is great, and it does not take anything away from creativity, learning, fun, or what have you. It’s a fantastic experience when the only interaction you have with the client OR the project manager, is when the PM addresses the team as a whole and says something like, “OK, the client wants something that does XYZ, let’s write up some story cards and estimate ’em!”

    I’d also be remiss in my critique if I forgot to mention that my Agile experience at was the single most fun, engaging, and interesting experience I’ve actually had since joining the “real world.”

    I’m telling you, man….If that Agile dev shop hired employees rather than subcontractors, I’d quit my current job without even giving two weeks notice, and I’d go to that Agile place and wouldn’t leave until they let me sign an employment contract, even if they called the police to get me out of the building.

    If Agile is done correctly…my God, there is no comparison possible. When I say “correctly” I’m specifically making an argument against places who say things like, “Oh, our programmers pair…that means that we’re following the XP book to the letter, right?” or “We have a project backlog…make sure to put Agile on our company website” etc…

    You do raise a number of interesting points though. Again, I have only had one single agile experience. Unfortunately, it’s all anecdotal, and it sounds like I subcontracted at a rare company indeed. Looking back, I miss it because I feel that I’ve essentially been run ragged ever since.

    • Hi Andy,

      Thanks for commenting and thank for reading my blog.

      I can definitely understand your viewpoint. You have some good points. I have had a similar experience when I first started working with Scrum.
      I also agree that Agile projects done right are much better than other environments and a much better way to develop software.

      I am definitely for Agile development, hopefully my post didn’t contradict my view on Agile in general too much.

      One interesting thing about what you said though, was that you worked on the project for 8 months and as a contractor.

      I think it is worth considering Bill’s analogy to running. You might be able to sprint for 8 months without getting tired, especially if you are being paid by the hour.
      Don’t get me wrong, I would definitely want to work an an Agile shop than not, but in this post I am focusing on that one aspect that seems to be missing when developers go to Agile.

      Don’t worry though, I have some good ideas on how to remedy that situation, which I will share in my next post. I been thinking about this for a while now, and like I said, while I am definitely much more content in an Agile environment, I still feel there is this one thing missing.

  3. Hm. pretend HTML is disabled. Edits:

    —————————————————-

    I’ve only worked at two other places, and their retention rates are worse than (insert fastfood chain here), and for about a million good reasons.

    —————————————————-

    I’d also be remiss in my critique if I forgot to mention that my Agile experience at (agile company name here) was the single most fun, engaging, and interesting experience I’ve actually had since joining the “real world.”

  4. Disclaimer up front: I’m no longer a developer. I’m a product owner these days.

    Although I determine (for the most part) what goes into a sprint backlog, and in fact prioritize all of the items in it, the team pulls items into active development as they see fit. Sometimes they pick things that are more interesting from the middle of the backlog, but often they know that the priorities are there for a reason and they work them in that order.

    As a team, we seem to be able to both do what we want and what the company needs.

    One of the reasons that this works so well for us is that we don’t do “sprint commitments.” When we get to the end of the sprint, whatever is not completely done (for us, done means code reviewed and merged into master branch) gets rolled forward into the next sprint. Things change, so in my estimation asking the team to “commit” to a certain amount of work can ONLY lead to disappointment and frustration on everyone’s part.

    So we don’t do stuff that frustrates us.

    –Bruce

    • Good point, Bruce. That makes sense. I imagine developer would feel more free and less restricted if they are selecting the backlogs and the pressure of committing to a time period is absolved. In my next post, I plan on talking about an idea of using bidding to allow developers to bid on items they would like to get done, so that they have more part in determining what gets worked on.

      • had exactly the same thing in my last agile run project.
        the biggest drain was the continual pair programming which wouldnt let you do your little breaks like checking out reddit and so on.

  5. […] A Drawback of Agile (John Sonmez) […]

  6. […] In my last post I had talked about a possible drawback of Agile basically being that Agile projects tend to drive developers instead of steering them. […]

  7. I have worked in a ‘sprint’ shop that was not officially Agile, and I can tell you that the stress level was extreme, with much developer burnout. Most of my career has been spent outside of Agile development, and this is what I enjoy.

    With the non-Agile freedom I have enjoyed over the years, I have been able to undertake proper R&D and create not only good technical solutions, but add features to applications that greatly enhanced the end-user experience. This could never happen in an Agile world.

    Although Agile development is efficient, there is more to life that efficiences. An individual, organization, or country requires creativity to grow. Innovation does not happen in an Agile world, just the constant drum of widgets being produced.

    Whether you enjoy Agile development depends on the programmer. The more creative ones likely shy away from Agile, whereas those who prefer stability tend to Agile development. The problem is that many shops have both types of programmers, and in fact need both types. So, how to keep both happy.

    I advocate a combination of both Agile and non-Agile development. We can call Agile development ‘bread and butter’ development, that pays the bills. Non-Agile development makes me want to go to work each day. This is the same concept with artists and scientists. Artists must create sellable ‘art’ which is not overly creative, to pay the bills and allow them to truly experiment. Science is the same. Scientists must perform the research necessary to crank out papers, to keep the funding coming. Outside of their ‘bread and butter’ research they often work on their more interesting research.

    Sometimes we need a little bit of ‘bread and butter’ to go along with our creativity.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: