I looked recently, and we started doing sprints at the middle of June last year. It's been over a year since I made the big push to add Scrum to our development methodology, and I'm happy with the overall results. Visibility of impediments was immediate, velocity was remarkably stable, and the amount of over-engineering that was done in the past was reduced. I don't want to diminish the gains that we have made, but it is more interesting to talk about what didn't work ideally, than to talk about what went according to plan. I was amused to learn that our software methodology has a name, "scrumbut".
I feel that many of our problems come from a lack of rhythm. Once iterating every two weeks becomes a habit, it is easier to keep doing things correctly. We've fallen prey to extending sprints a day or two "just to finish the stuff that is almost complete", with some of these extending to 3 weeks. While it is painful to reset a sprint when this is supposed to happen, I suspect that the real cause of our lack of rhythm is a lack of priority when it comes to software development.
At my company, we sell a hardware appliance that provides a service. As a result, we have many different units in the field, and our top priority is generally to get any failing unit back online, or at least to retrieve its data.
What this means in practice is that when we arrive in the morning, there may be crises waiting for us that take priority over the daily standup. Similarly, the vast majority of unplanned items throughout a sprint involve support tasks, or one-off customizations. Support intensive sprints are the primary cause for missing our sprint goals.
While I definitely value the support that we do, I have yet to figure out a good way to integrate it well with Scrum. A simple Scrum concept, the daily standup, is problematic for us at times, as there just isn't a good time of the day where everyone can be ready for a standup meeting.
The good news is that we were given a good amount of freedom when putting together our development methodology. The bad news is that management, in general, was ambivalent about Scrum. Where this caused the most trouble was in the role of Product Owner/Customer. Our VP of Engineering functioned as PO due to the company structure, but he was not provided with clear directions on the future or priorities of the product. As a result, we had a lot of churn as we changed directions several times, that could have been avoided if we had a single decision maker driving the product.
Our QA has always been problematic to integrate into development. We have a skilled QA Engineer, who unfortunately does not have enough background for development tasks, and the situation has led to some specialization of roles. Our story estimates have generally been done by estimating development, estimating QA, and then producing an aggregate number. When QA and development are not in sync, usually because of a regression test, the dreaded QA stories start to crop up. Unfortunately, I don't have a good solution to this problem except by changing team membership, and that is not an acceptable option.
Scrumbut, as we have implemented it, has not adapted well to personnel changes. Recently, some changes have been made to the organization that have resulted in our team becoming two teams working on separate projects. It has been difficult to see the value of a combined standup, but the new team sizes don't really justify separate standups. Similarly, we have tried to include our summer intern in our standups, but his work has been entirely separate from the core team. I believe that this impediment can be generalized as one of team sizes. Instead of one team, we have become three teams, two of which are too small to justify the use of scrum. What I take away from this is that the guideline of 7 members, plus or minus 2, has been accurate for us when it comes to agile development and ideal team size.