A lot of companies I talk to are “shifting left”. This is primarily driven by the question of how test fits into agile and feels good because it aligns with the classic fact that defects are cheaper to fix the earlier you find them.
Makes sense and personally I’m a big fan of shift left; but lately it seems to me that “shift left” has taken on a very waterfall form. This concerns me since I can see our industry about to take a massive back-step in terms of quality (both in terms of effectiveness and efficiency) similar to the naive test outsourcing done in the early 2000s. It took at least four years for the industry to right itself and get a net benefit from outsourcing and the same could easily happen again; and if nothing else that just makes my job (and all those genuinely trying to advance software engineering) frustrating and boring. This waterfall form appears most commonly in phrases such as:
“We are ‘shifting-left’ so all our testers are learning ruby.”
“We are ‘shifting-left’ so we are focussed on unit testing.”
What’s waterfall about this? Well, let’s consider what agile is all about.
- Let’s say product development comprises requirements, design, coding, integration, and system test.
- Waterfall organises these by function, i.e. you did all your requirements, then all your design, etc.
- Agile organises these by feature or story, i.e. in a sprint you do all of these for a small number of features/stories.
- Agile increased efficiency mainly by removing wasted work (i.e. design for requirements that were dropped at the coding stage), removing waiting time (see Poppendiek), removing unhelpful documentation, and other similar benefits.
Now let’s consider test. In the “old days” we used to have a mix of test types, e.g. unit test, integration test, and system test (obviously there are other types but for simplicity let’s stick with these). These were aligned with the waterfall stages development, integration, and test.
So what about test in agile? Well, if we were applying agile principles properly, then we’d take all the test phases and just do them in a single sprint, i.e. we’d really take all testing activities and shift them left. So for a feature/story being implemented in Sprint X, in Sprint X (or possibly Sprint X+1) we would write the unit tests to test the code classes, possibly some integration tests, and some system tests to test the story end-to-end. Obviously some rationalisation would make sense and is an opportunity for efficiency improvements.
But that’s NOT what a lot of people are doing. Instead they seem to be using the following logic:
- “Agile” means that we get rid of the test phase.
- There’s development going on in every sprint; so basically the whole lifecycle is now the “development phase”.
- Unit testing is the testing we used to do in the development phase of waterfall.
- So now all testing is unit testing.
So a clumsy application of waterfall concepts to agile has resulted in “shift left” meaning “unit test only”, i.e. we haven’t actually shifted anything, we’ve just dropped the stuff we used to do on the right.
This is ridiculous when you think about it this way. It’s like saying we don’t have any requirements (or stories) in agile. Of course we do, in fact both Scrum and even XP talk more about requirements/stories than they do about coding. We just do them on a feature-by-feature basis and only when we’re ready to actually implement them. Agile is about dividing a project into sprints defined by sets of features (and then doing all the activities for those features) rather than dividing a project into stages based on activities (and then deciding what features to apply the activities to). Agile is not about getting rid of everything except coding (or unit testing). This was clear and done properly for development, but some reason test has become confused.
Why is this happening? It’s hard to really understand the root-causes and they are different for different companies. People I’ve spoken to in companies that went down this route but then re-adjusted have suggested a mix of the following:
- Developers are often driving agile initiatives and developers know unit testing. So everything becomes unit testing.
- Naive project managers (or even development managers) believe this approach will save them money (though this rarely lasts long).
- Some testers see it as an opportunity to re-skill and move into a developer role.
If these reasons are true then I would suggest that the problem is a lack of real test leaders involved in the agile transformations within companies. As Paul Gerrard recently presented – “Will the test leaders please stand up!”.
Whatever the reason I think that the trend to “unit test only” is at best expensive (developers are generally more expensive than testers) and most likely also ineffective (replacing product experts with technical experts). “Unit test only” is like testing a bridge by getting a group of civil engineers to inspect each part as it arrives on-site; but if that’s the only testing done, I wouldn’t want to be the first person walking across that bridge.
So this is a call to people to really “shift left” and not just “drop right”.
At this point I should put in the disclaimer that although I obviously work for a test tools company with a vested interest, prior to this I worked for 15 years in a variety of software management roles. In all these roles I was a champion of agile (e.g. I was a certified scrum master in 2004) and I’ve introduced unit testing into several teams. So I’ve practised what I’m preaching above.