Background
I attended the STC Meetup in Birmingham on Tuesday evening and was struck by a realisation towards the end of Simon Knight‘s lightning talk on the pros and cons (they were mostly cons) of Cucumber. More on this epiphany later…
By happy coincidence I had recently started to look into Cucumber as a concept because our development team on the SAMS Programme here at Nottingham are trying to move towards embracing continuous integration as a methodology. I have read and digested the Cuke4Ninja how-to PDF and it seems, on the surface, to be addressing some of the failings of “traditional” software development. Its main thrust is the use of Gherkin, a form of Structured English, to enable requirements specifications, or derivations thereof, to be put forth in much tighter and more precise terms than are commonly used. Bluntly the idea is that by aiming to reduce ambiguity, better software will result faster. Riding on the coat-tails of this are programming language libraries (Cucumber is by far the dominant one) to make Gherkin statements executable and thus testable in some limited sense.
Criticism
With good timing James Bach (also at the Meetup) had recently weighed in with some strident criticism of the Behaviour-Driven Development (BDD) methodology that Cucumber is a tool for.
Realisation
I realised during Simon’s presentation that Cucumber tries to position itself at the intersection of Analysis, Development and Testing (visualise a Venn diagram of these roles) and is therefore essentially a compromise between all three.
To quote from Cuke4Ninja (page 13 of the PDF):
Cucumber is a functional test automation tool for lean and agile teams. It supports behaviour-driven development, specification by example and agile acceptance testing. You can use it to automate functional validation in a form that is easily readable and understandable to business users, developers and testers. [my emphasis]
The point I raised during the Q&A was that since each group already produces their own form of documentation particular to their viewpoint (Business Analysts write requirements specifications, Developers have their code, Testers produce test plans, etc) how can any documentation possibly sit between all three and not be compromised? Who would author such documentation? Who would own it? And who would realistically countenance adding what would be a fourth layer to the mix when documentation is hard enough to do anyway? It is yet another set of documents to be kept in synch with the true requirements, which are hard enough to ever nail down.
Caution
If BDD is half of what it’s cracked up to be, such an investment might be worthwhile. There are some interesting concepts there and not all are without merit. But it is the nature of people to overstate the worth of something new so we need to exercise caution while at the same time being open to new ideas. I’m reminded once again of Fred Brooks’ paper No Silver Bullet which still stands as an undefeated caveat to the entire software industry 25 years since it was published.
Futher Work
I leave you with these notes I jotted down after the session. Despite the scepticism above, I still intend to do this, though I admit my enthusiasm has been somewhat tempered:
Cucumber/Gherkin
– learn it
– apply it
– report on your experiences
– bask in the warm glow/icy frost of your peers
Watch this space.
I’d say Cucumber is a tool like any other. If you write bad code you get a bad or unmaintainable project, same for Cucumber. It’s a specification layer parser to focus development on meeting user requirements – yes it doesn’t MEAN exactly what it SAYS that it’s doing, and you can easily lie or fudge the results, but you can do that already. Cucumber requires the Agile mindset of fleshing out user stories without specific use case documentation. It’s a syntactic sugar layer. Syntactic frosting (if nobody else has thought of that term, I claim it now). I think that Agile groups like it because it applies directly to user stories, and I think that Cucumber attempts to substitute documentation rather than serve as a “information nexus” between departments. That is to say Gherkin becomes the test plan, and the user stories, and the requirements doc all in one – so it’s not a fourth layer (in this sense), but a combination of three from which each group can parse information (via Cucumber/webdriver/etc for testing, via eyes for the tester/development combo that is an Agile dev)
Let us know how you get on, though, I haven’t actually used it for a project
If it’s to be a replacement for all three layers, rather than a fourth as I perhaps erroneously claim, then that still means getting three sets of people to adopt an approach new to them all.
That’s a tough call so the benefits had better be dramatic.
Perhaps that’s why Agile groups leap on it – they change their practices to Agile, but the old methodologies of documentation and clear specification go out of the window and the story-to-product flow becomes difficult to communicate, which would give Cucumber a very nice place to live that all of the groups can work from. To be honest I think Cucumber is a development tool first and foremost. I think there are some that claim that the tests used to develop the software should be thrown away after production (i.e. not used for regression), but I’d have to check on that.
I think it would be very easy to fail at implementing Cucumber into the workflow without the whole dev/testing team very much on board.
Confused. Why are the BA’s still writing reqt specs ? And what does a tester writing a test plan have to do with this ?
> Why are the BA’s still writing reqt specs?
Because unless all parties adopt BDD you’re going to have a duplication of effort in terms of generating documentation. It would appear to be an all-or-nothing approach. It depends on what kind of team you work in but I think the transition to BDD could be too great for some (that doesn’t include me, however).
I’m new to this so have no direct experience to draw upon. If you do, please enlighten me.
BAs are writing the specs to deal with devs writing software that fits the requiements of the devs rather the requirements of the customers. Cucumber is not a testing tool, it only generates automated acceptance tests so that the devs code to the behaviour (functionality) of the system. Testers don’t write gherkin, development do.
And the requirement specs are not just written by BAs, they’re supposed to be written collaboratively by the team.
I’m not trying to defend the use (or non-use) of anything, but I think it’s very important that we call a spade a spade before we use it to serve soup.