A not-so-formal Report on Initial Observations and Experiences with the SEMAT Framework

by: Arthur Evensen                                                                  

Raison d'écrire: Professor Pekka requested a report following his discovery of my group's usage of the SEMAT alpha state cards in our TDT4140 software engineering project. We have used the cards to play a variant progress poker and then use the results to drive all – hopefully – aspects of our project forwards. This has generated lots of communication and improved group morale.

I believe a preliminary note on the extent of my knowledge as regards software development methodologies: I have read three books by 37signals (Getting real, Remote and Rework), the first third of Fred Brooks' The Mythical Man-Month, the majority of Henrik Kniberg's Scrum and XP from the Trenches – 2nd ed, the agile manifesto, and have a passing familiarity with the irreverent and partly satirical website programming-motherfucker.com. To summarize: As a second year student at the bachelor level, I most certainly don't qualify as an expert – but I have read on software development of my own volition.

How we played Progress Poker
Ivar Jacobsen's SEMAT presentation, where he mentioned the alpha state cards, inspired me to order a deck for myself. When the deck arrived I eagerly got my group aboard playing progress poker, but then came across a hurdle: Progress Poker per se requires several decks. Forced to improvise, we identified the simultaneous presentation of cards as the main idea behind the game, and as such laid the cards out on the table (following the format of Chase the State), gave everyone time to decide on a card, and then counted down from three and simultaneously pointed at our selected card(s). This gave a good simulation of progress poker, I think.

We typically pointed to two adjacent cards, though at the extreme one of us (from a group of four) pointed at a state four card, while the rest pointed at a state two card. Some players faced confusion regarding whether to select the card which required work (AKA some checkpoints unchecked) or the highest state card fully checked out (AKA highest finished state).

A Minor !Digression
In The Mythical Man-Month Fred Brooks points out that as a software project grows, a significant communication overhead also develops. Judging by the high failure rate of projects in general, I think it safe to say that teams rarely consist of only excellent communicators. Interestingly, then, we have primarily used the alpha states to spur discussion so far. Some of that discussion also contained actions, for example when we discussed our governance rules.

Several group members have expressed their enthusiasm towards the use of the alpha state cards to direct our discussions, saying that it added a level of formalism they enjoyed. As such, we have used the cards as scaffolding for our conversations, and to help us drive our work forwards.

However, the act of playing progress poker and arriving at a conclusion as to which state we currently inhabit had another effect as well: It forced us to re-talk areas we had previously talked about. It turned out our understanding of those areas differed, so while we had spent time talking those topics and themes over before, the conclusions we had drawn as individual members of the team differed from each other. This means we had had an illusion of communication, where we each thought everything had been sufficiently and clearly communicated. This clearly represented a fallacy, since understanding differed so much. Thus, the use of the alpha state cards helped us move from thinking we communicated towards actually communicating. (I guess, using the field of phenomenology as reference, that one can only approach full communication asymptotically.)

Critical Interfaces Demonstrated have Not Been Demonstrated
We found interpreting some of the checkpoints of the cards a challenge. The words used fall within the fairly abstract and general – probably correctly so – but we didn't find any help in semat.org/quick-reference-guide for understanding the checkpoints. Many of the 'explanations' or 'translations' had the approximate concreteness of a Newtonian fluid. I think a single example will suffice: When we looked up “Critical interfaces demonstrated” we found the not-so-helpful description “Critical interfaces have been demonstrated”. Ahem.

A Handsy Experience
When professor Pekka first presented the SEMAT kernel in class, with its seven alphas, I found it a confusing model. Certainly, I could see that all relevant activities to software development ought to sort into the seven alphas with some ease (the unease coming from certain convoluted activities that might fit into more than one alpha). However, I could not readily see how to use the kernel in any capacity. A nifty descriptive model, for sure, perhaps even lightly normative (since it implied that any quality project ought to not leave any alpha out of focus). But I could not see how to turn the kernel into something applicable right away.

The SEMAT alpha state cards worked wonders in bringing the kernel to life, and turning it from a descriptive model to an applicable set of actions. We have managed to accomplish most of those actions-undertaken-so-far by internal discussion in our team, where for a larger more industrial project we would have required bringing in other stakeholders for example. As teased in the introduction, the most marked changes have come about as a result of laying out governance rules and leadership model: in the true fashion of NTNU students we have implemented a dessert punishment system for meeting latecomers, where they have to bring dessert to the next meeting. I have also observed increased morale after our leadership model became explicit – it didn't change from what I previously implemented, but the model I had made up received sanction by the group as a whole. (Leader sets meeting agendas, and has decisive vote in cases of otherwise equal number of votes for-or-against something. Tipping vote?)

A Temporary Verdict on the SEMAT alpha state cards
Use of the alpha state cards has improved group morale, made us focus on under-prioritized areas, forced us to communicate and make sure everyone has a (mostly) shared understanding of our project and where we stand in it, given our meetings some formal frames to operate within, and – I think – added some lighthearted fun to our approach. The cards have greatly satisfied me and I would gladly buy another deck if I lost my current one.

An Inelegant Segue
I request payment for writing elegant segues, so I now inelegantly segue into a discussion of SEMAT!

SEMAT in Contrast with, say, Scrum
Scrum seems a traditional software method, much like (the much misunderstood) waterfall method: it presents a set of practices which it holds up as the way to develop software efficiently. I find this comical because it seems to me that everyone should find it self-evident that a method cannot be fit for each and every project, since projects have different goals and requirements.

If we define method similarly to how Merriam-webster's online dictionary, then we can say that a method is a procedure or process for doing something. If the something differs, obviously the method ought to differ to! No one tries to bake a cake the same way that they practice sportsball. As circumstantial evidence in favour of this, I will quote professor Pekka: “No one uses Scrum anymore. They all say 'Oh, we use Scrum: but-...'. They are using Scrumbut. Or Scrumban.'

SEMAT however, strikes me as a framework, not a method – it points out the key areas of a software project, tells you to focus on them all to avoid your project lacking in key areas, and then takes a step back. “Oh, by the way,” SEMAT says after you have had time to breathe. “You'll need a method to work with. Want to go look through my practice library? Perhaps see someone else's methods for inspiration and evaluation?”

In other words, SEMAT recognizes the generality of methods and goals, and the impossibility of making any one true method™.This allows developers to focus on development and finding out what works for the project in question, instead of trying to blindly follow a method prison developed by some guru(s).

A Tiny Personal Insight
A team measures success during a Scrum retrospective, strictly speaking, by how closely they have followed the Scrum method. The main reason to have a method at all is to avoid cowboy coding. During the Scrum course last semestre, this lead to rather trivial and blunt retrospectives: “What worked?” “Well, we Scrummed some.” “What should we do next sprint?” “Scrum even more?” “Alright. What didn't work?” “Maybe we didn't Scrum enough?”

In the third alpha state card for Way of Working – In Use – there are two checkpoints of particular interest to me: “[Practices & tools] regularly inspected” and “Feedback mechanisms in place”. This means that at a SEMAT retrospective we can evaluate not only whether we have successfully used our way of working, our method, but whether our way of working works.

The result of this is that while developing a project, you also develop – as a byproduct, if you please – a software development method based around the SEMAT kernel which is, hopefully, fit for purpose and the team itself. This means that the team efficiency ought to constantly improve as their way of working becomes refined and improved upon. There's an iterative development of the software method itself, which should become more and more fit for the particular project! It also seems to have particular use in larger projects, since the method would then have more time for refinement. (And it's precisely in larger projects that methods become important!)

I think there's some beauty and elegance in that.