SEMAT Blog SEMAT Blog

Back

Semat – A Status Report on The Kernel

Authors: Dave Cuningham, Michael Goedicke, Ivar Jacobson, Mira Kajko-Mattson, Paul Mc Mahon, Richard Soley, Ian Spence.

Note: The work presented here will continue within the responsibility of OMG.

Summary

Semat was started as an initiative to revolutionize software engineering “based on a solid theory, theory, proven principles and best practices, that includes a kernel of widely-agreed elements.”

We have developed software for more than 50 years.  Independent of what kind of software we build, the specific application we are developing, the size of the software, the organization tat develops it (in-house, outsourced, offshored), and the practices being used, there is a common ground, a kernel of elements always prevalent in any software methodology.  This kernel provides the essential elements of software engineering.

The Semat initiative is clear about the search for a kernel that is agnostic to any particular method or any practices.  It is a base to describe methods in use to deliver real software; it includes practices for developing software, extensible for new uses, new technologies, new experiences, and new innovations. Such a kernel, standing on a solid theoretical ground, once widely accepted will change the software engineering field.  A common “practice infrastructure” will enable software developers to more quickly understand, compose and compare individual practices and entire methods.  It will form the basis for the appropriate governance of software organizations; it will provide appropriate freedom to the developers to use their preferred practices, melded with those of their organizations; it will allow us to evaluate and validate comparable entities; it will guide research to useful results; it will change the way that software developers are educated; and it will reduce the fashions and fads prevalent in the software methodologies and decision-making of today.

The kernel is not a new unified methodology, it is not a new software process meta-model, it is not a new body of knowledge, it is not a new modeling language, and it is not a trick to get people to build or buy more tools. The kernel is as simple as what we already have (such as teams and projects), what we already do (such as specify and implement), what we already produce (such as software systems) when we develop software independent of whether we document it or not, independent of what approach we use when we do it, independent of whether the result is good or bad.  The kernel is concrete, focused and small. Never before has the software community built something like this kernel.  It is as fresh as tomorrow, but it stands on the experience of the knowledge we as a community have captured over many decades.

Thus a widely agreed upon kernel is the beginning of Semat.

1       Prerequisites

The following documents are heavily referenced in this report.  Reading these documents before reading this document will greatly facilitate understanding what we have been doing and where we are going.

  1. Call for Action statement [1]

  2. Vision statement [2]

  3. Alpha definition: Modeling project state through Abstract-Level Project Health Attributes [3]

2       Background

Semat was founded a year ago by Ivar Jacobson, Bertrand Meyer and Richard Soley (the Troika) who presented a call for action statement [1] which essentially said that a) “Software engineering is gravely hampered today by immature practices …” and b) “We support a process to refound software engineering based on a solid theory, proven principles and best practices…”

The statement has been endorsed by 35 signatories, all well-known individuals in the field of software engineering, 12 corporate and academic signatories representing organizations such as IBM, Microsoft, Fujitsu, Ericsson, ABB, Samsung, Peking University, and supported by close to 1400 individuals.

In Feb 2010, the Troika also presented a Vision statement [2] which outlined a solution and which laid out the work for the next 12 months.  This vision statement was adopted at the first Semat workshop in Zurich in March 2010 [4], and is guiding our work.

Since March we have been working in a number of tracks and we have had two more workshops, one in Washington DC in July [5] and another one in Milan at the end of September [6].  Since March we have focused on results and reduced our time spent on outward communication.  This is of course necessary to deliver on our goals within a tight schedule and with limited time to spend on this initiative. People who work on the initiative are all volunteers with their “day work” to do as well.  Since we now have come quite some way on the road we want to share what we have done so far with everybody.  This document intends to give a picture of what has been done so far, where the work is heading, and references other documents to complete the picture.

3       The vision – in one sentence

We will create a platform (the kernel) allowing people to describe their current and future practices and methods so that they can be composed, simulated, applied, compared, evaluated, measured, taught and researched.

4       Problem addressed by Semat

The original Semat Call for Action gave a broad definition of the problems that the Semat initiative is poised to address (here abbreviated for simplicity, see [1]):  We look like a fashion industry; we lack a theoretical basis; we have a huge number of methods which are hard to compare; we need experimental evaluation and validation;, there exists a split between industry practice and academic research.

As a consequence, we as a society are not moving forward with reasonable speed.  Exaggerating a little: we follow a zig-zag path in evolution; we don’t stand on previous work; every new good idea is accompanied by a redefinition of many already existing terms; we don’t reuse what we have already learned; in many cases universities teach software engineering only by examples (UP, XP, Scrum) and not with a solid theoretical base; research projects are not anchored in problems identified by the industry; the industry is ignorant of research that could help them; we don’t know how to distinguish good practices from bad practices; we can’t compare practices.   For example, individual practitioners get trapped into one particular method adopted by one organization and cannot easily move to another organization which has adopted a different method.

Summarizing the situation, the current status of the software engineering society costs industry and academics dearly in lost opportunities. Industry seeks ways to develop software systems that are dramatically better, faster and cheaper. Academics look to provide significantly improved education and research aligned with the practical needs of Industry.   When the troika suggested the call for action, we were asking people to join us to revolutionize software engineering.  These are strong words, but they match our conviction.

5       Key solution

In our call for action we envisioned that a solution should be “based on a solid theory, proven principles and best practices that:

  • Includes a kernel of widely-agreed elements, extensible for specific uses

  • Addresses both technology and people issues

  • Are supported by industry, academia, researchers and users

  • Supports extension in the face of changing requirements and technology”

The vision statement and the work since the vision statement was presented have developed these ideas into something more tangible:

  1. A method is a composition of practices as opposed to an interconnection of process components (discipline, or similar). Practices give value one by one, they are what users want to make lean, and they provide interesting measures, all of which is the critical differentiator.

  2. All methods have something in common – a kernel (a common ground).  The kernel consists of two things: the universals, and the kernel language

  3. The primary users of methods and practices are project participants (developers, testers, project leads, etc.). Process engineers are also users.

  4. Methods need theory – our work must stand on a solid theoretical basis.  Methods being composed of practices, practices being described in terms of universals and elements such as activities and work products; all formalized into a language is the beginning of such a theory.  Individual practices being supported by theories take our work across the whole research in software engineering.

  5. Methods are enactable and executable (or operational).  Methods can be queried to discover guidance aligned with where you think you are today and help you get to where you want to go tomorrow; they are not just descriptions for developers to read, they are dynamic, supporting your day to day activities. This turns the concept of a method upside down.  Method is not just a description, but what is actually done.

Our solution as outlined here is fundamentally different from earlier approaches, such as SME, SPEM, OPF, EPF, UP, SWEBOK and CMMI.  We still believe that earlier works will play a significant role in Semat. However, since Semat has a different basis, we will have to complement these earlier works in the context of this new basis.

To reiterate, it is important to make clear that Semat is not about developing a new unified methodology.  It is instead about finding the common ground of all methods and practices.  Semat is not about evaluating methods and practices, but about facilitating evaluations in the future.

The kernel is at the core of our solution

Our solution relies on the discovery that underneath all methods is a kernel including the essential elements (the universals) of software engineering. The kernel also includes a kernel language to be used to precisely describe the universals as well as practices and entire methods [2].

For the time being the following universals have been identified but more will come: Opportunity, Stakeholder Community, Requirement, Software System, Work, Team, Method, and Practice.  These are all what we call alphas [3].  In the months ahead other kinds of universals will be identified such as activity spaces, competences or skills, etc.

6       The products of Semat

Semat needs customers and users.  This is why we have diligently worked to get corporate signatories and supporters involved. We need to know that we are doing the right things and that we are doing them right.

Our requirements are the starting point. These must be anchored among our corporate signatories.  They need to be agreed upon by the developer community. The requirements of the Semat initiative as a whole are captured as use cases allowing us to understand the problem and to assess it.  From these requirements we have also derived the requirements on the kernel. Thus, we are also specifying a use case model for the kernel and a related domain model.

With the requirements for the kernel as a start we identify and describe the universals and we design the Kernel Language with two related models; a meta-model and a formal definition. The meta-model uses UML class diagrams to describe its abstract syntax and OCL (Object Constraint Language, also part of UML) to describe its well-formedness rules.  The operational semantics is described in narrative form.  The formal definition defines the language in a mathematical form using Graphs and Transformation rules.  The Kernel Language has also a concrete graphical syntax, which of course is what the users apply when specifying methods, practices and universals.  Finally, to assess the work an assessment framework is being developed.

Work is now occurring iteratively on all these work products.  From a list of 250+ practices identified from different forum we selected a prioritized list of six practices to drive our work: Scrum, Iterative Development, User stories, Use Cases, Agile Requirements Management, Risk-Value Life Cycle.

We also prioritized the use cases that would also drive our work: Define Practice, Compose Practice, and Enact Practice.

For our first iteration, just completed, we did an architectural spike through all our work products resulting in sketches of two practices: Scrum and Iterative Development. This exercised the use case Define Practice and the domain model, referenced the relevant universals, drawing on the Meta model and produced a sketch of the practices by utilizing the concrete syntax.  The goal of the iteration was to synchronize and drive through consistency between the work products.  The report of all this work will be part of the group’s response to OMG’s RFP:

Work on the next iteration, also an architectural spike, has started.  The goal has not changed compared to the goal set in the vision statement.  We work on a kernel with a set of universals and a kernel language to be used to describe methods, practices and universals.  At that time we also plan to have a user manual for the language and a few important practices defined.

References:

[1]    Ivar Jacobson, Bertrand Meyer, and Richard Soley: “Call for Action: The Semat Initiative” Dr. Dobb’s Journal December 10, 2009. Online at http://www.drdobbs.com/architecture-and-design/222001342

[2]    Ivar Jacobson, Bertrand Meyer, and Richard Soley: “The Semat Vision Statement” online at http://semat.org/documents/20181/27952/SEMAT-vision.pdf/16059a36-a0ba-4405-b883-4a11a2131cea

[3]    Bertrand Meyer: “Modeling project state through Abstract-Level Project Health Attributes” July 10, 2010 online at xxxxxxx

[4]    The 1st Semat Workshop Report March 17-19, 2010 Zurich. Online at:
http://semat.org/documents/20181/27952/Zurich_meeting_report.pdf/e4c13d7b-c53d-466a-a9ad-77746f0067c8

[5]    The 2nd Semat Workshop Report, July 13 – 14, 2010 Washington D.C. Online at: http://semat.org/documents/20181/27952/2nd_Semat_Workshop_Report.pdf/f35f1ecb-bdae-44d7-8b6b-9e8553a1dae4

[6]    The 3rd Semat Workshop Report. September 29 – October 1, 2010. Milan Italy. Online at:

http://semat.org/documents/20181/27952/3rd_Semat_Workshop_Report.pdf/846f7248-d0af-46f2-a52a-4b812389ce6d

[7]    Ivar Jacobson and Bertrand Meyer: “Methods need theory” Dr. Dobb’s Journal, August 06, 2009. Online at http://www.drdobbs.com/architecture-and-design/219100242

[8]    Ivar Jacobson and Ian Spence: “Why we need a theory for software engineering” Dr. Dobb’s Journal, October 02, 2009. Online at http://www.drdobbs.com/architecture-and-design/220300840

Comments
No comments yet. Be the first.