Product of an inspiring walk around campus with my colleague
Lee Martie.
What is a theory in Software Engineering?
Regardless of scientific field, a theory is a model used to predict and to explain phenomena. In Software engineering these types of models are respectively called by many authors (e.g., [Sca02] and [PW92]) as predictive and descriptive models.
How do we evaluate these models in software engineering?
A predictive model is evaluated by contrasting the expected outcomes against the actual outcomes of a software process/method. For that, outcomes must be described in a way conducive to measure and to compare results. Such description of outcomes requires a descriptive model (e.g., which set of metrics will be used to evaluate the outcome of this new software process?).
A descriptive model is evaluated by looking at the steps taken to produce an outcome. For instance, how can someone tell whether a requirements model is consistent with user goals? For that, one has to look at how requirements were elicited and validated against user goals. Such steps are part of a prescriptive model of goal-based requirements engineering.
How to evaluate if my tool has actually led me to a theory?
I believe that a “theory of a tool” is a model for a family of tools. As such it must be compared against other competing models. How to make that comparison? Karl Popper wrote that the measure of quality of a theory is how falsifiable that theory is [Pop63]. It does not mean that my model is more complete. Rather means that my model provides the theoretical or empirical means to demonstrate that under certain circumstances (e.g., usually, inputs and preconditions in a usage scenario) a certain outcome is expected (deterministically or probabilistically).
In other words. If my model claims that a set of features cause certain outcomes to happened, for models which don’t have the same features, one cannot refute these models by setting the same circumstances (inputs) and predicting the same outcomes.
Of the many tool features, which can actually be articulated in a model? (See picture below)
The features must be grounded in reality, thereby we can rely on empirical evidence produced by us or by previous research. The features must relate to the ontology of our family of tools. In other words, features must be intelligible to people who are acquainted to the family of tools. The importance of that is twofold. First, to guarantee that the model is understandable, therefore passive to be reified in a tool. Second, reduce the risk that the problem that the our model aims to solve was not already solved by a simpler solution (Occam Razor).
References:
[Sca02] Walt Scacchi. Process Models in Software Engineering, in J. Marciniak (ed.), Encyclopedia of Software Engineering, 2nd. Edition, Wiley, 993-1005, 2002
[PW92] Dewayne E. Perry and Alexander L. Wolf. Foundations for the Study of Software Architecture. ACM Software Engineering Notes 17(4):40-52, October 1992.
[Pop63] Karl Popper. Conjectures and Refutations: The Growth of Scientific Knowledge, 1963, ISBN 0-415-04318-2
Christian M. Adriano 