Title: Department of Computer Science Seminar
Date: Feb. 14, 2000
Time: 10:00 am to 11:00 am
Venue: Room N101, CSIT Building [108]
Speaker: Dr Clive Boughton (DCS at ANU)
Description: "Learning from Mistakes"


                        Abstract

Many so-called "software engineers" are dissuaded 
from reviewing a software development because 
it isn't exciting and they really don't believe 
that anything could be learned from such a task.  From my
experiences it seems that the great majority of 
"software engineers" over this last decade or 
so seem never to have had a desire to
research/investigate real data in order to discover better ways 
to develop software. On the other hand, there are plenty 
of "software engineers" who are prepared to uptake apparently 
new and untested proposals for making (apparent) improvements.  
Nearly all such proposals for improvement are technically 
oriented - this is usually not where
significant gains are made to the improvement 
of software developments. Only recently has there been 
some proposals backed-up with research
findings that are not purely technical in nature - 
eg. the Capability Maturity Model (CMM), 
the Personal & Team Software Process (PSP & TSP). 
Some of these non-technical approaches 
to improvement have yielded (measurably) 
quite high return on investment.

Improvements can only be realised 
when objectives are defined and appropriate measurements 
are effected to observe progress toward those
objectives.  However, before any objectives 
can be treated seriously and a consequent improvement program 
implemented, it is essential to determine the base 
from which one is launching into improvement
paradise.  The operative clause is - "Know thyself!" 

The CMM is a formal framework for helping to 
identify improvement strategies in software development.  
However, the assessment processes that are typically 
used to formally establish the maturity of an
organisation and therefore to identify possible improvements 
that can/should be implemented, are generally considered 
overkill unless an organisation is undertaking multiple 
software developments as a major part of its business.  
Most small software development companies cannot afford 
a CMM-based assessment, but then most small software development
organisations often don't do any form of assessment.  
Alternatively if they do, it's usually much too late.

So, on the one hand there is a good formal, 
but expensive option to evaluate the software development 
capability of an organsiation.  On the other there seems 
little inclination to make improvements in development.  
However, there is an increasing awareness by the software
acquisition community that it is important to 
obtain some measure of the capability of the software development 
companies they typically employ to effect projects on their behalf. 
Nonetheless, even the software acquisition organisations 
only become concerned about capability when it's all too late.

Consequently, over the last decade 
I have become increasingly involved
with "software evaluations".  
Typically, I or my company have been
approached to review the overall worth 
of the various products (artefacts) being delivered 
to a customer who has used, or is using, a
(sub)contractor to implement a set of software requirements.

As more information is collected for an ever increasing number of
developments that I and my company colleagues have "evaluated", 
we see recurring patterns emerging - 
irrespective of the type of development.  

Software evaluations are only effective 
when the evaluator or evaluator team is highly skilled (technically) 
and possesses considerable knowledge of software development 
and/or acquisition processes.  The evaluator (team) 
must also be very sensitive to the political
environment typically surrounding an evaluation.  
Hence the evaluation task is not for the immature.

An overview of the typical approaches that 
have been devised (by me) for undertaking an evaluation 
will be presented and discussed.  Some of the typical outcomes 
will also be presented and discussed together with some
of the ramifications and consequent reactions 
that customers have had toward the outcomes.



URL: http://cs.anu.edu.au/lib/seminars/seminars00/dept2000021