Extract from the main conclusions of the official report into the failure of the London Ambulance Service 's Computer Systems on October 26th and 27th and November 4th 1992.
4.0 STATIC TESTING
4.1 Overview
Static testing techniques is used to find errors before software is actually executed and contrasts therefore with dynamic testing techniques that are applied to a working system. The earlier we catch an error, the cheaper it is, usually, to correct. This module looks at a variety of different static testing techniques. Some are applied to documentation (e.g. walkthroughs, reviews and Inspections) and some are used to analyze the physical code (e.g. compilers, data flow analyzers). This is a huge subject and we can only hope to give an introduction in this module. You will be expected to appreciate the difference between the various review techniques and you will need to be aware of how and when static analysis tools are used.
4.2 Objectives
After completing this module you will:
Understand the various review techniques that you can apply to documentation and code.
Appreciate the difference between walkthroughs, formal reviews and inspections.
Understand how static analysis techniques can detect errors in code.
Understand two complexity metrics (lines of code and McCabe's metric).
4.3 REVIEWS AND THE TEST PROCESS
4.3.1 What is a review?
A review is a fundamental technique that must be used throughout the development lifecycle. Basically a review is any of a variety of activities involving evaluation of technical matter by a group of people working together. The objective of any review is to obtain reliable information, usually as to status and/or work quality.
4.3.2 Some history of reviews
During any project, management requires a means of assessing a measuring progress. The so-called progress review evolved as means of achieving this. However, results of those early reviews proved to be bitter experiences for many project managers. Just how long can a project remain at 90% complete? They found that they could not measure 'true' progress until they had a means of gauging the quality of the work performed. Thus the concept of the technical review emerged to examine the quality of the work and provide input to the progress reviews.
4.3.3 What can be reviewed?
There are many different types of reviews throughout the development life cycle. Virtually any work produced during development can be (and is) reviewed. This includes, requirements documents, designs, database specifications, designs, data models, code, test plans, test scripts, test documentation and so on.
4.3.4 What has this got to do with testing?
The old fashioned view that reviews and testing are totally different things stems from the fact that testing used to be tacked onto the end of the development lifecycle. However as we all now view testing as a continuous activity that must be started as early as possible you can begin to appreciate the benefits of reviews. Reviews are the only testing technique that is available to us in the early stages of testing. At early stages in the development lifecycle we obviously cannot use dynamic testing techniques since the software is simply not ready for testing.
Reviews share similarities to test activities in that they must be planned (what are we testing), what are the criteria for success (expected results) and who will do the work (responsibilities). The next section examines the different types of review techniques in more detail.
4.4 TYPES OF REVIEW
Walk-through, informal reviews, technical reviews and Inspections are fundamental techniques that must be used throughout the development process. All have their strengths and weaknesses and their place in the project development cycle. All four techniques have some ground rules in common as follows:
A structured approach must be for the review process.
Be sure to know what is being reviewed - each component must have a unique identifier.
• Changes must be configuration controlled.
• Reviewers must prepare.
• Reviewers must concentrate on their own specialization.
• Be sure to review the product, not the person.
There must be:
Total group responsibility.
Correct size of review group.
Correct allocation of time.
Correct application of standards.
Checklists must be used.
Reports must be produced.
Quality must be specified in terms of:
Adherence to standards.
Reliability required.
Robustness required.
Modularity.
Accuracy.
Ability to handle errors.
4.5 REVIEW TEAM SIZE AND COMPOSITION
Problems with small teams must be avoided; bring in extra people, (perhaps use the Testing Team) to bring extra minds to bear on the issues.
Opinion is often divided as to whether or not the author should participate in a review. There are advantages to both scenarios. Since specifications and designs must be capable of being understood without the author present, an Inspection without them tests the document. Another reason for excluding the author from the review is that there should be team ownership of the product and team responsibility for the quality of all the deliverables, maintaining ownership via the author runs against this.
Alternatively, including the author can be a valuable aid to team building. Equally, an author may well be able to clear up misunderstandings in a document more quickly than another team member, thus saving the reviewer valuable time. From a practical perspective however, it is worth remembering that an author is the least likely person to identify faults in the document.
The one person who should not attend reviews is the manager. If, as in some cases, the manager is also a contributor to the deliverables, he should be included but treated the same as other group members. It is important that reviewers are a peer group process.
4.6 TYPE 1, 2 AND 3 REVIEW PROCESSES
Review process is both most effective and universal test method and management needs to make sure that review process is working as effectively as possible. Useful model for manager is 1,2,3 model.
1, 2, 3 model is derived from work of first working party of British Computer Society Specialist Interest Group in Software Testing and book that came from this work; in Software Development.
Type 1 testing is process of making sure that product (document, program, screen design, clerical procedure or Functional Specification) is built to standards and contains those features that we would expect from the name of that product. It is a test to make sure that produce conforms to standards, is internally consistent, accurate and unambiguous.
Type 2 testing is process of testing to see if product conforms to requirements as specified by output of preceding project stage and ultimately Specifications of Requirements for whole project. Type 2 testing is backward looking and is checking that product is consistent with preceding documentation (including information on change).
Type 3 testing is forward looking and is about specification of certification process and test that are to be done on delivered product. It is asking question - Can we can build deliverables (test material, training material, next stage analysis documentation)?
4.6.1 Make reviews incremental
Whilst use of 1, 2, 3 model will improve review technique, reviewing task can be made easier by having incremental reviews throughout product construction.
This will enable reviewers to have more in-depth understanding of product that they are reviewing and to start construction type 3 material.
4.6.2 General review procedure for documents
Test team will need general procedure for reviewing documents, as this will probably form large part of team's work.
1. Establishing standards and format for document. 2. Check contents list.
3. Check appendix list.
4. Follow up references outside documents.
5. Cross check references inside document.
6. Check for correct and required outputs.
7. Review each screen layout against appropriate standard, data dictionary, processing rules and files/data base access.
8. Review each report layout against appropriate standard, data dictionary, processing rules and files/data base access.
9. Review comments and reports on work andreviews done prior to this review.
10. Documents reviewed will range from whole reports such as Specification of Requirements to pages from output that system is to produce. All documents will need careful scrutiny.
4.6.3 Report on review
Report should categorize products:
Be sensitive to voice of concerned but not necessarily assertive tester in team; this person may well have observed a fault that all others have missed. It should not be that person with load voice or strong personality is allowed to dominate.
4.7 INSPECTIONS
This section on Inspections is based on an edited version of selected extracts from Tom Gib and Dorothy Graham's book [Gib, Graham 93].
4.7.1 Introduction
Michael E. Fagan at IBM Kingston NY Laboratories developed the Inspection technique. Fagan, a certified quality control engineer, was a student of the methods of the quality gurus W. Edwards Deming and J. M. Juran.
Fagan decided, on his own initiative, to use industrial hardware statistical quality methods on a software project he was managing in 1972-74. The project consisted of the translation of IBM’s software from Assembler Language to PLS, a high-level programming language assembler. Fagan's achievement was to make statistical quality and process control methods work on 'ideas on paper'. In 1976 he reported his results outside IBM in a now famous paper [Fagan, 1976].
The Inspection technique was built further by Caroline L. Jones and Robert Mays at IBM [jones, 1985] who created a number of useful enhancements:
The kickoff meeting, for training, goal setting, and setting a strategy for the current inspection
Inspection cycle;
The causal analysis meeting;
The action database;
The action team.
Continue....
No comments:
Post a Comment