12th International Conference on Software Quality

Software Division of
The American Society for Quality

Home ] Up ] 13ICSQ Program ] Sponsor & Exhibitor Opportunities ] Special Needs ] Dallas Info ] Hotel/Airline/Rental Car/Driving Directions ] Paper & Tutorial Presenter Information ] Volunteers Opportunities ] ICSQ History.htm ] Contact Us ] About ASQ ]


Tutorial Descriptions
Tuesday Presentations
Wednesday Presentations
Tim Lister
John Musa
Alain Abran
Scott Ambler
Kevin Daily
Alec Dorling
Liz Keim
Tom McCabe
Linda Westfall


Technical Presentations - Wednesday, October 30, 2002:

Many definitions of software usability exist, making sometimes usability a confusing quality concept and not easily measurable compared to other quality factors. Generally speaking, usability of software refers to its ease of use, and its ease of learning. In this paper, we first define usability as an external software quality attribute – The user perspective of software quality or quality in use. Then, we present QUIM (Quality in Use Integrated Measurement/Map), a framework for specifying, modeling, quantifying and measuring quality in use. In particular, we will see how QUIM Editor, a software tool we developed, can help developers to model and measure usability goals for a user interface. 

Ahmed Seffah is an assistant professor of software and usability engineering. He is the Concordia research chair on human-centered software engineering. Dr. seffah is a member of the IFIP TC 1.2 working group on human-centered design methodologies and the co-founder of the Montreal chapter of the usability professional association (UPA). Dr. Seffah research interests and industry skills lie in the intersection between Human-Computer Interaction and Software Engineering. Specifically, he has been working on the following topics: usability engineering, human-centered software development, incorporating human factors in software development lifecycle, user requirements engineering, quality in use measurement and metrics, user interfaces development tools and techniques for Web, PDA and Mobile Applications. He joined in 1995 the Computer Research Institute of Montreal (CRIM) where he was a full member of the research staff for the human-computer interaction and the software development groups until 1999. From 1996 to 1999, he was also an adjunct professor at the University of Quebec at Montreal. Ahmed Seffah holds a master's degree in applied automatics and computer engineering from INSA de Lyon, and a Ph.D. in software engineering from the École Centrale de Lyon (France).

  • Makoto Nonaka presenting A Complexity-Weighted Functional Size Metric for Interactive Software 

This paper proposes a complexity-weighted functional size metric for interactive software using the user interface (UI) specification.  The UI specification can be obtained at earlier stages of the development process, as it is usually regarded as a major requirements specification in the development of interactive software.  By measuring functional size from the UI specification directly, early effort estimation can be expected. The functional size without complexity weights is also measured provisionally.  The proposed metric can be used as the complement of the COSMIC-FFP measurement method, as the provisional functional size nearly equals the COSMIC-FFP functional size. The proposed metric was applied to the interactive part of an actual car-navigation software system in order to validate its SLOC (Source Lines of Code) predictability.  The result showed that the metric could be a predictor of SLOC.

Makoto Nonaka received the BS and the MS degree in industrial engineering from Waseda University in 1995 and 1997 respectively.  He is a research associate in Department of Industrial and Management Systems Engineering, School of Science and Engineering, Waseda University.  He is a member of IEEE Computer Society, Information Processing Society of Japan (IPSJ).  He received the best paper award for young researcher of IPSJ National Convention in 2000.  

  • Session S: Metrics & Measurement

    • Pat Cross presenting The Uses and Abuses of Software Metrics 

Metrics is becoming, once again, a hot topic in software process improvement. The gathering and use of hard data is a required activity in the CMM journey at Level 3 and beyond. Yet practitioners are confused by the subject and good case studies showing defined improvement are hard to find. Many companies have invested money into metrics programs with little apparent return on investment. The author believes that it is because managers of software development and software quality are too eager to implement advanced techniques - they are trying to run before they can walk. They lack the fundamental theories of measurement and data analysis, and break rules they did not know existed. This may have the effect of triggering actions which not only may not be helpful, but may actually make results worse.

In this presentation, a very quick overview of some measurement theories and simple statistical techniques is given, with the emphasis on how they can help making good decisions and the pitfalls that must be avoided. The details of the methods and the formulas to use are not given, but good references are. The author stresses that in addition to aligning your metric with your goal, as in the Goal, Question, Metric method, the data analysis technique that is used to make decisions must be picked in advance and will affect the metrics collection.

Pat Cross has an extremely varied background. She has a B.Sc. in Physics, an M.Sc. in Textile Technology and a Ph.D in Consumer Studies. Her working life has been in teaching, technical management and consulting., mainly in the area of quality achievement. While her introduction to quality achievement was through manufacturing, she has been supporting the achievement of quality in control and IT software for the past 17 years. She was 12 years with Spar Aerospace, introducing and managing the discipline of Software Quality Assurance specifically for the Robot Arm for the International Space Station (Canadarm 2) and other projects. She presently runs London St. Consulting, supporting companies (mainly in IT) making a step-change in achieving software quality.

  • Subhas Misra presenting Heuristics for Controlling Quality of Object Oriented Systems Early in the Software Lifecycle

Devising and engineering methodologies to produce high quality software at low costs is an important concern of software quality researchers. Perhaps the most important attribute of software that characterizes its quality is maintainability, because it is the maintenance phase in the complete lifecycle of software that is the most expensive and time-consuming. Consequently, software maintainability plays an important role in improving quality of software. Predicting maintainability in the early phases of software development, viz., the design and implementation phases, should help software designers and quality controllers to alter and improve designs for better performance and as a consequence for reduced maintenance costs. 

This paper presents heuristics for assessing maintainability, and thus quality, early in the software lifecycle, from different widely accepted design-level metrics. The paper is based on a comprehensive intuitive analysis and an empirical study involving direct measurements of source-code of thirty projects. The chosen projects were written in C++ programming language and have varied characteristics. Maintainability was measured using the widely accepted “Maintainability Index” (MI) proposed by SEI (Welker, 1995). In order to generate the heuristics, a two-step analysis was performed. For performing the experimental analysis, the values of the chosen metrics were collected from static analyses of source-code delivered at the end of implementation phase of each project and the obtained value of each metric was investigated together with MI for the existence of statistically significant relationships. It was discovered that most of the metrics considered in the study, e.g., Average Lines of Code per Class (ACLOC), Depth of Inheritance Trees (DIT) and Density of Control Statements (CDENS), have strong impact on maintainability.  On the other hand, for few metrics, e.g., AVPATHS, which is a measure of the average depth of paths in the programs, no remarkable effect was observed. This led us to formulate a set of heuristics that should enable the software design and quality assurance professionals to administer quality in the early phases of software development and thus significantly reduce the software maintenance costs.  

Subhas Misra received B.Eng. degree in Electronics and Telecommunications Engineering from Andhra University and M.S. degree in Computer Science and Data Processing from the Indian Institute of Technology (IIT), Kharagpur, India. He is currently a research student in the Faculty of Computer Science in the University of New Brunswick, Fredericton, Canada, where he is carrying out research in software quality engineering. He has around ten years of experience working on R&D projects in software engineering and quality engineering. He has worked in the research wings of reputed industries, such as Nortel Networks, Ottawa, Canada and the Indian Telephone Industries, India. He has published several technical papers in different international journals.

  • Session T: Miscellaneous

    • David M. Phillips presenting CMMI V1.1: Improving Processes for Better Products

      David M. "Mike" Phillips is currently leading the CMMI Product Development Team, a joint government, industry and SEI effort to produce the CMMI Product Suite as an evolutionary improvement on the source mondels for software (SW-CMM) and systems engineering (SECM) as well as IPPD.  Before joining the SEI, Mike was responsilbe for managing development of software-intensive B-2 Stealth Bomber for the Air Force.  Mike has a bachelors degree in Astronautical Engineering from USAF Academy and masters degrees in Nuclear Enginering, in Systems Management and in International Affairs.

  • Kennet Henningson presenting Understanding the Relations Between Software Quality Attributes – A Survey Approach

Software quality is a complex concept containing a large number of quality attributes. These attributes often have different meaning for different people and different attributes are not of equal importance. Moreover, the actual relations between the attributes are mostly poorly understood. Companies have to cope with these relations in their daily software development. On the one hand, companies take management decisions based on experience. On the other hand, researchers address software quality too. However, the two views are not necessarily the same.

To increase the understanding of software quality attributes and their relations, two surveys have been conducted. The first survey focuses on the research literature and the second is an interview survey with people from industry. From these surveys, it is concluded that there is an agreement, in qualitative terms, that quality attributes are dependent. However, different opinions exist about the actual relations. No quantitative relations have been found. The main conclusion is that there is a gap between research literature that poses mostly generic relations between quality attributes and the tacit knowledge in industry. The tacit knowledge within industry is largely focused on system specific relations between quality attributes. The result from these surveys provides a compilation of relations between quality attributes that illustrates the gap between the views in industry and academia respectively. The understanding of the gap is the first step towards bringing the two views closer to each other.  

Kennet Henningsson is currently working as a PhD student at Blekinge Institute of Technology; there he completed his Master of Science in Software Engineering in 2001. His main working areas, besides research, are with the project courses at Blekinge Institute of Technology. Kennet has a interest in software engineering and the quality aspects of software engineering.

  • Exhibitor Track

    • NVP Software Testing Inc. presenting The Lighter Side of Software Automation

      This talk explores some of the more amusing things that have occurred during automation experiences in the last 10 years. These are things that have gone wrong even when they have been guarded against by all the best efforts. All experiences have occurred within NVP.

    • Exhibitor Speaker Opportunity - Slot 7

  • Session U: Invited Speaker - Kevin Daily presenting Towards Software Excellence – Capability Self Assessment for SME Developers

  • Session V: Miscellaneous 

    • Student Paper: Brad Graham presenting Mutation Testing of Hierarchical Finite State Machines

      Reactive systems are a class of systems characterized by the maintenance of a continuous and timely interaction with their environment. Testing techniques for such systems cannot be based on a functional view of the software. They must be based on some history preserving view such as one of the finite state machine formalisms. Mutation testing is a heuristic technique originally developed to encourage the construction of high quality test suites. It offers a potentially highly automatable, but computationally expensive way of creating thorough test suites. The success of mutation testing is largely dependent on the set of mutation operators defined for the particular language. The large computational cost of mutation testing has prevented its widespread use.

      This paper presents observations made while designing and using a system to perform mutation testing of reactive systems modeled by active objects whose behaviors are defined by hierarchical finite state machines (HFSM) as implemented by the Rational Rose Real Time (RoseRT) toolset. The modeling language is a variant of UML-RT. A set of mutation operators is presented and evaluated based on their use in testing a set of HFSMs. This is followed by a discussion of general observations made during the tests. Preliminary evidence is presented indicating that mutation testing of RoseRT HFSMs is an effective and feasible (with tool support) method of creating thorough test sets.

      Brad Graham received a B. Eng in computer engineering from Royal Military College of Canada in 2000. He received a Master of Computer Science from Queens University in 2001. Brad is currently a project officer working on the CF-18 modernization project in Ottawa Ontario. His interests include validation and verification of software intensive systems.

    • Jim Spigener presenting Behavior-Based Management as a Competitive Strategy

This presentation outlines the next frontier in quality management. It defines how to identify what behavior is critical to achieve the outcome measures necessary for success. It then outlines how to systematically use these critical behaviors as an upstream measure from which to initiate system improvement. This presentation also reveals the factors critical for viewing behavior management as a positive initiative.

Jim Spigener is Vice-President of Seminars and Training for Behavioral Science Technologies, Inc. (BST) where he has helped hundreds of companies worldwide prepare and implement successful behavioral systems in the fields of safety, TQM and reliability.  Before BST, Jim worked for a Fortune 500 company, as Superintendent of Fire, Security and Safety developing, implementing and coordinating a behavior-based accident prevention process.  Jim has presented at hundreds of national and international conference on the subject of behavior-based performance improvement and has authored and co-authored a nmumber of articles for industry and trade journals. 

  • Session W: Miscellaneous 

    • Student Paper: Edward Neubecker presenting Software Defect Prevention Techniques - Their Applicability in Automotive Embedded Systems

      Dependency on validation testing to find all software defects is a very expensive method of ensuring a quality software product. Awareness of this issue has created a great amount of research in methods to manage and assess software quality earlier in the software development lifecycle. These methods include monitoring process compliance; establishing and evaluating performance criteria; and detecting and analyzing software defects. Effective use of these methods can provide project managers the ability to strategically plan quality control actions such as testing, inspections or redesign. 

      The focus of this paper is on the applicability of defect prediction techniques in automotive embedded systems. This paper presents a software quality utility tree that illustrates some common quality attributes found in software systems. This graphical representation identifies the supporting factors for defect prediction techniques, their underlying measures and how they relate to software quality.  Following the utility tree is a description of a lifecycle for an automotive embedded system, and also an evaluation of four different prediction techniques: defect discovery profile; COQUALMO; empirical defect prediction; and fault-proneness. Within the evaluation I will qualify them as techniques that could be used in the field of automotive embedded systems. Finally, the most appropriate technique is recommended with justification.

Edward Neubecker is a software engineer at Visteon Corporation. He has an extensive and varied experience building software for embedded systems. His experience and interests also include safety critical software systems for the automotive industry and software quality assurance. Edward received his B.S. in Electrical Engineering from the University of Michigan and is currently a student in the Master of Software Engineering program at Carnegie Mellon University.

  • Kent Lacy presenting Requirements Management: A Quality-centric Approach

The software industry has been struggling with requirements management issues for decades. Millions have been spent on tools, training, and books - all developed to enhance our capabilities in this area. Yet we are still experiencing a high rate of project failures that are directly attributable to requirement management issues. The answer lies in our approach. Simplicity and quality are factors that will help guide us toward achieving our goal of improved performance.

Many requirements management techniques are extremely hard to learn and complex. It’s best to start simple and build in complexity based on experience and need rather than starting complex and attempting to scale back.

The requirement elicitation process requires a quality focus on understanding the customer’s wants. If we can’t articulate what our customer wants, project success will remain to be a rare event. Once customer wants are understood, we can identify customer needs and elicit clear, testable, consistent requirements. After requirements have been baselined, we move into change control. In this phase, the quality focus is on the management and control of change. Realistically, we have got to expect and plan for change. In order to accomplish this effectively, a well-documented and easy to follow change management process is required. The objective isn’t in preventing change, we simply want to measure and control change so our customers understand the impact of changes as they are recognized and requested. Tying this all together, we need the ability to verify that the customer’s requirements actually get build into the end product. This is accomplished through a traceability control process that traces requirements through design, into testing, and the final product. The quality focus for traceability control is on the linkage of requirements. Traceability is an iterative process where a matrix linking requirements, design, and tests is built in stages at specific milestones during the development lifecycle. By focusing on principles of simplicity and quality in a requirements management program we can improve performance and achieve greater project success.

Kent Lacy is a Certified Quality Manager, Certified Software Quality Engineer, and Project Management Professional (PMP) and a with 18 years experience in the Software Industry. Currently, he works for DBI Consulting where he serves as the Practice Lead for the Project Services practice. He has a degree in Information Systems Management, an MBA, and is currently working on a Doctorate in Organization Development at the University of St. Thomas in Minneapolis. Kent is an experienced public speaker, who presented at the 10th International Conference on Software Quality, in additional to several Sterling Software and Project Management Institute Conferences.

  • Session X: Software Testing 

    • Neil Price Jones presenting Don’t Test 

Frequently there is an assumption made at Software Testing and Quality Assurance conferences that: "I test therefore I am" (with apologies to Descartes). We talk about how much we save the organization by testing the software. We prove, with facts and figures, that we benefit the organization and its public and private appearance. However, when we return to the office after a conference these ideas and assumptions do not survive long. They succumb to the pressures of the need to get the software out the door for whatever reason is put forward by the current customer - market share, jump on the competition, technical requirements, integration with a major trading partner etc.

This talk explores the reasons we are met with not to test, our responses to those attitudes and some of the standard mechanisms we follow to deal with the situation. It the proposes a 'new' solution.

Neil Price Jones has 15 years experience in software testing and implementation of automation in a variety of environments including financial, telecommunications and pension administration. 

  • Theresa Hunt presenting Basis Path Techniques for Structural and Integration Testing

Basis path testing is a structural testing technique that identifies test cases based on the flows or logical paths that can be taken through the software. A basis path is a unique path through the software where no iterations are allowed; they’re atomic level paths, and all possible paths through the system are linear combinations of them. Basis path testing uses a Cyclomatic metric that measures the complexity of a source code unit by examining the control flow structure. Basis path testing can also be applied to integration testing when software units/components are integrated together. You’ll see how the use of the technique quantifies the integration effort involved as well as the design-level complexity.

Theresa Hunt is a lead consultant for The Westfall Team.  Her specialties include Software Quality Engineering, Process Definition and Improvement, and Testing.  Prior to joining The Westfall Team, Theresa was Principal Software Quality Engineer and a member of the Engineering Process Group responsible for corporate wide improvement of systems, software, and hardware engineering at ECC International Corp. based in Orlando, Florida.  Theresa has managed ECC’s Software Quality Engineering (SQE) department and directed Software Quality Engineers in the planning and execution of project-specific software quality functions. Theresa serves as Chair of the ASQ Software Division Programs Committee.

  • Session Y: Miscellaneous

    • Scott P. Duncan presenting Implications of Applying ISO 9000-3 (2000/3)

The revision to ISO 9000-3 to update it to meet the new structure of ISO 9001:2000 is currently underway with hopes of finalizing it by the end of this year (2002) or certainly early into next year.  As this paper is being prepared, concerns still exist with regard to how software-specific the guidance actually is in various places.  What this paper will try to do is highlight what, at this point in time, seem to be some of the software-specific elements in the Final Committee Draft (FCD) version of the document.  The talk delivered at the Conference will be based on the latest version of the document at that point in time.  

Scott Duncan has over 28 years of experience in internal and external software product development with commercial and government organizations, including over 14 years in the Telecom Industry.  The last 6 years, as an internal/external consultant, he has helped organizations achieve various quality/process registration and capability assessment goals.  Scott is Standards Chair for the ASQ’s Software Division and is a member of the U.S. Technical Advisory Group for ISO/IEC JTC1/SC7 software engineering standards as well as the IEEE CS Software Engineering Standards Committee’s Executive Committee. Since 1985, Scott has provided corporate training sessions on auditing/assessment standards, conducted public seminars in managing software development through metrics, and been a speaker at national and international conferences and user groups.

  • Richard Basque presenting Transitioning from SW-CMM1.1 to CMMI-SE/SW 1.1: a walk in the park?

For organizations who used SW-CMM for some years, the transition to CMMI could represent a not so obvious challenge. Part of the problem lies in the way the CMMI was launched some years ago: it left some pain in parts of the software community and generated lots of concerns and... lots of rumors. Now, since early 2002, CMMI is available in a clean and stable version that provides a lot of advantages, even for software-only organizations. How does it address the concerns of those who were reluctant to move from SW-CMM to CMMI? What are the differences? Based from concrete experience with both, we will discuss these aspects and propose a vision for a soft transition.

Richard Basque (30 years of experience in IT) is the founder of Alcyonix, now member of the international alliance Pi-Network, who provides services in process improvement. Alcyonix is a Transition Partner for teaching CMMI and for conducting appraisals with CMMI.  Richard is authorized by the SEI as a SW-CMM as well as CMMI instructor, as a SW-CMM (SCE and also CBA IPI) as well as a CMMI (SCAMPI) Lead Appraiser. He also works part time as a SEI Visiting Scientist.

  • Exhibitor Track

    • Exhibitor Speaker Opportunity - Slot 8

    • Exhibitor Speaker Opportunity - Slot 9