CEN 5070 - Software V & V Unit Testing Concepts

  • Published on
    12-Jan-2016

  • View
    28

  • Download
    0

DESCRIPTION

CEN 5070 - Software V & V Unit Testing Concepts. Purpose. This module presents the basic concepts of black-box and white-box testing for unit testing. A systematic approach is shown for deriving functional, boundary and white-box test cases. - PowerPoint PPT Presentation

Transcript

CEN 5070 - Software V & VUnit Testing ConceptsUnit Test ConceptsPurposeThis module presents the basic concepts of black-box and white-box testing for unit testing. A systematic approach is shown for deriving functional, boundary and white-box test cases." arbitrarily selected test set ... results in inefficient testing, leavingsome functions untested while performing redundant testing of others."Darlene Mackay, Quality Consultants UnlimitedUnit Test ConceptsAgendaWhat Is Unit TestingBlack-Box TestingWhite-Box TestingPutting It All TogetherUnit Test ConceptsWHAT IS UNIT TESTING?Executing a software element to determine whether it meets its specificationExecuting a software element to discover defects or anomaliesInspecting software element code to discover defects or anomalies.Unit Test ConceptsWHAT IS A UNIT?Named software element Separately invokablePerforms single functionExamplesSubprogram or scriptField with validationDatabase stored procedureJava class methodUnit Test ConceptsSPRAE: A Model for Testing PracticeSpecification -- basis for software testingPremeditation -- testing requires planning, forethought Repeatability -- process, results independent of testerAccountability -- testing artifacts maintainedEconomy in the use of human, time and computing resourcesUnit Test ConceptsA TESTING LIFECYCLEAnalysisDesignUnit Test ConceptsAgendaWhat is Unit TestingBlack-Box TestingWhite-Box TestingPutting It All TogetherUnit Test ConceptsBLACK-BOX TESTINGTesting based on the specification rather than the implementation. Specification defines the expected response(s) to stimuliStimuliResponse(s)Unit Test ConceptsBLACK-BOX TECHNIQUESFunctional testing -- tests the behavior of the software. Boundary testing -- tests behavior at the lower/upper bounds of input valuesRandom testing -- tests using randomly generated stimuli (load testing)Intuitive (ad hoc) testing -- error guessing Unit Test ConceptsFUNCTIONAL TEST DESIGN METHODOLOGYSpecificationIdentify behaviors Develop test casesWrite test scriptUnit Test ConceptsEXAMPLE A(1) SpecificationCompute pay for an hourly employee, given the number of hours worked and the hourly pay rate. Compute overtime at 1.5 times hourly rate for hours in excess of 40.Unit Test ConceptsEXAMPLE A(2) Identify BehaviorsCase 1: No overtime (Hours 40)Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)Unit Test ConceptsEXAMPLE A(3) Create Test CasesCase 1: No overtime (Hours 40)Use Rate = 10, Hours = 50 Expect Pay = 40*Rate+1.5*Rate*(Hours - 40) = 550Unit Test ConceptsEXAMPLE A(4) Write Test ScriptUnit Test ConceptsA MORE COMPLEX EXAMPLE (B)Increased number of behaviorsUse of decision table to document behaviorsTest case generation from decision tableUnit Test ConceptsEXAMPLE B(1) SpecificationCompute pay for employee, given the number of hours worked and the hourly pay rate. For hourly employees (rate < 30), compute overtime at 1.5 times hourly rate for hours in excess of 40. Salaried employees (rate >= 30) are paid for exactly 40 hours.Unit Test ConceptsEXAMPLE B(2) Identify BehaviorsCase 1: Hourly AND No overtime (Rate < 30) & (Hours 40)Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)Case 3: Salaried (Rate >= 30) Expect Pay = 40 * RateUnit Test ConceptsDECISION TABLEColumns defineBehaviorsUnit Test ConceptsEXAMPLE B(3) Create Test CasesOne test case per column of decision tableCase 1: Hourly, No OvertimeCase 2: Hourly, OvertimeCase 3: Salaried, No Extra HoursCase 4: Salaried, Extra HoursOrder the test cases by column Unit Test ConceptsEXAMPLE B(4) Write Test ScriptUnit Test ConceptsRULES -- DECISION TABLES Use 'Y', 'N', '-' or spaceUnit Test ConceptsYour Turn -- Problem P1(1) SpecificationCompute the dosage of drug X for patient, given the patient's Age and Weight. For patients 12 and under, the dosage is 1 pill. For patients over 65, the dosage is 2 pills. For all other patients, the dosage is 2 pills plus an extra pill for each 50 pounds above 120. The drug can not be given to patients over 300 pounds or over the age of 80.Unit Test ConceptsYour Turn(2a) Identify BehaviorsCaseExpected Stimulus Description#Pills 123456Unit Test ConceptsYour Turn (2b) Decision Table c1: Age 65 |c3: Age > 80 |c4: Weight > 300 |c5: Weight > 120 |a1: Pills = 0 |a2: Pills = 1 |a3: Pills = 2 |a4: Pills = 2+(W-120)/50 |Unit Test ConceptsYour Turn(3) Create Test Cases Case 1 2 3 4 5 6 7 8 9Age ___ ___ ___ ___ ___ ___ ___ ___ ___ Weight ___ ___ ___ ___ ___ ___ ___ ___ ___Pills ___ ___ ___ ___ ___ ___ ___ ___ ___Unit Test ConceptsYour Turn(4) Write Test ScriptStepStimuliPills= Age7891012Weight11Unit Test ConceptsSCALING UPThe heart of the approach is to use a decision table as a thinking tool. The most critical task in this process is to identify all the stimuli and responses. When there are many logical combinations of stimuli, the decision table can become large, indicating that the unit is probably too complex.Unit Test ConceptsIDENTIFYING BEHAVIORApproachesWork backwardsIdentify each responseIdentify conditions that provoke responseIdentify separate stimuliWork forwardIdentify each stimulusIdentify how stimulus influences what unit doesSpecify the responseTreat stimuli combinationsUnit Test ConceptsIDENTIFYING STIMULIArguments passed upon invocationInteractive user inputsInternal, secondary dataglobal or class variablesExternal data (sources)file or database status variablesfile or database dataExceptionsUnit Test ConceptsIT PAYS TO BE A GOOD STIMULUS DETECTIVE Failure to identify stimuli results in an incomplete, possibly misleading test caseThe search for stimuli exposesinterface assumptions -- a major source of integration problemsincomplete design of unitinadequate provision for exception handlingUnit Test ConceptsIDENTIFYING RESPONSESArguments/Results passed back on exitInteractive user outputsInternal, secondary dataupdated global or class variablesExternal data (sinks)output file or database status variablesoutput file or database dataExceptionsUnit Test ConceptsIT PAYS TO BE A GOOD RESPONSE DETECTIVE Failure to identify responses results in incomplete understanding of the software under testshallow test casesincomplete expected resultsincomplete test "success" verification -- certain effects not checkedTo test, one must know all the effects Unit Test ConceptsA SKETCHING TOOL Black-Box SchematicStimulus TypeResponse TypeSoftwareunderTestArgumentInputsGlobalsDatabaseExceptionArgumentOutputsGlobalsDatabaseExceptionUnit Test ConceptsBEFORE CONTINUTINGMuch of the discussion so far involves how to identify what software does. We have introduced thinking tools for systematically capturing our findings. These thought processes and tools can be used anywhere in the lifecycle, e.g., in software design!One Stone for Two Birds!!Unit Test ConceptsBOUNDARY TESTING DESIGN METHODOLOGYSpecificationIdentify elementary boundary conditions Identify boundary pointsGenerate boundary test casesUpdate test script (add boundary cases).Unit Test Concepts(1) SpecificationCompute pay for an hourly employee, given the number of hours worked and the hourly pay rate. Compute overtime at 1.5 times hourly rate for hours in excess of 40.HoursPayRateUnit Test Concepts(2) Identify Boundary ConditionsCondition 1 (bc1): Hours (3) Identify Boundary Pointsbc1 (Hrs (3a) BP Generalization bc x > y has TWO boundary pointsbp1: Just INside: x = y + precisionbp2: Just OUTside: x = ybc x = y has THREE boundary points:bp1: OUTlo: x = y - precisionbp2: OUThi: x = y + precisionbp3: AT: x = yUnit Test Concepts(3b) BP Generalization bc x != y has THREE boundary points:bp1: INlo: x = y - precisionbp2: INhi: x = y + precisionbp3: OUT: x = yUnit Test Concepts(4) Generate Test CasesCombine Hours boundary points with RateCase 1 (AT): Hours = 40, Rate = 10, Pay=400Case 2 (IN): Hours = 39, Rate = 10, Pay=390Case 3: (OUT): Hours = 41, Rate=10, Pay=415Observations:Test each boundary point individuallyThen consider pair-wise boundary pointsUnit Test Concepts(5) Update Test ScriptStepStimuliExpected Response HoursRatePay =1230501010300550340104004391039054110415Unit Test ConceptsYour TurnBoundary Testing Decision Table:Unit Test ConceptsYour Turn (2-3) Boundary Conditions/PointsBoundary ConditionBoundary Point (BP)ATINbc1:OUTbc2:bc3:bc4:bc5: Unit Test ConceptsYour Turn(4) Generate Boundary Test CasesTo create a test case, you must pair an Age with a WeightWeight boundary point + NOMINAL AgeAge boundary point + NOMINAL WeightOUT Age + OUT Weight A nominal value is one that is not close to a boundary point. For simplicity, use the same nominal value in all test cases. Unit Test ConceptsYour Turn(4) Boundary-Nominal Test CasesCondition btc BC Weight Age Expect Weight>300 1 IN 301 21 0 2 OUT 300 21 Age 65 6 IN 7 OUT Age > 80 9 IN 10 OUT Weight>120 11 IN 12 OUTUnit Test ConceptsYour Turn(4) Out-Out Boundary Test Cases Condition OUT BPWeight Age btc Weight Age Expect >300 65 14 >80 15 >120 65 17 >80 18Unit Test ConceptsOBSERVATIONSFunctional testing defines a minimal number of test casesBoundary testing adds a large number of test cases, but are EASY to createBoundary testing finds lots of errors!Unit Test ConceptsBOUNDARIES EXIST FOROptional fields: (present, missing).Variable length data: null string, max lengthDatabase tables: emptySearching: first/last positionFile: closed, emptyUnit Test ConceptsRANDOM TESTINGBeyond scope of this courseGenerally used to bombard software with inputsNo effort to identify expected resultsAppropriate for automated load testing, where concern is for capacity/volume.Unit Test ConceptsINTUITIVE (AD HOC) TESTINGMost common type of informal testingOften, no specification!!No scriptsNot repeatableNot systematicVery effectiveDoes not guarantee thorough testingUnit Test ConceptsINTUITIVE TESTINGAd hoc, exploratory testing, ideal for destructive testingGoal is to break the software via unexpected stimuli or sequences of stimuliBenefitsFlushes out holes in the specification .. What really should happen when the input is X? Forces treatment of error/exception handlingUnit Test ConceptsMAIN POINTSBLACK-BOX TESTINGBlack-box = spec-based testingTests intentionsKey knowledge = stimuli and responsesDecision table organizes S/R conditionsBoundary testing flushes coding errorsBlack-box test case design can drive software design -- same issues addressedUnit Test ConceptsAgendaWhat is Unit TestingBlack-Box TestingWhite-Box TestingPutting It All TogetherUnit Test ConceptsWHITE-BOX TESTINGStimuliResponse(s)Testing to ensure that software does not do what is not supposed to do. Test ALL of it!Tick -- Tick -- TickNo-Surprise Software!!Unit Test ConceptsWHITE-BOX TESTINGFocus is thorough execution of program elements during the testing process.Warning: Tests only what is built, not what was intended!StimuliResponse(s)Unit Test ConceptsWHITE-BOX TESTINGConcept of coverage. Numeric measure of thoroughness of testing, relative to StatementsBranchesConditionsPathsUnit Test ConceptsCONTROL FLOW GRAPHDefines the flow of control through unit. 12453Unit Test ConceptsCONTROL FLOW GRAPH TERMINOLOGY5 NODESsequential blocks of code terminated by a branch3 PATHS: [1,2,3,5], [1,2,5], [1,4,5] 2 BRANCHES1 and 2 are decision nodesUnit Test ConceptsCONTROL FLOW GRAPH COVERAGEOne test case forces execution of one path (red)Paths are determined by branches (decision nodes)A thorough test set forces execution of all paths (red, green, blue).Unit Test ConceptsCOVERAGE LEVELS (%)Statement -- a statement has been executed at least once during testingBranch -- each outcome of a branch has been performed at least once during testingPath -- a path through the code has been executed at least once during during testingCondition -- a condition has evaluated to true and to false at least once during testingUnit Test ConceptsCONTROL FLOW GRAPH COVERAGE MEASUREMENTFor 2 test cases (red, green)Node (statement) cov = 4/5Branch cov = 1/2 [2]Path cov = 2/3Acceptable coverage levelsStatement cov = 90%Branch cov = 80%Path cov = 70%12453Unit Test ConceptsBRANCH vs CONDITION COVERAGECode example 124100% Branch coverage [1] (x=0,y=2), (x=1,y=2) [TT,FT]But not 100% Condition coverage Need case TF (x=0,y=1)if (x1) x = x + y;else y = y - x; Unit Test ConceptsTHE PROBLEM WITH COMPOUND CONDITIONSMakes complex logic appear simpler than it really isTest cases may be omittedLogic results in 3 paths, not 2!!if (x1) x=x+y; else y=y-x;} else y=y-x;Unit Test ConceptsTHE PROBLEM WITH PATH COVERAGENot all paths are feasibleNo test case can force path [1,2,3,4,5], since consecutive decisions mutually exclusive.if (x= 1) y=3;z=y;Unit Test ConceptsMeasuring Path Testing DifficultyMcCabe metric -- logical code complexity Formula: 1 + #decisions in control flow graphTest Significance: #basis paths through codeDesign use: complexity of codeTest use: min #test cases for 100% path coverageMcCabe measures test (development) difficultyUnit Test ConceptsHow To Design White Box TestsTest cases must execute different pathsDecision tables Rows -- elementary conditions in the codeColumns -- combinations of conditions in the codeColumn based test case forces flow through different logic paths in the codeDecision table built from code reflects what was built versus intended (ie, from the spec)Decision analysis for white-box testing.Unit Test ConceptsWHITE-BOX TESTING TOOLSTools instrument source code and gathers coverage data when tests Compiled languagesScript languages -- coming to marketSome provide test case design assistanceList of paths not coveredData conditions to force branch/pathGraphic depiction of graph coverageUnit Test ConceptsProblem P1Code (v1) & Decision Tableif (Age>80 || Weight>300) return 0; if (Age 65) return 2;if (Weight < 120) return 2else return 2+(Weight-120)/50;McCabe = 6Unit Test ConceptsProblem P1Code (v2) & Decision Tableif (Age>80 || Weight>300)return 0; if (Age 65 || (AgeYour Turn -- White-Box Testing(1) Construct Decision Tablepills=0;if (Age < 80 && Weight = 65) pills=2; else if (Age > 12) pills=2+(Weight-120)/50;}return pills; Unit Test ConceptsYour Turn -- P1(2) Derive White-Box Test Cases Case 1 2 3 4 5 6 7 8 9Age ___ ___ ___ ___ ___ ___ ___ ___ ___ Weight ___ ___ ___ ___ ___ ___ ___ ___ ___Pills ___ ___ ___ ___ ___ ___ ___ ___ ___Unit Test ConceptsOBSERVATIONS -- WHITE-BOX TEST CASESCode may be incomplete with respect to input combinations from the specification Decision table constructed from code is simpler -- subset of black-box tableClaim: black-box test cases from decision table force coverage of logicUnless the code implements the wrong (a different) functionUnit Test ConceptsMAIN POINTSWHITE-BOX TESTINGWhite-box = logic testingLimitation: can't tell what's missingDon't forget exceptions -- throwing, catching, propagating (debugger)Perform decision analysis of codeCoverage tools help.Use black-box test cases.Unit Test ConceptsAgendaWhat is Unit TestingBlack-Box TestingWhite-Box TestingPutting It All TogetherUnit Test ConceptsPUTTING IT ALL TOGETHERTest design is a systematic process whether you use decision tables or not, you must understand the factors influencing and influenced by the behavior of the unitThe sooner you design test cases, the betterTest design is more crucial than running testsReveals assumptions, omissions and errorsUse test cases during design and coding inspectionsUnit Test ConceptsPUTTING IT ALL TOGETHERIt's okay to ask "What happens when ? Look for test design patterns Expect certain type units to be tested in a similar wayA way to identify best practicesEfficiency without cutting cornersView test design as a "design product", not just a test product!!Unit Test ConceptsFunctional/Boundary Testing WorksheetsStored in DominoMaster templateStimulus-Response AnalysisDecision TableFunctional Test CasesBoundary Test DesignBoundary Point AnalysisBoundary Test CasesUnit Test ConceptsSOLUTIONS TO EXERCISESUnit Test ConceptsYour Turn (Dosage Problem)(2a) Identify BehaviorsCaseExpected Stimulus Description#Pills 123456Any Age, Weight>300Age > 80, any WeightAge Your Turn(2b) Decision Table #1c1: Age 65 | Y Y N N Nc3: Age > 80 | Y Y c4: Weight > 300 | N Y N Y N Y N N Yc5: Weight > 120 | N Ya1: Pills = 0 | X X X X Xa2: Pills = 1 | X a3: Pills = 2 | X Xa4: Pills = 2+(W-120)/50 | XUnit Test ConceptsYour Turn (2b) Decision Table #2Unit Test ConceptsYour Turn(3a) Create Test Cases (testset1)Case 1 2 3 4 5 6 7 8 9Age 8 12 67 67 81 85 15 28 28Weight 40 305 180 360 120 315 100 220 320Pills 1 0 2 0 0 0 2 4 0Is more better? Every combination of stimuli is tested. Each column specifies at least one condition per variableUnit Test ConceptsYour Turn(3b) Create Test Cases (testset2)Case 1 2 3 4 5 6 Age 20 10 70 83 13 30 Weight 310 70 160 150 115 220 Pills 0 1 2 0 2 4 Is fewer better? Not every combination of stimuli is tested. Test case generation easier when each column specifies at least one condition per variableUnit Test ConceptsYour Turn(4) Write Test Script (testset1)Unit Test ConceptsYour Turn(2) Identify Boundary ConditionsCondition 1 (bc1): Age 65Condition 3 (bc3): Age > 80Condition 4 (bc4): Weight > 300Condition 5 (bc5): Weight > 120Unit Test ConceptsYour Turn(3) Identify Boundary PointsBoundary ConditionBoundary PointATINbc1: Age 65 bc3: Age > 80bc4: Weight > 300bc5: Weight > 120 12 11 6681 301 121 13 65 80300120Unit Test ConceptsYour Turn(4) Boundary-Nominal Test CasesCondition btc BC Weight Age Expect Weight>300 1 IN 301 21 0 2 OUT 300 21 5 Weight>120 3 IN 121 21 2 4 OUT 120 21 2 Age 65 8 IN 220 66 2 9 OUT 220 65 4 Age>80 10 IN 220 81 0 11 OUT 220 80 2Unit Test ConceptsYour Turn(4) Out-Out Boundary Test CasesCondition OUT B-pointWeight Age btc Weight Age Expect >300 65 13 300 65 5 >80 14 300 80 5 >120 65 16 120 65 2 >80 17 120 80 2Unit Test ConceptsYour Turn -- White-Box Testing(1) Construct Decision Table pills=0;if (Age < 80 && Weight = 65) pills=2; else if (Age > 12) pills=2+(Weight-120)/50;}return pills; | Age < 80 | N - Y Y Y | Weight= 65 | N Y N| Age > 12 | N - Y| Pills = | 0 0 1 2 C--------------Note: C: 2+(Weight/120)/50Unit Test ConceptsYour Turn(2) Derive White-Box Test Cases Case 1 2 3 4 5 Age 83 24 11 68 35 Weight 97 310 85 225 108Pills 0 0 1 2 2Unit Test Concepts1) Welcome to session #1 Unit Test CONCEPTS2) A few administrative matters1. Sign in -- the sheet is circulating2. Attend each session3. You will be asked to complete a course evaluation at the end of Session #3.3) How many of you areTest: [ ] Lead [ ] Analyst [ ] CoordinatorDevelopmentA&DNow, open your notebook.1. Read the welcome page2. Turn to Tab1This session takes around 2.5 hours.Handouts -- Unit Test Concepts5SYSTEMATIC testing = not too much, not too littleFOCUS is DESIGN OF TEST CASESHandouts -- Unit Test Concepts5The session is broken into these topics, with major emphasis on the second and third.Handouts -- Unit Test ConceptsTESTING implies EXECUTING software or INSPECTIING softwareThe intent of testing is TO FIND ERRORS!!A unit is the smallest software element that will be separately tested.Handouts -- Unit Test ConceptsELEMENTARY UNITS have these properties. The main property is that is can be INVOKED SEPARATELY -- so it can be isolated for the purpose of testing.Handouts -- Unit Test ConceptsThe practice of software testing should adhere to these five principles.1. Testing without a specification is invalid -- the specification defines "correct behavior"2. A systematic, consistent process must be used to design test cases.3. The overall testing process must be repeatable -- results must be independent of the tester.4. Testing must produce evidence showing what was attempted, the results of testing, and how the results were interpreted.5. Testing practice must be realistic in terms of time and labor required.The focus of this session is a REPEATABLE PROCESS for TEST CASE DESIGN.Handouts -- Unit Test ConceptsJust as there is a lifecycle for software development, there is one for testing.1. ANALYSIS -- Given the specification, determine a test strategy (and/or plan).2. DESIGN -- Given the specification and strategy, apply techniques to derive test cases.3. IMPLEMENTATION -- Build the machinery necessary to execute the tests, including data and manual scripts or automated scripts (drivers).4. EXECUTION -- Perform the tests according to the script/driver, and capture results.5. EVALUATION -- Evaluate test results to classify errors found, and record deficiencies.Handouts -- Unit Test Concepts5In this section we focus on black-box testing, which relies on the specification rather than the implementation.Handouts -- Unit Test ConceptsThe emphasis of black-box test case design is in gaining a COMPLETE UNDERSTANDING of the the data and conditions affecting the externally visible behavior of the software. In engineering, a black-box analysis is really an analysis of a system's RESPONSES to various input combinations (STIMULI). The relationships between stimuli and responses should be stated in the SPECIFICATION, or the specification must be modified to make those relationships explicit.Handouts -- Unit Test ConceptsThere are four black-box testing techniques. We will focus on SYSTEMATIC techniques for functional and boundary testing.INTUITIVE testing is non-systematic, but relies upon the tester's intuition about where errors may exist.Handouts -- Unit Test ConceptsThe process for designing functional test cases starts with the specification. The STIMULUS / RESPONSE behaviors are derived from the specification. Next, a systematic process is applied to derive test cases from the behaviors. Finally, the test cases are recorded in a test script (or driver), so they can be executed during testing.NOTE: This course does NOT treat the EXECUTION and EVALUATION phases of the test lifecycle.Handouts -- Unit Test ConceptsThe diagram, a BLACK-BOX SCHEMATIC, communicates information about the unit under test:1. TWO input arguments, Hours and Rate2. ONE output result, PayThe narrative specification, which is used to construct the schematic, contains explicit or implicit statement of the relationship between stimuli (Hours, Rate) and response (Pay). Handouts -- Unit Test ConceptsThere are TWO WAYS TO CALCULATE PAY:1. Overtime pay formula for over 40 hours2. Straight pay formula for 40 hours or lessThat is, to the outside observer, Pay has two BEHAVIORS, one when Pay is just the product of in the arguments (Hours * Rate), and another formula is used Hours > 40.Handouts -- Unit Test Concepts1. The function of Pay is tested only WHEN EACH BEHAVIOR is tested. So two cases are adequate for testing the function of Pay.NOTE: Note the behavior of Pay is the same for Hours = 10, 20, 30 and 40, since in all these cases Hours Incomplete and erroneous verification.2. Responses are not always visible -- effects may show up later on, for example:- Failure to initialize variable Sum to zero leads to overflow after adding 30 values.Handouts -- Unit Test ConceptsAgain we see the schematic template which lists the types of Stimuli and Responses.When analyzing behavior, question whether there are stimuli (responses) of each type. If so, list them by name (or description).RECALL: Stimuli tend to be used in the CONDITION part of the Decision Table. Responses appear in the ACTION part, along with those stimuli that provide values used in the computation of a response.Handouts -- Unit Test Concepts1. A Decision Table is a SPECIFCIATION.2. The specification is used for DESIGN AND FOR TESTING.3. Killing TWO BIRDS with ONE STONE!Handouts -- Unit Test ConceptsWe now show you a way to derive boundary test cases from the specification. The approach is1. Analyze specification to identify behaviors.2. From the decision table CONDITION section, identify boundary conditions.3. For each boundary condition, derive boundary points.4. Generate boundary test cases from the boundary points.5. Add the boundary test cases to the functional test cases already in the test script.Handouts -- Unit Test ConceptsGiven this specification, we determine potential boundary conditions.Handouts -- Unit Test ConceptsThere is just one -- relating to the OVERTIME criterion.NOTE: Boundary conditions are the condition rows in the decision table.Handouts -- Unit Test ConceptsA boundary point is a value as close as possible to the boundary stated by the decision table condition. How close a boundary point can be to the boundary depends on the PRECISION of the value. Example: when Hours are recorded down to 1/10 of an hour:IN = just inside, Hours = 40 - 0.1 = 39.9OUT = just outside, Hours = 40 + 0.1 = 40.1NOTE:a) For strict inequality, there is NO AT, and the value stated is OUT!b) SURPRISE: For strict EQUALITY, there is AT and TWO OUTS, outHI and outLO!Handouts -- Unit Test ConceptsA boundary point is a value as close as possible to the boundary stated by the decision table condition. How close a boundary point can be to the boundary depends on the PRECISION of the value. Example: when Hours are recorded down to 1/10 of an hour:IN = just inside, Hours = 40 - 0.1 = 39.9OUT = just outside, Hours = 40 + 0.1 = 40.1NOTE:a) For strict inequality, there is NO AT, and the value stated is OUT!b) SURPRISE: For strict EQUALITY, there is AT and TWO OUTS, outHI and outLO!Handouts -- Unit Test ConceptsA boundary point is a value as close as possible to the boundary stated by the decision table condition. How close a boundary point can be to the boundary depends on the PRECISION of the value. Example: when Hours are recorded down to 1/10 of an hour:IN = just inside, Hours = 40 - 0.1 = 39.9OUT = just outside, Hours = 40 + 0.1 = 40.1NOTE:a) For strict inequality, there is NO AT, and the value stated is OUT!b) SURPRISE: For strict EQUALITY, there is AT and TWO OUTS, outHI and outLO!Handouts -- Unit Test ConceptsRECALL that a test case is a PAIR of a RATE value with an HOURS value.1. Hold Rate CONSTANT at a NOMINAL (not near a boundary) value, and pair it with each of the HOURS boundary values.2. This provides an ISOLATED boundary test on HOURS, without possible interference with boundary values of RATE (if any).Handouts -- Unit Test ConceptsWe have added THREE boundary test cases.In general, each boundary condition has 2 or 3 boundary points, each of which must be used in a boundary test case.When the number of conditions grows, so can the number of boundary points, hence test cases!Handouts -- Unit Test ConceptsIt is now your turn to apply the technique to the DOSAGE calculation from the last session.Use the decision table shown as the starting point.This will take 10 minutes.Handouts -- Unit Test Concepts1) Extract boundary conditions from the decision table, in the order they appear in the decision table.2) For each boundary condition, DERIVE boundary points. Handouts -- Unit Test ConceptsBegin by testing boundary handling on one variable at a time.Next create test cases that pair OUT values of multiple variables.Suggested NOMINAL values:AGE = 21WEIGHT = 220Handouts -- Unit Test ConceptsNote the nominal values in red.We have already tested the function of the unit, so using the same nominal value each time a nominal value is required does not reduce the strength of the test, but does simplify determining the expected results.Handouts -- Unit Test ConceptsThese test cases exercise the handling of multiple variables being out of bounds at the same time RELATIVE TO THE BOUNDARY CONDITIONS.Handouts -- Unit Test Concepts1. Functional testing is a form of EQUIVALENCE PARTITIONING testing that has the goal of a minimum number of (POSITIVE) test cases that cover the functionality (behaviors) of the unit.2. Boundary testing generates far more test cases 3. But they're worth it: Boundary testing is especially effective in finding coding errors.Handouts -- Unit Test ConceptsBoundaries exist for situations that do not appear to be NUMERIC.- Any time there is a LIMIT, there are boundary points- Any time there is a notion of READY or NOT-READY, there are boundary points- Any time there is notion of ORDER, ARRANGEMENT or SORTEDNESS, there are boundary points based on the POSITION of an element in this collectionHandouts -- Unit Test Concepts(OPTIONAL)Random testing ignores the semantics of the inputs, and just churns out lots of values that fall within the range. Unless smart algorithms are used, random testing may produce lots of test cases that fall within the same equivalence partitions -- redundant test cases!There are situations, such as load or performance testing, where volume of data is more important than the meaning of the data.Handouts -- Unit Test ConceptsINTUITIVE TESTING is based on experience or "gut feel" about where errors may be. When all else fails, this is an effective technique but may have serious GAPS in the coverage of function, boundaries, and amount of the code that is exercised during the testing process.Handouts -- Unit Test ConceptsA benefit of intuitive testing is that it permits destructive (NEGATIVE) testing without the need to document ahead of time what the test cases will be.The "chimpanzee at the keyboard" approach forces discussion of responding to UNEXPECTED USAGE.Handouts -- Unit Test ConceptsMAJOR POINTS:1. Black-Box testing focuses on ensuring the software meets its specification. 2. Test case design is likely to result in the REFINEMENT OF THE SPECIFICATION.3. The decision table is an intermediate specification useful for designing and testing.4. Unit test design and software design are FLIP SIDES of the SAME COIN.Handouts -- Unit Test Concepts5We are now ready to discuss another approach to unit testing that focuses on HOW THE unit is BUILT rather than the specification.Handouts -- Unit Test ConceptsWhite-Box Testing is an UNDER-THE-HOOD testing approach. The goal is to ensure that the software does not contain a TICKING BOMB waiting to go off unexpectedly.Handouts -- Unit Test ConceptsThe one way to ensure that there is no ticking time bomb is to execute all the code under all the possible circumstances -- to see how the code responds.WARNING: Success in white-box testing is measured in terms of how thoroughly the code was executed -- EVEN IF IT'S THE WRONG CODE!Handouts -- Unit Test Concepts1) White-box testing can be MEASURED.2) The granularity of the measurement is defined in terms of code elementsa) Statements (or blocks)b) Branches caused by decision statements (if, loops, switches)c) Conditions used to make decisionsd) Execution paths through the code3) COVERAGE is the numeric measure -- a percentage of the countable code elementsHandouts -- Unit Test ConceptsThe execution of a unit can be described by a graph that depicts the FLOW OF CONTROL through elements of the code.Drawing this graph IS NOT REQUIRED (whew!), but the graph explains the important relationship between code complexity and testing difficulty. Handouts -- Unit Test ConceptsA unit can be described by this graph. Certain features of the graph are relevant to white-box testing.1) NODE -- block of statements in the code2) BRANCH -- code statement at which sequential flow is broken based on the outcome of a decision3) PATH -- end-to-end sequence of nodes visited during one execution of the unit.Handouts -- Unit Test ConceptsThe measure of thoroughness is called COVERAGE.The notion of THOROUGHNESS can be made VISUAL by coloring elements of the graph that have been executed during testing.There are tools that produce such pictures.Handouts -- Unit Test ConceptsThese coverage levels are arranged in order of RELATIVE STRENGTH and INCREASING DIFFICULTY. Handouts -- Unit Test ConceptsYou may be surprised that acceptable coverage levels are not all 100%.You will see in the next few slides some of the things take make it hard to achieve 100% coverage.Handouts -- Unit Test ConceptsIt is common to write code that uses COMPLEX CONDITIONS of logical expressions involving AND and OR operators.From experience, it is HARD to get programs with complex logic to work properly. If they are hard to get to work, they are also HARD TO TEST!Handouts -- Unit Test ConceptsHandouts -- Unit Test Concepts1) There may be paths in code that CAN NEVER BE EXECUTED. These are called INFEASIBLE PATHS.2) For the example:a) Test case x=0, path is [1,2,3,5] b) Test case x=2, path is [1,3,4.5] c) NO TEST CASE for path [1,2,3,4,5]3) One can TEST FOREVER and NEVER REACH 100% path coverage.4) HOWEVER -- These test cases achieved 100% BRANCH and CONDITION COVERAGE!!5) Designing test cases for path coverage results in high levels of branch/condition coverage.Handouts -- Unit Test Concepts1) There is a surprisingly simple way to determine the minimum number of test cases required to achieve 100% PATH COVERAGE (assuming feasibility)2) The metric, McCabe's Cyclomatic Complexity = 1 + #elementary_decisions 3) Most widely used metric for software designa) SPLIT unit when McCabe exceeds thresholdb) High McCabe means ERROR-PRONE4) McCabe = MIN# TEST CASES for path coverage Handouts -- Unit Test ConceptsThere are two options for designing white-box test cases:1. Reverse Engineer Decision Table from Codea) Each column represents a pathb) Create test case for each columnNOTE: Decisions based on computed values may complicate test case generation.2. Use Black-Box Test Casesa) Adequate scheme when implementation matches the specification -- or is very closeb) Misleading otherwiseNOTE: Use of black-box test cases during code inspection may keep implementation close to spec.Handouts -- Unit Test Concepts1) Fortunately, there are tools for white-box testing. Basic featuresa) Instrument source code by inserting probes to gather coverage data b) Report generation following test executionc) Graphical display of coveraged) Assistance with test case design 2) Unfortunately, these tools are not in widespread or consistent use in SC.Handouts -- Unit Test Concepts1) On the next two slides you will be shown how to build a decision table from code.2) Afterwards, you will be given "Your Turn" at doing the same.3) NOTE: This is an implementation of the Compute Dosage. It has McCabe of 6, so six test cases are needed to achieve 100% path coverage.4) The reverse engineered decision table has 6 columns, as expected. So deriving a test case for each column should force execution of the 6 different paths.Handouts -- Unit Test Concepts1) This is a different implementation of Compute Dosage. It is slightly more complex -- largely due to the use of complex conditions.NOTE: The decision table is different from your notes -- the condition rows have been reordered to match the order the conditions APPEAR IN THE CODE.2) WARNING: REMEMBER that these test cases force the execution of the paths of the AS BUILT CODE. Passing these tests does not mean the code matches the specification.Handouts -- Unit Test ConceptsYOUR TURN:(1) Reverse engineer the decision table from the code. In determining expected result, GO BY THE CODE, not the original spec.Handouts -- Unit Test ConceptsYOUR TURN(1) Create test cases for path coverage, one per column of the decision table.Handouts -- Unit Test ConceptsIn the ideal world, the CODE implements the SPEC, so that EACH black-box test case triggers a distinct behavior from the code .. and EACH DISTINCT BEHAVIOR has a UNIQUE PATH through the code.USE BLACK-BOX TEST CASES FOR WHITE-BOX TESTINGa) Measure actual coverage (using tool)b) Create only those test cases needed to achieve coverage goalWITHOUT TOOLS, USE SOUND ENGINEERING JUDGMENT.Handouts -- Unit Test Concepts1) The real goal of white-box testing is to TEST IT ALL!2) COVERAGE answers HOW MUCH SO FAR?3) TEST CASE DESIGN uses REVERSE ENGINEERING of code to decision table4) CODE INSPECTIONS are the best way to address whether black-box test cases are adequate for achieving white-box coverage.5) WITHOUT TOOLS you are limited in what you can BE SURE OF.Handouts -- Unit Test Concepts5We have nearly reached the end of this session. In the next few slides we will summarize what was presented in this session.Handouts -- Unit Test Concepts1. Test case creation requires a SYSTEMATIC DESIGN PROCESS2. The same design process is required for SOFTWARE DESIGN3. DECISION TABLES provide a simple way to SPECIFY THE INTENT of a software unit4. EARLY TEST CASE DESIGN forces EARLY CLARIFIICATION/REFINEMENT OF SPECS5. USE TEST CASES THROUGHOUT SOFTWARE DEVELOPMENT Handouts -- Unit Test Concepts1. The sooner you ask the question, the sooner you get an answer2. Since many units are similar (copy-and-paste coding), the test process is also likely to be similar -- PATTERNS3. TEST DESIGN is DESIGN -- treat it as a software design product.Handouts -- Unit Test Concepts5Handouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test ConceptsHandouts -- Unit Test Concepts

Recommended

View more >