Testing Example
The purpose of the demonstration program PassStatsBasic is to input a csv file of gender and percentage for exam results of a group of students and to output the pass rate (expressed as a percentage) for males and females. Make sure that you can understand the code before moving on to program PassStats, which includes plenty of validation in an attempt to make it robust.
program PassStatsBasic; {$APPTYPE CONSOLE} //Reads scores for males and females from a csv file //and outputs pass rate for each. uses SysUtils, Strutils; const FILE_NAME = 'marksheet.csv'; PASSMARK = 40; var intCurrentScore, intMales, intFemales, intMalePasses, intFemalePasses : integer; strCurrentScore, strMalePassRate, strFemalepassRate : string; charCurrentGender : char; rMalePassRate, rFemalePassRate : real; procedure CalcStats(FileName : string); var Marks : Text; strCurrentLine : string; ErrorCode : integer; begin intMales := 0; intFemales := 0; intMalePasses := 0; intFemalepasses := 0; intCurrentScore := 0; assignFile(Marks, FileName); reset(Marks); while not eof(Marks) do begin readln(Marks, strCurrentLine); charCurrentGender := LeftStr(strCurrentLine, 1)[1]; charCurrentGender := UpCase(charCurrentGender); strCurrentScore := rightStr(strCurrentLine, length(strCurrentLine) - 2); val(strCurrentScore, intCurrentScore, ErrorCode); if charCurrentGender = 'M' then begin inc(intMales); if intCurrentscore >= PASSMARK then inc(intMalePasses); end else //females begin inc(intFemales); if intCurrentscore >= PASSMARK then inc(intFemalePasses); end; end; closeFile(Marks); if intMales = 0 then begin writeln('No males'); strMalePassRate := 'NoMales'; end else begin rMalePassRate := intMalePasses * 100 / intMales; strMalePassRate := FloatToStrf(rMalePassRate, ffFixed, 6, 2); end; if intFemales = 0 then begin writeln('No females'); strFemalePassRate := 'NoFemales'; end else begin rFemalePassRate := intFemalePasses * 100 / intFemales; strFemalePassRate := FloatToStrf(rFemalePassRate, ffFixed, 6, 2); end; writeln('Male Pass Rate (%): ', strMalePassRate); writeln('Female Pass Rate (%): ', strFemalePassRate); end; begin CalcStats(FILE_NAME); readln; end.
The validated version of the program should be able to handle blank lines or lone commas in the csv file, report a missing data file, and reject files with erroneous data. It should accept groups of a single gender and calculate percentages correct to two places of decimals. We use the description lone comma to refer to a line containing only a comma. (The data in the csv file is likely to have originated in a spreadsheet, and a blank line will convert to a lone comma).
program PassStats; {$APPTYPE CONSOLE} //Reads scores for males and females from a csv file //and outputs pass rate for each. uses SysUtils, Strutils; const PASSMARK = 40; ERROR_MESSAGES : array [0..6] of string = ('', 'File not found', 'File empty', 'Comma should follow letter.', 'First letter must be M or F.', 'Mark must be between 0 and 100 inclusive.', 'An integer must be after the comma.'); var intCurrentScore, intMales, intFemales, intMalePasses, intFemalePasses, intLine, intError : integer; strCurrentScore : string; charCurrentGender : char; rMalePassRate, rFemalePassRate : real; ErrorFound : Boolean; procedure OutPutError(var intError : integer); begin ErrorFound := True; write(ERROR_MESSAGES[intError]); if intLine = 0 then writeln else writeln(' at line ', intLine); end; procedure CalcStats(FileName : string); var Marks : Text; strCurrentLine : string; CommaPos : integer; ErrorCode : integer; begin intError := 0; intLine := 0; if not fileExists(FileName) then begin OutPutError(1); end else begin intMales := 0; intFemales := 0; intMalePasses := 0; intFemalepasses := 0; intCurrentScore := 0; ErrorFound := False; assignFile(Marks, FileName); reset(Marks); if eof(Marks) then OutPutError(2) else begin while not eof(Marks) do begin repeat readln(Marks, strCurrentLine); inc(intLine); until (strCurrentLine <> ',') or eof(Marks); CommaPos := pos(',', strCurrentline); if (CommaPos <> 2) and (strCurrentLine <> ',') then OutputError(3) else begin charCurrentGender := LeftStr(strCurrentLine, 1)[1]; charCurrentGender := UpCase(charCurrentGender); if not (charCurrentGender in ['M', 'F']) then begin OutputError(4) else begin strCurrentScore := rightStr(strCurrentLine, length(strCurrentLine) - 2); val(strCurrentScore, intCurrentScore, ErrorCode); if not (ErrorCode = 0) then OutPutError(6) else begin if (intCurrentScore < 0) or (intCurrentScore > 100) then begin OutPutError(5); end else //no error detected begin if charCurrentGender = 'M' then begin inc(intMales); if intCurrentscore >= PASSMARK then inc(intMalePasses); end else //females begin inc(intFemales); if intCurrentscore >= PASSMARK then inc(intFemalePasses); end; end; //if (intCurrentScore < 0) or (intCurrentScore > 100) end; //if not (ErrorCode = 0) end; //if not (charCurrentGender in ['M', 'F']) end; //if CommaPos <> 2 end; //while closeFile(Marks); if not ErrorFound then begin if intMales = 0 then begin writeln('No males'); strMalePassRate := 'NoMales'; end else begin rMalePassRate := intMalePasses * 100 / intMales; strMalePassRate := FloatToStrf(rMalePassRate, ffFixed, 6, 2); writeln('Male Pass Rate (%): ', strMalePassRate); end; if intFemales = 0 then begin writeln('No females'); strFemalePassRate := 'NoFemales'; end else begin rFemalePassRate := intFemalePasses * 100 / intFemales; strFemalePassRate := FloatToStrf(rFemalePassRate, ffFixed, 6, 2); writeln('Female Pass Rate (%): ', strFemalePassRate); end; end; //if not ErrorFound end; //if eof(Marks) end; //if not fileExists(FileName) end; begin CalcStats('marksheet.csv'); readln; end.
Black box testing is a form of testing that is performed without reference to the source code. We can input examples of normal, borderline/boundary/extreme and erroneous data regardless of how the code is written. We tabulate below our test plan (columns 1-6) with results (column 7). We produced t7.csv by using the data generator at generatedata.com. Rather than provide the evidence in the form of seven screenshots here, we show the neat output of automated testing in the following two sections.
Test Case | File | Line | Data | Reason for test | Expected result | Actual Result |
1 | t1.csv | Missing file | 'Missing file' | As expected | ||
2 | t2.csv | Empty file | 'File empty' | As expected | ||
3 | t3.csv | 2 | M,-1 | Erroneous: out of range | 'Mark must be between 0 and 100 inclusive at line 2' | As expected |
4 | t3.csv | 3 | F,101 | Erroneous: out of range | 'Mark must be between 0 and 100 inclusive at line 3' | As expected |
5 | t3.csv | 4 | MM,10 | Erroneous: length check | 'Comma should follow letter at line 4' | As expected |
6 | t3.csv | 5 | F,36.4 | Erroneous: type check | 'An integer must be after the comma at line 5' | As expected |
7 | t3.csv | 6 | F, | Erroneous: presence check | 'An integer must be after the comma at line 6' | As expected |
8 | t3.csv | 7 | ,68 | Erroneous: presence check | 'Comma should follow letter at line 7' | As expected |
9 | t3.csv | 8 | F56 | Erroneous: missing comma and extreme: error at end of file | 'Comma should follow letter at line 8' | As expected |
10 | t4.csv | 1 | G,35 | Erroneous gender, error on first line | 'First letter must be M or F at line 1' | As expected |
11 | t4.csv | 2 | , | Valid | No message | As expected |
12 | t4.csv | 4 | blank line | Valid | No message | As expected |
13 | t4.csv | 6,7 | , | Consecutive lines with lone comma | No message | As expected |
14 | t4.csv | 9 | blank line | Extreme/borderline: Blank line at end of file | No Message | As expected |
15 | t5.csv | 1 | F,39 | Extreme file: no males, normal line | Output for file: No males, Female pass rate 66.67 | As expected |
16 | t5.csv | 2 | F,40 | Extreme | See Line 1 | As expected |
17 | t5.csv | 3 | F,100 | Extreme | See Line 1 | As expected |
18 | t6.csv | 1 | M,0 | Extreme file: no females, extreme data: lower mark limit. | Output for file: Male pass rate 66.67, No females | As expected |
19 | t6.csv | 2 | M,100 | Extreme: upper mark limit | See Line 1 | As expected |
20 | t6.csv | 3,4 | blank lines | Consecutive blank lines | See Line 1 | As expected |
21 | t6.csv | 5 | M,40 | Extreme: lower pass limit | See Line 1 | As expected |
22 | t6.csv | 6 | ' | Extreme: lone comma at end of file | See Line 1 | As expected |
23 | t7.csv | Typical data | Output for file: Male pass rate 75.56%, Female pass rate 78.18% | As expected |
For your convenience, we have put the test files and the source code of this tutorial's programs into testing.zip.
You may have thought of additional test cases for black box testing, such as various combinations of symbols. Please let us know if you discover test cases that reveal errors in this program. For commercial applications there will be time and budget considerations, and choices must be made about the test cases most likely to uncover a fault. For small programs of your own, it is more likely to be a question of how many tests you can be bothered to do! (Even after extensive testing, a program cannot be guaranteed to be bug-free).