Links

Testing the calculator

The source code for the calculator contains some test functions that allow automatic testing of some of its features. The code is located in the file jscicalc/complex/Test.java and to test it from the build directory, use:
java jscicalc/complex/Test
The test code needs a file called test.jsc with information on what to test. There is a suitable file in cvs in the jscicalc/complex directory.

Currently the automated tests test parsing of input and formatting of output and some of the scientific functions.

The parsing and formatting tests are particularly important because everything else uses standard, well-tetsted code. The current version of test.jsc contains data for over 870,000 tests. Each parsing/fomatiing test has an input number and a base. The number is put into the calculator Parser and then two comparisons are made for each of the eight possible combinations of notation. The first comparison compares the number stored in the calculator with the input. The second compares the output with the input.

The possible notations are the combinations of standard or scientific notation, complex (always shows both real and imaginary parts) and noncomplex notation, and rectangular and polar notation.

The comparisons use different techniques from the calculator for comparing input with output or internal representation. This is so that it is unlikely that a defect would be missed because both the calculator and the test make the same error. This is an attempt to produce independent verification.

The calculator has several thousand automated tests of the scientific functions. At the moment, the functions tested are sin, cosine, tangent and their inverses, log, ln, exp, inverse, square root, square, cube and cube root. The tests compare the result of the calculator’s calculation with a more accurate known value and estimate a relative error. The largest relative error so far found is about 0.000000000000008.

Basic arithmetic (addition, subtraction, multiplication and division) is also thoroughly tested. Addition and subtraction are accurate (relative error) to the test tolerance of 2.2204460492503131e−15. Almost all the multiplication and division tests achieve the same tolerance. A few have slightly greater error, about the same as the worst tests for cubes. The lower tolerance affects only numbers with nonzero imaginary parts and is fairly inevitable given the limits of numerical calculations in Java. Consider a complex multiplication like (17 + 10000i) × (5882353 + 10000i) = 1 + 58823700000i (which the calculator gets exactly right). The absolute size of the real part is much smaller than any of the products of real/imaginary parts. So if we did the calculation with more limited precision, we would quickly lose accuracy in the real part. It is very difficult to overcome this difficulty.

The calculator functions largely rely on code adapted from the gnu Scientific Library complex functions, which is highly tested. They are all tested manually. Nonethless, more automated tests would be valuable.

Last modified: Sat 31 May 2008 01:11 pm