NOA: Program for Constructing Orthogonal Arrays, Near-Orthogonal Arrays and Supersaturated Designs

  1. Introduction
  2. Using noa
  3. Output
  4. Examples
  5. References


A strength 2 orthogonal array (OA) of size n with k si-level columns (i = 1,...,k), denoted by Ln( is an n x k matrix in which all possible combinations of levels in any two columns appear the same number of times (Rao 1947). There is an OA library of over 200 OAs maintained by Prof. N. J. A. Sloane. This library has been recently updated by Dr. W. F. Kuhfeld of SAS at his OA site. This site contains all OAs listed in the Appendix of Kuhfeld & Tobias (2005) as well as new ones contributed by other authors. A simple introduction to OA can be found in most textbook on design of experiments (e.g. Chapter 7 of Wu & Hamada, 2000). A more comprehensive reference of OA is Hedayat, et. al. (1999).

In a near-OA L'n(, to reduce the run size, the orthogonality of some pairs of columns is necessarily sacrificed. The concept of near-OA (see Taguchi 1959; Wang and Wu 1992; Nguyen 1996b; Wu & Hamada 2000; Ma et. al. 2000 and Xu 2002) provides a genuine answer to situations when OAs are not available. An array is called a saturated design when ∑(si-1)=n-1 (e.g. a Hadamard matrix) and is called a supersaturated design when ∑(si-1)>n-1. The 2-level supersaturated designs were discussed in Booth & Cox (1962), Lin (1993), Nguyen (1996a), Tang & Wu (1997), Cheng (1997) and Wu & Hamada (2000) Section 8.6. The mixed level supersaturated designs were discussed in Fang, et. al. (2003, 2004). Additional references on this subject can be found in the references of these two papers.

NOA is a Gendex program for constructing mixed level OA, near-OAs and supersaturated designs. NOA is extremely useful when you want to augment OAs or near-OAs with additional columns. Several new OAs can be obtained by adding additional columns to the Sloane OA library. The optimality criterion used in NOA, called the average criterion is the minimization of E(d2)=∑∑d2ij /kC2 (where i and j are different columns of the array and k is the total number of columns of this array) discussed Ma et. al (2000), Lu et al. (2003) and Fang, et. al. (2003, 2004) (note that E(d2) is D2 in the first reference and E(fNOD) in the last two references). The following matrix shows the frequency distribution of the levels of the columns of an L'84(3161)

5       4       5       5       4       5
4       5       5       4       5       5
5       5       4       5       5       4

which say the combination (0,0) appears 5 times and the combination (0,1) appears 4 times, etc. The expected frequency for each level combination is

n/(s1s2)=84/(3x6)=4.6667. The differences between the observed and the expected frequencies of each cell are in the following matrix:

 0.3333  -0.6667   0.3333   0.3333  -0.6667   0.3333
-0.6667   0.3333   0.3333  -0.6667   0.33333  0.3333
 0.3333   0.3333  -0.6667   0.3333   0.3333  -0.6667

Here d212 (=4) is defined as the squared Euclidean distance between the observed and expected frequencies (or the sum of squares of the values in the difference matrix). Details of the NOA algorithm which uses this optimality criterion is discussed Nguyen & Liu (2007).

Using NOA

Let's assume all Gendex class files are in the directory c:\gendex and suppose you want to construct an L12(3124) (Example 1). At the working directory, type the following command at the command prompt (case is important):

java -cp c:\gendex NOA

The NOA GUI will pop up. Enter the the number of factors at the appropriate factor levels .

Now click START, two windows will pop up. The first one asks you to choose the number of runs: choose 12 and click OK. The second one asks you whether you want to force you want the first 3-level column to be orthogonal to the remaining columns: click No.

NOA will start running and after try 2, the plan of the constructed array for this try pops up in the NOA output window (as E(d2) reaches 0) and then NOA stops:

Note that the START button has been changed to the RESET one. If you click this RESET button, the output will disappear and you can use NOA for a new design problem. Also note that the default random seed is the one obtained from the system clock and the default number of tries is 1000. You can change these default values if you wish to.

Now, suppose you want to construct an L12(3129) (Example 11) by adding five 2-level columns to an array L12(3124) obtained in the previous NOA session and stored in a file called base.txt:

0  0  0  0  1
2  0  0  1  0
0  1  1  1  0
2  1  1  0  0
1  1  0  0  1
0  1  0  0  0
1  1  1  1  1
1  0  1  0  0
2  0  1  0  1
1  0  0  1  0
2  1  0  1  1
0  0  1  1  1

Enter base.txt at the File text field and enter 5 at the 2-level text field of the NOA GUI and click START and Yes to the question on whether you want to force the first 3-level column to be orthogonal to the remaining columns. In less than a second. the NOA output window will become (Example 11):


The result of the best try is displayed in the NOA output window and is also saved in the file NOA.htm in the working directory. This file can be read by a browser such as IE or Google Chrome. Information for this try includes:

  1. Try number;
  2. The number of iterations;
  3. f=∑d2ij (and g if you answer Yes for to the question on whether you want to force the first x columns to be orthogonal to the remaining columns). g=0 indicates that these columns are indeed orthogonal to the remaining columns;
  4. E(d2) and its lower bound. NOA bound improves the one of Fang, et. al. (2003, 2004) when the constructed design is a near-OA. Also, unlike the latter, the former does not make the restriction that n should be divisible by si (i=1,...,k). Details of the calculation of this lower bound appears in Nguyen & Liu (2008). Note that for 2-level column arrays, the popular criterion E(s2)=4E(d2) (Eq. (14) of Fang, et. al. (2003)).
  5. dmax and the frequency of dij=dmax;
  6. Np (the number of dij>0 or the number of pairs of nonorthogonal columns in the array).
  7. A2 (measurement of the overall aliasing or nonorthogonality between all possible pairs of columns).
  8. D-efficiency;
  9. The array;
  10. The d2 matrix;
  11. The Cramer's V matrix: Cramer's V is the most popular of the chi-square-based measures of association between two nominal variables.
  12. The number of protected columns if any;
  13. The time in seconds NOA used to construct the above design.

Note: The calculations of items 6-8 are described in Section 2.2 of Xu (2002). Items 10 and 11 are not printed when the constructed array is an OA (i.e. when f=0).


  1. An L12(3124) (
  2. An L18(2137) (
  3. An L20(5128) (
  4. An L25(56) (
  5. An L'6(3123) (
  6. An L'10(5125) (
  7. An L'12(4134) (
  8. An L'12(2334) (
  9. An L'12(6125) (
  10. An L'12(6126) (
  11. An L'12(3129) (
  12. An L'12(2135) (
  13. An L'12(3227) (
  14. An L'12(3227) (
  15. An L'12(2533) (
  16. An L'15(5135) (
  17. An L'18(2138) (
  18. An L'18(3723) (
  19. An L'18(9128) (
  20. An L'20(51215) (
  21. An L'24(8138) (
  22. An L'24(31221) (
  23. An L'24(61215) (
  24. An L'24(61218) (
  25. An S16(215) (
  26. An S11(211) (
  27. An S8(214) (
  28. An S12(218) (
  29. An S12(6544) (
  30. An S20(5541) (
  31. An S16(21688) (
  32. An S20(2194155) (



Booth, K.H.V. & Cox, D.R. (1962) Some systematic supersaturated designs. Technometrics 4, 489-495.
Cheng, C.S. (1997) E(s2)-optimal supersaturated designs. Statistica Sinica 7, 929-939.
Fang, K-T. , Lin, D. K. J. & Liu, M-Q. (2003) Optimal mixed-level supersaturated design. Metrika 58, 279-291.
Fang, K-T., Ge, G., Liu, M-Q., Qin, H. (2004) Combinatorial constructions for optimal supersaturated designs. Discrete Mathematics 279, 191-202.
Hedayat, A. S., Sloane, N. J. A. & Stufken J. (1999) Orthogonal arrays: Theory and Applications, New York:Springer-Verlag.
Koukouvinos, C. & Mantas, P. (2005) Construction of some E(fNOD) optimal mixed-level supersaturated designs. Statistics & Probability Letters 74, 312-321.
Kuhfeld, W. F. & Tobias, R. D. (2005) Large factorial designs for product engineering and market research applications. Technometrics 47, 122-132.
Lin, D.K.J. (1993) A new class of supersaturated designs. Technometrics 35, 28-31.
Lu, X., Hu, W. & Zheng, Y. (2003) A systematical procedure in the construction of multi-level supersaturated designs. J. of Statistical Planning & Inference 115, 287-310.
Ma, C-X., Fang, K-T & Liski, E. (2000) A new approach in constructing orthogonal and nearly orthogonal arrays. Metrika 50, 255-268.
Nguyen, N-K. (1996a) An algorithmic approach to constructing supersaturated designs. Technometrics 38, 205-209.
Nguyen, N-K. (1996b) A note on the construction of near-orthogonal arrays with mixed levels and economic run size. Technometrics 38, 279-283.
Nguyen, N-K & Liu, M-Q. (2008) Orthogonal and near-orthogonal arrays constructed from incomplete block designs. Computational Statistics & Data Analysis 52, 5269-5276.
Rao, C. R. (1947) Fractional experiments derivable from combinatorial arrangements of arrays. J. of the Statistical Society (Supp.) 9, 128-139.
Taguchi, G. (1959) Linear graphs for orthogonal arrays and their applications to experimental designs, with the aid of various techniques. Report of Statistical Applications Research, Japanese Union of Scientists and Engineers 6, 1-43.
Tang, B. & Wu, C.F.J. (1997) A method for constructing supersaturated designs and its E(s2)-property. Canadian J. of Statistics 25, 191-201
Wang, J.C. & Wu, C.F.J. (1992) Nearly orthogonal arrays with mixed levels and small runs. Technometrics 34, 409-422.
Wu, C.F.J & M. Hamada (2000) Experiments: Planning, Analysis and Parameter Design Optimization. New York: Wiley.
Xu, H. (2002) An algorithm for constructing orthogonal and nearly-orthogonal arrays with mixed levels and small runs. Technometrics 44, 356-368.

©2000-2014 Design Computing