RULE EXTRACTION FROM ANNS

In this section, we will solve the iris problem with an ANN, and then we will use GP to obtain the mathematical equations that explain the relation between the inputs and outputs of the ANN.

Martinez and Goddard (2001) proved that a maximum adjustment of 98.67 percent correct answers (two errors) is achieved with six neurons in the hidden layer. With the system put forward by Rabu ±al (1999) and five hidden neurons, tangent hyperbolic Activation functions and threshold function of 0.5 in the output neurons, the previous register has improved (with regard to the number of hidden neurons), reaching also a 98.67 percent of correct answers. In the cases with Iris setosa , no error is obtained; in the cases with Iris versicolor and Iris virginica , two errors are made, which are not detected because the ANN produces a valid classification (only one true output), but an erroneous one.

The architecture and connection weights obtained may be observed in Figure 4.

Figure 4: Obtained ANN

Now we have a network trained to solve the problem. However, the problem now is to understand how this network works. This is a problem present in the world of ANNs that has made many experts reluctant to use them because they need, not only to solve their problems, but also to know how the system solves them.

To understand how the ANN solves this problem, we will apply GP as we did before (this time, we will use the three-tree classifier system) to the pairs' input/ANN output. Now, the desired outputs will not be those of the problem, but the ones returned by the ANN to the inputs. So, we will obtain rules that explain the behavior of the ANN when it produces those outputs. If we apply this rule-discovery system with the same parameter configuration that we have previously seen, the following rules have been obtained.

The following rule has been obtained from the inputs-outputs of the ANN corresponding to the classification of Iris setosa, and produces an adjustment of 100 percent correct answers:

 (X  1  < 0.3116) 

The rule obtained from the inputs-outputs of the ANN corresponding to the classification of Iris versicolor produces an adjustment of 100 percent correct answers, being the following:

 ((0.2892 < X  1  < 0.5316) OR (((X  3  > X  2  ) OR ((X  4  > 0.7643)AND (X  2  > X  1  )))     AND (0.5316 < X  2  < 0.7268))) 

The rule obtained from the inputs-outputs of the ANN corresponding to the classification of Iris virginica produces an adjustment of 100 percent correct answers, being the following:

 (((X  1  > X  3  ) AND (X  1  > X  2  )) OR (((0.5497 < X  1  ) AND     ((0.5497 > X  3  ) OR (0.7279 < X  2  ))) AND (0.6787 < X  2  ))) 

If we analyze the results obtained, we may observe the distributions carried out by the ANN and the rules obtained by using the analysis file. It is shown in Figure 5.

Figure 5: Distribution Obtained of the Three Classes Produced by the Rules from the ANN