Reviewing the Problem

We have customer tests that let us write scripts and let us run the scripts to see if we get correct output. These tests run against the TextModel object. The theory is that the GUI is so simple that these tests are convincing.

Unfortunately, as a customer, I m no longer convinced that the product is solid. Even though my tests ran, there have been at least two very different defects that showed up only in the GUI. Again, one time the scrolling didn t work, and another time the test ran but the GUI didn t even have the feature built into it. What s up with that? As a customer, I m losing confidence. That makes me want to call for massive overtime, or reduced pizza rations, or something equally draconian. The programmers ought to be really interested in this issue.

Let s narrow the problem down a bit. Big problems beget big solutions. That way leads either to big infrastructure efforts or to manual testing. I forget which one of those is Charybdis, but either way it s bad.

Here s our problem in a nutshell : when the customer writes a test that sends AltS to the XML Notepad, we want to be sure that Alt+S is in the GUI and that it actually does what is intended ”in this case, InsertSectionTags.

I ll tell the tale of trying to drive the GUI automatically in a moment, but we couldn t really make it work well. Let s set aside whether we re driving the GUI and remember that we want to test the GUI to be sure that if the test script runs, the GUI will run as well.

Let s begin with a look at a couple of tests:

 *altS 
 *output 
 <sect1><title></title> 
 </sect1> 
 -------------------------------- 
 *input 
 <P>This is the first paragraph.</P> 
 <P>This is the second paragraph.</P> 
 *end 
 *enter 
 *output 
 <P>This is the first paragraph.</P> 
 <P></P> 
 <P>This is the second paragraph.</P> 

Recall that the lines with * are commands and the rest are data. The data lines show a vertical bar character () where the cursor is to be set. So the first test says User types Alt+S. The text window should then look like this. The rest of the stuff is the desired contents of the window. Similarly, the second test says Start with these two paragraphs, cursor in the first one. User types Enter. Now there are three paragraphs, a new one in the middle, with the cursor in it. The code to execute *enter or *altS looks like this:

 private void InterpretCommands(String commands, String message) { 
 StringReader reader = new StringReader(commands); 
 String line = reader.ReadLine(); 
 CreateModel(); 
 while ( line != null) { 
  if ( line == "*enter")  
  model.Enter();  
  if ( line == "*altS")  
  model.AltS();;  
   if ( line == "*altP") 
 model.AltP(); 
 if (line == "*display") 
 Console.WriteLine("display\r\n{0}\r\nend", model.TestText); 
 if (line == "*output") 
 CompareOutput(reader, message); 
 if (line == "*input") 
 SetInput(reader); 
 line = reader.ReadLine(); 
 } 
 } 

So when the *enter comes along, we just send the message Enter() to the model. That code looks like this:

 public void Enter() { 
 InsertParagraphTag(); 
 } 
 public void InsertParagraphTag() { 
 InsertTags(newParagraph, paragraphSkip); 
 } 
 
 private static string[] newParagraph = { "<P></P>" }; 
 private static string paragraphSkip = "<P>"; 

Note that the model is making the translation here from enter to InsertParagraphTag. In the GUI, we have this code:

 public void XMLKeyDownHandler(object objSender, KeyEventArgs kea) { 
 Console.WriteLine("Down" + kea.KeyCode); 
 model.SetLines(textbox.Lines); 
 model.SelectionStart = textbox.SelectionStart; 
 if (kea.KeyCode == Keys.Enter) { 
 model.Enter(); 
 kea.Handled = true; 
 } 
 PutText(textbox, model.LinesArray(), model.SelectionStart); 
 } 

The potential bug is that the code for the KeyDown might not support Keys.Enter. If we changed that to Keys.Tab, the tests would still run, but the GUI would not. This isn t as good as we d like. And it gets worse . Let s look at the Alt+S situation. Alt+S is implemented in the model similarly to Enter. AltS() calls InsertSectionTags(). But in the GUI, oh my! Look at this:

 MenuItem insertSection = new MenuItem ( 
 "Insert &Section", 
 new EventHandler(MenuInsertSection)); 
 void MenuInsertSection(object obj, EventArgs ea) { 
 model.SetLines(textbox.Lines); 
 model.SelectionStart = textbox.SelectionStart; 
 model.InsertSectionTags(); 
 PutText(textbox, model.LinesArray(), model.SelectionStart); 
 } 

The menu isn t going through AltS() in the model; it s calling InsertSectionTags() directly. Now, I think that s a better design, because the mapping from keystroke to menu to model call is more a function of the Form than of the GUI. But it means the test is completely broken. If we changed the AltS() method in the model, we could break the test without breaking the GUI. We could also find a way to break the GUI without breaking the test: one way is just not to implement that menu item at all. What can we do to make it better? Well, Chet and I worked on that.