Section 22.4. RESULTS

22.4. RESULTS

Our study led us to two kinds of insights: recommendations for designers of concern elaboration tools and hypotheses about the nature of concerns. We conclude this section with a discussion of the validity of our study.

22.4.1. Tool Design Recommendations

Our first two recommendations for designers of future concern elaboration tools follow directly from our results.

Concern elaboration tools should support a user in following the control flow of a program.

As we reported, despite the differences in searching supported by the concern elaboration tools studied, all participants tended to use a strategy of following the control flow in the program. Although all of the tools had affordances to help follow control flow, in some cases the lack of explicit support made following the control flow tedious and contributed to mistakes in concern elaboration.

Concern elaboration tools should help the user identify points in the source that correspond to conditions in the executing system.

For some change tasks, it was natural to describe the lines of code related to the concern. For other tasks, in particular those that add code, it was more natural to describe a condition in the executing system (e.g., a precondition). These conditions hold at one or more points in the execution, or simply execution points. A condition describing a set of execution points might be naturally defined, for example, in terms of the state of an object, the call stack, or a sequence of method calls. To assist code changes and other tasks, tools should map the execution points for a condition to corresponding points in the code.

Our third recommendation is based on our analysis of the differences between concerns identified in the context of the reasons given by the programmers' for those differences.

Concern elaboration tools should allow a user to determine the extent of the concern.

A tool does not exist in isolation. Concern elaboration tools should be expected to work within the context of other development tools. When a participant using a concern elaboration tool knew that a part of the concern could be easily found with another tool, they sometimes chose to limit their exploration with the concern elaboration tool. For example, participants did not include parts of the concern which they knew would be caught by the compiler. The task in which the concern elaboration was posed also affected the extent of a concern that was captured. In a refactoring task, for instance, a developer is typically working with a system that is compilable and for which the compiler can be used to expand the edges of the concern. Our recommendation is that designers of concern elaboration tools should be aware of how other tools might be used to compensate and complement the description of a concern such that a programmer can elaborate a concern cost-effectively within the tool suite. A concern elaboration tool should not prescribe to the user how much of the concern needs to be identified for the task at hand.

22.4.2. Concern Characterization

When we began this study, we assumed that concerns were definite entities that any motivated programmer could reliably identify. In particular, we spoke of a concern as though it were an unformed module that possessed an unambiguous, but possibly messy, interface. This implicit assumption was consistently contradicted by our three cases.

In fact, what we found was that the exact concern was heavily dependent on the particulars of the taskit was not reasonable to define it precisely prior to making an actual change, leading to our third design recommendation. Reflecting on our results, we found that each case taught us something different about the murkiness of concerns.

Concern boundaries are fuzzy. The Jex case involved the replacement of a concern's implementation. P_AMT wrote:

Some details [of the concern's code extent] depend on the actual implementation of the RTA algorithm. From the provided information it is not clear whether the new implementation can make use of (parts of) the helper procedures computeRelationships (Method). Since that is implementation dependent, I have marked it.

This quote shows how the extent of the concern was dependent upon the context in which the concern was identified. Although the task posed to the developers was to find the code related to existing functionality in the system, it was not always possible to clearly delineate the boundaries of that functionality, perhaps because that functionality had not been considered as a unit during any previous development tasks.

Concerns include execution points. The AMT case involved the insertion of new code into the concern. P_FEAT wrote:

It seemed that most of the "concern" code was actually stuff that had to be added, not so much existing code that needed to be changed. That's why I found the "concern code" by navigating parallel structures, and was often confused about what to add and what to leave out.

That is, the points where the code additions were legitimate depended on the code to be inserted. More importantly, these additions depended on a set of behavioral preconditions that had to be met for the inserted code to behave correctly.

The interface between concerns is a concern. The AspectBrowser case involved the refactoring of a concern. Prior to performing the traversal feature refactoring task, P_AMT wrote:

Are we to identify the complete [concern] or just the part that is relevant for the proposed refactoring task/change?

Later in the conversation, the AspectBrowser expert noted:

[D]ifferent design decisions will ultimately change what would be a concern.

A couple of insights emerged from this interview with the participants. First, most of the code in the traversal concern would not physically change; the real concern in some sense is the interface between the traversal concern and the code that invokes it. On the other hand, the scope of the code in the traversal concern is likely to change from the invokers' classes to a number of new classes. Also, arriving at a sensible design requires a possibly deep perusal of both the traversal concern and the invoking code, as well as an understanding of how the traversal feature might be enhanced in the future. It is not entirely clear whether just the edges of the traversal concern are the concern or if the core of the traversal concern should be included as well.^[2] If one can equate Parnas's notion of a design decision with a concern, discussions on modularity have weighed in on the side of an interface being distinct from its contents [4, 18, 19, 27].

^[2] This distinction was not manifested in the AspectBrowser case results because the ambiguity was arbitrarily resolved before the case was carried out (find the whole traversal concern).

Based on our experience through the three cases, we would now describe a concern as having the following elements:

• A core that determines the behavior of a concern

• An interface that may be a concern in its own right and is possibly ambiguous

• A set of execution pointsstates in the executionthat describe hooks at which behavior may be added to the concern

The complete and correct articulation of a concern is fraught with problems and is seemingly unnecessary, at least in support of software evolution tasks. For example, completely characterizing the execution points of a concern depends on a description of all possible changes to a concern. It is also unnecessary because the current change requires only the determination of sufficient execution points to add the code. According to the expert in the AMT case, two different developers need not even derive the same points in the concern.

22.4.3. Study Validity

Our study was exploratory. By using three participants and three tools on three different systems and tasks, we were able to uncover and consider a range of behaviors and issues in concern elaboration. Despite these differences, some commonalities emerged, such as the strategies used to investigate the code bases, and some differences emerged, such as what the participants determined was the edge of a concern and how they did so. The results of our study provide guidance on the functionality designers should consider when building future tools of this type and hypotheses upon which to base further, more in-depth studies.

We have avoided statements about the efficacy of the tools since programmer productivity is known to vary independently of tool use and because we did not compare our tools with existing, non-concern elaboration specific tools, such as grep. Furthermore, we recognize that we only examined a small part of the programming activity that contributes to productivity. To reflect on efficacy, a much larger study would be required.