1. Introduction

The use of tools to stimulate the curiosity and motivate the learning is a current theme for discussion, but few explored when related to Software Process. The learning about processes consists of curses in which are passed theoretical concepts and exercised some practical examples [1-3]. One way of making this teaching better and more interesting is to apply alternatives methods, like use case studies, project activities executed, games, simulators, tools, etc.

Using several educational tools, students will be trained through realistic experiences[3]. This approach allows the gain of knowledge away from risks of real projects. Therefore, applying tools permit the reduction of distance between theory and practice for the students and professional who wants to learn and practice[2].

In this paper, six prototypes of interface solutions for educational tools were created, structured in requirements identified from a research of tools and games developed in other papers proposed[2-3]. The objective of the paper will be reached through the evaluation of the mentioned prototypes.

The interfaces will be described qualitatively, based on a set of multiple criteria. Therefore, the paper researches an area called Multicriteria, an approach to support the process for decision making[22]. The prototypes were evaluated qualitatively by applying verbal decision analysis. Systems ORCLASS and PACOM, which belong to the VDA framework, were used[4] for solving problems that has qualitative nature and difficult to be formalized, called unstructured[5]. The first method mentioned has the objective to classify alternatives in different groups. After that, the second method will be applied structured in the preferences of the decision maker in order to rank the alternatives selected.

2. Verbal Decision Analysis

Decision making is a special activity of human behaviour aimed at the conclusion of an objective as for people, as for organizations. In the human world, emotions and reasons become hard to separate, and in personal decisions or when the consequences reach the decisors, the emotions often influences the decision making process[32].

These decision making processes can be easily solved when the problems are numerically described and analysed. Unfortunately, not anything can be evaluated numerically.

According to[23] in the majority of multi-criteria problems, exists a set of alternatives, which can be evaluated against the same set of characteristics (called criteria or attributes). These multi-criteria (or multi-attribute) descriptions of alternatives will be used to define the necessary solution.

The Verbal Decision Analysis (VDA) framework is structured on the assurance that most decision making problems can be qualitatively described. The Verbal Decision Analysis supports the decision making process by the verbal representation of problems[7-13][15-19][24-25][31].

The successful of the process of decision making may depend on an analysis of the problem’s circumstances, examination of the available information and the application of a verbal decision analysis method for selecting the alternatives[6].

The systems of decision making support allow evaluating the alternatives considering the multiple criteria and the decision maker’s preferences, which become the responsible for the decisions. As a multi-criteria decision support approach, the process does not have the objective to show a solution for the decision maker, but it has the objective to help the decision making process[22].

The decision maker’s ability to choose is very dependent on the occasion and the stakeholders involved, although the methods of the decision making are universal.

According to[6], the methods of verbal decision analysis are: ZAPRO-III, ZAPROS-LM, PACOM and ORCLASS. The three firsts have the goal to establish a ranking of the alternatives from some order of preference. The last one is the only methodology for classification from the VDA framework.

There are more DSS available, which does not belong to the same group of Verbal Decision Analysis framework defined by LARICHEV and MOSHKOVICH[29] (ZAPRO- III, ZAPROS-LM, PACOM, ORCLASS), which are: SAC, DIFCLASS and CYCLE for classification, and PARK for ranking[33].

Figure 1 introduces an easy visualization of Verbal Decision Analysis methodologies from the framework according to their objectives.

Figure 1. Methodologies from VDA framework visualization

The Systems that constitute the Framework of Verbal Decision Analysis can be evaluated against its objectives:

● The only methodology from the framework researched for CLASSIFICATION is ORCLASS. Classification means to categorize a group of multiattribute alternatives into a small number of decision classes or groups, which is the method intention.

○ Moreover, there are several other systems of classification that can be applied and analyzed for a future application[5-7] which does not belong to the Framework of VDA[42].

● The other objective identified is ordering (or ranking). This objective intends to organize the alternatives of solution for the problem in a rank, from the most preferable to the least preferable one. The three systems are proposed with this objective in the Framework VDA: ZAPROS-LM, ZAPROS-III, PACOM. Although they have the same final objective, the application form is different in both methodologies.

○ PACOM is the exclusively system created to be applied according to a pair compensation. It proposes the conception of comparing the advantages and disadvantages of multiattribute alternatives.

○ The System ZAPROS was created to be applied by pair comparison. It proposes the conception of comparing a pair of alternative with advantage of providing decision make using simple and understandable dialogue. It is also divided into two options of methods:

● ZAPROS-III differs from ZAPROS-LM in its level of treatment of inconsistence. ZAPROS-III can be considered an evolution of ZAPROS-LM in this concept.

○ Method PARK, which does not belong to the framework researched, is based on pairwise comparison and may be applied to problems with no more than 3 to 5 alternatives. Similar to the other DSS described above, PARK aims at ranking alternatives and its attribute scales are based only on verbal evaluation.

Each system presents a specificity that characterizes its application and objective. Therefore, the methodologies must be studied by the applicant in order to provide the perfect process of decision making for each situation.

3. Methodology ORCLASS

3.1. Overview

The ORCLASS methodology (Ordinal Classification) [6, 20] differs from the other verbal decision analysis methods (ZAPROS, PACOM) because it does not consist of ordering alternatives in rank, but aims at classifying the multi-criteria alternatives of a given set: the decision maker only needs that these alternatives are categorized into a small number of decision classes or groups, generally two groups.

Classification is a very important aspect in decision making. This means the prescription of projects to the particular classes.

In fact, many different methods for solving multicriteria classification problems are widely known[33]. The ordinary classification method ORCLASS was one of the first methods designed to solve these kinds of problems. Moreover, recent methods appeared, such as SAC, DIFCLASS and CYCLE systems.

The method ORCLASS allows to elicit information in traditional form for the human being: through verbal description of decision groups and criteria scales, about the verbal representation of problems. One of the main advantages of the method: dialog easily with the decision maker using verbal criteria values.

The method ORCLASS should compare a few quantity of criteria and criteria values, because the methodology works combining them. So, the combination may generate a high number of questions to the decision making.

The correct form to apply the methodology is presenting the combinations that generate new information, minimizing the number of combinations. This is the property called Transitivity. It means that the application of the method is based on the presentation of the alternatives to the decision maker for direct classification, but the elicitation procedure of ORCLASS allows reducing the number of presentations, through the defined property, simplifying the decision maker’s comprehension.

3.2. Structure

According to[20], Figure 2 presents the structure to apply the VDA method ORCLASS.

In accordance with the scheme described in Figure 2, the application of the method can be divided into three stages: Problem Formulation, Structuring of the Classification Rule and Analysis of the Information Obtained. In the Problem Formulation stage, the set of criteria and criteria values, and the groups to classify the alternatives are defined.

The “Structuring of the Classification Rule” stage will be done structured on the decision maker’s preferences. For this stage, classification boards will be developed for filling. Each cell from the mentioned board is composed by a combination of determined values of criteria. During the decision making process, the elicitation of preferences is done and as long as the filling is accomplished, the classification board became filled.

Figure 2. Procedure to apply ORCLASS method

According to the structure from the Figure 2, the last stage from the methodology will be the Analysis of the Information Obtained, which is responsible for the division of the real alternatives in decision groups, based on the study of the classification board and the explanation of the decision rule.

The results of the decision rules are verbally formulated to be easily explained to stakeholders.

3.3. Explaining the combination of criterion values

Initially, an ORCLASS matrix may be created with the main decision rules. The standard decision rule for any application of the method follows:

• An alternative composed by the best characteristics

([A1,B1,C1]), will always belong to Class I.

• An alternative composed by the worst characteristics

([A3,B3,C3]), will always belong to Class II.

As defined in[23], take an example if it is presented for the decision maker to judge the possibly alternative composed by the criterion values[A1,B1,C3]. In this case, if the decision maker chooses the alternative for the first group, then the better alternative ([A1, B1, C2]) naturally belongs to the first group too, since it is better than the previous. This question to the decision maker results in filling two cells from the board.

On the other hand, if the decision maker judges that the alternative example[A1,B1,C3] must be chosen for the second decision group, then the worst alternatives also must me classified for the second decision group. It means that the alternatives[A1,B2,C3],[A1,B3,C3],[A2,B1,C3],[A2,B2,C3],[A2,B3,C3],[A3,B1,C3],[A3,B2,C3] will belong to the second decision group too, since they are worst than the previous. This question to the decision maker results in filling eight cells from the board.

Below is shown the classification board for illustration and better visualization of the example.

Table 1. Classification Board Composed by the Quantities of Generated Information

It was concluded that the most informative alternative is the cell[A2,B2,C2] (which enables seven new classifications for either answer), so that is the better option to be presented to the decision maker for classification.

3.4. Presentation of the Prototypes

The prototypes represented by the Figures 3, 4, 5, 6, 7 and 8 were developed as examples of possible educational tools interfaces[8,26]. The prototypes of interfaces were created aiming to teach the software process “Analysis of Requirement”, but the intention is that the application, when developed, can be so flexible that the user will be able to configure the educational tool to learn about any software process as he needs.

Figure 3. Prototype 1

Figure 4. Prototype 2

It is important to clear that the interfaces must be friendly, because of the fact that this must be an educational tool, thus must be nice to use. Otherwise, the tool will not be pleasing for the students.

The purpose of the tool is to select the best techniques and tools to be used in the Analysis of Requirements discipline of a particular project. Each project has different characteristics, such as duration, cost and development process. As the educational tool will be flexible for teaching any software process, through configuring, the questions, the options and the project characteristics can change, according to the new configurations.

Figure 5. Prototype 3

Figure 6. Prototype 4

Figure 7. Prototype 5

Observe that the main difference between the prototypes is the form of navigation through the tool, and the availability of the information presented.

The first prototype’s navigation happens using the button for the application to present the next questions.

The second prototype has a similar navigation to the pages of search from internet.

The third prototype presents the navigation through tabs and information for each answer selected.

The fourth interface’s navigation is similar to desktop applications: using the scroll bar. And the information is presented to the user one for each question.

The fifth interface’s navigation happens through links. Each question is presented in the prototype as links that, when selected, expands the related question and its options to be chosen.

Figure 8. Prototype 6

In the sixth interface, the questions are disposed one below the other, and the navigation is also similar to desktop applications: using the scroll bar.

4. Computational Results of ORCLASS

4.1. Criteria and alternatives Definition

As the first step to apply ORCLASS, there were defined the criteria. After the study of another tools and games published, it was possible to describe a list of important requirements to be analyzed in the educational tool.

For each criterion, a set of values that represents the criteria are associated to each one. The criteria scale and its respective values are described in the table 2[8][26-27]:

Table 2. Criteria and Associated Values

Table 3 presents the characteristics of the prototypes about each criterion values defined in Table 2: through analysis of each prototype of interface, it was possible to classify the screens in criterion values.

Table 3. Alternatives Board

A set of decision groups must be defined.

● The first group was chosen to support the prototypes which can be future used to develop the real application, after the application of the methodology;

● The prototypes that will not be possible to became a truly tool will be at the second group.

4.2. Classification Board

According to Larichev and Moshkovich[6], follows the rules to choose the most informative cells:

● Choose the smaller difference between the numbers;

● Choose the cell with the maximum sum of two corresponding numbers;

● In case of several cells satisfying the conditions, choose any one of them.

The alternative[A2,B2,C2] is the most informative. Through the interview with the decision maker, that cell was selected to compose group l, what means that the decision maker believes that a prototype with the characteristics A2 (The information are not easily evident), B2 (Medium Navigability) and C2 (The facility of use is reduced) would be a satisfactory prototype to be developed for an educational tool.

Based on the first classification, the matrix was partially filled, by the dominance of the alternatives as described before. The next table presents the matrix with values better than the second value against each of the three criteria used were classified as Class 1. Besides, after the classification, it is necessary to remake the board, correcting the quantities of generated information. This process will be repeated as long as filling the cells. The next table also presents the cells recalculated.

Table 4. The Filling Process on the Basis of Dominance

Analogous, through the interview with the decision maker, it was filled the cells from the boards with the classification of all of the possible alternatives, and then it was done the dominance relation to fill all the empty cells (see Table 5).

Table 5. Final Classification Boards

4.5. Division in Groups

After the conclusion of the phase to question the decision maker about his preferences and fill all the board’s cells, initiates the phase to analyze the information obtained.

The second decision group will be composed by the prototype number 3, 5 and 6. The first group will be composed by the prototypes 1, 2 and 4, concluding that they are the solution interfaces proposed for a future development of educational tools.

4.6. Decision Rule

The decision rule[6] is an explanation of the results described in verbal way.

Analyzing the answers obtained from the application of ORCLASS, it is possible to identify that any prototype composed by the criterion values A3 or B3 or C3 were not accepted by the decision maker for constructing the aimed tool. The decision rules may be written as follows:

1. For the prototype which has the availability of the information hard to find, the prototype must never be accepted as model for development;

2. For the prototype which has hard navigability through the screen, the prototype must never be accepted as model for development;

3. For the prototype which can be hard to use, the prototype must never be accepted as model for development;

5. Methodology PACOM

5.1. Overview

The methodology PACOM belongs to the Verbal Decision Analysis framework, use to solve problems of qualitative characteristic and unstructured. The method’s name comes from PAired COMpansation

PACOM aims to select the best alternative through pair comparison[6], creating a rank of alternatives that presents the alternatives from the best one to the last preferable one.

According to[6], PACOM is used to compare few quantities of alternatives. In cases of high quantity of alternatives, that must be used another VDA methodologies, more robust, like ZAPROS to refine the rank[28-30].

For applying the methodology, first it is necessary to have a decision maker that knows deep about the problem to be solved. The decision maker must be capable of responding the comparison about his preferences, made by the analyst that applies PACOM.

In PACOM, there are four possible options for responses that the decision maker can take[6]:

● Alternative 1 is more preferable than alternative 2;

● Alternative 2 is more preferable than alternative 1;

● Alternatives 1 and 2 are equally preferable;

● It is difficult do give an answer;

Observe that the property transitivity is also applied in PACOM: according to the decision maker, if the alternative l is more preferable than the alternative 2, and the alternative 2 is more preferable than the alternative 3, it is possible to conclude that the alternative l is more preferable than the alternative 3.

5.2. Structure

According to[20], Figure 9 presents the structure to apply the VDA method PACOM.

In accordance with the scheme described in Figure 9, the application of the method can be divided in four stages: Problem Formulation, Elicitation of Preferences, Validation of decision maker’s preferences and Analysis.

In the Problem’s Formulation stage, the criteria and respective values for the problem are defined.

In the Elicitation of Preferences stage, a pair of alternatives is selected for comparison. Then, the first alternative is compared through advantages and disadvantages against the other alternative (from the most significant disadvantages to the least one).

In the Validation of decision maker’s preferences stage, the methods consist in elimination of dependences that may be found in the elicitation of preferences stage.

In the last stage of the application of the method, Analysis of the obtained information, the pair compensation of alternatives is concluded, and the elicitation preference between them is defined.

Figure 9. Procedure to Apply PACOM methodology

6. Computational Results of PACOM

6.1. Criteria Definition

The criteria and criteria values were defined in table 2. After concluding the application of ORCLASS, three prototypes were selected, and they will be the alternatives for applying the methodology PACOM. Table 6 presents the selected alternatives and its respective characteristics, according to the criteria and criteria values.

Table 6. Alternatives Board

6.2. Paired Compensation

Take the prototype l as the best alternative to begin the elicitation preferences process. The second alternative chosen was prototype 2 to form the first pair of alternatives for comparison. Observe that this is a particular case in the application of the method, because, naturally, the alternative prototype l is better than the alternative prototype 2, as can be seen that the criteria values for criteria B (Navigability) and C (Usability) are exactly the same for both, and they differ just in the criterion A (Information evidence), which alternative prototype l is more preferable than alternative prototype 2. For this case, it was not necessary to generate the hypothetical alternatives.

According to the method, the next step is to choose another alternative to compare to the alternative identified as the preferable in the last compensation. The other alternative existent is prototype 4.

The ranking of the disadvantages is made grouping the advantages identified in the prototype l over the prototype 4 (second prototype chosen for comparison) at the set Kl and the disadvantages at the set K2.

Considering table 6, it is possible to identify that set Kl will be composed by the criterion {C} and set K2, by criterion {B}, because prototype l is more preferable than the prototype 4 in the criteria value C, and less preferable in the criteria value B. Observe that they are equally preferable for criterion A, then criterion A does not belong to any group.

Hypothetical alternatives are generated as described in Table 7. The first one will be called P4’, which derives from the real alternative prototype l and they differ only at two criteria values: one advantage (criterion C) and one disadvantage (criterion B) which are replaced by the criteria values of the alternative prototype 4.

The second hypothetical alternative will be called Pl’ and derives from real alternative prototype 4. Analogously, one advantage (criterion B) and one disadvantage (criterion C) are replaced by the criteria values of the alternative prototype l.

Table 7. Real and Hypothetical Alternatives

The decision maker is part of the next step of the application. The decision maker must identify which is the most important disadvantage, comparing the pair of alternatives prototype l and P4’, and then, comparing the pair of alternatives: prototype 4 and Pl’.

Coincidentally, observing the criteria values, the comparison of the two pairs of alternatives is exactly the same. The decision maker must respond: what is the most important disadvantage: a prototype which has the facility of use reduced (C2) or medium navigability (B2)? Considering the decision maker knowledge and authority, he decided that the most disadvantage is the criterion value “Medium Navigability” (B2). It means that criterion value C2 is more preferable than the criterion value B2.

The preference elicitation was concluded and the result is that the real alternative Prototype 4 is more preferable than the real alternative Prototype 1.

6.3. Property “Transitivity”

In accordance with the theory of the method, Transitivity is a property that can be applied in this moment of the application.

The first comparison was made between alternatives prototype 1 and prototype 2, resulting that prototype 1 is more preferable than the prototype 2. The second comparison was made between alternatives prototype 1 and prototype 4 (applying hypothetical alternatives), resulting that prototype 4 is more preferable than the prototype 1. According to the property Transitivity, prototype 4 is more preferable than the prototype 2.

6.4. Rank of Alternatives

After the application of the method PACOM, the group of the prototypes selected for a future development of educational tools was ranked as described on Table 8.

Table 8. Final Rank of Alternatives

7. Conclusions

The teaching of software process must be stimulated and motivated, because the current learning consists of curses in which are passed theoretical concepts and exercised some practical examples. The use of tools for teaching is a current theme for discussion, but few explored when related to Software Process.

This paper presents several prototypes for educational tools for deciding, verbally, which of them could be really developed and disseminated for education. This paper contribution is to prove that the methodologies of verbal decision analysis can be applied in real problems of elicitation preferences process and decision making, including hybrid methods, composed by application of more than only one method from the verbal decision analysis framework.

Structured in the identified requirements from the research of tools and games developed in another papers already proposed, there was created several different prototypes of interfaces, with characteristics of an educational tool and latter, they were evaluated applying the verbal decision analysis ORCLASS method to select the prototypes interfaces more preferable for the decision maker for its possible development in future works.

After concluded the method application, the prototypes were divided in two decision groups: the first one will be composed by the interfaces which can be selected as a prototype for the real development of the educational tool, the second will be composed by the interfaces which should not be selected as a prototype for the real development.

The paper analyses the selected prototypes and applies the verbal decision analysis method PACOM. This one objects to elicit the decision maker preferences and rank the prototypes from the most preferable to the least preferable one for the decision maker.

8. Future Work

More research can be done applying other methodologies for classification[19,21,14]. Another future works could be to develop educational tools for dissemination based on the prototypes studied and created, selected from the decision analysis took in the paper. It is also interesting to extend the prototypes of educational tools, or research and apply different hybrid methodologies in other knowledge areas, like health.

ACKNOWLEDGEMENTS

The second and third authors are thankful to the National Counsel of Technological and Scientific Development (CNPq) for the support received on this project.