1. Introduction

In general terms, the University can be defined as an integrated structure, formed by different units, able to contribute to the generation of human resources in quantitative and qualitative terms, necessary for life and progress of society as a whole.

Thus, institution of higher learning can be defined as the structure formed by formal teaching, non formal (dissemination or divulgation) and research.

The desired outcome is to improve the quality of life of graduates and society as a whole, and to generate human beings able to sustain themselves, who can excel in life and participate fully in the society‘s progress.

Likewise, Faculty activities must be performed under complete academic freedom.

In that context, each student will benefit to the maximum extent of the academia.

Universidad Tecnologica de Panama was founded in 1981, and the National Center of Veraguas, located in the Pan-American road, whose location has coastline on both the Pacific and Caribbean oceans, was established in 1991.

Since 2004, the Campus is offering a total of 99 careers, including one Doctoral and 20 Master Degree programs, and this institution has a population of over 14,000 students, including locals and foreigners. In 2004, the University began adopting the College Standard International Admission Policy, which is based on the Puerto Rico’ College Board.

From the view of the community and society in general, the education offered by the University seems like the most powerful tool for achieving individual and collective capabilities.

By the same token, the institution is also open to innovations, critical thinking, progress and the search for the truth.

At the same time, the University is also conservative in regard to preserving their academic styles[1].

Nowadays, Faculty members perform not only the traditional practice of adoption and management of analytical tools by the students, but also a new conception of the academic formative process that focus in the student’s learning process[2].

There is not doubt that the accelerated transformations that are currently taking place in the world, are putting pressure in higher learning institutions to have more flexibility in their structures, processes and activities, like it is happening in different service’s organizations[3].

Although one of the most researched aspects of the academia was the faculty evaluation, until recently, the outcome of the university teaching-learning process was attributed as one of the aspects to be defined[4].

Therefore, it is imperative for Higher Learning Institutions to count with an integrated and accurate evaluation system that can measure results of the academic process.

At the administrative level, as the recognition of the importance and the necessity of incorporating an evaluation and assessment system into the university teaching-learning process grow, provisions must me made to include professional improvement, and development of the evaluated faculty integrated into the evaluation system.

The process should include elements for the process of reflection, decision making, and the achievement of the objectives.

Diverse models and methodologies have been proposed to respond to such challenges.

The topic of design and validation of instruments have been the object of scientific studies of certain in-depth and relevance.

However, one of the greatest dilemmas of Latin American universities is the availability of instruments that reflect with a high degree of accuracy the result of the academic performance as a tool for decision making.

Despite the fact that the application of structured methods of strategic support at the entrepreneur level has been widely disseminated, such as the BSC–The Balance Scorecard–[5], with all characteristics as a tool to provide direction to the managerial tasks[6],[7], its use in the academic environment have been scarce and divergent with the diagnostics of the educational process that shows a great need of improvement in the university processes[8].

As opposed from previous approaches, Reference[9], suggests a progressive development of the strategic planning in the context of the academic process based in the organizational learning and decision making[10].

Currently, due to the wide use of scales, following the directions of the “Cuadro del Mando Integral” by the Spanish anachronism of CMI, the objective of this article is to apply and review in a simple way, the concept of internal consistency and the interpretation of the coefficient of Crobach’s Alpha, based on the design and validation of instruments for exploring the internal environment of the Faculty members at Centro Regional de Veraguas, of Universidad Tecnologica de Panama.

The mentioned approach is focusing in the Planning, Development and the Faculty Activities inherent to their academic duties.

2. Materials and Method

The present research presents three (3) phases: (i) elaboration of the BSC - CMI, (ii) instruments design and (iii) procedures for validation of instruments.

2.1. Elaboration of BSC - CMI

The methodology suggested by Reference[11], was followed for implementing the Initial Faculty Evaluation Program for the Academic Achievement or “Programa Inicial de la Evaluación del Desempeño Académico Docente” by the Spanish anachronism of (PIEDAD) at Centro Regional de Veraguas, of Universidad Tecnologica de Panama.

At this stage of the research, a set of strategic indicators were defined. A system of information was designed in order to analyze the links among research, mission, long-term vision, intermediate term and short term of PIEDAD.

The same author’s citated, point out the definitions of three matrixes: strategic, evaluation of external factors -MSEF, and evaluation of internal factors-MSIF.

2.2. Instruments Design

The stage of design was developed to satisfy the need for the availability of simple instruments than can be standardized. Instruments must contain the academic environment of the Faculty members in their academic endeavours. They should consider the three (3) main focus of the evaluation process: Evaluation, Self-evaluation and Co- evaluation.

The inventory for collecting the information for the instrument’s design was based on the existing documents available at Universidad Tecnológica de Panamá and other universities. The aspects relevant to the academic dimensions to be evaluated were selected.

The dimensions considered for the construction of the instruments were identified in several reference sources and were three (3). From the available inventory, a number of items were written according to the corresponding standards.

Additionally, stakeholders were included in the process. From both sources, a list of questions was prepared, related to the academic responsabilities of the Faculty personnel.

The instruments were written in past tense, in the first or third person singular, with analogies pertinent to the academia. The listing of dimensions was as follows:

Planning Dimension: Includes the statements related to the academia planning tasks, including course plan and the communication with students.

Development Dimension: Includes situations related to the subject matter during the academic, derived from the organization and classes’ preparation until thecommunication to students.

Activity Dimension: Includes activities related to the teaching and learning process used by Faculty to achieve their academic goals and objectives, including references utilized, evaluation methods proposed, and the utilization of alternative teaching methods and pedagogical resources.

2.3. Validation Procedure

This stage had two steps: (i) Conceptual and Construct Validation and (ii) the Validation of Consistency or Reliability.

Table 1 presents different stages of the validation process, which contain different tools in the design of instruments to evaluate the academic achievement of faculty members. SPSS package for the PC was used to generate the consistency or reliability of the instruments.

Table 1. Relationship between the steps of instruments and their stages of validation Steps Stages of validation Types of validation Characteristics Definitions of dimensions Construct Delphi Technique = Expert Judgement Selection of items or indicators Content Revision of references used Delphi Technique Field tests or face validity Reliability Cronbach Alpha

To obtain the consistency or reliability index, which is the “accuracy of the instrument”[12],[13],[14], the model of internal consistency of Cronbach Alpha was utilized, which is based in the average of correlations of the instrument’s items[15],[16].

The closest the Cronbach’s Alpha index to 1, the highest is the value of the internal consistency of Cronbach Alpha of the analyzed items.[17]. The Cronbach’s Alpha index ranged from 0-1.

To perform the consistency of the different instruments, a field test was utilized. A sample of random and cluster sample of stakeholders were selected and they completed a self administrated questionnaires.

Participants were asked to rated the instrument base on their level of understanding,[18], utilizing the following scale: I do not understand at all (= 1); I do not understand (= 2); I understand more or less (= 3); I understand (= 4) I do understand completely (= 5) with the objective of establishing the work’s scale[19 - 22]. The variable of time was not taken into consideration as a variable for answering the instrument.

Figure 1. Summary of the methodology utilized for the design and validation of instruments regarding the academic achievement of faculty members

3. Results and discussion

3.1. Development of SBC - CMI

The first stage in the construction of integral control was the revision or construction of the Strategic Frame, which was the analysis and definition of the Mission and Vision of PIEDAD.

In the next stage, stakeholders were defined to include academic departments, Faculty members and students, and the responsible for the implementation of the evaluation, Co-Evaluation and Self-evaluation of the academic roles performed by the Faculty personnel at Universidad Tecnologica de Panama, Centro Regional de Veraguas.

Based on the information collected in the diagnostics of SOWT (Strengths, Opportunities, Weakness and Threats), and considering the importance of the external environment for its implementation, the matrix was elaborated.

The methodology of CMI recommends comparing the value obtained in the MSEF with an average value of (2.5) after analyzing the results. In this research, the value obtained in the MSEF matrix was of 1.37, which was below the desirable value. Thus, the value for the external environment was not appropriate for the PIEDAD implementation.

The mentioned output allowed determining the minimum needs required to implement PIEDAD; -needed resources –infrastructure, humans and financial resources for its implementation, and the availability of Evaluation, Co-evaluation and Self-evaluation instruments scientifically designed and validated.

The value for the internal environment was also included in the matrix –MSIF-, which allowed determining the internal capacity of the organization to confront the external environment and allowed the achievement of the mission and the survival of the organization.

Likewise, following the methodology, it is recommended to determine the value of the Matrix to be included in the MSIF and comparing with the average (2.5). The calculated value for the internal environment was 2.65. Thus, internally, the organization and their stakeholders are ready to implement PIEDAD with very good possibilities of success

3.2. Content and Construct Validation

Universidad Tecnologica de Panama–UTP–has been using the traditional evaluation model of assessing the faculty achievement, based only on students input. The evaluation form is collected on line.

The analysis of the existing instruments from internal records, from others in-country universities and from additional Latin American countries, were utilized as background information for the design of instruments.

Nevertheless, only instruments related to academic achievement of faculty were considered. Extension, Administration and Research related instruments were not considered.

For the content and construct validation, a consensus technique was used to reach a consensus in six (6) sessions. Sessions were used to revise the proposed instruments. Faculty participated in the sessions.

In each session, recommended modifications were incorporated, including the inclusion of new sentences or the allocation of the different dimensions of the academia.

Out of 110 sentences originated, 78 propositions were approved.

Finally, for the three structural components included in the content and construct validation, the number of statements for each component were as follow: 33 statements for the instrument of Evaluation; 33 for the instrument of Co-evaluation and 12 statements for the instrument of Self-evaluation.

3.3. Reliability Evaluation

For obtaining the reliability index, a total of 218 subjects participated. A total of 165 faculty members (75.7%) answered self administered instruments, and those were included in the data analysis process.

The global internal consistency of the three instruments was satisfactory, with a Cronbach’s Alpha coefficient ranging from 0.926 to 0.917 for the instruments of Evaluation and Self-evaluation respectively. Likewise, the instrument of Co-evaluation obtained a Cronbach Alpha index of 0.804.

Table 2 summarized the reliability coefficient index obtained for the instruments self –administered and their respective dimensions (Planning, Development, Educational activities, and type of instrument (Evaluation,Co-evaluation and Self-evaluation).

Table 2. Cronbach’s Alpha (α )scales for Evaluation’s dimensions and types of instruments

Taking into consideration the statements provided by Reference[17],who indicated that if the Cronbach’s Alpha index is greater than 0.9, the instrument ‘s reliability is excellent; in the range of 0.9-0.8 the instrument is good; between 0.8-0.7 it is acceptable; in the range of 0.7-0.6 the instrument is weak; between 0.6-0.5 it is poor; less than 0.5, the instrument is not acceptable; the conclusion is that the instruments for PIEDAD to evaluate the academic achievement of the Faculty Personnel for Universidad Tecnológica de Panamá, Centro Regional de Veraguas; in global terms, the reliability index ranged between excellent and good.

Table 2 shows a level of satisfactory reliability coefficient, although the dimension of Planning obtained a Conbach Alpha value of 0.51 (poor), and a similar value for the dimension of Self-evaluation (0.57). Likewise, the dimension of Academic Activities obtained a Crobach Alpha of 0.07 (no acceptable).

The described information allowed to revise and improved the more problematic sections and items and to visualize those that presented more variability. The objective was to make a more understandable instrument.

A more detailed table, with all sections and item by item are presented in figure 2. The figure includes the Mean, Standard Deviation and error by dimensions and types of instrument by these presentation was made it feasible to identify the problematic items, and it showed at the figure.

Figure 2. Box & Wisker Plot Graph, for the items of the instrument of Co-evaluation

The figure shows that for the Planning dimension, item presents the largest deviation for the answers provided by participants in both evaluation instruments: Evaluation and Self-evaluation. Some respondents said that that question does not imply necessarily that it was not comprehensible. (…) It could mean that the question was not applicable to all subject matter, or the answer might be very vague.

The item in question made reference to the possibility of the utilization of different methodological strategies for the course’s development; that’s why it is recommended a grammatical revision and a conceptual improvement of the instruments for its final edition.

Likewise, the dimension of the Pedagogical activities from the instrument of Co-evaluation obtained a Cronbach’s Alpha of 0.07 (no acceptable). Due to the fact that the Co-evaluation instrument has a number of items different from the Evaluation and Self-evaluation, the graph took into consideration the values minimum, maximum, mean and standard Deviation.

The analyzed dimension of the Co-evaluation instrument regarding the Pedagogic activities were four and are included in the right side of the previous graph: Agrado (Likes) Respeto (Respect), Equipo (Team) and Actividades (Activities).

In this graph it is observed that the item corresponding to the dimension to the attribute of Agrado (Likes) is getting away from the behavior of other items. That shows more variability in the provided answer. Thus, there is a need of revising the instrument to improve its comprehension. The original items made reference to the following item: “There is evidence that professors enjoy teaching their subject matter”. Some respondents said that that question is intrinsic and abstract. To summarize all this, the new validated instruments for evaluating the academic at Universidad Tecnologica de Panama at Centro Regional de Veraguas, as a part of PIEDAD is highly valid and reliable. That made it very relevant, due to the scarcity of evaluation instruments that measure the three main aspects of the evaluation process. The advantage of the availability of scientifically validated instruments make possible to detect the strong and weak points of the university teaching process, and the availability of the tools to improve them; findings will allow more evidence to the discussion of the strength and weakness of the planning, development and the methodology strategy used by faculty personnel.

3.4. Implementation: Lead Scale

In order to implement the institutional level PIEDAD conducted a pilot study involving 65 teachers, with their students and their academic coordinators respectively. Teachers were randomly selected as the subject for bids in the corresponding semester of implementation. In this case, it was First Semester 2011.

After applying the three instruments, we proceeded to analyze the results, to know, for example, which of the three dimensions, the teacher is "weaker" and how it was assessed both by his students, for their respective coordinator.

Data were graphed and for one of the teachers, the results are presented in the figures below.

Figure 3. Column Graph, overall weighting for the estate

If we consider that the maximum numerical value for each academic theme is 5, for example, in the Self-evaluation aspect, the teacher obtained a nonparametric 4.95, a Co-evaluation by the academic coordinator of 4.90 and an Evaluation 4.57 by their students. The situation observed, Self-evaluation, Evaluation, Co-evaluation, was not the standard behavior observed in the studied population

It is necessary to point out that those teachers whose overall average was around 3.5, feedback was provided to them in private.

On the other hand, the lead study could also show, for a particular teacher, which of the three main academic axes reflects higher or lower strength. The axes were assessed as follows: Planning (20%) Development (40%) and Methodology (40%). In this way it is worth mentioning that these aspects were defined and agreed during the validation phase of the instruments, specifically during meetings of the Delphi Technique.

Here we have, for example, that showed at figure 4, that for the teacher himself, it presents an overall score of 4.83 for the Methodology academic axis, that is, it shows evidence to use pedagogical teaching strategies used for the development of their academic activities.

Figure 4. Column Graph, for final Academic average for axle

Presenting the results like this, it allows personnel and teachers themselves to observe in which academic axis there is lack of content, which may be solved with corrective action required training.

You may be sharp in using different methodologies, but not in planning or executing, and as a result, you can be good in planning, but not in the other two academic axes

4. Conclusions

The present study had the following outcomes:

• During all the research process for validating evaluation’s instruments, the Management of the process was done with a great deal of rigor, in reference to the techniques (CMI, Delphy technique, and the determination of Alpha Value), during the different situations presented during the research’s development

• Three evaluation instruments were designed in order to collect information with a high degree of accuracy, to measure the academic achievement of faculty personnel at Universidad Tecnológica de Panamá, at Centro Regional de Veraguas. The compiled information was very relevant, due to the scarcity of valid and reliable instruments in the area, not only in Panama, but also in many higher education institutions in Latin-American countries, and in universities corresponding to industrialized societies. Presently, very few instruments addressed the three main areas of faculty assessment: Evaluation, Self evaluation and Co-evaluation.

• The main goal of the research was to achieve: the generation of useful instruments designed to evaluate the academic achievement of the faculty at Universidad Tecnológica de Panamá, Centro Regional de Veraguas.

• The advantage of the availability of scientifically validated instruments make possible to detect the strong and weak points of the university teaching process and the availability of the tools to improve them.

• Findings will allow more evidence to the discussion of the strengths and weakness of the planning, development and the methodology strategy used by faculty personnel.

• One of the strength’ of the evaluation process was the utilization of the strategic planning based on the information collected in the diagnostics of SOWT(Strengths, Opportunities, weakness and threats) and the importance of considering the internal and external environment.

• In this study, Stakeholders were defined to include academic departments, Faculty members and students, and the responsible for the implementation of the evaluation, Co-evaluation and Self-evaluation of the academic roles performed by the Faculty personnel at Universidad Tecnológica de Panamá, Centro Regional de Veraguas

• Cronbach’s Alpha scales were used for evaluating reliability and consistency index of the three dimensions: Planning, Development and Academic Activities; and for the three types of instruments: Evaluation, Self-evaluation and Co-evaluation ; and the definitions of three matrixes: strategic, evaluation of external factors -MSEF, and evaluation of internal factors-MSIF.