1. Introduction

“Publish or perish” has not been mandatory in universities in Thailand. But it may become inevitable in the near future. In responding to the need to differentiate, select, and promote research productivity, more appropriate, reliable, and valid measures are the prerequisite. This study aims at constructing and conducting preliminary validation of a measure on “research-based performance” (RBP) for use with Thai university lecturers and high school teachers.

2. Literature Review

2.1. Definition and Measurement of Research-based Performance

Research productivity has been measured by the number of published articles in academic journals and conference presentations (Marsh & Hattie, 2002). This type of assessment is popular in more advanced countries, but less appropriate in developing areas. An available study on productivity of research in economics revealed that Thailand, as well as other developing countries rarely published in top 10 international journals (Kocher, Luptacik, & Sutter, 2006). In addition, work concerns of faculty members at midlife in the Philippines did not appear to include research productivity, but might cover research teaching (Galeon, 2015).

Number of publications in well-known journals seems to be one of the important consensus indicators of successful researchers (Toutkoushian, Porter, Danielson, & Hollis, 2003, Kranzler, Grapin, & Daley, 2011). Besides article publications, a study on comparing research productivity suggested that individual’s research productivity should also be assessed in terms of grant funding, and number of Ph.D. students under supervision (Wootton, 2013).

However, such indicators of research productivity may be insensitive to differentiate levels of individual success. In Thailand, international academic journals and organized conferences entered academic arena only within the last ten years. Since English is not the official language, research publications in Thai national journals are becoming more popular (Sombatsompop, Chancheewa, Markpin, Premkamolnetr, Ittiritmeechai, Wongkaew, Yochai, & Ratchatahirun, 2012). Thus, the number of international publications may not be sufficiently sensitive to differentiate the amount of research-related work produced by Thai scholars, who are usually at the medium and low-end of the continuum.

In the present study, research-based performance of Thai scholars is originally defined in three dimensions, i.e., 1) applying research results and research skills in teaching and giving consultations, 2) performing research-related work, such as training, special lecturing, evaluating research proposals and reports, and 3) disseminating research at conferences and in academic journals (national / international). Thirty items with 6-unit rating scales ranged from “very true” to “not true at all” were constructed and tried-out in Thai scholars. Then two steps of Factor Analysis were carried out on the data.

2.2. Behavioral Intention and Research-Based Performance

According to Theory of Reason Action (Fishbein, & Ajzen, 1975), intention of a person to perform a certain behavior, as the part of attitude, is directly related to that behavior. Intention or readiness to act has been found to play an important role in performance. A study from 297 North Americans indicated that positive research attitudes related to research behavior (Milburn, Brown, & Paine, 2001). Thus, it can be expected that the higher the intention to conduct a research study (RBI), the more the individual performs research related activities (RBP).

2.3. Civic Moral Disengagement and Research-Based Performance

The cognitive mechanism related to immoral conducts and behaviors is one of the urgent issues of study in the academic community. This mechanism for psychological and social survival is investigated as moral disengagement proposed by Bandura (2002). In 2009, Caprara, Fida, Vecchione, Tramontano, and Barbaranelli had constructed a new measure on “civic moral disengagement”. This measure was used for comparing psychology and law university students (Sagone & De Caroli, 2013). The study revealed that psychology students used the mechanisms of moral disengagement more than law students. It can be expected that in general, psychology scholars will engage in research more often than law scholars. Thus, there is the possible link between civic moral disengagement (CMD) and research activities. As a consequence, this study employed civic moral disengagement as one of the variables for validating the RBP scale. However, the civic moral disengagement scale was expected to play less important role in convergent validation of the RBP scale than the intention to engage in research activities, according to the Theory of Reasoned Action (Fishbein & Ajzen, 1975).

2.4. Work Duration and Research-Based Performance

A study on career incentive and research performance employed at least one thousand Taiwanese faculty members (Tien, 2007). Two types of faculty members, namely, new faculty and experienced faculty from three fields of study (natural science and engineering, humanities, and social science) were compared. The findings indicated that faculty members from all fields were highly motivated to produce publications. However, after being promoted, it seems that the number of publications of these faculty members rapidly declined in humanities and social science, but not in natural science and engineering. Thus, it can be expected that the longer the researcher is in the fields of social science and humanities, the less the research activity they perform.

3. Research Hypotheses

There are three hypotheses in this study.

Hypothesis 1: By performing exploratory factor analysis, RBP measure is composed of at least three factors, with the total of at least 10 items, with the cumulative percentage of at least 60%.

Hypothesis 2: By performing confirmatory factor analysis, the underlying latent constructs emerging from the EFA for RBP measure are confirmed by another set of data.

Hypothesis 3: The relationships between other constructs and the RBP construct can be hypothesized as follows:

Hypothesis 3.1 The relationship between RBP and RBI is positive and greater than the relationship between RBP and CMD.

Hypothesis 3.2 RBI, CMD and work duration after last degree attainment are important predictors of RBP and together can account for the total variance of at least 40%.

4. Research Method

4.1. Samples

The sample in this study was 888 Thai scholars from schools and universities in Bangkok, and outer provinces. The sample was divided into three groups. The first group of 188 scholars was used for item quality analysis. It consisted of 48 males (25.90%) and 137 females (74.10%) with the average age of 40.13 years (SD = 9.32), and average work duration after the last degree was 13.88 years (SD = 11.78). The second group of 400 scholars was used for Exploratory Factor Analysis (EFA). It consisted of 138 males (34.50%) and 262 females (65.50%) with the average age of 44.51 years (SD = 12.88), and the average work duration after the last degree was 18.47 years (SD = 13.13). The third group of 300 scholars was used for Confirmatory Factor Analysis (CFA) and validation. It consisted of 87 males (29.60%) and 207 females (70.40%) with the average age of 41.87 years (SD = 12.07), and the average work duration after the last degree was 16.20 years (SD = 13.05).

4.2. Research-Based Performance (RBP)

The RBP scale was originally based on the literature review, and constructed with three factors preliminary covering the spectrum of low-end performance of “appearing to be a researcher” to medium performance of “being accepted as a researcher” to the high-end performance of “being a leading researcher”. The initial 30 items were endorsed from a group of experts in the field of psychology, education, and social science. Each item was accompanied with 6-unit Likert-type scale ranging from “very true” to “not true at all”.

After screening via two statistical approaches, item discrimination (t-ratio) and item-total correlation, using data from the first set, 25 items were selected. EFA was performed using the second set of data, to discover the emerging factors of RBP scale. Second-order confirmatory factor analysis was carried out using the third set of data to confirm the construct of the RBP scale. Criterion-related validity in both directions of convergence and discriminant validation was tested with RBI and CMD, respectively. Reliability of the final RBP scale was computed.

4.3. Measures

Research behavioral intention (RBI) consisted of five factors from EFA results with cumulative percentage of 62.71% in explaining the construct variance (Bhanthumnavin, 2014). The factors follow 1) intention to prepare for and ready to do research (4 items), e.g., I pay attention to national research policy in organizing my research proposal) 2) determination to carry out high standard-research work (4 items) e.g., I put efforts and time to produce high standard research. 3) efforts for carrying out research study (4 items) e.g., I try to avoid conducting rigorously advanced research study (with reversed scoring). 4) trying to enhance own research opportunity and ability (3 items), e.g., I try to keep up with new research trends, and 5) avoiding research work (2 items with reversed scoring)), e.g., my work position does not require me to do research study. The five-factor model with the total of 17 items was confirmed by second-order factor analysis with the test score reliability of 0.86.

Civic moral disengagement (CMD) was constructed by Caprara and others (2009) assessing 8 dimensions of civic moral disengagement with the total of 32 items. Only 12 items from three relevant dimensions were selected and translated for use in this study: 1) advantageous comparison (original items no. 4, 21, 23, and 27) 2) diffusion of responsibility (original items no. 6, 13, 16, and 30) and 3) dehumanization (original items no. 2,8,28 and 31). The reliability of this 12-items scale score was 0.87.

Work duration after last degree graduation (WDG) was measured in terms of working years (data in number of months) of the respondents after the last training in the highest educational level.

4.4. Data Analysis

Item quality of RBP was assessed by two statistical approaches: item discrimination and item-total correlation using the data in the first group. Using the data from the second group of scholars, the selected items scores were employed for EFA in order to reduce number of items and to identify the emerged factors. The three criteria for EFA were as follows (Hair, Jr., Black, Babin, & Anderson, 2010): 1) a measure of sampling adequacy (Kaiser-Meyer-Olkin test) should be greater than 0.60, 2) the Bartlett test of sphericity should be significant, and 3) the retained number of factors yields the total cumulative percentage for explaining the total variation of at least 60%. Second-order confirmatory factor analysis was performed using the third group of data to verify the emerged factors of the construct. The five most commonly used indices for CFA are non-significant Chi-Square value (Jöreskog, & Sörbom, 1996), a root mean square error of approximation (RMSEA) value of less than 0.50 (Browne & Cudeck, 1993), a comparative t index (CFI) of at least 0.90 (Bentler, 1990), Tucker-Lewis Index (TLI) of at least 0.95 (Tucker & Lewis, 1973), and a standardized root mean square residual (SRMR) of less than 0.80 (Hu & Bentler, 1998).

5. Research Results

5.1. Item Quality

The item quality of RBP measure was tested by performing two statistical analyses using the first data group of 188 scholars: 1) item discrimination (t-ratio) to examine the difference between the mean scores of the lower and upper groups (30% of total respondents for each group), and 2) the item-total correlation coefficient (r) revealed the relationships between the item scores and the total test scores without the target item. The results in Table 1 revealed that only 25 out of 30 items were retained (item no. 14,19,24,25 and 28 were excluded).

Table 1. Summary of t-ratios and item-total correlations of RBP measure

5.2. Exploratory Factor Analysis of RBP

EFA technique using the data from the second group of 400 scholars was computed for additional step based on 25 selected items from the previous step. The EFA results showed an adequate fit of Kaiser-Meyer-Olkin test (KMO = 0.86) with a significant Bartlett test of sphericity (χ² = 1848.24, df = 105, p<.000, n = 400).

The results indicated five-factor model of RBP measure (Table 2). Factor 1 with the eigenvalue of 4.91 was labelled “Research productivity”. It was considered high-end performance. This factor includes five positive items, explaining 32.75% of the variance of the RBP construct.

Table 2. Results of EFA for RBP measure

The second factor consisted of three negative items with the eigenvalue of 1.53 which was labelled “Research unproductivity” which was low-end performance. This factor could additionally explain the variance of RBP measure with 10.23%, which led to the total cumulative variance explanation of 42.98% of this construct.

The third factor, labelled “Absence of research-related activities” consisted of three negative low-end performance items with the eigenvalue of 1.31. This factor could additionally explain the variance of RBP measure with 8.76%, which led to the total cumulative variance explanation of 51.74% of this construct. The fourth factor, as medium performance, consisted of two positive items with the eigenvalue of 1.03 which was labelled “Dependable as researcher”. This factor could additionally explain the variance of RBP measure with 6.89%, which led to the total cumulative variance explanation of 58.63% of this construct. The fifth factor, labelled “Teaching research and award winning” consisted of two positive high-end performance items with the eigenvalue of 1.00. This factor could additionally explain the variance of RBP measure with 6.72%, which led to the total cumulative variance explanation of 65.35% of this construct. Thus, the results mostly supported hypothesis 1.

5.3. Second-Order Confirmatory Factor Analysis of RBP Measure

The results of a second-order confirmatory factor analysis using the third data group indicated a model fit with a Chi-Square value of 90.18 (df = 71, p value = 0.06), RMSEA = 0.03, CFI = 0.98, TLI = 0.98, SMRS = 0.03, which supported hypothesis 2. Therefore, the five-factor model with 10 items of RBP measure was confirmed (Figure 1).

Figure 1. Second-order factor analysis of Research-based performance (RBP) in Thai scholars (χ2 = 90.18, df = 71, p-value = 0.06, RMSEA = 0.03, CFI = 0.98, TLI = 0.98, SMRS = 0.03, N = 300)

The highest gamma value of 0.95 belonged to the factor of “Research unproductivity”. The second important factor of RBP construct was “Research productivity” with the gamma value of 0.94. The third important factor was “Dependable as researcher” with gamma value of 0.92. “Teaching research and award winning” was the fourth important factor with the gamma value of 0.79. The least important factor of “Absence of research-related activities” was found with the gamma value of 0.75.

5.4. Reliability and Validation of RBP Measure

Criterion validity of RBP measure was verified. From Table 3, it was found that RBP and RBI were positively correlated (r = 0.68, p<.01) to a moderate degree, while the relationship between RBP and CMD is not significant (r = 0.06, ns) with Z value of 9.278, p<.001. Thus, hypothesis 3.1 was supported. Next, Multiple Regression Analysis was performed on the data. Results indicated that RBI, CMD, and WDG accounted for 48.80% of the RBP variance. The first important predictor was RBI, followed by CMD and WDG (β = .67, .10, and -0.09, respectively). Thus, hypothesis 3.2 was supported. The reliability of the score of RBP is 0.87.

Table 3. Correlation coefficients among variables

6. Conclusions and Discussion

The aim of this study was to develop and preliminarily validate the “research-based performance” scale for use with social science and education scholars in Thailand. Evidently, this RBP measure is more effective in differentiating Thai scholars at the middle and the lower-end of the continuum. Thus, it can be said that the newly constructed RBP scale will be more suitable for use with Thai scholars than the regular “research productivity” measures employed in most other countries.

Furthermore, this new RBP scale employed the summated rating method. It assumed interval scale which has more advantage than other types of scale, such as the one employed in the “research involvement” measure (Whelan and others, 2013). The 6-unit rating scale which accompanied each of the 15 RBP items gives a wide continuous range of scores from 15 to 90. Evidently, the summated rating method used gave more suitable data for performing factor analyses (Spector, 1992).

Criterion-related validity of the scores on the RBP scale was investigated by using three approaches. First the convergent and discriminant validity was demonstrated by correlations and a Z-test. Secondly, the antecedent-like factors were used as predictors of the variance in RBP scale by performing Multiple Regression Analysis. Thirdly, correlations among the five factors of RBP scale were discussed. Thus the convergent validity of the RBP demonstrated in this study leads to high confidence in the new scale.

The relationships among five factors of RBP scale were examined. It was found that “research productivity” factor (factor 1 in Table 3) showed strong relationships with “dependable as researcher” (factor 4) and “research unproductivity” (factor 2) (r = 0.61 and 0.60, respectively). On the contrary, “teaching research and award winning” (factor 5) had lower relationships with the four other factors (r = 0.47, 0.31, 0.32, and 0.39, respectively). These results may imply that teachers of research methodology are not the most qualified researcher.

It should be mentioned that the item of “receiving research award within 5 years”, showed acceptable loading in factor 5. While factor 5 had less association with other four factors (see Table 3) which reflected research competence. One explanation might be that most award winners were among team members rather than being the major researchers.

However, some limitations and recommendations for further studies are offered as follows. The use of summated rating method for the RBP scale as well as the measure of research intention and civic moral disengagement (except the measure of work duration) may have a problem of common method bias which can inflate the results from multiple regression analysis and correlations. The benefit of using common method for measuring many important variables in a study may greatly outweigh the disadvantages. This argument is still going on with the pros and cons siders (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003; Spector, 2006).

It is also expected that the RBP measure in this study will be suitable for use with many types of Thai scholars, not only those in social science and education. Furthermore, the measure may be appropriate for use with scholars from other developing areas as well. Thus, studies on measurement invariance of this variable should also be carried out in future study.

Another important research finding was on the positive relationship between CMD and the factor 1 of RBP (Table 3). This means that the Thai scholars with high research productivity tended to use more moral disengagement as defense mechanism. A recent article by a psychologist in Florida, USA (Daneil, 2013) emphasized the importance of ethical issues in research process and report, such as plagiarism, bias, and false reporting. Misreporting in research or “lying” was conceptualized as having two goals as lying for benefit to self or lying for benefit to group or society (Borsellino, 2013). However, in most quantitative research studies, advanced methods in measurement and statistical analysis have protected the researchers from going astray. As for misreporting of research results, this type of lying is usually for the benefit to the researchers themselves. But at the same time, the wrong doing can harm others, society, and the advancement of knowledge.

Ethical research dilemma can easily occur. It was found that in such situation, moral disengagement was employed more frequently by the researchers who had high moral identity (Mulder & Aquino, 2013) which made them more sensitive to minor wrong doing. Thus, the present study in Thai scholars has pinpointed a complex phenomenon on the relationship between moral disengagement and RBP. One of the important possible moderators of this relationship may be the magnitude of moral identity of the researcher themselves. Consequently, further research investigations should be carried out in order to shed light on the advantages of research training and researcher development.