1. Introduction

Interorganizational collaboration and systemic change promise best realization of Building Information Modeling (BIM) benefits but a commonly accepted guide for evaluating its effectiveness has not been established. Pre-existing ineffective collaborative strategies is one of the reasons the construction industry has not adopted BIM to the fullest extent (Homayouni, et al., 2010). BIM has been cited in the literature as a promising technology that improves projects performance through increased efficiency (Azhar, et al., 2008; Becerik-Gerber & Rice, 2010; Becerik & Pollalis, 2006; Khemlani, 2007; Neelamkavil, 2009; Suermann & Issa, 2009; Woo, 2006; Yan & Damian, 2008). The industry is being challenged by the inability to fully utilize BIM technology that has the potential to increase efficiency of projects and, in turn, improve the industry’s declining productivity (Dyer, et al., 2012). Contractors, in particular, have been reported to lag behind architects and engineers in sharing BIM generated data (Gilligan & Kunz, 2007; Mutai, 2009; Suermann & Issa, 2009). Effective sharing of BIM generated data across organizations is referred to as interorganizational BIM - IBIM (Fox & Hietanen, 2007). In IBIM, various stakeholders interact when BIM technology is utilized on projects, to effectively implement activities interdependent beyond organizational boundaries. However, such interdependency of activities contradicts the industry’ competitive and fragmented work environment. This has proven difficult to adopt BIM to the fullest extent. Most literature calls for extensive collaboration with downstream project stakeholders to offer opportunities for sharing valuable input at early stages of BIM projects (Khanzode, et al., 2006).

Various approaches have been utilized, including motivating critical mass adoption, as well as identifying significant influential factors to guide BIM adoption process (Khanzode, et al., 2006; Mutai, 2009). Successful case studies to the adoption of BIM have been reported along with various challenges faced in practice (Becerik-Gerber & Rice, 2010; Suermann & Issa, 2009). Despite the efforts, most researchers agree that the industry has not best realized the benefits of BIM as a result of interorganizational interactivity challenges. Literature suggests, as BIM users focus on better management of the identified general influential factors, improvements will occur in effective interorganizational collaboration and systemic change. To date, there has been no systematic attempt to organize and synthesize the various sets of critical factors for collaboration and systemic change nor have a more complete evaluation guide been proposed. Even though there are long term accepted measures for general technological adoption such as technology-organization-environment (TOE) framework (Tornatzky & Fleischer, 1990), these are particularly focused on a company/organizational level. The extant literature lacks a clear consensus on interorganizational measures related to BIM adoption. Moreover, variations among interorganizational BIM studies and between the studies and the TOE factors, both in terms of categorizing and contextualizing the factors, suggest the TOE framework inadequately address the interorganizational context. In this present study the development and validation of an evaluation guide for interorganizational BIM adoptability (IBIMA) that is essential to assessing effectiveness of collaborative strategies and systemic change as practiced by BIM users is presented.

2. Methodology

The development and validation of the IBIMA evaluation guide presented in this study involved various methods (Figure 1.1). Ethical clearance for conducting this research was obtained from the Institutional Review Board (IRB) of Louisiana State University (LSU); IRB Approval #8345. Consent was sought from the study participants, in the form of a clearly written explanation of the aims and objectives of the study, prior to answering questions. The methodology approach is summarised next.

Figure 1.1. Process of Validating and Testing the IBIMA Evaluation Guide

Conceptualization

Conceptualization step involved identifying constructs and items from relevant literature. This was achieved through Formal Grounded Theory (FGT) method. The study determined and compared the factors influencing IBIM adoption as a function of existing theories. Based on scope of this study/unit of analysis, the classic Technology-Organization-Environment (TOE) theory was utilized as the basis of comparison. An interorganizational BIM adoption (IBIMA) theory was synthesized that extended the TOE theory to the topic of interorganizational BIM (Figure 1.2). Three key categorical factors (social, interoperability, legal), novel to the classic TOE framework, were referred to as collaborative factors that continuously overlap the TOE (basic) factors at an interorganizational level. The generated IBIMA theory theorizes how, in BIM adoption, interorganizational context variables relate to the organizational contextual factors presented in previous studies.

Figure 1.2. Extending the Technology-Organization-Environment Framework for Interorganizational BIM Adoption

Presenting the factors in a contextual specific approach would be useful to both decision makers and researchers. Identifying contextual specific strategies simplifies decision choices by decision makers and enhances their understanding of specific areas that have the potential to maximize BIM benefits. Researchers can use the identified measures to better understand the level of interactive practices necessary for companies to effectively adopt BIM. In addition, researchers can build theories and models that relate the identified critical measures to the companies’ overall BIM projects performance. The three collaborative factors represent critical areas of interorganizational interaction and systemic change where actions must be practiced to best realize the benefits of BIM. Measures of the collaborative factors, however, varied widely and were more numerous than expected-too numerous to embrace and use. Further examination of the collaborative measures was performed, through a survey methodology, to determine those measures that are critical to the interorganizational context.

Design

A representative sample of 165 US contractors completed an online survey about their adoption of BIM. Of these, 68 (41.2%) identified themselves as BIM users (59-sharing BIM data at an interorganizational level, and 9-companies sharing data within organizational boundaries). The remaining, 97 companies (58.8%), had not adopted BIM technology at the time of the survey. These were referred to as non-BIM users. Non-BIM users did not get the opportunity to evaluate the research variables because the research called for assessment of the variables based on practical experience. Instead, these were directed to a specific question that enquired the reason for not utilizing BIM technology on their projects. BIM users had the opportunity to evaluate the research variables based on their practical experience in utilizing BIM (see Table 1).

Table 1. Respondents’ characteristics

As referred to in this study, levels of interaction describe the following: communication - utilization of BIM for 3D objects visualization; coordination - utilization of BIM to a clash detection level; collaboration – utilization of BIM to a 3D BIM collaborative environment level; network-based - utilization of BIM to a supply chain integration level. Measures of the collaborative factors were tested for reliability and validity using perceptual data collected from a representative sample of 165 US contractors. Six critical factors (organizational variety, team BIM capability, duty of care, risk and liabilities, scope of work, and data preservation) were identified as more significant than other influential factors in practice. The identified six critical factors (scale) accounted for 71.089% of the total variance. The survey results demonstrated that reliability and validity of the scale are quite high. Hence, the scale captures most of the important aspects of interorganizational collaboration and systemic change discussed in today’s related research. It was recommended that identified factors (Figure 1.3) be practiced interactively as an interconnected facet rather than isolated. Study results suggested that attempts at either will not be successful without first establishing a comprehensive interactive strategy that supports both collaboration and systemic change within the interorganizational context.

Figure 1.3. Critical Factors for Interorganizational Collaboration and Systemic Change in BIM Adoption

Normalization

In addition to the Cronbach’s alpha values (ranging from 0.920 to 0.923), which indicated the scale is very reliable, a split-half analysis (Cronbach, 1951), was performed to further test reliability of scale. Spearman-Brown Coefficient (0.709) indicated the final 25-item instrument is an efficient measure of the interorganizational BIM adoptability (Figure 1.4). Overall, results showed that reliability of scale, based on both tests, is quite high and acceptable.

Figure 1.4. Split-Half Reliability Statistics

Alternate-form (enablers and inhibitors questionnaire) method was utilized to test the external reliability of the research instrument. Pearson correlation test was .354, while Spearman’s rho was .353. Correlation was found to be significant at the 0.01 level (2-tailed), p-value = .003. These measures indicated that the instrument has attained external reliability.

3. Validating the Interorganizational BIM Adoptability (IBIMA) Evaluation Guide

In this specific objective, the study demonstrates utilization of the proposed evaluation guide that includes a comprehensive set of six critical factors for interorganizational collaboration and systemic change in BIM adoption. The identified factors are literature-based, and have been examined for reliability analysis through a survey methodology. The evaluation guide was further validated through empirical research, utilizing quantitative data from three most recent BIM projects.

Case Study

Three companies were provided with an IBIMA evaluation guide to assess the evaluation guide based on their most recent BIM projects. The respondents were asked to rate their companies’ success on BIM use, based on each of the measurement items. The evaluation guide was measured using a five-point Likert-type scale with anchors ranging from 1 = “very low” to 5 = “very high”. To protect their identity, the three companies are referred to as company A, B, and C. The three companies’ projects were considered as being similar enough to offer a rational comparison. The companies’ revenue in million US dollars were 2,200 (A), 2,002 (B), and 1,300 (C). Companies A and B offered mostly general contracting services, while company C offered mostly design/build services.

The maximum IBIMA scale score (i.e. mean scores for each factor multiplied by the highest possible Likert scale score a company could get) for the six factors was 107.35 points. A threshold minimum for determining a company’s success or failure on IBIMA was calculated by multiplying the maximum score by the total amount of variance accounted (71.089%). Specific minimum scores for each factor’s ration contribution to the IBIMA scale was also calculated the same way. This meant that the company’s total score must, at the very least, meet the percentage rate of the total variance accounted by the scale, to be considered successful on IBIMA. Using this approach also helped to recommend specific areas that need improvement, where a company attained scores above the threshold but lower scores on either of the specific factors. A summary of this explanation is provided next, in the form of equations. S = social, I = interoperability, and L = legal, factors.

From the outlined equations, the minimum score that a company must attain from the three categories, to be considered successful on IBIMA are: 14.66 points (Social), 23.1 points (Interoperability) and 38.47 points (Legal), adding up to 76.2 points. Table 1.2 summarizes results of the IBIMA scores for the three projects.

Table 1.2. IBIMA Results of Three BIM Projects

4. Discussion

The identified critical measures provided a reliable guide for evaluating the effectiveness of interorganizational interaction in BIM projects. The utilization of the evaluation guide was quantitatively demonstrated using the data collected from three companies among the US contractors. The three projects were considered similar enough to provide a logical comparison (Table 1.2). Thirty six (36) companies that voluntarily provided their contacts during the initial survey were contacted. A request was sent for information related to their most recent BIM projects. However, only three companies responded with complete information. The three case study results shade light on specific areas that have the potential to maximize BIM adoption.

Companies A and B both implemented similar BIM projects, in terms of cost ($1 million) and time (1 month). The two companies also operated in the same geographical region (South) and had implemented more than 26 BIM projects, at the time of the study. However, their scores on collaborative (social, interoperability, legal) factors varied significantly. As indicated (Table 1.2), company A scored below the required minimum (unsuccessful) on all the three factors, whereas company B scored above the required minimum (successful) on all the three factors. Company A had an intermediate level of BIM experience, while interacting at level 1. Meanwhile, company B had an advanced level of BIM experience, while interacting at level 3. Company B was successful (scoring above the required minimum) on all the three factors while company A was not. While the two companies differ in terms of BIM experience and the level of interaction, a larger sample is necessary to conclude the impact of the two on the factor scores.

While companies B and C were both successful on the factor scores, they still indicated specific concerns with regard to the collaborative factors. Further, it is worth noting that their biggest concerns were significantly different; that is, company B (social) and company C (interoperability). Meanwhile, they both differed from company A (legal) that was unsuccessful on all scores of the collaborative factors. These findings suggest that the three collaborative (social, interoperability, and legal) factors are equally influential to the adoption of BIM at an interorganizational level. The IBIMA evaluation guide, comprising the six critical factors (presented as challenges /inhibitors on the left) and specific item measures (presented as strategic measures/enablers on the right) is presented in Figure 1.5.

Figure 1.5. Evaluation Guide for Interorganizational BIM Adoptability

Implication

The 25 – item instrument for interorganizational BIM adoptability (IBIMA) has undergone extensive evaluation and validation, which represents significant progress toward developing a standard instrument. Further, the study showed that the developed instrument is precise and can easily be utilized in practice. This instrument serves as a starting point for a detailed evaluation of the interorganizational collaboration and systemic change necessary to effectively adopt BIM.

Limitations and Suggestions

Demonstrating the interorganizational BIM adoptability guide utilized quantitative data from 3 companies with similar projects. Future studies could expand the sample size and compare results to corroborate the findings. The evaluation guide comprised 25-items within the collaborative category. Further research should consider combining the two categories, collaborative and basic, to compare results. Testing the 25-item instrument involved companies of large or very large size. Future studies should expand the sample across company sizes and compare results for potential differences. Involving only design/build companies would be another area of interest to further clarify the reasons why statistical analysis showed no significant differences between design/build and non-design/build companies, despite the argument that in design/build collaboration is made easier as everything is done under one roof.

Additionally, future research should consider involving other stakeholders (e.g. designers) who play an important role in BIM data exchange, to further validate the proposed instrument.

5. Conclusions – Interorganizational BIM Adoptability (IBIMA) Evaluation Guide

Different sets of organizational and interorganizational BIM requirements have been offered by different authors. However, no previously published research has developed a comprehensive set of requirements or critical factors that spans the literature. The present research offers a set of six critical factors for collaboration and systemic change, synthesized from various authors. Extant literature on BIM provides little guidance on how to interactively evaluate the proposed critical factors at an interorganizational level. This study successfully developed an instrument that can be used to evaluate interorganizational interactivity to maximize BIM adoption. The measures proposed were empirically based, and shown to be reliable and valid. The reliability coefficients (alphas) ranged from .920 to .923. Split-half reliability test (.709) and Pearson correlation test (.354) also indicated the instrument is reliable. Further, a systematic literature review, through grounded theory, and refinement of the survey by a panel of experts, helped ensure that the measures have the content validity. The correlation coefficients (≥ 0.5) further offered strong evidence of criterion-related validity.

The proposed evaluation guide for IBIMA permits managers to obtain a better understanding of the level of interaction in practice. It allows researchers to proceed with the task of developing and testing theories of effective interorganizational collaboration and systemic change in a fragmented and competitive work environment. Managers can use this guide to evaluate their companies’ interactivity level in practice. These measurements can help decision makers identify those areas with the highest potential to maximize BIM benefits. Also, comparisons of different organizations or divisions can be made to help prioritize interactivity practices. The findings presented in this study are encouraging but a great deal of further research remains to be done towards proposing a standard instrument for evaluating IBIMA across disciplines. Future research could replicate the empirical work reported here to corroborate these results. In addition, future studies could involve more items, and larger, more broadly based samples. It is expected that the findings of this study will provide momentum for future research aimed at gaining a better understanding of the collaboration and systemic change necessary to effectively adopt BIM. Overtime, future research will further validate the present findings toward a standard evaluation guide for interorganizational collaboration and systemic change necessary to adopt BIM to the fullest extent.