1. Introduction

In the realm of construction projects, accurate cost estimation plays a crucial role in project planning, budgeting, and decision-making. Estimating construction costs is a crucial task that offers essential financial information for making project decisions. During the initial stages, cost estimation is used to assess the feasibility of the project. Since there is a need for flexibility in adjusting the project scope, design, specifications, and standards at this stage, it is vital to conduct construction cost estimation as early as possible. However, it is impractical for construction companies to invest excessive time and effort in design during the early stage when resources are limited. The estimator's efforts become more valuable in the later stages of estimation. Consequently, numerous researchers have explored the use of data-based techniques for early construction cost estimation. These techniques rely on information from previous similar projects and the target (new) project, proving to be valuable in supporting early construction cost estimation. Conceptual estimating serves as a guideline for building decisions, project timelines, and budget-friendly design features. While conceptual estimating and cost estimating are similar, they have slight differences. Conceptual estimating occurs during the early conceptual phase of a project, providing a generalized cost estimate before construction work begins. It relies on industry information and historical data and may not be exact, but it serves as a basis for budget decisions. On the other hand, cost estimating takes place later in the process and considers more factors, producing more accurate results by analyzing all relevant cost data. Completing a conceptual estimate is important because it determines anticipated total construction costs, informs project decisions, and enables effective cost control. It helps clients plan within their financial means and saves time and money by making cost-effective decisions early on. Additionally, a good conceptual estimate considers various project factors, provides a comprehensive outlook on total costs, and serves as a foundation for construction planning. However, due to the inherent uncertainties and complexities associated with construction, cost estimates are often subject to various risks and uncertainties. To address this challenge, professionals in the construction industry have developed the Conceptual Cost Estimate Reliability Index (CCERI), a metric that quantifies the reliability and accuracy of conceptual cost estimates. The CCERI takes into account several factors that can impact the accuracy of conceptual cost estimates, including the level of project definition, availability of project information, expertise of the estimating team, project complexity, and historical data availability. By considering these factors, the CCERI provides a comprehensive assessment of the estimate's reliability. It is typically expressed as a percentage or a rating, indicating the degree of confidence in the estimate. Calculating the CCERI involves comparing estimated costs with actual cost data from similar completed projects or benchmarks. Historical data, industry standards, and expert judgment are utilized to establish a basis for comparison. Higher CCERI ratings or percentages reflect a higher degree of reliability in the estimate. The CCERI serves as a valuable tool for project stakeholders, including owners, developers, contractors, and financiers, providing them with a measure of confidence in the financial aspects of construction projects. It aids in understanding potential risks, making informed decisions regarding project feasibility, budget allocation, and financial planning. Furthermore, the CCERI contributes to risk management and contingency planning, allowing project managers to allocate appropriate contingencies and enhance overall financial management. The CCERI primarily focuses on conceptual cost estimates that are developed during the early stages of a construction project. However, as the project progresses and more detailed information becomes available, the reliability of cost estimates improves. In order to enhance this reliability, additional techniques such as Artificial Neural Networks (ANNs) can be employed, as they have proven to be superior in capturing complex relationships. Nevertheless, the CCERI remains an important tool for stakeholders to ensure the accuracy and reliability of conceptual cost estimates. [1-5]

On the other hand, the main reason for the inaccuracy of cost estimation is primarily linked to inadequate experience with similar projects, limited time for cost estimation, fluctuations in resource prices over time, and incomplete project information, even when performing detailed estimations. In situations where project information is lacking, conceptual cost estimation has been conducted using parametric estimation methods. By employing AI techniques, knowledge and experience from previous examples of cost estimation can be harnessed. These AI techniques, when integrated with parametric estimation methods, can establish a non-linear relationship between influential parameters and the estimated construction cost.

However, a significant unresolved issue is the fluctuation of unit prices for cost items over time. Relying on previous estimating experiences for construction cost estimates may lead to failure due to changes in unit prices. While previous approaches introduced the idea of an "overall price index" as a parameter for adjusting construction costs, individual cost items don't experience price fluctuations in the same manner simultaneously. Attempting to adjust all unit prices with a single "overall price index" does not accurately reflect the variation of unit prices in the market. A more effective approach involves separating and managing the two elements of the cost item. The quantity of a cost item can be influenced by factors such as structural design dimensions, construction methods, and site environmental conditions. The value of this element remains unchanged as long as the same structure is being constructed using the same method under the same environmental conditions. Conversely, the unit price of the cost item may vary over time. Therefore, it is reasonable to utilize the most updated unit prices for the cost items in order to reflect the real situation of the current marketplace. [6-10]

This article proposes an integrated approach that combines conceptual estimation, early construction cost estimation, and case-based reasoning (CBR) to assess and enhance the reliability of conceptual cost estimates in building construction projects. By integrating these methodologies, the aim is to develop a simple, user-friendly, and comprehensible assessment tool that leverages industry professionals' expertise and knowledge while being validated with real-world construction data. In other words, the proposed approach offers a comprehensive framework for improving the reliability of conceptual cost estimates by leveraging historical cost data from previous projects in the early estimation stages. In other words, The purpose of this study is to develop a practical and effective tool, the Conceptual Cost Estimate Reliability Index (CCERI), that incorporates the weighted influence of factors affecting cost estimate quality. The study aims to validate the CCERI through analysis of cost data from 100 building projects and determine its usefulness in assessing the reliability of conceptual cost estimates. Additionally, the research aims to utilize the Analytic Hierarchy Process (AHP) to assign weights to the influencing factors and employ the Welch ANOVA test for statistical analysis. The study also considers the utilization of Microsoft Azure Cloud for data storage and analysis. Ultimately, the goal is to provide construction practitioners with a reliable tool that supports decision-making processes and facilitates successful project management in building construction projects.

2. Importance of Factors in Evaluating the Reliability of Conceptual Cost Estimates

2.1. Evaluating the Reliability of Conceptual Cost Estimates

The evaluation of conceptual cost estimate reliability typically involves assessing the quality of the estimates. Quality, as defined in previous research, pertains to the estimated cost falling within the expected accuracy range in the cost estimate area, while reliability is gauged based on the range of accuracy. Consequently, the reliability of a conceptual cost estimate is contingent on whether the expected accuracy range aligns with the required accuracy range [11-13].

Figure 1 illustrates how conceptual cost estimation during pre-construction stages influences the optimization of actual project costs. The accuracy of a conceptual cost estimate can be described as the disparity between the actual and estimated costs and can be quantified by calculating the error rate using Equation (1):

(1)

Figure 1. Reliable conceptual cost estimates steps

2.2. Identifying the Influential Factors in Conceptual Cost Estimates

The reliability of conceptual cost estimates depends on various factors, including the level of available data and the time allocated for estimation. Identifying these influencing factors and understanding their impact is crucial in assessing the accuracy of conceptual cost estimates. To achieve this, an extensive literature review was conducted, and 35 influencing factors were identified from related studies [5-12]. These factors were further refined through interviews with experienced cost experts with 7 to 10 years of estimating background. Subsequently, using Microsoft Azure Cloud, the experts narrowed down the factors to a final selection of 20, which were then categorized into five different areas (Table 1).

Table 1. Influencing Factors on Conceptual Cost Estimates

Among these factors, the effectiveness of data collection and data measurability emerged as the most critical considerations. Consequently, certain theoretically important factors, such as the level of communication with the original architect/designer and the probability of market condition changes, were excluded from the research due to their challenging quantitative measurability. Additionally, factors that could be derived from other factors were also omitted from this study.

2.3. Factor Weighting through Analytic Hierarchy Process (AHP)

The Analytic Hierarchy Process (AHP), developed in the early 1970s, is a structured tool designed to address complex decision-making problems by breaking them down into smaller components and ranking these components based on expert knowledge. AHP has found applications in various construction areas as a systematic approach to decision-making and problem-solving. It employs pair-wise comparisons to determine the relative importance weights of factors for goals. Additionally, the consistency of judgments can be evaluated from the comparison matrix to ensure acceptable levels of consistency [13-17].

This study utilized a 6 categorized group, with 30 factors from Table 1. Level 1 represented the goals, which focused on the effects on conceptual cost estimates. Level 2 consisted of 6 categories, and Level 3 contained the decision elements, namely, the influencing factors on conceptual cost estimates. To assess the weight of each factor, a questionnaire was created, and experienced cost engineers from major Iranian construction companies were asked to provide pair-wise comparisons using a scale of 1–9 to evaluate the relative importance of the factors. Twelve cost engineers, with an average working experience of 11 years (including five years in the cost estimating area), returned completed questionnaires. Expert Choice, a software package for AHP, was employed to analyze the pair-wise comparisons, and the consistency was checked. The consistency ratios for the 12 questionnaires were 0.03, which falls within the acceptable level (0.1) proposed by Saaty [18-22].

The AHP approach effectively determined the relative importance weights of factors impacting conceptual cost estimates, representing the domain knowledge of the experts. Table 2 presents the weights of the 20 factors utilized in the AHP analysis.

Table 2. Importance of factors by AHP

2.4. Creating a Reliability Index for Conceptual Cost Estimates

Data Collection

To validate the proposed CCERI assessment model in this research, data was gathered from 100 completed building construction projects carried out by general contractors in Iran. The data encompassed both the evaluation of the conceptual cost estimate's reliability and measurements of various influencing factors. As previously discussed, the reliability of the conceptual cost estimate can be determined by its expected accuracy range. In this study, the reliability was assessed using three categories based on the error rate range: Class 1 (±0–5%), Class 2 (±5–10%), and Class 3 (over ±10%), representing the difference between the conceptual estimated cost and the actual cost. The classification of these error rate ranges was based on the study conducted by Ahuja and Campbell [9], which identified a 15% error rate as common during the concept stage of a project, while 10% in the detailed design phase and 5% in the tender preparation phase were deemed acceptable. Out of the collected data, 35 projects fell into Class 1, 30 into Class 2, and 35 into Class 3. To assess the reliability of the conceptual cost estimate, twenty factors were measured. Among these, 17 were evaluated using a numerical scale of 1–5. The remaining three factors, namely estimator's career experience, estimator's fieldwork experience, and time to estimate, were measured numerically. To convert these numerical values into ordinal data, Table 3 was utilized. Among the 17 factors measured on an ordinal scale, three factors (level of construction difficulty, level of competition, and contingency) initially displayed negative relationships with the conceptual cost estimate's reliability. However, for the sake of consistency with other factors, these were transformed into positive relationships during the data analysis and conversion process. The CCERI, as described in the following section, was developed through this data analysis and conversion process, as explained.

Table 3. The data process used to convert numerical data into ordinal data

Reliability Index for Conceptual Cost Estimates

The CCERI represents a numerical score incorporating the weights of 20 factors that influence the quality of a conceptual cost estimate. To calculate the CCERI score, each element's weight, as shown in Table 2, is multiplied by 1000 to assign values within the range of 0-1000. The spreadsheet in Figure 2 demonstrates the calculation process. Each element within four categories, as shown in Figure 2, is evaluated on a scale of 1-5 and then multiplied by its respective weight. The resulting values are summed to obtain subtotals for each category, and these subtotals are added to derive the total CCERI score. The maximum possible CCERI score is 5000.

Figure 2. Sample CCERI Score Sheet for Building Projects

3. Principals, Analysis and Verification

As mentioned earlier, real-world data from 100 building projects' cost estimates were utilized to analyze and validate the CCERI. The CCERI scores for these 100 projects are summarized in Table 4. To establish a meaningful guideline for categorizing projects into Class 1, Class 2, or Class 3, which represent distinct ranges of error rates in conceptual cost estimates, we conducted Welch analysis of variance (Welch ANOVA) tests. This step is crucial in the research process as it allows cost estimators to take necessary actions to enhance the reliability of conceptual cost estimates if the CCERI score indicates potential issues. For instance, cost estimators could be alerted when the CCERI score falls below 3000, indicating a possible error rate exceeding 10%. The Welch ANOVA test is a valuable statistical tool for assessing significant differences between variable means. Welch's ANOVA, also known as the Welch test or Welch's modified ANOVA, is a statistical test used for comparing the means of three or more groups to determine if there are significant differences between them. It is an alternative to the traditional ANOVA, specifically designed to handle situations where the groups have unequal variances or unequal sample sizes. The traditional ANOVA assumes that the variances of the groups being compared are equal, a condition known as homoscedasticity. However, in real-world scenarios, this assumption may be violated, and the variances between groups may differ significantly. In such cases, using Welch's ANOVA is more appropriate and reliable. The main advantage of Welch's ANOVA over traditional ANOVA is that it provides accurate results even when the groups being compared have unequal variances. By considering the variability within each group and accounting for unequal sample sizes, it offers increased statistical power and robustness.

Table 4. CCERI scoring Results

Here are some key points about Welch's ANOVA:

- Unequal variances: Welch's ANOVA does not assume equal variances between groups. It calculates separate variances for each group and adjusts the degrees of freedom accordingly.

- Unequal sample sizes: Welch's ANOVA can handle situations where the groups have different sample sizes. It accommodates this imbalance in the calculations, providing valid results.

- T-test extension: Welch's ANOVA can be seen as an extension of the t-test, allowing for comparisons of means among multiple groups rather than just two.

- F-value: Welch's ANOVA calculates a test statistic called the F-ratio or F-value. This value is compared to a critical value from the F-distribution to determine statistical significance.

- Interpretation: If the calculated F-value is greater than the critical value, it indicates that there are significant differences between the means of the groups. Post-hoc tests, such as Tukey's honestly significant difference (HSD) test or Dunnett's test, can be used to determine which specific group means differ significantly.

In this study, as the data belong to different groups with unequal variables and sizes, it was imperative to develop a CCERI score that is user-friendly, intuitive, and meaningful in assessing the reliability of conceptual cost estimates with statistical significance. To determine the appropriate CCERI score, initial Welch ANOVA tests were performed using a score of 2000 to examine if there were statistically significant differences between the scores of Class 1, Class 2, and Class 3. Repetitive Welch ANOVA tests were then carried out with CCERI scores of 2500, 3000, 3500, and 4000. The results presented in this section pertain to the Welch ANOVA tests conducted with a CCERI score of 3000.

The analysis between Class 1 and Class 2 showed that the calculated p-value (0.221) was higher than the significance level of 0.05, indicating support for H0: µ1=µ2. Therefore, there was no statistically significant difference between the means of the two samples. On the other hand, the analysis results between Class 1 and Class 3 demonstrated a significant difference in means at the 0.05 significance level (p=0.0001). Similar results were obtained in the analysis between Class 2 and Class 3, where the p-value was 0.0001. In summary, while there was no statistically significant difference between Class 1 and Class 2, Class 3 exhibited significant differences from both Class 1 and Class 2. Consequently, 3000 was determined to be a meaningful CCERI score for evaluating the reliability of conceptual cost estimates. Table 5 presents the error rate range for each Class along with the number of cases where the CCERI score was below 3000. For Class 3, out of the 35 cases, 19 had CCERI scores below 3000, indicating errors exceeding 10%. The analysis reveals that a conceptual cost estimate with a CCERI score below 3000 has a high probability of exceeding 10% error, rendering such conceptual cost estimates unreliable.

Table 5. Data numbers with CCERI scores below 3000

4. Discussion and Results

As depicted in Table 6, we identified ten key factors that significantly influence conceptual cost estimates based on the data in Table 2. These ten factors collectively account for 70% of the impact on conceptual cost estimates. Notably, among the twenty factors, the time to estimate emerged as the most crucial factor according to estimators, indicating that dedicating more time to the estimation process could enhance the quality of conceptual cost estimates.

Table 6. Influential Factors Affecting Conceptual Cost Estimate

Furthermore, our findings indicate that the key factors identified in this study differ slightly from those found in previous research. Specifically, the level of quality definition and the availability of data on underground factors, which were not considered significant in studies outside Iran, were recognized as important key factors in our research. This variation could be attributed to the prominence of residential buildings in Iran's construction projects and the increased significance of underground earthwork due to limited construction site areas. Consequently, the key factors identified in this study hold the potential to improve the quality of conceptual cost estimates for building projects in practical applications.

The reliability of conceptual cost estimates in the early stages of a project can be effectively assessed using the user-friendly and easy-to-understand CCERI calculation sheet developed in this study. When the CCERI score falls below 3000, it indicates a high likelihood of errors exceeding 10% in the conceptual cost estimates. In such cases, cost estimators should seek alternative approaches to enhance reliability.

By utilizing the CCERI score, decision-makers and clients can gauge the reliability of conceptual cost estimates, providing valuable support for decision-making processes based on these estimates. Additionally, with the aid of the CCERI and the aforementioned key factors, estimators can identify appropriate alternatives by considering the weights and scores of each element to minimize the error range. For instance, cost estimators can devise strategies to improve the CCERI score by adjusting the evaluation of key factors and implementing relevant actions to modify their evaluation. Subsequently, the cost estimator can re-evaluate the cost of the project, resulting in a more reliable estimate.

5. Conclusions

We have introduced a straightforward and user-friendly tool to evaluate the reliability of conceptual cost estimates in building construction projects, ensuring that the expected accuracy falls within an acceptable range. To develop this assessment method, we leveraged the expertise and knowledge of experts using the AHP approach to determine the relative weights of factors influencing conceptual cost estimates. The resulting CCERI provides a simple and comprehensible means of evaluating the reliability of conceptual cost estimates, utilizing data from 100 real-world building projects estimated by Iranian general contractors for analysis and validation. Our findings indicate that:

- Conceptual cost estimates with CCERI scores below 3000 have a high likelihood of exceeding a 10% error, indicating their unreliability. With the CCERI score, decision-makers and clients can assess the reliability of conceptual cost estimates to aid in decision-making processes. Moreover, by considering the CCERI and the relative importance weights of factors affecting conceptual cost estimates, estimators can identify and implement modifications to enhance conceptual cost estimates. These alternatives can mitigate risks associated with conceptual cost estimates and contribute to the successful management of construction projects.

- The proposed CCERI serves as a guide for cost estimators and decision-makers and does not guarantee successful conceptual cost estimates in the early stages of building construction projects. Further research should be conducted to incorporate previously studied construction management skills, including CCERI, to enhance the accuracy of early estimates. Additionally, while the CCERI was developed for building construction projects, exploring indicators for other types of construction projects is also essential.

- The CCERI stands apart from traditional conceptual estimation methods as it integrates various existing estimating approaches, such as parametric estimating, ratio estimating, and cost-significant models, with advanced mapping techniques. By distinguishing resource quantities from unit prices, it overcomes the limitations of analog-based approaches that treat cost items as indivisible entities. Moreover, the CCERI accommodates real-time fluctuations in prices by allowing estimators to base their estimates on prevailing unit prices in the marketplace. Additionally, it provides estimated quantities of principal resources, empowering experienced engineers to devise preconstruction plans based on this valuable information.

- To verify the effectiveness of the CCERI, a comprehensive case study was conducted, involving 100 types of residential building construction. The results of the demonstration case studies demonstrate that the CCERI can provide an estimate of the overall project cost using information from only 20% of all cost items. Furthermore, the CCERI exhibits high reliability even in scenarios characterized by significant price fluctuations over time. Therefore, it is concluded that the proposed CCERI method offers a superior option for conceptual cost estimation in the early stages of construction projects' progress compared to traditional approaches.

While this research has made notable contributions, there are several potential extensions that warrant further exploration. Interested researchers are encouraged to develop advanced modification indices that consider factors such as site locations, special designs, and other site conditions. Investigating these issues will contribute to the development of an enhanced estimating system.