International Journal of Information Science

p-ISSN: 2163-1921    e-ISSN: 2163-193X

2011;  1(1): 17-23

doi: 10.5923/j.ijis.20110101.03

Determining and Prioritizing the Criteria and Scales of Evaluating Cohesive Training Systems by AHP Method

Seyed Mohammad Mahmoudi , Gholamreza Jandaghi , Zolfa Haghgooya , Zahra Karimi Asl

Faculty of Management, University of Tehran, Iran

Correspondence to: Gholamreza Jandaghi , Faculty of Management, University of Tehran, Iran.

Email:

Copyright © 2012 Scientific & Academic Publishing. All Rights Reserved.

Abstract

The aim of present research is to determine and prioritize criteria and scales of evaluating cohesive training systems. Present paper attempts to provide a paradigm for better devising of universities' cohesive training systems by identifying such scales. In terms of its aim, this research is an applied one and in terms of data collection method, it is descriptive and survey one. Its population includes elite training experts in the colleges and institutes affiliated to . It attempts to study domestic and foreign papers and to identify criteria and scales for evaluating cohesive training systems. Library method is used to gather information on theoretical basics, literature and to identify aspects and scales. Identified aspects include: content, flexibility, organizational structure, usage convenience, usage services and apparent form. To prioritize identified aspects and scales, the opinions of elite training experts in are used. Data gathering tool was questionnaire. Data were analyzed by AHP method and Expert Choice software. According to research findings, the relative supremacy of content was 0.362; flexibility, 0.225; organizational structure, 0.155; usage convenience, 0.115, usage service, 0.087; and apparent form, 0.056. So, one can say that the importance of identified aspects to evaluate cohesive training systems in terms of priority is content, flexibility, organizational structure, usage convenience, usage services and apparent form.

Keywords: Evaluation Aspects, Evaluation Scales, Website, Internet

Cite this paper: Seyed Mohammad Mahmoudi , Gholamreza Jandaghi , Zolfa Haghgooya , Zahra Karimi Asl , "Determining and Prioritizing the Criteria and Scales of Evaluating Cohesive Training Systems by AHP Method", International Journal of Information Science, Vol. 1 No. 1, 2011, pp. 17-23. doi: 10.5923/j.ijis.20110101.03.

1. Introduction

Over a decade, web converted from a theory into a reality so that one can claim that web is now obvious in all areas of our social life. Hence, public and non-public companies and organizations, schools and universities have websites. The aim of designing web pages in such institutes particularly in high education and research centres is an important added-value role in public awareness and to achieve existing information in such centres more effectively, rapidly and with the lowest cost (Khanlarkhani et al, 2008, p. 67). Naturally, an organization whose users are facing with problems in using and networking its web pages exposes a poor image and weakens organizational status. Therefore, it is necessary that any organization evaluates its web pages by considering its users' perceptions and benchmarks (Khanlarkhani et al, 2008, p. 67)
The services provided by university have extended well beyond those offered at an on-site facility. The design, usability {“defined as the capacity of a system to allow users to carry out their tasks safely, effectively, efficiently, and enjoyably” (Li ,2005,p.253)}, and functionality of the websites University are critical if the to continue providing essential services to their patrons in a timely and efficient manner (Carole ,2005,p.167).
According to ISNA1, in terms of web measuring indicators, Iranian universities do not enjoy high ranks so that the ranks of top universities in terms of such indicators include University of Tehran (873), Tehran Medical University (1266), Sharif Industrial University (1560) and Mashhad Ferdousi University (1671) (http: //www.modir.ir/News/2602.aspx).
The plan to evaluate the websites of domestic universities and research institutes can compare their performance. Thus, their manager can be aware of their organization's situation compared to their counterparts or superior ones in web environment in terms of training and research (Ghane, 2010).
Since there has not been yet provided any model to evaluate cohesive training system, in present study the main question is that: "which are the main aspects and scales to evaluate cohesive training system and how can we prioritize them?
In below, the paper introduces Iranian Universities Training System (Golestan System), mentions the aspects and scales of evaluating websites and information systems, prioritizes them by using the opinions of elites, experts and AHP model and determine the importance ratio of scales.

2. Defining Websites and Information Systems

Websites are a set of current pages in world web network which may be backed by people or different trading, scientific, thematic, national and international organizations (Heydari, 2005, p. 18). Since websites are considered as information systems in organizations, the advantages of information system are represented below.
There are paramount definitions on information systems in disciplines such as management, computer sciences, software engineering, librarian sciences and public awareness. A definition in US Librarian Association encyclopaedia is a comprehensive definition of information system (a complete devised system to produce, gather, process, store, recover and disseminate information in an institute, organization or any other defined area of community" (Omidvar, 2006).
Today, managers recognize the strategic and competitive value of information system well. An organization should be able to establish an information system capable to meet most information needs inside the organization. Such a shared system enjoys following advantages: mitigating repetitive works in maintaining databases, representing data more carefully (since data are stored in one place and they only need to be updated), better communications inside the organization so that everyone can access his/her needed information, and harmonic treatment with inter-organizational information needs (Zavareghi, 2006).

3. General Criteria on Evaluating Websites and Information System

By comparing used models and methods throughout the world, we extracted some common aspects and scales some of which are evaluated here. Achieved aspects and scales can be used in our final paradigm. Some models are important for general usages and others for special ones. For example, it may be important to determine the validity of current information in the site in evaluating a website while access to information is more important for someone who uses FTP achieves (Heydari, 2005, p. 18). Pointed criteria in evaluating e-resources in most websites and Internet networks include: Correctness, Competency, Thematic coverage, density and intensiveness, newness, interaction, goal, velocity (Dragolanesco, 2002), usage convenience and users' satisfaction , Content, Website structure, Objectiveness (Heydari, 2005, pp. 27&28).
As a proper tool to evaluate web page quality, Webqual model attempts to provide an integrated and systematic structure in evaluating the quality of websites and webbed resources and to assess the quality from users' perspective and viewpoint in order to devise a user-oriented approach to evaluate the quality of web pages (Khanlarkhani et al, 2008, p. 68). In reviewing Webqual model, since it involves paramount versions and its elements are changing in adopting with each version, one can provide paramount elements in a qualitative evaluation by considering varied edits. In initial Webqual versions, these elements include 4 aspects, 12 structures and 35 factors while these factors and structures are changed in new versions. These aspects include: Profitability, Usage convenience, Amazements, Mutual communications (Khanlarkhani et al, 2008, p. 68).
In the most recent researches, a new version of Webqual titled Aqual is introduced. This model attempts to evaluate the quality of websites from users' perspective in 4 aspects and 34 factors at two status qua and expected status. The aspects include content quality, application, service interaction quality, interactions quality and security (Khanlarkhani et al, 2008, p. 68). An interesting point on this model is an applicable aspect in evaluating the quality of pages along with its factors and e-commerce factors in web quality services concept (Khanlarkhani et al, 2008, p. 68). Leo's measures to evaluate the site include: content, structure (in terms of visual designing), structure (in technical terms), author's right and applied scales (Baradar and Najafzadeh, 2008, p. 23).
"Software engineering: an approach to a technician" book also provides following software quality measures: Rightness, Maintenance ability, Comprehensiveness, Usability: the capability of physical or intellectual skills to learn needed system and the needed time to gain skills for using the system (Prisman, 2008, pp. 134 – 135). In paper on identifying portal social health training sites, following items were identified as effective parameters to evaluate training portals: Security, Management, Efficiency, User friendliness, Built-in applications, Flexibility, Trading, Interoperability, Support(Hejazi and Movahedi, 2007, pp. 82 – 84).
Huizingh (2000) distinguishes design from the information content, and identifies three dimensions: quality of navigation structure, multimedia capabilities and the presentation style. Paynter et al. (2001) take four categories into consideration: information, transaction services, trust, and non-functional requirements. Jenamani et al. (2002) present a thorough classification of the web site features, relating to marketing features, functional features, innovative features and accessibility features. However, important factors such as privacy, credibility, security and trust are missing from their classification, some of which are taken into account by Zhang and von Dran (2002). All these studies admit that the success of a web site design relies on the provision of a user-friendly environment for visitors. Sowards (1997) evaluates the effectiveness of the web sites from the user’s perspective and suggests that layouts, design, content and speed of a web site are important success factors. (Li & Holeckova, 2005, p.78)
Zhang and von Dran’s study was motivated by Herzberg’s hygiene and motivator factors and suggested that certain website features are necessary but not sufficient to elicit positive perceptions or prevent negative perceptions of website quality, while other features are not necessary, but do increase positive perceptions of website quality. They compiled an extensive list of 42- scale items grouped a priori into eleven dimensions: (1) information content, (2) cognitive outcomes, (3) enjoyment, (4) privacy, (5) user empowerment, (6) visual appearance, (7) technical support, (8) navigation, (9) organization of information, (10) credibility, and (11) impartiality. They operationalized web-site quality as a three-dimensional construct composed of (1) information quality, (2) response time, and (3) system accessibility. Path analysis supported the effect of the three website quality dimensions on usability and usefulness as antecedents of intention to reuse the site in the future (Kim & Stoe, 2004, p.620).
The quality of a website must be evaluated with a number of different criteria according to Thewall:
1- Site visibility in search engines; the issue of visibility is one that is easy for the inexperienced to ignore. An otherwise excellent website may be completely ignored because few potential customers ever find it.
2- Ease of use; Ease of use or usability comprises four main categories: accessibility; navigation (Web designers use the term “breadcrumbs” to describe navigational clues that show users where they are on a website. They trace the path the user has taken from the home page to their present location.( Leanne ,2005,p.183)); readability; download speed.
3- Design quality; Website design shares many features in common with print media, and many of its techniques transfer, but all too often even the basic rules of graphic design are forgotten. The design should be appropriate to the goal of the site, avoid `bad practice' and be memorable.
4- Ease of site maintenance and updating; a company opting for a website containing at least some product information needs to resolve the issue of how to keep it up to date. If an external contractor created the site, then it could be costly and inconvenient to ask for frequent updates. A better solution is to have the site created in such a way that company employees can update it themselves. There are various ways in which this can be done. (Thewall, 2000, pp.151- 154)
Cox & Dale (2002) suggested that Key quality factors (KQFs) include: Clarity of purpose, Design, Accessibility and speed, Content, Customer service (Cox & Dale, 2002, pp.863-870).
Huang et al provided a cohesive framework based on sites' traits and functions to evaluate trading websites that include: speeding up online tasks, establishing multiple communication channels, providing suitable access to contacts, making web site personal, providing company information and advertising online, facilitating customer feedback, allowing users to control information detail, aiding online shopping decisions, using multimedia tools. (Huang & et al, 2006, p. 525).
In web-measuring (web-metric) evaluations which are a branch of measurement science, global universities and high education institutes are categorized in terms of evaluation top trainings in web, volume, size, observations and impact of web pages published by universities and information resources. Other scales by which university and research centres can increase their ranks in global well-established categorizations systems are as follow: web pages and contents; resource quality; and observation. Other scales include public acceptance – which relates to users' satisfaction and measured by indicators such as user numbers, flexibility on various users' information finding behaviours, attempts to update and evolve website, judges' ideas and web impact ration; internationalization (communication with international scientists and researchers and cooperating to international faculty), training quality (the quantity of scientific staff able to create content in web, virtual training disciplines and the ratio of students to faculty and staff); research outcome (evaluated by research output access of institute in systems such as Google, Scholar, SSCI & SCI; scientific fame and web impact ration (the credit of university its faculty, observations through links to institutes' sites, the quantity of researchers according to institute and its famous staff throughout the world).

4. Research Background

In this section, we address to conducted researches on websites and information systems evaluation. One can divide conducted researches into three categories:
External researches whose aims are to identify the aspects and scales of website and information systems evaluation as follow:
The total of 31 of evaluation criteria that used by Li & Holeckova in Evaluation of UK car insurance brokers’ web sites are organized into five categories:(1) search (SEA); (2) Site characteristics that include: information (INF), system quality (SYQ), design (DES), navigation (NAV), credibility (CRE), privacy (PRI) and security (SEC); (3) quality of access (QUA);(4) quote (QUO); and (5) Purchase: purchase (PUR)(Li & Holeckova, 2005, p.79).
Liu and Arnett surveyed webmasters of Fortune 1000 companies about factors that contributed to website success. They originally proposed six dimensions of website quality, but exploratory factor analysis revealed four: (1) quality of information and service, (2) system use, (3) playfulness, and (4) system design quality (Kim & Stoe, 2004, p.621).
Loiacono explicitly measured website quality of sites selling goods and services (books, music CDs, airline tickets, and hotel reservations) and suggested that website quality is represented by 12 unique dimensions. In her study, 14 dimensions of website quality were originally proposed as a result of an extensive review of the marketing and IS literature and interviews with shoppers and website designers (Kim & Stoe, 2004, p.621).
Researches which address to foreign universities' websites such below two instances:
The results of the redesigning Carnegie Mellon University Libraries website indicated several key weaknesses with respect to navigation, screen design and labelling, leading to more revisions and the final release. Testing indicated that color and graphics attract attention; font, labels, and placement increase visibility; chunking and leading with keywords increase readability; and consistency increases usability (Carole, 2005, p.167).
Shelstad (2005) examined the work of the ’s American Heritage Centre (AHC) to revamp its website during 2003-2004. The task force analyzed the structure and content of the site to improve navigation, prioritized the presentation of content, and also researched the costs and benefits of outsourcing the design and maintenance of the site. The AHC also identified opportunities for expanding useful content with a relatively small investment of staff time and budgetary resources.(Shelstad,2005,p.210) Some of the less successful areas of the redesign included user feedback indicating that some portions of the site were not entirely up to date: this has been a great frustration, for the areas referred to ought to be in the forefront of providing archival services via the web. AHC’s user testing did not include more of the general public, but the AHC’s efforts to include them went unanswered (Shelstad, 2005, p.223).
Researches which address to evaluate Iranian universities' websites and categorize them based on their web quality.
In web-measuring (web-metric) evaluations which are a branch of measurement science, global universities and high education institutes are categorized in terms of evaluation top trainings in web, volume, size, observations and impact of web pages published by universities and information resources. The results are indicated below.
Based on this categorization of designing, observation, scientific docs (PDS), size and traffic, the ten top institutes in terms of acquired scores in above five scales include University of Tehran, Hawza and University Research Center, Iranian Sciences and IT Research Centre, Scientific database, Academic Jihad, Hawza website, Tehran Medical University, Payam Noor University (main portal), Shahid Beheshti University, and Mashhad Medical University.

5. Research Goal

The goal is to identify website scales evaluation on the cohesive system at .

6. Methodology, Sample, Population and Data Gathering Method

In terms of its aim, this research is an applied one and in terms of data collection method, it is descriptive and survey one. Its population includes elite training experts at who are able to recognize the weaknesses and deficiencies of the system since they are able to work a training cohesive system. The number of elite experts in is 27. Elite experts were selected from Pardis Qom, Sciences Pardis, art Parsdis, agricultural and natural resource Pardis as well as social sciences, technical, law, political sciences, environment, liberal arts, literature, management, foreign languages, entrepreneurship, psychology, economy, physical education, theology, Islamic sciences and geography. Library method is used to gather information on theoretical basics, literature and to identify aspects and scales. Field study method was used to gather information and the tool to gather information was questionnaire. The questionnaires were distributed among elite training experts in .

7. Scales and Aspects of Analysis and Evaluating Training Cohesive System

After broad study of literate and research background, aspects and scales were extracted as the aspects and scales of evaluating training cohesive system indicated in Table 1. All used aspects in this research are documented and each one is used in paramount researches as the measures to assess websites. As mentioned in previous section, the apparent form is use in studied by Lin and Arnt, Zhang & Von Dran and Loiacono and flexibility is used in a study by Hejazi and Movahedi. They are also seen in Leo's measures (in visual terms at structural aspect). Organizational structure is also seen in Leo's model (in technical terms at structural aspect) and Heydari's research. Application convenience is used in studies by Hang et al, Barns & Wigden, Hejazi & Movahedi. An aspect of webqual is application convenience. Content is both seen in both webqual and Leo's model. Heydari, Lin and Arnt, Cox and Dal and Swardes have used content in their studies. Finally, usage services are used in studies by Hejazi & Movahedi and Cox & Dal.
Table 2. Aspects prioritization of evaluating training cohesive systems to use AHP method.
The aspects of evaluating training cohesive systemsRelative advantage
Content0.362
Flexibility0.225
Organizational structure0.155
Usage convenience0.115
Usage services0.087
Apparent form0.056

8. Data Analysis

After identifying aspects and scales, this question arises: "How much is the Importance of each aspect? Which aspect has the highest and which has the lowest importance?" to determine rations and weights of scales, one can point such methods as Likert's model, unreal group, Borda's method and Expert Choice. An important usable method which is highly applied in management is AHP (Analytic Hierarch Process). AHP is a way by which a complicated situation is divided into smaller ones and then they are put into a hierarchical structure (Azar and Rajabzadeh, 2008).The results of aspects prioritization are outlined in Table 2.
Table 1. Aspects and scales of evaluating training cohesive systems.
AspectRowScales
Apparent forms1Picture and text coordination
2Proper fonts
3Proper colors
4Attractive logos and pictures
5Attractive designing
6Attractive environment
7Welcoming
8Animation
9Using multimedia tools
AspectRowScales
Flexibility1The possibility to change fonts and color
2The possibility to change language
3The possibility to convert into home page
4The possibility to ass sound
5The possibility to change background color
6The possibility to transfer and store information with different formats (word, PDF, excel, etc)
7The possibility to change page size
8The possibility to return desired page from any point or navigating the pages
9The possibility to link with other dates
10The possibility to attach and send via email
11The possibility to look at content without image or color
AspectRowScales
Organizational structure1Components integration
2Section interdependency
3Totality principle: full menu and needed lists in any section
4Proper structure of menus, hierarchies, …
5Menu title relevance to considered usage
6Proper layout
7Proper input/output
8Proper information structure
9Logic volume and relevance of menus and information
AspectRowScales
Usage convenience1Information access velocity
2Menus accessibility from any section
3Search and survey convenience
4Proper (low) interactions
5Different access (direct search)
6Effective search in site
7The convenience to modify programs when facing with errors
8System loading velocity
9Certain loading period of each page
10Download time
AspectRowScales
Content1Information relevance to needs
2Menus cohesiveness
3Information clarity
4Menus clarity
AspectRowScales
Usage services1The possibility to print information from any section
2The possibility to copy and share data
3The possibility to import and export data by everyone
4The possibility to edit information in any page
5The possibility to share information in official automation environment
6The possibility to share information in e-government
7Complete public awareness in each section
8The possibility to register and enter the system outside the university
9The possibility to issue forms like certification and so on by user
10The possibility to change information by user in any time
11The possibility to prepare structured reports
12Poll (feedback) system
13Search engine in the system
14Site efficient map
Table 3. The importance ratio of scales of each aspect, %.
AspectRowScalesVery important (a)Important (b)Relatively important (c)UnimportantImportance ratio
Apparent form1Picture and text coordination18.263.613.64.573.825
2Proper fonts36.454.59.1081.825
3Proper colors18.268.29.14.575.025
4Attractive logos and pictures22.740.913.622.765.85
5Attractive designing22.759.14.513.672.675
6Attractive environment27.3509.113.672.75
7Welcoming4.55022.722.759.025
8Animation9.136.418.236.454.6
9Using multimedia tools27.345.54.522.769.35
AspectRowScales
Flexibility1The possibility to change fonts and color18.268.29.14.575.025
2The possibility to change language40.945.54.59.179.55
3The possibility to convert into home page9.542.933.314.361.9
4The possibility to ass sound13.645.513.627.361.35
5The possibility to change background color9.15022.718.262.5
6The possibility to transfer and store information with different formats (word, PDF, excel, etc)90.99.10097.725
7The possibility to change page size40.940.918.2080.675
8The possibility to return desired page from any point or navigating the pages72.727.30093.175
9The possibility to link with other dates31.859.19.1080.675
10The possibility to attach and send via email40.940.99.19.178.4
11The possibility to look at content without image or color28.647.69.514.372.625
AspectRowScales
Organizational structure1Components integration54.540.50084.875
2Section interdependency59.131.89.1087.5
3Totality principle: full menu and needed lists in any section5036.413.6084.1
4Proper structure of menus, hierarchies, …54.531.813.6085.15
5Menu title relevance to considered usage63.631.84.5089.7
6Convenient communications63.636.40090.9
7Proper layout36.454.54.54.580.65
8Proper input/output59.140.90089.775
9Proper information structure59.131.804.586.325
10Logic volume and relevance of menus and information54.545.54.5088.625
AspectRowScales
Usage convenience1Information access velocity81.813.604.593.125
2Menus accessibility from any section77.313609.189.775
3Search and survey convenience72.722.704.590.85
4Proper (low) interactions30655081.25
5Different access (direct search)59.127.34.59.184.1
6Effective search in site59.140.90089.775
7The convenience to modify programs when facing with errors59.131.809.185.255
8System loading velocity86.44.54.54.593.15
9Certain loading period of each page36.436.418.29.175.075
10Download time5036.44.59.181.825
AspectRowScales
Content1Information relevance to needs72.727.30093.175
2Menus cohesiveness59.140.90089.775
3Information clarity72.727.30093.175
4Menus clarity68.231.80092.05
AspectRowScales
Usage services1The possibility to print information from any section54.536.44.54.585.175
2The possibility to copy and share data68.222.74.54.588.6
3The possibility to import and export data by everyone5036.413.6084.1
4The possibility to edit information in any page45.540.9013.679.575
5The possibility to share information in official automation environment68.227.304.589.8
6The possibility to share information in e-government5031.89.19.180.675
7Complete public awareness in each section36.463.60084.1
8The possibility to register and enter the system outside the university68.222.709.187.5
9The possibility to issue forms like certification and so on by user40.954.504.582.9
10The possibility to change information by user in any time45.531.89.113.677.3
11The possibility to prepare structured reports61.928.64.84.886.95
12Poll (feedback) system40.95009.18.675
13Search engine in the system59.136.404.587.525
14Site efficient map33.338.114.314.372.6
As seen in Table 2, relative advantage is as follow: content 0.362; flexibility, 0.225; organizational structure, 0.155; usage convenience, 115; usage services, 0.087; and apparent form, 0.056.. It shows that in evaluating training cohesive systems, content is more important than other aspects followed by flexibility, organizational structure, usage convenience, usage services and apparent form.
To determine the importance ratio of each scale, following formula is used. To facilitate the calculations, SPSS software is utilized. A, b, c and d columns show the percentage of relative frequency for each option.
Scales importance ratio = (a)*4+ (b)*3 + (c)*2+ (d)*1
The importance ratios of aspects and scales for evaluating training cohesive system in University of Tehran are indicated in Table 3.

9. Conclusions

The aim of present paper was to identify and prioritize the aspects and scales of evaluating training cohesive systems. In this line, we identified the aspects and scales by studying those internal/external papers which had addressed to identify the aspects and scales of evaluating websites and information systems. Identified aspects include: content, flexibility, organizational structure, usage convenience, usage services and apparent form. We used the opinions of elite experts in the colleges and institutes affiliated to in order to prioritize aspects and scales. To analyze data on prioritizing the aspects, AHP model and Expert Choice software were used. To determine the importance of scales, importance ratio was used whose results are shown in Table 3. The results of analysis showed that content has the highest and apparent form has the lowest importance in evaluating training cohesive systems. As a result, content aspect merit more ratio in evaluating training systems.

Notes

1. Iranian Student's News Agency

References

[1]  Axar, Ali & Rajabzadeh, Ali (2008), "applied decision-making (M.A.D.M approach), Negah Danesh Publications
[2]  Baradar, Roya & Najafinia, Sheida (2008), "Website assessment in e-journals", IT sciences, vol. 3, pp. 1- 34
[3]  Carole A.G, (2005),” Usability testing and design of a library website: an iterative approach”, OCLC Systems & Services, vol. 21, No.3, p.167-180
[4]  Cox.J & Dale. B. G ,(2002) ,” Key quality factors in Website Design and use: An examination” ,International Journal of Quality & Reliability Management، vol. 19, No .7 ,2002 ,pp. 862 – 888
[5]  Dragolansco, Nichola George (2002), "websites qualitative evaluation: tools and scales", translated by Gholam Heydari, Iranian Information and Scientific Docs online Journal, vol. 2.
[6]  Ghane, Mohhamad Reza ,(2010), "Establishing website assessment system in national universities and research centers", http: //bpdanesh.ir/detailnews.asp
[7]  Hejazi, Seyed Mahdi & Mofahedi, Farid (2007), "valuating training portal sites on community health", Health Information Management, vol. 1, pp. 81 – 88
[8]  Heydari, Gholam, (2005), "evaluation scales of online information resources by emphasize on websites", Information Sciences, vol. 3 & 4, pp. 17 – 32
[9]  Huang .W & et al ,(2006),”Categorizing web features and functions to evaluate commercial web sites”، Industrial Management & Data Systems ,vol.106 ,No.4, pp.523-539
[10]  Khnlarkhani, Elmira, Mohammad, Ismail, Sadigheh & Shokuie, Ali (2008), "quality and application elements in evaluating web pages", Public Awareness Monthly, vol.11
[11]  Kim .S & Stoe .L ,(2004) ,” Dimensional hierarchy of retail website quality”، Information & Management,vol.4,no.1 ,pp. 619–633
[12]  LaRue E.M ,(2008) ,” Development and evaluation of SPAT: a web page assessment tool”، Library Hi Tech، Vol. 26، No.2 ,pp. 274- 286
[13]  Leanne M.V ,(2005) ,” Usability analysis of Northern Illinois University Libraries’ website: a case study”، OCLC Systems & Services ,Vol. 21, No.3، pp.181-192
[14]  Li .P,(2005) ,” Cognitive task analysis: A cognitive approach to evaluateevidence-based nursing websites” ,OCLC Systems & Services ,Vol.21, No.3, pp. 252-256
[15]  Li .S & Holeckova .K ,(2005)، “Evaluation of UK car insurance brokers’ web sites: some preliminary findings”، Marketing Intelligence e & Planning، Vol.23 ,No.1, pp. 77 – 88
[16]  Manuel, (2009), "analyzing hierarchies in flexible production", http://inen.blogfa.com
[17]  Omidvar, Majid, (2006), "Management Guideline", http://www.tafda.org/management
[18]  Prisman, Roger (2008), "software engineering: an approach to a technician", translated by Hashemi Taba, pp. 134 – 135
[19]  Rahimi, Ghafur, (2006), "performance assessment and organizational continuous improvement", Tadbir Monthly, vol. 173, http://www.modiiran.ir
[20]  Shelstad. M ,(2005) ,” Content matters: analysis of a website redesign” ,OCLC Systems & Services, vol. 21 ,No.3 ,pp.209- 225
[21]  Thelwall .M ,(2000) ,” Effective websites for small and medium-sized enterprises” , Journal of Small Business and Enterprise Development, Volume 7, Number 2 ,pp.149-159
[22]  Zavareghi, Rasool, (2006), MIS, Iranian Information and Scientific Docs online Journal, http://www.modiiran.ir
[23]  http: //www.modir.ir/News/2602.aspx, 2010
[24]  PayaWeb.com, 2010