1. Introduction

The specificity of operation of public utility entities, particularly the fact that some of their services are free or only a partial fee is charged, makes it necessary to use appropriate evaluation criteria. One cannot settle on a purely commercial evaluation that boils down to an analysis in terms of financial performance. In the activities of these institutions there can also be seen other important factors that require a different perspective than the one typical for the assessment of profit-oriented enterprises. Such factors are displayed, inter alia, in basic work in this field[1-3].

It has been proposed that the assessment will be made in five dimensions. The first is the economic dimension, in which the assessed performance is defined as a ratio of the aggregated output to the total costs. The second dimension is a professional one, where the assessment relates to the effectiveness of operation of a given institution, understood as the ability to increase the well-being of an average consumer. The third aspect, called the process aspect, aims at evaluating the management of the institutions in terms of assuring quality of the provided services at the time of their “production.” The fourth dimension is a personal dimension, in which the subjective satisfaction of the consumer is measured. Finally, the last - fifth dimension is the social dimension, which measures the impact of a particular institution on social welfare.

Public utility entities run various activities. Among these institutions we may find elementary schools, universities, hospitals, and local government units. The core subject of the business and the recipient to which it is addressed determines a different approach to the problem of evaluation. For example, one evaluates the effectiveness of a university in a different way than effectiveness of a hospital. However, it is easy to note that certain elements are common, such as the fact that in the provision of services of both the university and hospital the goal is to increase the recipient’s holdings of intangible goods, in this case, health and knowledge.

Each process, and therefore the process of providing a service in a simplified way can be described as "transformation," a set of input into a set of output. Underlying the concept of measuring the quality of the idea is the assumption that the achievements and results can be standardized, and the variability of each process can be controlled. In many cases it is also necessary that the elements of the input and output can be presented in a quantitative manner. While assuming a quantitative approach in production of goods does not cause problems, in the context of the provision of services, which can be assessed with both quantitative and qualitative manner, can cause some complications. In the literature one may come across propositions to resolve this issue[4-7].

Comprehensive evaluation concept assumes that every individual, regardless of the subject of activity in each of these dimensions is evaluated in the same way on the basis of probably different input data taking into account the type of the public utility entity. For this purpose, the set characteristics A_ij was created, where i is the number of one of the five dimensions, and j the number of institution acting in the area in which the assessment is made.

Elements of sets of attributes form input data for each of the five methods of aggregation. The method for each of the dimensions is the same regardless of the type of unit being evaluated. Therefore, despite the fact that the sets A_ik and A_il may be different when k is different from l, the way of assessment being a transformation of the set the result of which is a number belonging to the interval[0,1], is the same. Since in any case the result of assessment is brought to a numerical value in the interval[0,1] any comparison becomes possible. Moreover, assuming that the assessment has a monitoring character, i.e. it is carried out regularly, one can set the historical trajectory of the evaluation and at the same time analyze the changes in the quality of services provided by individual institutions.

2. Economic Dimension of the Hospital Services Quality

Economic efficiency, or effectiveness of a given object is usually defined as the quotient of the economic result and productive contribution. Nonparametric method of Data Envelopment Analysis (DEA), which was suggested by Farrel[8], measures the relative efficiency. Farrel formulated the concept of productivity as the ratio of single output effect and a single input. This simple method offered the efficiency determining whether the object is on the so-called production possibilities curve.

Farrel’s method was further developed by Charnes, Cooper and Rhodes[9] who proposed a comparison between different objects. The authors understood the efficiency as a ratio of the sum of weighted outputs and sum of weighted inputs, thus giving rise to the method of DEA, which can be roughly understood as a method of analysis of the border data. In the simplest terms it can be said that this method, using real data, searches weights maximizing the efficiency and giving knowledge which of the objects being compared are on the effectiveness curve delineating the best combinations with the use certain selected expenditures. The effectiveness of an institution is achieved by maximizing the ratio of weighted services to weighted inputs, provided that the same indices for each institution are less than or equal to 1. Thus, DEA method identifies efficient objects, objects which produce most economically in given circumstances. It is therefore a measure of relative effectiveness which is dependent on the analyzed data set. The analysis can be focused on maximizing the effects or minimizing the consumption of expenditures. The measure of inefficiency of the object reflects its distance from the optimal area.

When assessing the economic dimension of a hospital’s activity one should take into consideration n entities, each of which utilizes P of different inputs in order to obtain R of different effects, that is services. The hospital uses at the same time x_ij input of i and produces y_rj of the effect of a service r. In this method, it is assumed that the input and output values are non-negative, and that for each of the analyzed units at least one input and one effect is positive. In terms of the mathematical the model is follows:

(1)

with the restriction

(2)

(3)

where μ = (μ₁, …, μ_R) and ν = (ν₁, …, ν_P) are the are the weights, which are determined by solving the above formula.

The solution of the problem (1)-(3) can be reduced to a linear form using the Charnes-Cooper transformation[10], i.e.

(4)

with the restriction

(5)

(6)

(7)

Equations (4)-(7) allow the numerical solution of the problem. In the literature one may also come across other ways to solve the problem (1)-(3)[11].

In case of a hospital as expenditure indicators one can propose: the number of patients per one doctor, the number of patients per one nurse, daily cost of a patient's stay treated as a hotel service, the value of medical equipment used for diagnosis and treatment, the value of hospital equipment without regard to medical equipment, assets of the hospital per patient, the proportion of grants awarded from the state budget per hospital. Indicators of the results are: the percentage of patients cured during the assessed period of time, the proportion of patients discharged from the hospital giving a positive assessment for the outcome of the treatment, in-hospital mortality and up to 30 days after discharge, the proportion of patients discharged with no improvement or worse health status, rate of in-hospital infections, percentage of patients with pressure ulcers, the percentage of court complaints filed by patients in relation to the number of patients, the average number of potential years of life lost.

3. Professional Dimension of Hospital Service Quality

In a praxeological approach efficiency is identified with the degree to which goals have been achieved. Thus, while measuring the effectiveness of services one has to measure its contribution to the achievement of its purpose as seen from the point of view of the customer, the staff of institutions providing services or managing the institution. In the assessment of institution in the professional dimension the assessment is limited to measuring the product and outcome. Thus, it were the objective achievements which were taken into consideration as the realization of intended public goals.

In case of the hospital, three main objectives of the institution were included, namely, the promotion of healthy lifestyles and prevention, restoration of health and maintenance of health. Within each of the objectives there were characteristics set out in the division between stimulants and destimulants. Within the first of the objectives the destimulants are: the percentage of obese people, and the proportion of people leading a sedentary lifestyle, and the stimulants are: the percentage of women aged 50-69 undergoing mammography examinations. Implementation of the second of the goals is assessed on the basis of five destimulants and one stimulant. To the category of destimulants we may include: waiting for admission to hospital after the diagnosis of the disease, the number of people waiting for admission to hospital, the number of court cases instituted by the hospital patients, the mortality rate of patients in the hospital or 30 days after discharge from the hospital, the proportion of patients discharged without improvement of the general or in a worse state of health, and the stimulant is the number of hospital’s accreditation points. The third objective is determined by one stimulant – one's life expectancy and three destimulants, namely the dynamics population’s mortality after a heart attack at the age of up to 64, the proportion of deaths due to cancer in the incidence of these diseases and potential years of life lost.

The measure presenting the assessment of a hospital’s performance is the arithmetic mean of standardized traits y_ij at the classical normalization

(8)

for the stimulant and

(9)

for destimulant where x_ij is the value of the j-th feature of the i-th object, max_ix_ij means the maximum value and min_ix_ij designated the minimal value of j-th feature at all objects.

4. Process Dimension of Hospital Service Quality

In order to assess the process dimension, in the first instance on the basis of Pareto-Lorenz diagram, and using the principle of Pareto one can determine 20% of the causes bringing about 80% of the effects or losses. This principle shows that the incidence of most types of events can be observed only in a small portion of the possible circumstances. The elimination or significant reduction of the said 20% significantly improves the final result and impact on the correctness of functioning of the process. At the same time identification of the 20% allows to leave out engaging into causes of minor importance, because the principle of Pareto analysis uses the unequal distribution of decision-making factors. Next, for the l factors chosen in this way the procedure which is utilized is similar to that which forms the basis of the idea of Shewhart’s control charts.

The choice of l significant factors can be associated with a pre-set list, which in the case of hospitals could be as follows: percentage of patients with pressure ulcers, percentage of in-hospital infections, percentage of patients discharged without improvement in health, percentage of court complaints filed by patients, quotient of the number of the hospitalized and the number of beds owned, average length of hospital stay, ratio of the number of in-hospital deaths to the number of patients, number of medical specialists per one patient, number of nurses and support staff per one patient, average salary of specialized personnel in relation to national average salary, average salary of nurses in relation to national average wage, average awaiting time for admission to hospital, number of infant deaths in relation to live births, percentage of accurate diagnoses, staff absenteeism in relation to the total number of working hours.

Based on data from n institutions across the country a mean μ_i is calculated and a standard deviation σ_i for each of the selected features i. Next, for each assessed hospital j in a given period of time, T measurements are made of the characteristics x_ij(t), and then with the use of the formula

(10)

one determines its deviation from the standard pattern. The next step is to calculate the average deviation

(11)

of i feature in the j hospital. It is easy to notice that if the value of Q(x_ij) is closer to unity, the better the service process offere by the entity, and vice versa a lower quality of work is demonstrated by the Q(x_ij) closer to zero. The ultimate consequence of the assessment of the j entity is the weighted average

(12)

where l is the number of features analyzed, and the weights w_ij are calculated based on the number of n_ij i.e. the number of states x_ij(t) which do not belong to the interval according to the formula

(13)

5. Personal Dimension of the Hospital Services Quality

Most commonly used method for obtaining information on the quality of services assessment in personal aspect is carrying out a survey. Surveys used for these purposes may have different design. The differences relate primarily to the number of questions, i.e. the number of different aspects of the evaluated issue, as well as the level of complexity individual responses[1],[12].

Among the recipients of the services of public utility entities, in particular, hospitals there are all citizens. When a random selection of the sample is made, one should expect a representative picture of the society. Therefore, the survey will include people with different educational backgrounds and different social status. Taking this into account a simple and as direct as possible evaluation survey form has to be ensured, preferably with a small number of possible variants of answers. Responses obtained in less complex and less extensive survey are provided spontaneously and, therefore, better reflect the real feelings.

Accordiongly, it is assumed that the evaluation of the quality of service hospitals is based on surveys, where m respondents answers to N questions regarding the quality of the various areas or in other words different service components. These spheres are mainly primary hospital activities carried out by the medical staff, auxiliary activities, in which the technical staff is involved and the hotel service most often supported to some extent by external companies. Specific, detailed questions about the elements of the other sphere, may have the following form:

Is the work of nurses of a particular health care centre on the level of:

And while answering a respondent selects one of the answers:

a) lower than expected,

b) the same as expected,

c) higher than expected.

In an extreme case the number of the questions can be limited to three, asking only about the general assessment of the above mentioned spheres.

The structure itself is similar to the survey on the analysis of inflation expectations of individual entities. The data obtained from the questionnaires is supposed to be merely a reflection of the mood of consumers related to the perception of the future dynamics of the inflation process. It does not provide direct information about what exactly the price level is expected in the future by this group of market participants, but only specified the direction of change.

It should be noted that the respondents are asked questions after using the services of the given unit, while the reference level, indicated by p_n is the value derived from a previous study of the quality of the work due to the n-th factor. In the case of inflation expectations, it was just the actual inflation. The initial value should be set at arbitrarily, for example on the level of half the value of the assessment that is 0.5 or the average response obtained from the survey, in which the surveyed with a number in the interval[0,1] would evaluate the level of service regarding a given factor.

For n-th question every respondent i formulates their own, subjective distribution of expected quality of the service received from the public utility entity regarding n-th factor with the average of and variation of This distribution may be different for each respondent, and may also vary depending on the question. It is assumed that the average for the whole group of respondents have a distribution , that is In the present discussion all that is considered are the cases of the same distributions in relation to the questions.

For the response function, it is assumed that the respondent with the subjective distribution of expected work quality will answer the survey question that the quality of the work of the evaluated unit rises or falls if the value exceeds the appropriate thresholds or . The respondent who is asked to determine the direction of change in the quality of services answers that in his/her opinion the value of quality remains unchanged, if a subjective standard of expected quality belongs to the interval called in the inflation expectations theory the sensitivity interval[13].

The main idea of quantifying quality expectations, using the assumptions described and disposition data obtained from the questionnaires, is to find estimates of the expected value and variation of the distribution expressed by the percentage of respondents expecting, respectively: an increase, consistency and falling of the expectations regarding the quality of the work of the unit. This can be done in a manner analogous to the quantification of inflation expectations of individual entities using, for example, a method by J.A. Carlson and M. Parkin[14], [15].

6. The Social Dimension of the Hospital Service Quality

Assessing the quality of hospital work in the social dimension one seeks to answer the question of how society benefits from the activities of this institution. In Poland, a large part of the medical services are provided free of charge and therefore is financed from the state budget, i.e. by all taxpayers. For this reason, the social dimension of evaluation is very important because it emphasizes the benefits the community receives in exchange for funding this activity. The proposed usage of numerical determining is taken from the elements of the mathematical theory of survival and modification of Sen’s shortened welfare function.

Purpose of the mathematical theory of survival is to estimate the distribution function of survival on the basis of certain characteristics of the studied population. The essential difference between the studied individuals are individual characteristics such as age, gender, the fact if an individual smokes, etc. The remaining variables should represent the current state of the patient’s health verified with medical examinations and laboratory tests. These factors, indicated as for individual i ∈ {1, 2, ..., n} constitute a set of explanatory variables in the present model.

Assuming that T is non-negative random variable, which represents the lifespan and F denote the cumulative distribution of the random variable T, the survival function can be defined as follows:

(14)

where t is left-sided limit of a cumulative distribution function F at the point of t. S(t) determines, therefore, the probability that a person will live at least t amount of years.

In Cox’s model is a survival function has the following form:

(15)

and entering the indication

(16)

The result is that

(17)

Assuming that h(t) = eα can with the method of maximum likelihood by Hosmer, Lemeshow[16], Magiera[17] on the basis of historical data to find estimates of the parameters of the model.

The variable is the age expressed in months of the i person and therefore the notion of

(18)

determines the chances of survival of at least one year by the i individual, provided that s/he survived longer months. Using standard formulas for conditional probability it can be shown that

(19)

Assuming that p_0ij and p_1ij mean the probability of surviving another year for the patient i of hospital j respectively at the time of his/hers admission and discharge, while and are average values for a given hospital j and a fixed group of patients treated in a given period of time, the value indicating an assessment of the entity in social dimension can be summarized as follows

(20)

where G_0j and G_1j are Gini’s coefficients of an inequality of distribution of survival probabilities for the next twelve months the community of patients at the time of their admission to the hospital j and leaving it.

7. Conclusions

Performing the evaluation of hospitals’ activities carried out in the manner presented allows both single and multi-faceted evaluation of the units. On the one hand, it takes into account the specific nature of the non-profit organization, on the other hand, it introduces elements characteristic for the assessment of its profit-driven activities. Multi-dimensional insight into the health care business unit provides a comprehensive assessment needed for public utility entities. The proposed use of standardized results in each of the evaluated dimensions allows homogeneous evaluation criteria. In addition, it creates the possibility of construction of a synthetic assessment measure by the use of arithmetic mean or the weighted mean of individual dimensions’ ratings. Given the fact that similar structures can be used in other areas of business of public utility entities one achieves a universal tool to be used in the evaluation process of this market segment.

ACKNOWLEDGEMENTS

Scientific work financed from the budget for science in 2010-2012 as a research project no N N 111 194439.