1. Introduction

While there is scholarly interest in the theory and practice of gerontological course evaluation, exemplified by the fact that a database search on the topic over the period from the year 2000 until today resulted in only 1-6 hits/year, there is a great need for more studies in this area of higher education. Most evaluation studies have focused on sociology (see e.g.[1, 2] and included evaluations of gerontological social work curricula, along with attitude, knowledge and skills assessments. Many of these papers involved evaluations of knowledge and attitude before and after gerontology programs, but few have evaluated the perceptions of health and medical sciences students[3]. With new generations of students, there is a continuous need to adapt the teaching and learning activities to their needs and expectations. This comes with challenges also for evaluation methods, requiring adaptation to match the preferences and the potential of the Millennial Generation [4]. One prerequisite for such development is to present efficient approaches to systematic analyses of curricula and courses, while only scarce international literature presenting such examples is available.

Starting in Bologna, Italy in 1999, higher education in Europe has seen major changes towards a unified structure [5]. The changes concern simplification of regulations, improvement of mobility, and quality improvements of training and research[6]. One goal implying considerable challenges for European universities is to develop a joint system of curricula, increasing cross-national comparability, ultimately nurturing cross-national collaboration and student as well as teacher mobility. This situation implies an increased importance of curricula development (on program and course level), beyond that of local interests[7]. The Bologna agreement rests on ten “action lines”, serving to support the development towards a coherent European Higher Education Area. Two themes link all the action lines, namely a focus on learners and a focus on learning outcomes[8]. In Sweden, this represents a major change from the former, content- and structure-based curriculum system, and there is a need for successive and continuous analysis and optimization of curricula.

Regarding training and education in gerontology in Europe, there are only a few full graduate academic programs available, unevenly distributed[6]. In contrast to in the U.S.[9], in many European countries it is only possible to follow individual courses, often linked to or integrated with Bachelor’s or Master’s programs in health or social sciences. Consequently, gerontology is an area of knowledge where students benefit from having the possibility to combine courses from different universities to obtain their degree. During recent years, several cross-national collaborations offer students who have limited access to regular gerontology programs the opportunity to combine courses from different universities [6]. Bearing this in mind, it is of course important that program and course curricula meet the European ambition of a focus on learning outcomes and cross-national comparability. In order to attain such goals, new courses should undergo critical analysis and subsequent revision to meet the new requirements.

The aim of this paper is to present and apply an approach to critical analysis of a Master course in gerontology, followed by a set of recommendations and reflections on the usefulness and potential of systematic analysis.

2. Material and Methods

2.1. The Nordic Master of Gerontology (NordMaG)

The context for this analysis is a joint Nordic endeavour for Master’s education in gerontology, initiated in 2006. Lund University in Sweden is part of this Nordic Master of Gerontology (NordMaG) initiative, and several courses have been developed, integrated with the Master of Medical Science Program at the Faculty of Medicine. The objective of this advanced-level program is to promoteinter-professional learning in the main fields of occupational therapy, nursing, midwifery and physiotherapy. There are five places for students specializing in gerontology, and for students doing at least 30 European Credits (ECTS) in gerontology in the University of Iceland or in Jyväskylä University, Finland (i.e. the other partners of the NordMaG), there is the possibility to receive a Nordic Master in gerontology.

2.2. The Course “Aging, Occupation and Health Promotion”

The course at target for this paper is an eligible course within the NordMaG program. At the time for the analysis, the course “Aging, Occupation and Health Promotion” (7.5 ECTS) was quite new, with a course syllabus valid from 1 January 2009. There were students with different professional and disciplinary backgrounds; since NordMaG students are eligible, not only those with applied professional backgrounds but also students with a theoretical background in e.g. psychology or gerontology have the possibility to apply. The first class of students had just been admitted and the course start was scheduled for November 2009. According to the syllabus, the course is given for four main fields, namely occupational therapy, gerontology, nursing and physiotherapy, each with a specialisation in scientific methodology. The course would be given in English, with a course leader with British English as his mother tongue.

By the end of the admission period, 23 students had applied for the course. Expressed with some disappointment, after the registration on site at Lund University, the course leader reported that only 10 students had shown up. In terms of ethnicity, the students represented far more diversity than expected. The students from abroad were recruited based on the NordMag partnership, but besides the fact that they came from the three different countries involved (Sweden, Finland, Iceland), several of them originally came from other countries. That is, in the course there were students from Sweden, Denmark, Finland, the U.S., Cameroon, Taiwan and Uganda. The students were of different ages (range 26-53 years; median age 29); seven were women. In terms of professional/disciplinary backgrounds those represented were medicine, gerontology, sociology/anthropology, nursing, physiotherapy, and occupational therapy.

2.3. Framework for the Analysis

The analysis was based on a three-folded framework based on different but compatible frames of reference, as well as current Swedish legislation and regulations of higher education. Since Sweden has signed the Bologna agreement, it was a straightforward decision to use the Framework for Qualifications in the European Higher Education Area (EQF-HE) as part of the analysis framework. The EQF-HE is intended to make coordination and comparison among the 46 countries that signed the Bologna agreement possible[10]. A main feature of the EQF-HE is the description and definition of the basic, advanced and research levels of higher education, and the general knowledge and skills requirements of each level. The EQF-HE strongly emphasises the importance of learning outcomes, but it should be noted that the structure of learning outcomes is not identical to that of the Swedish Higher Education Ordinance. That is, in the EQF-HE there are five learning outcome categories, namely “Knowledge and understanding”, “Applying knowledge and understanding”, “Making judgments”, “Communication”, and “Learning skills”, while the Swedish Higher Education Ordinance prescribes only three, worded somewhat differently. A specific asset for the analysis presented in this paper is the fact that the EQF-HE states rather detailed, yet general expected learning outcomes for the three cycles of higher education of the Bologna agreement.

Since constructive alignment is gaining increased recognition in higher education, in Europe as well as elsewhere, I selected Biggs & Tang’s[11] wellacknowledged work as the second frame of reference. Their approach fits well with the EQF-HE and guides the development of curriculum objectives in terms of performances that represent a suitably high cognitive level, in deciding teaching and learning activities judged to elicit those performances, and to assess and summatively report student performance. Most important, the structure of learning outcomes, teaching and learning activities, and examination according to Biggs and Tang was utilised for the subsequent three-step analysis of the course.

The EQF-HE approach to describe the overall learning outcome in different forms of knowledge has resulted in a renewed interest in existing taxonomies[7]. Consequently, the Structure of the Observed Learning Outcome (SOLO) Taxonomy, based on the study of outcomes of academic teaching, was selected as the third frame of reference of the analysis framework. This taxonomy has since long been widely recognised as an efficient and valid tool supporting the formulation of intended learning outcomes[12]. The five levels of the SOLO Taxonomy form an increasing order of structural complexity[13], as follows: 1) The pre-structural level; 2) the uni-structural level; 3) the multi-structural level; 4) the relational level; 5) the extended abstract level. The taxonomy can be used to define learning outcomes, forms of teaching and learning that support them, and forms of assessment that evaluate to what extent the learning outcomes were attained. That is, it fits well with Biggs & Tang’s[11] frame of reference of constructive alignment.

2.4. Documents

The documents available for the analysis were the Master of Medical Science Program syllabus, and the course syllabus, course schedule, list of participants and Student Guide for the course “Aging, Occupation and Health Promotion”. An additional source of information was short meetings with the course leader and one of the teachers involved. After course completion, the course leader made the student evaluation forms completed available.

3. Results and Critical Analysis

3.1. Learning Outcomes

The first step of the analysis of the course focused on the learning outcomes. The learning outcomes were stated under a separate heading in the course syllabus and categorised according to the three sub-headings; “Knowledge and Understanding”; “Skills and Abilities”; “Judgment and Approach”. While this makes a lot of sense and provides a systematic structure of the syllabus, later on there was a paragraph intended to describe the content of the course that contributed to some uncertainty concerning learning outcomes, as follows:

● to enable students to conduct in-depth study of the theory and methodology concerning activity and participation

● to apply this knowledge in the work of rehabilitation and health promotion in older people from the perspectives of the individual, the group and the community

● increased knowledge of how activity and participation are affected by increasing age and/or ill health and socio-cultural background

● ability to apply knowledge in analyses to promote activity and participation in older people

● develop ability to participate in interdisciplinary discussions of how activities can be organised to promote activity and participation in older people

● knowledge and understanding required for carrying out R&D projects

● ability to assess and analyse instruments that are relevant for measuring the development and evaluation of rehabilitation and health promotion in older people on the levels of the individual, the group and the community

That is, at first glance the course content to a high extent was expressed in terms of goals or outcomes, paralleling or extending those explicitly stated under the heading learning outcomes, leading to a somewhat unfocussed impression of the course syllabus as a whole. Taking both types of outcomes into consideration, the multitude of learning outcomes thus formulated was somewhat contradictory to a statement in the course syllabus emphasising that students will be encouraged to choose their own learning objectives. It should also be noted that according to Biggs & Tang[11], in course planning there is a tension between coverage and depth of understanding. Given the range of learning outcomes stated for this course, the course might have been somewhat over-loaded with content, with potential for negative consequences for the depth of understanding.

According to Biggs & Tang[11], intended learning outcomes should be aligned at three levels. Therefore, the Swedish Higher Education Ordinance was searched to identify what Biggs and Tang call “the graduate attributes” for a Master’s Degree. While it should be kept in mind that Biggs and Tang’s book was not written based on the current European system for higher education, the term graduate attributes was not found in the ordinance. Instead, the term “goals” was being used in the ordinance text, but the intention of stating these goals should be comparable to that of stating graduate attributes.

A comparison showed that there was close to perfect correspondence between the ordinance text and the learning outcomes stated in the syllabus for the Master of Medical Science Program of which the course was a part. In both documents the subheadings “Knowledge andunderstanding”, “Skills and abilities”, and “Judgment and approach” were used to provide a structure. Consequently, alignment was found between the structure of the graduate attributes and the program learning outcomes. Analysing the text under each sub-heading in detail, there was close to perfect correspondence also in how the graduate attributes and program learning outcomes were expressed; the only adaptation to the Master of Medical Science Program was that the wording “main field of study” in the ordinance text had been specified to occupational therapy/nursing/ physiotherapy. Taking a critical standpoint, it could be questioned whether such a high correspondence is optimal, in particular since the intended learning outcomes should reflect not only the university’s and the teacher’s perspective but rather be developed based on active student involvement[7]. While it might be an efficient way to ensure that a program has potential to fulfil the formal requirements, just copying the graduate attributes as stated in the national ordinance and adjusting them slightly to form the program learning outcomes is not an approach likely to fulfil the ambitions of student-centred learning, including involving students in the definition of learning outcomes.

Turning to the course syllabus, the learning outcomes were structured under sub-headings corresponding to those of the graduate attributes and program learning outcomes. As can be seen in Table 1, a first impression of the course learning outcomes was that they were unnecessarily wordy, and it could not be easily done for a student to overview and grasp the entity of the learning outcomes listed. For several outcomes, similar wording was repeated; it should not have been difficult to combine/integrate several of them into fewer outcome statements. Besides better readability, it should then have been easier to analyse the alignment between the program and course learning outcomes. One observation made is that there was some incongruence in under which sub-heading to categorise different types of learning outcomes. For example, the second program learning outcome under “Knowledge and understanding” concerned methodological knowledge, while most of the methodology-related learning outcomes in the course syllabus were found under “Skills and abilities”. It is of course not self-evident how to differentiate between knowledge and understanding and skills and abilities, but with a more conscious aspiration to optimize the alignment between the program and course learning outcomes, a higher level of consistency could have been attained. It is noteworthy that there was no course learning outcome corresponding to the program learning outcome on the ability to present and discuss knowledge and arguments with different groups. According to Lindberg-Sand[7], in second cycle learning outcomes the application of knowledge is central, as is originality and the ability to integrate knowledge from different fields. Moreover, this is an example of lack of alignment across the different levels, in particular since presentation and discussion were stated as prominent parts of the teaching and learning activities during the course and also constituted the final examination task.

Table 1. Overview of program and course learning outcomes, and teaching and learning activities of the course analysed

Table 2. Categorisation of the learning outcomes as expressed in the course syllabus according to the SOLO Taxonomy a

Related more to the overall course content, the fact that this course was part of a cross-Nordic collaboration and the diversity of students, it is surprising that there were no course learning outcomes targeting cross-national dimensions of aging. Increasingly, gerontology is becoming a globalised field of study, and according to a publication on global aging and gerontology education[14], there is definitely an international mandate for gerontology in higher education. Currently in Europe, there is a social agenda stressing collaboration, cooperation and solidarity between programs and universities. The NordMaG is one out of the few Master level programs in Europe with an international mandate[6]. In order for such education to be successful, a comparative perspective is highly recommended.

Another approach to analysing the course learning outcomes was to identify the verbs used under each of the three sub-headings in the course syllabus and classify them according to the SOLO Taxonomy[13]. Under “Skills and abilities” (Table 2), the verbs used were expressed in different combinations in five learning outcomes. Two of the verbs used were not found in the SOLO Taxonomy. It should be noted that in all of the five learning outcomes under “Skills and abilities”, the expression “in a scientific manner” was used. Since it is important that students grasp the essence of the learning outcomes, it could be questioned whether such an expression is sufficiently concrete to support student learning in an efficient way. Under the two first sub-headings of learning outcomes, “independently” was emphasised, while under “Judgment and approach” collaboration with others was introduced as an outcome (Table 1).

Scrutinising the verbs used in the course learning outcomes, it should be noted that the number of verbs were equally distributed under the SOLO Taxonomy categories uni-structural, multi-structural, and relational (Table 2). This implies that in several aspects, to really reach the advanced level requirements, the course might benefit from a reformulation of the learning outcomes, with a specific attention to which verbs to use. In particular, there might be gains to make from reformulating a few of the outcomes to include more verbs from the category “extended abstract” of the SOLO Taxonomy, since as it stands, this level was only reached for learning outcomes under “Skills and abilities”. It should also be noted that no verbs belonging to the fifth and highest level of the SOLO Taxonomy were identified. Searching for support in the literature of how to differentiate learning outcomes between the three cycles of the Bologna process, exemplification was found in Lindberg-Sand’s[7] report on learning outcomes as a starting-point for the development of program and course syllabi. Consequently, even more important than including more verbs representing the higher levels of the SOLO Taxonomy might be to reflect upon how to better target the EQF-HE recommendation to include originality and integration of knowledge from several fields of knowledge in the learning outcomes. Referring back to the observation that there was no course learning outcome targeting the ability to present and discuss knowledge and arguments with different groups, according to the EQF-HE, for both first and second cycles communication skills and the ability to disseminate information to laymen as well as specialists are crucial.

3.2. Teaching and Learning Activities

Another type of alignment of crucial importance is that between course learning outcomes and teaching and learning activities[11], constituting the focus of the second step of the analysis. According to the course syllabus, the teaching and learning activities planned were individual work and work in small groups. There would be online forms of instruction, but also lectures and seminars. The course schedule provided contained two and a half days of information activities and lectures on campus. Besides lectures on different aspects of aging and one lecture about journal clubs, time was allocated for instructions about how to use Lund University’s distance learning platform and library resources. Four teachers with different professional backgrounds were involved. Turning to the literature listed in the course syllabus, there were eleven entries of different character; original papers, books and book chapters, and policy documents. According to a statement at the end of the literature list, more original papers and legislation documents would be added.

It was stated in the course syllabus that the teaching would be student-oriented and based on the students’ knowledge, skills and experiences. According to the course syllabus they would also be encouraged to choose their own learning objectives and work methods, thus training them in knowledge searching, critical thinking and problem solving. The Student Guide comprised an overview of the course content and four assignments to be completed during the course. It should be noted that in the overview, the student was encouraged to self-determine an area of special interest within which the learning outcomes defined in the course syllabus could be achieved. While this is an explicitly student-active approach, it should be noted that this instruction was not congruent with the aforementioned statement in the course syllabus, and thus a source of indistinctness that might cause uncertainty among students.

According to the Student Guide, each student would be assigned to a smaller virtual study group for collaboration during assignments 1-3, each with a set deadline. In brief, the assignments were:

1. to formulate and publish a work plan over how the goals set for the course will be achieved. The assignment includes feedback to the group colleagues on their work plans, and a teacher will provide individual feedback

2. to apply the journal club format to produce a critical summary of two designated scientific papers. For each group, the course administration has appointed a journal club chairperson. The teachers monitor and give input along the working process

3. to produce an individual abstract based on the work plan produced in assignment 1, and to prepare two relevant questions for every abstract from all course participants. To avoid plagiarism, each abstract must be checked by special software

4. to participate and present at a mini-conference (15 min Power-Point presentation, including time for questions). Students from abroad shall participate and present using online technology. Each student will be designated several presentations at which they shall pose questions

There would be active teacher engagement in assignments 1-2, while not explicitly stated for assignments 3-4. Feedback from teachers is important to support high level understanding, and constitutes a basic prerequisite for interactive teaching[11]. Since there was only one course week scheduled on campus, it would be crucial to utilise up-to-date information technology, and an introduction including individual assistance on how to use the university’s distance learning platform was scheduled already during the first day on campus. Another facet of the organisation of the teaching and learning activities was the formation of “smaller virtual study groups” early on. There was no information available about the principles for organising these groups, but given the diversity among students it is an issue well worth reflecting upon. While there are authors arguing that random group assignment is the best way to organise student groups, others advocate systematic procedures[11].

The analyse of the alignment of the course learning outcomes and the teaching and learning activities as presented in the course schedule and the Student Guide demonstrated that they aligned seemingly well with the learning outcomes. Still, it might not have been quite obvious to the students, and there should be room for optimization of the Student Guide.

3.3. Examination

The third step of the analysis targeted the alignment between course learning outcomes and assessment[11]. The examination format stated in the course syllabus was to produce a PowerPoint or poster presentation and a written assessment of the student’s own learning process, i.e. a reflective journal[11, 15]. According to the course schedule, the examination would take place during a mini conference on campus or online. A pass on the course required active participation in all components of the course and a pass on the examination. While it is common and not negative that teaching and learning activities are intertwined[11], it was not obvious from the written material what parts of the course content constituted the actual examination. A weakness in the course syllabus and Student Guide was that the forms of assessment and grading were not described. According to Biggs & Tang[11], it is important to provide information about what will be graded, and how and why. Based on the description of the examination in the course syllabus and in the Student Guide, it seemed as if the task performance would be assessed, while instead, the learning outcome attainment should be assessed. A noteworthy and positive facet of the examination was the reflective journal, while no concrete guidance about what was expected was given. According to a paper on reflection in interprofessional education[15], there is much to be gained by introducing more specific and concrete methods and activities to guide the students in how to build up a reflective journal.

4. Discussion

Overall in course development in higher education, a critical approach to the steering documents produced is advantageous, but not often applied and communicated. The analysis presented in this paper is an example of an approach to such critical appraisal, utilizing three frames of reference. Even if the three frames of references have different origins, in terms of national source as well as when they were first presented, it made much sense to use them in combination. That is, although the EQF-HE (BWG-QF 2005) is specific for current higher education in Europe, the approach to analysis combining it with Biggs & Tang’s[11] framework of constructive alignment and the SOLO Taxonomy[13], both presented by authors affiliated with universities in the U.S., was instrumental. That said, the approach to analysis presented in this paper could serve as an inspiration for teachers not only in European universities but also in other continents.

As demonstrated by the analysis presented in this paper, systematic analysis can result in a set of recommendations, useful for further optimization of a course. The main challenge for the course analysed in this paper is that of constructive alignment between levels[11], i.e. between the graduate attributes, program, course, the teaching and learning activities, and the examination. According to the documents used for the analysis, the following challenges should be considered for further development of the course:

● is it positive or negative to have perfect alignment between the graduate attributes and the program learning outcomes?

● the alignment between program learning outcomes and course learning outcomes should be reconsidered

● the correspondence between the teaching and learning activities and the course learning outcomes, as well as between the examination and the course learning outcomes, should be improved

● information about what will be graded during the assessment constituting the base for the examination should be provided

Another type of challenge is the diversity of students in the course. The fact that students with different professional/ disciplinary backgrounds[16] and representing different generations[4] are part of the same course and program needs to be reflected in the course design[16]. While this certainly presents strength and a valuable source for fruitful developments, it is also important to reflect upon whether differences between professions could require different learning outcomes[11]. It is a challenge to provide a course context supporting and training students with different disciplinary backgrounds to be able to see and understand the perspective of other professions[17]. Ethnic and cultural diversity among the students is an asset and represents an interesting potential for the development of an internationally attractive course, but there was no course content or learning outcomes reflecting a comparative perspective on gerontology. With aging as one of the grand global challenges of our time[18], there is much to be gained from developing courses in gerontology not only attracting students from different countries and continents, but also fostering gerontologists with true competence for international contributions to the field. Adopting a critical appraisal to existing courses, followed by optimization and progressive development, is one way towards reaching such goals. The course analysed seems to have a strong potential for development, based on the interesting but challenging mix of students and the highly relevant outcomes stated. Gerontology education with a global perspective can inform policy, planning, service design, research, etc.[14], and the potential of this course to encompass cross-national comparative perspectives should be explored and utilised for further development.

Based on the findings of the current analysis, there is reason to recommend teachers to seriously consider the possibilities for optimizing the way learning outcomes are stated. For such development, national legislation and regulations of higher education as well as the three frameworks utilized in this paper serve as tools to make such work more efficient. To develop the course I analysed further, to avoid inaccuracy it would be important to scrutinise the course syllabus text under the heading course content and revise it to minimise the risk of mixing it up with the learning outcomes stated. Moreover, there are gains to be made in reformulating the course learning outcomes. In order to increase readability, several of them could be integrated, and there is also room for improvement in terms of optimized alignment at three levels; program learning outcomes, course learning outcomes, and teaching and learning outcomes.

While working on such optimization, the SOLO Taxonomy[13] as well as the EQF-HE framework could be useful to sharpen the learning outcomes in terms of second cycle qualifications. Most important, students should be involved in the reformulation of the learning outcomes, in particular since the teaching and learning activities of the current course to a great extent are based on student activity and linked to the student’s personal goal-setting. Another recommendation for further development of the course analysed is to structure the examination, and to make it clear what will be assessed and graded, why and how – in relation to the learning outcomes.

A limitation of the analysis presented is that it does not include any information about the course evaluation results. At the Faculty of Medicine, Lund University, course evaluation is mandatory, but as is often the case, the response rate to the evaluation of this course was very low. That is, only four of the students responded to the standardised online questionnaire that is routinely distributed after course completion. While these students were very positive in their evaluation, only when additional course cohorts have passed and evaluated the course, a fair judgment of its quality can be made. It would of course have been a strength if I had been given the opportunity to interview the students. However, since I was not at all engaged in the course, I did not have any possibility to influence course arrangements. The fact that students travelled from abroad, with only short stays on campus, represented a challenge to the possibilities to gather data on student opinions. Consequently, for logistical reasons I did not manage to evaluate the course from a student perspective. Actually, the value of student feedback at the end of a course is not to be taken for granted, with some provokingly claiming that such evaluation activities may be more effective in supporting teacher promotion than informing course improvement[19]. In fact, I suggest that engaging the students in the development of course syllabi and the formulation of learning outcomes might be a more efficient, while as yet not studied, way of course improvement.

5. Conclusions

Based on the combination of three different but complementary frames of reference, the analysis and findings presented in this paper contribute to laying the ground for the systematic revision and optimization of course curricula at Master’s level in gerontology, with potential to be applied in different national contexts. Future studies should include evaluation of the value of student engagement in course development and the formulation of learning outcomes – from the student perspective.

ACKNOWLEDGEMENTS

Thanks are extended to Associate Professor A. Malmgren Fänge, acting as a “critical friend” commenting on the analysis. This paper was prepared within the Centre for Ageing and Supportive Environments (CASE) at Lund University, financed by the Swedish Research Council for Health, Working Life and Welfare and the Ribbingska Foundation, Lund, Sweden.