Quality assurance regime fails to meet European standards

More than one in five of 189 study programmes evaluated by the Swedish National Agency for Higher Education (HSV) have been graded 'unsatisfactory'. And to the embarrassment of the Swedish government, the system of quality assurance itself has been rated unsatisfactory by a panel of European experts. The agency published its evaluations of courses at bachelor and master levels from 25 institutions, using a scale from 'very good' to 'good' and 'not satisfactory', on 25 April. Eight academic fields were evaluated and most were marked either good or very good. But 41 programmes were rated not satisfactory. The higher education institutions hosting those programmes have been given a year to improve their standards. Meanwhile, Sweden was found to have failed to meet European Association for Quality Assurance in Higher Education (ENQA) regulations by an ENQA review panel. Sweden has been a member of the association since it was established in 2000. The panel said Sweden's quality assurance system was “fundamentally at odds with ESG”, the European Standards and Guidelines for Quality Assurance in the European Higher Education Area. It said in a report that ESG’s first principle was that external quality assurance should build on the results of internal quality assurance. But the Swedish system “takes no account of institutions’ arrangements for internal quality assurance, except at the very margins”. The report added that while a basic principle of ESG was that quality assurance systems should lead to enhancement, the Swedish system made no recommendations for improvement. The impression of staff and students interviewed was that the purpose was merely to rank programmes. “ESG are about quality as much as standards, but the Swedish system is interested in standards almost exclusively,” the panel concluded. “It is a fundamental requirement of the ENQA criteria that agencies should be ‘independent’, in terms of the ‘definition and operation of [their] procedures and methods’. “HSV cannot be considered ‘independent’, due to the extent to which their procedures and methods, as well as overall aims and objectives, have been dictated by government.” Former education minister Ibrahim Baylan told the Swedish parliament that higher education in the country was ”at risk of being devalued internationally”. Kåre Bremer, the rector of Stockholm University, said the ENQA panel’s conclusion was embarrassing for the country. But ENQA Director Maria Kelo told University World News that the association had yet to evaluate or make a judgment on Sweden's new quality assurance system, having received the panel’s report only a few days previously. The report will be studied by an ENQA review committee of three members, which will make its recommendations to the association's board. A decision on HSV's membership of ENQA will be taken by the board in June. Minister of Education Jan Björklund said that the HSV had provided an accurate picture of the quality of the courses evaluated and had shown that many courses were not of passable quality. “It is the knowledge of the students that is important, and that is what is measured in the evaluation,” he said. The government is going to use the evaluation to inform the allocation of funds to universities. Those graded 'very good' will receive additional resources while those not satisfying quality criteria will have a year to improve courses. A further 6,000 programmes are due to be evaluated by 2014. However, the chair of the Swedish Rectors' Conference, Pam Fredman, who is the rector of Gothenburg University, vice-chair Marita Hilliges and secretary-general Marianne Granfelt issued a joint statement saying they were ”deeply worried” by the ENQA panel’s conclusion. They urged the government to “instruct the HSV to change the present evaluation system so that it complies with the European Standard and Guidelines”. The ENQA panel’s visit had been commissioned by HSV precisely to review compliance with ENQA regulations. Led by Professor Helmut Konrad, the former rector of Graz University, the panel's job was to “establish whether the agency met the ENQA standards with regard to both its external evaluation processes and its internal quality assurance”. In September 2009 the government rejected a proposal from the Association of Swedish Higher Education (SUHF) and the national student organisation (SFAS) for a new assurance system and introduced the current system. This led to the resignation in 2010 of the University Chancellor of Sweden, Professor Anders Flodström, as reported by University World News at the time. In the new evaluation system the HSV is using four criteria: students’ final theses, surveys of previous students, institutions’ self evaluations and students’ experience. But the majority of evaluation decisions were made on the sole basis of students’ theses, drawing criticism from Flodström, the SUHF and the ENQA panel. Lena Adamson, an associate professor of psychology now working as an expert in quality assurance issues for the European Institute of Innovation and Technology and the Council of Europe, who resigned from her position as secretary-general at HSV due to the introduction of the current quality assurance system, said: “Although important to include student work in a quality assurance system, it is very naïve to believe that this system will stimulate Swedish higher education in the right direction. “The assessments are done (necessarily) in relation to a sample of learning outcomes from the Swedish national qualification framework that are applicable only to theses work. “This will drive development in a pre-Bologna direction, not towards the competence-based higher education we need, in addition to its main driving force for universities: to search for ‘good’ students rather than developing their teaching quality.” But Maria Sundquist, head of the evaluation unit at HSV, said: “We have evaluated the outcomes of the study programmes by examining how well students achieve the objectives. We have found that it is possible to evaluate the quality based on this documentation and by the methods the evaluation system is using.” Of the first 189 courses evaluated and graded in eight fields of study, 29 were graded 'very good', while 41 did not pass the evaluation. Of the 29 top-graded courses, 79% were at old universities and élite professional institutions. Of the 41 courses that did not pass the evaluation, 58.5% were found at new universities. However, 17 courses that did not pass the evaluation were found at the older universities. The highest number of flunked courses was in national economics, where 31% of the courses received the lowest grading, and only two courses out of the 45 were awarded the top grade. Out of 48 courses examined in geoscience and cultural geography, a third were graded 'very good', while seven did not pass. Konrad told University World News: “The switch to learning outcomes, as against measuring input elements, is in principle a step in the right direction. “But the Swedish system is much too simple, excluding helpful information and fixed on the independent students' projects without having in mind the studying conditions.” However, he added: “Our references were the ESG, and the ESG themselves need a relaunch.”