Pilot studies

The University of Ghent –  the Centre for Higher Education Governance Ghent (CHEGG) has prepared two pilot pan-European comparative studies into how higher education institutions respond to the needs of students and society, based on thousands of external quality assurance reports being available in our Database for External Quality Assurance Results (DEQAR).

The research on which the two studies are based on was undertaken in the context of the DEQAR project (Database for External Quality Assurance Results). The project has been carried out with co-funding from the European Commission (ERASMUS+ KA3 grant). The content of the study and information contained therein remains the responsibilty of the authors.

Find out more about the concept (research questions) and the results (extract of conclusions) of the pilot studies in the section below. The papers may be also read in full, by downloading them into your devices.

The results of the pilot studies will be also presented and discussed with participants at the European Quality Assurance Forum (EQAF) taking place in Berlin on 21-23 November, during one of the three parallel plenary sessions discussing perspectives on QA.

First Pilot Study - The translation of the ESG in institutional QA reports

Research questions

For the first pilot study, published in April 2019, Jeroen Huisman and Maria Manatos analysed institutional QA review reports exploring the following research questions:

  • “Whether the ESG related to programme’s design, approval and monitoring are present in institutional external quality assurance reports;
  • Whether quality assurance agencies are translating the ESG differently when performing institutional audits of HEIs. Particularly, we aim to understand how quality assurance agencies are translating the ESG, their main concepts and dimensions, and how audit panels are applying the national guidelines for institutional external quality assurance reports and how they are approaching the ESG.”
Extracts from the concluding remarks from first pilot study
  • “Globally, our research responds positively to both our main research questions. On the one hand, the research concludes that the ESG related to programme design, approval and monitoring are present in institutional external quality assurance reports. A quantitative analysis to the frequency of ESG’s concepts and dimensions confirms that the ESG ‘appear’ in the reports, though to different extents across countries.”
  • “On the other hand, quality assurance agencies interpret and translate the main concepts and dimensions from the ESG differently, when performing institutional audits of HEIs. The differences are reflected in the ‘form’ of the reports, in particular, in the length, the focus and the guidelines for the reports. Indeed, quality assurance agencies have different foci and formulate different guidelines for institutional external quality assurance.”
  • “Under an umbrella of ‘standardisation’ and ‘homogenisation’ it is then interesting to notice how the quality assurance agencies translate and interpret the ESG, developing their own national guidelines, and how the audit panels apply such guidelines in the reports.” 
Read and download the 1st pilot study by UGent (CHEGG)

Second Pilot Study - The translation of the ESG in programme QA reports

Research questions

For the second pilot study, published in October 2019, Jeroen Huisman and Maria Manatos analysed programme QA accreditation exploring the following questions:

  • How do accreditation panels offer evidence for their findings with respect to:
    • encouraging students to take an active role in creating the learning process (ESG1.3)
    • and consistently applying pre-defined and published regulations covering all phases of the student “life cycle” (ESG 1.4)?
  • Are there differences – with respect to the findings between countries and types of institutions?
Extracts from the concluding remarks from the second pilot study
  • We conclude that globally the ESG 1.3 and 1.4 are ‘present’. […] However, they are subject to interpretation, first by the agency, which edits the ESG, though not radically changing them; second, by the panels, who also edit the ESG and present different evidences for the different dimensions inside the ESG
  • re ESG 1.3, “the panels offer more explicit evidences for findings related to student’s assessment (mostly assessment procedures) and also (though to a lesser extent, and sometimes less explicitly) to the adaptability of teaching and learning (according to students’ paths, labour market needs, etc); and more implicit or even no evidence for findings related to students’ motivation, complaints, engagement, student-teacher relationship.”
  • re ESG 1.4 “the panels tend to offer more evidences for findings related to the admission (mostly admission procedures) and progression (namely procedural aspects and ways to support students to progress in their studies), and certification, than related to informal learning and recognition (particularly important European references with regard to recognition). “
Read and download the 2nd pilot study by UGent (CHEGG)

As methodology, both studies follow a content analysis of the external quality assurance reports from in order to understand how quality assurance agencies translated the ESG for external quality assurance based on concepts (not only from the standards but also from the guidelines) related ESG 1.2 (2005), ESG 1.2 and 1.9 (2015) for the first study and ESG 1.3 and ESG 1.4 for the second study.