Database of Precedents
-
2.2 Designing methodologies fit for purpose – AKKORK – Partial compliance (2020) Clarity of activities; lack of stakeholder involvement in the development of methodologies and criteria;
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 2.2 Designing methodologies fit for purpose Keywords Clarity of activities; lack of stakeholder involvement in the development of methodologies and criteria; Panel conclusion Partial compliance Clarification request(s) – RC decision Partial compliance “Considering the design of AKKORK’s methodology of external QA procedures, the panel noted that the aims of the different activities were not clearly differentiated and that there were inconsistencies between the different language versions of the AKKORK ‘s website (English and Russian). In its additional representation the agency responded that it has made revisions to the information on its website. While the Register Committee could verify the publication of the procedures, the Committee further noted that the agency also has ‘on offer’ the activity quality assurance of educational programmes on the level of higher education and remains to demonstrate that the activity has been defined and designed to achieve the objectives set for it, as required by the standard. The panel further commented on the lack of involvement of external stakeholders, apart from the representatives from its own bodies, in the design and continuous improvement of the offered procedures. The agency commented in its additional representation that it has developed a Regulation on collaboration with partners designed to be implemented in AKKORK’s daily routine. The Register Committee welcomed the thorough work in the development of a cooperation regulation with stakeholders, but could not confirm that the Regulation is already in effect as no evidence of the stakeholders' engagement was provided for in the recent substantive changes introduced by the agency. ”
Full decision: see agency register entry
-
2.3 Implementing processes – AKKORK – Compliance (2020) Lack follow-up procedures
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 2.3 Implementing processes Keywords Lack follow-up procedures Panel conclusion Partial compliance Clarification request(s) – RC decision Compliance “In its decision of inclusion, the Register Committee noted that AKKORK’s follow-up procedures were not consistently implemented for all off the agency’s external quality assurance activities and therefore flagged this matter for future attention. In its 2019 review report, the panel showed that AKKORK had taken steps to address its flag by including follow-up processes as part of its contracts with higher education institutions. The panel found that - while follow-ups are not part of all contracts signed with the reviewed institutions, that they are nevertheless carried out after a conditional accreditation. The panel further underlined a number of shortcomings related to AKKORK’s independent accreditation reviews at institutional level and AKKORK’s IQAS procedures. Since these procedures are no longer on offer by AKKORK, the Register Committee found that the panel’s concerns were therefore addressed”
Full decision: see agency register entry
-
2.4 Peer-review experts – AKKORK – Partial compliance (2020) student involvement; Mismatch between number of experts enlisted in the reports and number of experts required according to agency’s review methodology; Unavailability of some reports
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 2.4 Peer-review experts Keywords student involvement; Mismatch between number of experts enlisted in the reports and number of experts required according to agency’s review methodology; Unavailability of some reports Panel conclusion Substantial compliance Clarification request(s) – RC decision Partial compliance “The Register Committee flagged AKKORK’s involvement of students in its previous decision of inclusion. The panel received confirmations during its multiple interviews that all of AKKORK’s peer review expert groups now included a student member.
The panel’s findings nevertheless show inconsistencies in the number of experts, listed by AKKORK in its published reports and the number of experts expected to be involved according to AKKORK’s own methodology. In particular, the review panel expressed concerns about the use of single experts for professional-public accreditations, although it was told that not all panel members were in fact listed in the prepared reports. The panel therefore concluded that while the formal procedure in the composition of review panel was met on paper, the practice differed. The Register Committee noted from AKKORK’s additional representation that its procedures are published, but noted that the above raised concerns were not addressed and remain unresolved. The Register Committee noted from the statement by the agency - in its further information on the report and additional representation - that the links have been checked and all review panel members are now listed in the published reports. Following a check of the agency’s website, the Register Committee found that a number of reports still have broken links (see the agency’s Register of programmes, under the Russian version of its website). ”
Full decision: see agency register entry
-
2.5 Criteria for outcomes – AKKORK – Partial compliance (2020) publication of procedures; Criteria are not applied consistently; Lack of consistency in decision making
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 2.5 Criteria for outcomes Keywords publication of procedures; Criteria are not applied consistently; Lack of consistency in decision making Panel conclusion Non-compliance Clarification request(s) – RC decision Partial compliance “In its decision of inclusion, the Register Committee flagged AKKORK’s publication of detailed criteria for all its procedures. In its additional representation AKKORK stated that the detailed procedures and criteria for decisions have now been published. The Register Committee could verify that that the criteria are now published. The panel learned that in many cases experts relied on their personal review experience, rather than following AKKORK’s guidelines and methodology, and that AKKORK’s criteria were not applied consistently in the agency’s decision making. According to AKKORK’s revised Guidelines for Reviewers on Conducting External Evaluation of Education Quality and Quality Assurance at Programme level (adopted as of 30/01/2020), experts are expected to follow a clear methodology in their evaluation, and not their personal review experience. In its additional information to the review report, AKKORK stated that its decision-making criteria had been checked for consistency following its external review. The agency further provided a mapping of the scale for its decision making on programme accreditation ”
Full decision: see agency register entry
-
2.6 Reporting – AKKORK – Partial compliance (2020) Publication of all reports
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 2.6 Reporting Keywords Publication of all reports Panel conclusion Non-compliance Clarification request(s) – RC decision Partial compliance “In its previous decision of inclusion, the Register Committee flagged AKKORK’s practice of ensuring the consistent publication of all external evaluation reports. In its additional information and additional representation to the review report the agency claimed that all its review reports and decisions were now published on its website, including the reports from its professional-public accreditation activity. The Register Committee could verify that with a few exceptions all of these reports are now published by the agency on its website i.e. links included under the Russian version of its agency’s website under Register of programmes. ”
Full decision: see agency register entry
-
2.7 Complaints and appeals – AKKORK – Compliance (2020) Lack of an independent appeals committee
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 2.7 Complaints and appeals Keywords Lack of an independent appeals committee Panel conclusion Substantial compliance Clarification request(s) – RC decision Compliance “The Register Committee noted that appeals procedures are published together with every procedure on the agency’s website, except the independent accreditation at the institutional level. To date AKKORK has only received a complaint, but no appeals. The panel’s analysis show that AKKORK’s Appeals Committee involves members of the International Accreditation Council, the same body that is involved in the accreditation decision-making process in addition to the members of its Supervisory Board. The Register Committee underlined that in such cases the impartiality of the decision making in considering appeals is not ensured. In its further information to the review report, AKKORK stated that its regulation on appeals have been updated and that its new appeals body, appointed by AKKORK’s Supervisory Board, includes members from the agency’s partner organisations. In its additional representation the agency has further provided a link to its revised regulation on appeals. ”
Full decision: see agency register entry
-
3.1 Activities, policy and processes for quality assurance – AKKORK – Partial compliance (2020) Mission implementation in daily work; Clear distinction between activities; Stakeholders’ involvement; Involvement of students
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 3.1 Activities, policy and processes for quality assurance Keywords Mission implementation in daily work; Clear distinction between activities; Stakeholders’ involvement; Involvement of students Panel conclusion Partial compliance Clarification request(s) – RC decision Partial compliance “In its analysis the panel commented that it was unclear which is the body in the agency responsible for developing a strategy, how AKKORK’s procedures are developed and what types of activities and services the agency is offering. Furthermore the panel could not find any clear evidence on how the agency’s mission is translated into the daily work of the agency. The panel also noted that AKKORK’s actual structure and relations between its governing bodies were different to the ones presented in the last version of the statutes, adopted in
2013. While stakeholders are part of AKKORK’s multiple bodies, the panel noted that their involvement could be broadened. In particular, the panel recommended that students be involved also in other bodies than the Advisory Council, where only representatives from professional organisations and higher education institutions are present. The Register Committee noted from the additional information that AKKORK has updated its mission statement and detailed its objectives and main activities on its website. The agency further stated that a new Strategic plan 2020-2024 has been approved by the Supervisory Board and that its Financial Plan also have been updated. The Register Committee could also verify that the new Strategic Plan has been adopted by the Supervisory Board at its meeting on December
2019. While the Committee welcomed the steps taken by the agency to address the panel’s concerns, the Committee further considers that these changes still have to be fully implemented and externally reviewed by a panel, i.e. showing evidence of a robust yearly work planning, implementation of the strategic plan. ”
Full decision: see agency register entry
-
3.4 Thematic analysis – AKKORK – Partial compliance (2020) Lack of systematic approach to thematic analysis; Absence of regular analysis;
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 3.4 Thematic analysis Keywords Lack of systematic approach to thematic analysis; Absence of regular analysis; Panel conclusion Partial compliance Clarification request(s) – RC decision Partial compliance “The Register Committee noted that the agency has produced two analyses since its last review in 2014, both written in Russian but only published on the English version of AKKORK’s website. According to the panel, the last publication, however, does not meet the requirements of a thematic analysis. In particular, the panel noted the great qualitative difference between the publications, indicating a missing systematic approach to the publication of thematic analysis. The Register Committee underlines the panel’s recommendation that the agency should establish a clear process to address all of its activities in thematic analyses. ”
Full decision: see agency register entry
-
3.6 Internal quality assurance and professional conduct – AKKORK – Partial compliance (2020) Lack of sufficient and fit for purpose IQA processes; Inconsistency of presentation of internal structures;
AKKORK
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 3.6 Internal quality assurance and professional conduct Keywords Lack of sufficient and fit for purpose IQA processes; Inconsistency of presentation of internal structures; Panel conclusion Partial compliance Clarification request(s) – RC decision Partial compliance “The Register Committee noted that while the responsibilities in internal quality assurance are defined in the ‘AKKORK Internal Regulations’, the main tool of the agency for internal quality assurance of its activities are the regular staff meetings. The panel’s analysis showed a number of concerns related to the agency’s internal workings, i.e. inconsistencies in the information presented on the agency’s website, the information regarding the publication of the reports or information available on external QA activities on the Russian and English parts of the website; inconsistency in the presentation of the agency’s organizational chart in ‘AKKORK Internal Regulations’ and the structure outlined in the review report, etc.. Considering the above examples, the panel considered that not all outcomes of the agency’s work, especially not the production of documentation, are covered by the existing QA cycles. ”
Full decision: see agency register entry
-
2.4 Peer-review experts – NCPA – Compliance (2020) learners in non-traditional higher education provision
NCPA
Application Renewal Review Full, coordinated by ENQA Decision of 16/03/2020 Standard 2.4 Peer-review experts Keywords learners in non-traditional higher education provision Panel conclusion Substantial compliance Clarification request(s) Panel (12/07/2025)
RC decision Compliance “While the panel underlined the different status of "listeners", as opposed to “students”, who enrol into these programmes, the Register Committee considered that the institutions and programmes in question were part of the higher education system and their accreditation thus within the scope of the ESG; this was also NCPA's own assessment at eligibility stage.
While the panel considered that, in light of the small number of these accreditations, a stricter judgement had “overstretched” the relevance of the matter, the Register Committee considered that this was not only a matter of numbers. The practice at the time of review meant that two types of accreditations did not fully comply with the standard.
In its additional representation, NCPA explained that it had revised the corresponding regulations and now required the presence of a “listener” (i.e. a student/learner enrolled in a further professional education programme, see above) on the expert panels. [...]
The Register Committee therefore considered that the issue has been addressed and was now able to concur with the panel’s conclusion that NCPA complies with the standard.
”
Full decision: see agency register entry
-
2.6 Reporting – ACCUA – Partial compliance (2020) publication of negative reports
ACCUA
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 2.6 Reporting Keywords publication of negative reports Panel conclusion Full compliance Clarification request(s) – RC decision Partial compliance “The Register Committee noted that AAC-DEVA publishes the full reports of its external reviews, except those with a negative result of its (ex-ante) verification programme. While the review panel found the agency’s explanations – that it is not necessary to publish information about study programmes that are not going to be offered – reasonable, the Register Committee underlined that it can be of interest for the public to know which concepts were denied accreditation and why. In particular, such information is important if the institutions applies for accreditation of the same programme with another quality assurance agency, which needs to be able to find out that it was earlier denied accreditation by AAC-DEVA. In its additional representation the agency stated that it is committed to transparency, but that the publication of the ex ante reports would create confusion among students and might create prejudice against the concerned university. AAC-DEVA added that that the current legal framework in Spain does not allow ex-ante accreditation to be carried out by a different QA agency and that the universities are required to wait at least two years before resubmitting the programme for the ex-ante verification. The Committee however could not follow the agency’s reasoning for not publishing reports where the agency takes a negative decision since such reports have been published in the past by registered agencies (in other regions of Spain and elsewhere), without the risk of confusion. The reputation of a higher education institutions should not be affected by an ex-ante negative report more than the publication of any other report with a negative outcome. The Register Committee added that the higher education institution could always have the option to withdraw the proposal of a study programme before a decision by AAC-DEVA is taken, so as to avoid any public decision from being taken. The Register Committee underlined that the standard requirement is clear, and does not include exceptions in regards to the publication of reports for any form of finalised external QA procedure. ”
Full decision: see agency register entry
-
2.7 Complaints and appeals – ACCUA – Partial compliance (2020) Lack of an impartial decision making → lack of an independent appeals committee
ACCUA
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 2.7 Complaints and appeals Keywords Lack of an impartial decision making → lack of an independent appeals committee Panel conclusion Substantial compliance Clarification request(s) – RC decision Partial compliance “The Register Committee noted that higher education institutions reviewed by AAC-DEVA may file an appeal before the Universities Council in case of verification/modification, accreditation renewal and institutional accreditation. If the claim is accepted, the appeal is sent to AAC-DEVA where a group of experts who have not intervened in the evaluation that led to the unfavourable decision will revise the report. The Committee underlined that AAC-DEVA’s current appeals process does not fully ensure the impartiality of decision making, as the Universities Council is both the body responsible for the accreditation decisions and the resolution of appeals after such procedures. In case of DOCENTIA, IMPLANTA and the accreditation of foreign language skills, the resolution of the appeal is with the same body who issued the initial decision. Thus the impartiality of the decision making is not fully ensured in any of the external QA activities carried out by the agency. In its additional representation the agency explained that according to the Spanish legal framework the agency cannot exclude the University Council in the consideration of the appeals procedure – in the case of verification/modification, accreditation renewal and institutional accreditation. The appeal is assessed by a commission which includes experts that have not intervened in the evaluation, and following their examination, if the appeal is ratified it will be further submitted to the corresponding evaluation body for reevaluation. The Register Committee noted that AAC-DEVA intends to establish an Appeals Committee to consider all appeals related to the evaluation of programmes. The decision of the Appeals Committee will be binding for the DEVA Director. Furthermore, while higher education institutions have been able to state their dissatisfaction with the review process i.e. by filling in an online ‘suggestions form’, AAC-DEVA lacks a formal procedure that explains what form of complaints may be submitted and considered by the agency. In its additional representation the agency stated that it has designed a specific procedure for complaints as part of its quality management system and that it has updated its online form for the submission of such complaints. The Register Committee could verify that a complaints protocol has been developed and that the modified online form allows the submission of various forms of complaints. ”
Full decision: see agency register entry
-
3.1 Activities, policy and processes for quality assurance – ACCUA – Partial compliance (2020) Stakeholders’ involvement; Involvement of students; Lack of strategical planning
ACCUA
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 3.1 Activities, policy and processes for quality assurance Keywords Stakeholders’ involvement; Involvement of students; Lack of strategical planning Panel conclusion Partial compliance Clarification request(s) – RC decision Partial compliance “The Register Committee noted the panel’s concerns in terms of the agency’s engagement with stakeholders. While the agency has strong connections with universities, the relationship with student representatives, and other civic and social groups is very weak. This lack of engagement is reflected in both the composition, working arrangements and internal interaction of the agency’s committees. The Register Committee considered the agency’s explanation that the current regulation of its statutes limit the involvement of students, but the Register Committee underlined that stakeholder involvement such as students is a key requirement of the standard. the agency lacked a strategic focus and that there was no evidence of governance level oversight of strategic planning or review by AAC-DEVA. In its additional representation the agency reported that steps have been already made to ensure the presence of students and professionals in all evaluation committees for ex-ante verification, modification and accreditation renewal. The agency stated that it intends to further strengthen the role of the Technical Committee for Evaluation and Accreditation (CTEyA) by incorporating professionals and by increasing the number of students. Considering the lack of a strategic focus and governance oversight, the agency added that its new Strategic Plan for 2021-2026 is in preparation and that it will reflect the engagement of stakeholders and the different levels of strategic governance. ”
Full decision: see agency register entry
-
3.5 Resources – ACCUA – Compliance (2020) Allocation of resources; Financial sustainability
ACCUA
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 3.5 Resources Keywords Allocation of resources; Financial sustainability Panel conclusion Substantial compliance Clarification request(s) – RC decision Compliance “In its last decision of inclusion the Register Committee flagged AAC-DEVA’s ability to acquire and consolidate the resources required to organise site-visits as part of its periodic re-accreditation procedures. As AAC-DEVA has been able to carry out site-visits within its procedures, the flag has been addressed. The Register Committee noted that AAC-DEVA acknowledged in its SWOT analysis that it did not achieved what it aspired it set out to do in the past five years. These concerns were mainly due to the current organisational structure of the agency and lack of strategic planning which has lead to difficulties in the proper allocation of resources and development of prospective units for the full implementation of thematic analysis. In its additional representation AAC-DEVA stated that it had been able to provide all the necessary resources to set up and maintain its external QA activities and that its budget has been approved without any reductions. Concerning the problems related to the structural organisation of the agency, AAC-DEVA explained that this are addressed in the development of its Strategic Plan. ”
Full decision: see agency register entry
-
3.6 Internal quality assurance and professional conduct – ACCUA – Partial compliance (2020) Lack of sufficient and consistent IQA processes
ACCUA
Application Renewal Review Full, coordinated by ENQA Decision of 22/06/2020 Standard 3.6 Internal quality assurance and professional conduct Keywords Lack of sufficient and consistent IQA processes Panel conclusion Substantial compliance Clarification request(s) – RC decision Partial compliance “In its previous decision of inclusion, the Register Committee flagged the effectiveness of AAC-DEVA’s internal quality assurance arrangements. In May 2018, AAC-DEVA established an Internal Quality Assurance Commission to review its own internal quality assurance methodology, but at the time of the site-visit the agency was still reviewing its internal documentation. The panel saw insufficient evidence of a pattern of continuous improvement in the internal quality assurance of AAC-DEVA. While there has been a spur of activity in preparation for the external review the panel did not consider this was sufficient to achieve meaningful results. In its additional representation the agency reasserted its commitment for improving its internal QA system and that it was in the process of reviewing its Services Charter and setting up a web application designed to support continuous improvement and facilitate the management of the agency’s activities. ”
Full decision: see agency register entry
-
2.3 Implementing processes – QQI – Partial compliance (2019) incomplete implementation of reviews for independent private providers
QQI
Application Renewal Review Full, coordinated by ENQA Decision of 05/11/2019 Standard 2.3 Implementing processes Keywords incomplete implementation of reviews for independent private providers Panel conclusion Substantial compliance Clarification request(s) – RC decision Partial compliance “In its 2016 decision on QQI's Substantive Change Report, the Register Committee flagged for attention the use of site visits. The Committee noted that site visits are not used in some processes, but that this was adequately explained by a “lighter touch in recognition of the greater responsibility held by those providers” (p. 27). The Register Committee concurred with the panel that the alternative approach used is effective and robust in the light of the process' objectives.The Register Committee noted that QQI has finalised its external quality assurance processes and moved to full implementation of most processes since the last review.The external review report, however, noted that for independent private providers “no cyclical institutional reviews have taken place as a result of the delay in approving those providers’ Quality Assurance Procedures through Re-engagement” (p. 28). While the report cited a combination of reasons for that and underlined that it was not the result of poor intentions on the part of the agency, the report noted that some providers may actually go up to 12 years without an institutional review. The panel further noted that the “risk of concerns about quality going unnoticed in these providers” was partly, but not wholly, mitigated by QQI having more intensive engagement with them through theirprogramme validation relationship (p. 28).In light of the incomplete implementation of reviews for independent private providers the Register Committee was unable to concur with the panel's conclusion of compliance, but considered that QQI only partially complies with the standard.”
Full decision: see agency register entry
-
2.4 Peer-review experts – QQI – Compliance (2019) Panels composition
QQI
Application Renewal Review Full, coordinated by ENQA Decision of 05/11/2019 Standard 2.4 Peer-review experts Keywords Panels composition Panel conclusion Full compliance Clarification request(s) – RC decision Compliance “In its 2016 decision on QQI's Substantive Change Report, the Register Committee flagged for attention the composition of QQI expert panels.The Register Committee noted that some specific QQI processes do not use traditional expert panels, but are based on desk assessments or dialogues by QQI staff, followed by subsequent decisions where applicable. The Committee noted that the decision-making bodies include all perspectives that are otherwise required to be on a panel.The Committee further noted that wherever panels are deployed their composition complies with the standard. The Register Committee therefore concluded that the flag was addressed and concurred with the panel's conclusion that QQI complies with the standard.”
Full decision: see agency register entry
-
2.7 Complaints and appeals – QQI – Compliance (2019) Scope of the appeals system
QQI
Application Renewal Review Full, coordinated by ENQA Decision of 05/11/2019 Standard 2.7 Complaints and appeals Keywords Scope of the appeals system Panel conclusion Full compliance Clarification request(s) – RC decision Compliance “The Register Committee noted the panel's remark that the outcomes of cyclical institutional reviews can currently not be appealed, while noting thata provider “may still wish to challenge the findings in a way that would be better suited to an appeals process rather than a complaints process” (p. 33).The Register Committee has generally understood the standard to require that all results of external quality assurance processes be open to appeal, hence including reports that do not include “categorical decisions”.The Committee therefore underlined that QQI should consider widening the scope of its appeal system in monitoring the fitness for purpose of the current arrangements, per the panel's remarks. ”
Full decision: see agency register entry
-
3.4 Thematic analysis – QQI – Compliance (2019) Producing effective thematic analysis reports
QQI
Application Renewal Review Full, coordinated by ENQA Decision of 05/11/2019 Standard 3.4 Thematic analysis Keywords Producing effective thematic analysis reports Panel conclusion Full compliance Clarification request(s) – RC decision Compliance “In its decision of 2015 to admit QQI to the Register, the Register Committee flagged for attention the production of thematic analyses by QQI.The Register Committee understood from the panel's report that QQI has been proactive and effective in producing a range of thematic analysis reports, which are considered useful in the sector for improving quality and quality assurance, even though not having completed full cycles in all external quality assurance activities.The Register Committee therefore considered that the flag has been addressed and concurred with the panel's conclusion that QQI complies withthe standard.”
Full decision: see agency register entry
-
3.6 Internal quality assurance and professional conduct – QQI – Compliance (2019) Efectiveness of internal QA
QQI
Application Renewal Review Full, coordinated by ENQA Decision of 05/11/2019 Standard 3.6 Internal quality assurance and professional conduct Keywords Efectiveness of internal QA Panel conclusion Full compliance Clarification request(s) – RC decision Compliance “The development of QQI's internal quality assurance system was flagged for attention when QQI was admitted to the Register in 2015.The Register Committee noted that the panel described QQI's internal quality assurance instruments as effective. They cover all its external quality assurance processes and thus respond to the recommendation made in the last external review of QQI.While the panel noted that some future adjustments might be necessary once further external QA processes are rolled out, the Register Committee considered that the flag has clearly been addressed.The Register Committee therefore concurred with the panel's conclusion that QQI complies with the standard.”
Full decision: see agency register entry