Database of Precedents

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.3 Implementing processes
Keywords	incomplete implementation of reviews for independent private providers
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“In its 2016 decision on QQI's Substantive Change Report, the Register Committee flagged for attention the use of site visits. The Committee noted that site visits are not used in some processes, but that this was adequately explained by a “lighter touch in recognition of the greater responsibility held by those providers” (p. 27). The Register Committee concurred with the panel that the alternative approach used is effective and robust in the light of the process' objectives.The Register Committee noted that QQI has finalised its external quality assurance processes and moved to full implementation of most processes since the last review.The external review report, however, noted that for independent private providers “no cyclical institutional reviews have taken place as a result of the delay in approving those providers’ Quality Assurance Procedures through Re-engagement” (p. 28). While the report cited a combination of reasons for that and underlined that it was not the result of poor intentions on the part of the agency, the report noted that some providers may actually go up to 12 years without an institutional review. The panel further noted that the “risk of concerns about quality going unnoticed in these providers” was partly, but not wholly, mitigated by QQI having more intensive engagement with them through theirprogramme validation relationship (p. 28).In light of the incomplete implementation of reviews for independent private providers the Register Committee was unable to concur with the panel's conclusion of compliance, but considered that QQI only partially complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.4 Peer-review experts
Keywords	Panels composition
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“In its 2016 decision on QQI's Substantive Change Report, the Register Committee flagged for attention the composition of QQI expert panels.The Register Committee noted that some specific QQI processes do not use traditional expert panels, but are based on desk assessments or dialogues by QQI staff, followed by subsequent decisions where applicable. The Committee noted that the decision-making bodies include all perspectives that are otherwise required to be on a panel.The Committee further noted that wherever panels are deployed their composition complies with the standard. The Register Committee therefore concluded that the flag was addressed and concurred with the panel's conclusion that QQI complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.7 Complaints and appeals
Keywords	Scope of the appeals system
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“The Register Committee noted the panel's remark that the outcomes of cyclical institutional reviews can currently not be appealed, while noting thata provider “may still wish to challenge the findings in a way that would be better suited to an appeals process rather than a complaints process” (p. 33).The Register Committee has generally understood the standard to require that all results of external quality assurance processes be open to appeal, hence including reports that do not include “categorical decisions”.The Committee therefore underlined that QQI should consider widening the scope of its appeal system in monitoring the fitness for purpose of the current arrangements, per the panel's remarks. ”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.4 Thematic analysis
Keywords	Producing effective thematic analysis reports
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“In its decision of 2015 to admit QQI to the Register, the Register Committee flagged for attention the production of thematic analyses by QQI.The Register Committee understood from the panel's report that QQI has been proactive and effective in producing a range of thematic analysis reports, which are considered useful in the sector for improving quality and quality assurance, even though not having completed full cycles in all external quality assurance activities.The Register Committee therefore considered that the flag has been addressed and concurred with the panel's conclusion that QQI complies withthe standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.6 Internal quality assurance and professional conduct
Keywords	Efectiveness of internal QA
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“The development of QQI's internal quality assurance system was flagged for attention when QQI was admitted to the Register in 2015.The Register Committee noted that the panel described QQI's internal quality assurance instruments as effective. They cover all its external quality assurance processes and thus respond to the recommendation made in the last external review of QQI.While the panel noted that some future adjustments might be necessary once further external QA processes are rolled out, the Register Committee considered that the flag has clearly been addressed.The Register Committee therefore concurred with the panel's conclusion that QQI complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	2.6 Reporting
Keywords	Publication of full reports for one procedure
Panel conclusion	Full compliance
Clarification request(s)	Panel (14/02/2020)
RC decision	Compliance

“In its previous decision, the Register Committee flagged ACSUG’s publication of full reports of the FIDES-AUDIT reviews. According to the analysis of the review panel, all completed reports by ACSUG are published. The Register Committee could further verify that the results of the FIDES-AUDIT review are made available on the agency’s web page. The review panel’s analysis also revealed that the reports resulting from the ex-ante accreditation (validation) assessment are not published by ACSUG, as deferred applications for initial accreditation are usually withdrawn by the higher education institution before a final decision is taken. The Register Committee has therefore asked the review panel to clarify whether ACSUG has ever reached a final decision of rejection in an ex-ante accreditation assessment and whether in such a scenario, ACSUG would publish its report with a negative decision? In its response letter, the panel clarified that ACSUG has not reached a final decision of rejection in an ex-ante accreditation (validation) assessment at that time. The panel added that it has no reason to doubt that ACSUG would publish its ex-ante reports with a negative decision, in case that such reports were completed. Having considered the clarification by the panel, and the reassurance that ACSUG would publish any negative reports resulting from its ex-ante accreditation, the Register Committee could follow the conclusion of the panel that ACSUG complies with ESG 2.6”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	2.7 Complaints and appeals
Keywords	Lack of an independent appeals committee
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that Galician Committee for Reports, Assessment, Certification and Accreditation (CGIACA) is both the body responsible for evaluation, certification and accreditation and also the body responsible for the appeals following these reviews. In order to ensure a fair decision-making, the panel recommends the establishment of an independent committee to review appeals. In its Additional information to the external review report (p. 4) the agency stated that following the CGIACA’s meeting in late autumn, the agency will consider the possible nomination of candidates for an independent appeals board. The Register Committee welcomed the ACSUG’s intention to set up a separate board to handle appeals, but the Committee underlined that such changes are yet implemented and that the nomination of its members and the revised appeal process is yet to be reviewed. The Register Committee was therefore unable to concur with the review panel’s judgment of (substantial) compliance, and concluded that ACSUG complies only partially with ESG 2.7.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	3.3 Independence
Keywords	Organisational independence: appointments in the majority of the agency’s made by the Galician government
Panel conclusion	Full compliance
Clarification request(s)	Panel (14/02/2020)
RC decision	Partial compliance

“The Register Committee noted that the president of the Governing Board of ACSUG, the director of ACSUG and the president of the CGIACA (following the nomination by the Governing Board) are all members appointed by the Galician Government. In view of the panel’s recommendation to ACSUG i.e. to set procedures in the selection of the director of the agency, president of the Governing Board and president of the CGIACA the Register Committee asked the panel to clarify whether the appointments of these bodies follow a set of criteria and if the panel has reviewed these criteria. The panel responded that it has only considered the criteria and tasks set in the Statutes of 2018, based on professional competence and prior experience, and that these criteria are intended to be used in the future appointment processes. While the Register Committee welcomed the appointment criteria based on professional competence and prior experience, the Committee underlined that the regional Galician Government nevertheless remains directly involved in the appointment of the majority of members in the agency’s governing body, in the appointment of president of the Governing Body, the president of the agency’s decision making body (CGIACA), the director of the agency and the president of the Advisory Council. The Register Committee welcomed ACSUG’s intention of appointing a representative from outside Galicia in the Governing Board and as part of CGIACA’s but noted that the current addition does not change the agencies reliance on the regional Spanish government. While the involvement of the government (as one of its stakeholders) is encouraged, the current organisational arrangements does not guarantee clear and sufficient safeguards to prevent the possible interference of the regional Galician government in ACSUG’s governance. Considering the close interlinkage between ACSUG and the Galician government, the Committee could not follow the panel’s judgment of (full) compliance, and concluded that ACSUG complies only partially with ESG 3.3.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	3.4 Thematic analysis
Keywords	Lack of ressources for thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The panel’s findings show that ACSUG engages in broader forms of analysis, however these studies do not fully answer the demand for thematic analysis as defined in the ESG. In its Additional information to the external review report, the agency stated that the development of thematic analysis is going to become its priority following the hiring of new staff. ACSUG intends to launch external working groups to develop thematic analysis, which are to be coordinated by the new staff. In addition ACSUG also is considering to develop thematic analysis through their involvement in REACU (the Spanish Network of quality assurance agencies). The Register Committee welcomed ACSUG’s plans of addressing its shortcomings in preparing thematic analysis, but as it currently stands ACSUG does not fully meet the requirement of the standard i.e. no studies have been undertaken yet by its new working group. The Register Committee therefore concurred with the review panel that ACSUG complies only partially with ESG 3.4.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.1 Consideration of internal quality assurance
Keywords	Part 1 reflected in evalag’s criteria
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“When evalag’s registration was last renewed, it was flagged for attention whether all standards of Part 1 of the ESG are consistently addressed in evalag’s accreditations and evaluations.The Register Committee noted the panel’s comprehensive analysis, based on evalag’s SAR and mapping tables, of how Part 1 of the ESG is reflected in the agency’s different sets of criteria. The Committee concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.2 Designing methodologies fit for purpose
Keywords	Involvement of students in developing the international accreditation and evaluation processes.
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that the processes and criteria for evalag’s work outside of the German accreditation system are designed by the Foundation Board. Based on the analysis by the review panel the Committee understood that students have little involvement in designing evalag’s own processes and are also not involved in the Foundation Board (see p. 30).The involvement of students in developing the international accreditation and evaluation processes was a recommendation from the previous ESG review of evalag in 2014, but has not yet been addressed.While the Register Committee welcomed evalag’s commitment expressed in its statement on the review report, no changes have been made as yet. The Committee was therefore unable to concur with the panel’s conclusion of substantial compliance, but considered that evalag only partially complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.3 Implementing processes
Keywords	Implementation of follow-up procedures; accreditations carried out without a site visit
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“For accreditation in Germany, the Register Committee underlined that evalag retains responsibility for follow-up to take place, even if GAC makes the accreditation decisions under the new legal framework. This does not exclude that GAC actually implements the follow-up processes, as long as evalag has assured itself that this indeed happens.Given the small number of accreditations under the new legal framework thus far, it was not possible to analyse the actual practice at this point. The Register Committee therefore noted that this is a matter for further attention in future reviews of evalag.While the German legal framework potentially allows for an accreditation procedure to be carried out without a site visit, the panel understood from evalag that the agency did not plan to make use of that option. The Register Committee underlined that it might be helpful if evalag would point that out in its official documentation.Notwithstanding the above remarks, the Register Committee concurred with the panel's conclusion that evalag complies with the standard”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	Publication of negative reports
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“When evalag’s registration was last renewed it was flagged for attention whether evalag has moved to publish reports where the accreditation decision was negative. The review panel observed that evalag has had no negative decisions so far. The panel nevertheless confirmed that evalag’s clear policy is to publish all reports regardless of the outcome; the panel had no doubts that a negative report will be published. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.7 Complaints and appeals
Keywords	Independence of the appeals committee
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“In the previous decision to renew evalag’s registration the Register Committee flagged for attention how evalag has followed up the recommendation to separate the bodies in charge of appeals from the bodies deciding on accreditation, and to rule out parallel memberships.The review panel confirmed the independence of the appeals committee from the evalag decision-making body for accreditation. The panel further reported that parallel memberships are now ruled out. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Involvement of student in evalag’s governing structure, and overarching strategy
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The review panel found that evalag had no clear overarching strategy, bringing together planning, budgeting and risk assessment. The review panel noted stakeholders are formally involved in evalag’s governance through the Foundation Board, while the Board does not include a student member. Stakeholders are further involved in evalag’s work through participation in workshops and similar activities. The panel recommended that a student be appointed to the Foundation Board.Considering the concerns re. the lack of an overarching strategy and that one important stakeholder group is not represented in evalag’s governing structure, the Register Committee was unable to concur with the panel’s conclusion of substantial compliance, but considered that evalag only partially complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.4 Thematic analysis
Keywords	Producing effective thematic analysis reports
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“When evalag’s registration was last renewed, the Register Committee noted that the current review should address whether evalag produced analyses of the general findings of all reviews carried out by the agency. The review panel described clearly how evalag conducts thematic analyses based on the findings from its reviews. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.4 Peer-review experts
Keywords	Involvement of students in panels.
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“The Register Committee noted that the agency has a wide range of experts involved in institutional, departmental and programme evaluations and that higher education institutions can make reasoned objections concerning the composition of the Expert Evaluation Committees (EEC - panel of experts conducting external evaluation).According to the analysis of the review panel, there is room for a more substantial role of students in evaluations, which is currently limited to a few issues (review report p. 41).While the panel found the arrangement for the appointment of EEC transparent, the panel also underlined that CYQAA should publish the procedures and criteria for the selection of all categories of experts and to explicitly define and communicate the criteria for the selection of student experts.In its additional representation the agency stated that it has published on its website the procedures and criteria for the selection of all categories of experts, including students. The agency further explained that the involvement of student experts is not restricted by the law or limited by CYQAA, and that the agency endeavours to support students’ participation in evaluation processes. The agency further included details about a recent training workshop organised for student experts in preparation for an external review. CYQAA added that the review panel extract referring to the participation of students on limited matters referred to a few students and that the review panel concluded that ‘all groups of experts feel that they participate on an equal footing’ (review report p. 41).14.The agency further argued that students are equal to the rest of the panel members, that they follow the same terms of reference, have the same responsibilities (defined in the “Guidelines for the Membersof External Evaluation Committees”), receive the same information, directions and clarifications and have the same rights and obligationsas the other panel members.15.Having considering the clarification provided by the agency, the Register Committee was able to follow the panel’s conclusion of (substantial) compliance with ESG 2.4.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	Publication and quality of reports and decisions
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“At the time of the review not all evaluation reports carried out by CYQAA were available on the agency’s website. In its statement to thereview report the agency explained that it has begun to upload the missing reports. The Register Committee confirmed at the time of its first consideration of CYQAA’s application (June 2019) that most reports have been published (including reports with a negative outcome), however the Register Committee observed that a number of reports were nevertheless still missing.The Register Committee further noted that the decision taken on cross-border higher education provisions are not published by CYQAA, as the agency only provides an explanatory note related to the outcome of these reviews. While the panel found this practice satisfactory, the Committee could not follow the reasoning of the panel and underlined that the standard requirements specify that any formal decision based on the reports, should be published together with the report.The analysis of the panel further showed that the quality of reports varies in terms of evidence provided, depth of analysis and consistency and that expert panels have a very short schedule to write evaluation reports.In its additional representation CYQAA stated that, as it has previously committed to publishing all reports, that the agency has proceeded in doing so. The agency has further provided direct links to the different sections of its website where the reports of accredited institutions, programmes, joint programmes, franchised programmes are published, including the accreditations resulting ina negative decision. The CYQAA added that it has also published the official decisions and reports for all the inbound ‘franchised programmes’ reviewed by theagency. The Register Committee took note of the explanation concerning the time allocated to drafting of the report and welcomed the agency’s development of a more detailed guideline to support the consistencyand quality of reports (according to CYQAA additional representation and statement to the review report). As the Committee could verify the link for all published reports and decisions by CYQAA, including the evaluation outcomes of cross-border higher education provision of ‘franchised programmes’, the Register Committee concluded that CYQAA now complies with the ESG 2.6.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.7 Complaints and appeals
Keywords	Limited scope of complaints procedure and implementation of appeals and complaints procedures still to be assessed
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“While the panel confirmed that CYQAA has in place formal and clearly defined processes for higher education institutions to appeal against its accreditation decision, the panel also pointed out that the process of considering appeals is not independent as it is being handled by the Council of CYQAA. The Register Committee understood that following the amendments to the law, CYQAA would appoint a three-member ad-hoc committee to handle appeals on a case by case basis. In addressing complaints, the analysis of the panel shows that while CYQAA has an established practice for handling complaints there is a ‘tendency to rely mainly on informal communication, and that the arrangements in place do not add up yet to a clearly defined and formal’ procedure. In its additional representation CYQAA stated that its revised appeals’procedure sets out the appointment of a three member Advisory Committee of Experts (ACE) to examine the appeals that have been approved by the Council of CYQAA. Additionally CYQAA has written a formal regulation concerning the withdrawal of the accreditation of an institution, department or program of study offered by the institution. The appeal in these cases is being handled by an Independent ad hoc Advisory Committees. In addressing complaints CYQAA stated that the agency has published its complaints policy on its website and explained that individuals and organisations may issue complains about an accredited institution, department or programme. The Register Committee however noted that the concept of complaints is limited to general issues concerning higher education institutions and that it does not allow the possibility to address complaints related to the conduct of a review or complaints concerning the agency’s own processes. While the Register Committee welcomed the steps taken by the agency to address its shortcomings related to the functioning of the Appeals Committee and the implementation of the Complaints Procedure, the Committee underlined the limited scope of the complaints procedure and that the implementation and functioning of these procedures are yet to be considered by a review panel. Considering these limitations, the Register Committee could not follow the panels judgment and therefore concluded that CYQAA complies only partially with ESG 2.7.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.3 Independence
Keywords	CYQAA’s link to the ministry of education
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“In its analysis the panel noted that the agency’s Council members are appointed by the Council of Ministers upon recommendation from the Minister of Education and Culture (MOEC) who consults the Rectors’ Conference and relevant professional bodies (except the student member, who is appointed by POFEN).In terms of organisational arrangements, the Ministry is further involved in the recruitment process of the agency’s staff and in CYQAA’s financial services. The panel noted that “MOEC provides what the agency needs, as evidenced by a big budget increase and new staff, and the agency itself is proposing amendments on fees to the law which would further increase its income. While the panel noted that the provision of the quality assurance law and the strict rules for Council members reduces the risk of the Ministry’s interference with the agency's independence, the panel also stated that the current arrangements could be further revised, especially in reconsidering CYQAA’s organisational ties with the Ministry of Education and Culture. The Register Committee underlined the recommendations of the panel to ensure a clear separation of CYQAA from MOEC in its staff recruitment process, infrastructure and management of its own finances. In its additional representation the agency presented its proposed legislation changes and argued that this would safeguard the agency’s independence and the Council’s autonomy. Considering the proposed changes the Register Committee was not convinced that the changes in the appointment of CYQAA’s Council members would result in an increase of independence from the Ministry as, according to the proposed changes most of the Council members (eight out of eleven), are proposed (following stakeholder consultation in some cases) by the Minister. The Register Committee took note of the provision regarding the operational independence of the agency and welcomed the steps taken to increase its financial independence i.e. including a budget provisions that will allow the agency to hire additional staff, if the agency intends to do so. While the Committee noted some progress towards safeguarding the independence and autonomy of the agency, the Committee underlined the existing close interlinkage between CYQAA and MOEC and therefore concluded that the agency complies only partially with ESG 3.3.”

Full decision: see agency register entry