Database of Precedents

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	2.6 Reporting
Keywords	Publication of full reports for one procedure
Panel conclusion	Full compliance
Clarification request(s)	Panel (14/02/2020)
RC decision	Compliance

“In its previous decision, the Register Committee flagged ACSUG’s publication of full reports of the FIDES-AUDIT reviews. According to the analysis of the review panel, all completed reports by ACSUG are published. The Register Committee could further verify that the results of the FIDES-AUDIT review are made available on the agency’s web page. The review panel’s analysis also revealed that the reports resulting from the ex-ante accreditation (validation) assessment are not published by ACSUG, as deferred applications for initial accreditation are usually withdrawn by the higher education institution before a final decision is taken. The Register Committee has therefore asked the review panel to clarify whether ACSUG has ever reached a final decision of rejection in an ex-ante accreditation assessment and whether in such a scenario, ACSUG would publish its report with a negative decision? In its response letter, the panel clarified that ACSUG has not reached a final decision of rejection in an ex-ante accreditation (validation) assessment at that time. The panel added that it has no reason to doubt that ACSUG would publish its ex-ante reports with a negative decision, in case that such reports were completed. Having considered the clarification by the panel, and the reassurance that ACSUG would publish any negative reports resulting from its ex-ante accreditation, the Register Committee could follow the conclusion of the panel that ACSUG complies with ESG 2.6”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	2.7 Complaints and appeals
Keywords	Lack of an independent appeals committee
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that Galician Committee for Reports, Assessment, Certification and Accreditation (CGIACA) is both the body responsible for evaluation, certification and accreditation and also the body responsible for the appeals following these reviews. In order to ensure a fair decision-making, the panel recommends the establishment of an independent committee to review appeals. In its Additional information to the external review report (p. 4) the agency stated that following the CGIACA’s meeting in late autumn, the agency will consider the possible nomination of candidates for an independent appeals board. The Register Committee welcomed the ACSUG’s intention to set up a separate board to handle appeals, but the Committee underlined that such changes are yet implemented and that the nomination of its members and the revised appeal process is yet to be reviewed. The Register Committee was therefore unable to concur with the review panel’s judgment of (substantial) compliance, and concluded that ACSUG complies only partially with ESG 2.7.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	3.3 Independence
Keywords	Organisational independence: appointments in the majority of the agency’s made by the Galician government
Panel conclusion	Full compliance
Clarification request(s)	Panel (14/02/2020)
RC decision	Partial compliance

“The Register Committee noted that the president of the Governing Board of ACSUG, the director of ACSUG and the president of the CGIACA (following the nomination by the Governing Board) are all members appointed by the Galician Government. In view of the panel’s recommendation to ACSUG i.e. to set procedures in the selection of the director of the agency, president of the Governing Board and president of the CGIACA the Register Committee asked the panel to clarify whether the appointments of these bodies follow a set of criteria and if the panel has reviewed these criteria. The panel responded that it has only considered the criteria and tasks set in the Statutes of 2018, based on professional competence and prior experience, and that these criteria are intended to be used in the future appointment processes. While the Register Committee welcomed the appointment criteria based on professional competence and prior experience, the Committee underlined that the regional Galician Government nevertheless remains directly involved in the appointment of the majority of members in the agency’s governing body, in the appointment of president of the Governing Body, the president of the agency’s decision making body (CGIACA), the director of the agency and the president of the Advisory Council. The Register Committee welcomed ACSUG’s intention of appointing a representative from outside Galicia in the Governing Board and as part of CGIACA’s but noted that the current addition does not change the agencies reliance on the regional Spanish government. While the involvement of the government (as one of its stakeholders) is encouraged, the current organisational arrangements does not guarantee clear and sufficient safeguards to prevent the possible interference of the regional Galician government in ACSUG’s governance. Considering the close interlinkage between ACSUG and the Galician government, the Committee could not follow the panel’s judgment of (full) compliance, and concluded that ACSUG complies only partially with ESG 3.3.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	3.4 Thematic analysis
Keywords	Lack of ressources for thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The panel’s findings show that ACSUG engages in broader forms of analysis, however these studies do not fully answer the demand for thematic analysis as defined in the ESG. In its Additional information to the external review report, the agency stated that the development of thematic analysis is going to become its priority following the hiring of new staff. ACSUG intends to launch external working groups to develop thematic analysis, which are to be coordinated by the new staff. In addition ACSUG also is considering to develop thematic analysis through their involvement in REACU (the Spanish Network of quality assurance agencies). The Register Committee welcomed ACSUG’s plans of addressing its shortcomings in preparing thematic analysis, but as it currently stands ACSUG does not fully meet the requirement of the standard i.e. no studies have been undertaken yet by its new working group. The Register Committee therefore concurred with the review panel that ACSUG complies only partially with ESG 3.4.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.1 Consideration of internal quality assurance
Keywords	Part 1 reflected in evalag’s criteria
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“When evalag’s registration was last renewed, it was flagged for attention whether all standards of Part 1 of the ESG are consistently addressed in evalag’s accreditations and evaluations.The Register Committee noted the panel’s comprehensive analysis, based on evalag’s SAR and mapping tables, of how Part 1 of the ESG is reflected in the agency’s different sets of criteria. The Committee concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.2 Designing methodologies fit for purpose
Keywords	Involvement of students in developing the international accreditation and evaluation processes.
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that the processes and criteria for evalag’s work outside of the German accreditation system are designed by the Foundation Board. Based on the analysis by the review panel the Committee understood that students have little involvement in designing evalag’s own processes and are also not involved in the Foundation Board (see p. 30).The involvement of students in developing the international accreditation and evaluation processes was a recommendation from the previous ESG review of evalag in 2014, but has not yet been addressed.While the Register Committee welcomed evalag’s commitment expressed in its statement on the review report, no changes have been made as yet. The Committee was therefore unable to concur with the panel’s conclusion of substantial compliance, but considered that evalag only partially complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.3 Implementing processes
Keywords	Implementation of follow-up procedures; accreditations carried out without a site visit
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“For accreditation in Germany, the Register Committee underlined that evalag retains responsibility for follow-up to take place, even if GAC makes the accreditation decisions under the new legal framework. This does not exclude that GAC actually implements the follow-up processes, as long as evalag has assured itself that this indeed happens.Given the small number of accreditations under the new legal framework thus far, it was not possible to analyse the actual practice at this point. The Register Committee therefore noted that this is a matter for further attention in future reviews of evalag.While the German legal framework potentially allows for an accreditation procedure to be carried out without a site visit, the panel understood from evalag that the agency did not plan to make use of that option. The Register Committee underlined that it might be helpful if evalag would point that out in its official documentation.Notwithstanding the above remarks, the Register Committee concurred with the panel's conclusion that evalag complies with the standard”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	Publication of negative reports
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“When evalag’s registration was last renewed it was flagged for attention whether evalag has moved to publish reports where the accreditation decision was negative. The review panel observed that evalag has had no negative decisions so far. The panel nevertheless confirmed that evalag’s clear policy is to publish all reports regardless of the outcome; the panel had no doubts that a negative report will be published. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.7 Complaints and appeals
Keywords	Independence of the appeals committee
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“In the previous decision to renew evalag’s registration the Register Committee flagged for attention how evalag has followed up the recommendation to separate the bodies in charge of appeals from the bodies deciding on accreditation, and to rule out parallel memberships.The review panel confirmed the independence of the appeals committee from the evalag decision-making body for accreditation. The panel further reported that parallel memberships are now ruled out. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Involvement of student in evalag’s governing structure, and overarching strategy
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The review panel found that evalag had no clear overarching strategy, bringing together planning, budgeting and risk assessment. The review panel noted stakeholders are formally involved in evalag’s governance through the Foundation Board, while the Board does not include a student member. Stakeholders are further involved in evalag’s work through participation in workshops and similar activities. The panel recommended that a student be appointed to the Foundation Board.Considering the concerns re. the lack of an overarching strategy and that one important stakeholder group is not represented in evalag’s governing structure, the Register Committee was unable to concur with the panel’s conclusion of substantial compliance, but considered that evalag only partially complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.4 Thematic analysis
Keywords	Producing effective thematic analysis reports
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“When evalag’s registration was last renewed, the Register Committee noted that the current review should address whether evalag produced analyses of the general findings of all reviews carried out by the agency. The review panel described clearly how evalag conducts thematic analyses based on the findings from its reviews. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.4 Peer-review experts
Keywords	Involvement of students in panels.
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“The Register Committee noted that the agency has a wide range of experts involved in institutional, departmental and programme evaluations and that higher education institutions can make reasoned objections concerning the composition of the Expert Evaluation Committees (EEC - panel of experts conducting external evaluation).According to the analysis of the review panel, there is room for a more substantial role of students in evaluations, which is currently limited to a few issues (review report p. 41).While the panel found the arrangement for the appointment of EEC transparent, the panel also underlined that CYQAA should publish the procedures and criteria for the selection of all categories of experts and to explicitly define and communicate the criteria for the selection of student experts.In its additional representation the agency stated that it has published on its website the procedures and criteria for the selection of all categories of experts, including students. The agency further explained that the involvement of student experts is not restricted by the law or limited by CYQAA, and that the agency endeavours to support students’ participation in evaluation processes. The agency further included details about a recent training workshop organised for student experts in preparation for an external review. CYQAA added that the review panel extract referring to the participation of students on limited matters referred to a few students and that the review panel concluded that ‘all groups of experts feel that they participate on an equal footing’ (review report p. 41).14.The agency further argued that students are equal to the rest of the panel members, that they follow the same terms of reference, have the same responsibilities (defined in the “Guidelines for the Membersof External Evaluation Committees”), receive the same information, directions and clarifications and have the same rights and obligationsas the other panel members.15.Having considering the clarification provided by the agency, the Register Committee was able to follow the panel’s conclusion of (substantial) compliance with ESG 2.4.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	Publication and quality of reports and decisions
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“At the time of the review not all evaluation reports carried out by CYQAA were available on the agency’s website. In its statement to thereview report the agency explained that it has begun to upload the missing reports. The Register Committee confirmed at the time of its first consideration of CYQAA’s application (June 2019) that most reports have been published (including reports with a negative outcome), however the Register Committee observed that a number of reports were nevertheless still missing.The Register Committee further noted that the decision taken on cross-border higher education provisions are not published by CYQAA, as the agency only provides an explanatory note related to the outcome of these reviews. While the panel found this practice satisfactory, the Committee could not follow the reasoning of the panel and underlined that the standard requirements specify that any formal decision based on the reports, should be published together with the report.The analysis of the panel further showed that the quality of reports varies in terms of evidence provided, depth of analysis and consistency and that expert panels have a very short schedule to write evaluation reports.In its additional representation CYQAA stated that, as it has previously committed to publishing all reports, that the agency has proceeded in doing so. The agency has further provided direct links to the different sections of its website where the reports of accredited institutions, programmes, joint programmes, franchised programmes are published, including the accreditations resulting ina negative decision. The CYQAA added that it has also published the official decisions and reports for all the inbound ‘franchised programmes’ reviewed by theagency. The Register Committee took note of the explanation concerning the time allocated to drafting of the report and welcomed the agency’s development of a more detailed guideline to support the consistencyand quality of reports (according to CYQAA additional representation and statement to the review report). As the Committee could verify the link for all published reports and decisions by CYQAA, including the evaluation outcomes of cross-border higher education provision of ‘franchised programmes’, the Register Committee concluded that CYQAA now complies with the ESG 2.6.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.7 Complaints and appeals
Keywords	Limited scope of complaints procedure and implementation of appeals and complaints procedures still to be assessed
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“While the panel confirmed that CYQAA has in place formal and clearly defined processes for higher education institutions to appeal against its accreditation decision, the panel also pointed out that the process of considering appeals is not independent as it is being handled by the Council of CYQAA. The Register Committee understood that following the amendments to the law, CYQAA would appoint a three-member ad-hoc committee to handle appeals on a case by case basis. In addressing complaints, the analysis of the panel shows that while CYQAA has an established practice for handling complaints there is a ‘tendency to rely mainly on informal communication, and that the arrangements in place do not add up yet to a clearly defined and formal’ procedure. In its additional representation CYQAA stated that its revised appeals’procedure sets out the appointment of a three member Advisory Committee of Experts (ACE) to examine the appeals that have been approved by the Council of CYQAA. Additionally CYQAA has written a formal regulation concerning the withdrawal of the accreditation of an institution, department or program of study offered by the institution. The appeal in these cases is being handled by an Independent ad hoc Advisory Committees. In addressing complaints CYQAA stated that the agency has published its complaints policy on its website and explained that individuals and organisations may issue complains about an accredited institution, department or programme. The Register Committee however noted that the concept of complaints is limited to general issues concerning higher education institutions and that it does not allow the possibility to address complaints related to the conduct of a review or complaints concerning the agency’s own processes. While the Register Committee welcomed the steps taken by the agency to address its shortcomings related to the functioning of the Appeals Committee and the implementation of the Complaints Procedure, the Committee underlined the limited scope of the complaints procedure and that the implementation and functioning of these procedures are yet to be considered by a review panel. Considering these limitations, the Register Committee could not follow the panels judgment and therefore concluded that CYQAA complies only partially with ESG 2.7.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.3 Independence
Keywords	CYQAA’s link to the ministry of education
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“In its analysis the panel noted that the agency’s Council members are appointed by the Council of Ministers upon recommendation from the Minister of Education and Culture (MOEC) who consults the Rectors’ Conference and relevant professional bodies (except the student member, who is appointed by POFEN).In terms of organisational arrangements, the Ministry is further involved in the recruitment process of the agency’s staff and in CYQAA’s financial services. The panel noted that “MOEC provides what the agency needs, as evidenced by a big budget increase and new staff, and the agency itself is proposing amendments on fees to the law which would further increase its income. While the panel noted that the provision of the quality assurance law and the strict rules for Council members reduces the risk of the Ministry’s interference with the agency's independence, the panel also stated that the current arrangements could be further revised, especially in reconsidering CYQAA’s organisational ties with the Ministry of Education and Culture. The Register Committee underlined the recommendations of the panel to ensure a clear separation of CYQAA from MOEC in its staff recruitment process, infrastructure and management of its own finances. In its additional representation the agency presented its proposed legislation changes and argued that this would safeguard the agency’s independence and the Council’s autonomy. Considering the proposed changes the Register Committee was not convinced that the changes in the appointment of CYQAA’s Council members would result in an increase of independence from the Ministry as, according to the proposed changes most of the Council members (eight out of eleven), are proposed (following stakeholder consultation in some cases) by the Minister. The Register Committee took note of the provision regarding the operational independence of the agency and welcomed the steps taken to increase its financial independence i.e. including a budget provisions that will allow the agency to hire additional staff, if the agency intends to do so. While the Committee noted some progress towards safeguarding the independence and autonomy of the agency, the Committee underlined the existing close interlinkage between CYQAA and MOEC and therefore concluded that the agency complies only partially with ESG 3.3.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.4 Thematic analysis
Keywords	Producing effective thematic analysis reports
Panel conclusion	Non-compliance
Clarification request(s)	–
RC decision	Compliance

“The review panel’s analysis show that CYQAA does not have a thorough and systematic analysis of findings from the evaluations conducted which could feed into a national quality assurance policy or guide quality improvement at higher education institutions. The panel nevertheless found that CYQAA’s Annual Report includes a few paragraphs referring to general issues and good practices in higher education in Cyprus. In its statement to the review report, the agency stated that it has collected the General Evaluation Reports from higher education institutions in Cyprus in November 2018 and has appointed a team of qualified members responsible for producing systematic analysis using the agency’s collected reports. The agency added that the findings of its thematic analysis are expected to be published within

2019. In its addition representation, the agency provided evidence of its implementation of thematic analysis and provided a link to its new publications. The agency has carried out an analysis of the “trends and dominant issues“ within i.e. e-learning programme evaluation, master programmes’ evaluation and the evaluation of short cycle programmes within colleges. The agency stated that it has made use of its thematic results, identifying weaknesses in aspects related to distance learning and recommending further attention to these issues. Considering the changes enacted by the agency, the Register Committee considered that CYQAA now complies with ESG 3.4.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.1 Consideration of internal quality assurance
Keywords	New standards published, but not reported to EQAR
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“The Register Committee noted that following the site-visit of its external review, CTI prepared and published (in February 2019) a new version of its standards and guidelines (R&O) for the accreditation of study programmes. While the revised R&O is expected to take into account the (minor) shortcomings identified by the panel in CTI’s coverage of ESG Part 1, these changes have not been included in the application to EQAR, nor reported via change report. The Register Committee therefore underlined that CTI is expected to report such substantial changes in its methodology immediately after they are adopted. CTI is thus expected to provide without delay a change report providing further information i.e. mapping of its new R&O against ESG Part 1.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.4 Peer-review experts
Keywords	Involvement of students in panels.
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“In its previous decision the Register Committee flagged CTI’s involvement of students in some of its review panels. The Register Committee noted that CTI works together with the French engineering students’ association, to ensures that CTI systematically nominates student experts in all its regular programme evaluations. The review panel also confirmed that in its CeQuInt evaluations, CTI’s panel include among its four experts also a student. ”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.5 Criteria for outcomes
Keywords	Lack of consistency in its decision making
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The panel’s analysis show that while the standards and criteria for accreditations are explicit and published, there are no clear deliberation rules detailing the basis upon which a specific decision is made and therefore concluded that consistency may not always be assured. In its response to the review report, CTI stated that it has taken a number of steps towards more consistency i.e. developing a new reporting template and updating its compliance table. The agency further declared that it will revise its rules for decision making for its different types of evaluation procedures following the analysis of its decision-making (after January 2020). The Register Committee welcomed the steps taken by the agency to address the shortcomings in ensuring consistency in its decision making but noted that the changes have not yet come into effect. The Register Committee therefore concurred with the panel’s conclusion that CTI complies only partially with ESG 2.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	Publication of full reports
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“In its previous decision the Register Committee flagged CTI’s practice ofnot publishing in full its review reports and not publishing its accreditation decisions. The panel’s analysis show that the agency’s practice it is to still publish summary evaluation reports. While the summary includes the decision, the main findings and recommendations of the review, the agency does not publish the full detailed evaluation results, called “minutes”. CTI argued thatit be difficult to ensure the consistency in the style and length of the differentpanel reports if it were to publish it in full, and that it would prove to be less valuable to its intended readership, including the reviewed higher education institutions which appreciate concise reports. In view of the panel’s recommendations of publishing the full accreditation reports, the agency stated that it has taken the decision that within the 2019-2020 external review campaign to make available the full reports for those higher education institutions that agree to having their report published (on a voluntary basis). CTI added that starting with its 2020-2021 external review campaign, that the agency is committed to publishing in full from that point all its review reports. The Register Committee acknowledged the actions taken by the agency towards the full publication of its reports, but stressed that the flag has not been addressed. As the agency does not currently meet the requirements of the standard (to publish full reports) at the Register Committee agrees with the panel’s conclusions that CTI complies only partially with ESG 2.6.”

Full decision: see agency register entry