Database of Precedents

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	15/03/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Clear separation of reviews and consultancy carried out for higher education institutions.
Panel conclusion	Full compliance
Clarification request(s)	Agency (31/01/2019) Panel (05/11/2018)
RC decision	Partial compliance

“The Register Committee sought clarification from the panel regarding the separation of reviews and consultancy to higher education institutions, including both services rendered by QAA itself (internationally) or through its subsidiary QAAE (domestically). The Committee took note of the various measures and policies the panel referred to. The Committee also noted the clarification that “no evidence was found which would bring any doubt about the proper separation between [of] consulting units and undertaking reviews”. The Register Committee considered QAA's activities in Albania, which QAA described as a capacity-building activity that involved the support of the Albanian Ministry of Education Sport and the Albanian national QA agency (initially PAAHE, later APAAL, now ASCAL) in (1) developing review materials, (2) training reviewers and (3) conducting the reviews of all Albanian higher education institutions. […] The Register Committee thus considered how QAA ensured a clear distinction of such work from its external quality assurance activities (see EQAR Policy on the Use and Interpretation of the ESG, standard 3.1 and Annex 5). In doing so, the Committee considered that also an uninformed reader needs to be able to distinguish the reviews in Albania from review reports (co-)owned by QAA, and thus not falling within the scope of QAA's registration on EQAR and not officially certified as in line with the ESG. While the final paragraph of the section “About this review” indeed states that QAA “provided expert support” in producing the report, the initial paragraph states that “Institutional Review is a peer-review process with each review team composed of a mix of UK reviewers appointed by QAA and Albanian reviewers appointed by the Albanian Public Accreditation Agency for Higher Education (APAAL)”. Considering the entire text, the Register Committee remained in doubt whether readers (at least lay readers) are able to clearly recognise the substantially different roles of QAA and ASCAL in the process, with QAA being “just” a consultant and ASCAL having full and sole ownership of and responsibility for the report.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.2 Designing methodologies fit for purpose
Keywords	clarity and consistency of standards, decision-making procedure
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee shared the view that NEAQA’s standards would benefit from a revision to ensure full clarity, consistency and readability, in particular in what concerns the employment of site-visits in programme accreditation procedures. The Register Committee concurred with the panel's analysis as regards the arrangement whereby NEAQA’s members combine their decision-making function with that of external experts and with the recommendation to separate these functions. In its additional representation the agency stated that the inherited regulations, procedures and standards of CAQA were revised considering the panel’s concerns and the new by-laws. According to NEAQA particular attention was devoted to ensuring the broader inclusion of external reviewers and the separation of review experts from other member and staff positions in the organisation.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.3 Implementing processes
Keywords	consistent implementation of a follow-up procedure & site visits
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The review panel’s analysis showed that the agency has made improvements, having also introduced follow-up procedure for its audits. The Register Committee found that while the panel was satisfied with this improvement, the approach to follow-up should allow higher education institutions to also report progress in the implementation of recommendations before all external review procedures. The Register Committee further noted that site-visits are not consistently carried out by NEAQA for programme accreditation.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.4 Peer-review experts
Keywords	student involvement in some reviews and involvement limited to student matters; anonymous review team experts, site-visit undertaken by an agency sub-commission instead of the group of exprts which prepared the report
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The panel’s findings show that students (and employer representatives) take part in periodic programme accreditations only when they are combined with institutional ones. The panel also noted that the involvement of students is limited to student matters.The Committee further noted that at the time of the application by CAQA review team experts remained anonymous and that their involvement was limited to assessing compliance based on documentation. The site visits were undertaken by a CAQA sub-commission, which subsequently prepared the final report integrating the experts' feedback.The Register Committee considered that this arrangement did not meet the requirement of the standard and that assessments should be undertaken by a “group of external experts”. ”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.6 Reporting
Keywords	Publication of initial accreditation reports, involvement of experts in preparing the reports, no fact-check verification process
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The panel noted that CAQA does not publish initial accreditation reports of higher education institutions and their programmes since another accreditation review in these cases is carried out after one year. While the panel considered this approach to some extent justifiable, the Register Committee underlined that, according to the standard, reports by the experts should be published in full. This applies to all types of reports, whether part of an initial or follow-up process. The review panel also found that the current reporting arrangements limit substantially the input coming from academic experts, students and labour market representative. The findings of the panel also showed that NEAQA’s reports are not sent to the institutions concerned for a factual check before the final version is published.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.7 Complaints and appeals
Keywords	Absence of a formal complaints procedure; no possibility to appeal with the body that carried out the procedure.
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that in the case of refusal of accreditation, the higher education institution concerned may lodge an appeal to the NCHE who is the responsible body for deciding on the appeals on the agency's decisions. The Register Committee further noted that NEAQA does not have in place any procedures for complaints. While the review panel found that representatives of the evaluated higher education were fully familiar with the appeals arrangements, the panel recommended that appeals should be nevertheless considered by a separate body within the structure of an agency. The panel further recommended the establishment of a complaints procedure.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Agency lacks explicit goals and objectives; limited stakeholder involvement i.e. consultations only where specific topics are addressed
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that while NEAQA’s strategy could provide a comprehensive framework for the agency's daily operations, the agency does not have in place mechanisms for effective forward planning and reviewing progress towards its objectives. Considering the involvement of stakeholders, the panel’s findings show that, since its last review, NEAQA has improved the engagement with employer representatives in its quality assurance activities. However, stakeholder involvement is still limited in agency's work given that neither students nor employers are involved in NEAQA’s governance.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	3.3 Independence
Keywords	Organisation and operational independence (lack of evidence due to change in legal framework)
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee had flagged the independence of NEAQA’s predecessor and the relationship between the agency and NCHE, in particular with regard to NCHE's double role of appointing members and being involved in the development of procedures and criteria as well as serving as the appeals body. The panel had also noted that CAQA was expected to become fully independent in terms of organisational and financial arrangements when incorporated into the new QA body, NEAQA, pursuant to the changed Law on Higher Education (LoHE) (review report, p. 19).The Committee nevertheless considered that the new organisational and financial arrangements still have to be fully implemented, and would need to be verified by an external panel.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	3.5 Resources
Keywords	financial sustainability
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“[...]The panel, however, added that NEAQA’s predecessor lacked human resources for other tasks which would enhance the agency’s performance. Considering the financial management of the agency, i.e. a bank account managed by the Ministry, the panel found that the planning, management and operational efficiency of the agency were limited due to its dependence on the Ministry’s administrative and financial services. The agency stated in its additional representation that it now has its own accounts and, while it continues to receive financial support from the Republic of Serbia for its initial activities, the agency’s further actions will be financed exclusively from its own funds.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	3.6 Internal quality assurance and professional conduct
Keywords	Implementation of processes for IQA
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that while CAQA/NEAQA has a number of mechanisms to ensure internal quality [...] that the agency lacks formal mechanisms to act upon the external and internal feedback on a regular basis and thus foster continuous improvement. In its additional representation NEAQA stated its plans to improve communication among all stakeholders in quality assurance processes; to introduce software to support its information management processes and ensure transparency in the work of the agency.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	04/04/2019
Standard	2.3 Implementing processes
Keywords	Consistency in implementation of EQA procedures
Panel conclusion	Full compliance
Clarification request(s)	Panel (26/02/2021)
RC decision	Compliance

“In its response letter the panel stated that it is the responsibility of each Permanent Speciality Commission to ensure that judgements in ARACIS reports are accurate and consistent and the panel was convinced that all processes defined in the Methodology are implemented consistently.

Having found limited information in the panel’s analysis on the functioning of provisional authorisation for programmes and higher education institutions, the Committee has asked the panel to confirm that the key features of ESG 2.3 (self-assessment, external assessment, site visit, review report, follow-up) are implemented by ARACIS in these reviews.

In its response letter the panel confirmed that following the detailed examination of ARACIS’s Methodology and Guide, the agency followed the same procedures for provisional authorisation as the ones employed for the evaluation of accredited programmes, which were addressed by the panel in its review report (p. 31).Having considered the panel’s clarification the Register Committee was able to concur with the panel’s conclusion that ARACIS complies with the standard.

The Committee nevertheless underlined the panel’s suggestion on the need to further develop the follow-up procedures of the agency and to consider how institutions have addressed the ARACIS’s recommendations in their evaluation reports.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	04/04/2019
Standard	2.4 Peer-review experts
Keywords	student involvement & training of experts
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“While the panel confirmed the practice of involving students as review experts in all ARACIS’s reviews, including at programme level, the panel added that at the time of its review the agency had had little practical experience. The panel further commented that “some evaluation panels treated student members as supernumeraries and not as full partner-evaluators in the review exercise and that some evaluators were not attuned to the necessity for ‘student centeredness’ in higher education and what that required of tutors and of them as evaluators.” (Review Report, p. 35)

In its additional representation ARACIS [...] commented that its policies and procedures gave students equal rights and obligations i.e. the coverage of mission costs and remuneration is the same as for the other members, and that students are required to participate in the Council, the Permanent Speciality Commission and expert panels. (p. 8-9) ARACIS further added that one of the aspects considered in the selection process of the Permanent Speciality Commission in 2018 was the understanding of students’ role and involvement in quality assurance activities.

The Register Committee noted that, while ARACIS regularly offers training sessions for its evaluators, the panel found that the format for training sessions offered few opportunities for learning through simulation and practical exercise and that the ‘virtual’ and ‘online’ training were lacking in efficacy. In the additional representation, ARACIS clarified that the e-platform is a complementary facility to the face to face training sessions and not a substitute. [..] The agency provided a list of past training sessions focused on the practical activity of the Permanent Speciality Commissions.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	04/04/2019
Standard	2.6 Reporting
Keywords	experts involvement in drafting the report
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“In its previous decision to renew registration the Register Committee flagged for attention ARACIS’s involvement of expert panels in drafting and agreeing upon the review report for institutional evaluations.

To address the flagged issues the panel noted that ARACIS decided to prepare a comprehensive synthetic report in which each of the expert panel members have a contribution. The panel also commented that separate reports from students and international evaluators are still a feature of the agency’s reporting arrangements.

While the Register Committee welcomed the panel's recommendations on improving the accessibility, storage, organisation and presentation of review decisions and reports on its website, it considered that ARACIS technically fulfilled the requirements of the standard by publishing the reports on the web. The Register Committee however noted that the full accessibility of reports remains an issue as long as some reports are published in an archived or scanned format.

The Register Committee concluded that ARACIS largely addressed the flag and was therefore unable to concur with the review panel’s judgement of partial compliance, but concluded that ARACIS complies with standard 2.6.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	04/04/2019
Standard	2.7 Complaints and appeals
Keywords	Concerns on the accesibility of the appeals procedure and issues of transparency in handling appeals
Panel conclusion	Full compliance
Clarification request(s)	Panel (26/02/2021)
RC decision	Partial compliance

“The Register Committee noted that ARACIS does not have a permanent appeals body but with each new case an ad hoc commission is established by the Executive Board to deal with. The agency explained that the small number of appeals and complaints it has received to date did not justify moving to more permanent arrangements of handling complaints and appeals.

The Register Committee found that the involvement of the Executive Board of the Council in establishing app [...].

The documentation provided by the agency in its additional representation shows that the nomination of Appeals Commission members follows ARACIS’ operational procedure ‘Solutions of appeals of higher education institutions’.

The agency explained that its (ad-hoc) Appeals Commission includes one member of the consultative commission (i.e. academics, former Council or Commissions members), external evaluators from the National Register of Evaluators and one inspector from the Quality Assurance Direction (technical department of ARACIS). Over the past four years the Appeals Commissions received 46 appeals, about 1.93% of the total of number of external evaluations performed by ARACIS.

ARACIS argued that the Executive Board is delegated to manage the activity of the agency and therefore is involved in the selection process of Appeals Commission members. The selection process follows strictly defined criteria that would not allow the Executive Board’s interference with the decision making on the appeal.

While the Register Committee welcomed the clarification on the appeals process and the publication of ‘ARACIS Solutions of appeals of higher education institutions’, the Committee noted that the members nominated to act in the Appeals Committee are not published and that the revised procedure is not easily accessible on the website.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	04/04/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	strategic plan, stakeholder involvement in the governance of the agency
Panel conclusion	Partial compliance
Clarification request(s)	Panel (26/02/2021)
RC decision	Compliance

“With regards to the agency’s vision mission and strategy, the panel stated that ARACIS “does not have a strategic plan that spans several years and does not appear to set targets for itself against which its Council and others can measure the Agency’s performance” (Review report p. 17).

Considering the involvement of stakeholders, the Register Committee noted that this is limited to representatives of students and employers sitting in ARACIS Council and Permanent Speciality Commissions. In the view of the panel, ARACIS tended to define the term ‘stakeholder’ narrowly, confining it to students, employers and trades union representatives and the panel was not confident that ARACIS understands the need to involve stakeholders more fully in the work of the Council and the work of ARACIS.

In its additional representation, ARACIS noted that it has followed up on the recommendations of the panel and it has developed a Strategic plan for the period 2018-2021 and the operational plans for 2018 and 2019, which were approved by the ARACIS Council. The Register Committee could verify the published plans.

The agency further provided a detailed overview on how representatives of stakeholders i.e. academics, representatives of students, representatives of employers and employers’ federation, representative of trade-unions, including one representative of ESU and ENQA.- participate in the council and different commissions of ARACIS.

ARACIS further explained that it has recently approved the inclusion of representatives of employers in the permanent Speciality Commissions, with priority the Commission of Engineering Sciences that awards the EUR-ACE label, and that the selection process for new members will be soon initiated.

The agency added that it has put forward a project application for “Stakeholder Engagement in QA” under a call of the Erasmus+ programme. The project aims to promote the diversification of stakeholders' involvement in quality assurance activities across EHEA and to provide the means for making the involvement of stakeholders effective.

The Register Committee welcomed the agency’s newly adopted strategic and operational work plans and was satisfied with the level of stakeholder involvement in the work and governance of the agency.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	04/04/2019
Standard	3.4 Thematic analysis
Keywords	thematic analysis conducted on a regular basis
Panel conclusion	Full compliance
Clarification request(s)	Panel (26/02/2021)
RC decision	Compliance

“While the panel found that none of the research projects described by ARACIS in its self assessment report met the requirement of the standard, the panel nevertheless formed the view that the agency is (fully) compliant with ESG 3.4, based on the Quality Barometer reports produced for 2009, 2010 and 2015.

The Register Committee could not conclude on the basis of the panel’s analysis whether the agency made use of the outcomes of these analysis and whether the Barometer reports are conducted on a regular basis. The Committee has therefore asked the panel for further clarifications.

In its response the panel stated that it was keen to see that ARACIS continues its commitment to the Quality Barometer series and reiterated its appreciation for the agency's dedication in preparing them, even though it comes at a substantial financial and management burden on the agency.

With a view to the application of Quality Barometers, the panel stated that they are used by ARACIS to engage with quality professionals, academics, students, and others throughout Romania and to disseminate aspects related to higher education to interested parties.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	04/04/2019
Standard	3.6 Internal quality assurance and professional conduct
Keywords	IQA processes are not fit for purpose
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The review report noted an improvement in the agency’s internal quality assurance process [...] but stressed the need for ARACIS’s internal QA to be improved so as to support the work of its speciality commissions, responsible for the consistency check of evaluation reports and for preparing the Council’s decisions. The panel found that the members of Permanent Speciality Commissions do not have access to the searchable digital copies of evaluation reports, and that they are provided with a pile of reports the day before the meeting, thus being prevented in making their own judgements on the findings of individual reports.

While the panel stated that ARACIS newly introduced comprehensive IQA procedures and new staff member will provide the agency with a sound basis for reviewing and improving the effectiveness with which it works, the Register Committee found this has not been implemented at the time of the review, in particular in supporting the internal activity of its speciality commissions.

ARACIS explained in its additional representation that the procedure to fill in the positions for the Internal public audit department has been delayed due to a temporary staff hiring interdiction in the public sector. As the interdiction has been lifted the agency started to fill in these positions.

The agency further stated that the Permanent Speciality Commissions are supported in their work by the experts and speciality inspectors for accreditation and quality assurance (permanent staff of the agency) providing all the logistics and necessary material. After the site visit each member of the Permanent Speciality Commission receives by e-mail, for analysis, the documents drafted by the visit panel. The panel coordinator of the site visit also presents the results in front of the Permanent Specialty Commission who takes the final decision.[…]. The Committee also noted that the agency has made little progress in making its reports machine-readable (p. 5) since its last review..”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	19/06/2019
Standard	2.6 Reporting
Keywords	no guarantee that all reports are published
Panel conclusion	Substantial compliance
Clarification request(s)	Panel (17/05/2019)
RC decision	Partial compliance

“The Register Committee noted that the reports and the formal accreditation decisions based on the assessment of degree programmes are published only on the website of NVAO; NQA does not publish these reports but provides a list of reviewed institutions. In its additional representation the agency stated that it has initiated a consultation with NVAO considering the possibility to create an online search-option linking NVAO’s database of degree assessment programme reports and decisions to the website of NQA. The agency added that its international assessments are also published by NVAO and that a web-link has been already added in the NQA website to the NVAO database. Concerning its publication policy, the agency reasserted that the owner of reports is the higher education institutions and it is up to the institution to use the assessment report for an accreditation by NVAO. If the institution decides to hold on to the report, no decision on accreditation is taken and therefore the report will not be published. While the Register Committee welcomed NQA’s intention to integrate a searchable database on its website, the Committee underlined that the agency itself bears the responsibility to follow the ESG and therefore needs to ensure (e.g. contractually) that it is in a position to publish all reports of its external quality assurance procedures.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	19/06/2019
Standard	2.7 Complaints and appeals
Keywords	Informal complaints only, appeals only with the decision-making body
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The review panel explained that NQA has no formal role in the appeal procedure on assessment of degree programmes as this is handled according to the Dutch accreditation system by NVAO. With regards to complaints, the panel noted that there is no formally defined procedure, rather, complaints are addressed by the agency in an informal way. The Register Committee underlined the panel’s recommendation of developing a complaints procedure that may handle any relevant issues in a formal manner. The Committee further underlined that the agency is also expected to have an appeals procedure in place. While appeals are handled by NVAO in case of assessments of degree programmes, NQA bears the responsibility for the report and its conclusion; higher education institution should thus be allowed to appeal (where the case) such results.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	19/06/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Publication of the mission or goals of the agency, stakeholder involvement in the governance of the agency (incl students) and separation of consulting and assessment activities
Panel conclusion	Partial compliance
Clarification request(s)	Panel (17/05/2019)
RC decision	Partial compliance

“n its additional representation the agency explained that it would be unwise from a commercial point of view, as a private organisation, to publish its strategic goals. The agency stated that it has chosen a more dynamic course, so that NQA is able to adopt swiftly if/when radical changes such as the introduction of institutional accreditations in the system occur. While the Register Committee understood that publishing explicit strategic goals might not be the most favourable option for NQA, that the agency could nevertheless publicly communicate (in broader terms) its mission statement along with its goals and objectives. The agency added that its yearly management review includes strategies based on its mission statement and goals. In its additional representation NQA stated that the agency meets with a diverse group of university employees (e.g. CEO’s, quality assurance officers, management of degree programmes and teachers), the NVAO, the Inspectorate and other quality assurance agencies. NQA emphasised that the results of such meetings are translated into the work of the agency, i.e. in its strategic goals, management and review processes. The Register Committee welcomed such consultations, but underscored the ESG understanding of stakeholders, which cover all actors within an institution and therefore also students. The Committee thus underlined the panel’s recommendation to ensure stakeholder involvement by establishing regular interactions with all relevant groups of interest. Concerning the separation between NQA’s external QA activities the panel reported that one of the measures adopted by NQA was to ensure that staff members involved in assessment activities do not provide consultancy to the same institution. In most of the examples provided to the panel, the consultancy projects were completed by the Director. This was regarded by the panel as a potential problem in ensuring a clear distinction between NQA’s consultancy and external quality assurance activities. From the review panel’s analysis the Committee noted that “NQA attempts to separate as much as possible quality assurance activities from consultancy services” and that “throughout the interviews, there was no feeling of a lack of independence”. The Committee further noted that while the panel concluded the agency is “substantially compliant” re. independence, the panel found it necessary to recommend the establishment of stricter internal procedures in order to further separate consulting and assessment activities. In particular, the panel suggested to avoid performing consultancy activities to institutions NQA reviews, at least during a certain time span. This would be in line with Annex 5 of the Use and Interpretation of the ESG.”

Full decision: see agency register entry