Quality Assurance and Learning Management Systems. Partner: IDP European Consultants.

At the end of this module you will be able to:

Comprehensive Understanding of EQAVET

Gain a deep understanding of the EQAVET Governance, Framework and Cycle

Effective Assessment Strategies

Develop the ability to plan, implement, and evaluate assessment strategies that align with the EQAVET principles for quality assurance (QA)

Evidence Analysis and Improvement

Get the knowledge to gather, analyse and interpret evidence of learning outcomes

Feedback, Planning and Action

Learn how to manage input and feedback for planning and enhancement, and implement actionable plans for learning management

Unit 1: Assessment Strategies

1.1. EQAVET: European Quality Assurance in Vocational Education and Training

1.2 The EQAVET Framework: Introduction and Indicators

1.3 The EQAVET Cycle: Planning, Implementation, Evaluation, Review

1.4 A Dual-Application Dimension: System and Provider Levels

1.5 Focus on the Provider Level’s Indicative Descriptors

1.6 Types of Assessment in VET: Formative, Summative and Diagnostic

1.7 Designing Effective Assessment Strategies

Unit 2: Analysing Evidences

2.1 Collecting Evidence of Learning Outcomes

2.2 Understanding the Role of Data Analytics

2.3 Qualitative and Quantitative Data Analysis

2.4 Focus on Identifying Areas for Improvement

Unit 3: Feedback and Planning

3.1 Effective Feedback Mechanisms

3.2 Three Levels of Feedback

3.3 Using Feedback to Inform Planning

3.4 Applying EQAVET Cycle in the Process

1.1 EQAVET: European Quality Assurance in Vocational Education and Training

The EQAVET initiative represents a collaborative network that brings out the European cooperation between Member States, Social Partners, and the European Commission aimed at improving and advancing quality assurance (QA) in Vocational Education and Training (VET).

EQAVET actively promotes the adoption and implementation of the European Quality Assurance Reference Framework for VET (EQAVET).

The Governance Structure

1.2 The EQAVET Framework: Introduction and Indicators

The EQAVET Framework is the EU standardised framework deployed to support and promote the QA mechanism and the improvement cycle in VET provision. It consists of a set of indicators and indicative descriptors.

During the EQAVET implementation and within the framework, ten (10) key reference indicators are available to help assess and improve the quality of VET.

EQAVET Indicators

  1. Relevance of quality assurance systems for VET providers
  2. Investment in training of teachers and trainers
  3. Participation rate in VET programmes
  4. Completion rate in VET programmes
  5. Placement rate of graduates from VET programmes
  6. Utilisation of acquired skills at the workplace
  7. Unemployment rate in the country
  8. Prevalence of vulnerable groups
  9. Mechanisms to identify training needs in the labour market
  10. Schemes used to promote better access to VET and provide guidance to (potential) VET learners

1.3 The EQAVET Cycle: Planning, Implementation, Evaluation, Review 

Inspired by the DEMING Cycle (Plan-Do-Check-Act), the EQAVET Framework is a four-step process aimed at supporting continuous improvement. This process follows the EQAVET Cycle.

1.4 A Dual-Application Dimension: System and Provider Levels 

The EQAVET Framework is applied in both VET systems and VET providers to strengthen the QA of:

  • Different learning environments (e.g., school/work-based learning, formal/informal/non-formal provisions)
  • Diverse learning contexts, including digital, face-to-face, and blended modes

The dual-application dimension stems from the use of the same set of indicators at both the system and provider levels, with the specifics and distinctions being provided by various and different indicative descriptors.

SYSTEM LEVEL

The EQAVET Indicative Descriptors are valuable tools for Member States in assessing the effectiveness of their QA systems and measuring strategies and progress.

Each phase of the quality phase has its own indicative descriptors (full list here) aiming at the long-term improvement of the whole VET ecosystem.

PROVIDER LEVEL

The framework provides indicative descriptors to help VET teachers/educators assess their QA strategies and measure achieved quality progress in the provision.

Within each phase of the cycle, there are distinct descriptors (full list here) for VET professionals.

1.5 Focus on the Provider Level’s Indicative Descriptors

1.5 Focus on the Provider Level’s Indicative Descriptors

1.6 Types of Assessment in VET: Formative, Summative and Diagnostic

In VET, there are various and different types of assessment for quality assurance and learning management. Let’s explore them.

FORMATIVE

  • Description: Formative assessment or evaluation is an ongoing and dynamic process aimed at assessing and providing feedback on learners’ level of understanding and performance during their training.
  • Use in VET: Formative assessment is placed to ensure the improvement of learning outcomes. It is used by trainers to identify learners’ strengths and weaknesses in real time, with the ultimate aim of possibly modifying methods and curriculum.
  • Tools: Classroom discussions, multiple-choice tests, assignments.

SUMMATIVE

  • Description: Summative assessment is a culminative assessment, i.e. conducted at the end of the training delivery. It measures the learners’ level of acquisition of knowledge and skills.
  • Use in VET: Summative assessments in VET are carried out to evaluate whether learners have achieved the required results and at what level. The result of the assessment reveals whether students are ready for certification or advancement.
  • Tools: Final examinations, certification assessments, and capstone projects.

DIAGNOSTIC

  • Description: Diagnostic assessment is used as a preliminary to a training course. It serves to identify the knowledge, skills, and learning needs of learners.
  • Use in VET: Diagnostic assessments help VET trainers adapt training methods and curriculum to the assessed needs. The goal is to minimise repetition and efficiently correct skill gaps while starting learners at the right level of teaching.
  • Tools: Pre-assessments, skills and knowledge inventories, and initial interviews.

1.7 Designing Effective Assessment Strategies (1)

Based on the acquisition of the diverse types of assessment in VET, let’s move on to the exploration of effective assessment strategies:

1. Alignment with EQAVET Principles:

  • Why Alignment is Fundamental? Assessments have to align with EQAVET principles in order to ensure they evaluate the VET-relevant competences and learning outcomes
  • Guarantee of Validity: Create tools that specifically reflect the nuances listed in the EQAVET framework

2. Priority on Reliability:

  • Reliable Assessments: Assessments should regularly give similar results when submitted to the same learners
  • Minimising Bias: Reduce bias to guarantee accuracy and impartiality

3. Multiple Assessment Models:

  • Diverse Approaches: Implement various assessment models for an holistic view of learner performance
  • Examples: Combine written tests with practical demonstrations and projects for a comprehensive assessment

4. Clear Guidelines and Criteria:

  • Transparency: Make sure learners have free access to clear rubrics and instructions outlining the evaluation criteria
  • Feedback and Improvement: In keeping with the EQAVET principles, this transparency makes it easier to gather feedback that leads to ongoing improvement

5. Periodic Review and Adaptation

  • Dynamism: The assessment strategies and tools must evolve and adapt to meet changing educational and industrial new needs and dynamics
  • Continuous Improvement: Regularly review, update and adapt assessment strategies and tools to maintain relevance and effectiveness

2.1 Collecting Evidence of Learning Outcomes

Collecting effective evidence is a key issue of QA. The following are practical methods and approaches for analysing and collecting evidence to assess and evaluate the effectiveness of VET programmes:

ASSESSMENTS

Use various assessment strategies and tools, including quizzes, tests and practical assessments, to measure students’ acquired knowledge and skills.

OBSERVATIONS

In order to evaluate practical skills and the application of knowledge, conduct systematic and ongoing observations in practical contexts.

PORTFOLIO ASSESSMENTS

Guide learners in compiling portfolios for tangible evidence of their work and projects, results and progress.

Data Collection in Evidence Analysis

Comprehensive data collection is the basis for making well-informed decisions on the effectiveness of VET programmes. It enables learners to adapt methods and curriculum to the learners’ needs by enabling continuous programme improvement and a holistic view of learner results and stats.

2.2 Understanding the Role of Data Analytics 

How data analytics contributes to the assessment of learning outcomes?

  • Correspondence Budget – Outcome: Data analysis allows the comparison of assessment results with desired learning outcomes to assess whether outcomes have been achieved.
  • Trend Detection: Analysis of data and statistics allows educators to detect trends and thus patterns in learner performance. This results in the emergence of areas of strength and areas of weakness, which require improvement.
  • Informed Decision-Making: Data analysis indirectly influences and promotes educational decision-making by providing institutions with data on curriculum and instructional improvements on which to base their interventions.
  • Ongoing Improvement: Regular and continuous data analysis enables VET institutions to undertake evidence-based continuous improvement processes, with the ultimate aim of adapting methods and programmes, e.g. to EQAVET standards – in our case.

2.3 Qualitative and Quantitative Data Analysis

QUALITATIVE

Qualitative data analysis is about examining non-numerical data, such as text, open-ended responses, narratives or observations.

  • Techniques and Application in VET: It comprises techniques such as content, thematic and comparative analysis. They are carried out to ascertain uncover aspects of learner experiences, perceptions, and behaviours such as themes, patterns, and insights.
  • Example: Thematic analyser software is one tool for qualitative data analysis. Trainers can use this tool as a process to analyse feedback from learners on surveys or tests. Trainers or VET providers in general can use this software to find recurrent words, phrases and themes in feedback, which serves as evidence of things like difficulties encountered during practical training – for example. This data can be used to adapt the methods to deliver programmes and the curriculum itself.

QUANTITATIVE

Quantitative data analysis is about dealing with numerical data and stats.

  • Techniques and Application in VET: Statistical techniques such as regression analysis, inferential statistics and descriptive statistics such as mean and standard deviation are used to analyse and interpret data. Through statistical testing, quantitative analysis offers numerical and statistical information and it allows VET learners to measure learning outcomes, performance trends and programme effectiveness.
  • Example: A platform for online learning can be analysed and assessed using quantitative data analysis. Trainers can differentiate between learners’ results based on whether or not they use the platform by gathering numerical data on their performance in modules completed offline and in modules completed online. This analysis identifies whether the use of e-learning tool improves trainers’ learning outcomes.

2.4 Focus on Identifying Areas for Improvement (1) 

By identifying areas for improvement, evidence and data analysis seeks to continuously improve the quality and management of training and learning. In other words, it is about tending towards targeted curriculum adjustments.

Here is a list of effective strategies for the identification of areas to improve VET programmes:

LEARNER FEEDBACK

  • Gather Feedback: Gather feedback from learners’ contributions through forms, surveys, focus groups, evaluations;
  • Analyse Input: Analyse feedback and comments to discover recurring words, phrases and themes. The aim is to bring out any challenges and areas of dissatisfaction;
  • Tool: Trainers can submit online surveys via platforms such as Google Forms.

ASSESSMENT DATA AND INSIGHTS

  • Analyse Evidence: Examine assessment formative and summative outcomes to identify areas where learners are performing poorly;
  • Trend Detection: Identify recurring dynamics and trends, such as particular topics or skills where learners struggle;
  • Tools: Resources can include spreadsheet software, data analytics, and learning management systems (LMS).

OBSERVATION AND PRACTICAL EVALUATIONS

  • Observe Learners: Conduct classroom or workplace practical observations to assess learners’ performance and evidence;
  • Assess Performance: Use checklists to practically and objectively evaluate and cross-check learners’ performance;
  • Tools: Templates for observations and standardised checklist.

INPUT BY STAKEHOLDERS (STKHs)

  • Stakeholders Engagements: Consult with and involve STKHs like teachers, employers, and specialists in the field to gain insights;
  • Gather Perspectives: Gather STKHs’ contributions on the topics and areas that are ‘under observation’;
  • Tools: Primary research (interviews and focus groups), advisory boards.

BENCHMARKING

  • Benchmark Against Standards: Compare VET programme outcomes against industry standards or EQAVET indicators;
  • Recognise Variances: Point out any areas where your programme falls short or shines;
  • Tools: Databases and reports.

DATA ANALYTICS SOFTWARE

  • Utilise Tools: Use data analytics software to collect and analyse datasets and stats;
  • Identify Trends: The aim remains that of identify trends, correlations, and anomalies in datasets;
  • Tools: Data visualisation tools (e.g., Power BI) and statistical software (e.g., R).

REVIEW COMMITTEES

  • Establish Committees: Form committees for evaluating programme;
  • Suggest Changes: Committee members should suggest changes for improvement based on data and proof;
  • Tools: Committee meetings and related reports.

3.1 Effective Feedback Mechanisms: Examples and Best Practices (1)

Of the previous effective strategies, let’s focus on examples and best practices.

Learners Feedback: A music school conducted quarterly student surveys to gather feedback. In a survey, 73% of students expressed the desire to carry out practical lessons in a more interactive way. In response, the school started an initiative in which students are both contestants and judges. This change improved the interactivity, with the students not limiting themselves to their performance but playing an interactive and protagonist role for the entire lesson.

Peer Assessment: Taking the example above, contest sessions not only improved the interactivity of the lessons, but also contributed to direct comparison between students. This initiative incorporated peer assessment into the assessment of practical skills and encouraged students to pay close attention to each partner’s best practices and different styles. The evaluation component promoted teamwork and professionalism among students.

Instructor Observations: A professional training institution regularly implements instructor observations as part of the training evaluation process. During a cybersecurity course for corporate devices, instructors observed particular gaps in basic IT skills, providing immediate feedback to the director of the institute after each lesson. It was decided to organise free reset workshops to recall the necessary basic IT skills to proceed as best as possible with the new course.

Digital Learning Analytics: A vocational institution’s learning management system now includes a cutting-edge learning analytics tool. For instance, this tool identified a significant number of students who had problems with a particular maths module. In response, the institution offered improved instructional materials for that module, additional tutoring sessions, and targeted online resources.

Review Committee: An industrial VET centre formed a review committee on the programmes composed of instructors, professionals and entrepreneurs in the sector and students. The committee examined the dataset and the feedback from the various assessment strategies already implemented, identifying the need for advanced training on robotic machinery, which leads to tailor-made curriculum updates. Higher employability rates are the result of the new graduates being better prepared for the workforce.

Stakeholder Engagement: As part of the sustainable tourism course, a technical institute established an advisory committee made up of industry experts and entrepreneurs who regularly reviewed the curriculum for the new role of sustainability manager for tourism. They provided feedback on the relevance of the course content, the skills and the protocols required. As a result, the programme produced graduates with skills that directly met the needs of local and national employers, addressing a need for new roles for businesses and leading to higher job placement rates.

3.2 Three Levels of Feedback

Three levels of feedback are present in VET to guarantee a feedback-driven approach to QA improvement. Peer assessment encourages peer collaboration, instructor observations offer individualised guidance with a top-down approach, and learner feedback offers holistic insights from the bottom up.

LEARNER FEEDBACK

  • Description: Learner feedback represents the most direct level of insights. Additionally, surveys give students a forum to express their opinions, preferences, and levels of satisfaction;
  • Particularity: Learner surveys put emphasis on gathering direct impressions with the training, environment and materials. They provide a comprehensive view of the educational experience for the learner

PEER ASSESSMENT

  • Description: Peer assessment involves the active involvement of learners in a collaborative effort between peers. In other words, learners analyse and provide feedback and comments on the work of their peers;
  • Particularity: Peer assessment focuses on analysing and evaluating tasks, activities, project and works completed by other learners. It enhances collaborative learning, critical thinking, self-evaluation, and peer-to-peer feedback.

INSTRUCTOR OBSERVATIONS

  • Description: Instructor observations provide direct and top-down information on student engagement and training progress. Instructors observe students during training activities and note their observations in real time, reporting at the end of the session;
  • Particularity: The information collected by the instructor is based on direct and real-time observation of the learners’ actions during the training sessions. These offer immediate feedback and guidance.

3.3 Using Feedback to Inform Planning (1)

Effectively incorporating feedback means not only gathering information, but also turning it into planning for continuous quality improvement. The following illustrates how feedback can be used to inform planning:

1. Data Analysis and Synthesis of Outputs and Insights:

  • Before planning, fully analyse datasets, feedback and comments from diverse sources (3-level feedback);
  • Extract key outputs and insights. Identify strengths, weaknesses and areas for improvement.

2. Prioritising Areas for Improvement:

  • Not all insights require immediate action. After identifying areas for improvement, prioritise them based on their positive impact on learners’ needs, learning outcomes and alignment with general objectives and goals of the programme.

3. Setting Specific Objectives:

  • Set precise, specific, and measurable objectives for enhancement. What do you aim to achieve based on the feedback received?
  • Consequently, anticipate results of interventions by setting expected results / outcomes with related indication of indicators of positive impact.

4. Actionable Plans and Actions:

  • Design and develop actionable plans with specific strategies, actions and tasks to address identified areas for improvement;
  • When planning, consult relevant stakeholders, e.g., instructors and learners.

5. Resource Allocation:

  • Provide the resources required to support the activities for improvement. These latter include initiatives such as more training materials, technological advancements, or faculty development.

6. Timing and Monitoring:

  • Impost deadlines for specific interventions and assign related tasks and responsibilities;
  • Design monitoring mechanisms to track and check progresses and be ready to contrast any challenges along the way with adjustments.

7. Ongoing Feedback:

  • Feed a continuous feedback loop by regularly and periodically gathering feedback on the effectiveness of implemented activities and improvements;
  • If necessary, fine-tune strategies and plans using feedback to ensure adaptive and continuous improvement.

8. Alignment with EQAVET Principles:

  • In our case: Make sure that improvement strategies and plans promote a culture of quality assurance in VET by being in line with the EQAVET Framework and its principles.

3.4 Applying EQAVET Cycle in the Process

The continuous quality improvement process of learning management and learner outcomes takes place within the different phases of the EQAVET cycle. As a result, quality assurance can start at any stage, including:

  • For new procedures: Beginning with the planning stage is the strongest foundation for quality;
  • When adapting existing procedures: Starting with the implementation of existing processes, followed by evaluation, is often better to assess and adapt current practices themselves;
  • For well-established processes: The process often begins with an evaluation of the effectiveness of the current arrangements;
  • When exploring new policies or approaches: The first stage frequently involves reviewing current arrangements in order to explore new policies or approaches.

Summing up

Summing up

Welcome to your Quality Assurance and Learning Management Systems

Name
What is the primary goal of EQAVET (European Quality Assurance in Vocational Education and Training)?

The EQAVET Framework is inspired by which well-known cycle?

What does qualitative data analysis primarily involve?

In the three levels of feedback – learner feedback, peer assessment, and instructor observations – which level provides insights from direct observations in real-time?

When using feedback to inform planning, what should be a primary consideration for prioritising areas of improvement?

Download Content:
.docx File
.pptx File

The European Commission’s support for the production of this publication does not constitute anendorsement of the contents, which reflect the views only of the authors, and the Commission cannotbe held responsible for any use which may be made of the information contained therein.

Legal description – Creative Commons licensing: The materials published on the DigComp4VET project website are classified as Open Educational Resources’ (OER) and can be freely (without permission of their creators): downloaded, used, reused, copied, adapted, and shared by users, with information about the source of their origin.

en_USEnglish
ro_RORomanian de_DEGerman it_ITItalian pl_PLPolish es_ESSpanish en_USEnglish