The Digital Platform for the Assessment of Competences (D-PAC) is an innovative tool for assessing a wide range of competences, based on the method of Comparative Judgement. It can be used within a variety of contexts (e.g., for writing ability, self-reflection, and problem-solving) and for different purposes (e.g., for formative evaluation, peer assessment, research, selection, etc.).
Comparative Judgement is based on Thurstone’s Law of Comparative Judgement. This law stipulates that people are better and more reliable in comparing two things than in assigning an absolute score to (only) one thing.
The next video makes clear why Comparative Judgement is a good alternative for absolute scoring and how it is done.
Comparative Judgment in D-PAC: why and how.
In practice, assessees are invited to upload their work. According to the competence being assessed, that work may be an academic paper or an animated film or a mood board or a speech file, etc.. In a next step, multiple assessors (e.g. teachers or peers) compare those representations in pairs. For each random pair presented, they identify – in a holistic manner – which work is better. As such, each work is assessed multiple times. The tool also offers the opportunity for assessors to provide feedback on each work. The gathered data is analysed using the Bradley-Terry-Luce model and results in a reliable interval (logit-) rank-order, ranging from the worst representation to the work, being assessed as the best.
Credible tool for reliable and valid judgements
D-PAC has been developed and validated by University of Antwerp, Ghent University, and imec, in close collaboration with end-users in the domains of Education and Human Resources .The D-PAC project is set-up as an interdisciplinary research project that integrates scientific knowledge from multiple fields of research. A meta-analysis of the results shows that D-PAC ratings are relatively stable across assessors, resulting in rank-orders with an average reliability between .70 and .80. The validity of the results is warranted as the method includes judgements of multiple assessors who all have their own perspective of what a quality performance looks like. Students are satisfied with the feedback they received from D-PAC. In sum, D-PAC is a credible tool for the assessment of competences, supporting assessors to make reliable and valid judgements without markings.
Growing interest to set out D-PAC
Since its development in 2014, D-PAC has already been used by 49 user groups: national and international scholars as well as educators in Belgium, The Netherlands, Germany, Cyprus, and China.
Curious about the method of Comparative Judgement and the user-friendliness of our tool? Try our demo version.
Want to discuss possibilities for using D-PAC in your context? Get in touch with us.