Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework

İsmail Demir

doi:10.30703/cije.1758405

EN TR

Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework

Abstract

This study aims to examine the criticisms and development suggestions expressed in the academic literature regarding digital assessment and evaluation applications, and the application orientations around which these applications are structured. The research was conducted using a mixed-content analysis approach that combines qualitative thematic coding with quantitative descriptive indicators. The dataset was created from peer-reviewed academic publications on digital assessment tools. Studies were searched in Scopus, Web of Science, ERIC, and Google Scholar databases using the keywords digital assessment, e-exam, e-portfolio, and learning analytics assessment, and selected according to predetermined inclusion criteria. In the analysis process, open, axial, and selective coding stages were applied; the obtained themes were classified within categories based on the pedagogical goals of digital assessment applications, the types of technology used, assessment strategies, and ethical-psychometric principles. The coding process was carried out by two researchers, and inter-coder agreement was checked with comparative evaluation. Qualitative thematic results were supported by frequency and trend analyses. The findings show that data security and access inequality issues are prominent in the literature, and that digital assessment applications largely focus on low and medium-level cognitive skills. Furthermore, the findings reveal that digital assessment processes are structured according to different application orientations based on pedagogical, ethical, and technological dimensions.

Keywords

Kavramsal Bir Çerçeve Bağlamında Dijital Ölçme Uygulamalarının Pedagojik, Etik ve Psikometrik Yönelimleri

Abstract

Bu çalışma, dijital ölçme ve değerlendirme uygulamalarına ilişkin akademik literatürde dile getirilen eleştirileri, gelişim önerilerini ve bu uygulamaların hangi uygulama yönelimleri etrafında yapılandığını incelemeyi amaçlamaktadır. Araştırma, nitel tematik kodlama ile nicel betimsel göstergeleri bir arada kullanan karma içerik analizi yaklaşımıyla yürütülmüştür. Veri seti, dijital değerlendirme araçlarını konu alan hakemli akademik yayınlardan oluşturulmuştur. Çalışmalar Scopus, Web of Science, ERIC ve Google Scholar veri tabanlarında digital assessment, e-exam, e-portfolio ve learning analytics assessment anahtar kavramları kullanılarak taranmış ve önceden belirlenen dâhil edilme ölçütlerine göre seçilmiştir. Analiz sürecinde açık, eksenel ve seçici kodlama aşamaları uygulanmış; elde edilen temalar dijital ölçme uygulamalarının pedagojik hedefleri, kullanılan teknoloji türleri, değerlendirme stratejileri ve etik-psikometrik ilkeleri temel alan kategoriler çerçevesinde sınıflandırılmıştır. Kodlama süreci iki araştırmacı tarafından yürütülmüş ve kodlayıcılar arası uyum karşılaştırmalı değerlendirme ile kontrol edilmiştir. Nitel tematik sonuçlar frekans ve eğilim analizleriyle desteklenmiştir. Bulgular, literatürde veri güvenliği ve erişim eşitsizliği konularının öne çıktığını ve dijital ölçme uygulamalarının büyük ölçüde düşük ve orta düzey bilişsel becerilere odaklandığını göstermektedir. Ayrıca, elde edilen bulgular dijital değerlendirme süreçlerinin pedagojik, etik ve teknolojik boyutlar temelinde farklı uygulama yönelimleri doğrultusunda yapılandığını ortaya koymaktadır.

Keywords

References

Andrade, H., & Du, Y. (2007). Student responses to criteria‐referenced self‐assessment. Assessment & Evaluation in Higher Education, 32(2), 159–181. https://doi.org/10.1080/02602930600801928
Barrett, H. C. (2007). Researching Electronic Portfolios and Learner Engagement: The REFLECT Initiative. Journal of Adolescent & Adult Literacy, 50(6), 436–449. https://doi.org/10.1598/JAAL.50.6.2
Bearman, M., Dawson, P., Ajjawi, R., Tai, J., & Boud, D. (2020). Re-imagining university assessment in a digital world. Assessment & Evaluation in Higher Education, 45(5), 785–798.
Bennett, R. (2015). The changing nature of educational assessment. Review of Research in Education.
Bennett, R., Dawson, P., Bearman, M., Molloy, E., & Boud, D. (2017). How technology shapes assessment design.
Biggs, J. B., & Tang, C. S. (with Society for Research into Higher Education). (2011). Teaching for quality learning at university: What the student does (4th edition). McGraw-Hill/Society for Research into Higher Education/Open University Press.
Biggs, J. B., Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy (structure of the observed learning outcome). Academic Press.
Bozkurt, A., & Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. https://doi.org/10.5281/ZENODO.3778083

Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676X.2019.1628806
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (Third edition). Sage.
Darling-Hammond, L., Zielezinski, M. B., & Goldman, S. (2020). Using technology to support at-risk students’ learning. Alliance for Excellent Education.
De Ayala, R. J. (2022). The theory and practice of item response theory (Second edition). The Guilford Press.
Deardorff, D. K. (2011). Assessing intercultural competence. New Directions for Institutional Research, 2011(149), 65–79. https://doi.org/10.1002/ir.381
Ferguson, R. (2019). Ethical challenges for learning analytics. Journal of Learning Analytics, 6(3), 25–30.
Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351. https://doi.org/10.1016/j.compedu.2011.06.004
Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and Validating Test Items (0 edn). Routledge. https://doi.org/10.4324/9780203850381
Henderson, M., Selwyn, N., & Aston, R. (2017). What works and why? Student perceptions of ‘useful’ digital technology in university teaching and learning. Studies in Higher Education, 42(8), 1567–1579. https://doi.org/10.1080/03075079.2015.1007946
Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial Intelligence in Education.
Ifenthaler, D., Mah, D.-K., & Yau, J. Y.-K. (2019). Utilising Learning Analytics for Study Success: Reflections on Current Empirical Findings. In D. Ifenthaler, D.-K. Mah, & J. Y.-K. Yau (Eds), Utilizing Learning Analytics to Support Study Success (pp. 27–36). Springer International Publishing. https://doi.org/10.1007/978-3-319-64792-0_2
JISC. (2020). Learning analytics code of practice. JISC.
Kizilcec, R. F., & Lee, H. (2022). Algorithmic fairness in education. In W. Holmes & K. Porayska-Pomsta, The Ethics of Artificial Intelligence in Education (1st edn, pp. 174–202). Routledge. https://doi.org/10.4324/9780429329067-10
Laurillard, D. (2012). Teaching as a design science: Building pedagogical patterns for learning and technology. Routledge.
Looney, J. (2011). Integrating Formative and Summative Assessment: Progress Toward a Seamless System? (OECD Education Working Papers No. 58; OECD Education Working Papers, Vol. 58). https://doi.org/10.1787/5kghx3kbl734-en
Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. (2016). Intelligence Unleashed: An Argument for AI in Education. Pearson.
McKenney, S., & Reeves, T. C. (2019). Conducting educational design research (Second edition). Routledge, Taylor & Francis Group. https://doi.org/10.4324/9781315105642
Miles, M. B., Huberman, A. M., & Saldaña, J. (2020). Qualitative data analysis: A methods sourcebook (Fourth edition). SAGE.
Nicol, D. (2021). The power of internal feedback: Exploiting natural comparison processes. Assessment & Evaluation in Higher Education, 46(5), 756–778.
Penetrating the Fog: Analytics in Learning and Education. (n.d.). EDUCAUSE Review. Retrieved 4 August 2025, from https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education
Prinsloo, P., & Slade, S. (2017). An elephant in the learning analytics room: Ethics and student privacy. Journal of Learning Analytics, 4(2), 24–29.
Redecker, C. (2017). European framework for digitally competent educators.
Redecker, C., & Johannessen, Ø. (2013). Changing assessment – towards a new assessment paradigm using ICT. European Journal of Education.
Selwyn, N. (2019). Should Robots Replace Teachers? AI and the Future of Education.
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
Tekin, H. (2019). Eğitimde ölçme ve değerlendirme (10. baskı). Yargı Yayınevi.
Tondeur, J., Scherer, R., Baran, E., Siddiq, F., Valtonen, T., & Sointu, E. (2019). Teacher educators as gatekeepers: Preparing the next generation of teachers for technology integration in education. British Journal of Educational Technology, 50(3), 1189–1209. https://doi.org/10.1111/bjet.12748
University of Southampton, UK, Alruwais, N., Wills, G., & Wald, M. (2018). Advantages and Challenges of Using e-Assessment. International Journal of Information and Education Technology, 8(1), 34–37. https://doi.org/10.18178/ijiet.2018.8.1.1008
Weiss, D. J., & Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21(4), 361–375. https://doi.org/10.1111/j.1745-3984.1984.tb01040.x
Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14. https://doi.org/10.1016/j.stueduc.2011.03.001
Wiliam, D. (2018). Embedded formative assessment (Second edition). Solution Tree Press.
Williamson, B. (2017). Big Data in Education.
Williamson, B., & Piattoeva, N. (2019). Objectivity as standardization in data-scientific education policy, technology and governance. Learning, Media and Technology, 44(1), 64–76. https://doi.org/10.1080/17439884.2018.1556215

Details

Primary Language

English

Subjects

Measurement and Evaluation in Education (Other)

Journal Section

Research Article

Authors

İsmail Demir ^*
0000-0001-9756-1901
Türkiye

Publication Date

May 15, 2026

Submission Date

August 4, 2025

Acceptance Date

April 24, 2026

Published in Issue

Year 2026 Volume: 15 Number: 1

DOI

https://doi.org/10.30703/cije.1758405

IZ

https://izlik.org/JA35EC85BP

Cite

RIS / Bibtex

APA

Demir, İ. (2026). Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework. Cumhuriyet Uluslararası Eğitim Dergisi, 15(1), 313-337. https://doi.org/10.30703/cije.1758405

AMA

1.Demir İ. Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework. CIJE. 2026;15(1):313-337. doi:10.30703/cije.1758405

Chicago

Demir, İsmail. 2026. “Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications Within a Conceptual Framework”. Cumhuriyet Uluslararası Eğitim Dergisi 15 (1): 313-37. https://doi.org/10.30703/cije.1758405.

EndNote

Demir İ (May 1, 2026) Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework. Cumhuriyet Uluslararası Eğitim Dergisi 15 1 313–337.

IEEE

[1]İ. Demir, “Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework”, CIJE, vol. 15, no. 1, pp. 313–337, May 2026, doi: 10.30703/cije.1758405.

ISNAD

Demir, İsmail. “Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications Within a Conceptual Framework”. Cumhuriyet Uluslararası Eğitim Dergisi 15/1 (May 1, 2026): 313-337. https://doi.org/10.30703/cije.1758405.

JAMA

1.Demir İ. Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework. CIJE. 2026;15:313–337.

MLA

Demir, İsmail. “Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications Within a Conceptual Framework”. Cumhuriyet Uluslararası Eğitim Dergisi, vol. 15, no. 1, May 2026, pp. 313-37, doi:10.30703/cije.1758405.

Vancouver

1.İsmail Demir. Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework. CIJE. 2026 May 1;15(1):313-37. doi:10.30703/cije.1758405

Pedagogical, Ethical, and Psychometric Orientations of Digital Assessment Applications within a Conceptual Framework

Abstract

Keywords

Kavramsal Bir Çerçeve Bağlamında Dijital Ölçme Uygulamalarının Pedagojik, Etik ve Psikometrik Yönelimleri

Abstract

Keywords

References

Details

Primary Language

Subjects

Journal Section

Authors

Publication Date

Submission Date

Acceptance Date

Published in Issue

DOI

IZ

Cite

© Cumhuriyet University, Faculty of Education