Automatized visualization methods for the qualitative analysis of teachers' collegial discussions
##plugins.themes.bootstrap3.article.sidebar##
##plugins.themes.bootstrap3.article.main##
https://orcid.org/0000-0003-2282-8071
Odd Tore Kaufmann
https://orcid.org/0000-0002-6998-2714
Andreas Ryve
https://orcid.org/0000-0002-3329-0177
Abstract
In this paper, we discuss a set of procedures for the visualization of conversation data, based on machine learning and text mining algorithms. The methodological function of these visualization methods is to support (rather than substitute) humanistic and interpretive qualitative data analysis. The context of our methodological discussion is the study of teachers’ learning communities. Enabling collegial learning among teachers has been argued to play a pivotal role in both teachers’ professional development and students’ learning. The problem is that the interpretive study of collegial discussions among teachers, which are typically considered a fruitful context for the study of teachers' collegial learning, is both time-consuming and cognitively demanding. Several scholars have discussed the potential of including machine learning and text mining in the toolbox of qualitative data analysis as a way to analyze large qualitative data sets and solve the time demand problem. However, the literature discussing the use of machine learning and text mining for qualitative research typically conceives the algorithms as substitutes for the work of the researcher. These automatized approaches are introduced to categorize the data, substituting the researcher in the analysis process. In contrast, we discuss the potential rewards of using automatized methods to support the analysis of teachers’ collegial discussions. We introduce a set of visualization procedures based on semi-supervised topic modeling and supervised classification algorithms and discuss how these visualizations can provide additional elements useful for the analysis of collegial discussions. Whereas these visualizations do not substitute interpretive analysis, we argue that they can support researchers’ interpretations by reducing data, identifying patterns of interactions, and possibly, visualizing group-level learning.
##plugins.themes.bootstrap3.article.details##
How to Cite
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
openAccess
Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2011). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103(1), 1–18.
Asoyan, K. (2013). Lärares syn på disciplin, ordning och ansvar i klassrummet: Sverige respektive Ryssland. [Master's thesis, Malmö University College]. http://urn.kb.se/resolve?urn=urn:nbn:se:mau:diva-34475
Benoit, K., Watanabe, K., Wang, H., Nulty, P., Obeng, A., Müller, S., Matsuo, A., Lowe, W., Müller, C., & Council (ERC-2011-StG 283794-QUANTESS), E. R. (2021). quanteda: Quantitative Analysis of Textual Data (3.2.0) [Computer software]. https://CRAN.R-project.org/package=quanteda
Blei, D. M. (2012). Topic modeling and digital humanities. Journal of Digital Humanities, 2(1), 8–11.
Buysse, V., Sparkman, K. L., & Wesley, P. W. (2003). Communities of Practice: Connecting What We Know with What We Do. Exceptional Children, 69(3), 263–277. https://doi.org/10.1177/001440290306900301
Chen, N.-C., Drouhard, M., Kocielnik, R., Suh, J., & Aragon, C. R. (2018). Using Machine Learning to Support Qualitative Coding in Social Science: Shifting the Focus to Ambiguity. ACM Transactions on Interactive Intelligent Systems, 8(2), 9:1-9:20. https://doi.org/10.1145/3185515
Fairclough, N. (1992). Discourse and social change. Polity.
Flyvbjerg, B. (2006). Five Misunderstandings About Case-Study Research. Qualitative Inquiry, 12(2), 219–245. https://doi.org/10.1177/1077800405284363
Forsberg, E., Hortlund, T., & Malmberg, K. (2016). The Assessment Culture of School Leadership. Nordic Studies in Education, 35(02), 141–158. https://doi.org/10.18261/issn.1891-5949-2016-02-05
Franke, M. L., Kazemi, E., & Battey, D. (2007). Mathematics teaching and classroom practice. In F. K. Lester (Ed.), Second Handbook of Research on Mathematics Teaching and Learning (2nd ed.). Information Age Publishing.
Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K., Wearne, D., & Murray, H. (1997). Making Sense: Teaching and Learning Mathematics with Understanding. Pearson Education Canada.
Hvitfeldt, E., & Silge, J. (2021). Supervised Machine Learning for Text Analysis in R. Chapman and Hall/CRC. https://doi.org/10.1201/9781003093459
Insulander, E., Brehmer, D., & Ryve, A. (2019). Teacher agency in professional development programmes – A case study of professional development material and collegial discussion. Learning, Culture and Social Interaction, 23, 1–9. https://doi.org/10.1016/j.lcsi.2019.100330
Janasik, N., Honkela, T., & Bruun, H. (2008). Text Mining in Qualitative Research: Application of an Unsupervised Learning Method. Organizational Research Methods, 12(3), 436–460. https://doi.org/10.1177/1094428108317202
Jurka, T. P., Collingwood, L., Boydstun, A. E., Grossman, E., & Atteveldt, W. van. (2020). RTextTools: Automatic Text Classification via Supervised Learning (1.4.3) [Computer software]. https://CRAN.R-project.org/package=RTextTools
Jurka, T. P., Collingwood, L., Boydstun, A. E., Grossman, E., & others. (2013). RTextTools: A Supervised Learning Package for Text Classification. RJournal, 5(1), 6–12.
Kaufmann, O. T., & Ryve, A. (2019). Construction of teachers’ roles in collegial discussions. In U. T. Jankvist, M. Van den Heuvel-Panhuizen, & M. Veldhuis (Eds.), Proceedings of the Eleventh Congress of the European Society for Research in Mathematics Education. Freudenthal Group & Freudenthal Institute, Utrecht University, Netherlands and ERME. https://hiof.brage.unit.no/hiof-xmlui/handle/11250/2642051
Kilpatrick, J., Swafford, J., & Findell, B. (2001). Adding it up: Helping children learn mathematics. National Academy Press.
Matta, C. (2014). Scientific Representation and Science Learning. Open Review of Educational Research, 1(1), 211–231. https://doi.org/10.1080/23265507.2014.989900
McPherson, G., & Thorne, S. (2006). Exploiting Exceptions to Enhance Interpretive Qualitative Health Research: Insights from a Study of Cancer Communication. International Journal of Qualitative Methods, 5(2), 73–86. https://doi.org/10.1177/160940690600500210
Munter, C. (2014). Developing Visions of High-Quality Mathematics Instruction. Journal for Research in Mathematics Education, 45(5), 584–635. https://doi.org/10.5951/jresematheduc.45.5.0584
Munter, C., Stein, M. K., & Smith, M. (2015). Dialogic and Direct Instruction: Two Distinct Models of Mathematics Instruction and the Debate(s) Surrounding Them. Teachers College Record, 117(11), 1–32.
Rietz, T., & Maedche, A. (2021). Cody: An AI-Based System to Semi-Automate Coding for Qualitative Research. Y. Kitamura (Ed.), Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–14. https://doi.org/10.1145/3411764.3445591
Rose, J., & Lennerholt, C. (2017). Low Cost Text Mining as a Strategy for Qualitative Researchers. Electronic Journal of Business Research Methods, 15(1), 2–16.
Ryve, A., & Hemmi, K. (2019). Educational policy to improve mathematics instruction at scale: Conceptualizing contextual factors. Educational Studies in Mathematics, 102(3), 379–394. https://doi.org/10.1007/s10649-019-09887-6
Sjoberg, D. (2022). Ggstream [Computer software]. https://github.com/davidsjoberg/ggstream (Original work published 2020)
Sjödin Fannoun, M., & Perlman, S. (2019). Demokratisk kompetens: En strukturerad litterturöversikt om värderingar och attityder i årskurs F-3 [Student thesis, Linköping University]. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-156408
Suárez, M. (2003). Scientific representation: Against similarity and isomorphism. International Studies in the Philosophy of Science, 17(3), 225–244. https://doi.org/10.1080/0269859032000169442
Steenbrugge, H. V., Larsson, M., Insulander, E., & Ryve, A. (2018). Curriculum Support for Teachers’ Negotiation of Meaning: A Collective Perspective. In Fan, L., Trouche, L., Qi, C., Rezat, S., Visnovska, J. (Eds.), Research on Mathematics Textbooks and Teachers’ Resources (pp. 167–191). Springer, Cham. https://doi.org/10.1007/978-3-319-73253-4_8
Svedberg, T., & Fjällström, K. (2010). Vägar till ett demokratiskt klassrum: - En kvalitativ undersökning [Student thesis, Örebro University]. http://urn.kb.se/resolve?urn=urn:nbn:se:oru:diva-9645
Valoyes-Chávez, L. (2019). On the making of a new mathematics teacher: Professional development, subjectivation, and resistance to change. Educational Studies in Mathematics, 100(2), 177–191. https://doi.org/10.1007/s10649-018-9869-5
Vangrieken, K., Dochy, F., Raes, E., & Kyndt, E. (2015). Teacher collaboration: A systematic review. Educational Research Review, 15, 17–40. https://doi.org/10.1016/j.edurev.2015.04.002
Vangrieken, K., Meredith, C., Packer, T., & Kyndt, E. (2017). Teacher communities as a context for professional development: A systematic review. Teaching and Teacher Education, 61, 47–59. https://doi.org/10.1016/j.tate.2016.10.001
Vescio, V., Ross, D., & Adams, A. (2008). A review of research on the impact of professional learning communities on teaching practice and student learning. Teaching and Teacher Education, 24(1), 80–91. https://doi.org/10.1016/j.tate.2007.01.004
Walfridsson, E., & Sanfridson, S. (2016). Traditionell matematikundervisning duger väl??: En litteraturstudie om alternativa undervisningsmetoder i matematik och hur dessa kan påverka elevers prestationer i och attityder till matematik. [Student thesis, Linköping University]. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-129064
Watanabe, K. (2022). Seeded-LDA for semisupervised topic modeling [Computer software]. https://github.com/koheiw/seededlda (Original work published 2019)
Watanabe, K., & Zhou, Y. (2022). Theory-Driven Analysis of Large Corpora: Semisupervised Topic Classification of the UN Speeches. Social Science Computer Review, 40(2), 346–366. https://doi.org/10.1177/0894439320907027
Wester, R. (2015). Matematikundervisning utifrån ett elevperspektiv: “det är klyddigt.” [Licentiate Dissertation, Malmö University College]. http://hdl.handle.net/2043/18169
Wickham, H., Chang, W., Henry, L., Pedersen, T. L., Takahashi, K., Wilke, C., Woo, K., Yutani, H., Dunnington, D., & RStudio. (2021). ggplot2: Create Elegant Data Visualisations Using the Grammar of Graphics (3.3.5) [Computer software]. https://CRAN.R-project.org/package=ggplot2
Wiedemann, G. (2016). Text Mining for Qualitative Data Analysis in the Social Sciences: A Study on Democratic Discourse in Germany. VS Verlag für Sozialwissenschaften. https://doi.org/10.1007/978-3-658-15309-0
Winther Jørgensen, M., & Phillips, L. (2002). Discourse analysis as theory and method. Sage.
Wittzell, F., & Nilsson, H. (2012). Lärares arbetssituation: En studie kring hur lärare erfar och uppfattar arbetsuppgifter och arbetstid [Student thesis, Krisianstad University College]. http://urn.kb.se/resolve?urn=urn:nbn:se:hkr:diva-10088
Yu, C. H., Jannasch-Pennell, A., & DiGangi, S. (2011). Compatibility between text mining and qualitative research in the perspectives of grounded theory, content analysis, and reliability. Qualitative Report, 16(3), 730-744.