RCT(s) without
Heffernan Authorship
Published or pending work without Neil Heffernan as an author,
conducted using E-TRIALS or its predecessor, the ASSISTments Testbed
Published Full/Short Papers // 18
ASSISTments closes achievement gaps (between lower and higher achieving students)
Fyfe, E. (2016). Providing feedback on computer-based algebra homework in middle-school classrooms. Computers in Human Behavior 63, 568-574. [PDF] University of Wisconsin-Madison
Motz, B. (2024). Concentration toward the mode: Estimating changes in the shape of a distribution of student data. Journal of School Psychology, 107, 101364. https://doi.org/10.1016/j.jsp.2024.101364 [PDF] University of Indiana
ASSISTments data used to answer a methods question:
Closser, A.H., Sales, A. & Botelho, A.F. (2024) Should We account for classrooms? Analyzing online experimental data with student-level randomization. Education Tech Research Dev (2024). https://doi.org/10.1007/s11423-023-10325-x [PDF] Purdue and Univ of Florida
Using ASSISTments to study spatial features in mathematical expressions
Harrison, A., Smith, H., Hulse, T., & Ottmar, E. (2020). Spacing out!: Manipulating Spatial Features in Mathematical Expressions Affects Performance. Journal of Numerical Cognition, 6(2), 186-203. DOI: 10.5964/jnc.v6i2.243. [PDF] Worcester Polytechnic Institute
Comparing hints to step-by-step scaffolding questions, solutions, and worked examples
Andres-Bray, M., Hutt, S., Zhou, Y, Ostrow, K. & Baker K. (2021). A Comparison of Hints vs. Scaffolding in a MOOC with Adult Learners. AIED 2021. [PDF] University of Pennsylvania and Worcester Polytechnic Institute
McGuire, P., Tu, S., Logue., M., Mason, C., Ostrow, K. (2017). Counterintuitive effects of online feedback in middle school math: results from a randomized controlled trial in ASSISTments. Educational Media International, 54(3), 231-244, DOI: 10.1080/09523987.2017.1384161. [PDF] University of Colorado Colorado Springs and University of Maine
Chan, J. Y.-C., Lee, J.-E., Mason, C. A., Sawrey, K., & Ottmar, E. (2022). From Here to There! A dynamic algebraic notation system improves understanding of equivalence in middle-school students. Journal of Educational Psychology, 114(1), 56–71. [PDF] Worcester Polytechnic Institute
Smith, H., Closser, A. H., Ottmar, E. R., & Chan, J. Y. C. (2022). The impact of algebra worked example presentations on student learning. Applied Cognitive Psychology, 36(2), 363-377. [PDF] Worcester Polytechnic Institute
Comparing the difficulty of different problem types
Walkington, C., Clinton, V., & Sparks, A. (2019). The effect of language modification of mathematics story problems on problem-solving in online homework. Instructional Science. 47, 499-529. [PDF] Southern Methodist University and University of North Dakota
Comparing the format of difference problem types
Jiang, Y., Almeda, M. V., Kai, S., Baker, R. S., Ostrow, K., Inventado, P. S., & Scupelli, P. (2020). Single Template vs. Multiple Templates: Examining the Effects of Problem Format on Performance. In Gresalfi, M. & Horn, I. S. (Eds.), The Interdisciplinarity of the Learning Sciences, 14th International Conference of the Learning Sciences (ICLS) 2020, Volume 2 (1015-1022). Nashville, Tennessee: International Society of the Learning Sciences. [PDF] ETS, TERC, Teachers College Columbia University, University of Pennsylvania, Worcester Polytechnic Institute, California State University Fullerton, and Carnegie Mellon University
Ngo, V., Lacera, L. P., Closser, A., & Ottmar, E. (2022). The effects of operator position and superfluous brackets on student performance in simple arithmetic. Journal of Numerical Cognition. [PDF] Worcester Polytechnic Institute
Comparing different instructional approaches
Unal, D. S., Arrington, C.M., Solovey, E., and Walker, E. (2020). Using Thinkalouds to Understand Rule Learning and Cognitive Control Mechanisms Within an Intelligent Tutoring System. In: Bitencourt I., Cukurova M., Muldner K., Luckin R., Millan E. (eds) Artificial Intelligence in Education. AIED 2020. Lecture Notes in Computer Science, vol 12163. Springer, Cham. https://doi.org/10.1007/978-3-030-52237-7_40. [PDF] University of Pittsburgh, Lehigh University, and Worcester Polytechnic Institute
Koedinger, K.R. & McLaughlin, E.A. (2010). Seeing language learning inside the math: Cognitive analysis yields transfer. In S. Ohlsson & R. Catrambone (Eds.), Proceedings of the 32nd Annual Conference of the Cognitive Science Society. (pp. 471-476.) Austin, TX: Cognitive Science Society. [PDF] Carnegie Mellon University
Koedinger, K. & McLaughlin, E. (2016) Closing the Loop with Quantitative Cognitive Task Analysis. In Barnes, Chi & Feng (eds) The 9th International Conference on Educational Data Mining. 412-417. [PDF] Carnegie Mellon University
Hurst, M. A., Cordes, S. (2018). Labeling Common and Uncommon Fractions Across Notation and Education. Proceedings of Cognitive Science, 1841-1846. [PDF] University of Chicago and Boston College
New Economics Journal uses ETRIALS
Duquennois, C. (2022). Fictional money, real costs: Impacts of financial salience on disadvantaged students. American Economic Review, 112(3), 798-826. [Author PDF][Journal version ] University of California Berkeley
Transferability of Findings on Student Supports for College Students
Smalenberger, M., & Smalenberger, K. (2022). Do College Students Learn Math the Same Way as Middle School Students? On the Transferability of Findings on Within-Problem Supports in Intelligent Tutoring Systems. In A. Mitrovic and N. Bosch, editors, Proceedings of the 15th International Conference on Educational Data Mining, pages 748– 752, Durham, United Kingdom, July 2022. International Educational Data Mining Society. [PDF] University of North Carolina Charlotte
Game-based learning vs. Immediate Feedback vs. No Feedback
Decker-Woodrow, L. E., Mason, C. A., Lee, J. E., Chan, J. Y. C., Sales, A., Liu, A., & Tu, S. (2023). The impacts of three educational technologies on algebraic understanding in the context of COVID-19. AERA open, 9, 23328584231165919. [PDF] Westat & WPI
Fractions Study
Oppenzato, C. O. (2024). Rebalancing fraction arithmetic practice. (Publication No. 30990188). [Doctoral dissertation, Columbia University]. ProQuest Dissertations & Theses Global. https://doi.org/10.7916/8hm7-0h26 [Columbia Univ.]
Posters/Workshops/Etc // 6
Harrison, A., Smith, H., Hulse, T., & Ottmar, E. (2020). Spacing out: Manipulating spatial features in math expressions affects performance. Paper to be presented in a roundtable session on “Design Considerations in Mathematics Learning” at the 2020 American Educational Research Association Annual Meeting in San Francisco, California. Worcester Polytechnic Institute
Smith, H., Harrison, A., Chan, J. C., & Ottmar, E. (2020). Dynamic vs. static: Which worked examples work best? Poster submission to the 2020 meeting of The Mathematical Cognition and Learning Society. [pre-registration Worcester Polytechnic Institute
Smith, H., Ramey, K., Heffernan, N., & Uttal, D. (June, 2022) Can mental rotation predict performance in an online geometry assignment? In C. Chinn, E. Tan, C. Chan, & Y. Kali (Eds.), International Collaboration toward Educational Innovation for All: Overarching Research, Development, and Practices - 16th International Conference of the Learning Sciences, ICLS 2022. International Society of the Learning Sciences (ISLS). ISBN 9781737330653 , ISSN 1814-9316. 1361-1364. Retrieved from https://www.dropbox.com/s/ws5sdcfi72aykj1/ICLS2022%20Proceedings.pdf?dl=0 or https://2022.isls.org/proceedings/
Ottmar, E. & Colleagues. (2022). Does where you start matter? The interaction between prior knowledge and effectiveness of game-based interventions. Conference proposal accepted by International Society of the Learning Sciences.
Ottmar, E. & Colleagues. (2022). From performance to perception: A laboratory-based task to detect changes in students’ perception of math equivalence in technology interventions. Conference proposal accepted by American Educational Research Association.
- Nasiar, N., Baker, R.S., Li, J., Gong, W. (2022). How do A/B Testing and Secondary Data Analysis on AIED Systems Influence Future Research?. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2022. Lecture Notes in Computer Science, vol 13355. Springer, Cham. https://doi.org/10.1007/978-3-031-11644-5_10
Under Review // 5
Vahey, P., Feng, M., Algama, M., & Liu, J. (Under Review). Using rapid experimental techniques to design more effective environments. SRI International, WestEd
Ngo, V., Perez, L., Closser, A. H., & Ottmar, E. The effects of operand position and superfluous brackets on student performance in math problem-solving. Manuscript in preparation. (Under Review)
Iannacchione, A., Ottmar, E., Ngo, V., Mason, Chan, C., J. Y., Smith, H., Drzewiecki, K., & Shaw, S. Examining relations between math anxiety, prior knowledge, hint usage, and math performance in two different online learning contexts. (Under Review)
Finster, M., Decker-Woodrow, L., Booker, B., Mason, C. A., & Tu, S. Cost-effectiveness of algebraic technological applications. (Under Review)
Ottmar, E. & Colleagues. In-person vs. Virtual: Learning modality selections and movement during COVID-19 and their influence on student learning. (Under Review)
In Preparation // 3
Closser, A. H., Sales, A., & Botelho, A. F. Should we account for classrooms? Analyzing online experimental data with student-level randomization. (Manuscript in preparation)
Smith, H., Ngo, V., Sales, A., Closser, A. H., Chan, J. Y. C., & Ottmar, E. R. To wait or not to wait: Adding to the debate on immediate vs. delayed feedback. (Manuscript in preparation)
St. John, J., Thompson, T., Closser, A. H., & Ottmar, E. et al. (In Progress). The vibrant side of math: The perceptual effects of color on mathematical Accuracy. [PDF]
Studies going on in the E-TRIALS Platform
Support Comparison Studies
Problem Varied Studies
