Disseminate

Access data on your study to analyze and publish your work - make sure to follow our Terms of Use with regard to data anonymity, open science, and citing ASSISTments

Terms of Use

As you Develop and Deploy your study it is important to remember that WPI's IRB requires all interventions created within ASSISTments (via E-TRIALS) and/or deployed to ASSISTments users fall within the boundaries of normal instructional practice. E-TRIALS use also requires that researchers receive - in the very least - an IRB exemption from their university or institution to access anonymized data. If using our ASSISTed Deploy model by working with our TFRF group or adding survey questions that likely extend beyond the bounds of normal instructional practice in mathematics, additional IRB Requirements may be imposed.

However, E-TRIALS also requires that users agree to Terms of Use that encompass the retrieval, analysis, and publication of study materials falling within Dissemination of findings. These Terms are meant to protect teacher and student users of ASSISTments, enforce open science policies, and outline proper citation of ASSISTments and E-TRIALS as tools for scientific research in online learning.

Legacy Data Requests

Once you have Developed and Deployed your study, you will want to access and analyze resulting anonymized data in order to Disseminate your findings. Although most ASSISTments users now access ASSISTments through Google Classroom, if you are using an Auto-Deploy or ASSISTed Deploy approach, your study may still reach teacher and student users of the legacy version of ASSISTments. You can request data from these users automatically using a Legacy Data Request.

You may have already received anonymized legacy data and are now looking for a Data Glossary. If you find variables are left undefined, please contact etrials@assistments.org for more information.

New System Data Requests

Most ASSISTments users are now using ASSISTments through Google Classroom - their anonymized data will NOT appear in a Legacy Data Request reports. If your report does not generate data as expected and/or to request access to anonymized data from users of ASSISTments through Google Classroom, please contact etrials@assistments.org.

You may have already received anonymized data from users of ASSISTments through Google Classroom and are now looking for a Data Glossary. If you find variables are left undefined, please contact korinn.ostrow[at]assistments.org for more information.

OSF Data Storage

If you have used our Auto-Deploy or ASSISTed Deploy approaches, you will most likely be reaching the majority of our users, those using ASSISTments through Google Classroom.

Once you have requested anonymized data that captures these users by contacting etrials@assistments.org, we suggest that - if you haven't already - you create a project page for your research with the Open Science Foundation (OSF). We support the OSF's emphasis on open science and data transparency and their project pages offer a way to preregister your work and store data publicly for citation. The OSF even makes it easy to register a free DOI for your project page(s), making citation clean and stable, indefinitely. An E-TRIALS team member can work with you to host this data on your OSF project page if you need assistance.

Publication

Once you've reached the point where you want to publish your findings, first let us offer our congratulations! Then, let us remind you to consult our Terms of Use with regard to publication and proper citation of the data you have received from ASSISTments.

We also remind you that we support open science and data transparency and urge you to do so as well. You can cite a legacy data report directly (this paper offers an example) and shorten URLs if desired. All legacy data reports will be archived for such purposes. You can also cite data from our new system by creating an OSF project page for your work and hosting the data you receive as part of a public project or registration.

If you wish to cite ASSISTments as a tool for research, we suggest using any of the following works:

  • Heffernan, N. & Heffernan, C. (2014). The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education. 24(4), 470-497. [Full Text]

  • Ostrow, K., Selent, D., Wang, Y., Van Inwegen, E., Heffernan, N., & Williams, J. J. (2016). Assessment of Learning Infrastructure (ALI): The Theory, Practice, and Scalability ofAutomated Assessment. Proceedings of 6th International Conference on Learning Analytics and Knowledge. [Full Text]

  • Ostrow, K.S., Heffernan, N.T., & Williams, J.J. (2017). Tomorrow’s EdTech Today: Establishing a Learning Platform as a Collaborative Research Tool for Sound Science. Teachers College Record, 119(3), 1-36. [Full Text]

  • Ostrow, K.S. (2018). A Foundation for Educational Research at Scale: Evolution and Application. PhD Dissertation, Learning Sciences & Technologies, Worcester Polytechnic Institute. [Full Text]

  • Ostrow, K. & Heffernan, N. (2019). Advancing the State of Online Learning: Stay Integrated, Stay Accessible, Stay Curious. In Robert Feldman (Ed.) Learning Science: Theory, Research, and Practice. McGraw Hill Education.

If you would like to acknowledge the grant funding that has made our work to build E-TRIALS and its predecessors possible, the following grants have supported research in ASSISTments:

NSF 1440753: SI2-SSE&SSI. "Adding Research Accounts to the ASSISTments Platform: Helping Researchers Do Randomized Controlled Studies with Thousands of Students." $500,000. 2014 - 2016.

NSF 1724889: CIF21 DIBBs: "PD: Enhancing and Personalizing Educational Resources through Tools for Experimentation." PIs Heffernan (WPI) & Korinn Ostrow (WPI). $544,644. 2017-2020.

Schmidt Futures 995: E-TRIALS. $2,000,000. 2019 - 2021.

NSF 1931523: Collaborative Research: CyberInfrastructure for Shared Algorithmic and Experimental Research in Online Learning. In collaboration with UPenn's Ryan Baker. $3,500,000 (WPI portion = $1,800,000). 2019 - 2024.