The 2018 Technology & Learning Insights Report: Towards Understanding App Effectiveness and Cost

The Technology & Learning Insights Report highlights the challenges and opportunities in evaluating the effectiveness of educational technology apps in schools. With over 2,500 apps available, educators face difficulties in selecting impactful tools amidst budget constraints and accountability pressures. The report emphasizes the need for authentic quantitative and qualitative data to assess app effectiveness, as usage and cost data alone cannot determine their impact on student learning. Using the BrightBytes Learning Outcomes module, researchers analyzed data from 392,603 students across 48 districts, focusing on 150 student-focused ed tech programs. The module integrates app usage data with standardized test scores in math, science, and English Language Arts to measure impact. Findings revealed that while most apps showed limited correlation between usage and learning outcomes, specific apps demonstrated significant positive effects. For example, ALEKS, DreamBox, and LearnZillion were among the most effective for math learning. The report also notes that app usage is generally lower than expected, with many licenses going unused. This highlights the importance of aligning procurement practices with actual usage and ensuring effective implementation. Additionally, the study underscores the need for correlational data to complement experimental research, enabling districts to make informed decisions based on their unique contexts. While the findings are preliminary and correlational, they provide valuable insights into app effectiveness and scalability. The report advocates for combining correlational evidence with randomized controlled trials to build a robust knowledge base for improving student outcomes through ed tech.

Research Citation:

OETC. (2018). The 2018 Technology & Learning Insights Report: Towards Understanding App Effectiveness.