District Profile: Morris School District
Seven districts ranging in various sizes and locality participated in the project, including New Jersey’s Morris Public School (MPS). This profile outlines the Morris team’s initial objectives at the beginning of the program and the insights they gleaned after seeing and reviewing their application usage and engagement data.
How Morris School District Leveraged Data Analytics to Track ROI, Engagement, and Opportunity Gaps
USING DATA TO ASSESS ROI
Like many school districts across the country, determining the cost per user of EdTech investments is critical to informing Morris School District’s (MSD’s) renewal and purchasing strategies. It has been difficult to calculate this amount without knowing the extent to which their paid apps and digital programs were being used by their students, especially this past year.
At the onset of the project, the MSD team immediately began using CatchOn to track application usage data across the district and identify which applications were being used and how often they were being used. Leveraging this data, they quickly gleaned which investments have been paying off for the district.
“We have a learning management system that costs us a lot of money, but I can see it is being used a lot,” said Erica Hartman, MSD’s Director of Instructional Technology. “And I’m also really proud that all the apps within our Trending Apps list are approved, so that means our teachers are using the apps that Brian Young, our Director of Curriculum and Instruction, has approved and introduced.”
IDENTIFYING PROFESSIONAL DEVELOPMENT BEST PRACTICES
The team is also using the actionable data generated to identify professional development best practices and opportunities. For example, this past year the district rolled out a program called Gizmos to its science teachers because they needed a way to conduct online labs for their virtual students. Much to the Morris team’s delight, they saw that the program was in their top 20 Trending Apps list within the CatchOn platform. CatchOn helped to validate their technology investments.
“From a curricular lens, we are making decisions based on need and support both for teachers and students,” said Brian Young. “For us, it’s not just a monetary investment in the platform itself. With Gizmos, for example, we hosted and paid for several professional development sessions. Looking at the usage data reaffirms that we are making effective decisions, and it shows me how valuable our teachers find that particular program for online science, investigation, and inquiry.”
CatchOn’s application usage data also prompted Young and Hartman to closely examine what made the Gizmos deployment so successful and strategize how they can replicate its success. “It is very reassuring to see that we are getting our return on investment on not only the technology itself but also the trainings we do to enable our teachers,” said Hartman.
TRACKING, MEASURING AND ANALYZING STUDENT ENGAGEMENT
Tracking and gaining a better understanding of student engagement in a digital learning environment has been another priority for MSD. Throughout the project, the MSD team reviewed their student engagement data to identify how often students were engaging with their digital tools, the time of day and days of the week students were engaging with their digital tools, and the duration of each engagement.
Having insight into their application usage and engagement data also enabled Morris’s team to identify apps with higher-than-expected engagements and then pinpoint professional development needs. “When we see excitement around a platform in the form of usage data, we should build upon that,” said Mackey Pendergrast, MSD’s Superintendent. “Let’s improve the learning process even more by leveraging an asset-driven professional development approach.”
This asset-driven strategy aligns with Superintendent Pendergrast’s two-part engagement strategy. “There are two questions that we need to answer regarding engagement,” said Pendergrast. “We need to understand what tools are being used and how they are being used. The data answers that first question for us, which is great. The second question is, ‘How are these tools meeting our learning goals and objectives?’ Some platforms may have really high digital engagement numbers, which is great, but we also purchased these platforms for specific reasons, and we want to make sure they are meeting our objectives.”
HERE ARE THREE KEY TAKEAWAYS AFTER REVIEWING THE DATA
✓ They were surprised by the high volume of engagement with some apps, particularly with one app because not much professional development was built around it, so the team was surprised to see how much it was being used by their students
✓ They were pleased to see that another tool also had significant engagement, even after the conclusion of the district’s assessment and testing window, and that the usage occurred mostly during daytime hours, meaning teachers were actively using the tool as part of their instruction and not just assigning the tool as homework.
✓ They were astonished by the sheer volume of apps being used by their students. In fact, the number of apps in use greatly surpassed the number of apps the MSD team estimated being used.
DISCOVERING OPPORTUNITY GAPS
A final challenge explored by the MSD team during the pilot related to opportunity gaps. Because the district was remote for most of the year, it was concerned about students having access to their digital tools and learning resources. Using CatchOn, the team was able to monitor how many devices were being used to help ensure their students were online and active. During the course of the pilot, the district did discover student engagement gaps that they attributed to student access, specifically usage gaps among specific grade levels and time of day engagement gaps.
Key Insights and Next Steps
Going forward, MSD intends to diver deeper into their application usage and engagement data to see what additional insights they can glean from it. “We want to look at a specific data point and identify the insight, the approach, and the questions we now have around that specific data point,” said Pendergrast. “From there, we can formulate the actions we need to take. This data certainly tells us a story and it will help guide us in our decision-making.”