Written by Molly Stewart, Research Scientist, Indiana University
When I think back to the beginning of our Ed-Fi implementation in the INsite Consortium (Indiana Student Information to Empower), I have to laugh. We were so naive! We didn’t even know what we didn’t know, which meant we didn’t know what questions to ask. For context, our goal was to “implement Ed-Fi” in ten pilot school districts in Indiana, with the end result for users being the Ed-Fi dashboards. From my non-technical perspective, my job was to focus on the users of the technology (assumed at that time to be only the dashboards). I put “implement Ed-Fi” in quotes because, at the time, I had only a vague idea as to what that meant! I was a victim of the assumption that getting the right data to flow to the right places in the dashboard would be fairly simple and quick (via IT Magic!).
Coming to this project as a researcher, my first thought was, “What a great opportunity to study how teachers utilize data! We’ll have before and after surveys and interviews and observe people in schools using data in action.” Over 18 months later, I can say that we did accomplish some of those things, but what the whole process looked like was significantly different than what I had expected.
INsite awarded Ed-Fi Rookie of the Year (2018)
My first step in any project is to review the existing literature on the topic. What mistakes should I avoid? What foundation can I build on? What I found in the literature (at the time) should have given me a heads-up: There wasn’t much existing research on the domains within the Ed-Fi Data Standard (i.e. attendance, behavior/discipline, course grades) other than looking at these data points as predictors in Early Warning Systems. For reference, this was back in March of 2018.
Most existing literature on data-based decision-making has examined teachers’ use of assessment scores. I didn’t even know enough about Ed-Fi versions, API integrations, or the assessment domain to know what kind of data points were going to be in our Operational Data Store (or not). Even today, as I am writing this blog and checking my facts, all of this is still somewhat in progress!
Fast-forward several months. We have now completed nearly 200 interviews with principals, teachers, and other school staff, and I have spent an incredible number of hours trying to teach my own qualitative brain to understand the technical limitations of data manipulation and visualizations (or at least the cost prohibition of trying to make some of our school partners’ ideas work in reality).
Nearing the end of our 18-month pilot period, I have several lessons learned to share with the Ed-Fi Community about working with district and school staff to see Ed-Fi-based technologies successfully implemented in schools.
Talking Like a Teacher & Communicating Progress
Those of you in the Ed-Fi technical world know all of this and could probably correct my accuracy on a few points. But how do you translate these “works in progress” to a middle school principal when she asks you, “Will we have X type of data available through this project?” And that’s the simplest question you’ll get!
“Will it look like this? Will we be able to filter it by A, B, and C? Can we customize start and end dates for this search?” The list of questions goes on, and you realize that there is a veritable chasm between the tech and the school–and I, hovering in this chasm, can’t answer anyone’s questions sufficiently. Educators do not think in rows and columns; not in the same way that software developers do, anyway. To bridge this knowledge gap, someone in the education world needs to understand the technology (the limitations as well as the capabilities), and someone in the technical world needs to truly understand how educators think about and use (or want to use) data.
Ed-Fi Implementation Lessons Learned
- Ask (what they need), solve (their need if you can), then share (your solution). Don’t start with a given solution and try to shoehorn their needs into it. It might work for a few people, but you’ll probably waste a lot of time (yours and theirs).
- Don’t show a tool with a demo data set to an end-user. Your real data set will not look that good (at least at first)!
- Find an end-user who is tech- and data-savvy (i.e. is interested in the data flow from their SIS to the ODS) and engage with them to do some initial data quality checking. Someone who has no interest in this will likely see incorrect data as a failure of the entire system as opposed to one step in the process. This person could be a teacher, instructional coach, principal or assistant principal, or counselor; with so many people changing roles in education, their current title doesn’t always reflect the extent of their data knowledge!
- Find an end-user who is a data evangelist—someone who knows what they want, is willing to put in the time to figure out the tech-to-end user process, and who, when given the data they want, is going to show everyone else in their district what that data (and their partnership with Ed-Fi) can do!
- Be patient—If your data tool still needs to be tweaked, limit who you show it to. Get feedback from those that understand the process, but don’t be in a hurry to share it out until your trusted contacts tell you that it is ready to be positively received by their staff members.
Finding Your Data Advocates
If your goal is to incorporate tools with underlying Ed-Fi technology throughout a particular school or district, as ours has been, having a solid advocate located in a school will make your work much easier.
INsite Team with two Representatives from Ed-Fi Pilot Districts
The best advocates are not always immediately apparent. We found our strongest advocates through a combination of perseverance and luck (for example, after sending several follow-up emails to one school leader, he carbon-copied someone else who ended up taking the idea and running with it).
Other advocates have been training participants who took the initiative to follow up with us via email, which let us know that they were actively using the tools on a regular basis. The best system that we have come up with so far is asking our primary contacts (technical, in instructional leadership) to put us in touch with their colleagues who are likely to be excited about the project and its possibilities.
Your best advocate may not even be in this first group of recommended individuals. Data leaders within a district tend to know and communicate with each other, and you can ask your contacts to help spread the word when they think their colleagues are ready to hear it.
Remember: This is Your Job, Not Theirs
A final lesson learned: the school staff that you want to work with probably have zero extra time to work with you, and they need what you’re working on for them yesterday. The faster you can deliver, and the more agile you are in your approach, the more buy-in you will get.
Many of the educators in this country are literally going to work every day to save lives; to help kids get on the right academic path, to connect them with social services, to positively intervene in emerging behavioral issues. Be ready to get on the phone at 7 am, because that’s when their day starts, if not before. And be prepared to quickly and accurately respond to questions like, “it’s not working, why is it not working?”
Don’t Forget Why The Work Matters
Keep your eye on the prize. If you embark on an Ed-Fi journey with school folks, you’re in for one of the most rewarding experiences of your life: knowing that you’ve had a hand in helping those superheroes help the kids who need it most.
Ed-Fi is about the community of people; we build this community one relationship at a time and prove out how the technology + people make it consistently better and more useful. Educators are failed by proprietary vendors every day, every year, consistently. Understanding that using this technology makes us part of a bigger movement and that putting in the work can help not only us today but others tomorrow, makes it a powerful and worthy investment of limited and precious educator time. If the outcome is more efficient, effective data access and organization, ultimately saving an even greater amount of instructional and relationship-building time with kids, then it’s even more worth it.
The Next Step: Making the Tech Work for Teachers
Ideally, we will get to the point of having these higher-level data points right alongside the micro, daily learning data points. The only way we can get there is by deciding that this is a worthy goal for the community as a whole, and by regularly seeking out the feedback of educators who are in the classroom daily, field testing their ideas and designs until we reach the perfect combination of interoperability and customization.
If getting Ed-Fi technology into the hands of teachers is the ultimate goal, then we must target the types of data that they actually need and use—including standards-based grading, running records, and daily assignments. A common term teachers used in our interviews was “taking data”: they primarily use data that they collect in the classroom. Currently, these data types are not usually populated (or even able to be populated) by certified integrations.
Ed-Fi is getting quite good at sending the easier data points—numbers that are required to be tracked for state reporting, for example. Now, as a community effort, we need to get better at facilitating interactive data tools that teachers can customize and self-populate with the data that really matters to them—what they observe and record in the classroom.
About the Author
Molly Stewart is an education researcher and program evaluator with Indiana University. Her areas of expertise include federal and state education policy and law, education markets and finance, school choice and privatization, and data-based decision-making. Molly also teaches evaluation and research methods courses for the School of Public and Environmental Affairs at Indiana University.
Note: This blog post reflects the viewpoint of the author, not their employer.
