Soon the Ed-Fi Alliance will release version 2.0 of our Analytics Middle Tier, welcoming it to the “big kids’ table” as a fully supported add-on to the ODS.
When we started this project in 2018, it was with the aim of creating a production-ready proof-of-concept (a seeming oxymoron, we know) for simplifying the Ed-Fi Data Model, thus making the ODS more useful for reporting and ad hoc analytics exploration (see From Diagrams & Definitions: Solving the Analytics Reporting Gap). As such, it was released on the Ed-Fi Exchange instead of being bundled into the core ODS/API platform.
The solution provides a set of database views, using dimension modeling to de-normalize the ODS table structure. Any business intelligence/reporting tool can consume these views—they are platform-neutral. Our first Analytics Middle Tier release targeted the Ed-Fi Data Standard version 2, and while initially there was no firm commitment to future updates, we supported this interest from the community by releasing several bug fixes and improvements in 2019.
Interest in the Analytics Middle Tier has been strong since day one, not only among school district data analysts but also among vendors and education service centers looking to speed up their development of standardized reports. This widespread support has been driven by two key features of the Analytics Middle Tier: the simplification of the data model, and the anticipation of a common interface that bridges the differences between Ed-Fi Data Standards.
Of course, there are costs associated with those features. Simplification requires some loss of detail, and we oversimplified in a few places. Furthermore, data standard version 3 introduced enhanced flexibility by moving student demographic information to the student-to-education organization relationship, whereas data standard version 2 placed it on the more specific student-to-school relationship. This is a complex gap to bridge between the two data standards.
Improvements Introduced in Analytics Middle Tier Version 2.0
The primary goals of version 2.0 are to resolve key architectural problems and to introduce breaking changes. By breaking a few things now, we hope to avoid more breaking changes in the future. You can read all about the version 2.0 requirements and design in Tech Docs. In summary, version 2.0 will:
- Expand Data Standard support:
| Engine | Data Standard 2.x | Data Standard 3.1 | Data Standard 3.2 |
| SQL Server | Limited | Yes | Yes |
| PostgreSQL | No | No | Yes |
- Trim dimension view names from “xyzDimension” to “xyzDim”.
- Introduce a “Descriptor Mapping” process that decouples the SQL queries from the default Ed-Fi descriptor values.
- Replace the StudentDimension view, recognizing that (a) students can be enrolled in multiple schools at the same time, and (b) Ed-Fi Data Standard 3.2 supports recording student demographics for any type of education organization, not just for the school.
- Standardize separation between the core collection of views and use case-specific collections, paving the way for community contributions and deployment customization.
- Introduce a SchoolYear column in a few relevant views.
- Integrate documentation fully into Tech Docs.
This design has been informed by review comments and ideas from several members of the Ed-Fi Community, ensuring it aligns with real-world needs. Our development team is hard at work and we plan to release version 2.0 by March 31. To keep up with the current developments, request features, or to comment on the design, please contact me or stop by our #dev-analytics-middle-tier channel on Slack.
