On December 15, 2020, Data Import 1.1 was released, in coordination with major community contributions from Certica Solutions, an Instructure company. Data Import is a tool provided by Ed-Fi that allows for CSV to API data mapping, transformation and loading to the ODS / API where preferred native, real-time integration within education products do not exist. Typically, when an educational agency has homegrown or legacy systems, Data Import can be useful to fill the gap for useful data that otherwise has no path to incorporate within the agency’s data ecosystem.
Certica has been using the Data Flow (a precursor) and Data Import for their client projects that experienced the lack of a directly connected API solution. From their experience with the tool, they approached the Ed-Fi team in early 2020 and offered to make customizations to Data Import and contribute back the updates. The proposed customizations were primarily for multi-connection handling within Data Import, common where Ed-Fi adopters have multi-instances of the ODS / API in their environments, such as in managed ODS / API platforms as Certica offers or within LEA collaboratives where a service agency will host ODS / API instances for member districts. Multi-connection handling has been frequently requested by the Ed-Fi community and we were very happy to accommodate these contributions by the Certica/Instructure team. The Ed-Fi team worked closely with the contributors to agree on designs and provide code reviews for the contributions and provided support for other features as well as documentation as part of every Ed-Fi release.
A summary of the updates to Data Import 1.1 is listed below:
Multi-Connection Support
Data Import 1.1 has been updated to accommodate multiple key/secret pairs either on the same ODS / API instance or across multiple ODS / API instances. This is useful to provide data loads across different ODS / API instances, such as a state assessment result set, across various individual district ODS / API instances within a collaborative or within a managed ODS / API platform. This is also useful to provide different security contexts for loading data on the same instance, such as data loads across different data domains, such as assessment and finance. Both use cases for Data Import have been signaled by the Ed-Fi community via the Technical Advisory Group (TAG) and Ed-Fi Tracker issue reporting tool.
Supplemental Information for the Template Sharing Service
Data Import has a feature that allows for data mapping to be shared across implementations, known as the Template Sharing Service. These data mappings may have additional documentation or usage notes to help aid in the reuse of the mappings. Data Import 1.1 now allows for an additional web link to be included with the templates for documentation and instructions.
Port Number Specification for SFTP/FTPS Connections
Via the Ed-Fi Tracker queue, numerous implementers have signaled the need for optional port numbers with SFTP/FTPS connections running in varying configurations. Data Import 1.1 now includes support for optional port number specification to support this need.
We are excited to announce these developments as driven and contributed mostly by the Certica team for the benefit of Ed-Fi adopters and Data Import users. As our President Troy Wheeler recently highlighted in his summary of 2020 goals, “2020: Great Change and Great Opportunity”, this update to Data Import is an example of what is possible from working with great community leaders and the power of collaborations to deliver on community needs. We have plans to continue community contributions for Data Import and look forward to delivering on additional needs in releases in the future.
Data Import 1.1 can be found on Ed-Fi TechDocs within the Ed-Fi Tools section. Ideas and issues for Data Import and other Ed-Fi technology can be reported on the Ed-Fi Tracker. For information about Certica Solutions, their team and technical services, please visit: https://certicasolutions.com/.
