18. August 2021

Modeling Import and Export-Mappings with neo4j

how neo4j can be used to handle mappings for data imports and generation of "views"

Using our Graph Designer we've built a graph based data platform for managing product data with neo4j. One of the main issues when orchestrating data from different source systems (e.g. PDM, CRM or ERP) is to handle quite a lot of data imports. The different systems do have different data models and use their own terminology.
We needed to find a way to map incoming data to our own graph structure and to "translate" the external to an internal terminology.

But importing data is only one side of the coin. You also need to be able to provide many different "views" of the stored data. Beside the department specific, internally used data formats - e.g. for data sheets or working lists - you probably need to have an eCl@ss "view" and several customer specific output formats (e.g. in XML oder JSON).

These "views" are generated on the fly based on the graph data model.

To be flexible enough and to guarantee a high performance on processing data imports and view generation, we decided to handle all data mappings for internal and external formats and terminology in neo4j as well.

The Mappings can be used both for data import and export (view generation).
The internal schema for handling product data properties is based on "DataAtoms" - the most fine grained way to manage data in a neutral format. External data has to be disaggregated and distributed to the DataAtoms during import. After that you can use the DataAtoms to generate every needed view on the fly and fill Forms, Grids, DataInterfaces, JSON sequences and XML templates.

By using our Graph Designer Toolbox we configured an own module for handling data mappings. This allows us to keep all mappings up to date and to see the dependencies and relationships ("what is used where?") between DataAtoms and their usage.

Interested in learning more details? See the details in German or in English