In Pricefx, importing data into the Analytics requires an understanding of how the system handles field mapping, existing records, and database locks. This article explains the behavior of the /datamart.importfile/ endpoint and the implications of direct data loads to a Datamart.
Direct Data Loading vs. Data Feeds
When using the /datamart.importfile/ endpoint, Pricefx loads the data straight to the Data Source. This process differs from the standard Data Feed and Data Load (Flush) workflow.
Because this endpoint bypasses the standard Data Feed layer, it also circumvents the locking mechanisms typically in place for Data Loads. This can lead to conflicts if an automated orchestration or another Data Load is running simultaneously.
Direct imports to a Datamart may break orchestrations because they do not respect system locks. It is strongly recommended to ensure no other data processes are active for the target Datamart during a direct import.
Field Mapping and Null Values
Pricefx uses the file header to match incoming data to Datamart fields. The following rules apply during the import process:
-
Missing Fields – If the provided import file is missing one or multiple Fields that exist in the target Datamart, those fields are populated with nulls.
-
Unmatched Columns – If a column is provided but does not match any field in the Datamart definition, the system ignores the extra column.
Upsert Behavior
When you import a row that matches an existing Data Source row (based on the business keys), the system will update the whole row.
This means that any values currently in the database for that row will be replaced by the values in the import file. If the import file contains null values for certain columns, those null values are imported and will overwrite existing data in the database. This behavior is often confusing for the user who expects only the provided non-null values to change.
You cannot perform a partial update of a row by omitting columns. If a column is missing from the file, the system erases the existing data in the database for that field by replacing it with a null value. This is a consequence of the Field Mapping and Null Values rules.
Recommendations for Data Integrity
To ensure data consistency during Datamart imports:
-
Verify Headers – Ensure your file headers exactly match the Datamart field names to avoid unintended null population.
-
Check Business Keys – Confirm which fields are defined as keys in the Datamart to understand which rows will be updated versus which will be created as new entries.
-
Coordinate Imports – Schedule direct API imports outside of standard Data Load windows to avoid breaking orchestrations.