Import CSV to Products (P)

Overview

This example demonstrates how to import product master data from a CSV file into Pricefx Products (object type P). The route reads a CSV file from a configured directory, unmarshals it using pfx-csv, and loads the data into Pricefx using pfx-api:loaddata with a mapper that maps CSV columns to product fields.

Files

routes/import-csv-to-products.xml

XML

<routes xmlns="http://camel.apache.org/schema/spring">
    <route id="import-csv-to-products">
        <from uri="file:{{import.products.directory}}?{{archive.file}}&amp;{{read.lock}}"/>
        <log message="Processing product file: ${header.CamelFileNameOnly}" loggingLevel="INFO"/>
        <split aggregationStrategy="recordsCountAggregation" streaming="true">
            <tokenize token="
" group="10000"/>
            <to uri="pfx-csv:unmarshal?skipHeaderRecord=true"/>
            <to uri="pfx-api:loaddata?objectType=P&amp;mapper=import-csv-to-products.mapper&amp;businessKeys=sku"/>
        </split>
        <log message="Product import complete. Total records: ${header.PfxTotalInputRecordsCount}" loggingLevel="INFO"/>
    </route>
</routes>

mappers/import-csv-to-products.mapper.xml

Note: Mapper filenames use the .mapper.xml suffix. The mapper ID must match the filename without .xml — e.g., import-csv-to-products.mapper.xml → id="import-csv-to-products.mapper".

XML

<mappers>
    <loadMapper id="import-csv-to-products.mapper">
        <body in="sku" out="sku"/>
        <body in="label" out="label"/>
        <body in="unitOfMeasure" out="unitOfMeasure"/>
        <body in="currency" out="currency"/>
        <body in="productGroup" out="attribute1"/>
        <body in="category" out="attribute2"/>
        <body in="brand" out="attribute3"/>
        <body in="status" out="attribute4"/>
        <body in="weight" out="attribute5"/>
    </loadMapper>
</mappers>

config/application.properties (snippet)

# Import directory for product CSV files
import.products.directory=/data/imports/products

How It Works

File Pickup: The file: component monitors the configured directory for CSV files. {{archive.file}} moves processed files to a timestamped archive; {{read.lock}} waits until the file is fully written before reading.
Splitting: The split with tokenize token=" " group="10000" breaks the file into batches of 10,000 lines for memory-efficient streaming processing. The recordsCountAggregation strategy tracks the total number of processed records.
CSV Unmarshalling: pfx-csv:unmarshal parses each batch of CSV lines into a list of maps. skipHeaderRecord=true tells the parser to use the first row as column names and skip it from the data.
Loading to Pricefx: pfx-api:loaddata sends the parsed data to Pricefx. The objectType=P targets the Products table. The mapper=import-csv-to-products.mapper references the mapper file that maps CSV columns to Pricefx product fields. The businessKeys=sku ensures upsert behavior based on the SKU field.
Completion Log: After all batches are processed, the total record count is logged via the PfxTotalInputRecordsCount header.

Common Pitfalls

Missing businessKeys: Without businessKeys=sku, every import creates new records instead of updating existing ones. Always specify business keys for upsert behavior.
Header mismatch: The CSV column names must exactly match the in attribute values in the mapper. A typo like SKU vs sku will result in null values.
Large files without streaming: Always use streaming="true" on the split and a reasonable group size to avoid loading the entire file into memory.
Encoding issues: If your CSV contains special characters, set the CamelCharsetName property to match the file encoding (e.g., UTF-8).
Ampersand in URIs: In XML route files, always use & instead of & in URI attributes.