pfx-csv

Summary: Reference for the pfx-csv Camel component — CSV parsing (unmarshal) and generation (marshal) with configurable delimiters, headers, and quoting.

Overview

The pfx-csv component is a producer-only Camel component that converts between CSV text and in-memory Java collections. It wraps Apache Commons CSV and adds Pricefx-specific conveniences such as automatic header caching across split batches and BOM-aware encoding.

Three methods are supported:

URI Format

pfx-csv:method?option=value&option=value

Where method is one of marshal, unmarshal, or streamingUnmarshal.

Examples:

pfx-csv:unmarshal
pfx-csv:unmarshal?header=sku,label,price&skipHeaderRecord=true&delimiter=,
pfx-csv:marshal
pfx-csv:marshal?delimiter=;&quoteMode=ALL
pfx-csv:streamingUnmarshal?useReusableParser=true

Parameters

All parameters below are query options on the endpoint URI. They apply to all three methods unless noted otherwise in the description.

respectively (e.g.CRLFbecomes\r
). On **marshal**, defaults to System.getProperty("line.separator")if not set. | |nullString|String| _(none)_ | String to interpret asnullwhen reading / to write fornullvalues when marshalling. | |trim|Boolean| _(none)_ | Whether to trim leading/trailing whitespace from field values. | |ignoreSurroundingSpaces|Boolean| _(none)_ | Whether to ignore spaces surrounding field values. | |ignoreEmptyLines|Boolean| _(none)_ | Whether to skip empty lines during unmarshal. | |ignoreHeaderCase|Boolean| _(none)_ | Whether header matching is case-insensitive. | |allowMissingColumnNames|Boolean| _(none)_ | Whether to tolerate missing column names in the header record. | |trailingDelimiter|Boolean| _(none)_ | Whether to add a trailing delimiter at the end of each record. | |camelSplitIndexAware|Boolean|true| Whentrue, the component uses the Camel SPLIT_INDEXexchange property to determine split-batch behavior: header is read only from the first batch and cached for subsequent batches. | |forceSkipHeaderWhenPartOfSplit|Boolean|true| **marshal only.** Whentrueand the exchange is part of a split (batch index > 0), the header line is automatically skipped to avoid duplicating it in every batch. | |useReusableParser|Boolean|false| **streamingUnmarshal only.** Whentrue, wraps the input in a ReusableCSVParserthat supports re-iteration over the same stream (the stream is marked and reset). | |lazyStartProducer|Boolean|false` | (Advanced) Whether to defer producer creation until the first message is processed. |

Method Details

unmarshal

Converts CSV text (from the exchange body) into a List<Map<String, String>> (default) or List<List<String>> (when useMaps=false).

Input: The exchange body must be convertible to an InputStream (e.g. String, byte[], File, or InputStream).

Output: The exchange body is replaced with the parsed collection.

Header detection:

• If header is set, those names are used.

• If header is not set, the first line is parsed as the header.

• When used inside a <split>, the header from the first batch (split index 0) is cached and reused for subsequent batches via ExchangeCache.

Encoding: Determined by the CamelCharsetName exchange property. Defaults to UTF-8. BOM (Byte Order Mark) is automatically stripped.

Error reporting: If a parse error occurs inside a split, the error message includes the absolute line number calculated from the split index and split size.

marshal

Converts a List<Map<String, String>> (from the exchange body) into a CSV string.

Input: The exchange body must be a List<Map<String, String>>.

Output: The exchange body is replaced with the CSV string.

Header resolution:

• If header is set, those column names are used (and values are extracted from maps in that order).

• If header is not set, the keys of the first map are used.

Split awareness: When forceSkipHeaderWhenPartOfSplit=true (the default) and the exchange has SPLIT_INDEX > 0, the header line is suppressed. This prevents duplicate header lines when marshalling batches that are later appended to a single file.

streamingUnmarshal

Returns a lazy CSVParser (or ReusableCSVParser) as the exchange body instead of loading all records into memory. The caller is expected to iterate over it (typically via a downstream <split>).

Restrictions:

• Cannot be used inside a <split> block. If a SPLIT_INDEX property is detected, a NonRecoverableException is thrown.

Header detection:

• If header is set, those names are used.

• If header is not set, withFirstRecordAsHeader() is applied so the first CSV line is consumed as the header.

Usage Examples

Basic CSV Import (DMDS)

Read a CSV file, split into 5000-line batches, unmarshal, and load into a Pricefx Data Source:

<route id="csvImportToDatasource">
    <from uri="file:{{import.fromUri}}"/>
    <split>
        <tokenize group="5000" token="
"/>
        <to uri="pfx-csv:unmarshal?header=sku,label,price&amp;skipHeaderRecord=true&amp;delimiter=,"/>
        <to uri="pfx-api:loaddata?mapper=myMapper&amp;objectType=DM&amp;dsUniqueName=Product"/>
    </split>
    <onCompletion onCompleteOnly="true">
        <to uri="pfx-api:flush?dataFeedName=DMF.Product&amp;dataSourceName=DMDS.Product"/>
    </onCompletion>
</route>

Basic CSV Export

Fetch data from Pricefx and write it as CSV:

<route id="exportToCsv">
    <from uri="timer://fetchData?repeatCount=1"/>
    <to uri="pfx-api:fetch?filter=myFilter&amp;objectType=PX"/>
    <to uri="pfx-csv:marshal"/>
    <to uri="file:export?fileName=data.csv"/>
</route>

Tab-Delimited File

Use a tab delimiter with the \\t escape sequence:

<to uri="pfx-csv:unmarshal?delimiter=\\t&amp;header=sku,name,price&amp;skipHeaderRecord=true"/>

Semicolon-Delimited with Explicit Quote Mode

Common for European locale CSV files:

<to uri="pfx-csv:marshal?delimiter=;&amp;quoteMode=ALL&amp;quoteCharacter=&quot;"/>

Pipe-Delimited with Quoting Disabled

<to uri="pfx-csv:unmarshal?delimiter=|&amp;quoteDisabled=true&amp;header=id,name,value"/>

Strict Header Validation

Fail the route if the CSV file header does not exactly match the expected columns:

<to uri="pfx-csv:unmarshal?header=sku,label,price&amp;headerPolicy=STRICT&amp;skipHeaderRecord=true"/>

Streaming Unmarshal for Large Files

For very large CSV files that should not be loaded entirely into memory:

<route id="streamingImport">
    <from uri="file:{{import.fromUri}}"/>
    <to uri="pfx-csv:streamingUnmarshal?header=sku,label,price"/>
    <split>
        <simple>${body}</simple>
        <!-- each iteration yields one CSVRecord -->
        <to uri="pfx-api:loaddata?mapper=myMapper&amp;objectType=DM&amp;dsUniqueName=Product"/>
    </split>
    <onCompletion onCompleteOnly="true">
        <to uri="pfx-api:flush?objectType=DM&amp;dsUniqueName=Product"/>
    </onCompletion>
</route>

Batched Export with Header Only on First Batch

When splitting fetched data into batches and appending to a file, forceSkipHeaderWhenPartOfSplit (default true) ensures the header appears only once:

<route id="batchedExport">
    <from uri="timer://fetchData?repeatCount=1"/>
    <to uri="pfx-api:fetch?filter=myFilter&amp;objectType=PX&amp;batchedMode=true&amp;batchSize=5000"/>
    <split>
        <simple>${body}</simple>
        <to uri="pfx-api:fetch?filter=myFilter&amp;objectType=PX"/>
        <to uri="pfx-csv:marshal"/>
        <to uri="file:export?fileName=data.csv&amp;fileExist=Append"/>
    </split>
</route>

Unmarshal to Lists Instead of Maps

When map keys are not needed (e.g. positional data):

<to uri="pfx-csv:unmarshal?useMaps=false&amp;header=col1,col2,col3&amp;skipHeaderRecord=true"/>

Custom Record Separator (Windows CRLF)

<to uri="pfx-csv:marshal?recordSeparator=CRLF"/>

Tips and Edge Cases

Quoting and Escaping

• By default, fields containing the delimiter, quote character, or newlines are automatically quoted using the double-quote character (").

• Set quoteMode=ALL to force-quote every field, which is useful when downstream systems require it.

• Set quoteDisabled=true only if you are certain that no field values contain the delimiter character. When quoting is disabled, fields with embedded delimiters will corrupt the output.

• The escapeCharacter option provides an alternative to quoting: special characters are escaped with the given character instead of being enclosed in quotes. Do not combine escapeCharacter with quoteCharacter unless the underlying CSVFormat supports it.

Encoding and BOM

• The component reads the CamelCharsetName exchange property to determine character encoding. If not set, UTF-8 is assumed.

• BOM (Byte Order Mark) bytes at the start of a file are automatically stripped via BOMInputStream, so UTF-8-BOM files are handled transparently.

• To set encoding explicitly in a route, use pfx-io:setupCharset or set the property before the pfx-csv step:

<setProperty name="CamelCharsetName">
    <constant>ISO-8859-1</constant>
</setProperty>
<to uri="pfx-csv:unmarshal?header=sku,price&amp;skipHeaderRecord=true"/>

Split Batching Behavior

• When camelSplitIndexAware=true (the default), the component detects whether it is running inside a Camel <split> by checking the CamelSplitIndex exchange property.

• On unmarshal, the header is read from the first batch (index 0) and cached. Subsequent batches reuse the cached header, so skipHeaderRecord only skips the first line of batch 0.

• On marshal, forceSkipHeaderWhenPartOfSplit=true suppresses the header for batches with index > 0 to avoid duplicating it when appending to a file.

• Set camelSplitIndexAware=false if each split chunk is an independent CSV document with its own header.

Null Handling

• Use nullString to control how null values are represented. For example, nullString=NULL will write the literal string NULL for null map values and interpret the string NULL as null during unmarshal.

• Without nullString, null values are written as empty fields.

Common Pitfalls

1. Forgetting skipHeaderRecord=true on unmarshal when a header line exists and the header parameter is also set. Without it, the first data row will be the header line itself (parsed as data).

2. Using streamingUnmarshal inside a <split> -- this throws a NonRecoverableException. Use streamingUnmarshal before the split and iterate the returned parser inside the split body.

3. Delimiter in XML -- remember to XML-escape the ampersand in URI query strings: use & not &. For the tab character, use \\t (double-escaped in XML).

4. Header mismatch with headerPolicy=STRICT -- the actual header in the CSV must match the header parameter exactly (same order, same case, same number of columns). Any difference throws an IllegalArgumentException.

Related XSD Elements

The Pricefx XSD (pfx.xsd) defines higher-level XML elements that internally use CSV processing:

These elements are alternatives to using pfx-csv:marshal / pfx-csv:unmarshal directly in route <to> URIs. They are typically used in legacy or simplified integration configurations.

Source Reference