pfx-api:flush

pfx-api:flush

Flush data from a Datamart data feed to a data source. This step is mandatory after loading data into a DMDS (Datamart Data Source) via loaddata or loaddataFile.

Overview

The flush method moves data that has been loaded into a data feed into its associated data source. In the Pricefx data pipeline:

1. Data is loaded into a data feed (via loaddata or loaddataFile with objectType=DMDS)

2. The data feed is flushed to the data source (via flush)

3. The datamart can then be refreshed to incorporate the new data (via refresh or calculate)

Without the flush step, data loaded into a data feed will not appear in the data source or datamart.

URI Format

pfx-api:flush?dataFeedName=MyFeed&dataSourceName=MyDS

Parameters

Examples

Basic Flush

<route>
    <from uri="direct:flushDM"/>
    <to uri="pfx-api:flush?dataFeedName=MyFeed&amp;dataSourceName=MyDS"/>
</route>

Incremental Flush

Flush only data newer than a specific date:

<route>
    <from uri="direct:flushIncremental"/>
    <to uri="pfx-api:flush?dataFeedName=MyFeed&amp;dataSourceName=MyDS&amp;incrementalDate=2026-01-01"/>
</route>

Full Load-Flush Pipeline

A typical DMDS pipeline: truncate the data source, load new data, then flush:

<route>
    <from uri="direct:fullDSLoad"/>
    <!-- Truncate existing data -->
    <to uri="pfx-api:truncate?objectType=DS&amp;targetName=MyDataSource"/>
    <!-- Load new data -->
    <to uri="pfx-api:loaddata?objectType=DMDS&amp;dsUniqueName=MyDataSource&amp;mapper=dsMapper&amp;businessKeys=sku,region"/>
    <!-- Flush to data source -->
    <to uri="pfx-api:flush?dataFeedName=MyFeed&amp;dataSourceName=MyDataSource"/>
</route>

Load from File, Flush, and Refresh

<route>
    <from uri="file:input/datasource?noop=true"/>
    <to uri="pfx-api:loaddataFile?objectType=DMDS&amp;dsUniqueName=MyDataSource&amp;mapper=dsMapper&amp;businessKeys=sku,region&amp;waitForCompletionStrategy=Always"/>
    <to uri="pfx-api:flush?dataFeedName=MyFeed&amp;dataSourceName=MyDataSource"/>
    <to uri="pfx-api:refresh?targetName=MyDatamart"/>
</route>

Common Pitfalls

1. Forgetting to flush -- This is the most common mistake with DMDS loads. Data loaded via loaddata or loaddataFile with objectType=DMDS sits in the data feed until flushed. If you skip the flush, the data source remains unchanged.

2. Wrong dataFeedName or dataSourceName -- The names must match exactly what is configured in Pricefx. A typo will cause the flush to fail silently or with an error.

3. Flushing before load completes -- If using loaddataFile with waitForCompletionStrategy=Never, the load may still be in progress when the route reaches the flush step. Use waitForCompletionStrategy=Always to ensure data is fully loaded before flushing.

4. Incremental date format -- The incrementalDate must be in a format recognized by Pricefx (typically yyyy-MM-dd). Invalid date formats will cause errors.

5. Not refreshing after flush -- Flushing moves data to the data source, but the datamart itself may need a refresh (via pfx-api:refresh) to recalculate derived columns and make data available in analytics.