Date Range Filters | Pricefx Knowledge Base

Overview

Date range filters are essential for incremental (delta) export patterns. By filtering on date fields like lastUpdateDate, you can fetch only records that have been modified since the last export run, dramatically reducing data transfer volumes and processing time.

Complete XML Examples

Greater Than (Records Modified After a Date)

Fetch records updated after a specific date:

XML

<filters>
    <filter id="recentlyUpdatedProducts">
        <and>
            <criterion fieldName="lastUpdateDate" operator="greaterThan"
                       value="2024-01-01T00:00:00"/>
        </and>
    </filter>
</filters>

Dynamic Date from Groovy Expression

Use a Groovy expression to compute the date value at runtime (e.g., from a header set earlier in the route):

XML

<filters>
    <filter id="incrementalExportFilter">
        <and>
            <criterion fieldName="lastUpdateDate" operator="greaterThan"
                       value="groovy:headers.lastExportedDate"/>
        </and>
    </filter>
</filters>

Date Range (Between Two Dates)

Combine greaterThan and lessThan to define a window:

XML

<filters>
    <filter id="dateRangeFilter">
        <and>
            <criterion fieldName="lastUpdateDate" operator="greaterThan"
                       value="groovy:headers.startDate"/>
            <criterion fieldName="lastUpdateDate" operator="lessThan"
                       value="groovy:headers.endDate"/>
        </and>
    </filter>
</filters>

Using iBetween for Date Ranges

The iBetween operator provides a compact syntax for range filtering:

XML

<filters>
    <filter id="betweenDatesFilter">
        <and>
            <criterion fieldName="attribute1" operator="iBetween"
                       start="10" end="20"/>
        </and>
    </filter>
</filters>

Incremental Export Route Pattern

A complete incremental export route that tracks the last export timestamp:

XML

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:pfx="http://www.pricefx.eu/schema/pfx"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans.xsd
           http://camel.apache.org/schema/spring
           http://camel.apache.org/schema/spring/camel-spring.xsd
           http://www.pricefx.eu/schema/pfx
           http://www.pricefx.eu/schema/pfx.xsd">

    <pfx:filter id="incrementalProductFilter">
        <pfx:and>
            <pfx:criterion fieldName="lastUpdateDate" operator="greaterThan"
                           value="groovy:headers.lastExportedDate"/>
        </pfx:and>
    </pfx:filter>

    <pfx:loadMapper id="productExportMapper">
        <pfx:body in="sku" out="sku"/>
        <pfx:body in="label" out="name"/>
        <pfx:body in="attribute1" out="price"/>
    </pfx:loadMapper>

    <routeContext id="incrementalExportRoutes" xmlns="http://camel.apache.org/schema/spring">
        <route id="incrementalProductExport">
            <from uri="direct:incrementalExport"/>

            <!-- Read last export timestamp from a tracking file or property -->
            <setHeader name="lastExportedDate">
                <simple>${exchangeProperty:lastRunTimestamp}</simple>
            </setHeader>

            <log message="Fetching products updated since ${header.lastExportedDate}"/>

            <toD uri="pfx-api:fetch?objectType=P&amp;filter=incrementalProductFilter"/>
            <toD uri="pfx-model:transform?mapper=productExportMapper"/>
            <toD uri="pfx-csv:marshal?delimiter=,"/>
            <to uri="file://{{rootFolder-outbound}}?fileName=products-delta.csv"/>

            <log message="Incremental export complete"/>
        </route>
    </routeContext>

</beans>

How It Works

greaterThan / lessThan operators compare the field value against the provided value. For date fields, Pricefx performs date-aware comparison.
Dynamic values use prefixes:
- groovy: -- evaluates a Groovy expression at runtime (e.g., groovy:headers.lastExportedDate).
- simple: -- evaluates a Camel Simple expression (e.g., simple:${exchangeProperty.startDate}).
Date format: Pricefx typically expects ISO 8601 format (yyyy-MM-dd'T'HH:mm:ss). Ensure your dynamic date values match this format.
iBetween uses start and end attributes instead of value, providing inclusive range matching.

Common Pitfalls

Date format mismatch: If the runtime date value does not match the expected format, the filter silently returns no results or all results. Always verify the date format.
Timezone handling: lastUpdateDate in Pricefx may be in UTC. Ensure your comparison date accounts for timezone differences.
Null lastExportedDate: On the first run, lastExportedDate may be null. Handle this with a fallback (e.g., a very old date like 1970-01-01T00:00:00) or an empty filter for the initial full load.
greaterThan vs greaterOrEqual: greaterThan is exclusive. If you need inclusive comparison, check if your Pricefx version supports greaterOrEqual.
Performance: Date range filters on indexed fields like lastUpdateDate are efficient. Filtering on non-indexed custom attributes may be slow on large datasets.