Data Feeds

Data Feed table allows you to upload raw data which is potentially not in a good condition (bad formatting, ...) and can include unwanted or duplicate rows. Data in the Data Feed is then cleaned with a Data Load process and uploaded into a Data Source.

A Data Feed is created automatically for each deployed Data Source, with an auto-generated name, generic field names (attributeX) and a label (DS.label). The data type is always text to prevent data type conversion issues (manipulation with the data comes later, in Data Loads).

Columns

This list provides an overview of the key attributes and status information for each Data Feed.

Name – This column indicates the name of the Data Feed, with no spaces.
Label – This column displays the label of the Data Feed.
# Rows – This column shows the number of rows in the associated Data Feed.
If the row count is very large, it may take some time for this value to update.
Last Load Date – This column indicates the date of the most recent change to the Data Feed's data.
Note that executing the Purge Cache operation automatically updates the Last Load Date of the associated Data Feed. By modifying the Last Load Date, the system recognizes that older cache files are invalid.
Last Update Date – This column reports the date of the most recent change to the metadata of the Data Feed (e.g., adding or modifying columns).
Last Update By – This column shows the user who performed the last change to the Data Feed's metadata.

Click the Data Feed's label to display its details. Each details page has two tabs:

• Fields tab – Here you can:

  • Edit the Data Feed's label.

  • Define the Field Separator if the data file that you intend to upload uses other than the default delimiter (comma). You must (re)deploy the Data Feed for this change to take effect.

  • Add fields from a data sample (CSV, XLS) to create a data structure for later data import. Use the Field Picker section to select which from the automatically identified fields you want to add to your Data Feed.

  • Import or export Data Feed definitions in the JSON format.

  • Delete selected fields.

   For a comprehensive example of using this tab, see How to Upload CSV Data into Analytics.

• Data tab – Here you can see all uploaded raw data and upload new data using the Data Load action. You can choose from:

  • Truncate – Deletes all/filtered rows from the Data Feed.

  • Upload – Loads an XLS/XLSX sheet or a CSV/TXT file. Note that the size limit is 1 GB.

  • Mass Edit – Allows you to apply mass changes to editable attributes.
    Whenever you perform a Mass Edit operation in Datamart, remember to run a default Publish Data Load to make the edited data available in your queries. For more information about Data Flow see Data Manager.

  • Status History – Shows information on the Data Feed status history – a list of Data Feed actions that have been executed.

  • Export Excel, CSV – Exports the Data Feed data to the XLSX or CSV format. For data tables containing a large amount of data, asynchronous download is available.

   If you have a large data set, you can disable automatic loading of the data grid content. Then you can set up a filter first and download only a subset of the data.

Limit the Number of Columns in Data Feed

Set the soft and hard limit for the number of columns. You can change the setting in pricefx-config.xml in the warnFieldCount and maxFieldCount properties.

Example

<warnFieldCount>150</warnFieldCount>
<maxFieldCount>500</maxFieldCount>

Default Field Limits

As part of the 16.0 release, Default Field Limits were introduced to provide the following benefits:

• Improved database and query performance – Very wide tables with a large number of columns negatively impact performance, particularly persisted Flex Configs and Data Sources or Models with many fields.

• Clearer guardrails for customers – Previously, the limit was arbitrary. Explicit warnings and errors now clearly communicate that field count has a direct impact on system performance.

• More realistic limits without breaking existing configurations – Earlier limits were intentionally set very high to avoid invalidating existing models. The new limits are lower and more reasonable, while backward compatibility is preserved through one-time exemptions created during migration.

You are nearing the recommended maximum (81–100). The effective field count is (81-100 + 9 System Fields) due to automatically generated fields. To keep performance stable, consider reviewing fields.

The effective field count (≧101 + 9 System Fields) exceeds the limit (100). Deployment isn’t possible until the number of fields is reduced.

The Default Field Limits can be modified later on an individual basis for each data source using the paMaxFcFieldCountExemptions Advanced Configuration Option.

Note on Last Update Date

When you run the operation Data Feeds to Data Source FLUSH and lastUpdateDate is present in the source (DF) data, the date is copied into the Data Source.

When lastUpdateDate is missing in the Data Feed data, the usual method applies – lastUpdateDate is set to the job's calculation start date.