Technical User Reference (Optimization - Shelf Price)

This logic contains some functions needed specifically for this Accelerator, such as reading its configuration from the application settings, applying the user filters in each part of the model, providing the problem description for the OE, preprocessing the data for the charts, and many small helpers for the charts rendering.

• The elements ParametersUtils, LabelsUtils, and TablesUtils contain the names of many elements and fields of the models.

• The elements DefinitionUtils, ScopeUtils, ConfigurationUtils, and ResultUtils contain functions specific to the corresponding step.

• The ProblemDescriptionDefinition element returns the problem description in a map. The content of the problem description is detailed in Problem Description. The specific Shelf Price Optimization problem is described as a chart in the logics repository of the accelerator. The element ProblemDescriptionUtils contains the names and field definitions for the problem description field.

This lib is the place where to change the names inside the model, to reflect the user business vocabulary. Here you can also write a function to be used in different places of the model class.

Any modification of the problem modeling and type of constraints to apply or objectives to reach implies a modification of the Problem Description, thus ProblemDescriptionDefinition.groovy should change accordingly.

⚠ If the problem description changes, do not forget to check if it is necessary to change or create some Problem tables.

These logics define the Data Source and the mapping of the entries for the transactions, in the configurator. The main logic calls the configurator and the code for the dashboard portlets.

Two portlets show the data that will be materialized in the model (table sales) and the filtered-out rows.

The mapping could be changed if a field is removed or added.

These logics define the Data Source and the mapping of the entries for the stock data, in the configurator. The main logic calls the configurator and the code for the dashboard portlets.

Two portlets show the data that will be materialized in the model (table stock) and the filtered-out rows.

The mapping could be changed if a field is removed or added.

These logics define the Data Source and the mapping of the entries for the competition data, in the configurator. The main logic calls the configurator and the code for the dashboard portlets.

Two portlets show the data that will be materialized in the model (table competition) and the filtered-out rows.

The mapping could be changed if a field is removed or added.

This logic validates some prerequisites and materializes the Data Sources in the three tables sales, stock, and competition.

The prerequisites are the consistency of the types of the product and store fields in the different sources and no negative values for the quantities and the prices.

There is no aggregation in the stored data, except to provide summary values, like revenue per product or per store.

The tables sales, stock, and competition are created in the model and can be used in further steps. These tables are the main connection to external data.

This calculation is the main connection to the external data and most often requires to be modified to accommodate the specifics of the customer data, such as mandatory filters. If the waterfall structure is not the standard one, maybe some columns should be added to the sales table. In this case, you may also change the user inputs to define this mapping, too (see Technical User Reference (Optimization - Shelf Price) | Definition Step).

These logics let the end user choose the filters to scope the optimization, through the configurator SP_Sco_Eval_Scope_Configurator. It means, for example, creating a model with a scope filtered on a set of product groups or a minimum store revenue. On the right, it displays some charts to evaluate how the scope is defined.

The scope applies to the Sales data only. The Competition and the Stock data are used as far as they join to the scope of the sales.

• A map of filters that are called later in the code by libs.SP_Lib.ScopeUtils.inputs(model). The source data query, filtered on the scope, is given by this sample code, in the next parts of the code:

def scope = libs.SP_Lib.ScopeUtils
scope.scopedSalesQuery(model, scope.inputs(model))

• A dashboard with information, charts, and tables that summarize the scope of transactions taken into account according to the user filters.

The main reason to modify these logics is to enrich the scope outputs with the data from the stock or the competition sources or with different filters. The filtering options that we expose to the user are modified here.

This logic materializes the aggregated table corresponding to the exact scope of the optimization. The reason for this logic is that with large data, a time overflow could happen with filtering on the flow to display the user dashboards.

A table called scoped_sales, to be used as a reference for the following steps (and not the aggregated table called sales, outputted by the Scope calculation).

It would be possible to add, remove, or change the table fields. Be careful, if you want to change the query to compute this table, the change has to be done in the Shelf Price library, libs.SP_Lib.ScopeUtils.scopedSalesQuery, because the same query is also used to display the Scope step dashboard.

The Configuration tabs are used to retrieve the user objectives and as such, are configurators. They contain all the information needed to guide the optimization process by setting the constraints and the goals to reach. The separation into five tabs is mainly for user experience purposes. Each tab has a different meaning.

The user inputs are stored to be aggregated in the following step with the rest of the data. The problem tables of the Optimization Engine use the configuration input to define the optimization goals.

The smallest common change is the change of the default values.

It is also quite common to change these logics by adding or modifying the constraints and objectives in the problem; for example, adding targets at some levels or setting thresholds to keep some values in check. These modifications are needed but not sufficient as the problem modeling itself must be changed to take them into account. It is possible to change the number of tabs in the configuration step, but then the Model Class definition has to be modified too.

The goal of this calculation is to create a simulation whose results will be used to initialize the optimization run in the next calculation. To do so, we need to create a Problem Description that details the structure of the problem to solve by the Optimization Engine and to give endpoints for the OE to get the data of the problem. The previous steps will change the problem by altering its scope and changing the objectives, and the data will be fed directly to the OE thanks to the model tables.

This step consists of:

• Validation of the elasticity model.

• Data manipulation to prepare the last tables needed by the OE. These logics are prefixed by “Create_” and create the model tables prefixed by “Problem_” that act as an endpoint for the OE. Be careful, their names follow a strict format: These endpoints must be named according to the Problem_nameOfTheSpace_nameOfTheScope present in the ProblemDescription.groovy and return the corresponding data. The library function problemTable creates automatically such a well-named problem table. The behavior of the OE and its way of reading data from endpoints highlight the need for a well-thought-out Scope step. Creating tables of the needed data, already computed and aggregated, implies being well aware of "where is the data I need" and "how do I need to transform it". That is why it is normal to refactor and improve the create table elements during the development of ProblemDescription.groovy.

• Run.groovy element – Contains the code that handles the problem description. It takes the description of the problem and the advanced parameters user inputs, and triggers the simulation job thanks to model.startJobTriggerCalculation. The run will create tables prefixed by “Initialization”.

The problem description is used to configure the instantiation of a job running an OE. The OE has to have access to the correct endpoints to get the data and to know where to write back the results when the computation is finished.

The Groovy code does some preparation work. It creates Problem_nameOfTheSpace_nameOfTheScope tables – data manipulation to prepare the last tables needed by the OE. These logics are prefixed by "Create_" and create the model tables that act as an endpoint for the OE. Be careful, their names follow a strict format.

At the end of its run, the OE will write a set of model tables containing its results. This writing is done directly by the OE job and is not related to a Groovy logic.

• The tables prefixed by “Results_” present the state of the objectives and constraints at the end of the optimization.

• The tables prefixed by “Simulation_” present the value of computed variables marked as exposed in the description, typically including values of interest.

Any modification of the problem modeling and type of constraints to apply or objectives to reach might imply a change or creation of some Problem tables.

In some cases, it could be useful to change the OE image and/or the OE tag that the job trigger refers to. Their values are in the element Run.groovy.

The goal of this calculation is to create a simulation and an optimization run and retrieve their results. To do so, we use a Problem Description that details the structure of the problem to solve by the Optimization Engine and to give endpoints for the OE to get the data of the problem. The previous steps will change the problem by altering its scope and changing the objectives, and the data will be fed directly to the OE thanks to the model tables.

This step consists of:

• Create_Global_All.groovy element – Prepares the last table needed by the OE based on the results of the previous Initialization calculation.

• Run.groovy element – Contains the code that handles the problem description. It takes the description of the problem and the advanced parameters user inputs, and triggers the two jobs thanks to model.startJobTriggerCalculation. Each run will return prefixed tables of similar structures. The jobs will run in parallel. The first one is the optimization itself and its outputs are prefixed by “Optimized”. The second one is a simulation: it simulates the first state of the optimization and will be a reference to compare before/after values in the results dashboards. Its outputs are prefixed by “Current”. The job type (optimization vs. simulation) is indicated by the input parameters of the model.startJobTriggerCalculation function.

 Once the problem description is created, it is used to trigger the instantiation of a job running an OE configured by this file. The OE has to have access to the correct endpoints to get the data and to know where to write back the results when the computation is finished.

The Groovy code does some preparation work. It creates Problem_Global_All table – data manipulation to prepare the last table needed by the OE. All the tables prefixed with Problem and created by the logic SP_Res_Calc_Run_Initialization are also needed by the OE.

A Groovy element also reads the problem description to retrieve a list of parameters used during the postprocessing step to reformat the Glassbox data.

At the end of its run, the OE will write a set of model tables containing its results and the Glassbox information needed to understand why this solution was used. This writing is done directly by the two OE jobs, simulation and optimization, and is not related to a Groovy logic.

• The Glassbox table provides optimization indicators for each pair of instantiated value finder – criterion. The simulation job does not create any Glassbox table.

• The tables prefixed by “Results_” present the state of the objectives and constraints at the end of the optimization.

• The tables prefixed by “Solution_” present the raw values that the system was meant to find (declared as Value_Finder in the Problem Description). The simulation job does not create any Solution table.

• The tables prefixed by “Simulation_” present the value of computed variables marked as exposed in the description, typically including values of interest such as forecasted quantities.

In some cases, it could be useful to change the OE image and/or the OE tag that the job trigger refers to. Their values are in the element Run.groovy.

This calculation retrieves the outputs of the RunOptimization logic and reformats them to provide tables that can be used to show user-friendly optimization results. Each element stores one model table or some similar tables.

• StoreGlassboxTables logic calls the Optimization Common Library (internal access only) to calculate aggregated metrics on the optimization agents criteria and value finders.

• Various other tables are created to make the calculation of the dashboards of the Results step faster.

This calculation writes a collection of tables:

• GlassboxVF_ tables – Their names are built as GlassboxVF_NameOfTheSpace_NameOfTheValueFinder, there is one table by value finder key (i.e. type of value finder). These tables store the overall values of each value finder.

• GlassboxCriteria_ tables – Their names are built as GlassboxCriteria_NameOfTheSpace_NameOfTheCriterion, there is one table by criterion key (i.e. type of criterion). These tables store the overall values of each criterion.

• Glassbox_AggregatedMetrics table – Summarizes the global interaction indicators between each value finder key and each criterion key.

• Glassbox_VFs_by_Key table – Summarizes the global overall indicators of each value finder key.

• Glassbox_Criteria_by_Key table – Summarizes the global overall indicators of each criterion key.

• stock_coverage table – Calculates the stock coverage of each product Pareto category, for each period of time.

• historical table – Represents the state for all periods of time before the optimization.

• forecasted table – Has the same structure as the historical table, but provides the optimized values for all future periods of time.

• details_product_store_period table – Contains all the information at product x store x period level for all periods of time, including more extended and unit values than there is in historical and forecasted tables. It is used as the main source of data for most of the Results step charts and tables.

• impact_competition_positioning table – Gives the comparisons metrics between the competition prices and the optimized prices.

• quantities_for_fallback_by_product – Gives the parameters, by product and store, to use the fallback elasticities.

If the problem description has been changed, the Create_Forecasted element, which refers to many of the tables created by the OE, may change too.

The other most common reason to change the logic is to reformat some data to ease the work of providing charts in the Result step tabs.

This tab exposes the results of the OE execution in an HTML summary and some complex graphs, to analyze the forecasted values. The data are filtered according to the user entries set by SP_Res_Eval_Filter_Configurator. This filter configurator is shared by both Impact and Details tabs.

The details of the provided charts are in Result Description (Optimization - Shelf Price).

Add, modify, or remove visualizations. This step is one of the most straightforward ones and its modification should not impact the previous steps.

This tab clearly shows the output data of the optimization. This way, the user can see the impact of the optimization on every adjusted value. The data are filtered according to the user entries set by SP_Res_Eval_Filter_Configurator. This filter configurator is shared by both Impact and Details tabs.

Some data tables. For details see Result Description (Optimization - Shelf Price) | Details.

Add, modify, or remove table outputs. In practice, if the problem description changes, this tab should provide tables to reflect the changes in the variables and constraints. The tables displayed here should allow the end user to access any useful information related to the optimization.

These tabs expose the technical state of the OE execution at the end of the process. It is a development tool to help finetune the model.

This logic displays charts that show the satisfaction, influences, impacts of the value finders and the criteria, initial movements of the value finders, and evolution of the criticality during the process of optimization. For details see Result Description (Optimization - Shelf Price) | Glassbox.

In general, there is no reason to change this dashboard. It is mainly a series of calls of the Optimization Common Library (internal link) functions.

This tab mocks the model evaluation.

The evaluation is used to access model results from outside of the model itself; for example in another logic. The first step is to use api.model("ModelName") to get the model and then use the function evaluate on it to retrieve an answer depending on the nature of the given parameters. The code needed to get these results is:

def model = api.model(“TheModelUniqueName”)
def results = model.evaluate(
  “query_results”, 
  [
    product: "someProductID",
    period_start_date: "aDate",
    store: "someStoreID",
  ]
)["Data"]

The product ID, Period Start Date, and Store ID are all optimal keys. The output of the evaluation is a result matrix with columns product, period_start_date, store, and unit_shelf_price, filtered on the values provided by the input parameters.

The output of the evaluation is a result matrix with columns product, period_start_date, store, and unit_shelf_price, filtered on the values provided by the input parameters.

If the optimization model depends on new fields and if it provides new values, the evaluator should be changed to take them into account.

It is also possible to add other evaluators to the same model.