Many companies operating in the eCommerce, Retail, Customer Relationship Management (CRM) and Digital Marketing industries collect large amounts of data about customers at different touch points across the so-called consumer journey. Data analytics provides a powerful means to gain customer insights, but their effectiveness depends on the data they are fed with. Data collected by individual companies often provide a partial view on the customer journey and the analytical models often neglect factors that have an important impact on customers’ decisions.
EW-Shopp aims at supporting companies to gain deeper customer insights by helping them develop analytical services that use rich models, which also consider events that impact on customer decisions, such as weather, marketing campaigns, holidays, etc.. The main project objective is to develop a toolkit to facilitate all the data processing steps required to develop reliable weather and event-based data-driven services, including data preparation and enrichment, analytics, and visualization. In addition, the project aims at demonstrating the effectiveness of weather and event-based data analytics for developing valuable business services and the usefulness of the delivered toolkit.
The EW-Shopp toolkit has been released as a set of open source interoperable components, which can be reused even beyond the scope of project. Six business services based on analytical models that use data about events and weather have been developed by companies in the project consortium and made available for their business units and clients. Different tools have been used in the development of the different business services, which has helped testing the components of the toolkit also on data of large size compared to the kind of analytics developed in these domains.
On the technical side, the EW-Shopp toolkit consists of four main open source components: the integrated tools Grafterizer and ASIA to prepare the data and enrich them with information about events and weather, the QMiner library to develop predictive models, and the KnowAge suite to set up live dashboards.
To facilitate Data Enrichment, ASIA (a semantic table annotation tool) has been integrated into Grafterizer, which has been extended to support graph data storage with ArangoDB. A pool of data reconciliation and extension services have been developed for ASIA to support semantic Data Enrichment. A solution to execute the enrichment transformations in batch mode on a scalable cloud-based infrastructure and to schedule pre-fetching of third-party data have been developed to support large-scale enrichment processes. To support Data Analytics, QMiner has been extended with tools and services for building and deploying event and weather-based analytic models. A keyword clustering methodology has been developed to classify keywords or short phrases to improve data analytics for Digital Marketing. As for Data Visualization, several customized cockpits and customized views have been developed using the KnowAge suite.
APIs, ontologies and mappings have been defined to facilitate enrichment and analytics with third-party data sources: for weather data (ECMWF and OpenWeatherMap); for event data (Media Attention API based on the Event Registry and the Event Ontology and APIs); for product data (GfK product catalog). Finally, a methodology to support cooperation in multi-party data analytics projects has been defined to support cooperation among partners.
On the business side, six business services that harness the power of event and weather-based analytics have been developed in the context of three business cases. The services have been developed after tests with five “pilots” developed during the first period.
- Enriched Purchase Insights is a widget informing users of a comparison shopping engine when a purchase is sensible.
- Performance Insights is a tool for planning pricing and campaigns targeted to retailers. Both services use weather information but exploit in particular price-related events tracked in the comparison shopping engine.
- Weather and Event-aware BI Sales Strategy Advisor is a tool enabling maximization of marketing budgets and optimization of workforce that leverages weather and event-based prediction of foot traffic and sales.
- A Workforce Optimization Management Service has been developed to leverage weather and several events to optimize the shifts of the agents used in contact centers, which support CRM and marketing campaigns.
- The Event and Weather Dashboard helps businesses optimize their ROI based on quantitative historical analysis of customer flows vs. seasonality and the prediction of foot traffic during marketing events.
- Campaign Booster is a set of modules to predict the performance of keywords in digital marketing campaigns based on weather and media attention data.
Each company has developed its exploitation plan to specify how its service will be further developed after the end of the project. Finally, toolkit-related solutions have been disseminated with scientific papers, demos and tutorials at best venues in the area of semantic computing.