Back to search


E!252: Real-time Sensor Data Mining Service

Alternative title: E!252: Tjeneste for sanntidsutvinning av sensordata

Awarded: NOK 5.2 mill.

Project Manager:

Project Number:


Project Period:

2022 - 2024

Funding received from:


Partner countries:

There are several tools and methods available in the space of data management, preparation, and cleaning. This is natural considering that it is both necessary and time-consuming. The product range spans from complete AI platforms to simple algorithms and scripts. However, data processing frameworks that cover all steps from raw time series measurement to structured and processed datasets are rare and insufficient. In the academic literature, much of the focus is placed on outlier detection, missing values, and data imputation. A recent survey by Wang and Wang[1] lists methods and implementations currently regarded as cutting-edge. Wang and Wang[1] write "There are many tools or systems for data cleaning, but they are not effective on time series cleaning problems.", and emphasize the need for more research and tools. They highlight Cleanits by Ding et al., which focuses on addressing missing, inconsistent, or abnormal values. A tool for data alignment, summarization, and processing similar to Squashy was not found during the writing of this project proposal. Nor has it been found during any of the patent applications filed related to the Squashy technology. We will hence adapt and 'productify' our proprietary production data mining tool (Squashy) for industries beyond Oil & Gas. The new standalone tool will support data scientists at industrial companies to leverage and operationalize the potential of their data, while also catering to data scientists at third-party AI/data science companies providing analytics services to such industrial companies. Squashy has been in operation since 2014 within O&G and shows great potential for applicability in other industries. As a first step in 2022, we are looking to expand into adjacent industries with similar sensor data workflows (mapped with our partners at the Boston Consulting Group) via questionnaires and meetings with BCG customers and Solution Seeker leads in different countries.

The DataMine project has significantly contributed to increasing Solution Seeker’s total addressable market and diversification, given the associated risks and the shift towards low-carbon investments worldwide. Along with our project partners, we have assessed the feasibility – and estimated the potential value impact – within industries such as mining, pulp and paper/forestry, drilling as well as fish farming. We have conducted two successful pilots during the project, which included testing our data mining algorithm, Squashy, both on historical data and on live data streams from industry partners including precision farming (sustainable forestry in Brazil) as well as a highly instrumented salmon farming partner (onshore in Norway). The technology has indeed proven to provide value in environments with sensor data in industries that go beyond the legacy O&G business. The ability to stream, mine, visualize, analyze, and extract value in real-time has shown to be a key competitive advantage for our industrial partners also with very relevant commercial value for Solution Seeker. One of the pilots has already progressed into a long-term commercial project ( and we are negotiating contract terms for a commercial project with our partner from the second successful pilot in Brazil. Squashy has proven to offer two main advantages to our partners. Firstly, it automates and quality controls the data mining. This is crucial in any real-time decision-making setting as sensor values are continuously streamed. Secondly, it provides higher-quality models and results due to domain-specific processing techniques, such as data and event categorization, alignment, and summarization.

Businesses today are awash with data but are not able to extract real value without proper data preparation. This process unnecessarily consumes a significant share of data scientists’ time (80% according to a recent analysis by The Economist). Solution Seeker's proprietary data mining tool (Data Janitor) has two main features; firstly it automates and quality-controls the data mining and, secondly, it provides higher quality models and results due to a framework for implementing domain-specific processing techniques, like event categorization, alignment and summarization. The Eurostars funding enables us to further productize our Data Janitor into a standalone product and also generalize it into a framework that can address more types of industrial data in addition to our core domain (Oil & Gas). Today we are using it as an internal tool for our own data scientists, but we truly see the potential benefit of extracting it as a standalone product catering to different heavy industries dealing with time series streamed in real-time from sensors. Together with our partners at The Boston Consulting Group (BCG) in France, we discovered that data mining is still a hurdle for many data scientists and we believe further development of our Data Janitor can address some of these issues in a meaningful way. BCG is helping us to identify industries, use cases and customers through their unique network worldwide so that we can jointly pilot the technology. We are excited to get to work on this project with our partners BCG France, and want to thank EU Horizon Europe/ Eurostars and the Norwegian Research Council for granting us funding. Evaluated by 3 independent experts, our project application achieved a score of 49 out of 54, and was summarized as follows: “The project reflects the deep knowledge about data science solutions in terms of the data mining process, data preparation, and deployment. (...)To my best knowledge, this is a state-of-the-art proposal.”

Funding scheme: