Automated short-term production planning processes for Hydro- and windpower using deep reinforced learning methods

It is easy to take for granted the electricity supplied to your home, but to supply it exactly when you need it requires careful planning. Consumption varies continuously as users turn on the TV or heaters, and power production must instantly respond to these changes. Every day power is traded is based on the estimated consumption, and the scheduled production on the Nord Pool Spot power exchange. On this exchange, power produced from different sources is bought and sold and is ready for delivery the next day. This is called the Day-ahead market. Even though the marked is planned to be in balance, the system is continuously influenced by factors that could lead to imbalances. This could be changes in consumption as result of colder weather or unplanned outage in a Power Station. During the last decades Statnett have introduced market solutions to ensure sufficient supply of reserves. To manage and plan for sales in an increasing number of markets, most power producers have engaged production planners. In production planning, the power producer attempts to optimize the value of the resources in a long and short-term perspective. This is done by applying a wide range of models and commercial competence The complexity in the planning and nomination process is increasing. The time from when information is acquired to decisions are made is getting shorter, and the degree of details modelled in the power systems, and the amount of information processed, is continuously increasing. In addition, restrictions given by local, state-dependent, concessional and environmental conditions tend to introduce additional requirements to models that are applied in the planning process. The objective of this project is to develop new methods for applied decision support for hydro- and windpower production planning. The long-term target is automatization of the nomination process using a combination of fundamental models, and machine learning methods In order to facilitate automation of complex processes, the problem should be divided into smaller and specific tasks to be solved. It is also important that a company that considers automating different processes, can be presented with a realistic picture of what the consequences of automating will be. One of the objectives of the project is therefore to build a simulation tool where different strategies can be assessed against each other. As a result of the requirement for decomposition and measurability, the following main activities have been identified, and will form the framework for further research in the project. I. Extend the method of heuristically calculating marginal cost to be applicable for a linked river system. II. Establish a simulation framework where different bidding strategies can be evaluated against each other. III. Apply different machine learning methods to investigate if different bidding strategies are optimal under certain conditions. IV. Handling unbalances in combination with load requirements in a power-system. V. Integration of an automated bidding-model in real life operations where choice of bidding strategy will be based on input criterions prior to bidding. Tasks 1-4 have been completed, and the thesis is submitted to the review committee. The method developed in task 1 has been implemented in an operational tool that can be applied by European power produces. The simulation framework developed in 2 has been tested on a section of a Norwegian river-system in Southwestern Norway, and has shown the stochastic bidding is a viable option for systems exposed to significant fluctuation in inflows and market prices. The concept developed in 3 where machine learning is applied to identify optimal bidding strategies has been investigated and published in a journal article. The results show that profits can be increased and calculation time reduced by applying the new concept. Finally, optimal balancing and load distribution of hydro- and wind production in intraday operations have been considered in the light of operating a combined portfolio of multiple energy assets as part of task 4. A method has been proposed for optimizing a common load requirement for the total portfolio with intraday prices. It has been demonstrated how the imbalance cost for a power producer with both wind and hydropower assets can be reduced by internal balancing in combination with sales and purchases in a pay-as-bid intraday market.

Målsetningen med prosjektet var å tilrettelegge for økt automatisering av produksjonsplanleggingsprosesser. Prosjektet har utviklet metoder som tilrettelegger for automatisering av prosesser som idag i stor grad utføres manuelt. Dette vil bidra til å frigjøre tid til å fokusere på prosesser som blir viktigere fremover. Dette kan blant annet være tilrettelegging for dataanalyse av en økt mengde informasjon som etterhvert vil bli tilgjengelig fra produksjonsenheter, nett og marked.

The objective of this project is to develop new methods for applied decision support for hydro- and windpower production planning. The long-term target is automatization of the nomination process using a combination of fundamental models, and deep reinforced learning methods. Traditionally Nordic power producers have optimized production towards the Nordpool spot market. Closer interactions with European markets, large scale introduction of wind and unregulated power production, and implementation of markets solutions for secondary and tertiary reserves, have increased the complexity in the planning and nomination process. The time from when information is acquired to decisions are made is getting shorter, and the degree of details modelled in the power systems, and the amount of information processed, is continuously increasing. In addition restrictions given by local, state-dependent, concessional and environmental conditions tend to introduce additional requirements to models that are applied in the planning process. In production planning, the power producer attempt to optimize the value of the available resources in a long and short term perspective. This is done by applying a wide range of models and commercial competence. A common challenge for the models applied in the existing planning process is the time requirements associated with complex modelling. The following models are used in the production planning process: o Price models to model expected long- and short-term prices o Long-term fundamental models to generate water values and weekly production plans o Short-term models to create hourly production schedules and marginal-cost o Marginal-cost models to translate results from short-term models in the price-dependent nomination process. The main tasks will focus on solving the identified, and capturing further, arising factors in scheduling.