AI-PRODUCER: AI-based Video Clipping and Summarization of Sport Events

As written in the project summary, the goal of the project is to develop an automated AI-based video producer generating clipped sport events and summaries. Such a service will reduce costs and additionally enable production services on both existing archives and in leagues not having the resources for a production. At the current stage, we have just initiated the project. We have collected a first initial dataset, and we have tested a few promising models for finding appropriate start and stop intervals for the video clip. Ongoing work include increasing the dataset and improving the initial models.

Video productions are part of an enormous industry where soccer alone has a market share of about 45% of the $500 billion global sports industry. There is a huge interest in sport event and summary consumption for both broadcasted and streamed on the web. A lot of resources are put into producing the content in an efficient and high-quality manner. However, generating such summaries and events requires expensive equipment, includes a lot of tedious expensive manual labor, and the operation is often performed redundantly by different actors for different purposes. Forzasys has already developed an improved annotation and production system optimising the manual procedures and reducing the need for expensive equipment. In this project, we are targeting an AI-based solution assisting the current production personnel in their daily work and extending the current system, i.e., allowing a production of highlights at a higher pace and enriched with additional metadata. Building on our experience in sport analysis, sport video delivery systems, and use of AI for video and image analysis in both sport and medicine, the idea is to use AI to analyse the video from the area around a detected event, using for example scene change detection, logo identifications, audio analysis, change of cameras, game transitions and replay recognition, i.e., all information to be used to enable an accurate automated AI-based clipping operation. Furthermore, other enhancement methods will also be investigated, like adding more detailed text descriptions, tags and appropriate thumbnail selection describing the events. Moreover, targeting the labour expensive generation of highlights after the game, and even in the half-time break, we also aim for AI-automated solutions for collecting the most relevant events and combining and clipping these into a highlight summary package to be used by various actors like the league streaming services, media and TV broadcasters.