Better Video workflows via Real-Time Collaboration and AI-Techniques in TV and New Media

It was the obvious trend for many years; people wanted to work with video wherever they were, at home, in the office and on travel by just using a web browser connecting to the cloud. Although there has been a lot of resistance to using tools like this a major shift happened with the Covid pandemic. Suddenly all employers wanted to supply their journalists with tools that let them work from home and make high quality content. The traditional process of making content in a TV station had been cumbersome and requiring a lot of time due to depending on multiple people and software packages, while you desired to deliver your story ASAP. This was further made more complicated when social media required different aspect ratios and adjusted ways to tell stories. When Vizrt started thinking about making Viz Story in 2016 our goal was to make a simple editing tool that worked in a web browser and could be hosted from the cloud and could be bought on a subscription basis. Today this is the strategy for all products in Vizrt, they are all converted into subscription models, multiple production suites go to the cloud and web-based tools is the general trend not only for Vizrt but for the whole media software industry. This project is actively researching in the following two areas: The first research is collaborative video editing. This has been a general trend over the last ten years in other applications, but the importance has been even more actualized during the Covid pandemic. This has aspects of technical integration on how we more effectively can work with multiple people on same story on same time timeline simultaneously. Here there should both be benefits for using people with different talent for doing different jobs but also simply having more people working on a story when the deadline is coming closer and when tasks must be picked up where the last person ended his day. Collaborative video is also very much about how the workflow should be between journalist and the leader or others involved unfold and this is being researched as well. The covid 19 situation also resulted in research on collaborative video editing when journalists are all in different locations. This had a major upswing during the Covid pandemic The second research area is the application of artificial intelligence (AI) techniques to the video story creation workflow. The advent of many novel AI techniques will allow us to automate repetitive tasks and explore new and exciting features; from voice recognition to object tracking in videos. Such techniques are currently being explored by Vizrt and prototypes have been created including AI assisted subtitling and object recognition using Machine Learning. Sometimes AI works alone, other times there is a human in the loop. What benefits we can gain from these workflows is also subject to our investigations. With Vizrt?s efforts into simultaneous multi aspect production there have been investigations into how AI techniques can be utilized to automate framing of interest points. This project as a collaboration between Vizrt and the Department of Information Science and Media Studies at the University of Bergen. The collaboration will provide valuable research to the academic community and allow Vizrt to incorporate state-of-the-art techniques into the Viz Story product. These enhanced workflows allow users to edit video from any remote office, home and add audio and customize stunning graphics from a web-browser, then publish them to Facebook, Youtube, Twitter, and more in multiple aspect ratios. This project will allow Vizrt to innovate in new ways that allow the end user to use a single tool to publish higher quality video stories in less time than ever.

From 2016 the vision was to make an editing software that worked in the web browser. We wanted to make a pilot software called Viz Story for the cloud and research AI and collaborative video editing. The project has served as a pivot point for our entire product portfolio. The product is today in use it as a daily tool to produce thousands of videos and publishes both to web and broadcast. A cloud version is in use by major cloud providers for their production needs. Since 2016 the change to cloud have been substantial and in todays product strategy Vizrt are on the way to make large product lines available for cloud. AI has grown to become something the customers expect to see in use in software from any provider. Vizrt has now set up a cross domain effort to use AI for use in many applications. Pandemic showed the need for collaborative video editing. The project has served as a pilot for our product lines and many elements are today a part of our entire product strategies.

It can be difficult to believe just how complicated the process of producing the quality of professional video stories produced by broadcasters and published on TV and New media. The current workflow is difficult, expensive and time-consuming, high barrier to entry, requires expertise and involving many different machines and expensive equipment. An example workflow might include research & writing scripts, recording video, recording audio, editing them together, adding text on video via subtitles or as can be found on 'silent videos' you might find on Facebook, as well as adding more advanced graphics as per many news channels. Each one of these activities requires different tools. Vizrt and The University of Bergen are partnering in this research project in order to explore how we might lower the barrier to entry and simplify this workflow such that these workflows become as available as possible. The research focuses on answering the following two questions: 1. How can collaborative tools help improve workflows? 2.How can we use AI techniques to aid workflows? We believe this project will benefit both Vizrt by potentially opening up new markets as well providing valuable industry data to researchers and helping democratise the media landscape. Success in this project could help bring about positive societal change through innovating and lowering the barrier to entry to a high quality video production workflows while providing valuable insights to the greater scientific community.