Tilbake til søkeresultatene

IKTPLUSS-IKT og digital innovasjon

ShuttleNet: Scalable Neural Models for Long Sequential Data

Alternativ tittel: ShuttleNet: Skalerbare nevrale modeller for lange sekvensielle data

Tildelt: kr 7,6 mill.

Vi ser på datadrevet kunstig intelligens hvor data er organisert i lange sekvenser. Data fra anvendelser er ofte sekvensielle, for eksempel tekst, tale, musikk, tidsserier, DNA-sekvenser og hendelsesforløp. Konvensjonelle datavitenskapsmetoder kan imidlertid kun behandle korte sekvenser med opptil noen få tusen trinn. I dette prosjektet utvikler vi en skalerbar metode som muliggjør effektiv og nøyaktig behandling av svært lange sekvenser. Slike sekvenser har opp til millioner eller til og med milliarder trinn. Ved prosjektets slutt vil vi levere teoretiske gjennombrudd i form av nye modeller med garantier, samt praktiske resultater som dataprogramvare og visualiseringsverktøy. Våre forskningsresultater vil bli demonstrert på to fokusområder: 1) mikrobiologi og infeksjonssykdomsepidemiologi og 2) mønstergjenkjenning for fjernåling. Dessuten, fordi lange sekvensielle data er ofte tilgjengelige på mange fagområder, kan vår metode brukes som en kritisk komponent i et bredt spekter av oppgaver, inkludert vitenskapelig forskning, medisin og helsevesen, analyse av naturlig språk, finansiell dataanalyse, markedsundersøkelser, mv. Se engelsk versjon for fremdrift i 01.12.2022 - 17.9.2023.

The project has made significant advancements in the field of machine learning methods for handling long sequences, both in terms of scalability and accuracy. Prior to the project, existing modeling methods were limited to processing sequences of only a few hundred steps. By the conclusion of the project, other established approaches were capable of handling sequence lengths of up to several thousand steps. In stark contrast, our method exhibits remarkable scalability, demonstrating successful inference for sequences as extensive as 1.5 million steps. Furthermore, our method has delivered substantial improvements in accuracy across a wide range of inference tasks, spanning synthetic data, text documents, images, and DNA sequences. It even outperformed Google's Enformer model in genetic variant prediction, achieving this without the need for gene expression supervised data. Additionally, our method surpassed the top competitor in open chromatin region detection with a mere 1% of supervised labels. These proposed neural attention models can serve as the foundation for networks and can be applied to various pattern recognition and generation tasks. The groundbreaking research achieved in this project promises to pave the way for even more disruptive applications in the fields of data science and industry.

In the past decade Machine Learning (ML), especially deep learning, has brought us many successful data-driven AI applications. Many real-world data are intrinsically sequential, for example, text, speech, music, time series, DNA sequences and unfolding of events. However, conventional deep learning methods can process only short sequences up to a few thousand steps. The existing approaches often face challenges like slow inference, vanishing (and exploding) gradients and difficulties in capturing long-term dependencies. In this project we develop a scalable machine learning method which enables efficient and accurate inference for very long sequences up to millions or even billions of steps. At the end of the project, we will deliver a versatile ML framework based on deep neural networks, as well as its efficient optimization algorithms, computer software, and visualization tools. Our research findings will be applied to two focus areas: 1) microbiology and infectious disease epidemiology and 2) remote sensing pattern recognition. Moreover, because long sequential data are commonly available in many areas, our method can be applied as a critical component in a wide range of tasks including scientific research, next-generation DNA sequence analysis, natural language processing, financial data analysis, market studies, etc.

Publikasjoner hentet fra Cristin

Ingen publikasjoner funnet

Ingen publikasjoner funnet

Ingen publikasjoner funnet

Budsjettformål:

IKTPLUSS-IKT og digital innovasjon