Back to search

IKTPLUSS-IKT og digital innovasjon

Logic-based Artificial Intelligence Everywhere: Tsetlin Machines in Hardware

Alternative title: Logikkbasert kunstig intelligens overalt: Tsetlin-maskiner i hardvare.

Awarded: NOK 14.6 mill.

Tsetlin machines are a new machine learning approach founded on sentence logic and Tsetlin Automata (finite-state automata for learning). They learn by extracting common patterns from data, decomposing the data into elementary logical expressions. The simplicity of the logical expressions enables ultra-low energy consumption and fast parallel hardware (HW). Despite this simplicity, Tsetlin machines have outperformed vanilla deep neural networks accuracy-wise on well-established benchmarks. In this project, we aim to solve three significant challenges in Tsetlin machine research: 1) Current hardware prototypes are limited to small-scale machine learning problems. 2) It is unclear how to do reinforcement learning (learning from penalties and rewards), which is a fundamental machine learning paradigm. 3) We do not know how to pre-train TMs from unlabelled data to deal with the well-known bottleneck of labelling data. By overcoming these three obstacles, we aim to architect a new software- and hardware ecosystem that outperforms state-of-the-art machine learning. This will enable powerful logic-based machine learning applications at the edge and in the cloud. For the first challenge, we have shown how to reduce energy consumption by introducing deterministic learning steps. We have further developed a scalable and asynchronous architecture that can utilize parallel hardware. Finally, we have proposed hardware designs using so-called Petri nets, including latency analysis. For the second challenge, we have developed the first Tsetlin machine framework for reinforcement learning and shown that the models built are interpretable. For the third challenge, we have developed a Tsetlin machine architecture that supports mapping between multidimensional input and output values. This is the first step towards learning from unlabelled data.

Tsetlin Machines (TMs) are a new machine learning (ML) approach founded on the Tsetlin Automaton. TMs use frequent pattern mining and resource allocation to extract common patterns in the data, rather than relying on minimizing output error, which is prone to overfitting. Unlike the intertwined nature of pattern representation in neural networks (NNs), TMs decompose problems into self-contained patterns, each represented as a conjunctive clause. The clause outputs, in turn, are combined into a classification decision through summation and thresholding, akin to a logistic regression function, however, with binary weights and a unit step output function. TM hardware (HW) has demonstrated up to three orders of magnitude reduced energy and faster learning, compared to NNs alike. Logic-driven fundamental blocks, organized in lean parallel processing units, are major contributors to this comparative advantage over NNs that are heavily arithmetic-based. The TM further outperforms vanilla CNNs and LSTMs accuracy-wise on well-established benchmarks. While the reported results on TMs are promising in terms of scalability, training time, accuracy, and energy, we here address three major obstacles. 1) Current FPGA and ASIC prototypes lack scalable memory elements, constraining them to small-scale ML problems. 2) Reinforcement learning (RL) is key to many ML problems, such as playing board games, however, it is unclear how to model RL in the TM framework. 3) State-of-the-art deep learning models support pre-training on unlabelled data, which significantly improves the accuracy of following supervised learning, dealing with shortage of labelled data. It is unclear how to pre-train TMs from unlabelled data. By overcoming these three obstacles we aim to architect a new TM HW/SW ecosystem that outperforms state-of-the-art ML in terms of energy efficiency and scalability, parametrised by accuracy. This will enable powerful logic-based ML applications at the edge and in the cloud.

Activity:

IKTPLUSS-IKT og digital innovasjon