Mike Heddes

DeepCrossAttention: Supercharging Transformer Residual Connections

Published at the International Conference on Machine Learning (ICML) 2025

DeepCrossAttention (DCA) is a novel enhancement to transformer networks that improves residual learning by dynamically combining layer outputs using input-dependent, learnable weights. This approach enables the model to focus on the most relevant information from previous layers while incorporating depth-wise cross-attention for richer layer interactions.

Always-Sparse Training by Growing Connections with Guided Stochastic Exploration

Published at the International Joint Conference on Neural Networks (IJCNN) 2025

The excessive computational requirements of modern deep learning are posing limitations on the machines that can run it. We propose an efficient, always-sparse training algorithm with excellent scaling to larger and sparser models. Moreover, our algorithm improves over the accuracy of previous sparse training methods.

Hyperdimensional Computing: A Framework for Stochastic Computation and Symbolic AI

Published in the Journal of Big Data 2024

In this manuscript, we provide an approachable, yet thorough, survey of the components of Hyperdimensional Computing (HDC). HDC is a neuro-inspired computing framework that exploits high-dimensional random vector spaces. We highlight the dual use of HDC, used for its learning capabilities, and more generally, as a probabilistic model for computation.

Convolution and Cross-Correlation of Count Sketches Enables Fast Cardinality Estimation of Multi-Join Queries

Published at the International Conference on Management of Data (SIGMOD) 2024

Estimating join cardinality is crucial for efficient database queries. This paper proposes a solution to a decades old problem using a novel sketching method that maintains fast updates even for multi-join queries by leveraging Count sketches. Our method significantly improves the efficiency and estimation accuracy over previous methods.

DotHash: Estimating Set Similarity Metrics for Link Prediction and Document Deduplication

Published at the International Conference on Knowledge Discovery and Data Mining (KDD) 2023

Metrics for set similarity are a core aspect of several data mining tasks. We propose DotHash, an unbiased estimator for the intersection size of two sets. DotHash can be used to estimate the Jaccard index and, to the best of our knowledge, is the first method that can also estimate the Adamic-Adar index and a family of related metrics.

Torchhd: An Open Source Python Library to Support Research on Hyperdimensional Computing and Vector Symbolic Architectures

Published at the Journal of Machine Learning Research (JMLR) 2023

We present Torchhd, a high-performance open-source Python library for Hypderdimensional Computing and Vector Symbolic Architectures (HD/VSA). Torchhd seeks to make HD/VSA more accessible and serves as an efficient foundation for further research and application development.

An Extension to Basis-Hypervectors for Learning from Circular Data in Hyperdimensional Computing

Published at the Design Automation Conference (DAC) 2023

We present a detailed study on basis-hypervector sets, which leads to practical contributions to Hyperdimensional Computing (HDC) in general: 1) an improvement for level-hypervectors, used to encode real numbers; 2) a method to learn from circular data, an important type of information never before addressed in machine learning with HDC.

Hyperdimensional Hashing: An Efficient and Robust Dynamic Hash Table

Published at the Design Automation Conference (DAC) 2022

Most cloud services and distributed applications rely on hashing algorithms that allow dynamic scaling of a robust and efficient hash table. Examples include AWS, Google Cloud and BitTorrent. We propose Hyperdimensional (HD) hashing and show that it has the efficiency to be deployed in large systems.

GraphHD: Efficient Graph Classification using Hyperdimensional Computing

Published at the Design, Automation and Test in Europe Conference (DATE) 2022

Graphs are among the most important forms of information representation, yet, to this day, Hyperdimensional Computing algorithms have not been applied to the graph learning problem in a general sense. In this paper, we present GraphHD — a baseline approach for graph classification with HDC.

Jet Fighter Ai

Personal project in Reinforcement Learning, 2021

A Reinforcement Learning agent learns to play the two-player Atari game Jet Fighter. Visitors can play against the agent in an Atari-style simulator on the project page.

EdgeAvatar: An Edge Computing System for Building Virtual Beings

Published in Electronics 2021

We describe EdgeAvatar, a system based on Edge Computing principles for the creation of virtual beings. The objective is to provide a streamlined and modular framework for virtual being applications that are to be deployed in public settings. EdgeAvatar can be adapted to fit different approaches for AI powered conversations.

Space Optimization Competition Platform

Project at the European Space Agency 2019

SpOC (Space optimization Competition) is an optimization challenge, hosted on the European Space Agency's Optimize platform, that has experts around the world compete to solve three complex optimization problems wrapped up in a stimulating and exotic space mission scenario.

Settlers of the Galaxy

Personal project in Animation & Interaction, 2019

Interactive animation of the Milky Way galaxy that includes our Solar System. Based on data from the GTOC X trajectory optimization challenge proposed by NASA's Jet Propulsion Laboratory (JPL).

Differentiable Cartesian Genetic Programming Interface

Project at the European Space Agency, 2019

Differentiable Genetic Cartesian Programming (dCGP) is a Machine Learning tool for symbolic regression. DCGP generates explicit formulas that can be understood and studied. Our software thus provides a form of explainable AI, which can be applied to any supervised learning task.

SpaceX Grid Fin Design

Personal project in Animation & Interaction, 2018

Interactive 3D model of the second generation Space X grid fin as flown on the Falcon 9 rocket. The grid fins are made from titanium so that they can be reused.

Music Production

During high school and college I dreamed of becoming an electronic dance music producer like Martin Garrix, the Swedish House Mafia, or Avicii. Over the years my dream of being a music producer faded away while my interest in science grew. Making music is now a hobby of mine.