Human activity recognition, OpenPack Challenge

Problem

In logistics centres, packing an order is a sequence of short operations: picking, reading labels, assembling a box, scanning, sealing. Knowing exactly which operation a worker is doing, second by second, is what lets a warehouse measure and improve the process. The OpenPack Challenge 2022 asked for exactly that: classify ten packaging operations for every one-second slot, from OpenPack’s 53 hours of multimodal recordings. Movements vary with the item, the worker’s speed and their skill, and the classes are heavily imbalanced, so the metric is macro F1.

What I did

I entered on my own, alongside my day job, and deliberately kept the inputs minimal: just the accelerometer and gyroscope from two wrist IMUs, 12 channels at 30 Hz, normalised and cut into 60-second windows.

The model is a DeepConvLSTM with self-attention: four 1D convolutional layers extract local motion features, two bidirectional LSTM layers (256 units) model how they evolve over time, and an 8-head self-attention layer lets the network weigh the time points that matter before a Conv1d head outputs per-timestep class probabilities.

The biggest lever was augmentation for time series: Gaussian noise, amplitude scaling, random crops, channel swapping, and permuting segments of the signal together with their labels, all tuned to vary the data without distorting it. Trained with PyTorch Lightning on Google Colab.

Result

Each idea added measurable accuracy: self-attention lifted macro F1 from 0.870 to 0.896, and augmentation took it to 0.911, with the biggest gains on the hardest test subject. That placed 5th of 33 teams on the final leaderboard (817 submissions in total), earned a top-five travel grant to the award ceremony at IEEE PerCom 2023, and was published as a workshop paper at BiRD 2023.