Detecting Anomalous Agent Decision Sequences Based on Offline Imitation Learning

Abstract

Anomaly detection in decision-making sequences is a challenging problem due to the complexity of normality representation learning, the sequential nature of the task and the difficulty of real-world implementation. In this work, we propose extracting two behaviour features: action optimality and sequential association to detect anomalous behaviour. Our offline imitation learning model is an adaptation of behavioural cloning with a transformer policy network, where we modify the training process to learn a Q function and a state value function from normal trajectories.