Unknown Signal Detection in Switching Linear Dynamical System Noise

Gabriel Ford, Benjamin J. Foster, Stephen A. Braun, Moshe Kam

Research output: Contribution to journalArticlepeer-review


A machine learning approach is presented for detecting unknown or anomalous signals in a complicated background of interfering signals and noise. The approach can be employed in RF spectrum monitoring applications to efficiently detect transmissions that deviate from a typical signal environment. For example, in the cognitive radio domain, the technique may be applied to learn the typical behavior of spectrum sharing secondary users and efficiently detect noncompliant transmissions. A switching linear dynamical system (SLDS) is trained to represent the interference and noise environment via a Bayesian nonparametric hierarchical Dirichlet process (HDP)-SLDS technique. An unknown signal is detected if the Viterbi hidden switching state path of the test data is sufficiently unlikely under the learned background SLDS. The detection scheme is derived as a generalized likelihood ratio test (GLRT) for an unknown deterministic signal in SLDS noise. The distribution of the Viterbi likelihood test statistic under the null hypothesis (signal absent) is analyzed and an asymptotic upper bound on the false alarm probability is derived as a function of the detection threshold. Numerical simulation and experimental results on a software-defined radio (SDR) testbed demonstrate that the empirical false alarm rate obeys the upper bound and that the SLDS detection approach substantially outperforms an earlier HMM-based scheme as well as a standard energy detector in a challenging interference and noise background.

Original languageEnglish (US)
Pages (from-to)2220-2234
Number of pages15
JournalIEEE Transactions on Signal Processing
StatePublished - 2023

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Electrical and Electronic Engineering


  • Bayesian nonparametrics
  • Markov jump linear system
  • Signal detection
  • generalized likelihood ratio test
  • switching linear dynamical system


Dive into the research topics of 'Unknown Signal Detection in Switching Linear Dynamical System Noise'. Together they form a unique fingerprint.

Cite this