This paper addresses the problem of continual learning of radar detectors from sequentially provided training data. Given a limited number of labeled samples for each of a number of tasks in a certain class, a model-agnostic meta-learning (MAML) approach is developed to enable few-shot learning of a new task in the class. In the radar detection problem, a task is associated with a set of specific environmental conditions such as target, clutter or interference properties. The proposed detector design is separated into two stages: a meta-training stage and an adaptation stage. The outcome of the meta-training stage is used to initialize the learning of the detector parameter vector, which in turn relies on the training data available for adaptation. Unlike a previously proposed offline approach, where all data available for meta-training was assumed available upfront, the proposed learning procedure is applied online, where meta-training data is made available incrementally, with each task. Numerical results validate that the proposed detector may learn a new task from a limited number of labeled samples, and that the performance improves as the meta-training relies on an increasing number of tasks.