The use of efficient computer-aided medical diagnosis has never been more critical, given the global extent of COVID-19 and the consequent depletion of hospital resources. Artificial intelligence (AI) powered COVID-19 detection can facilitate an early diagnosis of this highly contagious disease and further reduce the infectivity and mortality rates. The preferred imaging option for COVID-19 diagnosis is computed tomography (CT). However, there is an inevitable inter- and intra-observer variability when using CT scans for diagnosis leading to label noise. Label noise can significantly affect the performance of deep learning models. We present a robust COVID-19 classifier on chest CT scan images with noisy labels by proposing an ensemble deep learning model of pretrained Residual Attention and DenseNet architectures. The novelty of this method is that these two deep networks consolidate each other by focusing on complementary, attention-aware, and global sets of features. Specifically, the features extracted from the two deep networks (base-learners) are stacked together and processed by a meta-learner to provide the final robust prediction. Additionally, we have built a large and nationally diverse COVID-19 CT scan dataset by curating open-source datasets to improve the generalizability of our classifier. Extensive experimental evaluations on our dataset illustrate the improved performance of our ensemble model over the base-learners.