SaTC: CORE: Small: Collaborative: When Adversarial Learning Meets Differential Privacy: Theoretical Foundation and Applications

The pervasiveness of machine learning exposes new and severe vulnerabilities in software systems, where deployed deep neural networks can be exploited to reveal sensitive information in private training data, and to make the models misclassify. However, existing learning algorithms have not been designed to be simultaneously robust to such privacy and integrity attacks, in both theory and practice. In field trials, such lack of protection and efficacy significantly degrades the performance of machine learning-based systems, and puts sensitive data at high risk, thereby exposing service providers to legal action based on HIPAA/HITECH law and related regulations. This project aims to develop the first framework to advance and seamlessly integrate key techniques, including adversarial learning, privacy preserving, and certified defenses, offering tight and reliable protection against both privacy and integrity attacks, while retaining high model utility in deep neural networks. The system is being developed for scalable, complex, and commonly used machine learning frameworks, providing a fundamental impact to both industry and educational environments. An ultimate goal of this project is to build a core foundation of privacy preservation in adversarial learning, to better address the trade-off between model utility, privacy loss, and certified defenses. Accordingly, the team theoretically connects adversarial learning and privacy preservation by introducing a new set of rigorous theories to address the trade-off between model utility and privacy loss. To further strengthen the safety of the system, the team will conduct a new class of attacks towards discovering previously unknown and unprotected vulnerabilities, including highly sensitive and hidden correlation structures among data instances, which will be used to amplify existing model attacks. Based upon that effort, vulnerable features and correlations will be automatically identified and protected, towards unified robust and privacy preserving learning, given both model training and inference. Finally, the team will optimize the trade-off among model utility, privacy loss, and certified defenses. The project is expected to lay a theoretical and practical foundation of key privacy-preserving techniques to protect users' personal and highly sensitive data in adversarial learning under model attacks. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.