Jugal Parikh .jpg

Protecting the Protector - Hardening machine learning defenses against adversarial attacks


In today's threat landscape, it's not unusual for attackers to circumvent traditional machine learning based detections' by constantly scanning their malware samples against security products and modifying them until they are no longer detected. But more recently, we've seen a rise in attackers attempting to compromise these machine learning models directly by poisoning incoming telemetry.

In this talk, we discuss several strategies to make machine learning models more robust to such attacks. We'll discuss research that shows how singular models are susceptible to tampering, and some techniques, like stacked ensemble models, can be used to make them more resilient. We also talk about the importance of diversity in base ML models and technical details on how they can be optimized to handle different threat scenarios. Lastly, we'll describe suspected tampering activity we've witnessed using protection telemetry from over half a billion computers, and whether our mitigations worked.

The 4 main takeaways from this talk are:

  1. Technical analysis on attempted adversarial attacks used to compromise cloud and client ML models

  2. Importance of having a diverse set of features and classifiers for any ML pipeline and how to achieve that

  3. Tips and tricks to train interpretable, resilient and robust models against adversarial attacks followed by real world case studies on how it helped us block several large scale as well as targeted attacks.

  4. How to add real time monitoring on all deployed ML models to detect potential tampering activities in real time

Speaker Bio: Jugal Parikh has been working in the field of security and machine learning for seven years. He enjoys solving complex security problems like targeted attacks detection, static and behavioral file/ script based detection, and detecting adversarial attacks using machine learning. He is currently a Senior Data Scientist at Microsoft's Windows Defender Research team.