Visual analytics for convolutional neural network robustness

Abstract

Convolutional Neural Networks (CNNs) sind ein Typ von Machine Learning Modellen, der weit verbreitet ist bei Computer Vision Systemen. Trotz ihrer hohen Genauigkeit ist die Robustheit von CNNs oft schwach. Ein für Bildklassifizierung trainiertes Modell könnte beispielsweise ein Bild falsch klassifizieren nachdem das Bild leicht gedreht wurde, bei leichter Unschärfe, oder bei veränderter Farbsättigung. Außerdem sind CNNs anfällig gegen sogenannte “Adversarial Attacks”, Methoden, um analytisch minimale Veränderungen am Bild zu generieren. Diese sind für den Menschen nicht wahrnehmbar, können das Klassifizierungsmodell aber in die Irre führen. Es wurden verschiedene Trainingsmethoden entwickelt, um die Robustheit von CNNs zu verbessern.In dieser Arbeit untersuchen wir die Robustheit von CNNs mit zwei Ansätzen: Zuerst visualisieren wir Unterschiede zwischen standard und robusten Trainingsmethoden. Dafür verwenden wir Feature Visualization - eine Methode zur Visualisierung von Mustern, auf die individuelle Neuronen eines CNNs ansprechen. Darauf aufbauend stellen wir eine interaktive Visualisierungsanwendung vor, die die Nutzer eine 3d Szene manipulieren lässt, während sie gleichzeitig die Vorhersagen sowie Aktivierungen aus den versteckten Ebenen des CNNs beobachten können. Um standard und robust trainierte Modelle vergleichen zu können, erlaubt die Anwendung die gleichzeitige Beobachtung von zwei Modellen. Um die Nützlichkeit unserer Anwendung zu testen, führten wir fünf Case Studies mit Machine Learning Experten durch. Im Zuge dieser Case Studies und unserer eigenen Experimente konnten wir mehrere neue Erkenntnisse über robust trainierte Modelle gewinnen, von denen wir drei quantitativ verifizieren konnten. Trotz der Möglichkeit, zwei hochperformante CNNs in Echtzeit zu untersuchen, läuft unsere Anwendung clientseitig in einem standard Webbrowser und kann als statische Website übertragen werden, ohne einen performanten Backend-Server zu benötigen.

Convolutional neural networks (CNNs) are a type of machine learning model that is widely used for computer vision tasks. Despite their high performance, the robustness of CNNs is often weak. A model trained for image classification might misclassify an image when it is slightly rotated, blurred, or after a change in color saturation. Moreover, CNNs are vulnerable to so-called "adversarial attacks", methods where analytically computed perturbatiConvolutional neural networks (CNNs) are a type of machine learning model that is widely used for computer vision tasks. Despite their high performance, the robustness of CNNs is often weak. A model trained for image classification might misclassify an image when it is slightly rotated, blurred, or after a change in color saturation. Moreover, CNNs are vulnerable to so-called “adversarial attacks”, methods where analytically computed perturbations are generated which fool the classifier despite being imperceptible by humans. Various training methods have been designed to increase robustness in CNNs.In this thesis, we investigate CNN robustness with two approaches: First, we visualize differences between standard and robust training methods. For this, we use feature visualization, a method to visualize the patterns which individual units of a CNN respond to. Subsequently, we present an interactive visual analytics application which lets the user manipulate a 3d scene while simultaneously observing a CNN’s prediction, as well as intermediate neuron activations. To be able to compare standard and robustly trained models, the application allows simultaneously observing two models. To test the usefulness of our application, we conducted five case studies with machine learning experts. During these case studies and our own experiments, several novel insights about robustly trained models were made, three of which we verified quantitatively. Despite its ability to probe two high performing CNNs in real-time, our tool fully runs client-side in a standard web-browser and can be served as a static website, without requiring a powerful backend server.ons are generated which fool the classifier despite being imperceptible by humans. Various training methods have been designed to increase robustness in CNNs. In this thesis, we investigate CNN robustness with two approaches: First, we visualize differences between standard and robust training methods. For this, we use feature visualization, a method to visualize the patterns which individual units of a CNN respond to. Subsequently, we present an interactive visual analytics application which lets the user manipulate a 3d scene while simultaneously observing a CNN's prediction, as well as intermediate neuron activations. To be able to compare standard and robustly trained models, the application allows simultaneously observing two models. To test the usefulness of our application, we conducted five case studies with machine learning experts. During these case studies and our own experiments, several novel insights about robustly trained models were made, three of which we verified quantitatively. Despite its ability to probe two high performing CNNs in real-time, our tool fully runs client-side in a standard web-browser and can be served as a static website, without requiring a powerful backend server.