Implementation of the Simulator-based Explanations of DNN-Errors (SEDE) approach.

Overview

SEDE (Simulator-based Explanations of DNN-Errors)

Introduction

SEDE is a tool that is used to generate explanations for DNN errors with any simulator. SEDE is a cluster-based approach that relies on the clusters generated by HUDD, but might be used with any generated clusters.

This package provides the images generated by our approach and competing approaches appearing in "Simulator-based explanation and debugging of hazard-triggering events in DNN-based safety-critical systems" by Hazem Fahmy, Fabrizio Pastore, Member, IEEE, and Lionel Briand, Fellow, IEEE.

Project Description

To support safety analysis practices, we propose SEDE, a technique that generates readable descriptions for commonalities in failure-inducing, real-world images and improves the DNN through effective retraining. SEDE leverages the availability of simulators, which are commonly used for cyber-physical systems. It relies on genetic algorithms to drive simulators towards the generation of images that are similar to failure-inducing, real-world images in the test set; it then employs rule learning algorithms to derive expressions that capture commonalities in terms of simulator parameter values. The derived expressions are then used to generate additional images to retrain and improve the DNN. With DNNs performing in-car sensing tasks, SEDE successfully characterized hazard-triggering events leading to a DNN accuracy drop. Also, SEDE enabled retraining leading to significant improvements in DNN accuracy, up to 18 percentage points.

NOTICE on dependencies and libraries

SEDE is a toolset that might be used to generate explanations for DNN errors with any simulator. However, for our experiments we rely on the faces simulator developed by IEE S.A. (https://iee-sensing.com) We refer to such simulator as IEE-Face-Simulator. The released implementation of SEDE invokes the IEE-Face-Simulator through a program call (i.e., not through APIs). SEDE makes use of two versions of IEE-Face-Simulator, v0.5, which is in directory IEE_V1, and v1.0, which is in directory IEE_V2.

Both the two versions of the IEE-Face-Simulator are released with GPL v3 licence (see files 'gpl-3.0.txt' in both directories). The directory IEEPackage is Copyright (C) 2018-2022 IEE S.A. (https://iee-sensing.com) released with GPL v3 licence.

USAGE

The package contains the python code implementation of the SEDE approach and the files used to evaluate RQ1-5 (SEDE_RQx.py) along with IEE simulators and DeepJanus adoption of our case studies. We modified the implementation of DeepJanus for their BeamNG case study with the following changes:

  • individual.py: we do not deal with a pair of images but only one image; distance computation is performed directly in the archive class, where we import “doParamDist()” from “searchModule.py”
  • core.nsga2.py:
    • the budget is provided as execution time, not number of generations
    • ProblemClass.final_population is assigned with the population stored in the archive. The assignment occurs in “Archive.archive_metrics()”. This is slightly different from DeepJanus where the archive is stored inside BeamNGProblem.
  • core.archive_impl.py: the SmartArchive() function has been modified to add only individuals that are inside the cluster (F2 <=1); the distance function is replaced by ours (cosine parameters similarity)

To run the tool you need to have the case studies setup in a separated directory and then generate HUDD clusters using the command

python SEDE.py -o ./HPD -SEDE HUDD

To run SEDE, you need to specify the IEE-Face-Simulator version (options are 1 or 2) and the algorithm to use (options are SEDE or DeepJanus)

python SEDE.py -o ./HPD -iee 1 -SEDE SEDE python SEDE.py -o ./HPD -iee 1 -SEDE DeepJanus

To run the evaluation you need to use the following command:

python SEDE.py -SEDE RQ1

where RQx is the RQ to be evaluated (options are RQ1, RQ2, RQ3, RQ4, RQ5) N.B.: for RQ1 you need to have the RQ1 directory shared with the replicability package inside the parent directory of this file

The repository contains the following folders for evaluation:

  • 'SEDE Expressions' contains the PDF files with the expressions generated by SEDE for each case study DNN. For each RCC, the expression contains the constraints for the rules generated by PART (in bold) and the constraints generated by SEDE for the simulator parameter values in common for both safe and unsafe images.

  • 'RQ1' contains a folder for each case study DNN ('HPD-F', 'HPD-H', 'FLD'). Inside, you can find a folder for each RCC; such folder contains the real-world images belonging to the cluster along with the images generated by SEDE, NSGA-II and DeepNSGA-II in four different experiments. E.g.: 'RQ1/HPD-F/RCC-1' contains 'SEDE' and 'NSGA-II' and ‘HUDD', where ‘HUDD’ contains the real-world images in the cluster generated by HUDD. For ‘FLD’, the clusters are not included in the package due to copyrights purposes. ‘SEDE’, ‘DeepNSGA-II’ and ‘NSGA-II’ contains folders named ‘Run-1’, ‘Run-2’, ‘Run-3’, and ‘Run-4’, where each contains the images generated in each run reported for RQ1. Inside each folder, you can find ‘diversity.npy’ which reports the diversity observed over time (RQ1.1), ‘indvidiuals.npy’ which reports the percentage of individuals observed over time (RQ1.2), and ‘clusters.npy’ which reports the percentage of covered clusters observed over time (RQ1.3).

  • ‘RQ2-3’ contains a folder for each case study DNN (‘HPD-F’, ‘HPD-H’, ‘FLD’) with a folder for each RCC containing the images generated and evaluated by SEDE for these RQs. Since both RQ2 and RQ3 evaluate the same images generated by SEDE in Step 2.1, they are combined in a single folder.

  • 'RQ4' contains a folder for each case study DNN ('HPD-F', 'HPD-H', 'FLD') with images generated according to SEDE expressions for each cluster and the obtained accuracy. E.g.: 'RQ4/HPD-F/RCC-1_500_10.2%.gif' where 500 is the number of images and 10.2% is the obtained accuracy.

  • 'RQ5' contains a folder for each case study DNN ('HPD-F', 'HPD-H', 'FLD'). Each case study DNN contains a folder with the images selected/generated by competing approaches ('HUDD', 'RBL') for the 10 runs. E.g.: 'RQ5/HPD-F/HUDD/RCC-1.gif' contains the images selected by HUDD for Cluster 1.

Reference:

If you use our work, please cite SEDE in your publications. Here is an example BibTeX entry:

@misc{https://doi.org/10.48550/arxiv.2204.00480,
  doi = {10.48550/ARXIV.2204.00480},
  url = {https://arxiv.org/abs/2204.00480},
  author = {Fahmy, Hazem and Pastore, Fabrizio and Briand, Lionel},
  keywords = {Software Engineering (cs.SE), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Simulator-based explanation and debugging of hazard-triggering events in DNN-based safety-critical systems},
  publisher = {arXiv},
  year = {2022},
  copyright = {arXiv.org perpetual, non-exclusive license}
}
You might also like...

SLISEMAP: Combining supervised dimensionality reduction with local explanations

SLISEMAP: Combining supervised dimensionality reduction with local explanations SLISEMAP is a supervised dimensionality reduction method, that takes d

Dec 15, 2022

Implementations from scratch of YOLOv1 and YOLOv2 in Tensorflow 2.0 with Vietnamese detail explanations

YOLO Collections In this project, I have implemented YOLO algorithms from scratch with framework Tensorflow 2.0. Notice that the architecture of the m

May 27, 2022

Meaningfully debugging model mistakes with conceptual counterfactual explanations. ICML 2022

Meaningfully debugging model mistakes with conceptual counterfactual explanations. ICML 2022

Meaningfully debugging model mistakes with conceptual counterfactual explanations What is this work about? Understanding model mistakes is critical to

Dec 16, 2022

Public code repository accompanying "PredDiff: Explanations and Interactions from Conditional Expectations"

PredDiff: Explanations and Interactions from Conditional Expectations This repository provides resources to reproduce results from the paper: PredDiff

Aug 29, 2022

Contains key documentations of my masterthesis. Includes code for computing counterfactual explanations, layout of user study and analysis code of the study results.

Contains key documentations of my masterthesis. Includes code for computing counterfactual explanations, layout of user study and analysis code of the study results.

Evaluating the Practicality of Counterfactual Explanations This repository holds key information about my Masterthesis (18 EC) for the Master's degree

Sep 15, 2022

A Python based simulator for Oliver Darkshire's single page RPG "Trapped in a cabin with Lord Byron

byron_gh A Python 3 based simulator for Oliver Darkshire's single page RPG "Trapped in a cabin with Lord Byron This simple program is based on the rul

Apr 1, 2022

A python based data center topology. The project aims to present a comprehensive analysis of the Fat tree topology's performance using Mininet simulator.

Mininet-Fat-Tree-Topology This project aims to simulate a Data Center Network (DCN), using Fat tree topology proposed by M.Al-Fares, which is derived

Oct 23, 2022

A simple stock trading simulator website based on Flask.

A simple stock trading simulator website based on Flask.

Stock Trading Simulator A simple stock trading simulator website based on Flask web framework. Multiple operations including qouting, purchasing, sell

Nov 13, 2022

A Defensive Approach for Query-Based Audio Adversarial Attacks

A Defensive Approach for Query-Based Audio Adversarial Attacks We introduce a state detection defense for query-based attacks to detect the process of

May 13, 2022
Owner
null
code for paper: Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes

FedPS-AggregateOrNot code for paper: Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes Bas

LiXinchun 5 Nov 18, 2022
Zeus is an energy optimization framework for DNN training.

An Energy Optimization Framework for DNN Training Zeus automatically optimizes the energy and time of training a DNN to a target validation metric by

SymbioticLab 51 Jan 4, 2023
📦 PyTorch based visualization package for generating layer-wise explanations for CNNs.

Explainable CNNs ?? Flexible visualization package for generating layer-wise explanations for CNNs. It is a common notion that a Deep Learning model i

Ashutosh Hathidara 183 Dec 15, 2022
Script that checks all Meraki switch ports in an organization and returns the existence of errors.

MERAKI_ORG_PORT_CHECK Script that checks all Meraki switch ports in an organization and returns the existence of errors. HOW TO USE IT Open the file "

null 1 Mar 29, 2022
A core for easily initialise and run discord.py bots, and track logs and errors

Discore A core for initialise, run and tracks errors of discord.py bots, with a "Convention over configuration" philosophy Features One-line bot initi

Kyrela 2 Oct 31, 2022
Detect Dung Lai's common errors

Phát hiện Dũng Lại Lập Trình - Lỗi cơ bản Thư viện được lập trình để giúp người mới nhập môn lập trình khi học khóa cơ bản của Dũng Lại Lập Trình có t

Hiếu Deeptry 4 Jul 14, 2022
Super simple py logging module to log your errors.

OphanimLog Simple Python module to log your stuff Install The module uses colorama to display automatic logs, which are logs created automatically on

Accia 2 Sep 15, 2022
There is a script that has errors but i solved this error with some analyze that is about ORM.

Sqlite and benefits of ORM There is a script that has errors but i solved this error with some analyze that is about ORM. I noticed ORM is very import

null 1 Nov 24, 2022
This repo contains explanations of the most popular data structures used in interview questions.

Repository This repo contains explanations of the most popular data structures used in interview questions. Each data structure has its own separate R

Maria Roberta Prado 7 Oct 7, 2022
✏️ A comprehensive list of my Leetcode solutions and explanations

✏️ A comprehensive list of my Leetcode solutions and explanations

Martin 2 Apr 20, 2022