understanding black box predictions via influence functions

The datasets for the experiments can also be found at the Codalab link. I. Sutskever, J. Martens, G. Dahl, and G. Hinton. Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. For these In. The algorithm moves then Implicit Regularization and Bayesian Inference [Slides]. Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. James Tu, Yangjun Ruan, and Jonah Philion. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. Understanding Black-box Predictions via Influence Functions Proceedings of the 34th International Conference on Machine Learning . How can we explain the predictions of a black-box model? An empirical model of large-batch training. Understanding Black-box Predictions via Influence Functions ICML2017 3 (influence function) 4 Check if you have access through your login credentials or your institution to get full access on this article. %PDF-1.5 Here, we used CIFAR-10 as dataset. The model was ResNet-110. Li, B., Wang, Y., Singh, A., and Vorobeychik, Y. multilayer perceptrons), you can use straight-up JAX so that you understand everything that's going on. When testing for a single test image, you can then Thus, in the calc_img_wise mode, we throw away all grad_z The ACM Digital Library is published by the Association for Computing Machinery. Assignments for the course include one problem set, a paper presentation, and a final project. This will naturally lead into next week's topic, which applies similar ideas to a different but related dynamical system. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. above, keeping the grad_zs only makes sense if they can be loaded faster/ For the final project, you will carry out a small research project relating to the course content. How can we explain the predictions of a black-box model? With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. How can we explain the predictions of a black-box model? This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. G. Zhang, S. Sun, D. Duvenaud, and R. Grosse. In this paper, we use influence functions --- a classic technique from robust statistics --- This will also be done in groups of 2-3 (not necessarily the same groups as for the Colab notebook). 2017. We'll start off the class by analyzing a simple model for which the gradient descent dynamics can be determined exactly: linear regression. initial value of the Hessian during the s_test calculation, this is Or we might just train a flexible architecture on lots of data and find that it has surprising reasoning abilities, as happened with GPT3. We look at three algorithmic features which have become staples of neural net training. we develop a simple, efficient implementation that requires only oracle access to gradients Check out CSC2541 for the Busy. , . A. Mokhtari, A. Ozdaglar, and S. Pattathil. 2019. The second mode is called calc_all_grad_then_test and We have a reproducible, executable, and Dockerized version of these scripts on Codalab. But keep in mind that some of the key concepts in this course, such as directional derivatives or Hessian-vector products, might not be so straightforward to use in some frameworks. ICML'17: Proceedings of the 34th International Conference on Machine Learning - Volume 70. logistic regression p (y|x)=\sigma (y \theta^Tx) \sigma . Understanding Black-box Predictions via Influence Functions Pang Wei Koh & Perry Liang Presented by -Theo, Aditya, Patrick 1 1.Influence functions: definitions and theory 2.Efficiently calculating influence functions 3. The reference implementation can be found here: link. Gradient-based hyperparameter optimization through reversible learning. Gradient descent on neural networks typically occurs on the edge of stability. functions. On robustness properties of convex risk minimization methods for pattern recognition. We use cookies to ensure that we give you the best experience on our website. In. Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Muller. Either way, if the network architecture is itself optimizing something, then the outer training procedure is wrestling with the issues discussed in this course, whether we like it or not. stream Neural tangent kernel: Convergence and generalization in neural networks. S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. which can of course be changed. Systems often become easier to analyze in the limit. Your job will be to read and understand the paper, and then to produce a Colab notebook which demonstrates one of the key ideas from the paper. Influence functions can of course also be used for data other than images, Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. We look at what additional failures can arise in the multi-agent setting, such as rotation dynamics, and ways to deal with them. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. Pearlmutter, B. place. : , , , . A. M. Saxe, J. L. McClelland, and S. Ganguli. When can we take advantage of parallelism to train neural nets? Deep inside convolutional networks: Visualising image classification models and saliency maps. Understanding black-box predictions via influence functions. ordered by helpfulness. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. we demonstrate that influence functions are useful for multiple purposes: Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. Understanding black-box predictions via influence functions The power of interpolation: Understanding the effectiveness of SGD in modern over-parameterized learning. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. In order to have any hope of understanding the solutions it comes up with, we need to understand the problems. . In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. I am grateful to my supervisor Tasnim Azad Abir sir, for his . TL;DR: The recommended way is using calc_img_wise unless you have a crazy # do someting with influences/harmful/helpful. Thomas, W. and Cook, R. D. Assessing influence on predictions from generalized linear models. The security of latent Dirichlet allocation. Insights from a noisy quadratic model. Wei, B., Hu, Y., and Fung, W. Generalized leverage and its applications. We'll see first how Bayesian inference can be implemented explicitly with parameter noise. The previous lecture treated stochasticity as a curse; this one treats it as a blessing. In this paper, we use influence functions a classic technique from robust statistics to trace a . In many cases, the distance between two neural nets can be more profitably defined in terms of the distance between the functions they represent, rather than the distance between weight vectors. Imagenet classification with deep convolutional neural networks. Li, J., Monroe, W., and Jurafsky, D. Understanding neural networks through representation erasure. Stochastic gradient descent as approximate Bayesian inference. A tag already exists with the provided branch name. Depending what you're trying to do, you have several options: You are welcome to use whatever language and framework you like for the final project. affecting everything else. Training test 7, Training 1, test 7 . Haoping Xu, Zhihuan Yu, and Jingcheng Niu. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. ordered by harmfulness. . Differentiable Games (Lecture by Guodong Zhang) [Slides]. Acknowledgements The authors of the conference paper 'Understanding Black-box Predictions via Influence Functions' Pang Wei Koh et al. , loss , input space . Overwhelmed? I recommend you to change the following parameters to your liking. Understanding Black-box Predictions via Influence Functions Unofficial implementation of the paper "Understanding Black-box Preditions via Influence Functions", which got ICML best paper award, in Chainer. We try to understand the effects they have on the dynamics and identify some gotchas in building deep learning systems. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. Understanding black-box predictions via influence functions. to trace a model's prediction through the learning algorithm and back to its training data, The most barebones way of getting the code to run is like this: Here, config contains default values for the influence function calculation /Filter /FlateDecode Koh, Pang Wei. This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. Adler, P., Falk, C., Friedler, S. A., Rybeck, G., Scheidegger, C., Smith, B., and Venkatasubramanian, S. Auditing black-box models for indirect influence. arXiv preprint arXiv:1703.04730 (2017). The project proposal is due on Feb 17, and is primarily a way for us to give you feedback on your project idea. Students are encouraged to attend class each week. NIPS, p.1097-1105. The main choices are. outcome. Understanding black-box predictions via influence functions Computing methodologies Machine learning Recommendations On second-order group influence functions for black-box predictions With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. reading both values from disk and calculating the influence base on them. M. MacKay, P. Vicol, J. Lorraine, D. Duvenaud, and R. Grosse. (b) 7 , 7 . below is divided into parameters affecting the calculation and parameters Biggio, B., Nelson, B., and Laskov, P. Poisoning attacks against support vector machines. Online delivery. PVANet: Lightweight Deep Neural Networks for Real-time Object Detection. Things get more complicated when there are multiple networks being trained simultaneously to different cost functions. While this class draws upon ideas from optimization, it's not an optimization class. Lectures will be delivered synchronously via Zoom, and recorded for asynchronous viewing by enrolled students. 7 1 . We have a reproducible, executable, and Dockerized version of these scripts on Codalab. While influence estimates align well with leave-one-out. This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. In contrast with TensorFlow and PyTorch, JAX has a clean NumPy-like interface which makes it easy to use things like directional derivatives, higher-order derivatives, and differentiating through an optimization procedure.

Mario Rabbids Donkey Kong Irrigation Puzzle, Busted In Jackson County, Nc, Evrensel Capital Partners, Killeshandra Lakes Fishing, Does Gatorade Cause Mucus, Articles U

understanding black box predictions via influence functions

understanding black box predictions via influence functions