- Concept\/Motivation<\/strong><\/li>
- Recent trends on XAI<\/strong><\/li>
- Method 1: LIME\/SHAP<\/strong>
- Example: Classification<\/li>
- Example: Regressio<\/li>
- Example: Image classification<\/li><\/ul><\/li>
- Method 2: ABN for image classification<\/li><\/ul>\n\n\n\n
\u25a0Concept\/Motivation<\/strong><\/h3>\n\n\n\n
Generally speaking, AI is a blackbox.We want AI to be explainable because\u2026<\/p>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_01.png\")
<\/figure>\n\n\n\n<\/div>\n\n\n\n1. Users should trust AI to actually use it (prediction itself, or model)<\/strong><\/p>\n\n\n\n
Ex: diagnosis\/medical check, credit screening<\/p>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_02.png\")
<\/figure>\n\n\n\nG. Tolomei, et. al., arXiv:1706.06691<\/mark><\/p>\n\n\n\n
People want to know why they were rejected by AI screening, and what they should do in order to pass the screening.<\/p>\n\n\n\n
<\/div>\n\n\n\n2. It helps to choose a model from some candidates<\/strong><\/p>\n\n\n\n
Classifier of text to \u201cChristianity\u201d or \u201cAtheism\u201d (\u7121\u795e\u8ad6)<\/p>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_03.png\")
<\/figure>\n\n\n\nBoth model give correct classification, <\/strong>
but it is apparent that model 1 is better than model 2.<\/strong><\/p>\n\n\n\n<\/div>\n\n\n\n3. It is useful to find overfitting, when train data is different from test data<\/strong><\/p>\n\n\n\n
Cf: Famous example of \u201chusky or wolf\u201d
Training dataset contains pictures of wolfs with snowy background.<\/p>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_04.png\")
<\/figure>\n\n\n\nThen, the classifier trained on that dataset outputs \u201cwolf\u201d if the input image contains snow.<\/p>\n\n\n\n
\u25a0Recent trends<\/strong><\/h3>\n\n\n\n
# of papers which includes one of explanation-related words (\u201cintelligible\u201d, \u201cinterpretable\u201d,\u2026)
AND
one of AI-related words (\u201cMachine learning\u201d, \u201cdeep learning\u201d,\u2026)
FROM
7 repositories (arXiv, Google scholar, \u2026)<\/p>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_05.png\")
<\/figure>\n\n\n\nRecently, researchers are studying XAI more and more.<\/strong><\/p>\n\n\n\n
\u25a0Method 1: Local Interpretable Model-agnostic Explanations (LIME)<\/strong><\/h3>\n\n\n\n
Objects: Classifier or Regressor<\/p>\n\n\n\n
M. Ribeiro, S. Singh, C. Guestrin, arXiv:1602.04938<\/mark><\/p>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_06.png\")
<\/figure>\n\n\n\nBasic idea: Approximate original ML model with interpretable model <\/strong>
(linear model\/DT), in the vicinity of specific features.<\/strong><\/p>\n\n\n\n<\/div>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_07.png\")
<\/figure>\n\n\n\n\u25a0Explaining individual explanation<\/strong><\/h3>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_08.png\")
<\/figure>\n\n\n\n- Original model predicts from features (sneeze, headache,\u2026) whether the patient is flu or not.<\/li>
- LIME approximates the model with linear model in the vicinity of the specific patient.<\/li>
- The weights of the linear model for each feature give \u201cexplanation\u201d of the prediction<\/li><\/ol>\n\n\n\n
\u25a0desirable features of LIME<\/strong><\/h3>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_09.png\")
<\/figure>\n\n\n\n- Interpretable<\/strong><\/li>
- Local fidelity<\/strong><\/li>
- Model-agnostic<\/strong>
(Original model is not affected by LIME at all)<\/li>- Global perspective<\/strong>
(sample different inputs and its predictions)<\/li><\/ul>\n\n\n\n\u25a0Shapley Additive exPlanations (SHAP)<\/strong><\/h3>\n\n\n\n
S. Lundberg, S-I. Lee, arXiv:1705.07874<\/mark><\/p>\n\n\n\n
Generalization of methods for XAI,<\/strong><\/p>\n\n\n\n
- LIME<\/li>
- DeepLIFT A. Shrikumar et. al., arXiv:1605.01713<\/mark><\/li>
- Layer-Wise Relevance Propagation Sebastian Bach et al. In: PloS One 10.7 (2015), e0130140<\/mark><\/li><\/ul>\n\n\n\n
Actually, they are utilizing concepts of cooperative game theory:<\/strong><\/p>\n\n\n\n
- Shapley regression values<\/li>
- Shapley sampling values<\/li>
- Quantitative Input Influence<\/li><\/ul>\n\n\n\n
\u25a0Example: Classification (Titanic dataset)<\/strong><\/h3>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_10.png\")
<\/figure>\n\n\n\n\u25a0GBDT<\/strong><\/h3>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_11.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_12.png\")
<\/figure>\n\n\n\nI used one of GBDT (XGBoost) as a ML model, under the following conditions:<\/p>\n\n\n\n
- No standardization of numerical features (as DT does not need it)<\/li>
- No postprocessing of NaN (GBDT treats NaN as it is)<\/li>
- No feature engineering<\/li>
- Hyperparameter tuning for n_estimators and max_depth (optuna is excellent)<\/li><\/ul>\n\n\n\n
Results:<\/strong><\/p>\n\n\n\n
Best parameters: {\u2018n_estimators\u2019: 20, \u2018max_depth\u2019: 12}
Validation score:0.8659217877094972
Test score: 0.77033
Cf: Baseline (all women survive, all men die): 0.76555<\/p>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_13.png\")
<\/figure>\n\n\n\nCf: Reported best score with ensemble method: 0.84210
Review of know-how on feature engineering by experts:
Kaggle notebook \u201cHow am I doing with my score?\u201d<\/mark><\/p>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_14.png\")
<\/figure>\n\n\n\nCf: Test score at Kaggle competition<\/strong><\/p>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_15.png\")
<\/figure>\n\n\n\nCheaters\u2026 (Memorizing all the names of the survivors? Repeat trying with GA?)<\/p>\n\n\n\n
<\/p>\n\n\n\n
Linear correlation between features and label<\/strong><\/p>\n\n\n\n
(explanation of data itself, not model)
The method which you try first, to select important features.
You can work when the number of features is small.<\/p>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_16.png\")
<\/figure>\n\n\n\n<\/div>\n\n\n\nImportance of features in GBDT<\/strong><\/p>\n\n\n\n
(explanation of whole model, not for specific sample)<\/p>\n\n\n\n
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_17.png\")
<\/figure>\n\n\n\nThe average gain (for mutual information, or inpurity) of splits which use the feature<\/p>\n\n\n\n
Top 3 agrees with that in linear correlation.<\/strong><\/p>\n\n\n\n
- Layer-Wise Relevance Propagation Sebastian Bach et al. In: PloS One 10.7 (2015), e0130140<\/mark><\/li><\/ul>\n\n\n\n
- Local fidelity<\/strong><\/li>
- Interpretable<\/strong><\/li>
- Recent trends on XAI<\/strong><\/li>
![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_18.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_19.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_20.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_21.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_22.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_23.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_24.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_25.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_26.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_27.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_28.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_29.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_30.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_31.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_32.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_33.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_34.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_35.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_36.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_37.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_38.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200711_39.png\")
<\/figure>\n\n\n\n