Black Box Model Explainability

Introduction

Welcome to the "Black Box Model Explainability" lab! This lab is designed to give you an example black box model to interpret to practice and understand how to interpret black box model explainability metrics.

Learning Objectives

• Increase understanding of how to explain black box models
• Recognize the use cases for black box models
• Review how to interpret LIME and SHAP interpretations of models

Prerequisites

A deep understanding of different AI/ML model structures and basic understanding of model interpretability.

Before creating the model you will start by creating the synthetic dataset for anomaly detection using the make_blobs function. Use the code below to generate a synthetic dataset applicable to anomaly detection.

X, _ = make_blobs(n_samples=300, centers=1, cluster_std=1.0, random_state=42)
X = pd.DataFrame(X, columns=["Feature1", "Feature2"])
``` This synthetic data does not need to be labeled as the model we will use to determine anomaly detection will determine which data points are anomalies based on how far away the points are from the cluster of "regular activity".

You will start by training our initial isolated forest. Isolated forests are black box models that use unstructured data, meaning the data is not labeled, to train and determine anomalies. This makes explainability difficult because the data is not labeled and the model struggles to display the logic used to draw conclusions in high dimensional spaces. Training the initial model will use the code below. Use the code below to train an isolated forest.

iso = IsolationForest(contamination=0.1, random_state=42)
iso.fit(X)
scores = iso.decision_function(X)
preds = iso.predict(X)  # -1 = anomaly, 1 = normal
labels = (preds == 1).astype(int)  # 1 = normal, 0 = anomaly
``` With the initial isolated forest created we can visualize the data to get a better understanding of what might be happening under the hood. In this case it is simple since the data is only two dimensional. For higher dimensional data, aspects such as Principle Component Analysis (PCA) may need to be used to reduce the dimensionality. Using the code below you can visualize the blob data to gain a better understanding of how Anomalies were identified.

plt.scatter(X['Feature1'], X['Feature2'], c=labels, cmap='coolwarm') plt.title('Anomaly Detection Results') plt.xlabel('Feature1') plt.ylabel('Feature2') plt.show()

X_train, X_test, y_train, y_test = train_test_split(X, labels, stratify=labels, random_state=42) clf = RandomForestClassifier(random_state=42) clf.fit(X_train, y_train)

To start with SHAP you will first need to acquire the values from the SHAP TreeExplainer functionality using our surrogate random forest. The code below will give SHAP the model and the expected results and determine how the model valued each feature.

explainer_shap = shap.TreeExplainer(clf)
shap_values = explainer_shap.shap_values(X_test)

From here you can use summary_plot, or force_plot. summary_plot is how the features affect the model overall samples given. Whereas the force_plot will give insight to the features that impacted a specific choice. For the force_plot you can alter the index to receive information on different data points. The graph will display where each feature is pulling with negative features pulled towards a negative classification, and a positive pull, pulling towards a positive classification. You should plot both the summary_plot

shap.summary_plot(shap_values[1], X_test, plot_type="bar", show=True)

and the force_plot, to get an understanding of the differences between each type of summary.

shap.initjs()
instance = X_test.iloc[0:1]
shap.force_plot(explainer_shap.expected_value[1], shap_values[1][0], instance)

LIME explains why a specific prediction was made by approximating the model locally with a simple, interpretable surrogate model (like linear regression or decision trees). The difference this has from SHAP is the use of a local approximating model as opposed to deriving values from game theory.

It perturbs the input slightly around the point of interest, observes how predictions change, and fits a simple model on those synthetic samples. Giving you a model that understands how different values within the surrogate model change depending on inputted parameters.

To start with Lime you will need to setup the explainability model. The code below, will set up the lime explainer before applying it to our local model.

explainer_lime = LimeTabularExplainer(
    training_data=np.array(X_train),
    feature_names=X_train.columns,
    class_names=["Anomaly", "Normal"],
    discretize_continuous=True
)

From here you can examine specific data points to determine how different features helped classify as either positive or negative classifications. The code below allows for the interpretability of a specific choice to be analyzed.

i = 0
exp = explainer_lime.explain_instance(X_test.iloc[i].values, clf.predict_proba, num_features=2)
print("\nLIME Explanation:")
exp.show_in_notebook(show_table=True)