在Databricks中进行渲染

Question

我正在尝试可视化Shap索引以解释xgboost机器学习模型。可以在Google合作实验室中实现这一目标，但是在Databricks中却很难实现这一目标。

import shap
import pandas as pd
import numpy as np
import matplotlib
import xgboost as xgb

df = pd.read_csv("/dbfs/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv").iloc[:,1:]

#label encode the categorical features:
df = pandasData.copy() 
df["clarity"] = df["clarity"].astype('category')
df["color"] = df["color"].astype('category')
df["cut"] = df["cut"].astype('category')

df["clarity_cat"] = df["clarity"].cat.codes
df["color_cat"] = df["color"].cat.codes
df["cut_cat"] = df["cut"].cat.codes
df= df.drop(["clarity","color","cut"],axis = 1)

Y = df['price']
X = df.drop(['price'], axis=1)

X_traintest, X_valid, Y_traintest, Y_valid = train_test_split(X, Y, test_size=0.3, random_state=7)

model = xgb.XGBRegressor(learning_rate=0.02, n_estimators=3161, max_depth=3)
model.fit(X_traintest, Y_traintest, eval_metric="rmse", verbose = False)

目前可行

shap.initjs() # load JS visualization code to notebook

explainer = shap.TreeExplainer(model) # explain the model's predictions using SHAP values

shap_values = explainer.shap_values(X_traintest)
    shap_explain = shap.force_plot(explainer.expected_value, shap_values[0,:], X.iloc[0,:]) # visualize the first prediction's explanation
    display(shap_display)

并产生

这不起作用（但在Colab中却有效）：

shap.initjs()
shap_display = shap.dependence_plot('y', shap_values, X_traintest)
display(shap_display) #tried matplotlib=True/False)

最终看起来像这样

什么时候看起来像这样

Answer 1

您能看到这行吗？

displayHTML(shap_display.to_html()