How to interpret machine learning models?
Can we understand what really happens inside machine learning models?
In my previous blog,I had talked about the trade-off between model performance and interpretability. More often than not you will observe business stakeholders preferring less accurate but highly interpretable linear models because of the ease of explainability. But as a data scientist you know that in real-world problems due to inherent high dimensionality and complex non-linear relationships amongst the features non-linear and complex models are always more robust and reliable. So in an ideal world, everyone wants to be in the holy grid in which you get the highest model performance with highly interpretable results.
In this blog, I have briefly mentioned some of the techniques I have used in my projects to add interpretability and to combat the trade-off between high model performance and high interpretability.
Following are the ways I have used to add interpretability to the modes:
1. Feature Importance: What degree to which the predictive model relies on a particular feature?
2. Partial Dependence Plots: How does a particular feature impact model predictions?
3. Shap: Which variables caused this prediction?
4. Lime: Which variables caused this prediction?
To explain the above methods, I have used datasets from the Education sector from Open Government Data (OGD) Platform India — https://data.gov.in/ — is a platform for supporting Open Data initiative of Government of India. This data has the dropout rate, % schools with electricity, computer facilities, toilets etc. for all the states in India. Out of curiosity I wanted to test whether the dropout rate in primary schools is linked to the facilities available in schools. So I used dropout rate as the y-variable (target) and the other metrics as the independent features.
1. Feature Importance:
Gini Importance or Mean Decrease Impurity
For classification problems, Scikit-learn uses Gini Impurity by default for calculating feature importance. Feature importance is called as ‘Gini Importance’ or ‘Mean Decrease Impurity’. It is defined as the total decrease in node impurity averaged over all the trees of the ensemble. The total decrease in node impurity is weighted by the probability of reaching a particular node (which is out of the total proportion of samples reaching that node). In short this ensures that if the ‘Mean Decrease Impurity’ is same for 2 variables but variable 1 is at top of the tree and other is at depth 5, then variable 1 will get more importance as it is easier to get to that node as compared to the node at depth 5. So higher the value of the feature more important it is. Finally, the feature importance is normalized between 0 and 1 by dividing by the sum of all feature importance values.
Below is the feature importance graph from the case study mentioned above.
This suggests that across states in India, the dropout rate in primary schools is most impacted by availability of electricity, water supply, and toilet facilities more than availability of computers.
Permutation Importance
Permutation Importance is an alternative to calculate feature importance that can be used for any model. It basically shuffles 1 feature row wise at a time, all other features and target remaining constant. Then it checks how this affects the accuracy of model predictions in the reshuffled data. So for a column that is impacting the predictions the most, the model performance reduces.
In the above table the top features are ranked from most important to least important. The weight column measures how much the model performance (accuracy) decreased with random shuffling. As you can see the order of the top variables from both the methods is the same, but this might not be always true.
2. Partial Dependence Plots (PDP)
Now feature importance helps us get to 1 part of the puzzle. But to know how each variable impacts the predictions, we can analyze the Partial Dependence Plots. This can help us identify whether the relationship between the target and feature is linear, monotonous, or more complex.
In this method, we alter the values of a single variable at a time, other features and target remaining constant. With this we can understand how the predicted outcome varies with change in a particular variable. The logic of PDP is very simple, for the feature of interest, it basically forces a single value across all the data points in the data set and then averages the prediction across all the data points. This exercise is repeated for all the values of the particular feature.
The downward sloping chart indicates that as the % of schools with electricity increases, the dropout rate reduces as compared to what it is at the smallest value of the feature. The flat regions of the chart indicate that in those regions increase in % electricity doesn’t yield any change in the dropout %. PDPs are always measured with respect to the smallest value of the feature, so the leftmost point always has zero on the y-axis. The blue region is a measure of uncertainty.
You only have to ensure that the feature for which you are analyzing the PDP is not highly correlated with the other features in the data set. For example in our data set if % schools with electricity is positively correlated with % schools with computer, then when we are calculating PDP there will be records for which the combination of these 2 features is unlikely to occur in reality. If in your case the correlation is high amongst the features then you can look up for Marginal Plots and Accumulated Local Effects (ALE) Plots.
I will write about LIME and SHAP in the upcoming blogs.
Let me know your thoughts about the blog in comments. Also, I would like to know whether you have developed case studies from Open Government Data (OGD) Platform India.
This content was originally published on my personal blogging website: http://datascienceninja.com/. Click here to view it and subscribe to receive updates on latest blogs instantly.
