SHAP and LIME: a practical guide to opening up AI's black box

The guide frames explainability as an operational problem as much as a technical one. It points to credit scoring, medical diagnosis, e-commerce recommendations, and autonomous driving as areas where models are already embedded, and notes that under rules like Europe's GDPR, automated decisions increasingly come with a right to explanation. That shifts the burden onto engineers: when a loan application is rejected or a diagnosis is issued, someone has to justify the result, audit it for bias by race or gender, and stand behind it with customers and regulators.

The piece then separates the two tool families by use case. SHAP, built on Shapley values from game theory, spans both a model-wide importance view and individual predictions, with a TreeExplainer optimized for tree-based models. LIME instead samples around a single point and fits a simple local model, trading global insight for speed and intuition. The caveats are where the practical value sits: model-agnostic methods slow down sharply on large datasets, strong correlations between features can skew apparent importance, and per-inference explanations strain production latency — so the guide recommends precomputed summaries or rule-based substitutes when explanations must be served live.