Researchers investigate explainability of linear min-max neural networks, showing single activated neuron traces decision process

A research team studied the explainability properties of linear min-max neural networks, which can be interpreted as k-medoids (a clustering method) at initialization and are trained using subgradient descent.

The model enables tracing of decisions because a single most activated neuron is responsible for the output value. Researchers designed a pixel fragility measure to determine whether changes to a single pixel may cause classification output changes.

On the PneumoniaMnist dataset, the explanation method compared favorably to SHAP and Integrated Gradient, two existing explainability techniques.