PyTorch 2.12.0 test: inference_mode alone leaves Dropout on, outputs 0.0 and 2.0

The test was set up deliberately: all inputs were 1, the Linear weight was the identity matrix, and the only nonlinearity was Dropout with p=0.5. That makes the output values a direct readout of whether Dropout was active. In the training rows, outputs contained 0.0 and 2.0; in the evaluation rows, they were 1.0 throughout.

The distinction the write-up draws is between two separate switches. eval() calls train(False), which recursively sets training=False on child modules, and Dropout reads that flag. inference_mode() targets autograd bookkeeping instead, and the same PyTorch documentation carries the note that it does not automatically set the model to evaluation mode. Because the switches are independent, a test that only checks requires_grad can pass while Dropout is still running.

The suggested practice is to set evaluation mode first, disable gradient recording, and include layers whose behavior changes at evaluation time in output tests, rerunning them when the model or wrapper is updated.