Pooling vs output query: choosing how models form predictions

This sits between two earlier discussions: how outputs depend on each other (parallel versus sequential generation) and how variable-length inputs are represented (aggregated features or entity tokens). Its focus is the layer in between — how an encoded set of representations H=(h_1,...,h_n) is converted into predictions.

It separates the unit of input from the unit of output. A model may predict one property for an entire input, a prediction per input element, or predictions for a separate set of output slots that need not match the input tokens one-to-one. The three designs map onto these cases, and increasing input tokens does not force a matching number of outputs.

Exchanging information among output queries is distinct from autoregressive conditioning. With query self-attention but independent sampling per output, the joint distribution factorizes as a product of per-output distributions given the input. Autoregressive generation instead conditions each output on previously chosen outputs, a separate design axis from how representations are read out.