Constrained decoding: Can language models generate cute pixel art?

Recently, I’ve been experimenting with low-level token generation using models like gpt-oss-20b and gemma-3-27b-it. I became particularly interested in constrained decoding, where you apply a mask to the model’s probability distribution at each generation step.

That led me to a fun question: what if you constrain the probabilities to a predefined set of “color tokens” and force the model to output them in a grid, essentially generating pixel art?

Predictably, the results were… chaotic. But the failures were fascinating, and I’m convinced the limitations were mostly due to my rushed implementation and time constraints.