When it comes to text diffusion models, we often face the challenge of using a fixed number of inference steps for all inputs, regardless of their complexity. This can be inefficient, as simpler sentences require less compute power compared to more complex ones. To tackle this issue, I’ve been exploring an alternative architecture, which I call ‘Adaptive Refinement Diffusion.’
The core idea is to iteratively refine the sequence, calculating a confidence score for every token based on its embedding stability and prediction probability. If a token’s score passes a certain threshold, it gets ‘frozen’ and is excluded from future computation. The entire generation process stops dynamically once all tokens in the sequence are frozen.
This approach allows the model to focus compute on the more difficult or ambiguous tokens, finishing simple sentences much faster. I’d love to hear your thoughts on this idea and whether it already exists in some form. What potential flaws or failure modes do you see with this approach?
By sharing your insights, we can work together to create more efficient text diffusion models that adapt to the complexity of the input.