Hi and welcome to Part 3 of the series on Autoregressive Generative Models; last time we explained how to incorporate probabilistic interpretations to a network to allow it to generate new image samples. This time we cover another key principle underlying these models; causality. Let’s get on with it… Causality One of the fundamental properties of a generative model is causality; which is the requirement that predictions made for a single element of a sequence (for example a pixel of an image, word of a sentence or note of a piece of music) only depends on previously generated elements and not future elements. This is easiest to understand in the context of 1D or 2D convolutions where a regular convolution[…]