Hello and welcome to Part 4 in the series on autoregressive generative models. We have covered some of the theory of these models and have implemented causality and probability to allow a CNN to be capable of generating new images based on a training set. In this post we are going to focus on the building blocks that are used to construct the leading model of this class; pixelSNAIL (as well as some of the other variants of pixelCNNs). This brings in quite a few ideas from the field of deep learning and in some cases I will touch on them only briefly; and aim to cover them in more detail in other posts. Gated Residual Block We have discussed[…]
Tag: autoregressive generative models
Hi and welcome to Part 3 of the series on Autoregressive Generative Models; last time we explained how to incorporate probabilistic interpretations to a network to allow it to generate new image samples. This time we cover another key principle underlying these models; causality. Let’s get on with it… Causality One of the fundamental properties of a generative model is causality; which is the requirement that predictions made for a single element of a sequence (for example a pixel of an image, word of a sentence or note of a piece of music) only depends on previously generated elements and not future elements. This is easiest to understand in the context of 1D or 2D convolutions where a regular convolution[…]