I was going through this paper: Multi-Grid Neural Architectures. It proposes an alternate architecture to conventional CNN where at each depth there is a pyramid with bottom