EQUIVARIANT & COORDINATE INDEPENDENT CONVOLUTIONAL NETWORKS

What is the appropriate geometric structure for neural networks that process spatial signals on Euclidean spaces or more general manifolds? This question takes us on a journey which leads to a gauge field theory of convolutional networks.

Feature vector fields: The spatial signals we are interested in are fields of feature vectors. Feature fields allow to describe data like images, audio, videos, point clouds, or tensor fields, such as fluid flows and electromagnetic fields.

Equivariant networks commute with actions of some symmetry group on their feature spaces. The relevant group actions in this work are geometric transformations of feature fields, like translations, rotations, or reflections of images. Equivariant models generalize everything they learn over the considered group of transformations. This property makes them significantly more data efficient, interpretable, and robust in comparison to non-equivariant models.

Convolutional Neural Networks (CNNs) are the most common network architecture for processing feature fields. Conventional CNNs operate on Euclidean spaces and are translation equivariant, i.e. position independent. This work explains how to extend CNNs to be equivariant under more general symmetries of space.

Coordinate independence: Manifolds are in general not equipped with a canonical choice of coordinates. Feature fields and neural network layers are hence required to be coordinate independent, that is, expressible relative to different frames of reference. The ambiguity of local frames represents the gauge freedom of our neural field theory. We show that the demand for coordinate independence requires CNNs to be equivariant under local gauge transformations.

To offer an easy entry, the first part of this work focuses on the representation theory of equivariant convolutional networks on Euclidean spaces. The insights gained in the Euclidean setting are subsequently leveraged to develop the full gauge theory of coordinate independent CNNs on Riemannian manifolds. In the last part, we turn to a discussion of practical applications on specific manifolds. A comprehensive literature review demonstrates the generality of our theory by showing for more than 100 models from the literature how they can be understood as specific instantiations of 'Equivariant and Coordinate Independent CNNs'.

Contents:

• Preface
• Introduction
• Equivariant Convolutional Networks on Euclidean Spaces:
  • Invariant and Equivariant Machine Learning Models
  • Translation Equivariance & Conventional Euclidean CNNs
  • Affine Group Equivariance & Steerable Euclidean CNNs
  • G-Steerable Convolution Kernels
  • Empirical Evaluation of Steerable CNNs
• An Introduction to Coordinate Independent CNNs:
  • Gauges, Gauge Transformations and G-Structures
  • Coordinate Independent Feature Vector Fields
  • Coordinate Independent Networks and GM-Convolutions
  • Reflection Steerable Möbius CNNs
• Fiber Bundle Theory of Coordinate Independent CNNs:
  • Associated Bundles and Coordinate Free Feature Fields
  • Coordinate Free Formulation of Kernel Field Transforms and GM-Convolutions
  • Isometry Equivariance
• Applications & Literature Review:
  • Design Choices and Overview
  • Euclidean Coordinate Independent CNNs
  • Rotation Equivariant CNNs on Punctured Euclidean Spaces
  • Spherical Coordinate Independent CNNs
  • Coordinate Independent CNNs on General Surfaces
• Appendices:
  • Groups, Representations and Equivariant Maps
  • Coordinate Chart Formalism of Differential Geometry
  • Integration Over Tangent Spaces
  • Equivariant MLPs
  • Equivariant Convolutions on Homogeneous Spaces
  • Coordinate Independent Kernel Weight Sharing
  • An Intuition for the Wigner–Eckart Theorem
  • Existence and Smoothness of Kernel Field Transforms
  • Regular Feature Fields as Scalar Functions on G-Structure
  • Quotient Representative Kernel Fields — Proofs
  • Spherical Convolutions as GM-Convolutions — Proofs
• Bibliography
• List of Theorems and Definitions
• Index

Readership: Research scientists, engineers and graduate students in computer vision and deep learning who are interested in the equivariant processing of spatial data. Typical applications are medical imaging, satellite image analysis, physical simulations, climate modeling, molecule and protein generation, geometry processing or robotics.