Residual MFN

Abstract

Coordinate networks like Multiplicative Filter Networks (MFNs) and BACON offer some control over the frequency spectrum used to represent continuous signals such as images or 3D volumes. Yet, they are not readily applicable to problems for which coarse-to-fine estimation is required, including various inverse problems in which coarse-to-fine optimization plays a key role in avoiding poor local minima. We introduce a new coordinate network architecture and training scheme that enables coarse-to-fine optimization with fine-grained control over the frequency support of learned reconstructions. This is achieved with two key innovations. First, we incorporate skip connections so that structure at one scale is preserved when fitting finer-scale structure. Second, we propose a novel initialization scheme to provide control over the model frequency spectrum at each stage of optimization. We demonstrate how these modifications enable multiscale optimization for coarseto-fine fitting to natural images. We then evaluate our model on synthetically generated datasets for the the problem of single-particle cryo-EM reconstruction. We learn high resolution multiscale structures, on par with the state-of-the art.

Motivating 2D Example

When employing a coarse-to-fine training strategy to train a BACON, namely outputs at each scale is optimized in separate rounds of optimization, the network completely forgets the learned representations of coarse scale outputs from previous optimization rounds. In fact, reconstructions at different scales are highly coupled, since new layers can establish new frequencies anywhere within the band limit. We modify the architecture by adding skip connections and propose a new the initialization scheme to make such coordinate network amenable to coarse-to-fine multiscale reconstruction scenarios.

Cryo-EM

3D Reconstruction. Single particle cryo-EM is an increasingly popular experimental technique to recover the 3D structure of macromolecular complexes, such as proteins. This has enabled atomic or near-atomic resolution reconstruction of challenging structures. Multiscale, coarse-to-fine reconstructions has long played a critical role in cryo-EM estimation, making it an ideal testbed for our proposed methods.

Coarse-to-fine. In practice, naively optimizing for both the poses and potential map suffers from poor local minima due to high frequency noise. To avoid this, we start from estimating low frequency content of the map as it has higher SNR. This allows for reliable initial pose estimation. We then further refine to higher frequency content which allows for more accurate pose estimation. Previous approaches employ variants of this technique, called Frequency Marching.

Results. We evaluate our proposal on two synthetic datasets obtained from PDB1ol5 and PDB4ake. From low to high scales, increasingly more structural features are resolved. The low-scale reconstructions capture only the global and low-resolution features, whereas the full-scale ones contain fine-grained details.

Evolution. The coarse-to-fine evolution of the reconstructed structure for each dataset during training. We first start from a random structure, and then obtain a coarse reconstruction which will be further refined to fine-grained details at later stages (i.e. high-resolution).

Model Fitting. We also use Chimera to fit the available atomic model of pdb4ake into the corresponding final reconstruction. As shown in video model_fit.mp4, various secondary structures, such as α-helices, are well aligned with the reconstructed 3D density map.