Code for the experiments in Regularized linear convolutional networks inherit frequency sensitivity from image statistics.
Core functionality computing model gradients with respect to the Fourier basis
as well as various statistics thereof is in freq_sens.py. Code for training
the models used in our experiments is in zoo.py and its imports. The remaining
scripts are either dependencies of the previous two or used for analysis (e.g.
the aptly named generate_all_plots.py).
We gratefully acknowledge dependence on two submodules, namely learning_with_noise and pytorch-vgg-cifar10.
A conda environment is specified in freq_sens.yaml. In theory it can be
created using the following command.
conda env create --file=freq_sens.yaml Tables of training hyperparameters and model accuracies
can be found in ./model_details.
This repository was extracted from a larger research codebase to which Eleanor
Byler and Elise
Bishoff made many contributions. In particular,
Eleanor Byler wrote the first version of
training.py and both Charles Godfrey and
Elise Bishoff made further modifications, and
datasets.py was a collaborative effort of Charles
Godfrey and Eleanor
Byler. The procedural generation (using the
wavelet marginal model) and unsupervised training of AlexNets using
learning_with_noise was implemented by Davis
Brown. The remainder of the code was written by
Charles Godfrey, although it should be noted
that all authors listed in the citation below contributed substantially in the
form of experiment ideas, feedback, suggestions and debugging advice.
If you find this repository useful, please cite:
@article{frequency_sensitivity,
doi = {10.48550/ARXIV.2210.01257},
url = {https://arxiv.org/abs/2210.01257},
author = {Godfrey, Charles and Bishoff, Elise and Mckay, Myles and Brown, Davis and Jorgenson, Grayson and Kvinge, Henry and Byler, Eleanor},
keywords = {Machine Learning (cs.LG), Machine Learning (stat.ML), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Regularized linear convolutional networks inherit frequency sensitivity from image statistics},
publisher = {arXiv},
year = {2022},
copyright = {arXiv.org perpetual, non-exclusive license}
}