p7cond-mat.dis-nn

cond-mat.dis-nn

93 papers

An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model

Erwin Bolthausen

1201.2891

Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

Greg Yang, Edward J. Hu, Igor Babuschkin, Szymon Sidor, Xiaodong Liu, David Farhi, Nick Ryder, Jakub Pachocki, Weizhu Chen, Jianfeng Gao

2203.03466

Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation

Greg Yang

1902.04760

A Mean Field Theory of Batch Normalization

Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, Samuel S. Schoenholz

1902.08129

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli

1503.03585

A typical reconstruction limit of compressed sensing based on Lp-norm minimization

Y. Kabashima, T. Wadayama, T. Tanaka

0907.0914

Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach

Ryo Karakida, Shotaro Akaho, Shun-ichi Amari

1806.01316

Power-law out of time order correlation functions in the SYK model

Dmitry Bagrets, Alexander Altland, Alex Kamenev

1702.08902

Exponential expressivity in deep neural networks through transient chaos

Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, Surya Ganguli

1606.05340

Message Passing Algorithms for Compressed Sensing

David L. Donoho, Arian Maleki, Andrea Montanari

0907.3574

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit

Blake Bordelon, Lorenzo Noci, Mufan Bill Li, Boris Hanin, Cengiz Pehlevan

2309.16620

Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks

Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou

2310.02244

Mean Field Residual Networks: On the Edge of Chaos

Greg Yang, Samuel S. Schoenholz

1712.08969

Towards Understanding Grokking: An Effective Theory of Representation Learning

Ziming Liu, Ouail Kitouni, Niklas Nolte, Eric J. Michaud, Max Tegmark, Mike Williams

2205.10343

Understanding Deep Neural Networks with Rectified Linear Units

Raman Arora, Amitabh Basu, Poorya Mianjy, Anirbit Mukherjee

1611.01491

Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications

Aurelien Decelle, Florent Krzakala, Cristopher Moore, Lenka Zdeborová

1109.3041

A mean-field limit for certain deep neural networks

Dyego Araújo, Roberto I. Oliveira, Daniel Yukimura

1906.00193

The performance of modularity maximization in practical contexts

Benjamin H. Good, Yves-Alexandre de Montjoye, Aaron Clauset

0910.0165

Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks

Blake Bordelon, Cengiz Pehlevan

2205.09653

On the geometry of generalization and memorization in deep neural networks

Cory Stephenson, Suchismita Padhy, Abhinav Ganesh, Yue Hui, Hanlin Tang, SueYeon Chung

2105.14602

Contrasting random and learned features in deep Bayesian linear regression

Jacob A. Zavatone-Veth, William L. Tong, Cengiz Pehlevan

2203.00573

Feature Learning in Infinite-Width Neural Networks

Greg Yang, Edward J. Hu

2011.14522

Explaining Neural Scaling Laws

Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, Utkarsh Sharma

2102.06701

Sparse Estimation with the Swept Approximated Message-Passing Algorithm

Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborová

1406.4311

Scaling description of generalization with number of parameters in deep learning

Mario Geiger, Arthur Jacot, Stefano Spigler, Franck Gabriel, Levent Sagun, Stéphane d'Ascoli, Giulio Biroli, Clément Hongler, Matthieu Wyart

1901.01608

A jamming transition from under- to over-parametrization affects loss landscape and generalization

Stefano Spigler, Mario Geiger, Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Matthieu Wyart

1810.09665

Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes

Greg Yang

1910.12478

The edge of chaos: quantum field theory and deep neural networks

Kevin T. Grosvenor, Ro Jefferson

2109.13247

Geometric GAN

Jae Hyun Lim, Jong Chul Ye

1705.02894

Stochastic Interpolants: A Unifying Framework for Flows and Diffusions

Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden

2303.08797

Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

Andrew M. Saxe, James L. McClelland, Surya Ganguli

1312.6120

Why does deep and cheap learning work so well?

Henry W. Lin, Max Tegmark, David Rolnick

1608.08225

Asymptotics of representation learning in finite Bayesian neural networks

Jacob A. Zavatone-Veth, Abdulkadir Canatar, Benjamin S. Ruben, Cengiz Pehlevan

2106.00651

Performance of Bayesian linear regression in a model with mismatch

Jean Barbier, Wei-Kuo Chen, Dmitry Panchenko, Manuel Sáenz

2107.06936

From Integrable to Chaotic Systems: Universal Local Statistics of Lyapunov exponents

Gernot Akemann, Zdzislaw Burda, Mario Kieburg

1809.05905

The Quantization Model of Neural Scaling

Eric J. Michaud, Ziming Liu, Uzay Girit, Max Tegmark

2303.13506

Bias-variance decomposition of overparameterized regression with random linear features

Jason W. Rocks, Pankaj Mehta

2203.05443

Learning curves of generic features maps for realistic datasets with a teacher-student model

Bruno Loureiro, Cédric Gerbelot, Hugo Cui, Sebastian Goldt, Florent Krzakala, Marc Mézard, Lenka Zdeborová

2102.08127

Locality defeats the curse of dimensionality in convolutional teacher-student scenarios

Alessandro Favero, Francesco Cagnetta, Matthieu Wyart

2106.08619

The Gaussian equivalence of generative models for learning with shallow neural networks

Sebastian Goldt, Bruno Loureiro, Galen Reeves, Florent Krzakala, Marc Mézard, Lenka Zdeborová

2006.14709

Comparing Dynamics: Deep Neural Networks versus Glassy Systems

M. Baity-Jesi, L. Sagun, M. Geiger, S. Spigler, G. Ben Arous, C. Cammarota, Y. LeCun, M. Wyart, G. Biroli

1803.06969

Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group

Johanna Erdmenger, Kevin T. Grosvenor, Ro Jefferson

2107.06898

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Rodrigo Veiga, Ludovic Stephan, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

2202.00293

Pathological spectra of the Fisher information metric and its variants in deep neural networks

Ryo Karakida, Shotaro Akaho, Shun-ichi Amari

1910.05992

The role of regularization in classification of high-dimensional noisy Gaussian mixture

Francesca Mignacco, Florent Krzakala, Yue M. Lu, Lenka Zdeborová

2002.11544

Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks

Blake Bordelon, Cengiz Pehlevan

2304.03408

A Dynamical Model of Neural Scaling Laws

Blake Bordelon, Alexander Atanasov, Cengiz Pehlevan

2402.01092

Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics

Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel L. K. Yamins, Hidenori Tanaka

2012.04728

A Theory of Solving TAP Equations for Ising Models with General Invariant Random Matrices

Manfred Opper, Burak Çakmak, Ole Winther

1509.01229

Universality of jamming of non-spherical particles

Carolina Brito, Harukuni Ikeda, Pierfrancesco Urbani, Matthieu Wyart, Francesco Zamponi

1807.01975

Triple descent and the two kinds of overfitting: Where & why do they appear?

Stéphane d'Ascoli, Levent Sagun, Giulio Biroli

2006.03509

Tensor Programs IVb: Adaptive Optimization in the Infinite-Width Limit

Greg Yang, Etai Littwin

2308.01814

Nonperturbative renormalization for the neural network-QFT correspondence

Harold Erbin, Vincent Lahoche, Dine Ousmane Samary

2108.01403

Solving the Quantum Many-Body Problem with Artificial Neural Networks

Giuseppe Carleo, Matthias Troyer

1606.02318

Typical $l_1$-recovery limit of sparse vectors represented by concatenations of random orthogonal matrices

Yoshiyuki Kabashima, Mikko Vehkapera, Saikat Chatterjee

1208.4696

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)

Cedric Gerbelot, Alia Abbara, Florent Krzakala

2006.06581

Kac-Rice fixed point analysis for single- and multi-layered complex systems

J. R. Ipsen, P. J. Forrester

1807.05790

Entropic gradient descent algorithms and wide flat minima

Fabrizio Pittorino, Carlo Lucibello, Christoph Feinauer, Gabriele Perugini, Carlo Baldassi, Elizaveta Demyanenko, Riccardo Zecchina

2006.07897

Neural Networks and Quantum Field Theory

James Halverson, Anindita Maiti, Keegan Stoner

2008.08601

Exponential number of equilibria and depinning threshold for a directed polymer in a random potential

Yan V Fyodorov, Pierre Le Doussal, Alberto Rosso, Christophe Texier

1703.10066

All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation

Jean Barbier, Nicolas Macris, Cynthia Rush

2006.07971

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Benjamin Aubin, Florent Krzakala, Yue M. Lu, Lenka Zdeborová

2006.06560

Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula

Jean Barbier, Mohamad Dia, Nicolas Macris, Florent Krzakala, Thibault Lesieur, Lenka Zdeborova

1606.04142

Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems

Silvio Franz, Giorgio Parisi, Maksim Sevelev, Pierfrancesco Urbani, Francesco Zamponi

1702.06919

Isotropic Brownian motions over complex fields as a solvable model for May-Wigner stability analysis

J. R. Ipsen, H. Schomerus

1602.06364

Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime

Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli, Florent Krzakala

2003.01054

Statistical mechanics approach to 1-bit compressed sensing

Yingying Xu, Yoshiyuki Kabashima

1301.1423

The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks

Ryo Karakida, Shotaro Akaho, Shun-ichi Amari

1906.02926

Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks

Abdulkadir Canatar, Blake Bordelon, Cengiz Pehlevan

2006.13198

Nonlinear Analogue of the May-Wigner Instability Transition

Yan V. Fyodorov, Boris A. Khoruzhenko

1509.05737

Fermionic neural-network states for ab-initio electronic structure

Kenny Choo, Antonio Mezzacapo, Giuseppe Carleo

1909.12852

Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitions

Valentina Ros, Gerard Ben Arous, Giulio Biroli, Chiara Cammarota

1804.02686

Statistical Physics of Hard Optimization Problems

Lenka Zdeborová

0806.4112

Non-Gaussian processes and neural networks at finite widths

Sho Yaida

1910.00019

The jamming transition in high dimension: an analytical study of the TAP equations and the effective thermodynamic potential

Ada Altieri, Silvio Franz, Giorgio Parisi

1607.00966

On statistics of bi-orthogonal eigenvectors in real and complex Ginibre ensembles: combining partial Schur decomposition with supersymmetry

Yan V Fyodorov

1710.04699

Does a growing static length scale control the glass transition?

Matthieu Wyart, Michael E. Cates

1705.06588

Quantum generative adversarial learning in a superconducting quantum circuit

Ling Hu, Shu-Hao Wu, Weizhou Cai, Yuwei Ma, Xianghao Mu, Yuan Xu, Haiyan Wang, Yipu Song, Dong-Ling Deng, Chang-Ling Zou, Luyan Sun

1808.02893

Probing transfer learning with a model of synthetic correlated datasets

Federica Gerace, Luca Saglietti, Stefano Sarao Mannelli, Andrew Saxe, Lenka Zdeborová

2106.05418

Properties of the geometry of solutions and capacity of multi-layer neural networks with Rectified Linear Units activations

Carlo Baldassi, Enrico M. Malatesta, Riccardo Zecchina

1907.07578

A random critical point separates brittle and ductile yielding transitions in amorphous materials

Misaki Ozawa, Ludovic Berthier, Giulio Biroli, Alberto Rosso, Gilles Tarjus

1803.11502

Following the evolution of glassy states under external perturbations: the full replica symmetry breaking solution

Corrado Rainone, Pierfrancesco Urbani

1512.00341

Scaling collapse at the jamming transition

Yoav Kallus

1507.02325

Glass and Jamming Transitions: From Exact Results to Finite-Dimensional Descriptions

Patrick Charbonneau, Jorge Kurchan, Giorgio Parisi, Pierfrancesco Urbani, Francesco Zamponi

1605.03008

Growing timescales and lengthscales characterizing vibrations of amorphous solids

Ludovic Berthier, Patrick Charbonneau, Yuliang Jin, Giorgio Parisi, Beatriz Seoane, Francesco Zamponi

1511.04201

Compression Driven Jamming of Athermal Frictionless Spherocylinders in Two Dimensions

Theodore Marschall, S. Teitel

1709.01973

The jamming transition as a paradigm to understand the loss landscape of deep neural networks

Mario Geiger, Stefano Spigler, Stéphane d'Ascoli, Levent Sagun, Marco Baity-Jesi, Giulio Biroli, Matthieu Wyart

1809.09349

Network Cosmology

Dmitri Krioukov, Maksim Kitsak, Robert S. Sinkovits, David Rideout, David Meyer, Marian Boguna

1203.2109

Solving Constraint Satisfaction Problems through Belief Propagation-guided decimation

Andrea Montanari, Federico Ricci-Tersenghi, Guilhem Semerjian

0709.1667

Growing multiplex networks

Vincenzo Nicosia, Ginestra Bianconi, Vito Latora, Marc Barthelemy

1302.7126

Statistical Mechanics of Multiplex Ensembles: Entropy and Overlap

Ginestra Bianconi

1303.4057

Deciphering the global organization of clustering in real complex networks

Pol Colomer-de-Simon, M. Angeles Serrano, Mariano G. Beiro, J. Ignacio Alvarez-Hamelin, Marian Boguna

1306.0112

Signal recovery using expectation consistent approximation for linear observations

Yoshiyuki Kabashima, Mikko Vehkapera

1401.5151

cond-mat.dis-nn

An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model

Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation

A Mean Field Theory of Batch Normalization

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

A typical reconstruction limit of compressed sensing based on Lp-norm minimization

Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach

Power-law out of time order correlation functions in the SYK model

Exponential expressivity in deep neural networks through transient chaos

Message Passing Algorithms for Compressed Sensing

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit

Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks

Mean Field Residual Networks: On the Edge of Chaos

Towards Understanding Grokking: An Effective Theory of Representation Learning

Understanding Deep Neural Networks with Rectified Linear Units

Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications

A mean-field limit for certain deep neural networks

The performance of modularity maximization in practical contexts

Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks

On the geometry of generalization and memorization in deep neural networks

Contrasting random and learned features in deep Bayesian linear regression

Feature Learning in Infinite-Width Neural Networks

Explaining Neural Scaling Laws

Sparse Estimation with the Swept Approximated Message-Passing Algorithm

Scaling description of generalization with number of parameters in deep learning

A jamming transition from under- to over-parametrization affects loss landscape and generalization

Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes

The edge of chaos: quantum field theory and deep neural networks

Geometric GAN

Stochastic Interpolants: A Unifying Framework for Flows and Diffusions

Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

Why does deep and cheap learning work so well?

Asymptotics of representation learning in finite Bayesian neural networks

Performance of Bayesian linear regression in a model with mismatch

From Integrable to Chaotic Systems: Universal Local Statistics of Lyapunov exponents

The Quantization Model of Neural Scaling

Bias-variance decomposition of overparameterized regression with random linear features

Learning curves of generic features maps for realistic datasets with a teacher-student model

Locality defeats the curse of dimensionality in convolutional teacher-student scenarios

The Gaussian equivalence of generative models for learning with shallow neural networks

Comparing Dynamics: Deep Neural Networks versus Glassy Systems

Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Pathological spectra of the Fisher information metric and its variants in deep neural networks

The role of regularization in classification of high-dimensional noisy Gaussian mixture

Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks

A Dynamical Model of Neural Scaling Laws

Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics

A Theory of Solving TAP Equations for Ising Models with General Invariant Random Matrices

Universality of jamming of non-spherical particles

Triple descent and the two kinds of overfitting: Where &amp; why do they appear?

Tensor Programs IVb: Adaptive Optimization in the Infinite-Width Limit

Nonperturbative renormalization for the neural network-QFT correspondence

Solving the Quantum Many-Body Problem with Artificial Neural Networks

Typical $l_1$-recovery limit of sparse vectors represented by concatenations of random orthogonal matrices

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)

Kac-Rice fixed point analysis for single- and multi-layered complex systems

Entropic gradient descent algorithms and wide flat minima

Neural Networks and Quantum Field Theory

Exponential number of equilibria and depinning threshold for a directed polymer in a random potential

All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula

Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems

Isotropic Brownian motions over complex fields as a solvable model for May-Wigner stability analysis

Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime

Statistical mechanics approach to 1-bit compressed sensing

The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks

Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks

Nonlinear Analogue of the May-Wigner Instability Transition

Fermionic neural-network states for ab-initio electronic structure

Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitions

Statistical Physics of Hard Optimization Problems

Non-Gaussian processes and neural networks at finite widths

The jamming transition in high dimension: an analytical study of the TAP equations and the effective thermodynamic potential

On statistics of bi-orthogonal eigenvectors in real and complex Ginibre ensembles: combining partial Schur decomposition with supersymmetry

Does a growing static length scale control the glass transition?

Quantum generative adversarial learning in a superconducting quantum circuit

Probing transfer learning with a model of synthetic correlated datasets

Triple descent and the two kinds of overfitting: Where & why do they appear?