p7stat.ML

stat.ML

3925 papers

Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models

Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, Quanquan Gu

5 annotations

2401.01335

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean

1701.06538

Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators

Yann Dubois, Balázs Galambosi, Percy Liang, Tatsunori B. Hashimoto

2404.04475

Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling

Hakan Inan, Khashayar Khosravi, Richard Socher

1611.01462

Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks

Ji Young Lee, Franck Dernoncourt

1603.03827

Layer Normalization

Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton

1607.06450

Neural Semantic Encoders

Tsendsuren Munkhdalai, Hong Yu

1607.04315

On the Computational Efficiency of Training Neural Networks

Roi Livni, Shai Shalev-Shwartz, Ohad Shamir

1410.1141

On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses

Anish Athalye, Nicholas Carlini

1804.03286

A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

Yarin Gal, Zoubin Ghahramani

1512.05287

Listen, Attend and Spell

William Chan, Navdeep Jaitly, Quoc V. Le, Oriol Vinyals

1508.01211

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio

1406.1078

Neural Machine Translation by Jointly Learning to Align and Translate

Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio

1409.0473

Underspecification Presents Challenges for Credibility in Modern Machine Learning

Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley

2011.03395

Character-Level Language Modeling with Deeper Self-Attention

Rami Al-Rfou, Dokook Choe, Noah Constant, Mandy Guo, Llion Jones

1808.04444

How Much Knowledge Can You Pack Into the Parameters of a Language Model?

Adam Roberts, Colin Raffel, Noam Shazeer

2002.08910

Non-Gaussianity of Stochastic Gradient Noise

Abhishek Panigrahi, Raghav Somani, Navin Goyal, Praneeth Netrapalli

1910.09626

Improving GANs Using Optimal Transport

Tim Salimans, Han Zhang, Alec Radford, Dimitris Metaxas

1803.05573

Expert Gate: Lifelong Learning with a Network of Experts

Rahaf Aljundi, Punarjay Chakravarty, Tinne Tuytelaars

1611.06194

Input Warping for Bayesian Optimization of Non-stationary Functions

Jasper Snoek, Kevin Swersky, Richard S. Zemel, Ryan P. Adams

1402.0929

Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity

Amit Daniely, Roy Frostig, Yoram Singer

1602.05897

Character-Aware Neural Language Models

Yoon Kim, Yacine Jernite, David Sontag, Alexander M. Rush

1508.06615

Document Context Language Models

Yangfeng Ji, Trevor Cohn, Lingpeng Kong, Chris Dyer, Jacob Eisenstein

1511.03962

GraphMix: Improved Training of GNNs for Semi-Supervised Learning

Vikas Verma, Meng Qu, Kenji Kawaguchi, Alex Lamb, Yoshua Bengio, Juho Kannala, Jian Tang

1909.11715

GLU Variants Improve Transformer

Noam Shazeer

2002.05202

Training with Quantization Noise for Extreme Model Compression

Angela Fan, Pierre Stock, Benjamin Graham, Edouard Grave, Remi Gribonval, Herve Jegou, Armand Joulin

2004.07320

GAN and VAE from an Optimal Transport Point of View

Aude Genevay, Gabriel Peyré, Marco Cuturi

1706.01807

Weight Uncertainty in Neural Networks

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, Daan Wierstra

1505.05424

Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

Orhan Firat, Kyunghyun Cho, Yoshua Bengio

1601.01073

End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results

Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio

1412.1602

Distributed Representations of Words and Phrases and their Compositionality

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean

1310.4546

Disentangling by Factorising

Hyunjik Kim, Andriy Mnih

1802.05983

Unsupervised Learning of Disentangled Representations from Video

Remi Denton, Vighnesh Birodkar

1705.10915

On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, Yoshua Bengio

1409.1259

An Autoencoder Approach to Learning Bilingual Word Representations

Sarath Chandar A P, Stanislas Lauly, Hugo Larochelle, Mitesh M. Khapra, Balaraman Ravindran, Vikas Raykar, Amrita Saha

1402.1454

Music Transformer

Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, Douglas Eck

1809.04281

Galactica: A Large Language Model for Science

Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, Robert Stojnic

2211.09085

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, Luke Zettlemoyer

1910.13461

Noisy Activation Functions

Caglar Gulcehre, Marcin Moczulski, Misha Denil, Yoshua Bengio

1603.00391

Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition

Haşim Sak, Andrew Senior, Kanishka Rao, Françoise Beaufays

1507.06947

WaveGrad: Estimating Gradients for Waveform Generation

Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, William Chan

2009.00713

DiffWave: A Versatile Diffusion Model for Audio Synthesis

Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, Bryan Catanzaro

2009.09761

Neural Machine Translation and Sequence-to-sequence Models: A Tutorial

Graham Neubig

1703.01619

Augment-and-Conquer Negative Binomial Processes

Mingyuan Zhou, Lawrence Carin

1209.1119

Variational Dropout and the Local Reparameterization Trick

Diederik P. Kingma, Tim Salimans, Max Welling

1506.02557

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu

1910.10683

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Yarin Gal, Zoubin Ghahramani

1506.02142

A Comprehensive Survey of Deep Learning for Image Captioning

Md. Zakir Hossain, Ferdous Sohel, Mohd Fairuz Shiratuddin, Hamid Laga

1810.04020

Bayesian Dark Knowledge

Anoop Korattikara, Vivek Rathod, Kevin Murphy, Max Welling

1506.04416

The Hidden Vulnerability of Distributed Learning in Byzantium

El Mahdi El Mhamdi, Rachid Guerraoui, Sébastien Rouault

1802.07927

Controlling Computation versus Quality for Neural Sequence Models

Ankur Bapna, Naveen Arivazhagan, Orhan Firat

2002.07106

High-Performance Large-Scale Image Recognition Without Normalization

Andrew Brock, Soham De, Samuel L. Smith, Karen Simonyan

2102.06171

Tutorial on Variational Autoencoders

Carl Doersch

1606.05908

Attention-Based Models for Speech Recognition

Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, Yoshua Bengio

1506.07503

Practical Bayesian Optimization of Machine Learning Algorithms

Jasper Snoek, Hugo Larochelle, Ryan P. Adams

1206.2944

Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks

Yuhang Li, Xin Dong, Wei Wang

1909.13144

Label-Only Membership Inference Attacks

Christopher A. Choquette-Choo, Florian Tramer, Nicholas Carlini, Nicolas Papernot

2007.14321

Pointer Networks

Oriol Vinyals, Meire Fortunato, Navdeep Jaitly

1506.03134

Reducing Transformer Depth on Demand with Structured Dropout

Angela Fan, Edouard Grave, Armand Joulin

1909.11556

Adafactor: Adaptive Learning Rates with Sublinear Memory Cost

Noam Shazeer, Mitchell Stern

1804.04235

Compressive Transformers for Long-Range Sequence Modelling

Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Timothy P. Lillicrap

1911.05507

Continuous control with deep reinforcement learning

Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra

1509.02971

LeanDojo: Theorem Proving with Retrieval-Augmented Language Models

Kaiyu Yang, Aidan M. Swope, Alex Gu, Rahul Chalamala, Peiyang Song, Shixing Yu, Saad Godil, Ryan Prenger, Anima Anandkumar

2306.15626

Multi-task Sequence to Sequence Learning

Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, Lukasz Kaiser

1511.06114

A Spectral Energy Distance for Parallel Speech Synthesis

Alexey A. Gritsenko, Tim Salimans, Rianne van den Berg, Jasper Snoek, Nal Kalchbrenner

2008.01160

Stable LM 2 1.6B Technical Report

Marco Bellagente, Jonathan Tow, Dakota Mahan, Duy Phung, Maksym Zhuravinskyi, Reshinth Adithyan, James Baicoianu, Ben Brooks, Nathan Cooper, Ashish Datta, Meng Lee, Emad Mostaque, Michael Pieler, Nikhil Pinnaparju, Paulo Rocha, Harry Saini, Hannah Teufel, Niccolo Zanichelli, Carlos Riquelme

2402.17834

Convolutional Dictionary Learning: A Comparative Review and New Algorithms

Cristina Garcia-Cardona, Brendt Wohlberg

1709.02893

Denoising Diffusion Probabilistic Models

Jonathan Ho, Ajay Jain, Pieter Abbeel

2006.11239

Improved Techniques for Training Score-Based Generative Models

Yang Song, Stefano Ermon

2006.09011

Deep Reinforcement Learning that Matters

Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, David Meger

1709.06560

True Few-Shot Learning with Language Models

Ethan Perez, Douwe Kiela, Kyunghyun Cho

2105.11447

Distilling the Knowledge in a Neural Network

Geoffrey Hinton, Oriol Vinyals, Jeff Dean

1503.02531

Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks

José Miguel Hernández-Lobato, Ryan P. Adams

1502.05336

Radioactive data: tracing through training

Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Hervé Jégou

2002.00937

Rényi Differential Privacy of the Sampled Gaussian Mechanism

Ilya Mironov, Kunal Talwar, Li Zhang

1908.10530

Fantastic Generalization Measures and Where to Find Them

Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, Samy Bengio

1912.02178

Analysing Mathematical Reasoning Abilities of Neural Models

David Saxton, Edward Grefenstette, Felix Hill, Pushmeet Kohli

1904.01557

Learning Transferable Architectures for Scalable Image Recognition

Barret Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V. Le

1707.07012

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, Soumith Chintala

1912.01703

Memory Networks

Jason Weston, Sumit Chopra, Antoine Bordes

1410.3916

Semi-Supervised Learning with Deep Generative Models

Diederik P. Kingma, Danilo J. Rezende, Shakir Mohamed, Max Welling

1406.5298

Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks

Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M. Rush, Bart van Merriënboer, Armand Joulin, Tomas Mikolov

1502.05698

ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding

Uri Shaham, Maor Ivgi, Avia Efrat, Jonathan Berant, Omer Levy

2305.14196

Deep Double Descent: Where Bigger Models and More Data Hurt

Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever

1912.02292

Large Language Models as Tool Makers

Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, Denny Zhou

2305.17126

DDSP: Differentiable Digital Signal Processing

Jesse Engel, Lamtharn Hantrakul, Chenjie Gu, Adam Roberts

2001.04643

Jukebox: A Generative Model for Music

Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever

2005.00341

Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser

Zahra Kadkhodaie, Eero P. Simoncelli

2007.13640

Toxicity Prediction using Deep Learning

Thomas Unterthiner, Andreas Mayr, Günter Klambauer, Sepp Hochreiter

1503.01445

Parameter Space Noise for Exploration

Matthias Plappert, Rein Houthooft, Prafulla Dhariwal, Szymon Sidor, Richard Y. Chen, Xi Chen, Tamim Asfour, Pieter Abbeel, Marcin Andrychowicz

1706.01905

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, Zhifeng Chen

2006.16668

Adaptive Attention Span in Transformers

Sainbayar Sukhbaatar, Edouard Grave, Piotr Bojanowski, Armand Joulin

1905.07799

Signature Kernel Conditional Independence Tests in Causal Discovery for Stochastic Processes

Georg Manten, Cecilia Casolo, Emilio Ferrucci, Søren Wengel Mogensen, Cristopher Salvi, Niki Kilbertus

2402.18477

Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, Amin Karbasi

2312.02119

Adversarial score matching and improved sampling for image generation

Alexia Jolicoeur-Martineau, Rémi Piché-Taillefer, Rémi Tachet des Combes, Ioannis Mitliagkas

2009.05475

Natural Neural Networks

Guillaume Desjardins, Karen Simonyan, Razvan Pascanu, Koray Kavukcuoglu

1507.00210

Generating Diverse High-Fidelity Images with VQ-VAE-2

Ali Razavi, Aaron van den Oord, Oriol Vinyals

1906.00446

Generative Modeling by Estimating Gradients of the Data Distribution

Yang Song, Stefano Ermon

1907.05600

Empirical Evaluation of Rectified Activations in Convolutional Network

Bing Xu, Naiyan Wang, Tianqi Chen, Mu Li

1505.00853

WaveGlow: A Flow-based Generative Network for Speech Synthesis

Ryan Prenger, Rafael Valle, Bryan Catanzaro

1811.00002

Sliced Score Matching: A Scalable Approach to Density and Score Estimation

Yang Song, Sahaj Garg, Jiaxin Shi, Stefano Ermon

1905.07088

Learning without Forgetting

Zhizhong Li, Derek Hoiem

1606.09282

Normalizing Flows for Probabilistic Modeling and Inference

George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, Balaji Lakshminarayanan

1912.02762

Generating Long Sequences with Sparse Transformers

Rewon Child, Scott Gray, Alec Radford, Ilya Sutskever

1904.10509

Spectral Normalization for Generative Adversarial Networks

Takeru Miyato, Toshiki Kataoka, Masanori Koyama, Yuichi Yoshida

1802.05957

Identity Crisis: Memorization and Generalization under Extreme Overparameterization

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Michael C. Mozer, Yoram Singer

1902.04698

Resolving Gendered Ambiguous Pronouns with BERT

Matei Ionita, Yury Kashnitsky, Ken Krige, Vladimir Larin, Denis Logvinenko, Atanas Atanasov

1906.01161

DARTS: Differentiable Architecture Search

Hanxiao Liu, Karen Simonyan, Yiming Yang

1806.09055

Deep Learning with Differential Privacy

Martín Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, Li Zhang

1607.00133

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov

1901.02860

Rényi Divergence and Kullback-Leibler Divergence

Tim van Erven, Peter Harremoës

1206.2459

Efficient Neural Architecture Search via Parameter Sharing

Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, Jeff Dean

1802.03268

Deeply-Supervised Nets

Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, Zhuowen Tu

1409.5185

Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience

Vaishnavh Nagarajan, J. Zico Kolter

1905.13344

Analyzing Federated Learning through an Adversarial Lens

Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, Seraphin Calo

1811.12470

Do ImageNet Classifiers Generalize to ImageNet?

Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, Vaishaal Shankar

1902.10811

Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin

Colin Wei, Tengyu Ma

1910.04284

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Mingxing Tan, Quoc V. Le

1905.11946

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks

Alexander Robey, Eric Wong, Hamed Hassani, George J. Pappas

2310.03684

Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation

Colin Wei, Tengyu Ma

1905.03684

Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging

Nils Reimers, Iryna Gurevych

1707.09861

Reformer: The Efficient Transformer

Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya

2001.04451

GANSynth: Adversarial Neural Audio Synthesis

Jesse Engel, Kumar Krishna Agrawal, Shuo Chen, Ishaan Gulrajani, Chris Donahue, Adam Roberts

1902.08710

Massively Multitask Networks for Drug Discovery

Bharath Ramsundar, Steven Kearnes, Patrick Riley, Dale Webster, David Konerding, Vijay Pande

1502.02072

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent

Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, Jeffrey Pennington

1902.06720

MelNet: A Generative Model for Audio in the Frequency Domain

Sean Vasquez, Mike Lewis

1906.01083

Multi-task Neural Networks for QSAR Predictions

George E. Dahl, Navdeep Jaitly, Ruslan Salakhutdinov

1406.1231

Robust and interpretable blind image denoising via bias-free convolutional neural networks

Sreyas Mohan, Zahra Kadkhodaie, Eero P. Simoncelli, Carlos Fernandez-Granda

1906.05478

Deconvolutional Latent-Variable Model for Text Sequence Matching

Dinghan Shen, Yizhe Zhang, Ricardo Henao, Qinliang Su, Lawrence Carin

1709.07109

Non-square matrix sensing without spurious local minima via the Burer-Monteiro approach

Dohyung Park, Anastasios Kyrillidis, Constantine Caramanis, Sujay Sanghavi

1609.03240

Autoregressive Entity Retrieval

Nicola De Cao, Gautier Izacard, Sebastian Riedel, Fabio Petroni

2010.00904

Neural Empirical Bayes

Saeed Saremi, Aapo Hyvarinen

1903.02334

Adaptive Neural Networks for Efficient Inference

Tolga Bolukbasi, Joseph Wang, Ofer Dekel, Venkatesh Saligrama

1702.07811

Neural source-filter waveform models for statistical parametric speech synthesis

Xin Wang, Shinji Takaki, Junichi Yamagishi

1904.12088

Scaling Laws for Neural Language Models

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei

2001.08361

Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings

Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, Alan W Black

1904.04047

Autoregressive Quantile Networks for Generative Modeling

Georg Ostrovski, Will Dabney, Rémi Munos

1806.05575

Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement

Jason Lee, Elman Mansimov, Kyunghyun Cho

1802.06901

Interpretable Word Embeddings via Informative Priors

Miriam Hurtado Bodell, Martin Arvidsson, Måns Magnusson

1909.01459

Large Scale Adversarial Representation Learning

Jeff Donahue, Karen Simonyan

1907.02544

Better Fine-Tuning by Reducing Representational Collapse

Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, Sonal Gupta

2008.03156

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, François Fleuret

2006.16236

Sparse GPU Kernels for Deep Learning

Trevor Gale, Matei Zaharia, Cliff Young, Erich Elsen

2006.10901

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, Justin Gilmer

2006.16241

Adversarial Video Generation on Complex Datasets

Aidan Clark, Jeff Donahue, Karen Simonyan

1907.06571

Large Scale GAN Training for High Fidelity Natural Image Synthesis

Andrew Brock, Jeff Donahue, Karen Simonyan

1809.11096

Functional Variational Bayesian Neural Networks

Shengyang Sun, Guodong Zhang, Jiaxin Shi, Roger Grosse

1903.05779

Wasserstein Auto-Encoders

Ilya Tolstikhin, Olivier Bousquet, Sylvain Gelly, Bernhard Schoelkopf

1711.01558

Rethinking Attention with Performers

Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, Adrian Weller

2009.14794

Noisy Networks for Exploration

Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot, Jacob Menick, Ian Osband, Alex Graves, Vlad Mnih, Remi Munos, Demis Hassabis, Olivier Pietquin, Charles Blundell, Shane Legg

1706.10295

Evolution Strategies as a Scalable Alternative to Reinforcement Learning

Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, Ilya Sutskever

1703.03864

The Power of Depth for Feedforward Neural Networks

Ronen Eldan, Ohad Shamir

1512.03965

Learning deep kernels for exponential family densities

Li Wenliang, Danica J. Sutherland, Heiko Strathmann, Arthur Gretton

1811.08357

A Style-Based Generator Architecture for Generative Adversarial Networks

Tero Karras, Samuli Laine, Timo Aila

1812.04948

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?

Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel

2204.05832

Analyzing and Improving the Image Quality of StyleGAN

Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila

1912.04958

On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models

Erik Nijkamp, Mitch Hill, Tian Han, Song-Chun Zhu, Ying Nian Wu

1903.12370

Gradient Estimators for Implicit Models

Yingzhen Li, Richard E. Turner

1705.07107

Curiosity-driven Exploration by Self-supervised Prediction

Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, Trevor Darrell

1705.05363

A Spectral Approach to Gradient Estimation for Implicit Distributions

Jiaxin Shi, Shengyang Sun, Jun Zhu

1806.02925

Deep Learning Based Text Classification: A Comprehensive Review

Shervin Minaee, Nal Kalchbrenner, Erik Cambria, Narjes Nikzad, Meysam Chenaghlu, Jianfeng Gao

2004.03705

Show Your Work: Improved Reporting of Experimental Results

Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, Noah A. Smith

1909.03004

CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data

Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, Edouard Grave

1911.00359

A Little Is Enough: Circumventing Defenses For Distributed Learning

Moran Baruch, Gilad Baruch, Yoav Goldberg

1902.06156

Dual Supervised Learning

Yingce Xia, Tao Qin, Wei Chen, Jiang Bian, Nenghai Yu, Tie-Yan Liu

1707.00415

A Neural Representation of Sketch Drawings

David Ha, Douglas Eck

1704.03477

Neural Architecture Search with Bayesian Optimisation and Optimal Transport

Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, Eric Xing

1802.07191

Column Networks for Collective Classification

Trang Pham, Truyen Tran, Dinh Phung, Svetha Venkatesh

1609.04508

Hierarchical Representations for Efficient Architecture Search

Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu

1711.00436

Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding

Nan Rosemary Ke, Anirudh Goyal, Olexa Bilaniuk, Jonathan Binas, Michael C. Mozer, Chris Pal, Yoshua Bengio

1809.03702

i-RevNet: Deep Invertible Networks

Jörn-Henrik Jacobsen, Arnold Smeulders, Edouard Oyallon

1802.07088

Sensitivity and Generalization in Neural Networks: an Empirical Study

Roman Novak, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein

1802.08760

FiLM: Visual Reasoning with a General Conditioning Layer

Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, Aaron Courville

1709.07871

Towards Federated Learning at Scale: System Design

Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, Jason Roselander

1902.01046

The Implicit Bias of Gradient Descent on Separable Data

Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, Nathan Srebro

1710.10345

Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data

Yuanzhi Li, Yingyu Liang

1808.01204

Size-Independent Sample Complexity of Neural Networks

Noah Golowich, Alexander Rakhlin, Ohad Shamir

1712.06541

Mesh-TensorFlow: Deep Learning for Supercomputers

Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanantakool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, Ryan Sepassi, Blake Hechtman

1811.02084

Bridging the Gap between Training and Inference for Neural Machine Translation

Wen Zhang, Yang Feng, Fandong Meng, Di You, Qun Liu

1906.02448

Neural Text Generation with Unlikelihood Training

Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, Jason Weston

1908.04319

Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates

Dong Yin, Yudong Chen, Kannan Ramchandran, Peter Bartlett

1803.01498

Adversarial Feature Matching for Text Generation

Yizhe Zhang, Zhe Gan, Kai Fan, Zhi Chen, Ricardo Henao, Dinghan Shen, Lawrence Carin

1706.03850

Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks

Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruosong Wang

1901.08584

The Evolved Transformer

David R. So, Chen Liang, Quoc V. Le

1901.11117

Sanity Checks for Saliency Maps

Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, Been Kim

1810.03292

Deep AutoRegressive Networks

Karol Gregor, Ivo Danihelka, Andriy Mnih, Charles Blundell, Daan Wierstra

1310.8499

Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks

Difan Zou, Yuan Cao, Dongruo Zhou, Quanquan Gu

1811.08888

On the importance of single directions for generalization

Ari S. Morcos, David G. T. Barrett, Neil C. Rabinowitz, Matthew Botvinick

1803.06959

Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers

Zeyuan Allen-Zhu, Yuanzhi Li, Yingyu Liang

1811.04918

A Constructive Prediction of the Generalization Error Across Scales

Jonathan S. Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, Nir Shavit

1909.12673

The Unusual Effectiveness of Averaging in GAN Training

Yasin Yazıcı, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, Vijay Chandrasekhar

1806.04498

Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors

Soumya Ghosh, Jiayu Yao, Finale Doshi-Velez

1806.05975

Deep Learning with Dynamic Computation Graphs

Moshe Looks, Marcello Herreshoff, DeLesley Hutchins, Peter Norvig

1702.02181

Neural Processes

Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J. Rezende, S. M. Ali Eslami, Yee Whye Teh

1807.01622

Learning values across many orders of magnitude

Hado van Hasselt, Arthur Guez, Matteo Hessel, Volodymyr Mnih, David Silver

1602.07714

CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information

Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, Lawrence Carin

2006.12013

Noise Contrastive Priors for Functional Uncertainty

Danijar Hafner, Dustin Tran, Timothy Lillicrap, Alex Irpan, James Davidson

1807.09289

Safe and Efficient Off-Policy Reinforcement Learning

Rémi Munos, Tom Stepleton, Anna Harutyunyan, Marc G. Bellemare

1606.02647

Towards Conceptual Compression

Karol Gregor, Frederic Besse, Danilo Jimenez Rezende, Ivo Danihelka, Daan Wierstra

1604.08772

Unifying Count-Based Exploration and Intrinsic Motivation

Marc G. Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, Remi Munos

1606.01868

A Study of Reinforcement Learning for Neural Machine Translation

Lijun Wu, Fei Tian, Tao Qin, Jianhuang Lai, Tie-Yan Liu

1808.08866

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel

1611.02779

Overcoming catastrophic forgetting in neural networks

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, Raia Hadsell

1612.00796

Differentiable Compositional Kernel Learning for Gaussian Processes

Shengyang Sun, Guodong Zhang, Chaoqi Wang, Wenyuan Zeng, Jiaman Li, Roger Grosse

1806.04326

BEGAN: Boundary Equilibrium Generative Adversarial Networks

David Berthelot, Thomas Schumm, Luke Metz

1703.10717

Self-Attention Generative Adversarial Networks

Han Zhang, Ian Goodfellow, Dimitris Metaxas, Augustus Odena

1805.08318

Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling

Carlos Riquelme, George Tucker, Jasper Snoek

1802.09127

Adversarial Removal of Demographic Attributes from Text Data

Yanai Elazar, Yoav Goldberg

1808.06640

Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model

Guodong Zhang, Lala Li, Zachary Nado, James Martens, Sushant Sachdeva, George E. Dahl, Christopher J. Shallue, Roger Grosse

1907.04164

Scaling limit of the Stein variational gradient descent: the mean field regime

Jianfeng Lu, Yulong Lu, James Nolen

1805.04035

cGANs with Projection Discriminator

Takeru Miyato, Masanori Koyama

1802.05637

Towards Accurate Generative Models of Video: A New Metric & Challenges

Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, Sylvain Gelly

1812.01717

Interactive Differentiable Simulation

Eric Heiden, David Millard, Hejia Zhang, Gaurav S. Sukhatme

1905.10706

InfoVAE: Information Maximizing Variational Autoencoders

Shengjia Zhao, Jiaming Song, Stefano Ermon

1706.02262

Tensor2Tensor for Neural Machine Translation

Ashish Vaswani, Samy Bengio, Eugene Brevdo, Francois Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Łukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, Jakob Uszkoreit

1803.07416

MMD GAN: Towards Deeper Understanding of Moment Matching Network

Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, Barnabás Póczos

1705.08584

Revealing the Dark Secrets of BERT

Olga Kovaleva, Alexey Romanov, Anna Rogers, Anna Rumshisky

1908.08593

Energy-based models for atomic-resolution protein conformations

Yilun Du, Joshua Meier, Jerry Ma, Rob Fergus, Alexander Rives

2004.13167

Linformer: Self-Attention with Linear Complexity

Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, Hao Ma

2006.04768

Maximum Entropy Generators for Energy-Based Models

Rithesh Kumar, Sherjil Ozair, Anirudh Goyal, Aaron Courville, Yoshua Bengio

1901.08508

Hierarchical Implicit Models and Likelihood-Free Variational Inference

Dustin Tran, Rajesh Ranganath, David M. Blei

1702.08896

Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model

Erik Nijkamp, Mitch Hill, Song-Chun Zhu, Ying Nian Wu

1904.09770

Acquisition of Chess Knowledge in AlphaZero

Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, Adam Pearce, Demis Hassabis, Been Kim, Ulrich Paquet, Vladimir Kramnik

2111.09259

Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit

Boaz Barak, Benjamin L. Edelman, Surbhi Goel, Sham Kakade, Eran Malach, Cyril Zhang

2207.08799

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, Lucy Vasserman

1903.04561

Real Time Image Saliency for Black Box Classifiers

Piotr Dabkowski, Yarin Gal

1705.07857

How Does Adaptive Optimization Impact Local Neural Network Geometry?

Kaiqi Jiang, Dhruv Malik, Yuanzhi Li

2211.02254

On the Spectral Bias of Neural Networks

Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, Aaron Courville

1806.08734

Measuring Sample Quality with Kernels

Jackson Gorham, Lester Mackey

1703.01717

Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference

Geoffrey Roeder, Yuhuai Wu, David Duvenaud

1703.09194

Frame-Recurrent Video Super-Resolution

Mehdi S. M. Sajjadi, Raviteja Vemulapalli, Matthew Brown

1801.04590

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst

1606.09375

Computational Optimal Transport

Gabriel Peyré, Marco Cuturi

1803.00567

Predicting Parameters in Deep Learning

Misha Denil, Babak Shakibi, Laurent Dinh, Marc'Aurelio Ranzato, Nando de Freitas

1306.0543

Integrated Model, Batch and Domain Parallelism in Training Neural Networks

Amir Gholami, Ariful Azad, Peter Jin, Kurt Keutzer, Aydin Buluc

1712.04432

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, Wenzhe Shi

1609.04802

Memory Augmented Neural Networks with Wormhole Connections

Caglar Gulcehre, Sarath Chandar, Yoshua Bengio

1701.08718

Stochastic Pooling for Regularization of Deep Convolutional Neural Networks

Matthew D. Zeiler, Rob Fergus

1301.3557

Private Learning and Sanitization: Pure vs. Approximate Differential Privacy

Amos Beimel, Kobbi Nissim, Uri Stemmer

1407.2674

Spectrally-normalized margin bounds for neural networks

Peter Bartlett, Dylan J. Foster, Matus Telgarsky

1706.08498

Mitigating Sybils in Federated Learning Poisoning

Clement Fung, Chris J. M. Yoon, Ivan Beschastnikh

1808.04866

DeepArchitect: Automatically Designing and Training Deep Architectures

Renato Negrinho, Geoff Gordon

1704.08792

Federated Learning for Ultra-Reliable Low-Latency V2V Communications

Sumudu Samarakoon, Mehdi Bennis, Walid Saad, Merouane Debbah

1805.09253

Horovod: fast and easy distributed deep learning in TensorFlow

Alexander Sergeev, Mike Del Balso

1802.05799

Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach

Wenda Zhou, Victor Veitch, Morgane Austern, Ryan P. Adams, Peter Orbanz

1804.05862

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

Han Xiao, Kashif Rasul, Roland Vollgraf

1708.07747

Spectral Signatures in Backdoor Attacks

Brandon Tran, Jerry Li, Aleksander Madry

1811.00636

Expanding the Reach of Federated Learning by Reducing Client Resource Requirements

Sebastian Caldas, Jakub Konečny, H. Brendan McMahan, Ameet Talwalkar

1812.07210

Smart Augmentation - Learning an Optimal Data Augmentation Strategy

Joseph Lemley, Shabab Bazrafkan, Peter Corcoran

1703.08383

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering

Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, Biplav Srivastava

1811.03728

Gaussian Process Kernels for Pattern Discovery and Extrapolation

Andrew Gordon Wilson, Ryan Prescott Adams

1302.4245

Deep Neural Networks as Gaussian Processes

Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein

1711.00165

Learning Deep Features via Congenerous Cosine Loss for Person Recognition

Yu Liu, Hongyang Li, Xiaogang Wang

1702.06890

Applied Federated Learning: Improving Google Keyboard Query Suggestions

Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, Françoise Beaufays

1812.02903

Structured Prediction Energy Networks

David Belanger, Andrew McCallum

1511.06350

Towards Principled Methods for Training Generative Adversarial Networks

Martin Arjovsky, Léon Bottou

1701.04862

Towards a Human-like Open-Domain Chatbot

Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, Quoc V. Le

2001.09977

Sub-Gaussian estimators of the mean of a random vector

Gábor Lugosi, Shahar Mendelson

1702.00482

Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation

Greg Yang

1902.04760

Blocks and Fuel: Frameworks for deep learning

Bart van Merriënboer, Dzmitry Bahdanau, Vincent Dumoulin, Dmitriy Serdyuk, David Warde-Farley, Jan Chorowski, Yoshua Bengio

1506.00619

Variational Dropout Sparsifies Deep Neural Networks

Dmitry Molchanov, Arsenii Ashukha, Dmitry Vetrov

1701.05369

Completing Any Low-rank Matrix, Provably

Yudong Chen, Srinadh Bhojanapalli, Sujay Sanghavi, Rachel Ward

1306.2979

A Statistical Perspective on Algorithmic Leveraging

Ping Ma, Michael W. Mahoney, Bin Yu

1306.5362

On the Quality of the Initial Basin in Overspecified Neural Networks

Itay Safran, Ohad Shamir

1511.04210

Relational inductive biases, deep learning, and graph networks

Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, Razvan Pascanu

1806.01261

How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking

Nicola De Cao, Michael Schlichtkrull, Wilker Aziz, Ivan Titov

2004.14992

Communication-Computation Efficient Gradient Coding

Min Ye, Emmanuel Abbe

1802.03475

Progressive Growing of GANs for Improved Quality, Stability, and Variation

Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen

1710.10196

High-Accuracy Low-Precision Training

Christopher De Sa, Megan Leszczynski, Jian Zhang, Alana Marzoev, Christopher R. Aberger, Kunle Olukotun, Christopher Ré

1803.03383

Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm

Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, Sune Lehmann

1708.00524

Negative Momentum for Improved Game Dynamics

Gauthier Gidel, Reyhane Askari Hemmat, Mohammad Pezeshki, Remi Lepriol, Gabriel Huang, Simon Lacoste-Julien, Ioannis Mitliagkas

1807.04740

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli

1503.03585

Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks

Yuanzhi Li, Colin Wei, Tengyu Ma

1907.04595

Black-box Importance Sampling

Qiang Liu, Jason D. Lee

1610.05247

ReZero is All You Need: Fast Convergence at Large Depth

Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Garrison W. Cottrell, Julian McAuley

2003.04887

Molecular geometry prediction using a deep generative graph neural network

Elman Mansimov, Omar Mahmood, Seokho Kang, Kyunghyun Cho

1904.00314

Measuring Sample Quality with Diffusions

Jackson Gorham, Andrew B. Duncan, Sebastian J. Vollmer, Lester Mackey

1611.06972

Toward Controlled Generation of Text

Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, Eric P. Xing

1703.00955

Near-optimal-sample estimators for spherical Gaussian mixtures

Jayadev Acharya, Ashkan Jafarpour, Alon Orlitsky, Ananda Theertha Suresh

1402.4746

Deep Information Propagation

Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein

1611.01232

Global and Local Uncertainty Principles for Signals on Graphs

Nathanael Perraudin, Benjamin Ricaud, David Shuman, Pierre Vandergheynst

1603.03030

Interpretation of Natural Language Rules in Conversational Machine Reading

Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Singh, Tim Rocktäschel, Mike Sheldon, Guillaume Bouchard, Sebastian Riedel

1809.01494

Gaussian Process Behaviour in Wide Deep Neural Networks

Alexander G. de G. Matthews, Mark Rowland, Jiri Hron, Richard E. Turner, Zoubin Ghahramani

1804.11271

Norm-Based Capacity Control in Neural Networks

Behnam Neyshabur, Ryota Tomioka, Nathan Srebro

1503.00036

Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks

Ali Shafahi, W. Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, Tom Goldstein

1804.00792

Explaining and Harnessing Adversarial Examples

Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy

1412.6572

Learning to Drive in a Day

Alex Kendall, Jeffrey Hawke, David Janz, Przemyslaw Mazur, Daniele Reda, John-Mark Allen, Vinh-Dieu Lam, Alex Bewley, Amar Shah

1807.00412

Born Again Neural Networks

Tommaso Furlanello, Zachary C. Lipton, Michael Tschannen, Laurent Itti, Anima Anandkumar

1805.04770

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, Zehan Wang

1609.05158

Parseval Networks: Improving Robustness to Adversarial Examples

Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, Nicolas Usunier

1704.08847

Sever: A Robust Meta-Algorithm for Stochastic Optimization

Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jerry Li, Jacob Steinhardt, Alistair Stewart

1803.02815

Harmonic Networks: Deep Translation and Rotation Equivariance

Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, Gabriel J. Brostow

1612.04642

Nonparametric regression using deep neural networks with ReLU activation function

Johannes Schmidt-Hieber

1708.06633

Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters

Hao Zhang, Zeyu Zheng, Shizhen Xu, Wei Dai, Qirong Ho, Xiaodan Liang, Zhiting Hu, Jinliang Wei, Pengtao Xie, Eric P. Xing

1706.03292

Theoretical insights into the optimization landscape of over-parameterized shallow neural networks

Mahdi Soltanolkotabi, Adel Javanmard, Jason D. Lee

1707.04926

Learning mixtures of spherical Gaussians: moment methods and spectral decompositions

Daniel Hsu, Sham M. Kakade

1206.5766

Stochastic Variational Deep Kernel Learning

Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, Eric P. Xing

1611.00336

Variational Lossy Autoencoder

Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, Pieter Abbeel

1611.02731

Train longer, generalize better: closing the generalization gap in large batch training of neural networks

Elad Hoffer, Itay Hubara, Daniel Soudry

1705.08741

To prune, or not to prune: exploring the efficacy of pruning for model compression

Michael Zhu, Suyog Gupta

1710.01878

Learning Scalable Deep Kernels with Recurrent Structure

Maruan Al-Shedivat, Andrew Gordon Wilson, Yunus Saatchi, Zhiting Hu, Eric P. Xing

1610.08936

Categorical Reparameterization with Gumbel-Softmax

Eric Jang, Shixiang Gu, Ben Poole

1611.01144

Entropy-SGD: Biasing Gradient Descent Into Wide Valleys

Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, Riccardo Zecchina

1611.01838

Practical Secure Aggregation for Federated Learning on User-Held Data

Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, Karn Seth

1611.04482

Generalization in Deep Learning

Kenji Kawaguchi, Leslie Pack Kaelbling, Yoshua Bengio

1710.05468

Gated Graph Sequence Neural Networks

Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard Zemel

1511.05493

Understanding Batch Normalization

Johan Bjorck, Carla Gomes, Bart Selman, Kilian Q. Weinberger

1806.02375

Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks

Minmin Chen, Jeffrey Pennington, Samuel S. Schoenholz

1806.05394

Pseudo Numerical Methods for Diffusion Models on Manifolds

Luping Liu, Yi Ren, Zhijie Lin, Zhou Zhao

2202.09778

Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques, Craig Ferguson, Noah Jones, Agata Lapedriza, Rosalind Picard

1906.09308

Deep Convolutional Networks as shallow Gaussian Processes

Adrià Garriga-Alonso, Carl Edward Rasmussen, Laurence Aitchison

1808.05587

Federated Multi-Task Learning

Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, Ameet Talwalkar

1705.10467

Loss minimization and parameter estimation with heavy tails

Daniel Hsu, Sivan Sabato

1307.1827

Prototypical Networks for Few-shot Learning

Jake Snell, Kevin Swersky, Richard S. Zemel

1703.05175

Learning to Optimize

Ke Li, Jitendra Malik

1606.01885

Grammar as a Foreign Language

Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, Geoffrey Hinton

1412.7449

Bootstrap your own latent: A new approach to self-supervised Learning

Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko

2006.07733

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

Dan Hendrycks, Thomas Dietterich

1903.12261

Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks

Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, Jeffrey Pennington

1806.05393

Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes

Roman Novak, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Greg Yang, Jiri Hron, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein

1810.05148

On the Convergence Rate of Training Recurrent Neural Networks

Zeyuan Allen-Zhu, Yuanzhi Li, Zhao Song

1810.12065

cpSGD: Communication-efficient and differentially-private distributed SGD

Naman Agarwal, Ananda Theertha Suresh, Felix Yu, Sanjiv Kumar, H. Brendan Mcmahan

1805.10559

Can SGD Learn Recurrent Neural Networks with Provable Generalization?

Zeyuan Allen-Zhu, Yuanzhi Li

1902.01028

Revisiting Self-Training for Neural Sequence Generation

Junxian He, Jiatao Gu, Jiajun Shen, Marc'Aurelio Ranzato

1909.13788

Query-limited Black-box Attacks to Classifiers

Fnu Suya, Yuan Tian, David Evans, Paolo Papotti

1712.08713

Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking

Michael Sejr Schlichtkrull, Nicola De Cao, Ivan Titov

2010.00577

Group Equivariant Convolutional Networks

Taco S. Cohen, Max Welling

1602.07576

Agnostic Estimation of Mean and Covariance

Kevin A. Lai, Anup B. Rao, Santosh Vempala

1604.06968

Relational Deep Reinforcement Learning

Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, Murray Shanahan, Victoria Langston, Razvan Pascanu, Matthew Botvinick, Oriol Vinyals, Peter Battaglia

1806.01830

Adaptive Gradient Methods with Dynamic Bound of Learning Rate

Liangchen Luo, Yuanhao Xiong, Yan Liu, Xu Sun

1902.09843

Learned Optimizers that Scale and Generalize

Olga Wichrowska, Niru Maheswaranathan, Matthew W. Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando de Freitas, Jascha Sohl-Dickstein

1703.04813

Matching Networks for One Shot Learning

Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra

1606.04080

Low-Rank Matrix and Tensor Completion via Adaptive Sampling

Akshay Krishnamurthy, Aarti Singh

1304.4672

Graph networks as learnable physics engines for inference and control

Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, Peter Battaglia

1806.01242

MolGAN: An implicit generative model for small molecular graphs

Nicola De Cao, Thomas Kipf

1805.11973

SGD on Neural Networks Learns Functions of Increasing Complexity

Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L. Edelman, Fred Zhang, Boaz Barak

1905.11604

Robust Estimators in High Dimensions without the Computational Intractability

Ilias Diakonikolas, Gautam Kamath, Daniel Kane, Jerry Li, Ankur Moitra, Alistair Stewart

1604.06443

Adversarial Attacks on Neural Networks for Graph Data

Daniel Zügner, Amir Akbarnejad, Stephan Günnemann

1805.07984

The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least Squares

Rong Ge, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli

1904.12838

Visualizing the Loss Landscape of Neural Nets

Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, Tom Goldstein

1712.09913

Learning with Differential Privacy: Stability, Learnability and the Sufficiency and Necessity of ERM Principle

Yu-Xiang Wang, Jing Lei, Stephen E. Fienberg

1502.06309

Recurrent Dirichlet Belief Networks for Interpretable Dynamic Relational Data Modelling

Yaqiong Li, Xuhui Fan, Ling Chen, Bin Li, Zheng Yu, Scott A. Sisson

2002.10235

Benefits of depth in neural networks

Matus Telgarsky

1602.04485

Learning Neural Networks with Two Nonlinear Layers in Polynomial Time

Surbhi Goel, Adam Klivans

1709.06010

Semi-Supervised Learning with Generative Adversarial Networks

Augustus Odena

1606.01583

Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)

Andrew Gordon Wilson, Hannes Nickisch

1503.01057

Crowdsourcing Multiple Choice Science Questions

Johannes Welbl, Nelson F. Liu, Matt Gardner

1707.06209

Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes

Lei Wu, Zhanxing Zhu, Weinan E

1706.10239

Masked Autoregressive Flow for Density Estimation

George Papamakarios, Theo Pavlakou, Iain Murray

1705.07057

Understanding Black-box Predictions via Influence Functions

Pang Wei Koh, Percy Liang

1703.04730

Bayesian GAN

Yunus Saatchi, Andrew Gordon Wilson

1705.09558

Self-Normalizing Neural Networks

Günter Klambauer, Thomas Unterthiner, Andreas Mayr, Sepp Hochreiter

1706.02515

Maxout Networks

Ian J. Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, Yoshua Bengio

1302.4389

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, Quoc V. Le

1904.08779

A Method of Moments for Mixture Models and Hidden Markov Models

Animashree Anandkumar, Daniel Hsu, Sham M. Kakade

1203.0683

On the Expressive Power of Deep Learning: A Tensor Analysis

Nadav Cohen, Or Sharir, Amnon Shashua

1509.05009

Reliably Learning the ReLU in Polynomial Time

Surbhi Goel, Varun Kanade, Adam Klivans, Justin Thaler

1611.10258

On the Expressive Power of Deep Neural Networks

Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, Jascha Sohl-Dickstein

1606.05336

A la Carte - Learning Fast Kernels

Zichao Yang, Alexander J. Smola, Le Song, Andrew Gordon Wilson

1412.6493

Stochastic Activation Pruning for Robust Adversarial Defense

Guneet S. Dhillon, Kamyar Azizzadenesheli, Zachary C. Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, Anima Anandkumar

1803.01442

MCMC for Variationally Sparse Gaussian Processes

James Hensman, Alexander G. de G. Matthews, Maurizio Filippone, Zoubin Ghahramani

1506.04000

Recovery Guarantees for One-hidden-layer Neural Networks

Kai Zhong, Zhao Song, Prateek Jain, Peter L. Bartlett, Inderjit S. Dhillon

1706.03175

Tensor decompositions for learning latent variable models

Anima Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, Matus Telgarsky

1210.7559

Convolutional Networks on Graphs for Learning Molecular Fingerprints

David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparraguirre, Rafael Gómez-Bombarelli, Timothy Hirzel, Alán Aspuru-Guzik, Ryan P. Adams

1509.09292

Auxiliary Deep Generative Models

Lars Maaløe, Casper Kaae Sønderby, Søren Kaae Sønderby, Ole Winther

1602.05473

Identity Matters in Deep Learning

Moritz Hardt, Tengyu Ma

1611.04231

Deep Neural Network Approximation Theory

Dennis Elbrächter, Dmytro Perekrestenko, Philipp Grohs, Helmut Bölcskei

1901.02220

From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood

Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, Percy Liang

1704.07926

Accelerated Mini-Batch Stochastic Dual Coordinate Ascent

Shai Shalev-Shwartz, Tong Zhang

1305.2581

Steerable CNNs

Taco S. Cohen, Max Welling

1612.08498

PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications

Tim Salimans, Andrej Karpathy, Xi Chen, Diederik P. Kingma

1701.05517

Neural Combinatorial Optimization with Reinforcement Learning

Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, Samy Bengio

1611.09940

Pylearn2: a machine learning research library

Ian J. Goodfellow, David Warde-Farley, Pascal Lamblin, Vincent Dumoulin, Mehdi Mirza, Razvan Pascanu, James Bergstra, Frédéric Bastien, Yoshua Bengio

1308.4214

A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach

Aryan Mokhtari, Asuman Ozdaglar, Sarath Pattathil

1901.08511

Thoughts on Massively Scalable Gaussian Processes

Andrew Gordon Wilson, Christoph Dann, Hannes Nickisch

1511.01870

Variational Auto-encoded Deep Gaussian Processes

Zhenwen Dai, Andreas Damianou, Javier González, Neil Lawrence

1511.06455

Higher-Order Explanations of Graph Neural Networks via Relevant Walks

Thomas Schnake, Oliver Eberle, Jonas Lederer, Shinichi Nakajima, Kristof T. Schütt, Klaus-Robert Müller, Grégoire Montavon

2006.03589

Orthogonal Random Features

Felix X. Yu, Ananda Theertha Suresh, Krzysztof Choromanski, Daniel Holtmann-Rice, Sanjiv Kumar

1610.09072

Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation

Nikolay Bogoychev, Rico Sennrich

1911.03362

Hierarchical Reinforcement Learning for Open-Domain Dialog

Abdelrhman Saleh, Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, Rosalind Picard

1909.07547

Deep convolutional Gaussian processes

Kenneth Blomqvist, Samuel Kaski, Markus Heinonen

1810.03052

The Emergence of Spectral Universality in Deep Networks

Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli

1802.09979

Predictive Entropy Search for Bayesian Optimization with Unknown Constraints

José Miguel Hernández-Lobato, Michael A. Gelbart, Matthew W. Hoffman, Ryan P. Adams, Zoubin Ghahramani

1502.05312

A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay

Leslie N. Smith

1803.09820

AIDE: Fast and Communication Efficient Distributed Optimization

Sashank J. Reddi, Jakub Konečný, Peter Richtárik, Barnabás Póczós, Alex Smola

1608.06879

Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice

Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli

1711.04735

Learning without Concentration

Shahar Mendelson

1401.0304

Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach

Ryo Karakida, Shotaro Akaho, Shun-ichi Amari

1806.01316

Convex Optimization: Algorithms and Complexity

Sébastien Bubeck

1405.4980

How to Start Training: The Effect of Initialization and Architecture

Boris Hanin, David Rolnick

1803.01719

Unifying Human and Statistical Evaluation for Natural Language Generation

Tatsunori B. Hashimoto, Hugh Zhang, Percy Liang

1904.02792

Learning ReLUs via Gradient Descent

Mahdi Soltanolkotabi

1705.04591

Diffusion Models Beat GANs on Image Synthesis

Prafulla Dhariwal, Alex Nichol

2105.05233

Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures

Sergey Bartunov, Adam Santoro, Blake A. Richards, Luke Marris, Geoffrey E. Hinton, Timothy Lillicrap

1807.04587

ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models

Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, Cho-Jui Hsieh

1708.03999

How Robust are Reconstruction Thresholds for Community Detection?

Ankur Moitra, William Perry, Alexander S. Wein

1511.01473

Rotation-invariant convolutional neural networks for galaxy morphology prediction

Sander Dieleman, Kyle W. Willett, Joni Dambre

1503.07077

Semi-Supervised Learning with Ladder Networks

Antti Rasmus, Harri Valpola, Mikko Honkala, Mathias Berglund, Tapani Raiko

1507.02672

Binary Space Partitioning Forests

Xuhui Fan, Bin Li, Scott Anthony Sisson

1903.09348

Deep Poisson gamma dynamical systems

Dandan Guo, Bo Chen, Hao Zhang, Mingyuan Zhou

1810.11209

Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs

Alon Brutzkus, Amir Globerson

1702.07966

European Union regulations on algorithmic decision-making and a "right to explanation"

Bryce Goodman, Seth Flaxman

1606.08813

Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models

Pouya Samangouei, Maya Kabkab, Rama Chellappa

1805.06605

Dirichlet belief networks for topic structure learning

He Zhao, Lan Du, Wray Buntine, Mingyuan Zhou

1811.00717

Scalable Deep Generative Relational Models with High-Order Node Dependence

Xuhui Fan, Bin Li, Scott Anthony Sisson, Caoyuan Li, Ling Chen

1911.01535

Adversarial Feature Learning

Jeff Donahue, Philipp Krähenbühl, Trevor Darrell

1605.09782

Large scale distributed neural network training through online distillation

Rohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E. Dahl, Geoffrey E. Hinton

1804.03235

Rectangular Bounding Process

Xuhui Fan, Bin Li, Scott Anthony Sisson

1903.03906

Conditional Generative Adversarial Nets

Mehdi Mirza, Simon Osindero

1411.1784

Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks

Kuan Fang, Alexander Toshev, Li Fei-Fei, Silvio Savarese

1903.03878

Poisson-Randomized Gamma Dynamical Systems

Aaron Schein, Scott W. Linderman, Mingyuan Zhou, David M. Blei, Hanna Wallach

1910.12991

On the Variance of the Adaptive Learning Rate and Beyond

Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han

1908.03265

Complex Embeddings for Simple Link Prediction

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard

1606.06357

Understanding the Difficulty of Training Transformers

Liyuan Liu, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Jiawei Han

2004.08249

Learning Activation Functions to Improve Deep Neural Networks

Forest Agostinelli, Matthew Hoffman, Peter Sadowski, Pierre Baldi

1412.6830

Generalization Properties of Learning with Random Features

Alessandro Rudi, Lorenzo Rosasco

1602.04474

XGNN: Towards Model-Level Explanations of Graph Neural Networks

Hao Yuan, Jiliang Tang, Xia Hu, Shuiwang Ji

2006.02587

Auto-Encoding Sequential Monte Carlo

Tuan Anh Le, Maximilian Igl, Tom Rainforth, Tom Jin, Frank Wood

1705.10306

Adversarial examples in the physical world

Alexey Kurakin, Ian Goodfellow, Samy Bengio

1607.02533

Global Optimality in Tensor Factorization, Deep Learning, and Beyond

Benjamin D. Haeffele, Rene Vidal

1506.07540

Attention in Natural Language Processing

Andrea Galassi, Marco Lippi, Paolo Torroni

1902.02181

Importance Weighted Autoencoders

Yuri Burda, Roger Grosse, Ruslan Salakhutdinov

1509.00519

Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting

Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Adam Tauman Kalai

1901.09451

On the Number of Linear Regions of Deep Neural Networks

Guido Montúfar, Razvan Pascanu, Kyunghyun Cho, Yoshua Bengio

1402.1869

f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization

Sebastian Nowozin, Botond Cseke, Ryota Tomioka

1606.00709

A Network-based End-to-End Trainable Task-oriented Dialogue System

Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gasic, Lina M. Rojas-Barahona, Pei-Hao Su, Stefan Ultes, Steve Young

1604.04562

Optimal Rates for Random Fourier Features

Bharath K. Sriperumbudur, Zoltan Szabo

1506.02155

Incoherence-Optimal Matrix Completion

Yudong Chen

1310.0154

Low-rank Matrix Completion using Alternating Minimization

Prateek Jain, Praneeth Netrapalli, Sujay Sanghavi

1212.0467

Neural network identifiability for a family of sigmoidal nonlinearities

Verner Vlačić, Helmut Bölcskei

1906.06994

Ladder Variational Autoencoders

Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther

1602.02282

Recurrent Models of Visual Attention

Volodymyr Mnih, Nicolas Heess, Alex Graves, Koray Kavukcuoglu

1406.6247

Adversarial Attacks on Neural Network Policies

Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, Pieter Abbeel

1702.02284

Generative Poisoning Attack Method Against Neural Networks

Chaofei Yang, Qing Wu, Hai Li, Yiran Chen

1703.01340

Conditional Image Synthesis With Auxiliary Classifier GANs

Augustus Odena, Christopher Olah, Jonathon Shlens

1610.09585

How degenerate is the parametrization of neural networks with the ReLU activation function?

Julius Berner, Dennis Elbrächter, Philipp Grohs

1905.09803

Weakly Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation

Nelson Yalta, Shinji Watanabe, Kazuhiro Nakadai, Tetsuya Ogata

1807.01126

Exponential expressivity in deep neural networks through transient chaos

Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, Surya Ganguli

1606.05340

Deep ReLU Networks Have Surprisingly Few Activation Patterns

Boris Hanin, David Rolnick

1906.00904

Are Transformers universal approximators of sequence-to-sequence functions?

Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar

1912.10077

Improved Denoising Diffusion Probabilistic Models

Alex Nichol, Prafulla Dhariwal

2102.09672

Regularization for Deep Learning: A Taxonomy

Jan Kukačka, Vladimir Golkov, Daniel Cremers

1710.10686

Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?

Boris Hanin

1801.03744

A Walk with SGD

Chen Xing, Devansh Arpit, Christos Tsirigotis, Yoshua Bengio

1802.08770

Tight Complexity Bounds for Optimizing Composite Objectives

Blake Woodworth, Nathan Srebro

1605.08003

Analyzing the Structure of Attention in a Transformer Language Model

Jesse Vig, Yonatan Belinkov

1906.04284

Stochastic Variance Reduction for Nonconvex Optimization

Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabas Poczos, Alex Smola

1603.06160

L2 Regularization versus Batch and Weight Normalization

Twan van Laarhoven

1706.05350

Bayesian Optimization with Unknown Constraints

Michael A. Gelbart, Jasper Snoek, Ryan P. Adams

1403.5607

Linear Coupling: An Ultimate Unification of Gradient and Mirror Descent

Zeyuan Allen-Zhu, Lorenzo Orecchia

1407.1537

Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization

Yuchen Zhang, Lin Xiao

1409.3257

Don't Decay the Learning Rate, Increase the Batch Size

Samuel L. Smith, Pieter-Jan Kindermans, Chris Ying, Quoc V. Le

1711.00489

Achieving Optimal Misclassification Proportion in Stochastic Block Model

Chao Gao, Zongming Ma, Anderson Y. Zhang, Harrison H. Zhou

1505.03772

Biologically Motivated Algorithms for Propagating Local Target Representations

Alexander G. Ororbia, Ankur Mali

1805.11703

Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks

Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, Nathan Srebro

1805.12076

Deep Residual Networks and Weight Initialization

Masato Taki

1709.02956

Stacked What-Where Auto-encoders

Junbo Zhao, Michael Mathieu, Ross Goroshin, Yann LeCun

1506.02351

Learning agile and dynamic motor skills for legged robots

Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, Marco Hutter

1901.08652

Domain-Adversarial Training of Neural Networks

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, Victor Lempitsky

1505.07818

Gradient Descent Learns Linear Dynamical Systems

Moritz Hardt, Tengyu Ma, Benjamin Recht

1609.05191

Certifying and removing disparate impact

Michael Feldman, Sorelle Friedler, John Moeller, Carlos Scheidegger, Suresh Venkatasubramanian

1412.3756

Distributional Smoothing with Virtual Adversarial Training

Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, Ken Nakae, Shin Ishii

1507.00677

PacGAN: The power of two samples in generative adversarial networks

Zinan Lin, Ashish Khetan, Giulia Fanti, Sewoong Oh

1712.04086

Algorithmic Fairness

Dana Pessach, Erez Shmueli

2001.09784

Self-Attention Graph Pooling

Junhyun Lee, Inyeop Lee, Jaewoo Kang

1904.08082

Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings

Hongyu Ren, Weihua Hu, Jure Leskovec

2002.05969

Distribution-Specific Hardness of Learning Neural Networks

Ohad Shamir

1609.01037

Mondrian Forests: Efficient Online Random Forests

Balaji Lakshminarayanan, Daniel M. Roy, Yee Whye Teh

1406.2673

RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, Jian Tang

1902.10197

Generative Moment Matching Networks

Yujia Li, Kevin Swersky, Richard Zemel

1502.02761

On the Error of Random Fourier Features

Danica J. Sutherland, Jeff Schneider

1506.02785

Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

Balaji Lakshminarayanan, Alexander Pritzel, Charles Blundell

1612.01474

Adversarially Learned Inference

Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Arjovsky, Aaron Courville

1606.00704

Surpassing Human-Level Face Verification Performance on LFW with GaussianFace

Chaochao Lu, Xiaoou Tang

1404.3840

Learning a Driving Simulator

Eder Santana, George Hotz

1608.01230

Explainability Techniques for Graph Convolutional Networks

Federico Baldassarre, Hossein Azizpour

1905.13686

WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling

Hao Zhang, Bo Chen, Dandan Guo, Mingyuan Zhou

1803.01328

State-of-the-art Speech Recognition With Sequence-to-Sequence Models

Chung-Cheng Chiu, Tara N. Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J. Weiss, Kanishka Rao, Ekaterina Gonina, Navdeep Jaitly, Bo Li, Jan Chorowski, Michiel Bacchiani

1712.01769

Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC

Yulai Cong, Bo Chen, Hongwei Liu, Mingyuan Zhou

1706.01724

Augmenting Neural Networks with First-order Logic

Tao Li, Vivek Srikumar

1906.06298

Density Estimation in Infinite Dimensional Exponential Families

Bharath Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Aapo Hyvärinen, Revant Kumar

1312.3516

Leveraging Node Attributes for Incomplete Relational Data

He Zhao, Lan Du, Wray Buntine

1706.04289

Fairness Under Composition

Cynthia Dwork, Christina Ilvento

1806.06122

Explainable Prediction of Medical Codes from Clinical Text

James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, Jacob Eisenstein

1802.05695

Fast Randomized Kernel Methods With Statistical Guarantees

Ahmed El Alaoui, Michael W. Mahoney

1411.0306

Graph U-Nets

Hongyang Gao, Shuiwang Ji

1905.05178

Improving Deep Neural Networks with Probabilistic Maxout Units

Jost Tobias Springenberg, Martin Riedmiller

1312.6116

Detecting Adversarial Samples from Artifacts

Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, Andrew B. Gardner

1703.00410

How Powerful are Graph Neural Networks?

Keyulu Xu, Weihua Hu, Jure Leskovec, Stefanie Jegelka

1810.00826

Optimal Completion Distillation for Sequence Learning

Sara Sabour, William Chan, Mohammad Norouzi

1810.01398

A Lower Bound for the Optimization of Finite Sums

Alekh Agarwal, Leon Bottou

1410.0723

Variational Inference with Normalizing Flows

Danilo Jimenez Rezende, Shakir Mohamed

1505.05770

Training generative neural networks via Maximum Mean Discrepancy optimization

Gintare Karolina Dziugaite, Daniel M. Roy, Zoubin Ghahramani

1505.03906

Meta Networks

Tsendsuren Munkhdalai, Hong Yu

1703.00837

The Binary Space Partitioning-Tree Process

Xuhui Fan, Bin Li, Scott Anthony Sisson

1903.09343

Learning a Natural Language Interface with Neural Programmer

Arvind Neelakantan, Quoc V. Le, Martin Abadi, Andrew McCallum, Dario Amodei

1611.08945

Quaternion Knowledge Graph Embeddings

Shuai Zhang, Yi Tay, Lina Yao, Qi Liu

1904.10281

Mondrian Forests for Large-Scale Regression when Uncertainty Matters

Balaji Lakshminarayanan, Daniel M. Roy, Yee Whye Teh

1506.03805

Unsupervised Monocular Depth Estimation with Left-Right Consistency

Clément Godard, Oisin Mac Aodha, Gabriel J. Brostow

1609.03677

Score-Based Generative Modeling through Stochastic Differential Equations

Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole

2011.13456

Community Detection in Degree-Corrected Block Models

Chao Gao, Zongming Ma, Anderson Y. Zhang, Harrison H. Zhou

1607.06993

The Shattered Gradients Problem: If resnets are the answer, then what is the question?

David Balduzzi, Marcus Frean, Lennox Leary, JP Lewis, Kurt Wan-Duo Ma, Brian McWilliams

1702.08591

Convergence rates of sub-sampled Newton methods

Murat A. Erdogdu, Andrea Montanari

1508.02810

Sub-Sampled Newton Methods I: Globally Convergent Algorithms

Farbod Roosta-Khorasani, Michael W. Mahoney

1601.04737

Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equations

Julius Berner, Philipp Grohs, Arnulf Jentzen

1809.03062

Variance Reduction for Faster Non-Convex Optimization

Zeyuan Allen-Zhu, Elad Hazan

1603.05643

Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting

Jakub Konečný, Jie Liu, Peter Richtárik, Martin Takáč

1504.04407

An Overview of Multi-Task Learning in Deep Neural Networks

Sebastian Ruder

1706.05098

An optimal randomized incremental gradient method

Guanghui Lan, Yi Zhou

1507.02000

Curriculum Dropout

Pietro Morerio, Jacopo Cavazza, Riccardo Volpi, Rene Vidal, Vittorio Murino

1703.06229

Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?

Nitin Bansal, Xiaohan Chen, Zhangyang Wang

1810.09102

A Linearly-Convergent Stochastic L-BFGS Algorithm

Philipp Moritz, Robert Nishihara, Michael I. Jordan

1508.02087

Communication Complexity of Distributed Convex Learning and Optimization

Yossi Arjevani, Ohad Shamir

1506.01900

Exponential Regret Bounds for Gaussian Process Bandits with Deterministic Observations

Nando de Freitas, Alex Smola, Masrour Zoghi

1206.6457

Adversarial Machine Learning at Scale

Alexey Kurakin, Ian Goodfellow, Samy Bengio

1611.01236

Optimal Black-Box Reductions Between Optimization Objectives

Zeyuan Allen-Zhu, Elad Hazan

1603.05642

Gotta Go Fast When Generating Data with Score-Based Models

Alexia Jolicoeur-Martineau, Ke Li, Rémi Piché-Taillefer, Tal Kachman, Ioannis Mitliagkas

2105.14080

Optimal Cluster Recovery in the Labeled Stochastic Block Model

Se-Young Yun, Alexandre Proutiere

1510.05956

Katyusha: The First Direct Acceleration of Stochastic Gradient Methods

Zeyuan Allen-Zhu

1603.05953

Houdini: Fooling Deep Structured Prediction Models

Moustapha Cisse, Yossi Adi, Natalia Neverova, Joseph Keshet

1707.05373

Universal Adversarial Perturbations Against Semantic Image Segmentation

Jan Hendrik Metzen, Mummadi Chaithanya Kumar, Thomas Brox, Volker Fischer

1704.05712

Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling

Zeyuan Allen-Zhu, Zheng Qu, Peter Richtárik, Yang Yuan

1512.09103

Bounding and Counting Linear Regions of Deep Neural Networks

Thiago Serra, Christian Tjandraatmadja, Srikumar Ramalingam

1711.02114

Graph-Revised Convolutional Network

Donghan Yu, Ruohong Zhang, Zhengbao Jiang, Yuexin Wu, Yiming Yang

1911.07123

Conducting Credit Assignment by Aligning Local Representations

Alexander G. Ororbia, Ankur Mali, Daniel Kifer, C. Lee Giles

1803.01834

Learning to Adapt by Minimizing Discrepancy

Alexander G. Ororbia, Patrick Haffner, David Reitter, C. Lee Giles

1711.11542

Multi-Layered Gradient Boosting Decision Trees

Ji Feng, Yang Yu, Zhi-Hua Zhou

1806.00007

Accelerated Proximal Stochastic Dual Coordinate Ascent for Regularized Loss Minimization

Shai Shalev-Shwartz, Tong Zhang

1309.2375

MMSE of probabilistic low-rank matrix estimation: Universality with respect to the output channel

Thibault Lesieur, Florent Krzakala, Lenka Zdeborová

1507.03857

Competing with the Empirical Risk Minimizer in a Single Pass

Roy Frostig, Rong Ge, Sham M. Kakade, Aaron Sidford

1412.6606

Newton Sketch: A Linear-time Optimization Algorithm with Linear-Quadratic Convergence

Mert Pilanci, Martin J. Wainwright

1505.02250

Information Limits for Recovering a Hidden Community

Bruce Hajek, Yihong Wu, Jiaming Xu

1509.07859

Minimizing Finite Sums with the Stochastic Average Gradient

Mark Schmidt, Nicolas Le Roux, Francis Bach

1309.2388

How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD

Zeyuan Allen-Zhu

1801.02982

Few-Shot Unsupervised Image-to-Image Translation

Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, Jan Kautz

1905.01723

Submatrix localization via message passing

Bruce Hajek, Yihong Wu, Jiaming Xu

1510.09219

Scalable Recommendation with Poisson Factorization

Prem Gopalan, Jake M. Hofman, David M. Blei

1311.1704

Non-strongly-convex smooth stochastic approximation with convergence rate O(1/n)

Francis Bach, Eric Moulines

1306.2119

Phase transitions and sample complexity in Bayes-optimal matrix factorization

Yoshiyuki Kabashima, Florent Krzakala, Marc Mézard, Ayaka Sakata, Lenka Zdeborová

1402.1298

Phase Transitions in Sparse PCA

Thibault Lesieur, Florent Krzakala, Lenka Zdeborova

1503.00338

Deep metric learning using Triplet network

Elad Hoffer, Nir Ailon

1412.6622

Dropout Training as Adaptive Regularization

Stefan Wager, Sida Wang, Percy Liang

1307.1493

Semidefinite Programs for Exact Recovery of a Hidden Community

Bruce Hajek, Yihong Wu, Jiaming Xu

1602.06410

A Stochastic Quasi-Newton Method for Large-Scale Optimization

R. H. Byrd, S. L. Hansen, J. Nocedal, Y. Singer

1401.7020

Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields

Mark Schmidt, Reza Babanezhad, Mohamed Osama Ahmed, Aaron Defazio, Ann Clifton, Anoop Sarkar

1504.04406

Adding Gradient Noise Improves Learning for Very Deep Networks

Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens

1511.06807

Multimodal Unsupervised Image-to-Image Translation

Xun Huang, Ming-Yu Liu, Serge Belongie, Jan Kautz

1804.04732

Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithm

Deanna Needell, Nathan Srebro, Rachel Ward

1310.5715

A Proximal Stochastic Gradient Method with Progressive Variance Reduction

Lin Xiao, Tong Zhang

1403.4699

Sparse PCA via Covariance Thresholding

Yash Deshpande, Andrea Montanari

1311.5179

A Literature Study of Embeddings on Source Code

Zimin Chen, Martin Monperrus

1904.03061

Learning the Difference that Makes a Difference with Counterfactually-Augmented Data

Divyansh Kaushik, Eduard Hovy, Zachary C. Lipton

1909.12434

A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks

Umut Simsekli, Levent Sagun, Mert Gurbuzbalaban

1901.06053

Domain Adaptive Neural Networks for Object Recognition

Muhammad Ghifary, W. Bastiaan Kleijn, Mengjie Zhang

1409.6041

Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks

Devansh Arpit, Yingbo Zhou, Bhargava U. Kota, Venu Govindaraju

1603.01431

Off-grid Direction of Arrival Estimation Using Sparse Bayesian Inference

Zai Yang, Lihua Xie, Cishen Zhang

1108.5838

Towards Understanding the Spectral Bias of Deep Learning

Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, Quanquan Gu

1912.01198

How to Escape Saddle Points Efficiently

Chi Jin, Rong Ge, Praneeth Netrapalli, Sham M. Kakade, Michael I. Jordan

1703.00887

Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference

Yarin Gal, Zoubin Ghahramani

1506.02158

Deep Reinforcement Learning

Yuxi Li

1810.06339

Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation

Qiang Liu, Lihong Li, Ziyang Tang, Dengyong Zhou

1810.12429

Investigating practical linear temporal difference learning

Adam White, Martha White

1602.08771

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, Olivier Bachem

1811.12359

World Models

David Ha, Jürgen Schmidhuber

1803.10122

Auto-Encoding Variational Bayes

Diederik P Kingma, Max Welling

1312.6114

Dota 2 with Large Scale Deep Reinforcement Learning

OpenAI, :, Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemysław Dębiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, Rafal Józefowicz, Scott Gray, Catherine Olsson, Jakub Pachocki, Michael Petrov, Henrique P. d. O. Pinto, Jonathan Raiman, Tim Salimans, Jeremy Schlatter, Jonas Schneider, Szymon Sidor, Ilya Sutskever, Jie Tang, Filip Wolski, Susan Zhang

1912.06680

On the Fairness of Disentangled Representations

Francesco Locatello, Gabriele Abbati, Tom Rainforth, Stefan Bauer, Bernhard Schölkopf, Olivier Bachem

1905.13662

Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning

Michael Lutter, Christian Ritter, Jan Peters

1907.04490

Triangle Generative Adversarial Networks

Zhe Gan, Liqun Chen, Weiyao Wang, Yunchen Pu, Yizhe Zhang, Hao Liu, Chunyuan Li, Lawrence Carin

1709.06548

The Expressive Power of Word Embeddings

Yanqing Chen, Bryan Perozzi, Rami Al-Rfou, Steven Skiena

1301.3226

Kernel Mean Embedding of Distributions: A Review and Beyond

Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, Bernhard Schölkopf

1605.09522

Consistent On-Line Off-Policy Evaluation

Assaf Hallak, Shie Mannor

1702.07121

Optimal and Adaptive Off-policy Evaluation in Contextual Bandits

Yu-Xiang Wang, Alekh Agarwal, Miroslav Dudik

1612.01205

Dynamic Bernoulli Embeddings for Language Evolution

Maja Rudolph, David Blei

1703.08052

Understanding and Mitigating the Tradeoff Between Robustness and Accuracy

Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, Percy Liang

2002.10716

SCROLLS: Standardized CompaRison Over Long Language Sequences

Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, Omer Levy

2201.03533

Deep Energy Estimator Networks

Saeed Saremi, Arash Mehrjou, Bernhard Schölkopf, Aapo Hyvärinen

1805.08306

Implicit Regularization in Deep Matrix Factorization

Sanjeev Arora, Nadav Cohen, Wei Hu, Yuping Luo

1905.13655

Does Learning Require Memorization? A Short Tale about a Long Tail

Vitaly Feldman

1906.05271

Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language

Yusuke Yasuda, Xin Wang, Shinji Takaki, Junichi Yamagishi

1810.11960

Further Optimal Regret Bounds for Thompson Sampling

Shipra Agrawal, Navin Goyal

1209.3353

Thompson Sampling for Complex Bandit Problems

Aditya Gopalan, Shie Mannor, Yishay Mansour

1311.0466

Generalized Emphatic Temporal Difference Learning: Bias-Variance Analysis

Assaf Hallak, Aviv Tamar, Remi Munos, Shie Mannor

1509.05172

Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Model

Tian Han, Erik Nijkamp, Xiaolin Fang, Mitch Hill, Song-Chun Zhu, Ying Nian Wu

1812.10907

Improved Precision and Recall Metric for Assessing Generative Models

Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, Timo Aila

1904.06991

Bilevel Programming for Hyperparameter Optimization and Meta-Learning

Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, Massimilano Pontil

1806.04910

Contextual Parameter Generation for Universal Neural Machine Translation

Emmanouil Antonios Platanios, Mrinmaya Sachan, Graham Neubig, Tom Mitchell

1808.08493

ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching

Chunyuan Li, Hao Liu, Changyou Chen, Yunchen Pu, Liqun Chen, Ricardo Henao, Lawrence Carin

1709.01215

Progressive Neural Architecture Search

Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy

1712.00559

Exponential Family Embeddings

Maja R. Rudolph, Francisco J. R. Ruiz, Stephan Mandt, David M. Blei

1608.00778

Generalizing Hamiltonian Monte Carlo with Neural Networks

Daniel Levy, Matthew D. Hoffman, Jascha Sohl-Dickstein

1711.09268

Dynamic Word Embeddings for Evolving Semantic Discovery

Zijun Yao, Yifan Sun, Weicong Ding, Nikhil Rao, Hui Xiong

1703.00607

A Convergent Gradient Descent Algorithm for Rank Minimization and Semidefinite Programming from Random Linear Measurements

Qinqing Zheng, John Lafferty

1506.06081

Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models

Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling, Carlos Outeiral, Pedro Luis Cunha Farias, Alán Aspuru-Guzik

1705.10843

The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables

Chris J. Maddison, Andriy Mnih, Yee Whye Teh

1611.00712

The Generalized Reparameterization Gradient

Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei

1610.02287

Essentially No Barriers in Neural Network Energy Landscape

Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred A. Hamprecht

1803.00885

Are GANs Created Equal? A Large-Scale Study

Mario Lucic, Karol Kurach, Marcin Michalski, Sylvain Gelly, Olivier Bousquet

1711.10337

Multi-Stage Influence Function

Hongge Chen, Si Si, Yang Li, Ciprian Chelba, Sanjiv Kumar, Duane Boning, Cho-Jui Hsieh

2007.09081

Measuring Compositional Generalization: A Comprehensive Method on Realistic Data

Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc van Zee, Olivier Bousquet

1912.09713

Neural Networks can Learn Representations with Gradient Descent

Alex Damian, Jason D. Lee, Mahdi Soltanolkotabi

2206.15144

Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines

Li Huang, Lei Wang

1610.02746

A-NICE-MC: Adversarial Training for MCMC

Jiaming Song, Shengjia Zhao, Stefano Ermon

1706.07561

Object-Centric Learning with Slot Attention

Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, Thomas Kipf

2006.15055

Reconciling modern machine learning practice and the bias-variance trade-off

Mikhail Belkin, Daniel Hsu, Siyuan Ma, Soumik Mandal

1812.11118

Representation Learning with Contrastive Predictive Coding

Aaron van den Oord, Yazhe Li, Oriol Vinyals

1807.03748

A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation

Qiang Liu, Jason D. Lee, Michael I. Jordan

1602.03253

Concept Whitening for Interpretable Image Recognition

Zhi Chen, Yijie Bei, Cynthia Rudin

2002.01650

Up or Down? Adaptive Rounding for Post-Training Quantization

Markus Nagel, Rana Ali Amjad, Mart van Baalen, Christos Louizos, Tijmen Blankevoort

2004.10568

On Unifying Deep Generative Models

Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing

1706.00550

Variational Diffusion Models

Diederik P. Kingma, Tim Salimans, Ben Poole, Jonathan Ho

2107.00630

Implicit Bias of Gradient Descent on Linear Convolutional Networks

Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro

1806.00468

Convex and Non-convex Optimization Under Generalized Smoothness

Haochuan Li, Jian Qian, Yi Tian, Alexander Rakhlin, Ali Jadbabaie

2306.01264

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit

Blake Bordelon, Lorenzo Noci, Mufan Bill Li, Boris Hanin, Cengiz Pehlevan

2309.16620

Depth Dependence of $μ$P Learning Rates in ReLU MLPs

Samy Jelassi, Boris Hanin, Ziwei Ji, Sashank J. Reddi, Srinadh Bhojanapalli, Sanjiv Kumar

2305.07810

Qualitatively characterizing neural network optimization problems

Ian J. Goodfellow, Oriol Vinyals, Andrew M. Saxe

1412.6544

Layered Adaptive Importance Sampling

L. Martino, V. Elvira, D. Luengo, J. Corander

1505.04732

Neural networks and rational functions

Matus Telgarsky

1706.03301

Prodigy: An Expeditiously Adaptive Parameter-Free Learner

Konstantin Mishchenko, Aaron Defazio

2306.06101

GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution

Matt J. Kusner, José Miguel Hernández-Lobato

1611.04051

Learning Disentangled Representations with Semi-Supervised Deep Generative Models

N. Siddharth, Brooks Paige, Jan-Willem van de Meent, Alban Desmaison, Noah D. Goodman, Pushmeet Kohli, Frank Wood, Philip H. S. Torr

1706.00400

Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees

François-Xavier Briol, Chris J. Oates, Mark Girolami, Michael A. Osborne

1506.02681

Learning Latent Subspaces in Variational Autoencoders

Jack Klys, Jake Snell, Richard Zemel

1812.06190

A Kernel Test of Goodness of Fit

Kacper Chwialkowski, Heiko Strathmann, Arthur Gretton

1602.02964

Information Constraints on Auto-Encoding Variational Bayes

Romain Lopez, Jeffrey Regier, Michael I. Jordan, Nir Yosef

1805.08672

Randomized Distributed Mean Estimation: Accuracy vs Communication

Jakub Konečný, Peter Richtárik

1611.07555

Bandits with heavy tail

Sébastien Bubeck, Nicolò Cesa-Bianchi, Gábor Lugosi

1209.1727

How Can We Be So Dense? The Benefits of Using Highly Sparse Representations

Subutai Ahmad, Luiz Scheinkman

1903.11257

Uniform convergence may be unable to explain generalization in deep learning

Vaishnavh Nagarajan, J. Zico Kolter

1902.04742

Recent Advances in Autoencoder-Based Representation Learning

Michael Tschannen, Olivier Bachem, Mario Lucic

1812.05069

Measuring Sample Quality with Stein's Method

Jackson Gorham, Lester Mackey

1506.03039

Learning-Rate-Free Learning by D-Adaptation

Aaron Defazio, Konstantin Mishchenko

2301.07733

Fast Two-Sample Testing with Analytic Representations of Probability Measures

Kacper Chwialkowski, Aaditya Ramdas, Dino Sejdinovic, Arthur Gretton

1506.04725

On the relation between Gaussian process quadratures and sigma-point methods

Simo Särkkä, Jouni Hartikainen, Lennart Svensson, Fredrik Sandblom

1504.05994

Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering

Simon Lacoste-Julien, Fredrik Lindsten, Francis Bach

1501.02056

Captum: A unified and generic model interpretability library for PyTorch

Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, Orion Reblitz-Richardson

2009.07896

Making SGD Parameter-Free

Yair Carmon, Oliver Hinder

2205.02160

Probabilistic Numerics and Uncertainty in Computations

Philipp Hennig, Michael A Osborne, Mark Girolami

1506.01326

Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature

Tom Gunter, Michael A. Osborne, Roman Garnett, Philipp Hennig, Stephen J. Roberts

1411.0439

SmoothGrad: removing noise by adding noise

Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, Martin Wattenberg

1706.03825

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin

1602.04938

How to Explain Individual Classification Decisions

David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, Klaus-Robert Mueller

0912.1128

Sparse DNNs with Improved Adversarial Robustness

Yiwen Guo, Chao Zhang, Changshui Zhang, Yurong Chen

1810.09619

Logit Pairing Methods Can Fool Gradient-Based Attacks

Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, Dietrich Klakow

1810.12042

Generalization Bounds for Uniformly Stable Algorithms

Vitaly Feldman, Jan Vondrak

1812.09859

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems

Sébastien Bubeck, Nicolò Cesa-Bianchi

1204.5721

CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning

Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul, Curtis Langlotz, Katie Shpanskaya, Matthew P. Lungren, Andrew Y. Ng

1711.05225

Large-Scale Feature Learning With Spike-and-Slab Sparse Coding

Ian Goodfellow, Aaron Courville, Yoshua Bengio

1206.6407

An Investigation into Neural Net Optimization via Hessian Eigenvalue Density

Behrooz Ghorbani, Shankar Krishnan, Ying Xiao

1901.10159

On Variational Bounds of Mutual Information

Ben Poole, Sherjil Ozair, Aaron van den Oord, Alexander A. Alemi, George Tucker

1905.06922

Blockwise Parallel Decoding for Deep Autoregressive Models

Mitchell Stern, Noam Shazeer, Jakob Uszkoreit

1811.03115

Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks

Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, Ananthram Swami

1511.04508

Stealing Machine Learning Models via Prediction APIs

Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, Thomas Ristenpart

1609.02943

The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures

Joseph Anderson, Mikhail Belkin, Navin Goyal, Luis Rademacher, James Voss

1311.2891

Learning to Explain: An Information-Theoretic Perspective on Model Interpretation

Jianbo Chen, Le Song, Martin J. Wainwright, Michael I. Jordan

1802.07814

Scaling Laws for Reward Model Overoptimization

Leo Gao, John Schulman, Jacob Hilton

2210.10760

The Limitations of Deep Learning in Adversarial Settings

Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, Ananthram Swami

1511.07528

Smoothed Analysis of Tensor Decompositions

Aditya Bhaskara, Moses Charikar, Ankur Moitra, Aravindan Vijayaraghavan

1311.3651

Distributed Gaussian Processes

Marc Peter Deisenroth, Jun Wei Ng

1502.02843

Challenges in Representation Learning: A report on three machine learning contests

Ian J. Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, Yingbo Zhou, Chetan Ramaiah, Fangxiang Feng, Ruifan Li, Xiaojie Wang, Dimitris Athanasakis, John Shawe-Taylor, Maxim Milakov, John Park, Radu Ionescu, Marius Popescu, Cristian Grozea, James Bergstra, Jingjing Xie, Lukasz Romaszko, Bing Xu, Zhang Chuang, Yoshua Bengio

1307.0414

Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization

Shuo Xie, Zhiyuan Li

2404.04454

Nested Variational Compression in Deep Gaussian Processes

James Hensman, Neil D. Lawrence

1412.1370

Gaussian Processes for Big Data

James Hensman, Nicolo Fusi, Neil D. Lawrence

1309.6835

Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group

Mario Lezcano-Casado, David Martínez-Rubio

1901.08428

Fast and robust tensor decomposition with applications to dictionary learning

Tselil Schramm, David Steurer

1706.08672

Deep Learning using Linear Support Vector Machines

Yichuan Tang

1306.0239

Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions

Yanshuai Cao, David J. Fleet

1410.7827

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Yarin Gal, Mark van der Wilk, Carl E. Rasmussen

1402.1389

Gaussian Processes for Data-Efficient Learning in Robotics and Control

Marc Peter Deisenroth, Dieter Fox, Carl Edward Rasmussen

1502.02860

Integrated perception with recurrent multi-task neural networks

Hakan Bilen, Andrea Vedaldi

1606.01735

Provable Robustness of ReLU networks via Maximization of Linear Regions

Francesco Croce, Maksym Andriushchenko, Matthias Hein

1810.07481

DyNet: The Dynamic Neural Network Toolkit

Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, Kevin Duh, Manaal Faruqui, Cynthia Gan, Dan Garrette, Yangfeng Ji, Lingpeng Kong, Adhiguna Kuncoro, Gaurav Kumar, Chaitanya Malaviya, Paul Michel, Yusuke Oda, Matthew Richardson, Naomi Saphra, Swabha Swayamdipta, Pengcheng Yin

1701.03980

Defining and Characterizing Reward Hacking

Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, David Krueger

2209.13085

A Framework for Evaluating Approximation Methods for Gaussian Process Regression

Krzysztof Chalupka, Christopher K. I. Williams, Iain Murray

1205.6326

Products of Many Large Random Matrices and Gradients in Deep Neural Networks

Boris Hanin, Mihai Nica

1812.05994

Asymptotics of Wide Networks from Feynman Diagrams

Ethan Dyer, Guy Gur-Ari

1909.11304

Investigating the influence of noise and distractors on the interpretation of neural networks

Pieter-Jan Kindermans, Kristof Schütt, Klaus-Robert Müller, Sven Dähne

1611.07270

Complexity of Linear Regions in Deep Networks

Boris Hanin, David Rolnick

1901.09021

Deep multi-scale video prediction beyond mean square error

Michael Mathieu, Camille Couprie, Yann LeCun

1511.05440

Manifold Gaussian Processes for Regression

Roberto Calandra, Jan Peters, Carl Edward Rasmussen, Marc Peter Deisenroth

1402.5876

A Unified Approach to Interpreting Model Predictions

Scott Lundberg, Su-In Lee

1705.07874

Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability

Alex Damian, Eshaan Nichani, Jason D. Lee

2209.15594

Avoiding pathologies in very deep networks

David Duvenaud, Oren Rippel, Ryan P. Adams, Zoubin Ghahramani

1402.5836

Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability

Kai Y. Xiao, Vincent Tjeng, Nur Muhammad Shafiullah, Aleksander Madry

1809.03008

On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport

Lenaic Chizat, Francis Bach

1805.09545

Simple, Efficient, and Neural Algorithms for Sparse Coding

Sanjeev Arora, Rong Ge, Tengyu Ma, Ankur Moitra

1503.00778

Neural Tangent Kernel: Convergence and Generalization in Neural Networks

Arthur Jacot, Franck Gabriel, Clément Hongler

1806.07572

Towards Fast Computation of Certified Robustness for ReLU Networks

Tsui-Wei Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Duane Boning, Inderjit S. Dhillon, Luca Daniel

1804.09699

Understanding intermediate layers using linear classifier probes

Guillaume Alain, Yoshua Bengio

1610.01644

Provable learning of Noisy-or Networks

Sanjeev Arora, Rong Ge, Tengyu Ma, Andrej Risteski

1612.08795

Robustness May Be at Odds with Accuracy

Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, Aleksander Madry

1805.12152

A Mean Field View of the Landscape of Two-Layers Neural Networks

Song Mei, Andrea Montanari, Phan-Minh Nguyen

1804.06561

Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Method

Boaz Barak, Jonathan A. Kelner, David Steurer

1407.1543

Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors

Samuel B. Hopkins, Tselil Schramm, Jonathan Shi, David Steurer

1512.02337

Scaling provable adversarial defenses

Eric Wong, Frank R. Schmidt, Jan Hendrik Metzen, J. Zico Kolter

1805.12514

LambdaNet: Probabilistic Type Inference using Graph Neural Networks

Jiayi Wei, Maruth Goyal, Greg Durrett, Isil Dillig

2005.02161

Hierarchical Graph Representation Learning with Differentiable Pooling

Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, Jure Leskovec

1806.08804

On the (Statistical) Detection of Adversarial Examples

Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, Patrick McDaniel

1702.06280

Universality, Characteristic Kernels and RKHS Embedding of Measures

Bharath K. Sriperumbudur, Kenji Fukumizu, Gert R. G. Lanckriet

1003.0887

Embedding Logical Queries on Knowledge Graphs

William L. Hamilton, Payal Bajaj, Marinka Zitnik, Dan Jurafsky, Jure Leskovec

1806.01445

Bayesian Models of Graphs, Arrays and Other Exchangeable Random Structures

Peter Orbanz, Daniel M. Roy

1312.7857

Sum-Product Networks: A New Deep Architecture

Hoifung Poon, Pedro Domingos

1202.3732

Gated Path Planning Networks

Lisa Lee, Emilio Parisotto, Devendra Singh Chaplot, Eric Xing, Ruslan Salakhutdinov

1806.06408

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia X. Chen, Ye Jia, Anjuli Kannan, Tara Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel A. U. Bacchiani, Thomas B. Jablin, Rob Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon

1902.08295

Unsupervised Domain Adaptation by Backpropagation

Yaroslav Ganin, Victor Lempitsky

1409.7495

Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families

Heiko Strathmann, Dino Sejdinovic, Samuel Livingstone, Zoltan Szabo, Arthur Gretton

1506.02564

Learning Sets with Separating Kernels

Ernesto De Vito, Lorenzo Rosasco, Alessandro Toigo

1204.3573

Latent Sequence Decompositions

William Chan, Yu Zhang, Quoc Le, Navdeep Jaitly

1610.03035

Embedding Java Classes with code2vec: Improvements from Variable Obfuscation

Rhys Compton, Eibe Frank, Panos Patros, Abigail Koay

2004.02942

Stochastic Quasi-Newton Langevin Monte Carlo

Umut Şimşekli, Roland Badeau, A. Taylan Cemgil, Gaël Richard

1602.03442

Representation Learning on Graphs with Jumping Knowledge Networks

Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, Stefanie Jegelka

1806.03536

Particle Filter Networks with Application to Visual Localization

Peter Karkus, David Hsu, Wee Sun Lee

1805.08975

Neural Variational Inference and Learning in Belief Networks

Andriy Mnih, Karol Gregor

1402.0030

Learning to relate images: Mapping units, complex cells and simultaneous eigenspaces

Roland Memisevic

1110.0107

Poisson--Gamma Dynamical Systems

Aaron Schein, Mingyuan Zhou, Hanna Wallach

1701.05573

SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair

Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, Martin Monperrus

1901.01808

Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks

Ying Zhang, Mohammad Pezeshki, Philemon Brakel, Saizheng Zhang, Cesar Laurent Yoshua Bengio, Aaron Courville

1701.02720

Measuring abstract reasoning in neural networks

David G. T. Barrett, Felix Hill, Adam Santoro, Ari S. Morcos, Timothy Lillicrap

1807.04225

Explaining Explanations: An Overview of Interpretability of Machine Learning

Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, Lalana Kagal

1806.00069

Particle Gibbs for Bayesian Additive Regression Trees

Balaji Lakshminarayanan, Daniel M. Roy, Yee Whye Teh

1502.04622

Towards better decoding and language model integration in sequence to sequence models

Jan Chorowski, Navdeep Jaitly

1612.02695

Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning

Wen Sun, J. Andrew Bagnell, Byron Boots

1805.11240

Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks

Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, Zheng Zhang

1909.01315

Learning Features by Watching Objects Move

Deepak Pathak, Ross Girshick, Piotr Dollár, Trevor Darrell, Bharath Hariharan

1612.06370

Improved training of end-to-end attention models for speech recognition

Albert Zeyer, Kazuki Irie, Ralf Schlüter, Hermann Ney

1805.03294

Delving into adversarial attacks on deep policies

Jernej Kos, Dawn Song

1705.06452

Hierarchical Density Order Embeddings

Ben Athiwaratkun, Andrew Gordon Wilson

1804.09843

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel

1606.03657

Asymptotic study of stochastic adaptive algorithm in non-convex landscape

Sébastien Gadat, Ioana Gavra

2012.05640

Convergence of Value Aggregation for Imitation Learning

Ching-An Cheng, Byron Boots

1801.07292

Calibration for the (Computationally-Identifiable) Masses

Úrsula Hébert-Johnson, Michael P. Kim, Omer Reingold, Guy N. Rothblum

1711.08513

Learning End-to-end Autonomous Driving using Guided Auxiliary Supervision

Ashish Mehta, Adithya Subramanian, Anbumani Subramanian

1808.10393

Delayed Impact of Fair Machine Learning

Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, Moritz Hardt

1803.04383

Predicting multicellular function through multi-layer tissue networks

Marinka Zitnik, Jure Leskovec

1707.04638

Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models

Rohit Prabhavalkar, Tara N. Sainath, Yonghui Wu, Patrick Nguyen, Zhifeng Chen, Chung-Cheng Chiu, Anjuli Kannan

1712.01818

A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation

Nikolaus Mayer, Eddy Ilg, Philip Häusser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, Thomas Brox

1512.02134

Gamma Belief Networks

Mingyuan Zhou, Yulai Cong, Bo Chen

1512.03081

Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks

Chunyuan Li, Changyou Chen, David Carlson, Lawrence Carin

1512.07666

Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures

Luke Vilnis, Xiang Li, Shikhar Murty, Andrew McCallum

1805.06627

Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors

Rico Jonschkowski, Divyam Rastogi, Oliver Brock

1805.11122

Hierarchical Transformers for Long Document Classification

Raghavendra Pappagari, Piotr Żelasko, Jesús Villalba, Yishay Carmiel, Najim Dehak

1910.10781

Autoencoding beyond pixels using a learned similarity metric

Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, Ole Winther

1512.09300

Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition

Titouan Parcollet, Ying Zhang, Mohamed Morchid, Chiheb Trabelsi, Georges Linarès, Renato De Mori, Yoshua Bengio

1806.07789

Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability

Jeremy M. Cohen, Simran Kaur, Yuanzhi Li, J. Zico Kolter, Ameet Talwalkar

2103.00065

Canonical Tensor Decomposition for Knowledge Base Completion

Timothée Lacroix, Nicolas Usunier, Guillaume Obozinski

1806.07297

Quaternion Recurrent Neural Networks

Titouan Parcollet, Mirco Ravanelli, Mohamed Morchid, Georges Linarès, Chiheb Trabelsi, Renato De Mori, Yoshua Bengio

1806.04418

Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments

Jiyan Yang, Xiangrui Meng, Michael W. Mahoney

1502.03032

Towards Understanding Knowledge Distillation

Mary Phuong, Christoph H. Lampert

2105.13093

Communication Efficient Distributed Optimization using an Approximate Newton-type Method

Ohad Shamir, Nathan Srebro, Tong Zhang

1312.7853

Sub-Sampled Newton Methods II: Local Convergence Rates

Farbod Roosta-Khorasani, Michael W. Mahoney

1601.04738

Achieving Exact Cluster Recovery Threshold via Semidefinite Programming: Extensions

Bruce Hajek, Yihong Wu, Jiaming Xu

1502.07738

Improved Dropout for Shallow and Deep Learning

Zhe Li, Boqing Gong, Tianbao Yang

1602.02220

Spectral Sparsification and Regret Minimization Beyond Matrix Multiplicative Updates

Zeyuan Allen-Zhu, Zhenyu Liao, Lorenzo Orecchia

1506.04838

On Detecting Adversarial Perturbations

Jan Hendrik Metzen, Tim Genewein, Volker Fischer, Bastian Bischoff

1702.04267

Natasha 2: Faster Non-Convex Optimization Than SGD

Zeyuan Allen-Zhu

1708.08694

Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization

Roy Frostig, Rong Ge, Sham M. Kakade, Aaron Sidford

1506.07512

Introduction to Online Convex Optimization

Elad Hazan

1909.05207

Robustness of classifiers: from adversarial to random noise

Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, Pascal Frossard

1608.08967

Communication-Efficient Distributed Dual Coordinate Ascent

Martin Jaggi, Virginia Smith, Martin Takáč, Jonathan Terhorst, Sanjay Krishnan, Thomas Hofmann, Michael I. Jordan

1409.1458

Model-Agnostic Interpretability of Machine Learning

Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin

1606.05386

Reinforcement Learning with a Corrupted Reward Channel

Tom Everitt, Victoria Krakovna, Laurent Orseau, Marcus Hutter, Shane Legg

1705.08417

Virtual Worlds as Proxy for Multi-Object Tracking Analysis

Adrien Gaidon, Qiao Wang, Yohann Cabon, Eleonora Vig

1605.06457

Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs

Ryan L. Murphy, Balasubramaniam Srinivasan, Vinayak Rao, Bruno Ribeiro

1811.01900

Katyusha X: Practical Momentum Method for Stochastic Sum-of-Nonconvex Optimization

Zeyuan Allen-Zhu

1802.03866

Semi-Supervised Classification with Graph Convolutional Networks

Thomas N. Kipf, Max Welling

1609.02907

Deep Ensembles: A Loss Landscape Perspective

Stanislav Fort, Huiyi Hu, Balaji Lakshminarayanan

1912.02757

Finding Approximate Local Minima Faster than Gradient Descent

Naman Agarwal, Zeyuan Allen-Zhu, Brian Bullins, Elad Hazan, Tengyu Ma

1611.01146

Orthogonal Recurrent Neural Networks with Scaled Cayley Transform

Kyle Helfrich, Devin Willmott, Qiang Ye

1707.09520

Uniform Sampling for Matrix Approximation

Michael B. Cohen, Yin Tat Lee, Cameron Musco, Christopher Musco, Richard Peng, Aaron Sidford

1408.5099

mixup: Beyond Empirical Risk Minimization

Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, David Lopez-Paz

1710.09412

BART: Bayesian additive regression trees

Hugh A. Chipman, Edward I. George, Robert E. McCulloch

0806.3286

UnICORNN: A recurrent model for learning very long time dependencies

T. Konstantin Rusch, Siddhartha Mishra

2103.05487

Multiview Deep Learning for Predicting Twitter Users' Location

Tien Huu Do, Duc Minh Nguyen, Evaggelia Tsiligianni, Bruno Cornelis, Nikos Deligiannis

1712.08091

Modeling polypharmacy side effects with graph convolutional networks

Marinka Zitnik, Monica Agrawal, Jure Leskovec

1802.00543

SCAN: Learning Hierarchical Compositional Visual Concepts

Irina Higgins, Nicolas Sonnerat, Loic Matthey, Arka Pal, Christopher P Burgess, Matko Bosnjak, Murray Shanahan, Matthew Botvinick, Demis Hassabis, Alexander Lerchner

1707.03389

Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network

Wengong Jin, Connor W. Coley, Regina Barzilay, Tommi Jaakkola

1709.04555

A Reliable Effective Terascale Linear Learning System

Alekh Agarwal, Olivier Chapelle, Miroslav Dudik, John Langford

1110.4198

Hamiltonian ABC

Edward Meeds, Robert Leenders, Max Welling

1503.01916

To understand deep learning we need to understand kernel learning

Mikhail Belkin, Siyuan Ma, Soumik Mandal

1802.01396

Fair and Diverse DPP-based Data Summarization

L. Elisa Celis, Vijay Keswani, Damian Straszak, Amit Deshpande, Tarun Kathuria, Nisheeth K. Vishnoi

1802.04023

Stochastic Gradient Hamiltonian Monte Carlo

Tianqi Chen, Emily B. Fox, Carlos Guestrin

1402.4102

Statistical guarantees for the EM algorithm: From population to sample-based analysis

Sivaraman Balakrishnan, Martin J. Wainwright, Bin Yu

1408.2156

Infinite Edge Partition Models for Overlapping Community Detection and Link Prediction

Mingyuan Zhou

1501.06218

On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators

Changyou Chen, Nan Ding, Lawrence Carin

1610.06665

Nonparametric Bayesian Negative Binomial Factor Analysis

Mingyuan Zhou

1604.07464

Non-convex Robust PCA

Praneeth Netrapalli, U N Niranjan, Sujay Sanghavi, Animashree Anandkumar, Prateek Jain

1410.7660

Order Matters: Sequence to sequence for sets

Oriol Vinyals, Samy Bengio, Manjunath Kudlur

1511.06391

Monotonic Chunkwise Attention

Chung-Cheng Chiu, Colin Raffel

1712.05382

Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling

Xiaocheng Shang, Zhanxing Zhu, Benedict Leimkuhler, Amos J. Storkey

1510.08692

Efficient Neural Network Robustness Certification with General Activation Functions

Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, Luca Daniel

1811.00866

On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models

Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, Pushmeet Kohli

1810.12715

Measuring the tendency of CNNs to Learn Surface Statistical Regularities

Jason Jo, Yoshua Bengio

1711.11561

A Sufficient Condition for Convergences of Adam and RMSProp

Fangyu Zou, Li Shen, Zequn Jie, Weizhong Zhang, Wei Liu

1811.09358

Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, Krishna P. Gummadi

1610.08452

SchNet: A continuous-filter convolutional neural network for modeling quantum interactions

Kristof T. Schütt, Pieter-Jan Kindermans, Huziel E. Sauceda, Stefan Chmiela, Alexandre Tkatchenko, Klaus-Robert Müller

1706.08566

Convergence and Dynamical Behavior of the ADAM Algorithm for Non-Convex Stochastic Optimization

Anas Barakat, Pascal Bianchi

1810.02263

AdaGrad stepsizes: Sharp convergence over nonconvex landscapes

Rachel Ward, Xiaoxia Wu, Leon Bottou

1806.01811

Deep Sets

Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, Alexander Smola

1703.06114

KATE: K-Competitive Autoencoder for Text

Yu Chen, Mohammed J. Zaki

1705.02033

Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data

Maximilian Karl, Maximilian Soelch, Justin Bayer, Patrick van der Smagt

1605.06432

A general system of differential equations to model first order adaptive algorithms

André Belotto da Silva, Maxime Gazeau

1810.13108

Fast Linear Model for Knowledge Graph Embeddings

Armand Joulin, Edouard Grave, Piotr Bojanowski, Maximilian Nickel, Tomas Mikolov

1710.10881

QMDP-Net: Deep Learning for Planning under Partial Observability

Peter Karkus, David Hsu, Wee Sun Lee

1703.06692

Dependent Multinomial Models Made Easy: Stick Breaking with the Pólya-Gamma Augmentation

Scott W. Linderman, Matthew J. Johnson, Ryan P. Adams

1506.05843

The Poisson Gamma Belief Network

Mingyuan Zhou, Yulai Cong, Bo Chen

1511.02199

Learning to Perform Physics Experiments via Deep Reinforcement Learning

Misha Denil, Pulkit Agrawal, Tejas D Kulkarni, Tom Erez, Peter Battaglia, Nando de Freitas

1611.01843

Unsupervised Discovery of Interpretable Directions in the GAN Latent Space

Andrey Voynov, Artem Babenko

2002.03754

Controlling generative models with continuous factors of variations

Antoine Plumerault, Hervé Le Borgne, Céline Hudelot

2001.10238

Less than a Single Pass: Stochastically Controlled Stochastic Gradient Method

Lihua Lei, Michael I. Jordan

1609.03261

Training Generative Adversarial Networks with Limited Data

Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, Timo Aila

2006.06676

Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization

Shai Shalev-Shwartz, Tong Zhang

1209.1873

Manitest: Are classifiers really invariant?

Alhussein Fawzi, Pascal Frossard

1507.06535

Distributed Coordinate Descent Method for Learning with Big Data

Peter Richtárik, Martin Takáč

1310.2059

Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss

Yuchen Zhang, Lin Xiao

1501.00263

Matrix Completion has No Spurious Local Minimum

Rong Ge, Jason D. Lee, Tengyu Ma

1605.07272

Global Optimality of Local Search for Low Rank Matrix Recovery

Srinadh Bhojanapalli, Behnam Neyshabur, Nathan Srebro

1605.07221

The Natural Language Decathlon: Multitask Learning as Question Answering

Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, Richard Socher

1806.08730

Unsupervised Learning of 3D Structure from Images

Danilo Jimenez Rezende, S. M. Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, Nicolas Heess

1607.00662

Deep Models of Interactions Across Sets

Jason Hartford, Devon R Graham, Kevin Leyton-Brown, Siamak Ravanbakhsh

1803.02879

BRUNO: A Deep Recurrent Model for Exchangeable Data

Iryna Korshunova, Jonas Degrave, Ferenc Huszár, Yarin Gal, Arthur Gretton, Joni Dambre

1802.07535

Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring

Sungjin Ahn, Anoop Korattikara, Max Welling

1206.6380

Negative Binomial Process Count and Mixture Modeling

Mingyuan Zhou, Lawrence Carin

1209.3442

Improved Sum-of-Squares Lower Bounds for Hidden Clique and Hidden Submatrix Problems

Yash Deshpande, Andrea Montanari

1502.06590

Sum-of-Squares Lower Bounds for Sparse PCA

Tengyu Ma, Avi Wigderson

1507.06370

Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies

T. Konstantin Rusch, Siddhartha Mishra

2010.00951

Symplectic Recurrent Neural Networks

Zhengdao Chen, Jianyu Zhang, Martin Arjovsky, Léon Bottou

1909.13334

Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget

Anoop Korattikara, Yutian Chen, Max Welling

1304.5299

Lipschitz Recurrent Neural Networks

N. Benjamin Erichson, Omri Azencot, Alejandro Queiruga, Liam Hodgkinson, Michael W. Mahoney

2006.12070

Deep Scale-spaces: Equivariance Over Scale

Daniel E. Worrall, Max Welling

1905.11697

Learning Structural Node Embeddings Via Diffusion Wavelets

Claire Donnat, Marinka Zitnik, David Hallac, Jure Leskovec

1710.10321

Value Iteration Networks

Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, Pieter Abbeel

1602.02867

Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

Manuel Watter, Jost Tobias Springenberg, Joschka Boedecker, Martin Riedmiller

1506.07365

On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes

Xiaoyu Li, Francesco Orabona

1805.08114

Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks

Jinghui Chen, Dongruo Zhou, Yiqi Tang, Ziyan Yang, Yuan Cao, Quanquan Gu

1806.06763

Modeling Documents with Deep Boltzmann Machines

Nitish Srivastava, Ruslan R Salakhutdinov, Geoffrey E. Hinton

1309.6865

Universal Stagewise Learning for Non-Convex Problems with Convergence on Averaged Solutions

Zaiyi Chen, Zhuoning Yuan, Jinfeng Yi, Bowen Zhou, Enhong Chen, Tianbao Yang

1808.06296

AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods

Zhiming Zhou, Qingru Zhang, Guansong Lu, Hongwei Wang, Weinan Zhang, Yong Yu

1810.00143

Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice

Hongzhou Lin, Julien Mairal, Zaid Harchaoui

1712.05654

SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator

Cong Fang, Chris Junchi Li, Zhouchen Lin, Tong Zhang

1807.01695

Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration

Soham De, Anirbit Mukherjee, Enayat Ullah

1807.06766

Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functions

Mohammad Emtiyaz Khan, Reza Babanezhad, Wu Lin, Mark Schmidt, Masashi Sugiyama

1511.00146

On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization

Xiangyi Chen, Sijia Liu, Ruoyu Sun, Mingyi Hong

1808.02941

Deep Exponential Families

Rajesh Ranganath, Linpeng Tang, Laurent Charlin, David M. Blei

1411.2581

Composing graphical models with neural networks for structured representations and fast inference

Matthew J. Johnson, David Duvenaud, Alexander B. Wiltschko, Sandeep R. Datta, Ryan P. Adams

1603.06277

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

Andrew Brock, Theodore Lim, J. M. Ritchie, Nick Weston

1608.04236

WNGrad: Learn the Learning Rate in Gradient Descent

Xiaoxia Wu, Rachel Ward, Léon Bottou

1803.02865

Convergence Analysis for Rectangular Matrix Completion Using Burer-Monteiro Factorization and Gradient Descent

Qinqing Zheng, John Lafferty

1605.07051

Dropping Convexity for Faster Semi-definite Optimization

Srinadh Bhojanapalli, Anastasios Kyrillidis, Sujay Sanghavi

1509.03917

When Are Nonconvex Problems Not Scary?

Ju Sun, Qing Qu, John Wright

1510.06096

Confident Multiple Choice Learning

Kimin Lee, Changho Hwang, KyoungSoo Park, Jinwoo Shin

1706.03475

Direct Feedback Alignment Provides Learning in Deep Neural Networks

Arild Nøkland

1609.01596

Neural Ordinary Differential Equations

Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, David Duvenaud

1806.07366

Lagrangian Neural Networks

Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, Shirley Ho

2003.04630

Trivializations for Gradient-Based Optimization on Manifolds

Mario Lezcano-Casado

1909.09501

Recurrent Neural Networks in the Eye of Differential Equations

Murphy Yuezhen Niu, Lior Horesh, Isaac Chuang

1904.12933

An Inexact Variable Metric Proximal Point Algorithm for Generic Quasi-Newton Acceleration

Hongzhou Lin, Julien Mairal, Zaid Harchaoui

1610.00960

MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks

Ariel Gordon, Elad Eban, Ofir Nachum, Bo Chen, Hao Wu, Tien-Ju Yang, Edward Choi

1711.06798

Catalyst Acceleration for Gradient-Based Non-Convex Optimization

Courtney Paquette, Hongzhou Lin, Dmitriy Drusvyatskiy, Julien Mairal, Zaid Harchaoui

1703.10993

Learning Stochastic Recurrent Networks

Justin Bayer, Christian Osendorfer

1411.7610

Variational Inference: A Review for Statisticians

David M. Blei, Alp Kucukelbir, Jon D. McAuliffe

1601.00670

Nostalgic Adam: Weighting more of the past gradients when designing the adaptive learning rate

Haiwen Huang, Chang Wang, Bin Dong

1805.07557

Black box variational inference for state space models

Evan Archer, Il Memming Park, Lars Buesing, John Cunningham, Liam Paninski

1511.07367

Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks

Difan Zou, Ziniu Hu, Yewen Wang, Song Jiang, Yizhou Sun, Quanquan Gu

1911.07323

Stochastic Video Generation with a Learned Prior

Remi Denton, Rob Fergus

1802.07687

Complete Dictionary Recovery over the Sphere

Ju Sun, Qing Qu, John Wright

1504.06785

Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

Rie Johnson, Tong Zhang

1412.1058

Supervised Contrastive Learning

Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, Dilip Krishnan

2004.11362

Learning Distributed Representations from Reviews for Collaborative Filtering

Amjad Almahairi, Kyle Kastner, Kyunghyun Cho, Aaron Courville

1806.06875

prDeep: Robust Phase Retrieval with a Flexible Deep Network

Christopher A. Metzler, Philip Schniter, Ashok Veeraraghavan, Richard G. Baraniuk

1803.00212

Invertible Residual Networks

Jens Behrmann, Will Grathwohl, Ricky T. Q. Chen, David Duvenaud, Jörn-Henrik Jacobsen

1811.00995

Adaptive Proximal Gradient Method for Convex Optimization

Yura Malitsky, Konstantin Mishchenko

2308.02261

Bias and Generalization in Deep Generative Models: An Empirical Study

Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman, Stefano Ermon

1811.03259

Learning Gender-Neutral Word Embeddings

Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, Kai-Wei Chang

1809.01496

Pruning Random Forests for Prediction on a Budget

Feng Nan, Joseph Wang, Venkatesh Saligrama

1606.05060

Learning Structured Sparsity in Deep Neural Networks

Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, Hai Li

1608.03665

Whitening for Self-Supervised Representation Learning

Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe

2007.06346

Big Self-Supervised Models are Strong Semi-Supervised Learners

Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, Geoffrey Hinton

2006.10029

Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks

Santiago Pascual, Mirco Ravanelli, Joan Serrà, Antonio Bonafonte, Yoshua Bengio

1904.03416

Black Box Variational Inference

Rajesh Ranganath, Sean Gerrish, David M. Blei

1401.0118

One Model To Learn Them All

Lukasz Kaiser, Aidan N. Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, Jakob Uszkoreit

1706.05137

Bayesian Reinforcement Learning: A Survey

Mohammad Ghavamzadeh, Shie Mannor, Joelle Pineau, Aviv Tamar

1609.04436

Self-Supervised GANs via Auxiliary Rotation Loss

Ting Chen, Xiaohua Zhai, Marvin Ritter, Mario Lucic, Neil Houlsby

1811.11212

Adversarial Generation of Natural Language

Sai Rajeswar, Sandeep Subramanian, Francis Dutil, Christopher Pal, Aaron Courville

1705.10929

Neural source-filter-based waveform model for statistical parametric speech synthesis

Xin Wang, Shinji Takaki, Junichi Yamagishi

1810.11946

Stochastic Cubic Regularization for Fast Nonconvex Optimization

Nilesh Tripuraneni, Mitchell Stern, Chi Jin, Jeffrey Regier, Michael I. Jordan

1711.02838

Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?

Samet Oymak, Mahdi Soltanolkotabi

1812.10004

Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks

Reinhard Heckel, Paul Hand

1810.03982

Adversarial Robustness through Local Linearization

Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli

1907.02610

Optimal Time Bounds for Approximate Clustering

Ramgopal Mettu, Greg Plaxton

1301.0587

Dynamic Word Embeddings

Robert Bamler, Stephan Mandt

1702.08359

Edward: A library for probabilistic modeling, inference, and criticism

Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, David M. Blei

1610.09787

Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect

Kaihua Tang, Jianqiang Huang, Hanwang Zhang

2009.12991

Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA

Aapo Hyvarinen, Hiroshi Morioka

1605.06336

Learning Representations by Maximizing Mutual Information Across Views

Philip Bachman, R Devon Hjelm, William Buchwalter

1906.00910

Rethinking Pre-training and Self-training

Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin D. Cubuk, Quoc V. Le

2006.06882

On Self Modulation for Generative Adversarial Networks

Ting Chen, Mario Lucic, Neil Houlsby, Sylvain Gelly

1810.01365

On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks

Hoang Thanh-Tung, Truyen Tran

1807.04015

Robustness via curvature regularization, and vice versa

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Jonathan Uesato, Pascal Frossard

1811.09716

Theoretically Principled Trade-off between Robustness and Accuracy

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, Michael I. Jordan

1901.08573

(More) Efficient Reinforcement Learning via Posterior Sampling

Ian Osband, Daniel Russo, Benjamin Van Roy

1306.0940

ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing

Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rihawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, Debsindhu Bhowmik, Burkhard Rost

2007.06225

Top2Vec: Distributed Representations of Topics

Dimo Angelov

2008.09470

Do Deep Generative Models Know What They Don't Know?

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, Balaji Lakshminarayanan

1810.09136

Glow: Generative Flow with Invertible 1x1 Convolutions

Diederik P. Kingma, Prafulla Dhariwal

1807.03039

Enhanced Membership Inference Attacks against Machine Learning Models

Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, Reza Shokri

2111.09679

Training behavior of deep neural network in frequency domain

Zhi-Qin John Xu, Yaoyu Zhang, Yanyang Xiao

1807.01251

Symmetric Variational Autoencoder and Connections to Adversarial Learning

Liqun Chen, Shuyang Dai, Yunchen Pu, Chunyuan Li, Qinliang Su, Lawrence Carin

1709.01846

Adaptive Scheduling for Multi-Task Learning

Sébastien Jean, Orhan Firat, Melvin Johnson

1909.06434

Thompson Sampling for Contextual Bandits with Linear Payoffs

Shipra Agrawal, Navin Goyal

1209.3352

Prior-free and prior-dependent regret bounds for Thompson Sampling

Sébastien Bubeck, Che-Yu Liu

1304.5758

Few-shot classification in Named Entity Recognition Task

Alexander Fritzler, Varvara Logacheva, Maksim Kretov

1812.06158

Thompson Sampling for 1-Dimensional Exponential Family Bandits

Nathaniel Korda, Emilie Kaufmann, Remi Munos

1307.3400

Optimal Algorithms for Distributed Optimization

César A. Uribe, Soomin Lee, Alexander Gasnikov, Angelia Nedić

1712.00232

Thompson Sampling: An Asymptotically Optimal Finite Time Analysis

Emilie Kaufmann, Nathaniel Korda, Rémi Munos

1205.4217

Consistency by Agreement in Zero-shot Neural Machine Translation

Maruan Al-Shedivat, Ankur P. Parikh

1904.02338

FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation

Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, Maosong Sun

1810.10147

DreamFusion: Text-to-3D using 2D Diffusion

Ben Poole, Ajay Jain, Jonathan T. Barron, Ben Mildenhall

2209.14988

Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective

Muhammad Abdullah Jamal, Matthew Brown, Ming-Hsuan Yang, Liqiang Wang, Boqing Gong

2003.10780

Data Augmentation Generative Adversarial Networks

Antreas Antoniou, Amos Storkey, Harrison Edwards

1711.04340

Semi-supervised Vocabulary-informed Learning

Yanwei Fu, Leonid Sigal

1604.07093

Boundary-Seeking Generative Adversarial Networks

R Devon Hjelm, Athul Paul Jacob, Tong Che, Adam Trischler, Kyunghyun Cho, Yoshua Bengio

1702.08431

Non-convex Optimization for Machine Learning

Prateek Jain, Purushottam Kar

1712.07897

A Large-Scale Study on Regularization and Normalization in GANs

Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski, Sylvain Gelly

1807.04720

Aligning Language Models with Preferences through f-divergence Minimization

Dongyoung Go, Tomasz Korbak, Germán Kruszewski, Jos Rozen, Nahyeon Ryu, Marc Dymetman

2302.08215

MINE: Mutual Information Neural Estimation

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, R Devon Hjelm

1801.04062

Efficient fair PCA for fair representation learning

Matthäus Kleindessner, Michele Donini, Chris Russell, Muhammad Bilal Zafar

2302.13319

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

Yujun Lin, Song Han, Huizi Mao, Yu Wang, William J. Dally

1712.01887

Can recurrent neural networks warp time?

Corentin Tallec, Yann Ollivier

1804.11188

A Wild Bootstrap for Degenerate Kernel Tests

Kacper Chwialkowski, Dino Sejdinovic, Arthur Gretton

1408.5404

Resilience: A Criterion for Learning in the Presence of Arbitrary Outliers

Jacob Steinhardt, Moses Charikar, Gregory Valiant

1703.04940

On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions

Francis Bach

1502.06800

Being Robust (in High Dimensions) Can Be Practical

Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jerry Li, Ankur Moitra, Alistair Stewart

1703.00893

How do infinite width bounded norm networks look in function space?

Pedro Savarese, Itay Evron, Daniel Soudry, Nathan Srebro

1902.05040

Towards Understanding Sycophancy in Language Models

Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, Ethan Perez

2310.13548

A Survey of Available Corpora for Building Data-Driven Dialogue Systems

Iulian Vlad Serban, Ryan Lowe, Peter Henderson, Laurent Charlin, Joelle Pineau

1512.05742

Maximum Likelihood Training of Score-Based Diffusion Models

Yang Song, Conor Durkan, Iain Murray, Stefano Ermon

2101.09258

Compositional Explanations of Neurons

Jesse Mu, Jacob Andreas

2006.14032

Nonparametric Estimation of Renyi Divergence and Friends

Akshay Krishnamurthy, Kirthevasan Kandasamy, Barnabas Poczos, Larry Wasserman

1402.2966

Score-based Generative Modeling in Latent Space

Arash Vahdat, Karsten Kreis, Jan Kautz

2106.05931

Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD

Matthew Faw, Litu Rout, Constantine Caramanis, Sanjay Shakkottai

2302.06570

Dual Discriminator Generative Adversarial Nets

Tu Dinh Nguyen, Trung Le, Hung Vu, Dinh Phung

1709.03831

Convergence of Adam Under Relaxed Assumptions

Haochuan Li, Alexander Rakhlin, Ali Jadbabaie

2304.13972

Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming

Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, Daniel Soudry

2006.10518

On the infinite-depth limit of finite-width neural networks

Soufiane Hayou

2210.00688

Grounded Language Learning in a Simulated 3D World

Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis, Phil Blunsom

1706.06551

VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning

Akash Srivastava, Lazar Valkov, Chris Russell, Michael U. Gutmann, Charles Sutton

1705.07761

Stable ResNet

Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau

2010.12859

Fast $ε$-free Inference of Simulation Models with Bayesian Conditional Density Estimation

George Papamakarios, Iain Murray

1605.06376

The Efficiency Misnomer

Mostafa Dehghani, Anurag Arnab, Lucas Beyer, Ashish Vaswani, Yi Tay

2110.12894

Stochastic Nested Variance Reduction for Nonconvex Optimization

Dongruo Zhou, Pan Xu, Quanquan Gu

1806.07811

Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition

Rong Ge, Furong Huang, Chi Jin, Yang Yuan

1503.02101

A Simple Convergence Proof of Adam and Adagrad

Alexandre Défossez, Léon Bottou, Francis Bach, Nicolas Usunier

2003.02395

Pac-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning

Olivier Catoni

0712.0248

Towards A Rigorous Science of Interpretable Machine Learning

Finale Doshi-Velez, Been Kim

1702.08608

Probabilistic Interpretation of Linear Solvers

Philipp Hennig

1402.2058

Machine Learning of Linear Differential Equations using Gaussian Processes

Maziar Raissi, George Em. Karniadakis

1701.02440

FastMMD: Ensemble of Circular Discrepancy for Efficient Two-Sample Test

Ji Zhao, Deyu Meng

1405.2664

InterpretML: A Unified Framework for Machine Learning Interpretability

Harsha Nori, Samuel Jenkins, Paul Koch, Rich Caruana

1909.09223

Beyond the Golden Ratio for Variational Inequality Algorithms

Ahmet Alacaoglu, Axel Böhm, Yura Malitsky

2212.13955

Towards a Definition of Disentangled Representations

Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, Alexander Lerchner

1812.02230

Gaussian Process Regression Networks

Andrew Gordon Wilson, David A. Knowles, Zoubin Ghahramani

1110.4411

Gaussian Process Prior Variational Autoencoders

Francesco Paolo Casale, Adrian V Dalca, Luca Saglietti, Jennifer Listgarten, Nicolo Fusi

1810.11738

A Hitting Time Analysis of Stochastic Gradient Langevin Dynamics

Yuchen Zhang, Percy Liang, Moses Charikar

1702.05575

Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization

Jelena Diakonikolas, Constantinos Daskalakis, Michael I. Jordan

2011.00364

Hyperprior Induced Unsupervised Disentanglement of Latent Representations

Abdul Fatir Ansari, Harold Soh

1809.04497

End-to-End Kernel Learning with Supervised Convolutional Kernel Networks

Julien Mairal

1605.06265

Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies

Alessandro Achille, Tom Eccles, Loic Matthey, Christopher P. Burgess, Nick Watters, Alexander Lerchner, Irina Higgins

1808.06508

Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels

Haim Avron, Vikas Sindhwani, Jiyan Yang, Michael Mahoney

1412.8293

Understanding disentangling in $β$-VAE

Christopher P. Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, Alexander Lerchner

1804.03599

Fast Variational Inference in the Conjugate Exponential Family

James Hensman, Magnus Rattray, Neil D. Lawrence

1206.5162

Equivalence of distance-based and RKHS-based statistics in hypothesis testing

Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, Kenji Fukumizu

1207.6076

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization

Dongruo Zhou, Jinghui Chen, Yuan Cao, Ziyan Yang, Quanquan Gu

1808.05671

Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation

Sarah Tan, Rich Caruana, Giles Hooker, Yin Lou

1710.06169

DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method

Ahmed Khaled, Konstantin Mishchenko, Chi Jin

2305.16284

Latent Intention Dialogue Models

Tsung-Hsien Wen, Yishu Miao, Phil Blunsom, Steve Young

1705.10229

Holographic Embeddings of Knowledge Graphs

Maximilian Nickel, Lorenzo Rosasco, Tomaso Poggio

1510.04935

The Goldilocks zone: Towards better understanding of neural network loss landscapes

Stanislav Fort, Adam Scherlis

1807.02581

Neon2: Finding Local Minima via First-Order Oracles

Zeyuan Allen-Zhu, Yuanzhi Li

1711.06673

Maximum a Posteriori Policy Optimisation

Abbas Abdolmaleki, Jost Tobias Springenberg, Yuval Tassa, Remi Munos, Nicolas Heess, Martin Riedmiller

1806.06920

An Interpretable Model with Globally Consistent Explanations for Credit Risk

Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, Tong Wang

1811.12615

A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks

Sanjeev Arora, Nadav Cohen, Noah Golowich, Wei Hu

1810.02281

Variational Approaches for Auto-Encoding Generative Adversarial Networks

Mihaela Rosca, Balaji Lakshminarayanan, David Warde-Farley, Shakir Mohamed

1706.04987

Last Iterate is Slower than Averaged Iterate in Smooth Convex-Concave Saddle Point Problems

Noah Golowich, Sarath Pattathil, Constantinos Daskalakis, Asuman Ozdaglar

2002.00057

Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data

Wei-Ning Hsu, Yu Zhang, James Glass

1709.07902

Global Convergence and Variance-Reduced Optimization for a Class of Nonconvex-Nonconcave Minimax Problems

Junchi Yang, Negar Kiyavash, Niao He

2002.09621

Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers

Abraham J. Wyner, Matthew Olson, Justin Bleich, David Mease

1504.07676

Scalable Kernel Methods via Doubly Stochastic Gradients

Bo Dai, Bo Xie, Niao He, Yingyu Liang, Anant Raj, Maria-Florina Balcan, Le Song

1407.5599

Understanding the Role of Momentum in Stochastic Gradient Methods

Igor Gitman, Hunter Lang, Pengchuan Zhang, Lin Xiao

1910.13962

Isolating Sources of Disentanglement in Variational Autoencoders

Ricky T. Q. Chen, Xuechen Li, Roger Grosse, David Duvenaud

1802.04942

Implicit Reparameterization Gradients

Michael Figurnov, Shakir Mohamed, Andriy Mnih

1805.08498

Lossy Image Compression with Compressive Autoencoders

Lucas Theis, Wenzhe Shi, Andrew Cunningham, Ferenc Huszár

1703.00395

Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods

Maher Nouiehed, Maziar Sanjabi, Tianjian Huang, Jason D. Lee, Meisam Razaviyayn

1902.08297

Optimization for deep learning: theory and algorithms

Ruoyu Sun

1912.08957

SimplE Embedding for Link Prediction in Knowledge Graphs

Seyed Mehran Kazemi, David Poole

1802.04868

Convexified Modularity Maximization for Degree-corrected Stochastic Block Models

Yudong Chen, Xiaodong Li, Jiaming Xu

1512.08425

Gradients are Not All You Need

Luke Metz, C. Daniel Freeman, Samuel S. Schoenholz, Tal Kachman

2111.05803

How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets

Zhiyun Lu, Avner May, Kuan Liu, Alireza Bagheri Garakani, Dong Guo, Aurélien Bellet, Linxi Fan, Michael Collins, Brian Kingsbury, Michael Picheny, Fei Sha

1411.4000

Accelerating Smooth Games by Manipulating Spectral Shapes

Waïss Azizian, Damien Scieur, Ioannis Mitliagkas, Simon Lacoste-Julien, Gauthier Gidel

2001.00602

Improved Analysis of Clipping Algorithms for Non-convex Optimization

Bohang Zhang, Jikai Jin, Cong Fang, Liwei Wang

2010.02519

Understanding Deep Neural Networks with Rectified Linear Units

Raman Arora, Amitabh Basu, Poorya Mianjy, Anirbit Mukherjee

1611.01491

A spectral method for community detection in moderately-sparse degree-corrected stochastic block models

Lennart Gulikers, Marc Lelarge, Laurent Massoulié

1506.08621

Scalable Bayesian Optimization Using Deep Neural Networks

Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, Ryan P. Adams

1502.05700

Multiaccuracy: Black-Box Post-Processing for Fairness in Classification

Michael P. Kim, Amirata Ghorbani, James Zou

1805.12317

Distributed Algorithms for Learning and Cognitive Medium Access with Logarithmic Regret

Animashree Anandkumar, Nithin Michael, Ao Kevin Tang, Ananthram Swami

1006.1673

Fairer and more accurate, but for whom?

Alexandra Chouldechova, Max G'Sell

1707.00046

Generative Image Modeling Using Spatial LSTMs

Lucas Theis, Matthias Bethge

1506.03478

Privacy Risks of Securing Machine Learning Models against Adversarial Examples

Liwei Song, Reza Shokri, Prateek Mittal

1905.10291

Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference

Klas Leino, Matt Fredrikson

1906.11798

Last-iterate convergence rates for min-max optimization

Jacob Abernethy, Kevin A. Lai, Andre Wibisono

1906.02027

Momentum Improves Normalized SGD

Ashok Cutkosky, Harsh Mehta

2002.03305

The generalization error of random features regression: Precise asymptotics and double descent curve

Song Mei, Andrea Montanari

1908.05355

Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms

Jianyu Wang, Gauri Joshi

1808.07576

Spectral redemption: clustering sparse networks

Florent Krzakala, Cristopher Moore, Elchanan Mossel, Joe Neeman, Allan Sly, Lenka Zdeborová, Pan Zhang

1306.5550

Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations

Boris Hanin

1708.02691

Don't Use Large Mini-Batches, Use Local SGD

Tao Lin, Sebastian U. Stich, Kumar Kshitij Patel, Martin Jaggi

1808.07217

LION: Latent Point Diffusion Models for 3D Shape Generation

Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis

2210.06978

Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD

Jianyu Wang, Gauri Joshi

1810.08313

Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains

Aymeric Dieuleveut, Alain Durmus, Francis Bach

1707.06386

On the insufficiency of existing momentum schemes for Stochastic Optimization

Rahul Kidambi, Praneeth Netrapalli, Prateek Jain, Sham M. Kakade

1803.05591

Inductive Moment Matching

Linqi Zhou, Stefano Ermon, Jiaming Song

2503.07565

Understanding Gradient Clipping in Private SGD: A Geometric Perspective

Xiangyi Chen, Zhiwei Steven Wu, Mingyi Hong

2006.15429

A Simple Framework for Contrastive Learning of Visual Representations

Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton

2002.05709

Fairness Without Demographics in Repeated Loss Minimization

Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, Percy Liang

1806.08010

Detecting and Correcting for Label Shift with Black Box Predictors

Zachary C. Lipton, Yu-Xiang Wang, Alex Smola

1802.03916

Fairness in Criminal Justice Risk Assessments: The State of the Art

Richard A. Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, Aaron Roth

1703.09207

struc2vec: Learning Node Representations from Structural Identity

Leonardo F. R. Ribeiro, Pedro H. P. Savarese, Daniel R. Figueiredo

1704.03165

From Pixels to Torques: Policy Learning with Deep Dynamical Models

Niklas Wahlström, Thomas B. Schön, Marc Peter Deisenroth

1502.02251

Revisiting Membership Inference Under Realistic Assumptions

Bargav Jayaraman, Lingxiao Wang, Katherine Knipmeyer, Quanquan Gu, David Evans

2005.10881

Sparsified SGD with Memory

Sebastian U. Stich, Jean-Baptiste Cordonnier, Martin Jaggi

1809.07599

Two models of double descent for weak features

Mikhail Belkin, Daniel Hsu, Ji Xu

1903.07571

Single-Channel Multi-Speaker Separation using Deep Clustering

Yusuf Isik, Jonathan Le Roux, Zhuo Chen, Shinji Watanabe, John R. Hershey

1607.02173

On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization

Fan Zhou, Guojing Cong

1708.01012

On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games

Eric V. Mazumdar, Michael I. Jordan, S. Shankar Sastry

1901.00838

The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization

Constantinos Daskalakis, Ioannis Panageas

1807.03907

Surprises in High-Dimensional Ridgeless Least Squares Interpolation

Trevor Hastie, Andrea Montanari, Saharon Rosset, Ryan J. Tibshirani

1903.08560

Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimization

Constantinos Daskalakis, Ioannis Panageas

1807.04252

Asynchronous Decentralized Parallel Stochastic Gradient Descent

Xiangru Lian, Wei Zhang, Ce Zhang, Ji Liu

1710.06952

Pseudo-likelihood methods for community detection in large sparse networks

Arash A. Amini, Aiyou Chen, Peter J. Bickel, Elizaveta Levina

1207.2340

Benign Overfitting in Linear Regression

Peter L. Bartlett, Philip M. Long, Gábor Lugosi, Alexander Tsigler

1906.11300

The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning

Siyuan Ma, Raef Bassily, Mikhail Belkin

1712.06559

Priors for Random Count Matrices Derived from a Family of Negative Binomial Processes

Mingyuan Zhou, Oscar Hernan Madrid Padilla, James G. Scott

1404.3331

Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization

Blake Woodworth, Jialei Wang, Adam Smith, Brendan McMahan, Nathan Srebro

1805.10222

YellowFin and the Art of Momentum Tuning

Jian Zhang, Ioannis Mitliagkas

1706.03471

CodeSearchNet Challenge: Evaluating the State of Semantic Code Search

Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, Marc Brockschmidt

1909.09436

Accelerating Stochastic Gradient Descent For Least Squares Regression

Prateek Jain, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli, Aaron Sidford

1704.08227

Analysis of classifiers' robustness to adversarial perturbations

Alhussein Fawzi, Omar Fawzi, Pascal Frossard

1502.02590

Parallel Coordinate Descent Methods for Big Data Optimization

Peter Richtárik, Martin Takáč

1212.0873

Optimal algorithms for smooth and strongly convex distributed optimization in networks

Kevin Scaman, Francis Bach, Sébastien Bubeck, Yin Tat Lee, Laurent Massoulié

1702.08704

Variational Autoencoders and Nonlinear ICA: A Unifying Framework

Ilyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, Aapo Hyvärinen

1907.04809

Distributed Delayed Stochastic Optimization

Alekh Agarwal, John C. Duchi

1104.5525

Statistical-Computational Tradeoffs in Planted Problems and Submatrix Localization with a Growing Number of Clusters and Submatrices

Yudong Chen, Jiaming Xu

1402.1267

Generalized Conditional Gradient for Sparse Estimation

Yaoliang Yu, Xinhua Zhang, Dale Schuurmans

1410.4828

A Geometric Analysis of Phase Retrieval

Ju Sun, Qing Qu, John Wright

1602.06664

Accelerated, Parallel and Proximal Coordinate Descent

Olivier Fercoq, Peter Richtárik

1312.5799

Gradient Descent Converges to Minimizers

Jason D. Lee, Max Simchowitz, Michael I. Jordan, Benjamin Recht

1602.04915

node2vec: Scalable Feature Learning for Networks

Aditya Grover, Jure Leskovec

1607.00653

The Variational Fair Autoencoder

Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, Richard Zemel

1511.00830

Determinantal point processes for machine learning

Alex Kulesza, Ben Taskar

1207.6083

A Converse to Banach's Fixed Point Theorem and its CLS Completeness

Constantinos Daskalakis, Christos Tzamos, Manolis Zampetakis

1702.07339

Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow

T. Tony Cai, Xiaodong Li, Zongming Ma

1506.03382

Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networks

Tengyuan Liang, James Stokes

1802.06132

Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling

John Duchi, Alekh Agarwal, Martin Wainwright

1005.2012

Flexibly Fair Representation Learning by Disentanglement

Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa A. Weis, Kevin Swersky, Toniann Pitassi, Richard Zemel

1906.02589

Escaping Saddle Points with Adaptive Gradient Methods

Matthew Staib, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar, Suvrit Sra

1901.09149

When Do Neural Networks Outperform Kernel Methods?

Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz, Andrea Montanari

2006.13409

Weakly Supervised Disentanglement by Pairwise Similarities

Junxiang Chen, Kayhan Batmanghelich

1906.01044

Deep learning: a statistical viewpoint

Peter L. Bartlett, Andrea Montanari, Alexander Rakhlin

2103.09177

On the Ineffectiveness of Variance Reduced Optimization for Deep Learning

Aaron Defazio, Léon Bottou

1812.04529

Slow Learners are Fast

John Langford, Alexander Smola, Martin Zinkevich

0911.0491

Benign overfitting in ridge regression

A. Tsigler, P. L. Bartlett

2009.14286

The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization

Ben Adlam, Jeffrey Pennington

2008.06786

Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression

Aymeric Dieuleveut, Nicolas Flammarion, Francis Bach

1602.05419

Imagination-Augmented Agents for Deep Reinforcement Learning

Théophane Weber, Sébastien Racanière, David P. Reichert, Lars Buesing, Arthur Guez, Danilo Jimenez Rezende, Adria Puigdomènech Badia, Oriol Vinyals, Nicolas Heess, Yujia Li, Razvan Pascanu, Peter Battaglia, Demis Hassabis, David Silver, Daan Wierstra

1707.06203

Interpretable Explanations of Black Boxes by Meaningful Perturbation

Ruth Fong, Andrea Vedaldi

1704.03296

Disentangling Factors of Variation Using Few Labels

Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem

1905.01258

D$^2$: Decentralized Training over Decentralized Data

Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, Ji Liu

1803.07068

A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates

Zhi Li, Wei Shi, Ming Yan

1704.07807

On Optimal Probabilities in Stochastic Coordinate Descent Methods

Peter Richtárik, Martin Takáč

1310.3438

Second-Order Stochastic Optimization for Machine Learning in Linear Time

Naman Agarwal, Brian Bullins, Elad Hazan

1602.03943

The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICA

Luigi Gresele, Paul K. Rubenstein, Arash Mehrjou, Francesco Locatello, Bernhard Schölkopf

1905.06642

Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent

Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, Ji Liu

1705.09056

The Power of Normalization: Faster Evasion of Saddle Points

Kfir Y. Levy

1611.04831

On gradient regularizers for MMD GANs

Michael Arbel, Danica J. Sutherland, Mikołaj Bińkowski, Arthur Gretton

1805.11565

Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis

Maxim Raginsky, Alexander Rakhlin, Matus Telgarsky

1702.03849

On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset

Muhammad Waleed Gondal, Manuel Wüthrich, Đorđe Miladinović, Francesco Locatello, Martin Breidt, Valentin Volchkov, Joel Akpo, Olivier Bachem, Bernhard Schölkopf, Stefan Bauer

1906.03292

How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric View

Tengyuan Liang

1712.08244

AudioPaLM: A Large Language Model That Can Speak and Listen

Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, Hannah Muckenhirn, Dirk Padfield, James Qin, Danny Rozenberg, Tara Sainath, Johan Schalkwyk, Matt Sharifi, Michelle Tadmor Ramanovich, Marco Tagliasacchi, Alexandru Tudor, Mihajlo Velimirović, Damien Vincent, Jiahui Yu, Yongqiang Wang, Vicky Zayats, Neil Zeghidour, Yu Zhang, Zhishuai Zhang, Lukas Zilka, Christian Frank

2306.12925

Are Disentangled Representations Helpful for Abstract Visual Reasoning?

Sjoerd van Steenkiste, Francesco Locatello, Jürgen Schmidhuber, Olivier Bachem

1905.12506

Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability

Shaowu Pan, Karthik Duraisamy

1906.03663

Efficient approaches for escaping higher order saddle points in non-convex optimization

Anima Anandkumar, Rong Ge

1602.05908

Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems

Maziar Raissi, Paris Perdikaris, George Em Karniadakis

1801.01236

Discovering Symbolic Models from Deep Learning with Inductive Biases

Miles Cranmer, Alvaro Sanchez-Gonzalez, Peter Battaglia, Rui Xu, Kyle Cranmer, David Spergel, Shirley Ho

2006.11287

Proximal Quasi-Newton Methods for Regularized Convex Optimization with Linear and Accelerated Sublinear Convergence Rates

Hiva Ghanbari, Katya Scheinberg

1607.03081

Computational and Statistical Boundaries for Submatrix Localization in a Large Noisy Matrix

T. Tony Cai, Tengyuan Liang, Alexander Rakhlin

1502.01988

Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

Yann Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, Yoshua Bengio

1406.2572

Graph Convolutional Neural Networks for Web-Scale Recommender Systems

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, Jure Leskovec

1806.01973

Modeling Relational Data with Graph Convolutional Networks

Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, Max Welling

1703.06103

Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization

Jiong Zhang, Qi Lei, Inderjit S. Dhillon

1803.09327

Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations

Maziar Raissi, Paris Perdikaris, George Em Karniadakis

1711.10566

Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control

Yaofeng Desmond Zhong, Biswadip Dey, Amit Chakraborty

1909.12077

Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, Antonio Torralba

1810.01566

Sylvester Normalizing Flows for Variational Inference

Rianne van den Berg, Leonard Hasenclever, Jakub M. Tomczak, Max Welling

1803.05649

N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning

Anubhav Ashok, Nicholas Rhinehart, Fares Beainy, Kris M. Kitani

1709.06030

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons

Byeongho Heo, Minsik Lee, Sangdoo Yun, Jin Young Choi

1811.03233

Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations

Maziar Raissi

1801.06637

The local convexity of solving systems of quadratic equations

Chris D. White, Sujay Sanghavi, Rachel Ward

1506.07868

Adaptive Gradient Descent without Descent

Yura Malitsky, Konstantin Mishchenko

1910.09529

Null-sampling for Interpretable and Fair Representations

Thomas Kehrenberg, Myles Bartlett, Oliver Thomas, Novi Quadrianto

2008.05248

Sparse PCA via Bipartite Matchings

Megasthenis Asteris, Dimitris Papailiopoulos, Anastasios Kyrillidis, Alexandros G. Dimakis

1508.00625

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, Cho-Jui Hsieh

1905.07953

Hamiltonian Generative Networks

Peter Toth, Danilo Jimenez Rezende, Andrew Jaegle, Sébastien Racanière, Aleksandar Botev, Irina Higgins

1909.13789

Deep learning with Elastic Averaging SGD

Sixin Zhang, Anna Choromanska, Yann LeCun

1412.6651

Supervised Topic Models

David M. Blei, Jon D. McAuliffe

1003.0783

Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients

Lukas Balles, Philipp Hennig

1705.07774

Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent

Chi Jin, Praneeth Netrapalli, Michael I. Jordan

1711.10456

Phase Retrieval using Alternating Minimization

Praneeth Netrapalli, Prateek Jain, Sujay Sanghavi

1306.0160

DGM: A deep learning algorithm for solving partial differential equations

Justin Sirignano, Konstantinos Spiliopoulos

1708.07469

Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

Maziar Raissi, Paris Perdikaris, George Em Karniadakis

1711.10561

PDE-Net: Learning PDEs from Data

Zichao Long, Yiping Lu, Xianzhong Ma, Bin Dong

1710.09668

Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations

Yiping Lu, Aoxiao Zhong, Quanzheng Li, Bin Dong

1710.10121

Provable Bayesian Inference via Particle Mirror Descent

Bo Dai, Niao He, Hanjun Dai, Le Song

1506.03101

A trust-region method for stochastic variational inference with applications to streaming data

Lucas Theis, Matthew D. Hoffman

1505.07649

Towards Adversarial Retinal Image Synthesis

Pedro Costa, Adrian Galdran, Maria Inês Meyer, Michael David Abràmoff, Meindert Niemeijer, Ana Maria Mendonça, Aurélio Campilho

1701.08974

Fast Algorithms for Robust PCA via Gradient Descent

Xinyang Yi, Dohyung Park, Yudong Chen, Constantine Caramanis

1605.07784

Convex Optimization for Big Data

Volkan Cevher, Stephen Becker, Mark Schmidt

1411.0972

Provable Efficient Online Matrix Completion via Non-convex Stochastic Gradient Descent

Chi Jin, Sham M. Kakade, Praneeth Netrapalli

1605.08370

Residual Flows for Invertible Generative Modeling

Ricky T. Q. Chen, Jens Behrmann, David Duvenaud, Jörn-Henrik Jacobsen

1906.02735

FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models

Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, David Duvenaud

1810.01367

Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network

Risi Kondor, Zhen Lin, Shubhendu Trivedi

1806.09231

Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs

Nicholas Watters, Loic Matthey, Christopher P. Burgess, Alexander Lerchner

1901.07017

Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data

Yinhao Zhu, Nicholas Zabaras, Phaedon-Stelios Koutsourelakis, Paris Perdikaris

1901.06314

Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations

Maziar Raissi, George Em Karniadakis

1708.00588

Multimodal Generative Models for Scalable Weakly-Supervised Learning

Mike Wu, Noah Goodman

1802.05335

Deep learning for universal linear embeddings of nonlinear dynamics

Bethany Lusch, J. Nathan Kutz, Steven L. Brunton

1712.09707

Auto-Encoding Total Correlation Explanation

Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, Aram Galstyan

1802.05822

A General Distributed Dual Coordinate Optimization Framework for Regularized Loss Minimization

Shun Zheng, Jialei Wang, Fen Xia, Wei Xu, Tong Zhang

1604.03763

Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations

Maziar Raissi, Paris Perdikaris, George Em Karniadakis

1703.10230

Learning Latent Dynamics for Planning from Pixels

Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson

1811.04551

Multi-level Residual Networks from Dynamical Systems View

Bo Chang, Lili Meng, Eldad Haber, Frederick Tung, David Begert

1710.10348

A Distributed Quasi-Newton Algorithm for Empirical Risk Minimization with Nonsmooth Regularization

Ching-pei Lee, Cong Han Lim, Stephen J. Wright

1803.01370

Learning Disentangled Joint Continuous and Discrete Representations

Emilien Dupont

1804.00104

Differential Privacy for Functions and Functional Data

Rob Hall, Alessandro Rinaldo, Larry Wasserman

1203.2570

Variational Graph Auto-Encoders

Thomas N. Kipf, Max Welling

1611.07308

Multi-Object Representation Learning with Iterative Variational Inference

Klaus Greff, Raphaël Lopez Kaufman, Rishabh Kabra, Nick Watters, Chris Burgess, Daniel Zoran, Loic Matthey, Matthew Botvinick, Alexander Lerchner

1903.00450

Full-Capacity Unitary Recurrent Neural Networks

Scott Wisdom, Thomas Powers, John R. Hershey, Jonathan Le Roux, Les Atlas

1611.00035

Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning

Aapo Hyvarinen, Hiroaki Sasaki, Richard E. Turner

1805.08651

Generative Models of Visually Grounded Imagination

Ramakrishna Vedantam, Ian Fischer, Jonathan Huang, Kevin Murphy

1705.10762

Joint Multimodal Learning with Deep Generative Models

Masahiro Suzuki, Kotaro Nakayama, Yutaka Matsuo

1611.01891

VAE with a VampPrior

Jakub M. Tomczak, Max Welling

1705.07120

Learning Invariant Representations with Local Transformations

Kihyuk Sohn, Honglak Lee

1206.6418

Wide & Deep Learning for Recommender Systems

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah

1606.07792

Reversible Architectures for Arbitrarily Deep Residual Neural Networks

Bo Chang, Lili Meng, Eldad Haber, Lars Ruthotto, David Begert, Elliot Holtham

1709.03698

Learning image representations tied to ego-motion

Dinesh Jayaraman, Kristen Grauman

1505.02206

From Averaging to Acceleration, There is Only a Step-size

Nicolas Flammarion, Francis Bach

1504.01577

Perturbed Iterate Analysis for Asynchronous Stochastic Optimization

Horia Mania, Xinghao Pan, Dimitris Papailiopoulos, Benjamin Recht, Kannan Ramchandran, Michael I. Jordan

1507.06970

Collaborative Learning of Stochastic Bandits over a Social Network

Ravi Kumar Kolla, Krishna Jagannathan, Aditya Gopalan

1602.08886

On Distributed Cooperative Decision-Making in Multiarmed Bandits

Peter Landgren, Vaibhav Srivastava, Naomi Ehrich Leonard

1512.06888

Diversity-Sensitive Conditional Generative Adversarial Networks

Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tianchen Zhao, Honglak Lee

1901.09024

Deep Variational Reinforcement Learning for POMDPs

Maximilian Igl, Luisa Zintgraf, Tuan Anh Le, Frank Wood, Shimon Whiteson

1806.02426

Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion

Jacob Buckman, Danijar Hafner, George Tucker, Eugene Brevdo, Honglak Lee

1807.01675

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, Sergey Levine

1910.10897

Variational Autoencoder for Deep Learning of Images, Labels and Captions

Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, Lawrence Carin

1609.08976

Lagging Inference Networks and Posterior Collapse in Variational Autoencoders

Junxian He, Daniel Spokoyny, Graham Neubig, Taylor Berg-Kirkpatrick

1901.05534

Imitating Human Behaviour with Diffusion Models

Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, Sam Devlin

2301.10677

Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models

Kurtland Chua, Roberto Calandra, Rowan McAllister, Sergey Levine

1805.12114

AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks

Bo Chang, Minmin Chen, Eldad Haber, Ed H. Chi

1902.09689

A Neural Autoregressive Approach to Collaborative Filtering

Yin Zheng, Bangsheng Tang, Wenkui Ding, Hanning Zhou

1605.09477

DARLA: Improving Zero-Shot Transfer in Reinforcement Learning

Irina Higgins, Arka Pal, Andrei A. Rusu, Loic Matthey, Christopher P Burgess, Alexander Pritzel, Matthew Botvinick, Charles Blundell, Alexander Lerchner

1707.08475

Iterative Amortized Inference

Joseph Marino, Yisong Yue, Stephan Mandt

1807.09356

Collaborative Deep Learning for Recommender Systems

Hao Wang, Naiyan Wang, Dit-Yan Yeung

1409.2944

MONet: Unsupervised Scene Decomposition and Representation

Christopher P. Burgess, Loic Matthey, Nicholas Watters, Rishabh Kabra, Irina Higgins, Matt Botvinick, Alexander Lerchner

1901.11390

Bilingual Learning of Multi-sense Embeddings with Discrete Autoencoders

Simon Šuster, Ivan Titov, Gertjan van Noord

1603.09128

Filtering Variational Objectives

Chris J. Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, Yee Whye Teh

1705.09279

Addressing Function Approximation Error in Actor-Critic Methods

Scott Fujimoto, Herke van Hoof, David Meger

1802.09477

A Deep Generative Deconvolutional Image Model

Yunchen Pu, Xin Yuan, Andrew Stevens, Chunyuan Li, Lawrence Carin

1512.07344

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, Sergey Levine

1801.01290

Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification

Daniel Michelsanti, Zheng-Hua Tan

1709.01703

Signal Recovery from Pooling Representations

Joan Bruna, Arthur Szlam, Yann LeCun

1311.4025

Learning protein sequence embeddings using information from structure

Tristan Bepler, Bonnie Berger

1902.08661

Learning Dexterous In-Hand Manipulation

OpenAI, Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, Jonas Schneider, Szymon Sidor, Josh Tobin, Peter Welinder, Lilian Weng, Wojciech Zaremba

1808.00177

Inference Compilation and Universal Probabilistic Programming

Tuan Anh Le, Atilim Gunes Baydin, Frank Wood

1610.09900

Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models

Bradly C. Stadie, Sergey Levine, Pieter Abbeel

1507.00814

Structured Inference Networks for Nonlinear State Space Models

Rahul G. Krishnan, Uri Shalit, David Sontag

1609.09869

Tighter Variational Bounds are Not Necessarily Better

Tom Rainforth, Adam R. Kosiorek, Tuan Anh Le, Chris J. Maddison, Maximilian Igl, Frank Wood, Yee Whye Teh

1802.04537

Describing Videos by Exploiting Temporal Structure

Li Yao, Atousa Torabi, Kyunghyun Cho, Nicolas Ballas, Christopher Pal, Hugo Larochelle, Aaron Courville

1502.08029

A Minimalist Approach to Offline Reinforcement Learning

Scott Fujimoto, Shixiang Shane Gu

2106.06860

Synthesizing Programs for Images using Reinforced Adversarial Learning

Yaroslav Ganin, Tejas Kulkarni, Igor Babuschkin, S. M. Ali Eslami, Oriol Vinyals

1804.01118

Offline RL Without Off-Policy Evaluation

David Brandfonbrener, William F. Whitney, Rajesh Ranganath, Joan Bruna

2106.08909

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

Zhisheng Xiao, Karsten Kreis, Arash Vahdat

2112.07804

A0C: Alpha Zero in Continuous Action Space

Thomas M. Moerland, Joost Broekens, Aske Plaat, Catholijn M. Jonker

1805.09613

ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero

Yuandong Tian, Jerry Ma, Qucheng Gong, Shubho Sengupta, Zhuoyuan Chen, James Pinkerton, C. Lawrence Zitnick

1902.04522

The Role of ImageNet Classes in Fréchet Inception Distance

Tuomas Kynkäänniemi, Tero Karras, Miika Aittala, Timo Aila, Jaakko Lehtinen

2203.06026

Consistency Models

Yang Song, Prafulla Dhariwal, Mark Chen, Ilya Sutskever

2303.01469

Re-evaluating Evaluation

David Balduzzi, Karl Tuyls, Julien Perolat, Thore Graepel

1806.02643

I$^2$SB: Image-to-Image Schrödinger Bridge

Guan-Horng Liu, Arash Vahdat, De-An Huang, Evangelos A. Theodorou, Weili Nie, Anima Anandkumar

2302.05872

Simple diffusion: End-to-end diffusion for high resolution images

Emiel Hoogeboom, Jonathan Heek, Tim Salimans

2301.11093

NAS-Bench-101: Towards Reproducible Neural Architecture Search

Chris Ying, Aaron Klein, Esteban Real, Eric Christiansen, Kevin Murphy, Frank Hutter

1902.09635

Diffusion Posterior Sampling for General Noisy Inverse Problems

Hyungjin Chung, Jeongsol Kim, Michael T. Mccann, Marc L. Klasky, Jong Chul Ye

2209.14687

Evaluating the Search Phase of Neural Architecture Search

Kaicheng Yu, Christian Sciuto, Martin Jaggi, Claudiu Musat, Mathieu Salzmann

1902.08142

Rethinking Collapsed Variational Bayes Inference for LDA

Issei Sato, Hiroshi Nakagawa

1206.6435

A Review of Relational Machine Learning for Knowledge Graphs

Maximilian Nickel, Kevin Murphy, Volker Tresp, Evgeniy Gabrilovich

1503.00759

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Han Cai, Ligeng Zhu, Song Han

1812.00332

Neural Architecture Optimization

Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, Tie-Yan Liu

1808.07233

Convolutional neural networks with low-rank regularization

Cheng Tai, Tong Xiao, Yi Zhang, Xiaogang Wang, Weinan E

1511.06067

Understanding the Limitations of Variational Mutual Information Estimators

Jiaming Song, Stefano Ermon

1910.06222

It Takes (Only) Two: Adversarial Generator-Encoder Networks

Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky

1704.02304

Flow Matching for Generative Modeling

Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, Matt Le

2210.02747

An analytic theory of generalization dynamics and transfer learning in deep linear networks

Andrew K. Lampinen, Surya Ganguli

1809.10374

Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview

Yuejie Chi, Yue M. Lu, Yuxin Chen

1809.09573

BOHB: Robust and Efficient Hyperparameter Optimization at Scale

Stefan Falkner, Aaron Klein, Frank Hutter

1807.01774

Learning to Search Better Than Your Teacher

Kai-Wei Chang, Akshay Krishnamurthy, Alekh Agarwal, Hal Daumé, John Langford

1502.02206

Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks

Gauthier Gidel, Francis Bach, Simon Lacoste-Julien

1904.13262

Improving Diffusion Models for Inverse Problems using Manifold Constraints

Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, Jong Chul Ye

2206.00941

Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation

Kenton Murray, Jeffery Kinnison, Toan Q. Nguyen, Walter Scheirer, David Chiang

1910.06717

Building Normalizing Flows with Stochastic Interpolants

Michael S. Albergo, Eric Vanden-Eijnden

2209.15571

Inference and Uncertainty Quantification for Noisy Matrix Completion

Yuxin Chen, Jianqing Fan, Cong Ma, Yuling Yan

1906.04159

Nonconvex Low-Rank Tensor Completion from Noisy Data

Changxiao Cai, Gen Li, H. Vincent Poor, Yuxin Chen

1911.04436

Riemannian Score-Based Generative Modelling

Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, Arnaud Doucet

2202.02763

Non-Denoising Forward-Time Diffusions

Stefano Peluchetti

2312.14589

PointPainting: Sequential Fusion for 3D Object Detection

Sourabh Vora, Alex H. Lang, Bassam Helou, Oscar Beijbom

1911.10150

Deep Generative Models for Distribution-Preserving Lossy Compression

Michael Tschannen, Eirikur Agustsson, Mario Lucic

1805.11057

Assessing Generative Models via Precision and Recall

Mehdi S. M. Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, Sylvain Gelly

1806.00035

No More Pesky Learning Rates

Tom Schaul, Sixin Zhang, Yann LeCun

1206.1106

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel

2001.07685

Lessons from Building Acoustic Models with a Million Hours of Speech

Sree Hari Krishnan Parthasarathi, Nikko Strom

1904.01624

Progress & Compress: A scalable framework for continual learning

Jonathan Schwarz, Jelena Luketina, Wojciech M. Czarnecki, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, Raia Hadsell

1805.06370

CURL: Contrastive Unsupervised Representations for Reinforcement Learning

Aravind Srinivas, Michael Laskin, Pieter Abbeel

2004.04136

Stacked Generative Adversarial Networks

Xun Huang, Yixuan Li, Omid Poursaeed, John Hopcroft, Serge Belongie

1612.04357

Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss

Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, Tengyu Ma

1906.07413

Measuring Compositionality in Representation Learning

Jacob Andreas

1902.07181

General-Purpose In-Context Learning by Meta-Learning Transformers

Louis Kirsch, James Harrison, Jascha Sohl-Dickstein, Luke Metz

2212.04458

Sequence Transduction with Recurrent Neural Networks

Alex Graves

1211.3711

A Focus on Neural Machine Translation for African Languages

Laura Martinus, Jade Z. Abbott

1906.05685

Scalability in Perception for Autonomous Driving: Waymo Open Dataset

Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Sheng Zhao, Shuyang Cheng, Yu Zhang, Jonathon Shlens, Zhifeng Chen, Dragomir Anguelov

1912.04838

Distribution Matching in Variational Inference

Mihaela Rosca, Balaji Lakshminarayanan, Shakir Mohamed

1802.06847

Multi-Task Learning as Multi-Objective Optimization

Ozan Sener, Vladlen Koltun

1810.04650

WAIC, but Why? Generative Ensembles for Robust Anomaly Detection

Hyunsun Choi, Eric Jang, Alexander A. Alemi

1810.01392

Sample Efficient Adaptive Text-to-Speech

Yutian Chen, Yannis Assael, Brendan Shillingford, David Budden, Scott Reed, Heiga Zen, Quan Wang, Luis C. Cobo, Andrew Trask, Ben Laurie, Caglar Gulcehre, Aäron van den Oord, Oriol Vinyals, Nando de Freitas

1809.10460

Deep Anomaly Detection with Outlier Exposure

Dan Hendrycks, Mantas Mazeika, Thomas Dietterich

1812.04606

On the Accuracy of Influence Functions for Measuring Group Effects

Pang Wei Koh, Kai-Siang Ang, Hubert H. K. Teo, Percy Liang

1905.13289

Generating Sentences by Editing Prototypes

Kelvin Guu, Tatsunori B. Hashimoto, Yonatan Oren, Percy Liang

1709.08878

Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching

Chih-Kuan Yeh, Jianshu Chen, Chengzhu Yu, Dong Yu

1812.09323

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, Ziyi Wu

2206.04615

Neural Programmer: Inducing Latent Programs with Gradient Descent

Arvind Neelakantan, Quoc V. Le, Ilya Sutskever

1511.04834

Interpreting Black Box Predictions using Fisher Kernels

Rajiv Khanna, Been Kim, Joydeep Ghosh, Oluwasanmi Koyejo

1810.10118

ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

Stéphane d'Ascoli, Hugo Touvron, Matthew Leavitt, Ari Morcos, Giulio Biroli, Levent Sagun

2103.10697

A note on the evaluation of generative models

Lucas Theis, Aäron van den Oord, Matthias Bethge

1511.01844

Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks

Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R. Kosiorek, Seungjin Choi, Yee Whye Teh

1810.00825

HEAR: Holistic Evaluation of Audio Representations

Joseph Turian, Jordie Shier, Humair Raj Khan, Bhiksha Raj, Björn W. Schuller, Christian J. Steinmetz, Colin Malloy, George Tzanetakis, Gissel Velarde, Kirk McNally, Max Henry, Nicolas Pinto, Camille Noufi, Christian Clough, Dorien Herremans, Eduardo Fonseca, Jesse Engel, Justin Salamon, Philippe Esling, Pranay Manocha, Shinji Watanabe, Zeyu Jin, Yonatan Bisk

2203.03022

Harnessing Structures in Big Data via Guaranteed Low-Rank Matrix Estimation

Yudong Chen, Yuejie Chi

1802.08397

Recurrent Environment Simulators

Silvia Chiappa, Sébastien Racaniere, Daan Wierstra, Shakir Mohamed

1704.02254

Disentangling factors of variation in deep representations using adversarial training

Michael Mathieu, Junbo Zhao, Pablo Sprechmann, Aditya Ramesh, Yann LeCun

1611.03383

Fair Regression: Quantitative Definitions and Reduction-based Algorithms

Alekh Agarwal, Miroslav Dudík, Zhiwei Steven Wu

1905.12843

The large learning rate phase of deep learning: the catapult mechanism

Aitor Lewkowycz, Yasaman Bahri, Ethan Dyer, Jascha Sohl-Dickstein, Guy Gur-Ari

2003.02218

Rectified Flow: A Marginal Preserving Approach to Optimal Transport

Qiang Liu

2209.14577

Accelerated Hierarchical Density Clustering

Leland McInnes, John Healy

1705.07321

Neural Networks as Kernel Learners: The Silent Alignment Effect

Alexander Atanasov, Blake Bordelon, Cengiz Pehlevan

2111.00034

The Big Data Bootstrap

Ariel Kleiner, Ameet Talwalkar, Purnamrita Sarkar, Michael Jordan

1206.6415

Towards Understanding Hierarchical Learning: Benefits of Neural Representations

Minshuo Chen, Yu Bai, Jason D. Lee, Tuo Zhao, Huan Wang, Caiming Xiong, Richard Socher

2006.13436

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

Xuezhe Ma, Eduard Hovy

1603.01354

Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction

Dominik Stöger, Mahdi Soltanolkotabi

2106.15013

Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks

Blake Bordelon, Cengiz Pehlevan

2205.09653

What Happens after SGD Reaches Zero Loss? --A Mathematical Framework

Zhiyuan Li, Tianhao Wang, Sanjeev Arora

2110.06914

Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK

Yuanzhi Li, Tengyu Ma, Hongyang R. Zhang

2007.04596

Distilling a Neural Network Into a Soft Decision Tree

Nicholas Frosst, Geoffrey Hinton

1711.09784

Toward Multimodal Image-to-Image Translation

Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A. Efros, Oliver Wang, Eli Shechtman

1711.11586

Gradient Descent Happens in a Tiny Subspace

Guy Gur-Ari, Daniel A. Roberts, Ethan Dyer

1812.04754

A Deep and Tractable Density Estimator

Benigno Uria, Iain Murray, Hugo Larochelle

1310.1757

Unrolled Generative Adversarial Networks

Luke Metz, Ben Poole, David Pfau, Jascha Sohl-Dickstein

1611.02163

Universal Transformers

Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, Łukasz Kaiser

1807.03819

Deep Image Prior

Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky

1711.10925

Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values

Julius Adebayo, Justin Gilmer, Ian Goodfellow, Been Kim

1810.03307

Priors in Bayesian Deep Learning: A Review

Vincent Fortuin

2105.06868

Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection

Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, Song Mei

2306.04637

Learning to Simulate Complex Physics with Graph Networks

Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, Peter W. Battaglia

2002.09405

Characterizing Implicit Bias in Terms of Optimization Geometry

Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro

1802.08246

The Fast Cauchy Transform and Faster Robust Linear Regression

Kenneth L. Clarkson, Petros Drineas, Malik Magdon-Ismail, Michael W. Mahoney, Xiangrui Meng, David P. Woodruff

1207.4684

On the Convergence of Adam and Beyond

Sashank J. Reddi, Satyen Kale, Sanjiv Kumar

1904.09237

Optimal approximation of piecewise smooth functions using deep ReLU neural networks

Philipp Petersen, Felix Voigtlaender

1709.05289

Calibration of Encoder Decoder Models for Neural Machine Translation

Aviral Kumar, Sunita Sarawagi

1903.00802

Towards better understanding of gradient-based attribution methods for Deep Neural Networks

Marco Ancona, Enea Ceolini, Cengiz Öztireli, Markus Gross

1711.06104

The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks

Emmanuel Abbe, Enric Boix-Adsera, Theodor Misiakiewicz

2202.08658

Towards causal generative scene models via competition of experts

Julius von Kügelgen, Ivan Ustyuzhaninov, Peter Gehler, Matthias Bethge, Bernhard Schölkopf

2004.12906

Learning to Search with MCTSnets

Arthur Guez, Théophane Weber, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals, Daan Wierstra, Rémi Munos, David Silver

1802.04697

Finite-Sample Analysis of Proximal Gradient TD Algorithms

Bo Liu, Ji Liu, Mohammad Ghavamzadeh, Sridhar Mahadevan, Marek Petrik

2006.14364

Optimization Methods for Large-Scale Machine Learning

Léon Bottou, Frank E. Curtis, Jorge Nocedal

1606.04838

How Does Batch Normalization Help Optimization?

Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, Aleksander Madry

1805.11604

Hessian-based Analysis of Large Batch Training and Robustness to Adversaries

Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, Michael W. Mahoney

1802.08241

High-dimensional dynamics of generalization error in neural networks

Madhu S. Advani, Andrew M. Saxe

1710.03667

Label Noise SGD Provably Prefers Flat Global Minimizers

Alex Damian, Tengyu Ma, Jason D. Lee

2106.06530

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, Wieland Brendel

1811.12231

Do CIFAR-10 Classifiers Generalize to CIFAR-10?

Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, Vaishaal Shankar

1806.00451

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization

Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar

1603.06560

GMAT: Global Memory Augmentation for Transformers

Ankit Gupta, Jonathan Berant

2006.03274

Compressed Sensing with Deep Image Prior and Learned Regularization

Dave Van Veen, Ajil Jalal, Mahdi Soltanolkotabi, Eric Price, Sriram Vishwanath, Alexandros G. Dimakis

1806.06438

xLSTM: Extended Long Short-Term Memory

Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter

2405.04517

Recurrent Independent Mechanisms

Anirudh Goyal, Alex Lamb, Jordan Hoffmann, Shagun Sodhani, Sergey Levine, Yoshua Bengio, Bernhard Schölkopf

1909.10893

Stochastic Gradient Descent as Approximate Bayesian Inference

Stephan Mandt, Matthew D. Hoffman, David M. Blei

1704.04289

A Practical Algorithm for Topic Modeling with Provable Guarantees

Sanjeev Arora, Rong Ge, Yoni Halpern, David Mimno, Ankur Moitra, David Sontag, Yichen Wu, Michael Zhu

1212.4777

Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning

Zhiyuan Li, Yuping Luo, Kaifeng Lyu

2012.09839

Enhanced Convolutional Neural Tangent Kernels

Zhiyuan Li, Ruosong Wang, Dingli Yu, Simon S. Du, Wei Hu, Ruslan Salakhutdinov, Sanjeev Arora

1911.00809

On Layer Normalization in the Transformer Architecture

Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, Tie-Yan Liu

2002.04745

Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks

Chunjie Luo, Jianfeng Zhan, Lei Wang, Qiang Yang

1702.05870

A Retrieve-and-Edit Framework for Predicting Structured Outputs

Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren, Percy Liang

1812.01194

A New Training Pipeline for an Improved Neural Transducer

Albert Zeyer, André Merboldt, Ralf Schlüter, Hermann Ney

2005.09319

Understanding overfitting peaks in generalization error: Analytical risk curves for $l_2$ and $l_1$ penalized interpolation

Partha P Mitra

1906.03667

Safely Learning to Control the Constrained Linear Quadratic Regulator

Sarah Dean, Stephen Tu, Nikolai Matni, Benjamin Recht

1809.10121

Discriminator Rejection Sampling

Samaneh Azadi, Catherine Olsson, Trevor Darrell, Ian Goodfellow, Augustus Odena

1810.06758

Pareto Multi-Task Learning

Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qingfu Zhang, Sam Kwong

1912.12854

Differentiable Patch Selection for Image Recognition

Jean-Baptiste Cordonnier, Aravindh Mahendran, Alexey Dosovitskiy, Dirk Weissenborn, Jakob Uszkoreit, Thomas Unterthiner

2104.03059

On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)

Zhiyuan Li, Sadhika Malladi, Sanjeev Arora

2102.12470

Plex: Towards Reliability using Pretrained Large Model Extensions

Dustin Tran, Jeremiah Liu, Michael W. Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang, Zelda Mariet, Huiyi Hu, Neil Band, Tim G. J. Rudner, Karan Singhal, Zachary Nado, Joost van Amersfoort, Andreas Kirsch, Rodolphe Jenatton, Nithum Thain, Honglin Yuan, Kelly Buchanan, Kevin Murphy, D. Sculley, Yarin Gal, Zoubin Ghahramani, Jasper Snoek, Balaji Lakshminarayanan

2207.07411

Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, Frank Hutter

1605.07079

Structured Neural Summarization

Patrick Fernandes, Miltiadis Allamanis, Marc Brockschmidt

1811.01824

Sharpness-Aware Minimization for Efficiently Improving Generalization

Pierre Foret, Ariel Kleiner, Hossein Mobahi, Behnam Neyshabur

2010.01412

Spatially Transformed Adversarial Examples

Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, Dawn Song

1801.02612

RNADE: The real-valued neural autoregressive density-estimator

Benigno Uria, Iain Murray, Hugo Larochelle

1306.0186

Evaluating and Understanding the Robustness of Adversarial Logit Pairing

Logan Engstrom, Andrew Ilyas, Anish Athalye

1807.10272

On the geometry of generalization and memorization in deep neural networks

Cory Stephenson, Suchismita Padhy, Abhinav Ganesh, Yue Hui, Hanlin Tang, SueYeon Chung

2105.14602

Gradient Surgery for Multi-Task Learning

Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, Chelsea Finn

2001.06782

Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search

Xiangxiang Chu, Tianbao Zhou, Bo Zhang, Jixiang Li

1911.12126

On the Origin of Implicit Regularization in Stochastic Gradient Descent

Samuel L. Smith, Benoit Dherin, David G. T. Barrett, Soham De

2101.12176

Nuclear norm penalization and optimal rates for noisy low rank matrix completion

Vladimir Koltchinskii, Alexandre B. Tsybakov, Karim Lounici

1011.6256

Spectral Methods for Data Science: A Statistical Perspective

Yuxin Chen, Yuejie Chi, Jianqing Fan, Cong Ma

2012.08496

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine

Pedro Domingos

2012.00152

The Principles of Deep Learning Theory

Daniel A. Roberts, Sho Yaida, Boris Hanin

2106.10165

Using Pre-Training Can Improve Model Robustness and Uncertainty

Dan Hendrycks, Kimin Lee, Mantas Mazeika

1901.09960

Deep Networks and the Multiple Manifold Problem

Sam Buchanan, Dar Gilboa, John Wright

2008.11245

Learning to Control PDEs with Differentiable Physics

Philipp Holl, Vladlen Koltun, Nils Thuerey

2001.07457

Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization

Jonas Kohler, Hadi Daneshmand, Aurelien Lucchi, Ming Zhou, Klaus Neymeyr, Thomas Hofmann

1805.10694

Extractive Adversarial Networks: High-Recall Explanations for Identifying Personal Attacks in Social Media Posts

Samuel Carton, Qiaozhu Mei, Paul Resnick

1809.01499

Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs

Inbar Seroussi, Gadi Naveh, Zohar Ringel

2112.15383

Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs

Li Jing, Yichen Shen, Tena Dubček, John Peurifoy, Scott Skirlo, Yann LeCun, Max Tegmark, Marin Soljačić

1612.05231

Unsupervised speech representation learning using WaveNet autoencoders

Jan Chorowski, Ron J. Weiss, Samy Bengio, Aäron van den Oord

1901.08810

Deep Parametric Continuous Convolutional Neural Networks

Shenlong Wang, Simon Suo, Wei-Chiu Ma, Andrei Pokrovsky, Raquel Urtasun

2101.06742

Multi-Agent Diverse Generative Adversarial Networks

Arnab Ghosh, Viveka Kulharia, Vinay Namboodiri, Philip H. S. Torr, Puneet K. Dokania

1704.02906

Between-class Learning for Image Classification

Yuji Tokozume, Yoshitaka Ushiku, Tatsuya Harada

1711.10284

Global optimality conditions for deep neural networks

Chulhee Yun, Suvrit Sra, Ali Jadbabaie

1707.02444

Limitations of Lazy Training of Two-layers Neural Networks

Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz, Andrea Montanari

1906.08899

Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss

Lenaic Chizat, Francis Bach

2002.04486

Neural Execution of Graph Algorithms

Petar Veličković, Rex Ying, Matilde Padovano, Raia Hadsell, Charles Blundell

1910.10593

Accurate Uncertainties for Deep Learning Using Calibrated Regression

Volodymyr Kuleshov, Nathan Fenner, Stefano Ermon

1807.00263

A Scalable Bootstrap for Massive Data

Ariel Kleiner, Ameet Talwalkar, Purnamrita Sarkar, Michael I. Jordan

1112.5016

Tensor principal component analysis via sum-of-squares proofs

Samuel B. Hopkins, Jonathan Shi, David Steurer

1507.03269

Energy-based Generative Adversarial Network

Junbo Zhao, Michael Mathieu, Yann LeCun

1609.03126

Contrasting random and learned features in deep Bayesian linear regression

Jacob A. Zavatone-Veth, William L. Tong, Cengiz Pehlevan

2203.00573

Neural Variational Inference for Text Processing

Yishu Miao, Lei Yu, Phil Blunsom

1511.06038

Shape Matters: Understanding the Implicit Bias of the Noise Covariance

Jeff Z. HaoChen, Colin Wei, Jason D. Lee, Tengyu Ma

2006.08680

DiffTaichi: Differentiable Programming for Physical Simulation

Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, Frédo Durand

1910.00935

On the Relationship between Self-Attention and Convolutional Layers

Jean-Baptiste Cordonnier, Andreas Loukas, Martin Jaggi

1911.03584

Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks

Yu Bai, Jason D. Lee

1910.01619

The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes

Alexander Atanasov, Blake Bordelon, Sabarish Sainathan, Cengiz Pehlevan

2212.12147

Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel

Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M. Roy, Surya Ganguli

2010.15110

Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model

Anjuli Kannan, Arindrima Datta, Tara N. Sainath, Eugene Weinstein, Bhuvana Ramabhadran, Yonghui Wu, Ankur Bapna, Zhifeng Chen, Seungji Lee

1909.05330

More Algorithms for Provable Dictionary Learning

Sanjeev Arora, Aditya Bhaskara, Rong Ge, Tengyu Ma

1401.0579

Big Bird: Transformers for Longer Sequences

Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, Amr Ahmed

2007.14062

Deep Set Prediction Networks

Yan Zhang, Jonathon Hare, Adam Prügel-Bennett

1906.06565

Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam

Mohammad Emtiyaz Khan, Didrik Nielsen, Voot Tangkaratt, Wu Lin, Yarin Gal, Akash Srivastava

1806.04854

Do Vision Transformers See Like Convolutional Neural Networks?

Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, Alexey Dosovitskiy

2108.08810

Decomposing Overcomplete 3rd Order Tensors using Sum-of-Squares Algorithms

Rong Ge, Tengyu Ma

1504.05287

What Does it Mean for a Language Model to Preserve Privacy?

Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, Florian Tramèr

2202.05520

Neural Tangents: Fast and Easy Infinite Neural Networks in Python

Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, Samuel S. Schoenholz

1912.02803

Train faster, generalize better: Stability of stochastic gradient descent

Moritz Hardt, Benjamin Recht, Yoram Singer

1509.01240

Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs

Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, Andrew Gordon Wilson

1802.10026

Interpretation of Neural Networks is Fragile

Amirata Ghorbani, Abubakar Abid, James Zou

1710.10547

The Robust Manifold Defense: Adversarial Training using Generative Models

Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, Alexandros G. Dimakis

1712.09196

Relative Positional Encoding for Transformers with Linear Complexity

Antoine Liutkus, Ondřej Cífka, Shih-Lun Wu, Umut Şimşekli, Yi-Hsuan Yang, Gaël Richard

2105.08399

A Bayesian Perspective on Generalization and Stochastic Gradient Descent

Samuel L. Smith, Quoc V. Le

1710.06451

Learning deep representations by mutual information estimation and maximization

R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, Yoshua Bengio

1808.06670

Explaining Neural Scaling Laws

Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, Utkarsh Sharma

2102.06701

Improved Training of Wasserstein GANs

Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, Aaron Courville

1704.00028

Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization

Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand, Giancarlo Kerg, Huan Wang, Caiming Xiong, Richard Socher, Kyunghyun Cho, Krzysztof Geras

2012.14193

Harmless interpolation of noisy data in regression

Vidya Muthukumar, Kailas Vodrahalli, Vignesh Subramanian, Anant Sahai

1903.09139

A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation

Akhilesh Gotmare, Nitish Shirish Keskar, Caiming Xiong, Richard Socher

1810.13243

Harnessing Deep Neural Networks with Logic Rules

Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, Eric Xing

1603.06318

High-dimensional covariance estimation by minimizing $\ell_1$-penalized log-determinant divergence

Pradeep Ravikumar, Martin J. Wainwright, Garvesh Raskutti, Bin Yu

0811.3628

Flat minima generalize for low-rank matrix recovery

Lijun Ding, Dmitriy Drusvyatskiy, Maryam Fazel, Zaid Harchaoui

2203.03756

Dropout-GAN: Learning from a Dynamic Ensemble of Discriminators

Gonçalo Mordido, Haojin Yang, Christoph Meinel

1807.11346

Motivating the Rules of the Game for Adversarial Example Research

Justin Gilmer, Ryan P. Adams, Ian Goodfellow, David Andersen, George E. Dahl

1807.06732

Generalized End-to-End Loss for Speaker Verification

Li Wan, Quan Wang, Alan Papir, Ignacio Lopez Moreno

1710.10467

The Lipschitz Constant of Self-Attention

Hyunjik Kim, George Papamakarios, Andriy Mnih

2006.04710

Methods for Interpreting and Understanding Deep Neural Networks

Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller

1706.07979

Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks

Itay Safran, Ohad Shamir

1610.09887

A Closer Look at Memorization in Deep Networks

Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, Simon Lacoste-Julien

1706.05394

Dynamics of Deep Neural Networks and Neural Tangent Hierarchy

Jiaoyang Huang, Horng-Tzer Yau

1909.08156

Fashion-Gen: The Generative Fashion Dataset and Challenge

Negar Rostamzadeh, Seyedarian Hosseini, Thomas Boquet, Wojciech Stokowiec, Ying Zhang, Christian Jauvin, Chris Pal

1806.08317

Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks

Sanjeev Arora, Simon S. Du, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang, Dingli Yu

1910.01663

Feature Purification: How Adversarial Training Performs Robust Deep Learning

Zeyuan Allen-Zhu, Yuanzhi Li

2005.10190

code2seq: Generating Sequences from Structured Representations of Code

Uri Alon, Shaked Brody, Omer Levy, Eran Yahav

1808.01400

Efficient Content-Based Sparse Attention with Routing Transformers

Aurko Roy, Mohammad Saffar, Ashish Vaswani, David Grangier

2003.05997

Deep Equilibrium Models

Shaojie Bai, J. Zico Kolter, Vladlen Koltun

1909.01377

Transfer Learning for Named-Entity Recognition with Neural Networks

Ji Young Lee, Franck Dernoncourt, Peter Szolovits

1705.06273

Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang

1707.09457

Counterfactual Fairness

Matt J. Kusner, Joshua R. Loftus, Chris Russell, Ricardo Silva

1703.06856

Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)

Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, Rory Sayres

1711.11279

Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models

Jesse Engel, Matthew Hoffman, Adam Roberts

1711.05772

Alias-Free Generative Adversarial Networks

Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, Timo Aila

2106.12423

Fast Graph Representation Learning with PyTorch Geometric

Matthias Fey, Jan Eric Lenssen

1903.02428

Stochastic Blockmodels meet Graph Neural Networks

Nikhil Mehta, Lawrence Carin, Piyush Rai

1905.05738

How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?

Ferenc Huszár

1511.05101

Neural Controlled Differential Equations for Irregular Time Series

Patrick Kidger, James Morrill, James Foster, Terry Lyons

2005.08926

Inductive Representation Learning on Large Graphs

William L. Hamilton, Rex Ying, Jure Leskovec

1706.02216

Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation

Reinhard Heckel, Mahdi Soltanolkotabi

2005.03991

Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit

Song Mei, Theodor Misiakiewicz, Andrea Montanari

1902.06015

Just Interpolate: Kernel "Ridgeless" Regression Can Generalize

Tengyuan Liang, Alexander Rakhlin

1808.00387

Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach

Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, Luca Daniel

1801.10578

Transformers Learn Shortcuts to Automata

Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang

2210.10749

Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech

Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, Mikhail Kudinov

2105.06337

An Explicit Sampling Dependent Spectral Error Bound for Column Subset Selection

Tianbao Yang, Lijun Zhang, Rong Jin, Shenghuo Zhu

1505.00526

Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound

Zhao Song, Xin Yang

1906.03593

Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?

Raja Giryes, Guillermo Sapiro, Alex M. Bronstein

1504.08291

PointPillars: Fast Encoders for Object Detection from Point Clouds

Alex H. Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, Oscar Beijbom

1812.05784

Lipschitz and Comparator-Norm Adaptivity in Online Learning

Zakaria Mhammedi, Wouter M. Koolen

2002.12242

Explaining NonLinear Classification Decisions with Deep Taylor Decomposition

Grégoire Montavon, Sebastian Bach, Alexander Binder, Wojciech Samek, Klaus-Robert Müller

1512.02479

The Conditional Analogy GAN: Swapping Fashion Articles on People Images

Nikolay Jetchev, Urs Bergmann

1709.04695

Learning Latent Representations for Speech Generation and Transformation

Wei-Ning Hsu, Yu Zhang, James Glass

1704.04222

Differentially Private Fine-tuning of Language Models

Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A. Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, Huishuai Zhang

2110.06500

Stochastic Normalizing Flows

Hao Wu, Jonas Köhler, Frank Noé

2002.06707

An Empirical Model of Large-Batch Training

Sam McCandlish, Jared Kaplan, Dario Amodei, OpenAI Dota Team

1812.06162

Multi-Task Learning of Keyphrase Boundary Classification

Isabelle Augenstein, Anders Søgaard

1704.00514

EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples

Pin-Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh

1709.04114

NSML: Meet the MLaaS platform with a real-world case study

Hanjoo Kim, Minkyu Kim, Dongjoo Seo, Jinwoong Kim, Heungseok Park, Soeun Park, Hyunwoo Jo, KyungHyun Kim, Youngil Yang, Youngkwan Kim, Nako Sung, Jung-Woo Ha

1810.09957

The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks

Wei Hu, Lechao Xiao, Ben Adlam, Jeffrey Pennington

2006.14599

Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages

Wittawat Jitkrittum, Arthur Gretton, Nicolas Heess, S. M. Ali Eslami, Balaji Lakshminarayanan, Dino Sejdinovic, Zoltán Szabó

1503.02551

A Deep Reinforcement Learning Chatbot

Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, Sai Rajeshwar, Alexandre de Brebisson, Jose M. R. Sotelo, Dendi Suhubdy, Vincent Michalski, Alexandre Nguyen, Joelle Pineau, Yoshua Bengio

1709.02349

Graph Neural Networks: A Review of Methods and Applications

Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun

1812.08434

Lower Bounds for Non-Convex Stochastic Optimization

Yossi Arjevani, Yair Carmon, John C. Duchi, Dylan J. Foster, Nathan Srebro, Blake Woodworth

1912.02365

On Exact Computation with an Infinitely Wide Neural Net

Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang

1904.11955

A Universally Optimal Multistage Accelerated Stochastic Gradient Method

Necdet Serhat Aybat, Alireza Fallah, Mert Gurbuzbalaban, Asuman Ozdaglar

1901.08022

Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks

Jost Tobias Springenberg

1511.06390

Visual Causal Feature Learning

Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt

1412.2309

Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron

Sharan Vaswani, Francis Bach, Mark Schmidt

1810.07288

Precise Tradeoffs in Adversarial Training for Linear Regression

Adel Javanmard, Mahdi Soltanolkotabi, Hamed Hassani

2002.10477

Learning A Task-Specific Deep Architecture For Clustering

Zhangyang Wang, Shiyu Chang, Jiayu Zhou, Meng Wang, Thomas S. Huang

1509.00151

Explaining Recurrent Neural Network Predictions in Sentiment Analysis

Leila Arras, Grégoire Montavon, Klaus-Robert Müller, Wojciech Samek

1706.07206

Deep Kalman Filters

Rahul G. Krishnan, Uri Shalit, David Sontag

1511.05121

On the Turing Completeness of Modern Neural Network Architectures

Jorge Pérez, Javier Marinković, Pablo Barceló

1901.03429

The Cramer Distance as a Solution to Biased Wasserstein Gradients

Marc G. Bellemare, Ivo Danihelka, Will Dabney, Shakir Mohamed, Balaji Lakshminarayanan, Stephan Hoyer, Rémi Munos

1705.10743

The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization

Yifei Min, Lin Chen, Amin Karbasi

2002.11080

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Tongzhou Wang, Phillip Isola

2005.10242

What Can ResNet Learn Efficiently, Going Beyond Kernels?

Zeyuan Allen-Zhu, Yuanzhi Li

1905.10337

TapNet: Neural Network Augmented with Task-Adaptive Projection for Few-Shot Learning

Sung Whan Yoon, Jun Seo, Jaekyun Moon

1905.06549

Hierarchical Imitation and Reinforcement Learning

Hoang M. Le, Nan Jiang, Alekh Agarwal, Miroslav Dudík, Yisong Yue, Hal Daumé

1803.00590

Sparse Estimation with the Swept Approximated Message-Passing Algorithm

Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborová

1406.4311

Unsupervised Learning by Predicting Noise

Piotr Bojanowski, Armand Joulin

1704.05310

Inference Suboptimality in Variational Autoencoders

Chris Cremer, Xuechen Li, David Duvenaud

1801.03558

Towards Efficient Data Valuation Based on the Shapley Value

Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gurel, Bo Li, Ce Zhang, Dawn Song, Costas Spanos

1902.10275

FSPool: Learning Set Representations with Featurewise Sort Pooling

Yan Zhang, Jonathon Hare, Adam Prügel-Bennett

1906.02795

Evolutionary Generative Adversarial Networks

Chaoyue Wang, Chang Xu, Xin Yao, Dacheng Tao

1803.00657

Deep Learning without Poor Local Minima

Kenji Kawaguchi

1605.07110

Insights on representational similarity in neural networks with canonical correlation

Ari S. Morcos, Maithra Raghu, Samy Bengio

1806.05759

Krylov Subspace Descent for Deep Learning

Oriol Vinyals, Daniel Povey

1111.4259

One-Shot Generalization in Deep Generative Models

Danilo Jimenez Rezende, Shakir Mohamed, Ivo Danihelka, Karol Gregor, Daan Wierstra

1603.05106

A Study on Overfitting in Deep Reinforcement Learning

Chiyuan Zhang, Oriol Vinyals, Remi Munos, Samy Bengio

1804.06893

Machine Learning Force Fields

Oliver T. Unke, Stefan Chmiela, Huziel E. Sauceda, Michael Gastegger, Igor Poltavsky, Kristof T. Schütt, Alexandre Tkatchenko, Klaus-Robert Müller

2010.07067

code2vec: Learning Distributed Representations of Code

Uri Alon, Meital Zilberstein, Omer Levy, Eran Yahav

1803.09473

Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks

Kazuki Osawa, Yohei Tsuji, Yuichiro Ueno, Akira Naruse, Rio Yokota, Satoshi Matsuoka

1811.12019

White-box vs Black-box: Bayes Optimal Strategies for Membership Inference

Alexandre Sablayrolles, Matthijs Douze, Yann Ollivier, Cordelia Schmid, Hervé Jégou

1908.11229

Batched High-dimensional Bayesian Optimization via Structural Kernel Learning

Zi Wang, Chengtao Li, Stefanie Jegelka, Pushmeet Kohli

1703.01973

Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks

Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo, Zheng Ma

1905.10264

Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel

Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, Ruslan Salakhutdinov

1908.11775

Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning

Dilin Wang, Qiang Liu

1611.01722

Pitfalls of Graph Neural Network Evaluation

Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, Stephan Günnemann

1811.05868

AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty

Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, Balaji Lakshminarayanan

1912.02781

Computational Lower Bounds for Community Detection on Random Graphs

Bruce Hajek, Yihong Wu, Jiaming Xu

1406.6625

Spurious Local Minima are Common in Two-Layer ReLU Neural Networks

Itay Safran, Ohad Shamir

1712.08968

Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures

John R. Hershey, Jonathan Le Roux, Felix Weninger

1409.2574

Structured Pruning of Deep Convolutional Neural Networks

Sajid Anwar, Kyuyeon Hwang, Wonyong Sung

1512.08571

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

Ossama Ahmed, Frederik Träuble, Anirudh Goyal, Alexander Neitz, Yoshua Bengio, Bernhard Schölkopf, Manuel Wüthrich, Stefan Bauer

2010.04296

Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, Dmitry Kalenichenko

1712.05877

Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks

Guodong Zhang, James Martens, Roger Grosse

1905.10961

Community Detection in Random Networks

Ery Arias-Castro, Nicolas Verzelen

1302.7099

Learning Curve Theory

Marcus Hutter

2102.04074

Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization

Tian Ye, Simon S. Du

2106.14289

Multiscale Deep Equilibrium Models

Shaojie Bai, Vladlen Koltun, J. Zico Kolter

2006.08656

Diffusion Probabilistic Modeling for Video Generation

Ruihan Yang, Prakhar Srivastava, Stephan Mandt

2203.09481

Instabilities of Offline RL with Pre-Trained Neural Representation

Ruosong Wang, Yifan Wu, Ruslan Salakhutdinov, Sham M. Kakade

2103.04947

Approximate Gradient Coding via Sparse Random Graphs

Zachary Charles, Dimitris Papailiopoulos, Jordan Ellenberg

1711.06771

Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling

Tong Che, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, Yoshua Bengio

2003.06060

Probabilistic Model-Agnostic Meta-Learning

Chelsea Finn, Kelvin Xu, Sergey Levine

1806.02817

Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription

Nicolas Boulanger-Lewandowski, Yoshua Bengio, Pascal Vincent

1206.6392

No bad local minima: Data independent training error guarantees for multilayer neural networks

Daniel Soudry, Yair Carmon

1605.08361

Gradient descent GAN optimization is locally stable

Vaishnavh Nagarajan, J. Zico Kolter

1706.04156

Stabilizing Transformers for Reinforcement Learning

Emilio Parisotto, H. Francis Song, Jack W. Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant M. Jayakumar, Max Jaderberg, Raphael Lopez Kaufman, Aidan Clark, Seb Noury, Matthew M. Botvinick, Nicolas Heess, Raia Hadsell

1910.06764

GENIE: Higher-Order Denoising Diffusion Solvers

Tim Dockhorn, Arash Vahdat, Karsten Kreis

2210.05475

Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution

Cong Ma, Kaizheng Wang, Yuejie Chi, Yuxin Chen

1711.10467

A General Theoretical Paradigm to Understand Learning from Human Preferences

Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, Rémi Munos

2310.12036

A DIRT-T Approach to Unsupervised Domain Adaptation

Rui Shu, Hung H. Bui, Hirokazu Narui, Stefano Ermon

1802.08735

Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks

Yuan Cao, Quanquan Gu

1905.13210

Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition

Naoya Takeishi, Yoshinobu Kawahara, Takehisa Yairi

1710.04340

Domain-Adversarial Neural Networks

Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand

1412.4446

$O(n)$ Connections are Expressive Enough: Universal Approximability of Sparse Transformers

Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar

2006.04862

Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels

Simon S. Du, Kangcheng Hou, Barnabás Póczos, Ruslan Salakhutdinov, Ruosong Wang, Keyulu Xu

1905.13192

Matrix Completion from a Few Entries

Raghunandan H. Keshavan, Andrea Montanari, Sewoong Oh

0901.3150

Statistical and computational trade-offs in estimation of sparse principal components

Tengyao Wang, Quentin Berthet, Richard J. Samworth

1408.5369

Abstract Syntax Networks for Code Generation and Semantic Parsing

Maxim Rabinovich, Mitchell Stern, Dan Klein

1704.07535

Stability and Convergence Trade-off of Iterative Optimization Algorithms

Yuansi Chen, Chi Jin, Bin Yu

1804.01619

On the Power of Over-parametrization in Neural Networks with Quadratic Activation

Simon S. Du, Jason D. Lee

1803.01206

Meta-Learning with Latent Embedding Optimization

Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, Raia Hadsell

1807.05960

Communication Compression for Decentralized Training

Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, Ji Liu

1803.06443

From optimal transport to generative modeling: the VEGAN cookbook

Olivier Bousquet, Sylvain Gelly, Ilya Tolstikhin, Carl-Johann Simon-Gabriel, Bernhard Schoelkopf

1705.07642

Bayesian Model-Agnostic Meta-Learning

Taesup Kim, Jaesik Yoon, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, Sungjin Ahn

1806.03836

Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning

Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, Susanne Saminger-Platz

1702.08811

Stochastic Variational Inference

Matt Hoffman, David M. Blei, Chong Wang, John Paisley

1206.7051

Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-$1$ Updates

Animashree Anandkumar, Rong Ge, Majid Janzamin

1402.5180

Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced

Simon S. Du, Wei Hu, Jason D. Lee

1806.00900

Revisiting Model Stitching to Compare Neural Representations

Yamini Bansal, Preetum Nakkiran, Boaz Barak

2106.07682

Optimizing Neural Networks with Kronecker-factored Approximate Curvature

James Martens, Roger Grosse

1503.05671

Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics

Christoph Wehmeyer, Frank Noé

1710.11239

Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence

Nicolas Loizou, Sharan Vaswani, Issam Laradji, Simon Lacoste-Julien

2002.10542

Exploring the Landscape of Spatial Robustness

Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, Aleksander Madry

1712.02779

RuleMatrix: Visualizing and Understanding Classifiers with Rules

Yao Ming, Huamin Qu, Enrico Bertini

1807.06228

Disentangling Trainability and Generalization in Deep Neural Networks

Lechao Xiao, Jeffrey Pennington, Samuel S. Schoenholz

1912.13053

No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis

Rong Ge, Chi Jin, Yi Zheng

1704.00708

Elucidating the Design Space of Diffusion-Based Generative Models

Tero Karras, Miika Aittala, Timo Aila, Samuli Laine

2206.00364

Tighter Low-rank Approximation via Sampling the Leveraged Element

Srinadh Bhojanapalli, Prateek Jain, Sujay Sanghavi

1410.3886

TACO: Learning Task Decomposition via Temporal Alignment for Control

Kyriacos Shiarlis, Markus Wulfmeier, Sasha Salter, Shimon Whiteson, Ingmar Posner

1803.01840

A jamming transition from under- to over-parametrization affects loss landscape and generalization

Stefano Spigler, Mario Geiger, Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Matthieu Wyart

1810.09665

Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo

Yu-Xiang Wang, Stephen E. Fienberg, Alex Smola

1502.07645

Graph Attention Networks

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio

1710.10903

Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexity

Kaiqing Zhang, Sham M. Kakade, Tamer Başar, Lin F. Yang

2007.07461

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

Noah Hollmann, Samuel Müller, Katharina Eggensperger, Frank Hutter

2207.01848

Optimal Regularization Can Mitigate Double Descent

Preetum Nakkiran, Prayaag Venkat, Sham Kakade, Tengyu Ma

2003.01897

Complex and Holographic Embeddings of Knowledge Graphs: A Comparison

Théo Trouillon, Maximilian Nickel

1707.01475

Strong Baselines for Neural Semi-supervised Learning under Domain Shift

Sebastian Ruder, Barbara Plank

1804.09530

Zero-Shot Visual Imitation

Deepak Pathak, Parsa Mahmoudieh, Guanghao Luo, Pulkit Agrawal, Dian Chen, Yide Shentu, Evan Shelhamer, Jitendra Malik, Alexei A. Efros, Trevor Darrell

1804.08606

Hierarchical Variational Models

Rajesh Ranganath, Dustin Tran, David M. Blei

1511.02386

Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS

Lin Chen, Sheng Xu

2009.10683

The edge of chaos: quantum field theory and deep neural networks

Kevin T. Grosvenor, Ro Jefferson

2109.13247

Practical Inexact Proximal Quasi-Newton Method with Global Complexity Analysis

Katya Scheinberg, Xiaocheng Tang

1311.6547

Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge

Emmanuel de Bezenac, Arthur Pajot, Patrick Gallinari

1711.07970

Membership Inference Attacks against Machine Learning Models

Reza Shokri, Marco Stronati, Congzheng Song, Vitaly Shmatikov

1610.05820

Contrastive Learning with Hard Negative Samples

Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, Stefanie Jegelka

2010.04592

Geometric Inference for General High-Dimensional Linear Inverse Problems

T. Tony Cai, Tengyuan Liang, Alexander Rakhlin

1404.4408

Spectral algorithms for tensor completion

Andrea Montanari, Nike Sun

1612.07866

Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problems

Christopher De Sa, Kunle Olukotun, Christopher Ré

1411.1134

Deep Gaussian Processes with Convolutional Kernels

Vinayak Kumar, Vaibhav Singh, P. K. Srijith, Andreas Damianou

1806.01655

Natasha: Faster Non-Convex Stochastic Optimization Via Strongly Non-Convex Parameter

Zeyuan Allen-Zhu

1702.00763

Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited

Wesley J. Maddox, Gregory Benton, Andrew Gordon Wilson

2003.02139

Large-Scale Learnable Graph Convolutional Networks

Hongyang Gao, Zhengyang Wang, Shuiwang Ji

1808.03965

Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers

Guodong Zhang, Aleksandar Botev, James Martens

2203.08120

AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery

Izhar Wallach, Michael Dzamba, Abraham Heifets

1510.02855

Minimax Theory for High-dimensional Gaussian Mixtures with Sparse Mean Separation

Martin Azizyan, Aarti Singh, Larry Wasserman

1306.2035

Fisher GAN

Youssef Mroueh, Tom Sercu

1705.09675

Deep Exploration via Randomized Value Functions

Ian Osband, Benjamin Van Roy, Daniel Russo, Zheng Wen

1703.07608

Adversarial Uncertainty Quantification in Physics-Informed Neural Networks

Yibo Yang, Paris Perdikaris

1811.04026

High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation

Jimmy Ba, Murat A. Erdogdu, Taiji Suzuki, Zhichao Wang, Denny Wu, Greg Yang

2205.01445

GNNExplainer: Generating Explanations for Graph Neural Networks

Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, Jure Leskovec

1903.03894

Deep Directed Generative Autoencoders

Sherjil Ozair, Yoshua Bengio

1410.0630

The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs

Han Liu, John Lafferty, Larry Wasserman

0903.0649

Deep Generative Models with Learnable Knowledge Constraints

Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Xiaodan Liang, Lianhui Qin, Haoye Dong, Eric Xing

1806.09764

What Algorithms can Transformers Learn? A Study in Length Generalization

Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Josh Susskind, Samy Bengio, Preetum Nakkiran

2310.16028

RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment

Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, Tong Zhang

2304.06767

Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate

Mikhail Belkin, Daniel Hsu, Partha Mitra

1806.05161

A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness

Jeremiah Zhe Liu, Shreyas Padhy, Jie Ren, Zi Lin, Yeming Wen, Ghassen Jerfel, Zack Nado, Jasper Snoek, Dustin Tran, Balaji Lakshminarayanan

2205.00403

Large-Scale Study of Curiosity-Driven Learning

Yuri Burda, Harri Edwards, Deepak Pathak, Amos Storkey, Trevor Darrell, Alexei A. Efros

1808.04355

Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination

Nathan Kallus, Xiaojie Mao, Angela Zhou

1906.00285

Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks

Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, Zheng Ma

1901.06523

Interacting Particle Markov Chain Monte Carlo

Tom Rainforth, Christian A. Naesseth, Fredrik Lindsten, Brooks Paige, Jan-Willem van de Meent, Arnaud Doucet, Frank Wood

1602.05128

Modular Duality in Deep Learning

Jeremy Bernstein, Laker Newhouse

2410.21265

Deep Hyperspherical Learning

Weiyang Liu, Yan-Ming Zhang, Xingguo Li, Zhiding Yu, Bo Dai, Tuo Zhao, Le Song

1711.03189

Neural Autoregressive Flows

Chin-Wei Huang, David Krueger, Alexandre Lacoste, Aaron Courville

1804.00779

An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization

Hamid Reza Feyzmahdavian, Arda Aytekin, Mikael Johansson

1505.04824

Learning Deep Generative Models of Graphs

Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, Peter Battaglia

1803.03324

Adversarial Attacks and Defences Competition

Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, Alan Yuille, Sangxia Huang, Yao Zhao, Yuzhe Zhao, Zhonglin Han, Junjiajia Long, Yerkebulan Berdibekov, Takuya Akiba, Seiya Tokui, Motoki Abe

1804.00097

Task-Driven Dictionary Learning

Julien Mairal, Francis Bach, Jean Ponce

1009.5358

The Rate of Convergence of AdaBoost

Indraneel Mukherjee, Cynthia Rudin, Robert E. Schapire

1106.6024

Deeper Insights into Weight Sharing in Neural Architecture Search

Yuge Zhang, Zejun Lin, Junyang Jiang, Quanlu Zhang, Yujing Wang, Hui Xue, Chen Zhang, Yaming Yang

2001.01431

Multilingual Autoregressive Entity Linking

Nicola De Cao, Ledell Wu, Kashyap Popat, Mikel Artetxe, Naman Goyal, Mikhail Plekhanov, Luke Zettlemoyer, Nicola Cancedda, Sebastian Riedel, Fabio Petroni

2103.12528

Learning Sparse Neural Networks through $L_0$ Regularization

Christos Louizos, Max Welling, Diederik P. Kingma

1712.01312

Convergence guarantees for kernel-based quadrature rules in misspecified settings

Motonobu Kanagawa, Bharath K. Sriperumbudur, Kenji Fukumizu

1605.07254

Real or Fake? Learning to Discriminate Machine from Human Generated Text

Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc'Aurelio Ranzato, Arthur Szlam

1906.03351

Kernel-based Conditional Independence Test and Application in Causal Discovery

Kun Zhang, Jonas Peters, Dominik Janzing, Bernhard Schoelkopf

1202.3775

Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

Abhishek Kumar, Prasanna Sattigeri, Avinash Balakrishnan

1711.00848

Beyond neural scaling laws: beating power law scaling via data pruning

Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari S. Morcos

2206.14486

Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting

Aditya Grover, Jiaming Song, Alekh Agarwal, Kenneth Tran, Ashish Kapoor, Eric Horvitz, Stefano Ermon

1906.09531

ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates

Zhimin Peng, Yangyang Xu, Ming Yan, Wotao Yin

1506.02396

Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization

John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, Ludwig Schmidt

2107.04649

Geometric GAN

Jae Hyun Lim, Jong Chul Ye

1705.02894

Deep Bayesian Active Learning with Image Data

Yarin Gal, Riashat Islam, Zoubin Ghahramani

1703.02910

Exact tensor completion with sum-of-squares

Aaron Potechin, David Steurer

1702.06237

Decoupling Representation Learning from Reinforcement Learning

Adam Stooke, Kimin Lee, Pieter Abbeel, Michael Laskin

2009.08319

Learning without Concentration for General Loss Functions

Shahar Mendelson

1410.3192

Estimating the Hessian by Back-propagating Curvature

James Martens, Ilya Sutskever, Kevin Swersky

1206.6464

On the linearity of large non-linear models: when and why the tangent kernel is constant

Chaoyue Liu, Libin Zhu, Mikhail Belkin

2010.01092

Variational Autoencoders Pursue PCA Directions (by Accident)

Michal Rolinek, Dominik Zietlow, Georg Martius

1812.06775

Deep Learning with Gaussian Differential Privacy

Zhiqi Bu, Jinshuo Dong, Qi Long, Weijie J. Su

1911.11607

Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition

Hamed Karimi, Julie Nutini, Mark Schmidt

1608.04636

Learning Discrete Structures for Graph Neural Networks

Luca Franceschi, Mathias Niepert, Massimiliano Pontil, Xiao He

1903.11960

Adversarial Logit Pairing

Harini Kannan, Alexey Kurakin, Ian Goodfellow

1803.06373

The challenge of realistic music generation: modelling raw audio at scale

Sander Dieleman, Aäron van den Oord, Karen Simonyan

1806.10474

Quantized Adam with Error Feedback

Congliang Chen, Li Shen, Haozhi Huang, Wei Liu

2004.14180

Finite Versus Infinite Neural Networks: an Empirical Study

Jaehoon Lee, Samuel S. Schoenholz, Jeffrey Pennington, Ben Adlam, Lechao Xiao, Roman Novak, Jascha Sohl-Dickstein

2007.15801

Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning

Supasorn Suwajanakorn, Noah Snavely, Jonathan Tompson, Mohammad Norouzi

1807.03146

Fixing a Broken ELBO

Alexander A. Alemi, Ben Poole, Ian Fischer, Joshua V. Dillon, Rif A. Saurous, Kevin Murphy

1711.00464

Convergence rates of efficient global optimization algorithms

Adam D. Bull

1101.3501

Effects of Depth, Width, and Initialization: A Convergence Analysis of Layer-wise Training for Deep Linear Neural Networks

Yeonjong Shin

1910.05874

Image Augmentations for GAN Training

Zhengli Zhao, Zizhao Zhang, Ting Chen, Sameer Singh, Han Zhang

2006.02595

StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks

Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dimitris Metaxas

1612.03242

Improving GANs for Speech Enhancement

Huy Phan, Ian V. McLoughlin, Lam Pham, Oliver Y. Chén, Philipp Koch, Maarten De Vos, Alfred Mertins

2001.05532

Individual Privacy Accounting via a Renyi Filter

Vitaly Feldman, Tijana Zrnic

2008.11193

Doubly Robust Off-policy Value Evaluation for Reinforcement Learning

Nan Jiang, Lihong Li

1511.03722

Stochastic Optimization of Sorting Networks via Continuous Relaxations

Aditya Grover, Eric Wang, Aaron Zweig, Stefano Ermon

1903.08850

Compositional generalization in a deep seq2seq model by separating syntax and semantics

Jake Russin, Jason Jo, Randall C. O'Reilly, Yoshua Bengio

1904.09708

Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF

Anand Siththaranjan, Cassidy Laidlaw, Dylan Hadfield-Menell

2312.08358

Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks

Pratik Chaudhari, Stefano Soatto

1710.11029

Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control

Sanket Kamthe, Marc Peter Deisenroth

1706.06491

Position-aware Graph Neural Networks

Jiaxuan You, Rex Ying, Jure Leskovec

1906.04817

Fairness and Missing Values

Fernando Martínez-Plumed, Cèsar Ferri, David Nieves, José Hernández-Orallo

1905.12728

Global Convergence of a Grassmannian Gradient Descent Algorithm for Subspace Estimation

Dejiao Zhang, Laura Balzano

1506.07405

DSNAS: Direct Neural Architecture Search without Parameter Retraining

Shoukang Hu, Sirui Xie, Hehui Zheng, Chunxiao Liu, Jianping Shi, Xunying Liu, Dahua Lin

2002.09128

Near-Optimal Algorithms for Minimax Optimization

Tianyi Lin, Chi Jin, Michael. I. Jordan

2002.02417

Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks

Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion

2404.02151

MixMatch: A Holistic Approach to Semi-Supervised Learning

David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin Raffel

1905.02249

The relativistic discriminator: a key element missing from standard GAN

Alexia Jolicoeur-Martineau

1807.00734

Multi-Generator Generative Adversarial Nets

Quan Hoang, Tu Dinh Nguyen, Trung Le, Dinh Phung

1708.02556

Taming the Wild: A Unified Analysis of Hogwild!-Style Algorithms

Christopher De Sa, Ce Zhang, Kunle Olukotun, Christopher Ré

1506.06438

Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training

Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Hongjun Choi, Blake Hechtman, Shibo Wang

2004.13336

Additive Kernels for Gaussian Process Modeling

Nicolas Durrande, David Ginsbourger, Olivier Roustant

1103.4023

Implicit Gradient Regularization

David G. T. Barrett, Benoit Dherin

2009.11162

Searching for Efficient Multi-Scale Architectures for Dense Image Prediction

Liang-Chieh Chen, Maxwell D. Collins, Yukun Zhu, George Papandreou, Barret Zoph, Florian Schroff, Hartwig Adam, Jonathon Shlens

1809.04184

Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling

Jacob Menick, Nal Kalchbrenner

1812.01608

Aligning where to see and what to tell: image caption with region-based attention and scene factorization

Junqi Jin, Kun Fu, Runpeng Cui, Fei Sha, Changshui Zhang

1506.06272

Runaway Feedback Loops in Predictive Policing

Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger, Suresh Venkatasubramanian

1706.09847

The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit

Lorenzo Noci, Chuning Li, Mufan Bill Li, Bobby He, Thomas Hofmann, Chris Maddison, Daniel M. Roy

2306.17759

High-recall causal discovery for autocorrelated time series with latent confounders

Andreas Gerhardus, Jakob Runge

2007.01884

Conditional Image Generation with Score-Based Diffusion Models

Georgios Batzolis, Jan Stanczuk, Carola-Bibiane Schönlieb, Christian Etmann

2111.13606

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, Kunal Talwar

1610.05755

Learning with convolution and pooling operations in kernel methods

Theodor Misiakiewicz, Song Mei

2111.08308

LieTransformer: Equivariant self-attention for Lie Groups

Michael Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Teh, Hyunjik Kim

2012.10885

Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets

Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, Nicholas Carlini

2204.00032

Learning to Decompose and Disentangle Representations for Video Prediction

Jun-Ting Hsieh, Bingbin Liu, De-An Huang, Li Fei-Fei, Juan Carlos Niebles

1806.04166

Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training

Tasnim Mohiuddin, Shafiq Joty

1904.04116

Learning One-hidden-layer Neural Networks with Landscape Design

Rong Ge, Jason D. Lee, Tengyu Ma

1711.00501

Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

Andrew M. Saxe, James L. McClelland, Surya Ganguli

1312.6120

Exploratory Gradient Boosting for Reinforcement Learning in Complex Domains

David Abel, Alekh Agarwal, Fernando Diaz, Akshay Krishnamurthy, Robert E. Schapire

1603.04119

Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra

Vardan Papyan

2008.11865

Directional convergence and alignment in deep learning

Ziwei Ji, Matus Telgarsky

2006.06657

Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs

Sangwoo Mo, Minsu Cho, Jinwoo Shin

2002.10964

Certified Adversarial Robustness via Randomized Smoothing

Jeremy M Cohen, Elan Rosenfeld, J. Zico Kolter

1902.02918

Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design

Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, Pieter Abbeel

1902.00275

A High Probability Analysis of Adaptive SGD with Momentum

Xiaoyu Li, Francesco Orabona

2007.14294

A Complete Recipe for Stochastic Gradient MCMC

Yi-An Ma, Tianqi Chen, Emily B. Fox

1506.04696

Slice sampling covariance hyperparameters of latent Gaussian models

Iain Murray, Ryan Prescott Adams

1006.0868

Reinforcement Learning for Relation Classification from Noisy Data

Jun Feng, Minlie Huang, Li Zhao, Yang Yang, Xiaoyan Zhu

1808.08013

Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision

Fredrik K. Gustafsson, Martin Danelljan, Thomas B. Schön

1906.01620

Gradient Descent with Random Initialization: Fast Global Convergence for Nonconvex Phase Retrieval

Yuxin Chen, Yuejie Chi, Jianqing Fan, Cong Ma

1803.07726

Contextual Explanation Networks

Maruan Al-Shedivat, Avinava Dubey, Eric P. Xing

1705.10301

Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?

Abhishek Das, Harsh Agrawal, C. Lawrence Zitnick, Devi Parikh, Dhruv Batra

1606.05589

High-Fidelity Image Generation With Fewer Labels

Mario Lucic, Michael Tschannen, Marvin Ritter, Xiaohua Zhai, Olivier Bachem, Sylvain Gelly

1903.02271

How rotational invariance of common kernels prevents generalization in high dimensions

Konstantin Donhauser, Mingqi Wu, Fanny Yang

2104.04244

Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree

Chen-Yu Lee, Patrick W. Gallagher, Zhuowen Tu

1509.08985

Deep Neural Networks Motivated by Partial Differential Equations

Lars Ruthotto, Eldad Haber

1804.04272

Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge

Pasquale Minervini, Sebastian Riedel

1808.08609

Systematic Evaluation of Privacy Risks of Machine Learning Models

Liwei Song, Prateek Mittal

2003.10595

Random Search and Reproducibility for Neural Architecture Search

Liam Li, Ameet Talwalkar

1902.07638

Recurrent Neural Network Training with Dark Knowledge Transfer

Zhiyuan Tang, Dong Wang, Zhiyong Zhang

1505.04630

Learning Hierarchical Features from Generative Models

Shengjia Zhao, Jiaming Song, Stefano Ermon

1702.08396

Compositional Generalization via Neural-Symbolic Stack Machines

Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, Denny Zhou

2008.06662

Deep Learning Techniques for Inverse Problems in Imaging

Gregory Ongie, Ajil Jalal, Christopher A. Metzler, Richard G. Baraniuk, Alexandros G. Dimakis, Rebecca Willett

2005.06001

Information Leakage in Embedding Models

Congzheng Song, Ananth Raghunathan

2004.00053

Transformers without Tears: Improving the Normalization of Self-Attention

Toan Q. Nguyen, Julian Salazar

1910.05895

Open Category Detection with PAC Guarantees

Si Liu, Risheek Garrepalli, Thomas G. Dietterich, Alan Fern, Dan Hendrycks

1808.00529

A Theoretical Analysis of Deep Neural Networks and Parametric PDEs

Gitta Kutyniok, Philipp Petersen, Mones Raslan, Reinhold Schneider

1904.00377

Loss landscapes and optimization in over-parameterized non-linear systems and neural networks

Chaoyue Liu, Libin Zhu, Mikhail Belkin

2003.00307

Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark

Cody Coleman, Daniel Kang, Deepak Narayanan, Luigi Nardi, Tian Zhao, Jian Zhang, Peter Bailis, Kunle Olukotun, Chris Re, Matei Zaharia

1806.01427

Orthogonal Gradient Descent for Continual Learning

Mehrdad Farajtabar, Navid Azizan, Alex Mott, Ang Li

1910.07104

Review of Deep Learning

Rong Zhang, Weiping Li, Tong Mo

1804.01653

Non-Stationary Spectral Kernels

Sami Remes, Markus Heinonen, Samuel Kaski

1705.08736

Implicit Quantile Networks for Distributional Reinforcement Learning

Will Dabney, Georg Ostrovski, David Silver, Rémi Munos

1806.06923

Adversarial examples for generative models

Jernej Kos, Ian Fischer, Dawn Song

1702.06832

The Fairness-Accuracy Pareto Front

Susan Wei, Marc Niethammer

2008.10797

Stochastic Backpropagation and Approximate Inference in Deep Generative Models

Danilo Jimenez Rezende, Shakir Mohamed, Daan Wierstra

1401.4082

Outlier-robust moment-estimation via sum-of-squares

Pravesh K. Kothari, David Steurer

1711.11581

Approximating Continuous Functions by ReLU Nets of Minimal Width

Boris Hanin, Mark Sellke

1710.11278

Gated Feedback Recurrent Neural Networks

Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, Yoshua Bengio

1502.02367

A statistical model for tensor PCA

Andrea Montanari, Emile Richard

1411.1076

Robust Spectral Compressed Sensing via Structured Matrix Completion

Yuxin Chen, Yuejie Chi

1304.8126

BYOL works even without batch statistics

Pierre H. Richemond, Jean-Bastien Grill, Florent Altché, Corentin Tallec, Florian Strub, Andrew Brock, Samuel Smith, Soham De, Razvan Pascanu, Bilal Piot, Michal Valko

2010.10241

Early Visual Concept Learning with Unsupervised Deep Learning

Irina Higgins, Loic Matthey, Xavier Glorot, Arka Pal, Benigno Uria, Charles Blundell, Shakir Mohamed, Alexander Lerchner

1606.05579

Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression

Tim Salimans, David A. Knowles

1206.6679

Streaming PCA: Matching Matrix Bernstein and Near-Optimal Finite Sample Guarantees for Oja's Algorithm

Prateek Jain, Chi Jin, Sham M. Kakade, Praneeth Netrapalli, Aaron Sidford

1602.06929

Variational Sequential Monte Carlo

Christian A. Naesseth, Scott W. Linderman, Rajesh Ranganath, David M. Blei

1705.11140

Adversarial examples from computational constraints

Sébastien Bubeck, Eric Price, Ilya Razenshteyn

1805.10204

On the Limitations of Representing Functions on Sets

Edward Wagstaff, Fabian B. Fuchs, Martin Engelcke, Ingmar Posner, Michael Osborne

1901.09006

Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Method

Ju Sun, Qing Qu, John Wright

1511.04777

Large-Margin Softmax Loss for Convolutional Neural Networks

Weiyang Liu, Yandong Wen, Zhiding Yu, Meng Yang

1612.02295

Unity: A General Platform for Intelligent Agents

Arthur Juliani, Vincent-Pierre Berges, Ervin Teng, Andrew Cohen, Jonathan Harper, Chris Elion, Chris Goy, Yuan Gao, Hunter Henry, Marwan Mattar, Danny Lange

1809.02627

Fluctuation-dissipation relations for stochastic gradient descent

Sho Yaida

1810.00004

TRAK: Attributing Model Behavior at Scale

Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, Aleksander Madry

2303.14186

Measuring the Effects of Data Parallelism on Neural Network Training

Christopher J. Shallue, Jaehoon Lee, Joseph Antognini, Jascha Sohl-Dickstein, Roy Frostig, George E. Dahl

1811.03600

A mathematical theory of semantic development in deep neural networks

Andrew M. Saxe, James L. McClelland, Surya Ganguli

1810.10531

Implicit Regularization for Optimal Sparse Recovery

Tomas Vaškevičius, Varun Kanade, Patrick Rebeschini

1909.05122

Scaling up Differentially Private Deep Learning with Fast Per-Example Gradient Clipping

Jaewoo Lee, Daniel Kifer

2009.03106

On Quadratic Penalties in Elastic Weight Consolidation

Ferenc Huszár

1712.03847

When Explanations Lie: Why Many Modified BP Attributions Fail

Leon Sixt, Maximilian Granz, Tim Landgraf

1912.09818

A Flexible Generative Framework for Graph-based Semi-supervised Learning

Jiaqi Ma, Weijing Tang, Ji Zhu, Qiaozhu Mei

1905.10769

Junction Tree Variational Autoencoder for Molecular Graph Generation

Wengong Jin, Regina Barzilay, Tommi Jaakkola

1802.04364

Assessing Social and Intersectional Biases in Contextualized Word Representations

Yi Chern Tan, L. Elisa Celis

1911.01485

Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet

Wieland Brendel, Matthias Bethge

1904.00760

The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance

Matthew Faw, Isidoros Tziotis, Constantine Caramanis, Aryan Mokhtari, Sanjay Shakkottai, Rachel Ward

2202.05791

Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis

Guangzhi Sun, Yu Zhang, Ron J. Weiss, Yuan Cao, Heiga Zen, Yonghui Wu

2002.03785

A New Perspective on Shampoo's Preconditioner

Depen Morwani, Itai Shapira, Nikhil Vyas, Eran Malach, Sham Kakade, Lucas Janson

2406.17748

Learning the Structure of Deep Sparse Graphical Models

Ryan Prescott Adams, Hanna M. Wallach, Zoubin Ghahramani

1001.0160

The State of Sparsity in Deep Neural Networks

Trevor Gale, Erich Elsen, Sara Hooker

1902.09574

Infinite attention: NNGP and NTK for deep attention networks

Jiri Hron, Yasaman Bahri, Jascha Sohl-Dickstein, Roman Novak

2006.10540

Structured Transforms for Small-Footprint Deep Learning

Vikas Sindhwani, Tara N. Sainath, Sanjiv Kumar

1510.01722

BilBOWA: Fast Bilingual Distributed Representations without Word Alignments

Stephan Gouws, Yoshua Bengio, Greg Corrado

1410.2455

Variational Bayesian Inference with Stochastic Search

John Paisley, David Blei, Michael Jordan

1206.6430

On Mutual Information Maximization for Representation Learning

Michael Tschannen, Josip Djolonga, Paul K. Rubenstein, Sylvain Gelly, Mario Lucic

1907.13625

Firefly Monte Carlo: Exact MCMC with Subsets of Data

Dougal Maclaurin, Ryan P. Adams

1403.5693

A Unified Lottery Ticket Hypothesis for Graph Neural Networks

Tianlong Chen, Yongduo Sui, Xuxi Chen, Aston Zhang, Zhangyang Wang

2102.06790

Neural Network Matrix Factorization

Gintare Karolina Dziugaite, Daniel M. Roy

1511.06443

Improving Palliative Care with Deep Learning

Anand Avati, Kenneth Jung, Stephanie Harman, Lance Downing, Andrew Ng, Nigam H. Shah

1711.06402

Diffusion-GAN: Training GANs with Diffusion

Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, Mingyuan Zhou

2206.02262

Online Learning with Predictable Sequences

Alexander Rakhlin, Karthik Sridharan

1208.3728

Fairness in the Eyes of the Data: Certifying Machine-Learning Models

Shahar Segal, Yossi Adi, Benny Pinkas, Carsten Baum, Chaya Ganesh, Joseph Keshet

2009.01534

EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis

Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch

1612.07919

Iterative Hessian sketch: Fast and accurate solution approximation for constrained least-squares

Mert Pilanci, Martin J. Wainwright

1411.0347

Autoencoding Variational Inference For Topic Models

Akash Srivastava, Charles Sutton

1703.01488

Random sampling of bandlimited signals on graphs

Gilles Puy, Nicolas Tremblay, Rémi Gribonval, Pierre Vandergheynst

1511.05118

Similarity of Neural Network Representations Revisited

Simon Kornblith, Mohammad Norouzi, Honglak Lee, Geoffrey Hinton

1905.00414

Concrete Dropout

Yarin Gal, Jiri Hron, Alex Kendall

1705.07832

Knowledge Graph Completion via Complex Tensor Factorization

Théo Trouillon, Christopher R. Dance, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard

1702.06879

Modeling Human Decision-making in Generalized Gaussian Multi-armed Bandits

Paul Reverdy, Vaibhav Srivastava, Naomi E. Leonard

1307.6134

Clustering via Mode Seeking by Direct Estimation of the Gradient of a Log-Density

Hiroaki Sasaki, Aapo Hyvärinen, Masashi Sugiyama

1404.5028

From neural PCA to deep unsupervised learning

Harri Valpola

1411.7783

Character-based Neural Machine Translation

Marta R. Costa-Jussà, José A. R. Fonollosa

1603.00810

Stochastic Recursive Gradient Descent Ascent for Stochastic Nonconvex-Strongly-Concave Minimax Problems

Luo Luo, Haishan Ye, Zhichao Huang, Tong Zhang

2001.03724

TherML: Thermodynamics of Machine Learning

Alexander A. Alemi, Ian Fischer

1807.04162

BackPACK: Packing more into backprop

Felix Dangel, Frederik Kunstner, Philipp Hennig

1912.10985

Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels

Eran Malach, Pritish Kamath, Emmanuel Abbe, Nathan Srebro

2103.01210

Global Convergence of Online Limited Memory BFGS

Aryan Mokhtari, Alejandro Ribeiro

1409.2045

The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization

Mufan Bill Li, Mihai Nica, Daniel M. Roy

2106.04013

Machine Teaching: A New Paradigm for Building Machine Learning Systems

Patrice Y. Simard, Saleema Amershi, David M. Chickering, Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, John Wernsing

1707.06742

Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning

Zhendong Wang, Jonathan J Hunt, Mingyuan Zhou

2208.06193

Vector Quantized Models for Planning

Sherjil Ozair, Yazhe Li, Ali Razavi, Ioannis Antonoglou, Aäron van den Oord, Oriol Vinyals

2106.04615

Intriguing Properties of Contrastive Losses

Ting Chen, Calvin Luo, Lala Li

2011.02803

Automatic Differentiation Variational Inference

Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, David M. Blei

1603.00788

Phase Retrieval Meets Statistical Learning Theory: A Flexible Convex Relaxation

Sohail Bahmani, Justin Romberg

1610.04210

Agnostic Federated Learning

Mehryar Mohri, Gary Sivek, Ananda Theertha Suresh

1902.00146

Gradient Sparsification for Communication-Efficient Distributed Optimization

Jianqiao Wangni, Jialei Wang, Ji Liu, Tong Zhang

1710.09854

An Exponential Learning Rate Schedule for Deep Learning

Zhiyuan Li, Sanjeev Arora

1910.07454

Measuring the Intrinsic Dimension of Objective Landscapes

Chunyuan Li, Heerad Farkhoor, Rosanne Liu, Jason Yosinski

1804.08838

Learning Sparse Nonparametric DAGs

Xun Zheng, Chen Dan, Bryon Aragam, Pradeep Ravikumar, Eric P. Xing

1909.13189

Bayesian Deep Ensembles via the Neural Tangent Kernel

Bobby He, Balaji Lakshminarayanan, Yee Whye Teh

2007.05864

Iterate averaging as regularization for stochastic gradient descent

Gergely Neu, Lorenzo Rosasco

1802.08009

Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems

Yuxin Chen, Emmanuel J. Candes

1505.05114

Less is More: Nyström Computational Regularization

Alessandro Rudi, Raffaello Camoriano, Lorenzo Rosasco

1507.04717

ProjE: Embedding Projection for Knowledge Graph Completion

Baoxu Shi, Tim Weninger

1611.05425

Score matching enables causal discovery of nonlinear additive noise models

Paul Rolland, Volkan Cevher, Matthäus Kleindessner, Chris Russel, Bernhard Schölkopf, Dominik Janzing, Francesco Locatello

2203.04413

Learning Neural PDE Solvers with Convergence Guarantees

Jun-Ting Hsieh, Shengjia Zhao, Stephan Eismann, Lucia Mirabella, Stefano Ermon

1906.01200

Towards Neural Machine Translation for African Languages

Jade Z. Abbott, Laura Martinus

1811.05467

Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness

Raphael Suter, Đorđe Miladinović, Bernhard Schölkopf, Stefan Bauer

1811.00007

Corrupted Sensing: Novel Guarantees for Separating Structured Signals

Rina Foygel, Lester Mackey

1305.2524

Can Adversarial Weight Perturbations Inject Neural Backdoors?

Siddhant Garg, Adarsh Kumar, Vibhor Goel, Yingyu Liang

2008.01761

Why does deep and cheap learning work so well?

Henry W. Lin, Max Tegmark, David Rolnick

1608.08225

Differentiable Game Mechanics

Alistair Letcher, David Balduzzi, Sebastien Racaniere, James Martens, Jakob Foerster, Karl Tuyls, Thore Graepel

1905.04926

Hinge-loss Markov Random Fields: Convex Inference for Structured Prediction

Stephen Bach, Bert Huang, Ben London, Lise Getoor

1309.6813

Fair Inference On Outcomes

Razieh Nabi, Ilya Shpitser

1705.10378

On the Power and Limitations of Random Features for Understanding Neural Networks

Gilad Yehudai, Ohad Shamir

1904.00687

Regularization, sparse recovery, and median-of-means tournaments

Gábor Lugosi, Shahar Mendelson

1701.04112

Sentence-State LSTM for Text Representation

Yue Zhang, Qi Liu, Linfeng Song

1805.02474

InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization

Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, Jian Tang

1908.01000

LazySVD: Even Faster SVD Decomposition Yet Without Agonizing Pain

Zeyuan Allen-Zhu, Yuanzhi Li

1607.03463

Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Control

Glen Berseth, Cheng Xie, Paul Cernek, Michiel Van de Panne

1802.04765

Deep Temporal Sigmoid Belief Networks for Sequence Modeling

Zhe Gan, Chunyuan Li, Ricardo Henao, David Carlson, Lawrence Carin

1509.07087

Pathwise Derivatives Beyond the Reparameterization Trick

Martin Jankowiak, Fritz Obermeyer

1806.01851

Dataset Meta-Learning from Kernel Ridge-Regression

Timothy Nguyen, Zhourong Chen, Jaehoon Lee

2011.00050

Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization

Xiangru Lian, Yijun Huang, Yuncheng Li, Ji Liu

1506.08272

Universal Differential Equations for Scientific Machine Learning

Christopher Rackauckas, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, Ali Ramadhan, Alan Edelman

2001.04385

A Universal Algorithm for Variational Inequalities Adaptive to Smoothness and Noise

Francis Bach, Kfir Y. Levy

1902.01637

A guide to convolution arithmetic for deep learning

Vincent Dumoulin, Francesco Visin

1603.07285

Stabilizing DARTS with Amended Gradient Estimation on Architectural Parameters

Kaifeng Bi, Changping Hu, Lingxi Xie, Xin Chen, Longhui Wei, Qi Tian

1910.11831

An empirical study on evaluation metrics of generative adversarial networks

Qiantong Xu, Gao Huang, Yang Yuan, Chuan Guo, Yu Sun, Felix Wu, Kilian Weinberger

1806.07755

Structure Discovery in Nonparametric Regression through Compositional Kernel Search

David Duvenaud, James Robert Lloyd, Roger Grosse, Joshua B. Tenenbaum, Zoubin Ghahramani

1302.4922

Nonparametric Variational Auto-encoders for Hierarchical Representation Learning

Prasoon Goyal, Zhiting Hu, Xiaodan Liang, Chenyu Wang, Eric Xing

1703.07027

Gradient Descent Maximizes the Margin of Homogeneous Neural Networks

Kaifeng Lyu, Jian Li

1906.05890

Solving Rubik's Cube with a Robot Hand

OpenAI, Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, Jonas Schneider, Nikolas Tezak, Jerry Tworek, Peter Welinder, Lilian Weng, Qiming Yuan, Wojciech Zaremba, Lei Zhang

1910.07113

TensorFlow Distributions

Joshua V. Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Srinivas Vasudevan, Dave Moore, Brian Patton, Alex Alemi, Matt Hoffman, Rif A. Saurous

1711.10604

Memory-Efficient Pipeline-Parallel DNN Training

Deepak Narayanan, Amar Phanishayee, Kaiyu Shi, Xie Chen, Matei Zaharia

2006.09503

Tropical Geometry of Deep Neural Networks

Liwen Zhang, Gregory Naitzat, Lek-Heng Lim

1805.07091

A Variational Perspective on Accelerated Methods in Optimization

Andre Wibisono, Ashia C. Wilson, Michael I. Jordan

1603.04245

Molecular Graph Convolutions: Moving Beyond Fingerprints

Steven Kearnes, Kevin McCloskey, Marc Berndl, Vijay Pande, Patrick Riley

1603.00856

Pyro: Deep Universal Probabilistic Programming

Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, Noah D. Goodman

1810.09538

Towards Accurate Binary Convolutional Neural Network

Xiaofan Lin, Cong Zhao, Wei Pan

1711.11294

Efficient Algorithms and Lower Bounds for Robust Linear Regression

Ilias Diakonikolas, Weihao Kong, Alistair Stewart

1806.00040

Deep Fried Convnets

Zichao Yang, Marcin Moczulski, Misha Denil, Nando de Freitas, Alex Smola, Le Song, Ziyu Wang

1412.7149

Deep Transfer Reinforcement Learning for Text Summarization

Yaser Keneshloo, Naren Ramakrishnan, Chandan K. Reddy

1810.06667

Learning with Differentiable Perturbed Optimizers

Quentin Berthet, Mathieu Blondel, Olivier Teboul, Marco Cuturi, Jean-Philippe Vert, Francis Bach

2002.08676

Quantifying Generalization in Reinforcement Learning

Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, John Schulman

1812.02341

RL with KL penalties is better viewed as Bayesian inference

Tomasz Korbak, Ethan Perez, Christopher L Buckley

2205.11275

Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect

Xiang Wei, Boqing Gong, Zixia Liu, Wei Lu, Liqiang Wang

1803.01541

Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog

Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, Rosalind Picard

1907.00456

Ray Interference: a Source of Plateaus in Deep Reinforcement Learning

Tom Schaul, Diana Borsa, Joseph Modayil, Razvan Pascanu

1904.11455

Human Alignment of Large Language Models through Online Preference Optimisation

Daniele Calandriello, Daniel Guo, Remi Munos, Mark Rowland, Yunhao Tang, Bernardo Avila Pires, Pierre Harvey Richemond, Charline Le Lan, Michal Valko, Tianqi Liu, Rishabh Joshi, Zeyu Zheng, Bilal Piot

2403.08635

Learning Controllable Fair Representations

Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, Stefano Ermon

1812.04218

Multimodal Deep Learning

Cem Akkus, Luyang Chu, Vladana Djakovic, Steffen Jauch-Walser, Philipp Koch, Giacomo Loss, Christopher Marquardt, Marco Moldovan, Nadja Sauter, Maximilian Schneider, Rickmer Schulte, Karol Urbanczyk, Jann Goschenhofer, Christian Heumann, Rasmus Hvingelby, Daniel Schalk, Matthias Aßenmacher

2301.04856

Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regression

Theodor Misiakiewicz

2204.10425

Adversarial Training Can Hurt Generalization

Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C. Duchi, Percy Liang

1906.06032

Formal Limitations on the Measurement of Mutual Information

David McAllester, Karl Stratos

1811.04251

Kernel Bayes' rule

Kenji Fukumizu, Le Song, Arthur Gretton

1009.5736

Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks

Neil Band, Tim G. J. Rudner, Qixuan Feng, Angelos Filos, Zachary Nado, Michael W. Dusenberry, Ghassen Jerfel, Dustin Tran, Yarin Gal

2211.12717

A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction

Thomas Wiatowski, Helmut Bölcskei

1512.06293

Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning

Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, Andrew Gordon Wilson

1902.03932

Stationary signal processing on graphs

Nathanaël Perraudin, Pierre Vandergheynst

1601.02522

Invertibility and Robustness of Phaseless Reconstruction

Radu Balan, Yang Wang

1308.4718

Straggler Mitigation in Distributed Optimization Through Data Encoding

Can Karakus, Yifan Sun, Suhas Diggavi, Wotao Yin

1711.04969

From ImageNet to Image Classification: Contextualizing Progress on Benchmarks

Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Andrew Ilyas, Aleksander Madry

2005.11295

Distilled One-Shot Federated Learning

Yanlin Zhou, George Pu, Xiyao Ma, Xiaolin Li, Dapeng Wu

2009.07999

Model Selection in Bayesian Neural Networks via Horseshoe Priors

Soumya Ghosh, Finale Doshi-Velez

1705.10388

Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions

Carl-Johann Simon-Gabriel, Bernhard Schölkopf

1604.05251

Meta-learners' learning dynamics are unlike learners'

Neil C. Rabinowitz

1905.01320

Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices

Ayan Acharya, Joydeep Ghosh, Mingyuan Zhou

1512.08996

Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations

Dan Hendrycks, Thomas G. Dietterich

1807.01697

Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality

Song Mei, Theodor Misiakiewicz, Andrea Montanari, Roberto I. Oliveira

1703.08729

Deep Reinforcement and InfoMax Learning

Bogdan Mazoure, Remi Tachet des Combes, Thang Doan, Philip Bachman, R Devon Hjelm

2006.07217

On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime

Stephan Wojtowytsch

2005.13530

Sparse Optimization on Measures with Over-parameterized Gradient Descent

Lenaic Chizat

1907.10300

Universality of empirical risk minimization

Andrea Montanari, Basil Saeed

2202.08832

Implicit Regularization in Matrix Factorization

Suriya Gunasekar, Blake Woodworth, Srinadh Bhojanapalli, Behnam Neyshabur, Nathan Srebro

1705.09280

A comparison of recent waveform generation and acoustic modeling methods for neural-network-based speech synthesis

Xin Wang, Jaime Lorenzo-Trueba, Shinji Takaki, Lauri Juvela, Junichi Yamagishi

1804.02549

Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

Qiang Liu, Dilin Wang

1608.04471

Soft Weight-Sharing for Neural Network Compression

Karen Ullrich, Edward Meeds, Max Welling

1702.04008

Data-Free Quantization Through Weight Equalization and Bias Correction

Markus Nagel, Mart van Baalen, Tijmen Blankevoort, Max Welling

1906.04721

Toward Communication Efficient Adaptive Gradient Method

Xiangyi Chen, Xiaoyun Li, Ping Li

2109.05109

On Evaluating Adversarial Robustness

Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, Alexey Kurakin

1902.06705

OmniMAE: Single Model Masked Pretraining on Images and Videos

Rohit Girdhar, Alaaeldin El-Nouby, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, Ishan Misra

2206.08356

Accelerating ABC methods using Gaussian processes

Richard D Wilkinson

1401.1436

Independently Controllable Factors

Valentin Thomas, Jules Pondard, Emmanuel Bengio, Marc Sarfati, Philippe Beaudoin, Marie-Jean Meurs, Joelle Pineau, Doina Precup, Yoshua Bengio

1708.01289

Combinatorial clustering and the beta negative binomial process

Tamara Broderick, Lester Mackey, John Paisley, Michael I. Jordan

1111.1802

On the Expressive Efficiency of Sum Product Networks

James Martens, Venkatesh Medabalimi

1411.7717

Tractable Function-Space Variational Inference in Bayesian Neural Networks

Tim G. J. Rudner, Zonghao Chen, Yee Whye Teh, Yarin Gal

2312.17199

Error Feedback Fixes SignSGD and other Gradient Compression Schemes

Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian U. Stich, Martin Jaggi

1901.09847

Reducing Noise in GAN Training with Variance Reduced Extragradient

Tatjana Chavdarova, Gauthier Gidel, François Fleuret, Simon Lacoste-Julien

1904.08598

Minimax sparse principal subspace estimation in high dimensions

Vincent Q. Vu, Jing Lei

1211.0373

Escaping Saddles with Stochastic Gradients

Hadi Daneshmand, Jonas Kohler, Aurelien Lucchi, Thomas Hofmann

1803.05999

Gaussian Mixture Generative Adversarial Networks for Diverse Datasets, and the Unsupervised Clustering of Images

Matan Ben-Yosef, Daphna Weinshall

1808.10356

Learning Nonlinear Functions Using Regularized Greedy Forest

Rie Johnson, Tong Zhang

1109.0887

Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs

Cristóbal Esteban, Stephanie L. Hyland, Gunnar Rätsch

1706.02633

Asymptotics of representation learning in finite Bayesian neural networks

Jacob A. Zavatone-Veth, Abdulkadir Canatar, Benjamin S. Ruben, Cengiz Pehlevan

2106.00651

Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications

Songtao Lu, Ioannis Tsaknakis, Mingyi Hong, Yongxin Chen

1902.08294

Adaptive Federated Optimization

Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, H. Brendan McMahan

2003.00295

Bibliographic Analysis on Research Publications using Authors, Categorical Labels and the Citation Network

Kar Wai Lim, Wray Buntine

1609.06532

A Simple Baseline for Bayesian Uncertainty in Deep Learning

Wesley Maddox, Timur Garipov, Pavel Izmailov, Dmitry Vetrov, Andrew Gordon Wilson

1902.02476

On the Complexity Analysis of Randomized Block-Coordinate Descent Methods

Zhaosong Lu, Lin Xiao

1305.4723

Understanding training and generalization in deep learning by Fourier analysis

Zhiqin John Xu

1808.04295

A new regret analysis for Adam-type algorithms

Ahmet Alacaoglu, Yura Malitsky, Panayotis Mertikopoulos, Volkan Cevher

2003.09729

On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems

Tianyi Lin, Chi Jin, Michael I. Jordan

1906.00331

Simplifying Graph Convolutional Networks

Felix Wu, Tianyi Zhang, Amauri Holanda de Souza, Christopher Fifty, Tao Yu, Kilian Q. Weinberger

1902.07153

Import2vec - Learning Embeddings for Software Libraries

Bart Theeten, Frederik Vandeputte, Tom Van Cutsem

1904.03990

Predictive Entropy Search for Efficient Global Optimization of Black-box Functions

José Miguel Hernández-Lobato, Matthew W. Hoffman, Zoubin Ghahramani

1406.2541

Real Time Speech Enhancement in the Waveform Domain

Alexandre Defossez, Gabriel Synnaeve, Yossi Adi

2006.12847

Multiple Source Adaptation and the Renyi Divergence

Yishay Mansour, Mehryar Mohri, Afshin Rostamizadeh

1205.2628

Depth-Width Trade-offs for ReLU Networks via Sharkovsky's Theorem

Vaggos Chatziafratis, Sai Ganesh Nagarajan, Ioannis Panageas, Xiao Wang

1912.04378

Understanding Deep Convolutional Networks

Stéphane Mallat

1601.04920

Improving Variational Inference with Inverse Autoregressive Flow

Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, Max Welling

1606.04934

Dynamic Poisson Factorization

Laurent Charlin, Rajesh Ranganath, James McInerney, David M. Blei

1509.04640

Deconvolutional Paragraph Representation Learning

Yizhe Zhang, Dinghan Shen, Guoyin Wang, Zhe Gan, Ricardo Henao, Lawrence Carin

1708.04729

Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT

Antti Koskela, Joonas Jälkö, Lukas Prediger, Antti Honkela

2006.07134

A Variational Inequality Perspective on Generative Adversarial Networks

Gauthier Gidel, Hugo Berard, Gaëtan Vignoud, Pascal Vincent, Simon Lacoste-Julien

1802.10551

Harmless interpolation in regression and classification with structured features

Andrew D. McRae, Santhosh Karnik, Mark A. Davenport, Vidya Muthukumar

2111.05198

Expectation-Maximization for Learning Determinantal Point Processes

Jennifer Gillenwater, Alex Kulesza, Emily Fox, Ben Taskar

1411.1088

Linear dynamical neural population models through nonlinear embeddings

Yuanjun Gao, Evan Archer, Liam Paninski, John P. Cunningham

1605.08454

AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients

Juntang Zhuang, Tommy Tang, Yifan Ding, Sekhar Tatikonda, Nicha Dvornek, Xenophon Papademetris, James S. Duncan

2010.07468

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lillicrap, David Silver

1911.08265

Bayesian Nonparametric Causal Inference: Information Rates and Learning Algorithms

Ahmed M. Alaa, Mihaela van der Schaar

1712.08914

Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations

Diane Bouchacourt, Ryota Tomioka, Sebastian Nowozin

1705.08841

Learning multiple visual domains with residual adapters

Sylvestre-Alvise Rebuffi, Hakan Bilen, Andrea Vedaldi

1705.08045

Information-Theoretic Bounded Rationality

Pedro A. Ortega, Daniel A. Braun, Justin Dyer, Kee-Eung Kim, Naftali Tishby

1512.06789

Achieving Exact Cluster Recovery Threshold via Semidefinite Programming

Bruce Hajek, Yihong Wu, Jiaming Xu

1412.6156

The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization

Mufan Bill Li, Mihai Nica, Daniel M. Roy

2206.02768

Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems

Weijie Zhao, Deping Xie, Ronglai Jia, Yulei Qian, Ruiquan Ding, Mingming Sun, Ping Li

2003.05622

Robust Estimation via Robust Gradient Estimation

Adarsh Prasad, Arun Sai Suggala, Sivaraman Balakrishnan, Pradeep Ravikumar

1802.06485

A Call for More Rigor in Unsupervised Cross-lingual Learning

Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, Eneko Agirre

2004.14958

HiPPO: Recurrent Memory with Optimal Polynomial Projections

Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, Christopher Re

2008.07669

Stronger Data Poisoning Attacks Break Data Sanitization Defenses

Pang Wei Koh, Jacob Steinhardt, Percy Liang

1811.00741

Symbolic Music Generation with Diffusion Models

Gautam Mittal, Jesse Engel, Curtis Hawthorne, Ian Simon

2103.16091

Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data

Marc Finzi, Samuel Stanton, Pavel Izmailov, Andrew Gordon Wilson

2002.12880

What Are Bayesian Neural Network Posteriors Really Like?

Pavel Izmailov, Sharad Vikram, Matthew D. Hoffman, Andrew Gordon Wilson

2104.14421

Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

William Fedus, Mihaela Rosca, Balaji Lakshminarayanan, Andrew M. Dai, Shakir Mohamed, Ian Goodfellow

1710.08446

Safety Verification of Deep Neural Networks

Xiaowei Huang, Marta Kwiatkowska, Sen Wang, Min Wu

1610.06940

Classification Accuracy Score for Conditional Generative Models

Suman Ravuri, Oriol Vinyals

1905.10887

A General Approach to Adding Differential Privacy to Iterative Training Procedures

H. Brendan McMahan, Galen Andrew, Ulfar Erlingsson, Steve Chien, Ilya Mironov, Nicolas Papernot, Peter Kairouz

1812.06210

Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling

Valentin De Bortoli, James Thornton, Jeremy Heng, Arnaud Doucet

2106.01357

Federated Learning with Non-IID Data

Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, Vikas Chandra

1806.00582

GPT-GNN: Generative Pre-Training of Graph Neural Networks

Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, Yizhou Sun

2006.15437

Wasserstein Fair Classification

Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, Silvia Chiappa

1907.12059

Sequential Neural Models with Stochastic Layers

Marco Fraccaro, Søren Kaae Sønderby, Ulrich Paquet, Ole Winther

1605.07571

TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency

Adji B. Dieng, Chong Wang, Jianfeng Gao, John Paisley

1611.01702

A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case

Greg Ongie, Rebecca Willett, Daniel Soudry, Nathan Srebro

1910.01635

Fine-Tuning Language Models from Human Preferences

Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, Geoffrey Irving

1909.08593

A Distributional Perspective on Reinforcement Learning

Marc G. Bellemare, Will Dabney, Rémi Munos

1707.06887

Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information

Peng Xu, Fred Roosta, Michael W. Mahoney

1708.07164

Visual Reinforcement Learning with Imagined Goals

Ashvin Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, Sergey Levine

1807.04742

Link Prediction Based on Graph Neural Networks

Muhan Zhang, Yixin Chen

1802.09691

TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets

Chengrun Yang, Gabriel Bender, Hanxiao Liu, Pieter-Jan Kindermans, Madeleine Udell, Yifeng Lu, Quoc Le, Da Huang

2204.07615

ATOMO: Communication-efficient Learning via Atomic Sparsification

Hongyi Wang, Scott Sievert, Zachary Charles, Shengchao Liu, Stephen Wright, Dimitris Papailiopoulos

1806.04090

Robustly Learning a Gaussian: Getting Optimal Error, Efficiently

Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jerry Li, Ankur Moitra, Alistair Stewart

1704.03866

Gated Orthogonal Recurrent Units: On Learning to Forget

Li Jing, Caglar Gulcehre, John Peurifoy, Yichen Shen, Max Tegmark, Marin Soljačić, Yoshua Bengio

1706.02761

Likelihood-free inference via classification

Michael U. Gutmann, Ritabrata Dutta, Samuel Kaski, Jukka Corander

1407.4981

Auto-Keras: An Efficient Neural Architecture Search System

Haifeng Jin, Qingquan Song, Xia Hu

1806.10282

Bayesian Compression for Deep Learning

Christos Louizos, Karen Ullrich, Max Welling

1705.08665

Playing hard exploration games by watching YouTube

Yusuf Aytar, Tobias Pfaff, David Budden, Tom Le Paine, Ziyu Wang, Nando de Freitas

1805.11592

Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates

Jeffrey Negrea, Mahdi Haghifam, Gintare Karolina Dziugaite, Ashish Khisti, Daniel M. Roy

1911.02151

Adversarially Robust Generalization Just Requires More Unlabeled Data

Runtian Zhai, Tianle Cai, Di He, Chen Dan, Kun He, John Hopcroft, Liwei Wang

1906.00555

Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition

Vincent Dorie, Jennifer Hill, Uri Shalit, Marc Scott, Dan Cervone

1707.02641

ClusterCluster: Parallel Markov Chain Monte Carlo for Dirichlet Process Mixtures

Dan Lovell, Jonathan Malmaud, Ryan P. Adams, Vikash K. Mansinghka

1304.2302

Emergent Tool Use From Multi-Agent Autocurricula

Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, Igor Mordatch

1909.07528

On Iterative Hard Thresholding Methods for High-dimensional M-Estimation

Prateek Jain, Ambuj Tewari, Purushottam Kar

1410.5137

Bayesian Recurrent Neural Networks

Meire Fortunato, Charles Blundell, Oriol Vinyals

1704.02798

Learning model-based planning from scratch

Razvan Pascanu, Yujia Li, Oriol Vinyals, Nicolas Heess, Lars Buesing, Sebastien Racanière, David Reichert, Théophane Weber, Daan Wierstra, Peter Battaglia

1707.06170

Targeted free energy estimation via learned mappings

Peter Wirnsberger, Andrew J. Ballard, George Papamakarios, Stuart Abercrombie, Sébastien Racanière, Alexander Pritzel, Danilo Jimenez Rezende, Charles Blundell

2002.04913

Operator Variational Inference

Rajesh Ranganath, Jaan Altosaar, Dustin Tran, David M. Blei

1610.09033

Does Neural Machine Translation Benefit from Larger Context?

Sebastien Jean, Stanislas Lauly, Orhan Firat, Kyunghyun Cho

1704.05135

Regularization by architecture: A deep prior approach for inverse problems

Sören Dittmer, Tobias Kluth, Peter Maass, Daniel Otero Baguer

1812.03889

A Supervised Approach to Extractive Summarisation of Scientific Papers

Ed Collins, Isabelle Augenstein, Sebastian Riedel

1706.03946

Do semidefinite relaxations solve sparse PCA up to the information limit?

Robert Krauthgamer, Boaz Nadler, Dan Vilenchik

1306.3690

Learning Video Representations using Contrastive Bidirectional Transformer

Chen Sun, Fabien Baradel, Kevin Murphy, Cordelia Schmid

1906.05743

Random Walk Initialization for Training Very Deep Feedforward Networks

David Sussillo, L. F. Abbott

1412.6558

Follow the Compressed Leader: Faster Online Learning of Eigenvectors and Faster MMWU

Zeyuan Allen-Zhu, Yuanzhi Li

1701.01722

Question Answering by Reasoning Across Documents with Graph Convolutional Networks

Nicola De Cao, Wilker Aziz, Ivan Titov

1808.09920

On Empirical Comparisons of Optimizers for Deep Learning

Dami Choi, Christopher J. Shallue, Zachary Nado, Jaehoon Lee, Chris J. Maddison, George E. Dahl

1910.05446

Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization

Changyou Chen, David Carlson, Zhe Gan, Chunyuan Li, Lawrence Carin

1512.07962

Distributed Matrix Completion and Robust Factorization

Lester Mackey, Ameet Talwalkar, Michael I. Jordan

1107.0789

Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation

Junliang Guo, Xu Tan, Linli Xu, Tao Qin, Enhong Chen, Tie-Yan Liu

1911.08717

Explorations on high dimensional landscapes

Levent Sagun, V. Ugur Guney, Gerard Ben Arous, Yann LeCun

1412.6615

Tight Analyses for Non-Smooth Stochastic Gradient Descent

Nicholas J. A. Harvey, Christopher Liaw, Yaniv Plan, Sikander Randhawa

1812.05217

A Generic Acceleration Framework for Stochastic Composite Optimization

Andrei Kulunchakov, Julien Mairal

1906.01164

Density estimation using Real NVP

Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio

1605.08803

An Empirical Study of Rich Subgroup Fairness for Machine Learning

Michael Kearns, Seth Neel, Aaron Roth, Zhiwei Steven Wu

1808.08166

On the Robustness of Interpretability Methods

David Alvarez-Melis, Tommi S. Jaakkola

1806.08049

A Simple Practical Accelerated Method for Finite Sums

Aaron Defazio

1602.02442

Professor Forcing: A New Algorithm for Training Recurrent Networks

Alex Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron Courville, Yoshua Bengio

1610.09038

On the Generalization Benefit of Noise in Stochastic Gradient Descent

Samuel L. Smith, Erich Elsen, Soham De

2006.15081

Dimension free ridge regression

Chen Cheng, Andrea Montanari

2210.08571

Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis

Jikai Jin, Bohang Zhang, Haiyang Wang, Liwei Wang

2110.12459

Mastering Atari with Discrete World Models

Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, Jimmy Ba

2010.02193

The Implicit Bias of Depth: How Incremental Learning Drives Generalization

Daniel Gissin, Shai Shalev-Shwartz, Amit Daniely

1909.12051

Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile

Panayotis Mertikopoulos, Bruno Lecouat, Houssam Zenati, Chuan-Sheng Foo, Vijay Chandrasekhar, Georgios Piliouras

1807.02629

GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation

Edward Meeds, Max Welling

1401.2838

SGAS: Sequential Greedy Architecture Search

Guohao Li, Guocheng Qian, Itzel C. Delgadillo, Matthias Müller, Ali Thabet, Bernard Ghanem

1912.00195

A Dual Approach to Scalable Verification of Deep Networks

Krishnamurthy, Dvijotham, Robert Stanforth, Sven Gowal, Timothy Mann, Pushmeet Kohli

1803.06567

EDUCE: Explaining model Decisions through Unsupervised Concepts Extraction

Diane Bouchacourt, Ludovic Denoyer

1905.11852

Statistical Inference for Model Parameters in Stochastic Gradient Descent

Xi Chen, Jason D. Lee, Xin T. Tong, Yichen Zhang

1610.08637

Reinforcement and Imitation Learning via Interactive No-Regret Learning

Stephane Ross, J. Andrew Bagnell

1406.5979

The Frontiers of Fairness in Machine Learning

Alexandra Chouldechova, Aaron Roth

1810.08810

Robust Compressed Sensing MRI with Deep Generative Priors

Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alexandros G. Dimakis, Jonathan I. Tamir

2108.01368

IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression

Rianne van den Berg, Alexey A. Gritsenko, Mostafa Dehghani, Casper Kaae Sønderby, Tim Salimans

2006.12459

Tactics of Adversarial Attack on Deep Reinforcement Learning Agents

Yen-Chen Lin, Zhang-Wei Hong, Yuan-Hong Liao, Meng-Li Shih, Ming-Yu Liu, Min Sun

1703.06748

MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims

Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, Jakob Grue Simonsen

1909.03242

Fast Exact Matrix Completion with Finite Samples

Prateek Jain, Praneeth Netrapalli

1411.1087

The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training

Andrea Montanari, Yiqiao Zhong

2007.12826

The loss surface of deep and wide neural networks

Quynh Nguyen, Matthias Hein

1704.08045

Superpolynomial Lower Bounds for Learning One-Layer Neural Networks using Gradient Descent

Surbhi Goel, Aravind Gollakota, Zhihan Jin, Sushrut Karmalkar, Adam Klivans

2006.12011

Fast Structured Decoding for Sequence Models

Zhiqing Sun, Zhuohan Li, Haoqing Wang, Zi Lin, Di He, Zhi-Hong Deng

1910.11555

Factoring nonnegative matrices with linear programs

Victor Bittorf, Benjamin Recht, Christopher Re, Joel A. Tropp

1206.1270

Preference Completion: Large-scale Collaborative Ranking from Pairwise Comparisons

Dohyung Park, Joe Neeman, Jin Zhang, Sujay Sanghavi, Inderjit S. Dhillon

1507.04457

Neural Adaptive Sequential Monte Carlo

Shixiang Gu, Zoubin Ghahramani, Richard E. Turner

1506.03338

On Provable Copyright Protection for Generative Models

Nikhil Vyas, Sham Kakade, Boaz Barak

2302.10870

Fast Differentiable Sorting and Ranking

Mathieu Blondel, Olivier Teboul, Quentin Berthet, Josip Djolonga

2002.08871

Primal Method for ERM with Flexible Mini-batching Schemes and Non-convex Losses

Dominik Csiba, Peter Richtárik

1506.02227

Fixup Initialization: Residual Learning Without Normalization

Hongyi Zhang, Yann N. Dauphin, Tengyu Ma

1901.09321

Analyzing Inverse Problems with Invertible Neural Networks

Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W. Pellegrini, Ralf S. Klessen, Lena Maier-Hein, Carsten Rother, Ullrich Köthe

1808.04730

Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Christopher De Sa, Zhiru Zhang

1901.09504

Deep Gaussian Processes

Andreas C. Damianou, Neil D. Lawrence

1211.0358

A Kernel Test for Three-Variable Interactions

Dino Sejdinovic, Arthur Gretton, Wicher Bergsma

1306.2281

Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization

Xiao Wang, Shiqian Ma, Donald Goldfarb, Wei Liu

1607.01231

Deeply Learning the Messages in Message Passing Inference

Guosheng Lin, Chunhua Shen, Ian Reid, Anton van den Hengel

1506.02108

A Convex Framework for Fair Regression

Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, Aaron Roth

1706.02409

VeLO: Training Versatile Learned Optimizers by Scaling Up

Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, Jascha Sohl-Dickstein

2211.09760

Datamodels: Predicting Predictions from Training Data

Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclerc, Aleksander Madry

2202.00622

Inference Networks for Sequential Monte Carlo in Graphical Models

Brooks Paige, Frank Wood

1602.06701

Automatic Gradient Descent: Deep Learning without Hyperparameters

Jeremy Bernstein, Chris Mingard, Kevin Huang, Navid Azizan, Yisong Yue

2304.05187

A Kernel Independence Test for Random Processes

Kacper Chwialkowski, Arthur Gretton

1402.4501

Density Ratio Estimation via Infinitesimal Classification

Kristy Choi, Chenlin Meng, Yang Song, Stefano Ermon

2111.11010

Supersparse Linear Integer Models for Optimized Medical Scoring Systems

Berk Ustun, Cynthia Rudin

1502.04269

Observe and Look Further: Achieving Consistent Performance on Atari

Tobias Pohlen, Bilal Piot, Todd Hester, Mohammad Gheshlaghi Azar, Dan Horgan, David Budden, Gabriel Barth-Maron, Hado van Hasselt, John Quan, Mel Večerík, Matteo Hessel, Rémi Munos, Olivier Pietquin

1805.11593

Hyperparameter Selection for Offline Reinforcement Learning

Tom Le Paine, Cosmin Paduraru, Andrea Michi, Caglar Gulcehre, Konrad Zolna, Alexander Novikov, Ziyu Wang, Nando de Freitas

2007.09055

Latent ODEs for Irregularly-Sampled Time Series

Yulia Rubanova, Ricky T. Q. Chen, David Duvenaud

1907.03907

The Space of Transferable Adversarial Examples

Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel

1704.03453

Deep Learning: A Critical Appraisal

Gary Marcus

1801.00631

Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning

Milad Nasr, Reza Shokri, Amir Houmansadr

1812.00910

On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants

Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabás Póczos, Alex Smola

1506.06840

An exact mapping between the Variational Renormalization Group and Deep Learning

Pankaj Mehta, David J. Schwab

1410.3831

Frequency Principle in Deep Learning with General Loss Functions and Its Potential Application

Zhi-Qin John Xu

1811.10146

Recurrent Poisson Factorization for Temporal Recommendation

Seyed Abbas Hosseini, Keivan Alizadeh, Ali Khodadadi, Ali Arabzadeh, Mehrdad Farajtabar, Hongyuan Zha, Hamid R. Rabiee

1703.01442

Recent Developments on Factor Models and its Applications in Econometric Learning

Jianqing Fan, Kunpeng Li, Yuan Liao

2009.10103

Spectral norm of random tensors

Ryota Tomioka, Taiji Suzuki

1407.1870

Exploration by Random Network Distillation

Yuri Burda, Harrison Edwards, Amos Storkey, Oleg Klimov

1810.12894

Robust Filtering and Smoothing with Gaussian Processes

Marc Peter Deisenroth, Ryan Turner, Marco F. Huber, Uwe D. Hanebeck, Carl Edward Rasmussen

1203.4345

Nonparametric Canonical Correlation Analysis

Tomer Michaeli, Weiran Wang, Karen Livescu

1511.04839

Elliptical slice sampling

Iain Murray, Ryan Prescott Adams, David J. C. MacKay

1001.0175

Exploring the Limits of Large Scale Pre-training

Samira Abnar, Mostafa Dehghani, Behnam Neyshabur, Hanie Sedghi

2110.02095

Attentive Neural Processes

Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, Yee Whye Teh

1901.05761

State Representation Learning for Control: An Overview

Timothée Lesort, Natalia Díaz-Rodríguez, Jean-François Goudou, David Filliat

1802.04181

Minimax Regret Bounds for Reinforcement Learning

Mohammad Gheshlaghi Azar, Ian Osband, Rémi Munos

1703.05449

Information Dropout: Learning Optimal Representations Through Noisy Computation

Alessandro Achille, Stefano Soatto

1611.01353

Spurious Valleys in Two-layer Neural Network Optimization Landscapes

Luca Venturi, Afonso S. Bandeira, Joan Bruna

1802.06384

Self-training with Noisy Student improves ImageNet classification

Qizhe Xie, Minh-Thang Luong, Eduard Hovy, Quoc V. Le

1911.04252

Hierarchical Neural Architecture Search via Operator Clustering

Guilin Li, Xing Zhang, Zitong Wang, Matthias Tan, Jiashi Feng, Zhenguo Li, Tong Zhang

1909.11926

Avoiding Discrimination through Causal Reasoning

Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, Bernhard Schölkopf

1706.02744

An Introduction to Variational Autoencoders

Diederik P. Kingma, Max Welling

1906.02691

Kernel Adaptive Metropolis-Hastings

Dino Sejdinovic, Heiko Strathmann, Maria Lomeli Garcia, Christophe Andrieu, Arthur Gretton

1307.5302

Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations

Liu Yang, Dongkun Zhang, George Em Karniadakis

1811.02033

On Causal and Anticausal Learning

Bernhard Schoelkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, Joris Mooij

1206.6471

Feature-Budgeted Random Forest

Feng Nan, Joseph Wang, Venkatesh Saligrama

1502.05925

Learned Step Size Quantization

Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, Dharmendra S. Modha

1902.08153

Gated End-to-End Memory Networks

Julien Perez, Fei Liu

1610.04211

Understanding and Robustifying Differentiable Architecture Search

Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, Frank Hutter

1909.09656

What Makes Convolutional Models Great on Long Sequence Modeling?

Yuhong Li, Tianle Cai, Yi Zhang, Deming Chen, Debadeepta Dey

2210.09298

Optimally-Weighted Herding is Bayesian Quadrature

Ferenc Huszár, David Duvenaud

1204.1664

Excessive Invariance Causes Adversarial Vulnerability

Jörn-Henrik Jacobsen, Jens Behrmann, Richard Zemel, Matthias Bethge

1811.00401

The frontier of simulation-based inference

Kyle Cranmer, Johann Brehmer, Gilles Louppe

1911.01429

Bayesian Poisson Tucker Decomposition for Learning the Structure of International Relations

Aaron Schein, Mingyuan Zhou, David M. Blei, Hanna Wallach

1606.01855

Deep Graph Infomax

Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, R Devon Hjelm

1809.10341

Gradient Descent Finds Global Minima of Deep Neural Networks

Simon S. Du, Jason D. Lee, Haochuan Li, Liwei Wang, Xiyu Zhai

1811.03804

Solving Schrödinger Bridges via Maximum Likelihood

Francisco Vargas, Pierre Thodoroff, Neil D. Lawrence, Austen Lamacraft

2106.02081

Analyzing Finite Neural Networks: Can We Trust Neural Tangent Kernel Theory?

Mariia Seleznova, Gitta Kutyniok

2012.04477

Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture

Ju Sun, Qing Qu, John Wright

1511.03607

BayesNAS: A Bayesian Approach for Neural Architecture Search

Hongpeng Zhou, Minghao Yang, Jun Wang, Wei Pan

1905.04919

A Tight and Unified Analysis of Gradient-Based Methods for a Whole Spectrum of Games

Waïss Azizian, Ioannis Mitliagkas, Simon Lacoste-Julien, Gauthier Gidel

1906.05945

Stochastic modified equations and adaptive stochastic gradient algorithms

Qianxiao Li, Cheng Tai, Weinan E

1511.06251

You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle

Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, Bin Dong

1905.00877

Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search

Arthur Guez, David Silver, Peter Dayan

1205.3109

Interpretable Distribution Features with Maximum Testing Power

Wittawat Jitkrittum, Zoltan Szabo, Kacper Chwialkowski, Arthur Gretton

1605.06796

Encoder Based Lifelong Learning

Amal Rannen Triki, Rahaf Aljundi, Mathew B. Blaschko, Tinne Tuytelaars

1704.01920

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

Vivek Jayaram, John Thickstun

2105.08164

Disentanglement via Latent Quantization

Kyle Hsu, Will Dorrell, James C. R. Whittington, Jiajun Wu, Chelsea Finn

2305.18378

Disentangling Adaptive Gradient Methods from Learning Rates

Naman Agarwal, Rohan Anil, Elad Hazan, Tomer Koren, Cyril Zhang

2002.11803

Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net

Anirudh Goyal, Nan Rosemary Ke, Surya Ganguli, Yoshua Bengio

1711.02282

Conditional Neural Processes

Marta Garnelo, Dan Rosenbaum, Chris J. Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo J. Rezende, S. M. Ali Eslami

1807.01613

Compositional Fairness Constraints for Graph Embeddings

Avishek Joey Bose, William L. Hamilton

1905.10674

Improving End-to-End Speech Recognition with Policy Learning

Yingbo Zhou, Caiming Xiong, Richard Socher

1712.07101

The Recurrent Neural Tangent Kernel

Sina Alemohammad, Zichao Wang, Randall Balestriero, Richard Baraniuk

2006.10246

Offline RL Policies Should be Trained to be Adaptive

Dibya Ghosh, Anurag Ajay, Pulkit Agrawal, Sergey Levine

2207.02200

Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks

Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, Yang Liu

1909.03496

Neural Kernels Without Tangents

Vaishaal Shankar, Alex Fang, Wenshuo Guo, Sara Fridovich-Keil, Ludwig Schmidt, Jonathan Ragan-Kelley, Benjamin Recht

2003.02237

Compressive Spectral Clustering

Nicolas Tremblay, Gilles Puy, Remi Gribonval, Pierre Vandergheynst

1602.02018

VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification

Zhibin Lu, Pan Du, Jian-Yun Nie

2004.05707

Benchmarks for Deep Off-Policy Evaluation

Justin Fu, Mohammad Norouzi, Ofir Nachum, George Tucker, Ziyu Wang, Alexander Novikov, Mengjiao Yang, Michael R. Zhang, Yutian Chen, Aviral Kumar, Cosmin Paduraru, Sergey Levine, Tom Le Paine

2103.16596

The Implicit and Explicit Regularization Effects of Dropout

Colin Wei, Sham Kakade, Tengyu Ma

2002.12915

A Note on the Inception Score

Shane Barratt, Rishi Sharma

1801.01973

Communication trade-offs for synchronized distributed SGD with large step size

Kumar Kshitij Patel, Aymeric Dieuleveut

1904.11325

Effective Data Augmentation with Multi-Domain Learning GANs

Shin'ya Yamaguchi, Sekitoshi Kanai, Takeharu Eda

1912.11597

Solving Large-scale Systems of Random Quadratic Equations via Stochastic Truncated Amplitude Flow

Gang Wang, Georgios B. Giannakis, Jie Chen

1610.09540

Block-Coordinate Frank-Wolfe Optimization for Structural SVMs

Simon Lacoste-Julien, Martin Jaggi, Mark Schmidt, Patrick Pletscher

1207.4747

Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction

Hyungjin Chung, Byeongsu Sim, Jong Chul Ye

2112.05146

Multimodal Word Distributions

Ben Athiwaratkun, Andrew Gordon Wilson

1704.08424

Generative Adversarial Networks

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio

1406.2661

Consistency and fluctuations for stochastic gradient Langevin dynamics

Yee Whye Teh, Alexandre Thiéry, Sebastian Vollmer

1409.0578

How to Train Your Energy-Based Models

Yang Song, Diederik P. Kingma

2101.03288

First-order Stochastic Algorithms for Escaping From Saddle Points in Almost Linear Time

Yi Xu, Rong Jin, Tianbao Yang

1711.01944

Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection

Julie Nutini, Mark Schmidt, Issam H. Laradji, Michael Friedlander, Hoyt Koepke

1506.00552

Fast and Robust Recursive Algorithms for Separable Nonnegative Matrix Factorization

Nicolas Gillis, Stephen A. Vavasis

1208.1237

Hyperparameter Importance Across Datasets

J. N. van Rijn, F. Hutter

1710.04725

Unitary Evolution Recurrent Neural Networks

Martin Arjovsky, Amar Shah, Yoshua Bengio

1511.06464

Reinterpreting Importance-Weighted Autoencoders

Chris Cremer, Quaid Morris, David Duvenaud

1704.02916

Distributional Reinforcement Learning with Quantile Regression

Will Dabney, Mark Rowland, Marc G. Bellemare, Rémi Munos

1710.10044

Fair prediction with disparate impact: A study of bias in recidivism prediction instruments

Alexandra Chouldechova

1610.07524

Snomed2Vec: Random Walk and Poincaré Embeddings of a Clinical Knowledge Base for Healthcare Analytics

Khushbu Agarwal, Tome Eftimov, Raghavendra Addanki, Sutanay Choudhury, Suzanne Tamang, Robert Rallo

1907.08650

A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs

Gadi Naveh, Zohar Ringel

2106.04110

Recycling Randomness with Structure for Sublinear time Kernel Expansions

Krzysztof Choromanski, Vikas Sindhwani

1605.09049

Structured Bayesian Pruning via Log-Normal Multiplicative Noise

Kirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha, Dmitry Vetrov

1705.07283

Adversarially Robust Generalization Requires More Data

Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, Aleksander Mądry

1804.11285

Differential Privacy Has Disparate Impact on Model Accuracy

Eugene Bagdasaryan, Vitaly Shmatikov

1905.12101

Model-Based Reinforcement Learning for Atari

Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, Afroz Mohiuddin, Ryan Sepassi, George Tucker, Henryk Michalewski

1903.00374

Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, Jasper Snoek

1906.02530

Stabilizing Transformer Training by Preventing Attention Entropy Collapse

Shuangfei Zhai, Tatiana Likhomanenko, Etai Littwin, Dan Busbridge, Jason Ramapuram, Yizhe Zhang, Jiatao Gu, Josh Susskind

2303.06296

DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps

Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu

2206.00927

Simple Black-Box Adversarial Perturbations for Deep Networks

Nina Narodytska, Shiva Prasad Kasiviswanathan

1612.06299

Learning to Represent Edits

Pengcheng Yin, Graham Neubig, Miltiadis Allamanis, Marc Brockschmidt, Alexander L. Gaunt

1810.13337

Low-Rank Matrix Recovery with Scaled Subgradient Methods: Fast and Robust Convergence Without the Condition Number

Tian Tong, Cong Ma, Yuejie Chi

2010.13364

Gaussian variational approximation with sparse precision matrices

Linda S. L. Tan, David J. Nott

1605.05622

Diffusion Models for Video Prediction and Infilling

Tobias Höppe, Arash Mehrjou, Stefan Bauer, Didrik Nielsen, Andrea Dittadi

2206.07696

Amortised MAP Inference for Image Super-resolution

Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi, Ferenc Huszár

1610.04490

Three Mechanisms of Weight Decay Regularization

Guodong Zhang, Chaoqi Wang, Bowen Xu, Roger Grosse

1810.12281

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning

Briland Hitaj, Giuseppe Ateniese, Fernando Perez-Cruz

1702.07464

Unsupervised Data Augmentation for Consistency Training

Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, Quoc V. Le

1904.12848

Few-Shot Learning with Graph Neural Networks

Victor Garcia, Joan Bruna

1711.04043

How to train your MAML

Antreas Antoniou, Harrison Edwards, Amos Storkey

1810.09502

Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax

Yu Li, Tao Wang, Bingyi Kang, Sheng Tang, Chunfeng Wang, Jintao Li, Jiashi Feng

2006.10408

Efficient Algorithms for Smooth Minimax Optimization

Kiran Koshy Thekumparampil, Prateek Jain, Praneeth Netrapalli, Sewoong Oh

1907.01543

The Knowledge Within: Methods for Data-Free Model Compression

Matan Haroush, Itay Hubara, Elad Hoffer, Daniel Soudry

1912.01274

Reducing Overfitting in Deep Networks by Decorrelating Representations

Michael Cogswell, Faruk Ahmed, Ross Girshick, Larry Zitnick, Dhruv Batra

1511.06068

Step Size Matters in Deep Learning

Kamil Nar, S. Shankar Sastry

1805.08890

nuScenes: A multimodal dataset for autonomous driving

Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, Oscar Beijbom

1903.11027

Neural Structure Learning with Stochastic Differential Equations

Benjie Wang, Joel Jennings, Wenbo Gong

2311.03309

What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation

Vitaly Feldman, Chiyuan Zhang

2008.03703

Reparameterization Gradients through Acceptance-Rejection Sampling Algorithms

Christian A. Naesseth, Francisco J. R. Ruiz, Scott W. Linderman, David M. Blei

1610.05683

Stein Variational Gradient Descent as Gradient Flow

Qiang Liu

1704.07520

Accurate and Conservative Estimates of MRF Log-likelihood using Reverse Annealing

Yuri Burda, Roger B. Grosse, Ruslan Salakhutdinov

1412.8566

Fast Stochastic Algorithms for SVD and PCA: Convergence Properties and Convexity

Ohad Shamir

1507.08788

Sockeye: A Toolkit for Neural Machine Translation

Felix Hieber, Tobias Domhan, Michael Denkowski, David Vilar, Artem Sokolov, Ann Clifton, Matt Post

1712.05690

Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai

1607.06520

Sub-sampled Cubic Regularization for Non-convex Optimization

Jonas Moritz Kohler, Aurelien Lucchi

1705.05933

A totally unimodular view of structured sparsity

Marwa El Halabi, Volkan Cevher

1411.1990

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Marius Mosbach, Maksym Andriushchenko, Dietrich Klakow

2006.04884

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis

Jiatao Gu, Lingjie Liu, Peng Wang, Christian Theobalt

2110.08985

Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion

Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, Stefano Ermon

2310.02279

Sequence-to-Sequence Learning as Beam-Search Optimization

Sam Wiseman, Alexander M. Rush

1606.02960

Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior

Guangzhi Sun, Yu Zhang, Ron J. Weiss, Yuan Cao, Heiga Zen, Andrew Rosenberg, Bhuvana Ramabhadran, Yonghui Wu

2002.03788

Overcoming Forgetting in Federated Learning on Non-IID Data

Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Israel, Daniel Benditkis, Liron Mor-Yosef, Itai Zeitak

1910.07796

A Large Dimensional Analysis of Least Squares Support Vector Machines

Zhenyu Liao, Romain Couillet

1701.02967

Differentially Private Coordinate Descent for Composite Empirical Risk Minimization

Paul Mangold, Aurélien Bellet, Joseph Salmon, Marc Tommasi

2110.11688

Spectral Norm Regularization for Improving the Generalizability of Deep Learning

Yuichi Yoshida, Takeru Miyato

1705.10941

Once-for-All: Train One Network and Specialize it for Efficient Deployment

Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, Song Han

1908.09791

Omnigrok: Grokking Beyond Algorithmic Data

Ziming Liu, Eric J. Michaud, Max Tegmark

2210.01117

Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model

Benjamin Letham, Cynthia Rudin, Tyler H. McCormick, David Madigan

1511.01644

Wasserstein GAN

Martin Arjovsky, Soumith Chintala, Léon Bottou

1701.07875

More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch

Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward H. Adelson, Sergey Levine

1805.11085

Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods

Nicolas Loizou, Peter Richtárik

1712.09677

Bias-variance decomposition of overparameterized regression with random linear features

Jason W. Rocks, Pankaj Mehta

2203.05443

Hybrid Models with Deep and Invertible Features

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, Balaji Lakshminarayanan

1902.02767

Scaling ResNets in the Large-depth Regime

Pierre Marion, Adeline Fermanian, Gérard Biau, Jean-Philippe Vert

2206.06929

Large-Scale Methods for Distributionally Robust Optimization

Daniel Levy, Yair Carmon, John C. Duchi, Aaron Sidford

2010.05893

Learning perturbation sets for robust machine learning

Eric Wong, J. Zico Kolter

2007.08450

The combinatorial structure of beta negative binomial processes

Creighton Heaukulani, Daniel M. Roy

1401.0062

Understanding Membership Inferences on Well-Generalized Learning Models

Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A. Gunter, Kai Chen

1802.04889

AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates

Ning Liu, Xiaolong Ma, Zhiyuan Xu, Yanzhi Wang, Jian Tang, Jieping Ye

1907.03141

Hyperspherical Variational Auto-Encoders

Tim R. Davidson, Luca Falorsi, Nicola De Cao, Thomas Kipf, Jakub M. Tomczak

1804.00891

MaskGAN: Better Text Generation via Filling in the______

William Fedus, Ian Goodfellow, Andrew M. Dai

1801.07736

Deep Successor Reinforcement Learning

Tejas D. Kulkarni, Ardavan Saeedi, Simanta Gautam, Samuel J. Gershman

1606.02396

LSQ+: Improving low-bit quantization through learnable offsets and better initialization

Yash Bhalgat, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, Nojun Kwak

2004.09576

Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching

Cheng Lu, Kaiwen Zheng, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu

2206.08265

Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours

Dimitrios Stamoulis, Ruizhou Ding, Di Wang, Dimitrios Lymberopoulos, Bodhi Priyantha, Jie Liu, Diana Marculescu

1904.02877

Poincaré Embeddings for Learning Hierarchical Representations

Maximilian Nickel, Douwe Kiela

1705.08039

The Implicit Bias of Benign Overfitting

Ohad Shamir

2201.11489

Learning curves of generic features maps for realistic datasets with a teacher-student model

Bruno Loureiro, Cédric Gerbelot, Hugo Cui, Sebastian Goldt, Florent Krzakala, Marc Mézard, Lenka Zdeborová

2102.08127

Copula Processes

Andrew Gordon Wilson, Zoubin Ghahramani

1006.1350

MixUp as Locally Linear Out-Of-Manifold Regularization

Hongyu Guo, Yongyi Mao, Richong Zhang

1809.02499

A Fourier Perspective on Model Robustness in Computer Vision

Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin D. Cubuk, Justin Gilmer

1906.08988

Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules

Daniel Ho, Eric Liang, Ion Stoica, Pieter Abbeel, Xi Chen

1905.05393

Stabilizing Training of Generative Adversarial Networks through Regularization

Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, Thomas Hofmann

1705.09367

First-order Methods Almost Always Avoid Saddle Points

Jason D. Lee, Ioannis Panageas, Georgios Piliouras, Max Simchowitz, Michael I. Jordan, Benjamin Recht

1710.07406

Latent Alignment and Variational Attention

Yuntian Deng, Yoon Kim, Justin Chiu, Demi Guo, Alexander M. Rush

1807.03756

Stochastic AUC Maximization with Deep Neural Networks

Mingrui Liu, Zhuoning Yuan, Yiming Ying, Tianbao Yang

1908.10831

Exploring Interpretable LSTM Neural Networks over Multi-Variable Data

Tian Guo, Tao Lin, Nino Antulov-Fantulin

1905.12034

FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search

Xiangxiang Chu, Bo Zhang, Ruijun Xu

1907.01845

Global universal approximation of functional input maps on weighted spaces

Christa Cuchiero, Philipp Schmocker, Josef Teichmann

2306.03303

Theoretical guarantees for approximate sampling from smooth and log-concave densities

Arnak S. Dalalyan

1412.7392

Spectral Representations for Convolutional Neural Networks

Oren Rippel, Jasper Snoek, Ryan P. Adams

1506.03767

Representer Point Selection for Explaining Deep Neural Networks

Chih-Kuan Yeh, Joon Sik Kim, Ian E. H. Yen, Pradeep Ravikumar

1811.09720

Controllable Guarantees for Fair Outcomes via Contrastive Information Estimation

Umang Gupta, Aaron M Ferber, Bistra Dilkina, Greg Ver Steeg

2101.04108

Hypothesis Testing Interpretations and Renyi Differential Privacy

Borja Balle, Gilles Barthe, Marco Gaboardi, Justin Hsu, Tetsuya Sato

1905.09982

Near-Optimal Performance Bounds for Orthogonal and Permutation Group Synchronization via Spectral Methods

Shuyang Ling

2008.05341

Bayesian Bits: Unifying Quantization and Pruning

Mart van Baalen, Christos Louizos, Markus Nagel, Rana Ali Amjad, Ying Wang, Tijmen Blankevoort, Max Welling

2005.07093

Beta-Negative Binomial Process and Poisson Factor Analysis

Mingyuan Zhou, Lauren Hannah, David Dunson, Lawrence Carin

1112.3605

Decentralized learning for wireless communications and networking

Georgios B. Giannakis, Qing Ling, Gonzalo Mateos, Ioannis D. Schizas, Hao Zhu

1503.08855

RvS: What is Essential for Offline RL via Supervised Learning?

Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, Sergey Levine

2112.10751

Finite Depth and Width Corrections to the Neural Tangent Kernel

Boris Hanin, Mihai Nica

1909.05989

Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization

Navid Azizan, Sahin Lale, Babak Hassibi

1906.03830

Adversarial Filters of Dataset Biases

Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew E. Peters, Ashish Sabharwal, Yejin Choi

2002.04108

Scalable Differentiable Physics for Learning and Control

Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, Ming C. Lin

2007.02168

Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers

Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, Sebastien Bubeck

1906.04584

Causality for Machine Learning

Bernhard Schölkopf

1911.10500

Learning Convolutional Neural Networks for Graphs

Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov

1605.05273

Metric Learning for Adversarial Robustness

Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, Baishakhi Ray

1909.00900

Experimental Design for Regret Minimization in Linear Bandits

Andrew Wagenmaker, Julian Katz-Samuels, Kevin Jamieson

2011.00576

Learning to Prove Theorems via Interacting with Proof Assistants

Kaiyu Yang, Jia Deng

1905.09381

Unified Optimal Analysis of the (Stochastic) Gradient Method

Sebastian U. Stich

1907.04232

Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow

Gang Wang, Georgios B. Giannakis, Yonina C. Eldar

1605.08285

Grammar Variational Autoencoder

Matt J. Kusner, Brooks Paige, José Miguel Hernández-Lobato

1703.01925

Approximation and Learning with Deep Convolutional Models: a Kernel Perspective

Alberto Bietti

2102.10032

Nested Sequential Monte Carlo Methods

Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön

1502.02536

Deep Reinforcement Learning at the Edge of the Statistical Precipice

Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron Courville, Marc G. Bellemare

2108.13264

Attacks Which Do Not Kill Training Make Adversarial Learning Stronger

Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, Mohan Kankanhalli

2002.11242

Flow Contrastive Estimation of Energy-Based Models

Ruiqi Gao, Erik Nijkamp, Diederik P. Kingma, Zhen Xu, Andrew M. Dai, Ying Nian Wu

1912.00589

iCaRL: Incremental Classifier and Representation Learning

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, Christoph H. Lampert

1611.07725

High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normality

Zhaoran Wang, Quanquan Gu, Yang Ning, Han Liu

1412.8729

Capsule Network Performance on Complex Data

Edgar Xi, Selina Bing, Yang Jin

1712.03480

Randomized Nonlinear Component Analysis

David Lopez-Paz, Suvrit Sra, Alex Smola, Zoubin Ghahramani, Bernhard Schölkopf

1402.0119

Dynamic Backdoor Attacks Against Machine Learning Models

Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, Yang Zhang

2003.03675

On the Equivalence between Herding and Conditional Gradient Algorithms

Francis Bach, Simon Lacoste-Julien, Guillaume Obozinski

1203.4523

Transformers Can Do Bayesian Inference

Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, Frank Hutter

2112.10510

Is Out-of-Distribution Detection Learnable?

Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, Feng Liu

2210.14707

Improved Consistency Regularization for GANs

Zhengli Zhao, Sameer Singh, Honglak Lee, Zizhao Zhang, Augustus Odena, Han Zhang

2002.04724

Maximum Principle Based Algorithms for Deep Learning

Qianxiao Li, Long Chen, Cheng Tai, Weinan E

1710.09513

Neural Network Encapsulation

Hongyang Li, Xiaoyang Guo, Bo Dai, Wanli Ouyang, Xiaogang Wang

1808.03749

Phase Transitions of Spectral Initialization for High-Dimensional Nonconvex Estimation

Yue M. Lu, Gen Li

1702.06435

On Nonconvex Optimization for Machine Learning: Gradients, Stochasticity, and Saddle Points

Chi Jin, Praneeth Netrapalli, Rong Ge, Sham M. Kakade, Michael I. Jordan

1902.04811

Locality defeats the curse of dimensionality in convolutional teacher-student scenarios

Alessandro Favero, Francesco Cagnetta, Matthieu Wyart

2106.08619

Learning Latent Space Energy-Based Prior Model

Bo Pang, Tian Han, Erik Nijkamp, Song-Chun Zhu, Ying Nian Wu

2006.08205

Frequency Bias in Neural Networks for Input of Non-Uniform Density

Ronen Basri, Meirav Galun, Amnon Geifman, David Jacobs, Yoni Kasten, Shira Kritchman

2003.04560

Variational Inference in Nonconjugate Models

Chong Wang, David M. Blei

1209.4360

Classification vs regression in overparameterized regimes: Does the loss function matter?

Vidya Muthukumar, Adhyyan Narang, Vignesh Subramanian, Mikhail Belkin, Daniel Hsu, Anant Sahai

2005.08054

SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation

Yang Zhao, Xiaohan Chen, Yue Wang, Chaojian Li, Haoran You, Yonggan Fu, Yuan Xie, Zhangyang Wang, Yingyan Lin

2005.03403

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards

Haoxiang Wang, Yong Lin, Wei Xiong, Rui Yang, Shizhe Diao, Shuang Qiu, Han Zhao, Tong Zhang

2402.18571

RES: Regularized Stochastic BFGS Algorithm

Aryan Mokhtari, Alejandro Ribeiro

1401.7625

Visual Object Networks: Image Generation with Disentangled 3D Representation

Jun-Yan Zhu, Zhoutong Zhang, Chengkai Zhang, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum, William T. Freeman

1812.02725

A new method for parameter estimation in probabilistic models: Minimum probability flow

Jascha Sohl-Dickstein, Peter Battaglino, Michael R. DeWeese

2007.09240

Boosting the concordance index for survival data - a unified framework to derive and evaluate biomarker combinations

Andreas Mayr, Matthias Schmid

1307.6417

Continual Learning Through Synaptic Intelligence

Friedemann Zenke, Ben Poole, Surya Ganguli

1703.04200

The Ingredients of Real-World Robotic Reinforcement Learning

Henry Zhu, Justin Yu, Abhishek Gupta, Dhruv Shah, Kristian Hartikainen, Avi Singh, Vikash Kumar, Sergey Levine

2004.12570

Benchmarking Model-Based Reinforcement Learning

Tingwu Wang, Xuchan Bao, Ignasi Clavera, Jerrick Hoang, Yeming Wen, Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, Jimmy Ba

1907.02057

A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics

Weinan E, Chao Ma, Lei Wu

1904.04326

A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices

Till Speicher, Hoda Heidari, Nina Grgic-Hlaca, Krishna P. Gummadi, Adish Singla, Adrian Weller, Muhammad Bilal Zafar

1807.00787

Mean-Field Networks

Yujia Li, Richard Zemel

1410.5884

Neural Optimizer Search with Reinforcement Learning

Irwan Bello, Barret Zoph, Vijay Vasudevan, Quoc V. Le

1709.07417

When Does Label Smoothing Help?

Rafael Müller, Simon Kornblith, Geoffrey Hinton

1906.02629

Nested Hierarchical Dirichlet Processes

John Paisley, Chong Wang, David M. Blei, Michael I. Jordan

1210.6738

Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification

Yingxue Zhou, Zhiwei Steven Wu, Arindam Banerjee

2007.03813

Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection

Shiqian Ma, Lingzhou Xue, Hui Zou

1206.1275

Model Accuracy and Runtime Tradeoff in Distributed Deep Learning:A Systematic Study

Suyog Gupta, Wei Zhang, Fei Wang

1509.04210

Augmented Neural ODEs

Emilien Dupont, Arnaud Doucet, Yee Whye Teh

1904.01681

A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm

Chi Jin, Praneeth Netrapalli, Rong Ge, Sham M. Kakade, Michael I. Jordan

1902.03736

Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction

Christina Wadsworth, Francesca Vera, Chris Piech

1807.00199

Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations

Weinan E, Jiequn Han, Arnulf Jentzen

1706.04702

Matrix Completion from Noisy Entries

Raghunandan H. Keshavan, Andrea Montanari, Sewoong Oh

0906.2027

Simple random search provides a competitive approach to reinforcement learning

Horia Mania, Aurelia Guy, Benjamin Recht

1803.07055

Trellis Networks for Sequence Modeling

Shaojie Bai, J. Zico Kolter, Vladlen Koltun

1810.06682

L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data

Jianbo Chen, Le Song, Martin J. Wainwright, Michael I. Jordan

1808.02610

Minimum Width for Universal Approximation

Sejun Park, Chulhee Yun, Jaeho Lee, Jinwoo Shin

2006.08859

Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection

Bingzhe Wu, Shiwan Zhao, ChaoChao Chen, Haoyang Xu, Li Wang, Xiaolu Zhang, Guangyu Sun, Jun Zhou

1908.07882

Learning Likelihoods with Conditional Normalizing Flows

Christina Winkler, Daniel Worrall, Emiel Hoogeboom, Max Welling

1912.00042

Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent

Trevor Campbell, Tamara Broderick

1802.01737

End-to-End Learning of Semantic Grasping

Eric Jang, Sudheendra Vijayanarasimhan, Peter Pastor, Julian Ibarz, Sergey Levine

1707.01932

Learning and Evaluating General Linguistic Intelligence

Dani Yogatama, Cyprien de Masson d'Autume, Jerome Connor, Tomas Kocisky, Mike Chrzanowski, Lingpeng Kong, Angeliki Lazaridou, Wang Ling, Lei Yu, Chris Dyer, Phil Blunsom

1901.11373

Is Generator Conditioning Causally Related to GAN Performance?

Augustus Odena, Jacob Buckman, Catherine Olsson, Tom B. Brown, Christopher Olah, Colin Raffel, Ian Goodfellow

1802.08768

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

Stephane Ross, Geoffrey J. Gordon, J. Andrew Bagnell

1011.0686

Feature-Critic Networks for Heterogeneous Domain Generalization

Yiying Li, Yongxin Yang, Wei Zhou, Timothy M. Hospedales

1901.11448

Auditing Black-box Models for Indirect Influence

Philip Adler, Casey Falk, Sorelle A. Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, Suresh Venkatasubramanian

1602.07043

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music

Adam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, Douglas Eck

1803.05428

Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro

Du Phan, Neeraj Pradhan, Martin Jankowiak

1912.11554

Learning Longer-term Dependencies in RNNs with Auxiliary Losses

Trieu H. Trinh, Andrew M. Dai, Minh-Thang Luong, Quoc V. Le

1803.00144

Finite Sample Analysis of Approximate Message Passing Algorithms

Cynthia Rush, Ramji Venkataramanan

1606.01800

Asymptotics of Wide Convolutional Neural Networks

Anders Andreassen, Ethan Dyer

2008.08675

Dissecting Neural ODEs

Stefano Massaroli, Michael Poli, Jinkyoo Park, Atsushi Yamashita, Hajime Asama

2002.08071

The Fast Convergence of Incremental PCA

Akshay Balsubramani, Sanjoy Dasgupta, Yoav Freund

1501.03796

Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting

Frederic Koehler, Lijia Zhou, Danica J. Sutherland, Nathan Srebro

2106.09276

Deep Gaussian Processes for Regression using Approximate Expectation Propagation

Thang D. Bui, Daniel Hernández-Lobato, Yingzhen Li, José Miguel Hernández-Lobato, Richard E. Turner

1602.04133

What shapes feature representations? Exploring datasets, architectures, and training

Katherine L. Hermann, Andrew K. Lampinen

2006.12433

Fast matrix completion without the condition number

Moritz Hardt, Mary Wootters

1407.4070

Adaptive Sampling for Best Policy Identification in Markov Decision Processes

Aymen Al Marjani, Alexandre Proutiere

2009.13405

Bridging the Gap Between Value and Policy Based Reinforcement Learning

Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans

1702.08892

Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks

Ziwei Ji, Matus Telgarsky

1909.12292

Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels

Ilya Kostrikov, Denis Yarats, Rob Fergus

2004.13649

On the (In)fidelity and Sensitivity for Explanations

Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Sai Suggala, David I. Inouye, Pradeep Ravikumar

1901.09392

Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks

John Bradshaw, Alexander G. de G. Matthews, Zoubin Ghahramani

1707.02476

Theory of the Frequency Principle for General Deep Neural Networks

Tao Luo, Zheng Ma, Zhi-Qin John Xu, Yaoyu Zhang

1906.09235

Online Tensor Methods for Learning Latent Variable Models

Furong Huang, U. N. Niranjan, Mohammad Umar Hakeem, Animashree Anandkumar

1309.0787

Evaluating model calibration in classification

Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, Thomas B. Schön

1902.06977

Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima

Simon S. Du, Jason D. Lee, Yuandong Tian, Barnabas Poczos, Aarti Singh

1712.00779

Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method

Yifan Chen, Qi Zeng, Heng Ji, Yun Yang

2111.00035

Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes

Xianyan Jia, Shutao Song, Wei He, Yangzihao Wang, Haidong Rong, Feihu Zhou, Liqiang Xie, Zhenyu Guo, Yuanzhou Yang, Liwei Yu, Tiegang Chen, Guangxiao Hu, Shaohuai Shi, Xiaowen Chu

1807.11205

Implicit Regularization in Deep Learning May Not Be Explainable by Norms

Noam Razin, Nadav Cohen

2005.06398

Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks

Like Hui, Mikhail Belkin

2006.07322

Functional Anomaly Detection: a Benchmark Study

Guillaume Staerman, Eric Adjakossa, Pavlo Mozharovskyi, Vera Hofer, Jayant Sen Gupta, Stephan Clémençon

2201.05115

Invariant Representations without Adversarial Training

Daniel Moyer, Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, Aram Galstyan

1805.09458

How do Data Science Workers Collaborate? Roles, Workflows, and Tools

Amy X. Zhang, Michael Muller, Dakuo Wang

2001.06684

Parallel training of DNNs with Natural Gradient and Parameter Averaging

Daniel Povey, Xiaohui Zhang, Sanjeev Khudanpur

1410.7455

Aligned Cross Entropy for Non-Autoregressive Machine Translation

Marjan Ghazvininejad, Vladimir Karpukhin, Luke Zettlemoyer, Omer Levy

2004.01655

The Consciousness Prior

Yoshua Bengio

1709.08568

Simple Black-box Adversarial Attacks

Chuan Guo, Jacob R. Gardner, Yurong You, Andrew Gordon Wilson, Kilian Q. Weinberger

1905.07121

An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution

Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, Jason Yosinski

1807.03247

Reparameterizing the Birkhoff Polytope for Variational Permutation Inference

Scott W. Linderman, Gonzalo E. Mena, Hal Cooper, Liam Paninski, John P. Cunningham

1710.09508

Online Learning Without Prior Information

Ashok Cutkosky, Kwabena Boahen

1703.02629

Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning

Colin Wei, Sang Michael Xie, Tengyu Ma

2106.09226

Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators

Reinhard Heckel, Mahdi Soltanolkotabi

1910.14634

Ten Steps of EM Suffice for Mixtures of Two Gaussians

Constantinos Daskalakis, Christos Tzamos, Manolis Zampetakis

1609.00368

Dataset Distillation

Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, Alexei A. Efros

1811.10959

Curriculum Learning of Multiple Tasks

Anastasia Pentina, Viktoriia Sharmanska, Christoph H. Lampert

1412.1353

Ensemble Adversarial Training: Attacks and Defenses

Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel

1705.07204

Rethinking the Hyperparameters for Fine-tuning

Hao Li, Pratik Chaudhari, Hao Yang, Michael Lam, Avinash Ravichandran, Rahul Bhotika, Stefano Soatto

2002.11770

Adaptive Cross-Modal Few-Shot Learning

Chen Xing, Negar Rostamzadeh, Boris N. Oreshkin, Pedro O. Pinheiro

1902.07104

Long-tail learning via logit adjustment

Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, Sanjiv Kumar

2007.07314

Machine learning of solvent effects on molecular spectra and reactions

Michael Gastegger, Kristof T. Schütt, Klaus-Robert Müller

2010.14942

On the Practical Computational Power of Finite Precision RNNs for Language Recognition

Gail Weiss, Yoav Goldberg, Eran Yahav

1805.04908

Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach

Giorgio Patrini, Alessandro Rozza, Aditya Menon, Richard Nock, Lizhen Qu

1609.03683

On the diffusion approximation of nonconvex stochastic gradient descent

Wenqing Hu, Chris Junchi Li, Lei Li, Jian-Guo Liu

1705.07562

A Model of Double Descent for High-dimensional Binary Linear Classification

Zeyu Deng, Abla Kammoun, Christos Thrampoulidis

1911.05822

Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time

Yuzhou Gu, Zhao Song, Junze Yin, Lichen Zhang

2302.11068

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization

Christopher J. Cueva, Xue-Xin Wei

1803.07770

The Gaussian equivalence of generative models for learning with shallow neural networks

Sebastian Goldt, Bruno Loureiro, Galen Reeves, Florent Krzakala, Marc Mézard, Lenka Zdeborová

2006.14709

Learning Theory for Distribution Regression

Zoltan Szabo, Bharath Sriperumbudur, Barnabas Poczos, Arthur Gretton

1411.2066

Probably Approximately Correct Constrained Learning

Luiz F. O. Chamon, Alejandro Ribeiro

2006.05487

Adaptive scale-invariant online algorithms for learning linear models

Michał Kempka, Wojciech Kotłowski, Manfred K. Warmuth

1902.07528

The Benefit of Group Sparsity

Junzhou Huang, Tong Zhang

0901.2962

A Theory for Emergence of Complex Skills in Language Models

Sanjeev Arora, Anirudh Goyal

2307.15936

Learning Unitary Operators with Help From u(n)

Stephanie L. Hyland, Gunnar Rätsch

1607.04903

A Variant of the Wang-Foster-Kakade Lower Bound for the Discounted Setting

Philip Amortila, Nan Jiang, Tengyang Xie

2011.01075

Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficient

Botao Hao, Yaqi Duan, Tor Lattimore, Csaba Szepesvári, Mengdi Wang

2011.04019

Folded Recurrent Neural Networks for Future Video Prediction

Marc Oliu, Javier Selva, Sergio Escalera

1712.00311

PixelSNAIL: An Improved Autoregressive Generative Model

Xi Chen, Nikhil Mishra, Mostafa Rohaninejad, Pieter Abbeel

1712.09763

Variational approach for learning Markov processes from time series data

Hao Wu, Frank Noé

1707.04659

Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models

Mor Shpigel Nacson, Suriya Gunasekar, Jason D. Lee, Nathan Srebro, Daniel Soudry

1905.07325

Comparing Dynamics: Deep Neural Networks versus Glassy Systems

M. Baity-Jesi, L. Sagun, M. Geiger, S. Spigler, G. Ben Arous, C. Cammarota, Y. LeCun, M. Wyart, G. Biroli

1803.06969

Interpretable Deep Neural Networks for Single-Trial EEG Classification

Irene Sturm, Sebastian Bach, Wojciech Samek, Klaus-Robert Müller

1604.08201

Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods

Guoyin Li, Ting Kei Pong

1602.02915

Generalization Error Bounds for Noisy, Iterative Algorithms

Ankit Pensia, Varun Jog, Po-Ling Loh

1801.04295

Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects

Adam R. Kosiorek, Hyunjik Kim, Ingmar Posner, Yee Whye Teh

1806.01794

The (Un)reliability of saliency methods

Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, Been Kim

1711.00867

Episodic Curiosity through Reachability

Nikolay Savinov, Anton Raichuk, Raphaël Marinier, Damien Vincent, Marc Pollefeys, Timothy Lillicrap, Sylvain Gelly

1810.02274

Root Mean Square Layer Normalization

Biao Zhang, Rico Sennrich

1910.07467

Computational Limitations in Robust Classification and Win-Win Results

Akshay Degwekar, Preetum Nakkiran, Vinod Vaikuntanathan

1902.01086

Sequential Monte Carlo for Graphical Models

Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön

1402.0330

Differentially Private Federated Learning: A Client Level Perspective

Robin C. Geyer, Tassilo Klein, Moin Nabi

1712.07557

Bayesian Optimization with Exponential Convergence

Kenji Kawaguchi, Leslie Pack Kaelbling, Tomás Lozano-Pérez

1604.01348

On the impact of activation and normalization in obtaining isometric embeddings at initialization

Amir Joudaki, Hadi Daneshmand, Francis Bach

2305.18399

Finding Linear Structure in Large Datasets with Scalable Canonical Correlation Analysis

Zhuang Ma, Yichao Lu, Dean Foster

1506.08170

Predict then Propagate: Graph Neural Networks meet Personalized PageRank

Johannes Gasteiger, Aleksandar Bojchevski, Stephan Günnemann

1810.05997

Differentiable Dynamic Programming for Structured Prediction and Attention

Arthur Mensch, Mathieu Blondel

1802.03676

TUDataset: A collection of benchmark datasets for learning with graphs

Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, Marion Neumann

2007.08663

Scalable Gradients for Stochastic Differential Equations

Xuechen Li, Ting-Kam Leonard Wong, Ricky T. Q. Chen, David Duvenaud

2001.01328

Scalable Object Detection using Deep Neural Networks

Dumitru Erhan, Christian Szegedy, Alexander Toshev, Dragomir Anguelov

1312.2249

Learning Latent Permutations with Gumbel-Sinkhorn Networks

Gonzalo Mena, David Belanger, Scott Linderman, Jasper Snoek

1802.08665

Bayesian Hypernetworks

David Krueger, Chin-Wei Huang, Riashat Islam, Ryan Turner, Alexandre Lacoste, Aaron Courville

1710.04759

Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks

Peter L. Bartlett, David P. Helmbold, Philip M. Long

1802.06093

Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness

Lingjuan Lyu, Xuanli He, Yitong Li

2010.01285

Deep S$^3$PR: Simultaneous Source Separation and Phase Retrieval Using Deep Generative Models

Christopher A. Metzler, Gordon Wetzstein

2002.05856

Universality of Deep Convolutional Neural Networks

Ding-Xuan Zhou

1805.10769

A Unified Approach to Error Bounds for Structured Convex Optimization Problems

Zirui Zhou, Anthony Man-Cho So

1512.03518

Hierarchical Adversarially Learned Inference

Mohamed Ishmael Belghazi, Sai Rajeswar, Olivier Mastropietro, Negar Rostamzadeh, Jovana Mitrovic, Aaron Courville

1802.01071

A Unified Analysis of First-Order Methods for Smooth Games via Integral Quadratic Constraints

Guodong Zhang, Xuchan Bao, Laurent Lessard, Roger Grosse

2009.11359

The Uncertainty Bellman Equation and Exploration

Brendan O'Donoghue, Ian Osband, Remi Munos, Volodymyr Mnih

1709.05380

Ethical Adversaries: Towards Mitigating Unfairness with Adversarial Machine Learning

Pieter Delobelle, Paul Temple, Gilles Perrouin, Benoît Frénay, Patrick Heymans, Bettina Berendt

2005.06852

Scaling Graph Neural Networks with Approximate PageRank

Aleksandar Bojchevski, Johannes Gasteiger, Bryan Perozzi, Amol Kapoor, Martin Blais, Benedek Rózemberczki, Michal Lukasik, Stephan Günnemann

2007.01570

The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies

Ronen Basri, David Jacobs, Yoni Kasten, Shira Kritchman

1906.00425

Rigging the Lottery: Making All Tickets Winners

Utku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, Erich Elsen

1911.11134

SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications

Isabelle Augenstein, Mrinal Das, Sebastian Riedel, Lakshmi Vikraman, Andrew McCallum

1704.02853

Double descent in the condition number

Tomaso Poggio, Gil Kur, Andrzej Banburski

1912.06190

Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group

Johanna Erdmenger, Kevin T. Grosvenor, Ro Jefferson

2107.06898

Stochasticity in Neural ODEs: An Empirical Study

Viktor Oganesyan, Alexandra Volokhova, Dmitry Vetrov

2002.09779

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Rodrigo Veiga, Ludovic Stephan, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

2202.00293

Laplace Redux -- Effortless Bayesian Deep Learning

Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, Philipp Hennig

2106.14806

Deep backward schemes for high-dimensional nonlinear PDEs

Côme Huré, Huyên Pham, Xavier Warin

1902.01599

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review

Sergey Levine

1805.00909

Stochastic Gradient Push for Distributed Deep Learning

Mahmoud Assran, Nicolas Loizou, Nicolas Ballas, Michael Rabbat

1811.10792

Scaling Vision with Sparse Mixture of Experts

Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, Neil Houlsby

2106.05974

Efficient Learning by Directed Acyclic Graph For Resource Constrained Prediction

Joseph Wang, Kirill Trapeznikov, Venkatesh Saligrama

1510.07609

Co-clustering separately exchangeable network data

David Choi, Patrick J. Wolfe

1212.4093

A General Characterization of the Statistical Query Complexity

Vitaly Feldman

1608.02198

Learning with invariances in random features and kernel models

Song Mei, Theodor Misiakiewicz, Andrea Montanari

2102.13219

On Fairness and Calibration

Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, Kilian Q. Weinberger

1709.02012

Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields

Thomas Unterthiner, Bernhard Nessler, Calvin Seward, Günter Klambauer, Martin Heusel, Hubert Ramsauer, Sepp Hochreiter

1708.08819

Pathological spectra of the Fisher information metric and its variants in deep neural networks

Ryo Karakida, Shotaro Akaho, Shun-ichi Amari

1910.05992

Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks

Blake Bordelon, Abdulkadir Canatar, Cengiz Pehlevan

2002.02561

Conditional Restricted Boltzmann Machines for Structured Output Prediction

Volodymyr Mnih, Hugo Larochelle, Geoffrey E. Hinton

1202.3748

NeuroNER: an easy-to-use program for named-entity recognition based on neural networks

Franck Dernoncourt, Ji Young Lee, Peter Szolovits

1705.05487

SNAS: Stochastic Neural Architecture Search

Sirui Xie, Hehui Zheng, Chunxiao Liu, Liang Lin

1812.09926

Picking Winning Tickets Before Training by Preserving Gradient Flow

Chaoqi Wang, Guodong Zhang, Roger Grosse

2002.07376

Proximal Newton-type methods for minimizing composite functions

Jason D. Lee, Yuekai Sun, Michael A. Saunders

1206.1623

Model-Based Domain Generalization

Alexander Robey, George J. Pappas, Hamed Hassani

2102.11436

A Priori Estimates of the Population Risk for Two-layer Neural Networks

Weinan E, Chao Ma, Lei Wu

1810.06397

Integrating Document Clustering and Topic Modeling

Pengtao Xie, Eric P. Xing

1309.6874

Noisy Tensor Completion via the Sum-of-Squares Hierarchy

Boaz Barak, Ankur Moitra

1501.06521

Consistency Regularization for Generative Adversarial Networks

Han Zhang, Zizhao Zhang, Augustus Odena, Honglak Lee

1910.12027

Neural networks-based backward scheme for fully nonlinear PDEs

Huyen Pham, Xavier Warin, Maximilien Germain

1908.00412

Machine Learning from a Continuous Viewpoint

Weinan E, Chao Ma, Lei Wu

1912.12777

Adversarial Examples Are a Natural Consequence of Test Error in Noise

Nic Ford, Justin Gilmer, Nicolas Carlini, Dogus Cubuk

1901.10513

Asymmetry Helps: Eigenvalue and Eigenvector Analyses of Asymmetrically Perturbed Low-Rank Matrices

Yuxin Chen, Chen Cheng, Jianqing Fan

1811.12804

On the (im)possibility of fairness

Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian

1609.07236

Characterizing signal propagation to close the performance gap in unnormalized ResNets

Andrew Brock, Soham De, Samuel L. Smith

2101.08692

Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis

Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, Pushmeet Kohli

1805.04276

On the Convergence of FedAvg on Non-IID Data

Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, Zhihua Zhang

1907.02189

The Computational Limits of Deep Learning

Neil C. Thompson, Kristjan Greenewald, Keeheon Lee, Gabriel F. Manso

2007.05558

Online Censoring for Large-Scale Regressions with Application to Streaming Big Data

Dimitris Berberidis, Vassilis Kekatos, Georgios B. Giannakis

1507.07536

Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space

Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, Jianfeng Gao

2004.04092

Stochastic Recursive Variance-Reduced Cubic Regularization Methods

Dongruo Zhou, Quanquan Gu

1901.11518

The role of regularization in classification of high-dimensional noisy Gaussian mixture

Francesca Mignacco, Florent Krzakala, Yue M. Lu, Lenka Zdeborová

2002.11544

A Fine-Grained Spectral Perspective on Neural Networks

Greg Yang, Hadi Salman

1907.10599

Meta Pseudo Labels

Hieu Pham, Zihang Dai, Qizhe Xie, Minh-Thang Luong, Quoc V. Le

2003.10580

Distributed Distributional Deterministic Policy Gradients

Gabriel Barth-Maron, Matthew W. Hoffman, David Budden, Will Dabney, Dan Horgan, Dhruva TB, Alistair Muldal, Nicolas Heess, Timothy Lillicrap

1804.08617

Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data

Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, Michael I. Jordan

1805.12316

Towards Sparse Hierarchical Graph Classifiers

Cătălina Cangea, Petar Veličković, Nikola Jovanović, Thomas Kipf, Pietro Liò

1811.01287

Sparse Networks from Scratch: Faster Training without Losing Performance

Tim Dettmers, Luke Zettlemoyer

1907.04840

Reshaped Wirtinger Flow and Incremental Algorithm for Solving Quadratic System of Equations

Huishuai Zhang, Yi Zhou, Yingbin Liang, Yuejie Chi

1605.07719

CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs

Ahmed El-Kishky, Vishrav Chaudhary, Francisco Guzman, Philipp Koehn

1911.06154

Neural Photo Editing with Introspective Adversarial Networks

Andrew Brock, Theodore Lim, J. M. Ritchie, Nick Weston

1609.07093

On Nesting Monte Carlo Estimators

Tom Rainforth, Robert Cornish, Hongseok Yang, Andrew Warrington, Frank Wood

1709.06181

Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation

Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, Daniel Ulbricht

1903.04064

Few-Shot Learning via Learning the Representation, Provably

Simon S. Du, Wei Hu, Sham M. Kakade, Jason D. Lee, Qi Lei

2002.09434

Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models

Chin-Wei Huang, Laurent Dinh, Aaron Courville

2002.07101

Neural Code Comprehension: A Learnable Representation of Code Semantics

Tal Ben-Nun, Alice Shoshana Jakobovits, Torsten Hoefler

1806.07336

Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics

Michael Neunert, Abbas Abdolmaleki, Markus Wulfmeier, Thomas Lampe, Jost Tobias Springenberg, Roland Hafner, Francesco Romano, Jonas Buchli, Nicolas Heess, Martin Riedmiller

2001.00449

A New Convex Relaxation for Tensor Completion

Bernardino Romera-Paredes, Massimiliano Pontil

1307.4653

Finding Influential Training Samples for Gradient Boosted Decision Trees

Boris Sharchilev, Yury Ustinovsky, Pavel Serdyukov, Maarten de Rijke

1802.06640

Contrastive Multi-View Representation Learning on Graphs

Kaveh Hassani, Amir Hosein Khasahmadi

2006.05582

Capacity and Trainability in Recurrent Neural Networks

Jasmine Collins, Jascha Sohl-Dickstein, David Sussillo

1611.09913

Towards moderate overparameterization: global convergence guarantees for training shallow neural networks

Samet Oymak, Mahdi Soltanolkotabi

1902.04674

Efficient and Modular Implicit Differentiation

Mathieu Blondel, Quentin Berthet, Marco Cuturi, Roy Frostig, Stephan Hoyer, Felipe Llinares-López, Fabian Pedregosa, Jean-Philippe Vert

2105.15183

Understanding the Acceleration Phenomenon via High-Resolution Differential Equations

Bin Shi, Simon S. Du, Michael I. Jordan, Weijie J. Su

1810.08907

Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks

Blake Bordelon, Cengiz Pehlevan

2304.03408

Randomized Positional Encodings Boost Length Generalization of Transformers

Anian Ruoss, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Róbert Csordás, Mehdi Bennani, Shane Legg, Joel Veness

2305.16843

Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases

Ren Wang, Gaoyuan Zhang, Sijia Liu, Pin-Yu Chen, Jinjun Xiong, Meng Wang

2007.15802

A Simple Exponential Family Framework for Zero-Shot Learning

Vinay Kumar Verma, Piyush Rai

1707.08040

Soft Actor-Critic Algorithms and Applications

Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, Sergey Levine

1812.05905

Robust Estimation and Generative Adversarial Nets

Chao Gao, Jiyi Liu, Yuan Yao, Weizhi Zhu

1810.02030

Rolling Diffusion Models

David Ruhe, Jonathan Heek, Tim Salimans, Emiel Hoogeboom

2402.09470

Amortized Inference for Causal Structure Learning

Lars Lorch, Scott Sussex, Jonas Rothfuss, Andreas Krause, Bernhard Schölkopf

2205.12934

Monte Carlo Gradient Estimation in Machine Learning

Shakir Mohamed, Mihaela Rosca, Michael Figurnov, Andriy Mnih

1906.10652

SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient

Lam M. Nguyen, Jie Liu, Katya Scheinberg, Martin Takáč

1703.00102

A Dynamical Model of Neural Scaling Laws

Blake Bordelon, Alexander Atanasov, Cengiz Pehlevan

2402.01092

Bayesian Deep Learning and a Probabilistic Perspective of Generalization

Andrew Gordon Wilson, Pavel Izmailov

2002.08791

Efficient Exploration through Bayesian Deep Q-Networks

Kamyar Azizzadenesheli, Animashree Anandkumar

1802.04412

Contrastive Representation Learning: A Framework and Review

Phuc H. Le-Khac, Graham Healy, Alan F. Smeaton

2010.05113

Representational Strengths and Limitations of Transformers

Clayton Sanford, Daniel Hsu, Matus Telgarsky

2306.02896

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

Qimai Li, Zhichao Han, Xiao-Ming Wu

1801.07606

Stop Wasting My Gradients: Practical SVRG

Reza Babanezhad, Mohamed Osama Ahmed, Alim Virani, Mark Schmidt, Jakub Konečný, Scott Sallinen

1511.01942

GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training

Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, Jie Tang

2006.09963

Graphite: Iterative Generative Modeling of Graphs

Aditya Grover, Aaron Zweig, Stefano Ermon

1803.10459

Saccader: Improving Accuracy of Hard Attention Models for Vision

Gamaleldin F. Elsayed, Simon Kornblith, Quoc V. Le

1908.07644

Streaming automatic speech recognition with the transformer model

Niko Moritz, Takaaki Hori, Jonathan Le Roux

2001.02674

Exploring Model-based Planning with Policy Networks

Tingwu Wang, Jimmy Ba

1906.08649

DiffEqFlux.jl - A Julia Library for Neural Differential Equations

Chris Rackauckas, Mike Innes, Yingbo Ma, Jesse Bettencourt, Lyndon White, Vaibhav Dixit

1902.02376

The Variational Gaussian Process

Dustin Tran, Rajesh Ranganath, David M. Blei

1511.06499

Zero-Shot Learning via Semantic Similarity Embedding

Ziming Zhang, Venkatesh Saligrama

1509.04767

Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimization

Daniel Golovin, Qiuyi Zhang

2006.04655

Training Generative Adversarial Networks by Solving Ordinary Differential Equations

Chongli Qin, Yan Wu, Jost Tobias Springenberg, Andrew Brock, Jeff Donahue, Timothy P. Lillicrap, Pushmeet Kohli

2010.15040

The power of deeper networks for expressing natural functions

David Rolnick, Max Tegmark

1705.05502

TADAM: Task dependent adaptive metric for improved few-shot learning

Boris N. Oreshkin, Pau Rodriguez, Alexandre Lacoste

1805.10123

Community Detection and Stochastic Block Models

Emmanuel Abbe

1703.10146

High-Dimensional Robust Mean Estimation in Nearly-Linear Time

Yu Cheng, Ilias Diakonikolas, Rong Ge

1811.09380

Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model

Tsung-yuan Hsu, Chi-liang Liu, Hung-yi Lee

1909.09587

Adversarial Attacks on Stochastic Bandits

Kwang-Sung Jun, Lihong Li, Yuzhe Ma, Xiaojin Zhu

1810.12188

On the Continuity of Rotation Representations in Neural Networks

Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, Hao Li

1812.07035

Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables

Kate Rakelly, Aurick Zhou, Deirdre Quillen, Chelsea Finn, Sergey Levine

1903.08254

Fairness in Algorithmic Decision Making: An Excursion Through the Lens of Causality

Aria Khademi, Sanghack Lee, David Foley, Vasant Honavar

1903.11719

Uncertainty-guided Continual Learning with Bayesian Neural Networks

Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, Marcus Rohrbach

1906.02425

Deep Learning Scaling is Predictable, Empirically

Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, Yanqi Zhou

1712.00409

Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes

Sebastian W. Ober, Laurence Aitchison

2005.08140

Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning

Runa Eschenhagen, Erik Daxberger, Philipp Hennig, Agustinus Kristiadi

2111.03577

Task Agnostic Continual Learning via Meta Learning

Xu He, Jakub Sygnowski, Alexandre Galashov, Andrei A. Rusu, Yee Whye Teh, Razvan Pascanu

1906.05201

Particle Gibbs with Ancestor Sampling for Probabilistic Programs

Jan-Willem van de Meent, Hongseok Yang, Vikash Mansinghka, Frank Wood

1501.06769

Rademacher Complexity for Adversarially Robust Generalization

Dong Yin, Kannan Ramchandran, Peter Bartlett

1810.11914

A New Approach to Probabilistic Programming Inference

Frank Wood, Jan Willem van de Meent, Vikash Mansinghka

1507.00996

Identifying Mislabeled Data using the Area Under the Margin Ranking

Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, Kilian Q. Weinberger

2001.10528

CNN Architectures for Large-Scale Audio Classification

Shawn Hershey, Sourish Chaudhuri, Daniel P. W. Ellis, Jort F. Gemmeke, Aren Jansen, R. Channing Moore, Manoj Plakal, Devin Platt, Rif A. Saurous, Bryan Seybold, Malcolm Slaney, Ron J. Weiss, Kevin Wilson

1609.09430

Insertion Transformer: Flexible Sequence Generation via Insertion Operations

Mitchell Stern, William Chan, Jamie Kiros, Jakob Uszkoreit

1902.03249

Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs

Woojeong Jin, Meng Qu, Xisen Jin, Xiang Ren

1904.05530

Variational Autoencoders for Collaborative Filtering

Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, Tony Jebara

1802.05814

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model

Alex X. Lee, Anusha Nagabandi, Pieter Abbeel, Sergey Levine

1907.00953

Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics

Vinay V. Ramasesh, Ethan Dyer, Maithra Raghu

2007.07400

Counterfactual Data Augmentation using Locally Factored Dynamics

Silviu Pitis, Elliot Creager, Animesh Garg

2007.02863

Fairness in Learning: Classic and Contextual Bandits

Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth

1605.07139

Observational Scaling Laws and the Predictability of Language Model Performance

Yangjun Ruan, Chris J. Maddison, Tatsunori Hashimoto

2405.10938

Quantifying and Reducing Stereotypes in Word Embeddings

Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai

1606.06121

Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec

Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Kuansan Wang, Jie Tang

1710.02971

Generative Code Modeling with Graphs

Marc Brockschmidt, Miltiadis Allamanis, Alexander L. Gaunt, Oleksandr Polozov

1805.08490

Stochastic Optimization of PCA with Capped MSG

Raman Arora, Andrew Cotter, Nathan Srebro

1307.1674

Support union recovery in high-dimensional multivariate regression

Guillaume Obozinski, Martin J. Wainwright, Michael I. Jordan

0808.0711

Why Is My Classifier Discriminatory?

Irene Chen, Fredrik D. Johansson, David Sontag

1805.12002

On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent

Noah Golmant, Nikita Vemuri, Zhewei Yao, Vladimir Feinberg, Amir Gholami, Kai Rothauge, Michael W. Mahoney, Joseph Gonzalez

1811.12941

Intrinsic dimension of data representations in deep neural networks

Alessio Ansuini, Alessandro Laio, Jakob H. Macke, Davide Zoccolan

1905.12784

Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration

Song Mei, Theodor Misiakiewicz, Andrea Montanari

2101.10588

Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics

Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel L. K. Yamins, Hidenori Tanaka

2012.04728

Meta-Learning for Semi-Supervised Few-Shot Classification

Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenenbaum, Hugo Larochelle, Richard S. Zemel

1803.00676

Lookahead Optimizer: k steps forward, 1 step back

Michael R. Zhang, James Lucas, Geoffrey Hinton, Jimmy Ba

1907.08610

Stochastic blockmodels with growing number of classes

David S. Choi, Patrick J. Wolfe, Edoardo M. Airoldi

1011.4644

Equivariant Flows: sampling configurations for multi-body systems with symmetric energies

Jonas Köhler, Leon Klein, Frank Noé

1910.00753

Reinforcement Learning with Augmented Data

Michael Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, Aravind Srinivas

2004.14990

Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks

Takuhiro Kaneko, Hirokazu Kameoka

1711.11293

Training language GANs from Scratch

Cyprien de Masson d'Autume, Mihaela Rosca, Jack Rae, Shakir Mohamed

1905.09922

DimmWitted: A Study of Main-Memory Statistical Analytics

Ce Zhang, Christopher Ré

1403.7550

Overlearning Reveals Sensitive Attributes

Congzheng Song, Vitaly Shmatikov

1905.11742

Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization

Satrajit Chatterjee

2002.10657

XNAS: Neural Architecture Search with Expert Advice

Niv Nayman, Asaf Noy, Tal Ridnik, Itamar Friedman, Rong Jin, Lihi Zelnik-Manor

1906.08031

A continual learning survey: Defying forgetting in classification tasks

Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory Slabaugh, Tinne Tuytelaars

1909.08383

Sparse Inverse Covariance Selection via Alternating Linearization Methods

Katya Scheinberg, Shiqian Ma, Donald Goldfarb

1011.0097

Online Learning: Sufficient Statistics and the Burkholder Method

Dylan J. Foster, Alexander Rakhlin, Karthik Sridharan

1803.07617

Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations

Andrew Slavin Ross, Michael C. Hughes, Finale Doshi-Velez

1703.03717

DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score

Yuma Koizumi, Kenta Niwa, Yusuke Hioka, Kazunori Kobayashi, Yoichi Haneda

1810.09137

De-identification of Patient Notes with Recurrent Neural Networks

Franck Dernoncourt, Ji Young Lee, Ozlem Uzuner, Peter Szolovits

1606.03475

SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver

Po-Wei Wang, Priya L. Donti, Bryan Wilder, Zico Kolter

1905.12149

Analysis of boosting algorithms using the smooth margin function

Cynthia Rudin, Robert E. Schapire, Ingrid Daubechies

0803.4092

Image Classification at Supercomputer Scale

Chris Ying, Sameer Kumar, Dehao Chen, Tao Wang, Youlong Cheng

1811.06992

Transformation Autoregressive Networks

Junier B. Oliva, Avinava Dubey, Manzil Zaheer, Barnabás Póczos, Ruslan Salakhutdinov, Eric P. Xing, Jeff Schneider

1801.09819

Global analysis of Expectation Maximization for mixtures of two Gaussians

Ji Xu, Daniel Hsu, Arian Maleki

1608.07630

Neural Lyapunov Control

Ya-Chien Chang, Nima Roohi, Sicun Gao

2005.00611

Benchmarking Neural Network Training Algorithms

George E. Dahl, Frank Schneider, Zachary Nado, Naman Agarwal, Chandramouli Shama Sastry, Philipp Hennig, Sourabh Medapati, Runa Eschenhagen, Priya Kasimbeg, Daniel Suo, Juhan Bae, Justin Gilmer, Abel L. Peirson, Bilal Khan, Rohan Anil, Mike Rabbat, Shankar Krishnan, Daniel Snider, Ehsan Amid, Kongtao Chen, Chris J. Maddison, Rakshith Vasudev, Michal Badura, Ankush Garg, Peter Mattson

2306.07179

Parameter-Efficient Transfer Learning for NLP

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, Sylvain Gelly

1902.00751

On the Limitations of Unsupervised Bilingual Dictionary Induction

Anders Søgaard, Sebastian Ruder, Ivan Vulić

1805.03620

Stability and Generalization of Learning Algorithms that Converge to Global Optima

Zachary Charles, Dimitris Papailiopoulos

1710.08402

COLA: Decentralized Linear Learning

Lie He, An Bian, Martin Jaggi

1808.04883

Explainable Neural Networks based on Additive Index Models

Joel Vaughan, Agus Sudjianto, Erind Brahimi, Jie Chen, Vijayan N. Nair

1806.01933

Toward a unified theory of sparse dimensionality reduction in Euclidean space

Jean Bourgain, Sjoerd Dirksen, Jelani Nelson

1311.2542

Transformer-based Online CTC/attention End-to-End Speech Recognition Architecture

Haoran Miao, Gaofeng Cheng, Changfeng Gao, Pengyuan Zhang, Yonghong Yan

2001.08290

DeepMDP: Learning Continuous Latent Space Models for Representation Learning

Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, Marc G. Bellemare

1906.02736

Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation

Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L Smith, Yee Whye Teh

2302.10322

Are Labels Required for Improving Adversarial Robustness?

Jonathan Uesato, Jean-Baptiste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, Pushmeet Kohli

1905.13725

High Accuracy and High Fidelity Extraction of Neural Networks

Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, Nicolas Papernot

1909.01838

SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks

Fabian B. Fuchs, Daniel E. Worrall, Volker Fischer, Max Welling

2006.10503

Certified Robustness to Adversarial Examples with Differential Privacy

Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, Suman Jana

1802.03471

Deep Kernel Learning

Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, Eric P. Xing

1511.02222

Variance Reduction in SGD by Distributed Importance Sampling

Guillaume Alain, Alex Lamb, Chinnadhurai Sankar, Aaron Courville, Yoshua Bengio

1511.06481

A type of generalization error induced by initialization in deep neural networks

Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo, Zheng Ma

1905.07777

Adaptive Graph Encoder for Attributed Graph Embedding

Ganqu Cui, Jie Zhou, Cheng Yang, Zhiyuan Liu

2007.01594

The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems

Weinan E, Bing Yu

1710.00211

Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models

Cheng Lu, Yang Song

2410.11081

Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets

Jakob Runge

2003.03685

Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian

Samet Oymak, Zalan Fabian, Mingchen Li, Mahdi Soltanolkotabi

1906.05392

Metropolis-Hastings Generative Adversarial Networks

Ryan Turner, Jane Hung, Eric Frank, Yunus Saatci, Jason Yosinski

1811.11357

Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation

Raphael Gontijo Lopes, Dong Yin, Ben Poole, Justin Gilmer, Ekin D. Cubuk

1906.02611

Random Feature Expansions for Deep Gaussian Processes

Kurt Cutajar, Edwin V. Bonilla, Pietro Michiardi, Maurizio Filippone

1610.04386

On Estimating Many Means, Selection Bias, and the Bootstrap

Noah Simon, Richard Simon

1311.3709

Gradient Descent Provably Optimizes Over-parameterized Neural Networks

Simon S. Du, Xiyu Zhai, Barnabas Poczos, Aarti Singh

1810.02054

Meta-learning with differentiable closed-form solvers

Luca Bertinetto, João F. Henriques, Philip H. S. Torr, Andrea Vedaldi

1805.08136

GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework

Haotao Wang, Shupeng Gui, Haichuan Yang, Ji Liu, Zhangyang Wang

2008.11062

Graph HyperNetworks for Neural Architecture Search

Chris Zhang, Mengye Ren, Raquel Urtasun

1810.05749

On Tensor Completion via Nuclear Norm Minimization

Ming Yuan, Cun-Hui Zhang

1405.1773

A generative adversarial network approach to calibration of local stochastic volatility models

Christa Cuchiero, Wahid Khosrawi, Josef Teichmann

2005.02505

On the Similarity between the Laplace and Neural Tangent Kernels

Amnon Geifman, Abhay Yadav, Yoni Kasten, Meirav Galun, David Jacobs, Ronen Basri

2007.01580

Gradients as Features for Deep Representation Learning

Fangzhou Mu, Yingyu Liang, Yin Li

2004.05529

Demystifying MMD GANs

Mikołaj Bińkowski, Danica J. Sutherland, Michael Arbel, Arthur Gretton

1801.01401

Differentially Private Diffusion Models Generate Useful Synthetic Images

Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel L. Smith, Olivia Wiles, Borja Balle

2302.13861

An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices with Polynomial Scalings

Yue M. Lu, Horng-Tzer Yau

2205.06308

Optimal detection of sparse principal components in high dimension

Quentin Berthet, Philippe Rigollet

1202.5070

Exponential Lower Bounds for Planning in MDPs With Linearly-Realizable Optimal Action-Value Functions

Gellért Weisz, Philip Amortila, Csaba Szepesvári

2010.01374

Towards Robust Neural Networks via Random Self-ensemble

Xuanqing Liu, Minhao Cheng, Huan Zhang, Cho-Jui Hsieh

1712.00673

On the Convergence and Robustness of Training GANs with Regularized Optimal Transport

Maziar Sanjabi, Jimmy Ba, Meisam Razaviyayn, Jason D. Lee

1802.08249

Optimization as Estimation with Gaussian Processes in Bandit Settings

Zi Wang, Bolei Zhou, Stefanie Jegelka

1510.06423

Robust Accelerated Gradient Methods for Smooth Strongly Convex Functions

Necdet Serhat Aybat, Alireza Fallah, Mert Gurbuzbalaban, Asuman Ozdaglar

1805.10579

AdaCliP: Adaptive Clipping for Private SGD

Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar

1908.07643

Improved Algorithms for Convex-Concave Minimax Optimization

Yuanhao Wang, Jian Li

2006.06359

Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback

Shuai Zheng, Ziyue Huang, James T. Kwok

1905.10936

Advances and Open Problems in Federated Learning

Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, Sen Zhao

1912.04977

ROBEL: Robotics Benchmarks for Learning with Low-Cost Robots

Michael Ahn, Henry Zhu, Kristian Hartikainen, Hugo Ponte, Abhishek Gupta, Sergey Levine, Vikash Kumar

1909.11639

Learning to reinforcement learn

Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, Matt Botvinick

1611.05763

Privacy Amplification via Random Check-Ins

Borja Balle, Peter Kairouz, H. Brendan McMahan, Om Thakkar, Abhradeep Thakurta

2007.06605

On Smoothing and Inference for Topic Models

Arthur Asuncion, Max Welling, Padhraic Smyth, Yee Whye Teh

1205.2662

Efficient Gradient-Based Inference through Transformations between Bayes Nets and Neural Nets

Diederik P. Kingma, Max Welling

1402.0480

SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics

Emmanuel Abbe, Enric Boix-Adsera, Theodor Misiakiewicz

2302.11055

A comparative study of fairness-enhancing interventions in machine learning

Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P. Hamilton, Derek Roth

1802.04422

Adversarial Scene Editing: Automatic Object Removal from Weak Supervision

Rakshith Shetty, Mario Fritz, Bernt Schiele

1806.01911

Multi-Evidence Filtering and Fusion for Multi-Label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning

Weifeng Ge, Sibei Yang, Yizhou Yu

1802.09129

The Three Stages of Learning Dynamics in High-Dimensional Kernel Methods

Nikhil Ghosh, Song Mei, Bin Yu

2111.07167

Implicit Regularization of Random Feature Models

Arthur Jacot, Berfin Şimşek, Francesco Spadaro, Clément Hongler, Franck Gabriel

2002.08404

Nonparametric variational inference

Samuel Gershman, Matt Hoffman, David Blei

1206.4665

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One

Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, Kevin Swersky

1912.03263

Safe Exploration in Finite Markov Decision Processes with Gaussian Processes

Matteo Turchetta, Felix Berkenkamp, Andreas Krause

1606.04753

On exponential convergence of SGD in non-convex over-parametrized learning

Raef Bassily, Mikhail Belkin, Siyuan Ma

1811.02564

Stable Neural Flows

Stefano Massaroli, Michael Poli, Michelangelo Bin, Jinkyoo Park, Atsushi Yamashita, Hajime Asama

2003.08063

Beta processes, stick-breaking, and power laws

Tamara Broderick, Michael I. Jordan, Jim Pitman

1106.0539

Stochastic Frank-Wolfe Methods for Nonconvex Optimization

Sashank J. Reddi, Suvrit Sra, Barnabas Poczos, Alex Smola

1607.08254

Doubly Robust Policy Evaluation and Learning

Miroslav Dudik, John Langford, Lihong Li

1103.4601

Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance

Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin

1611.05817

Deep reinforcement learning from human preferences

Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, Dario Amodei

1706.03741

Learning Stable Deep Dynamics Models

Gaurav Manek, J. Zico Kolter

2001.06116

Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game

Alexander G. Reisach, Christof Seiler, Sebastian Weichwald

2102.13647

Sparse and Low-rank Tensor Estimation via Cubic Sketchings

Botao Hao, Anru Zhang, Guang Cheng

1801.09326

On the Identifiability of the Post-Nonlinear Causal Model

Kun Zhang, Aapo Hyvarinen

1205.2599

Meta-Learning: A Survey

Joaquin Vanschoren

1810.03548

Gradient Coding from Cyclic MDS Codes and Expander Graphs

Netanel Raviv, Itzhak Tamo, Rashish Tandon, Alexandros G. Dimakis

1707.03858

Decentralized Deep Learning with Arbitrary Communication Compression

Anastasia Koloskova, Tao Lin, Sebastian U. Stich, Martin Jaggi

1907.09356

Guided Conditional Diffusion for Controllable Traffic Simulation

Ziyuan Zhong, Davis Rempe, Danfei Xu, Yuxiao Chen, Sushant Veer, Tong Che, Baishakhi Ray, Marco Pavone

2210.17366

Fast Attention Requires Bounded Entries

Josh Alman, Zhao Song

2302.13214

Provable Bounds for Learning Some Deep Representations

Sanjeev Arora, Aditya Bhaskara, Rong Ge, Tengyu Ma

1310.6343

Troubling Trends in Machine Learning Scholarship

Zachary C. Lipton, Jacob Steinhardt

1807.03341

Convergence of score-based generative modeling for general data distributions

Holden Lee, Jianfeng Lu, Yixin Tan

2209.12381

Fast Threshold Tests for Detecting Discrimination

Emma Pierson, Sam Corbett-Davies, Sharad Goel

1702.08536

Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets?

Zhiyuan Li, Yi Zhang, Sanjeev Arora

2010.08515

Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence

Shakir Mohamed, Marie-Therese Png, William Isaac

2007.04068

Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models

Wieland Brendel, Jonas Rauber, Matthias Bethge

1712.04248

V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RL

Chi Jin, Qinghua Liu, Yuanhao Wang, Tiancheng Yu

2110.14555

Fast Adaptation with Linearized Neural Networks

Wesley J. Maddox, Shuai Tang, Pablo Garcia Moreno, Andrew Gordon Wilson, Andreas Damianou

2103.01439

Soft Threshold Weight Reparameterization for Learnable Sparsity

Aditya Kusupati, Vivek Ramanujan, Raghav Somani, Mitchell Wortsman, Prateek Jain, Sham Kakade, Ali Farhadi

2002.03231

On the training dynamics of deep networks with $L_2$ regularization

Aitor Lewkowycz, Guy Gur-Ari

2006.08643

On the Local Minima of the Empirical Risk

Chi Jin, Lydia T. Liu, Rong Ge, Michael I. Jordan

1803.09357

Symmetric Cross Entropy for Robust Learning with Noisy Labels

Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, James Bailey

1908.06112

Batch Bayesian Optimization via Local Penalization

Javier González, Zhenwen Dai, Philipp Hennig, Neil D. Lawrence

1505.08052

Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks

Kun Xu, Lingfei Wu, Zhiguo Wang, Yansong Feng, Michael Witbrock, Vadim Sheinin

1804.00823

Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance

Neal Jean, Sang Michael Xie, Stefano Ermon

1805.10407

AutoAugment: Learning Augmentation Policies from Data

Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, Quoc V. Le

1805.09501

One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers

Ari S. Morcos, Haonan Yu, Michela Paganini, Yuandong Tian

1906.02773

Diagnosing and Enhancing VAE Models

Bin Dai, David Wipf

1903.05789

Simple rules for complex decisions

Jongbin Jung, Connor Concannon, Ravi Shroff, Sharad Goel, Daniel G. Goldstein

1702.04690

Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate Label Spaces

Isabelle Augenstein, Sebastian Ruder, Anders Søgaard

1802.09913

Certifying Some Distributional Robustness with Principled Adversarial Training

Aman Sinha, Hongseok Namkoong, Riccardo Volpi, John Duchi

1710.10571

Topology and Geometry of Half-Rectified Network Optimization

C. Daniel Freeman, Joan Bruna

1611.01540

SparseMAP: Differentiable Sparse Structured Inference

Vlad Niculae, André F. T. Martins, Mathieu Blondel, Claire Cardie

1802.04223

There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average

Ben Athiwaratkun, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson

1806.05594

Entity Abstraction in Visual Model-Based Reinforcement Learning

Rishi Veerapaneni, John D. Co-Reyes, Michael Chang, Michael Janner, Chelsea Finn, Jiajun Wu, Joshua B. Tenenbaum, Sergey Levine

1910.12827

Seq2Slate: Re-ranking and Slate Optimization with RNNs

Irwan Bello, Sayali Kulkarni, Sagar Jain, Craig Boutilier, Ed Chi, Elad Eban, Xiyang Luo, Alan Mackey, Ofer Meshi

1810.02019

A comprehensive, application-oriented study of catastrophic forgetting in DNNs

B. Pfülb, A. Gepperth

1905.08101

A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning

Marco Fraccaro, Simon Kamronn, Ulrich Paquet, Ole Winther

1710.05741

Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

Iulian Vlad Serban, Tim Klinger, Gerald Tesauro, Kartik Talamadupula, Bowen Zhou, Yoshua Bengio, Aaron Courville

1606.00776

A Neural Scaling Law from the Dimension of the Data Manifold

Utkarsh Sharma, Jared Kaplan

2004.10802

Convergence of Adversarial Training in Overparametrized Neural Networks

Ruiqi Gao, Tianle Cai, Haochuan Li, Liwei Wang, Cho-Jui Hsieh, Jason D. Lee

1906.07916

Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates

Sharan Vaswani, Aaron Mishkin, Issam Laradji, Mark Schmidt, Gauthier Gidel, Simon Lacoste-Julien

1905.09997

Variational inference for Monte Carlo objectives

Andriy Mnih, Danilo J. Rezende

1602.06725

Temporal Pointwise Convolutional Networks for Length of Stay Prediction in the Intensive Care Unit

Emma Rocheteau, Pietro Liò, Stephanie Hyland

2007.09483

Controlling Fairness and Bias in Dynamic Learning-to-Rank

Marco Morik, Ashudeep Singh, Jessica Hong, Thorsten Joachims

2005.14713

A Tensor Approach to Learning Mixed Membership Community Models

Anima Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade

1302.2684

Learning time-scales in two-layers neural networks

Raphaël Berthier, Andrea Montanari, Kangjie Zhou

2303.00055

Learning Stability Certificates from Data

Nicholas M. Boffi, Stephen Tu, Nikolai Matni, Jean-Jacques E. Slotine, Vikas Sindhwani

2008.05952

Averaging Weights Leads to Wider Optima and Better Generalization

Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, Andrew Gordon Wilson

1803.05407

A Sharp Analysis of Model-based Reinforcement Learning with Self-Play

Qinghua Liu, Tiancheng Yu, Yu Bai, Chi Jin

2010.01604

Towards Robust Interpretability with Self-Explaining Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

1806.07538

Black-Box Reductions for Parameter-free Online Learning in Banach Spaces

Ashok Cutkosky, Francesco Orabona

1802.06293

MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction

Yuning Chai, Benjamin Sapp, Mayank Bansal, Dragomir Anguelov

1910.05449

Minimax Rates of Estimation for Sparse PCA in High Dimensions

Vincent Q. Vu, Jing Lei

1202.0786

A Closer Look at Deep Policy Gradients

Andrew Ilyas, Logan Engstrom, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, Aleksander Madry

1811.02553

Doubly Stochastic Variational Inference for Deep Gaussian Processes

Hugh Salimbeni, Marc Deisenroth

1705.08933

A Simple Neural Attentive Meta-Learner

Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel

1707.03141

LEEP: A New Measure to Evaluate Transferability of Learned Representations

Cuong V. Nguyen, Tal Hassner, Matthias Seeger, Cedric Archambeau

2002.12462

Sparse Quantized Spectral Clustering

Zhenyu Liao, Romain Couillet, Michael W. Mahoney

2010.01376

Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds

Raef Bassily, Adam Smith, Abhradeep Thakurta

1405.7085

Coordinate Friendly Structures, Algorithms and Applications

Zhimin Peng, Tianyu Wu, Yangyang Xu, Ming Yan, Wotao Yin

1601.00863

Triple descent and the two kinds of overfitting: Where & why do they appear?

Stéphane d'Ascoli, Levent Sagun, Giulio Biroli

2006.03509

DAWN: Dynamic Adversarial Watermarking of Neural Networks

Sebastian Szyller, Buse Gul Atli, Samuel Marchal, N. Asokan

1906.00830

Hinge-Loss Markov Random Fields and Probabilistic Soft Logic

Stephen H. Bach, Matthias Broecheler, Bert Huang, Lise Getoor

1505.04406

Watch Your Step: Learning Node Embeddings via Graph Attention

Sami Abu-El-Haija, Bryan Perozzi, Rami Al-Rfou, Alex Alemi

1710.09599

Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo

Markus Heinonen, Henrik Mannerström, Juho Rousu, Samuel Kaski, Harri Lähdesmäki

1508.04319

Sorting out Lipschitz function approximation

Cem Anil, James Lucas, Roger Grosse

1811.05381

Exact expressions for double descent and implicit regularization via surrogate random design

Michał Dereziński, Feynman Liang, Michael W. Mahoney

1912.04533

Fast Stochastic Methods for Nonsmooth Nonconvex Optimization

Sashank J. Reddi, Suvrit Sra, Barnabas Poczos, Alex Smola

1605.06900

A Survey on Methods and Theories of Quantized Neural Networks

Yunhui Guo

1808.04752

REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models

George Tucker, Andriy Mnih, Chris J. Maddison, Dieterich Lawson, Jascha Sohl-Dickstein

1703.07370

Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets

Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Sanjeev Arora, Rong Ge

1906.06247

Frequentist Consistency of Variational Bayes

Yixin Wang, David M. Blei

1705.03439

Automatic Program Synthesis of Long Programs with a Learned Garbage Collector

Amit Zohar, Lior Wolf

1809.04682

Toward Understanding Catastrophic Forgetting in Continual Learning

Cuong V. Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, Stefano Soatto

1908.01091

Explaining Predictions of Non-Linear Classifiers in NLP

Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Müller, Wojciech Samek

1606.07298

Learning from Noisy Labels with Distillation

Yuncheng Li, Jianchao Yang, Yale Song, Liangliang Cao, Jiebo Luo, Li-Jia Li

1703.02391

Revisiting Small Batch Training for Deep Neural Networks

Dominic Masters, Carlo Luschi

1804.07612

Adversarial Training Methods for Semi-Supervised Text Classification

Takeru Miyato, Andrew M. Dai, Ian Goodfellow

1605.07725

Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows

Ruizhi Deng, Bo Chang, Marcus A. Brubaker, Greg Mori, Andreas Lehrmann

2002.10516

Sparse Stochastic Inference for Latent Dirichlet allocation

David Mimno, Matt Hoffman, David Blei

1206.6425

EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL

Seyed Kamyar Seyed Ghasemipour, Dale Schuurmans, Shixiang Shane Gu

2007.11091

Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization

Jun Suzuki, Masaaki Nagata

1701.00138

Soft-Label Dataset Distillation and Text Dataset Distillation

Ilia Sucholutsky, Matthias Schonlau

1910.02551

Geometric descent method for convex composite minimization

Shixiang Chen, Shiqian Ma, Wei Liu

1612.09034

What is the Effect of Importance Weighting in Deep Learning?

Jonathon Byrd, Zachary C. Lipton

1812.03372

A Selective Overview of Deep Learning

Jianqing Fan, Cong Ma, Yiqiao Zhong

1904.05526

Semidefinite relaxations for certifying robustness to adversarial examples

Aditi Raghunathan, Jacob Steinhardt, Percy Liang

1811.01057

Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity

Úlfar Erlingsson, Vitaly Feldman, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Abhradeep Thakurta

1811.12469

Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes

Jerry Yao-Chieh Hu, Dennis Wu, Han Liu

2410.23126

Adaptive Aggregation Networks for Class-Incremental Learning

Yaoyao Liu, Bernt Schiele, Qianru Sun

2010.05063

Explanations can be manipulated and geometry is to blame

Ann-Kathrin Dombrowski, Maximilian Alber, Christopher J. Anders, Marcel Ackermann, Klaus-Robert Müller, Pan Kessel

1906.07983

Curiosity Driven Exploration of Learned Disentangled Goal Spaces

Adrien Laversanne-Finot, Alexandre Péré, Pierre-Yves Oudeyer

1807.01521

Automatic Variational Inference in Stan

Alp Kucukelbir, Rajesh Ranganath, Andrew Gelman, David M. Blei

1506.03431

On Uniform Convergence and Low-Norm Interpolation Learning

Lijia Zhou, Danica J. Sutherland, Nathan Srebro

2006.05942

Why do tree-based models still outperform deep learning on tabular data?

Léo Grinsztajn, Edouard Oyallon, Gaël Varoquaux

2207.08815

Learning Non-overlapping Convolutional Neural Networks with Multiple Kernels

Kai Zhong, Zhao Song, Inderjit S. Dhillon

1711.03440

PyTorch-BigGraph: A Large-scale Graph Embedding System

Adam Lerer, Ledell Wu, Jiajun Shen, Timothee Lacroix, Luca Wehrstedt, Abhijit Bose, Alex Peysakhovich

1903.12287

Measuring the Reliability of Reinforcement Learning Algorithms

Stephanie C. Y. Chan, Samuel Fishman, John Canny, Anoop Korattikara, Sergio Guadarrama

1912.05663

Overfitting in adversarially robust deep learning

Leslie Rice, Eric Wong, J. Zico Kolter

2002.11569

Towards Differentially Private Text Representations

Lingjuan Lyu, Yitong Li, Xuanli He, Tong Xiao

2006.14170

What is being transferred in transfer learning?

Behnam Neyshabur, Hanie Sedghi, Chiyuan Zhang

2008.11687

Batch Value-function Approximation with Only Realizability

Tengyang Xie, Nan Jiang

2008.04990

Bayesian nonparametric Plackett-Luce models for the analysis of preferences for college degree programmes

François Caron, Yee Whye Teh, Thomas Brendan Murphy

1211.5037

Nonconvex Matrix Factorization from Rank-One Measurements

Yuanxin Li, Cong Ma, Yuxin Chen, Yuejie Chi

1802.06286

On Completeness-aware Concept-Based Explanations in Deep Neural Networks

Chih-Kuan Yeh, Been Kim, Sercan O. Arik, Chun-Liang Li, Tomas Pfister, Pradeep Ravikumar

1910.07969

Jointly Measuring Diversity and Quality in Text Generation Models

Ehsan Montahaei, Danial Alihosseini, Mahdieh Soleymani Baghshah

1904.03971

The Break-Even Point on Optimization Trajectories of Deep Neural Networks

Stanislaw Jastrzebski, Maciej Szymczak, Stanislav Fort, Devansh Arpit, Jacek Tabor, Kyunghyun Cho, Krzysztof Geras

2002.09572

Deep context: end-to-end contextual speech recognition

Golan Pundak, Tara N. Sainath, Rohit Prabhavalkar, Anjuli Kannan, Ding Zhao

1808.02480

Random Feature Amplification: Feature Learning and Generalization in Neural Networks

Spencer Frei, Niladri S. Chatterji, Peter L. Bartlett

2202.07626

On the Global Linear Convergence of Frank-Wolfe Optimization Variants

Simon Lacoste-Julien, Martin Jaggi

1511.05932

Self-Supervised Policy Adaptation during Deployment

Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Alenyà, Pieter Abbeel, Alexei A. Efros, Lerrel Pinto, Xiaolong Wang

2007.04309

REPAIR: REnormalizing Permuted Activations for Interpolation Repair

Keller Jordan, Hanie Sedghi, Olga Saukh, Rahim Entezari, Behnam Neyshabur

2211.08403

Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness

Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, Xue Lin

2005.00060

Differentiable Ranks and Sorting using Optimal Transport

Marco Cuturi, Olivier Teboul, Jean-Philippe Vert

1905.11885

Lost Relatives of the Gumbel Trick

Matej Balog, Nilesh Tripuraneni, Zoubin Ghahramani, Adrian Weller

1706.04161

Lipschitz Adaptivity with Multiple Learning Rates in Online Learning

Zakaria Mhammedi, Wouter M. Koolen, Tim van Erven

1902.10797

Gradient Coding

Rashish Tandon, Qi Lei, Alexandros G. Dimakis, Nikos Karampatziakis

1612.03301

Invertible Generative Modeling using Linear Rational Splines

Hadi M. Dolatabadi, Sarah Erfani, Christopher Leckie

2001.05168

Learning Data Manipulation for Augmentation and Weighting

Zhiting Hu, Bowen Tan, Ruslan Salakhutdinov, Tom Mitchell, Eric P. Xing

1910.12795

ASAGA: Asynchronous Parallel SAGA

Rémi Leblond, Fabian Pedregosa, Simon Lacoste-Julien

1606.04809

Interpolation Consistency Training for Semi-Supervised Learning

Vikas Verma, Kenji Kawaguchi, Alex Lamb, Juho Kannala, Arno Solin, Yoshua Bengio, David Lopez-Paz

1903.03825

Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification

Xiaoyu Cao, Neil Zhenqiang Gong

1709.05583

The Road Less Scheduled

Aaron Defazio, Xingyu Alice Yang, Harsh Mehta, Konstantin Mishchenko, Ahmed Khaled, Ashok Cutkosky

2405.15682

A Transformer-based Approach for Source Code Summarization

Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang

2005.00653

Compressing Large-Scale Transformer-Based Models: A Case Study on BERT

Prakhar Ganesh, Yao Chen, Xin Lou, Mohammad Ali Khan, Yin Yang, Hassan Sajjad, Preslav Nakov, Deming Chen, Marianne Winslett

2002.11985

Deep Lattice Networks and Partial Monotonic Functions

Seungil You, David Ding, Kevin Canini, Jan Pfeifer, Maya Gupta

1709.06680

OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage

Raj Rao Nadakuditi

1306.6042

The Limitations of Adversarial Training and the Blind-Spot Attack

Huan Zhang, Hongge Chen, Zhao Song, Duane Boning, Inderjit S. Dhillon, Cho-Jui Hsieh

1901.04684

Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlators

Anindita Maiti, Keegan Stoner, James Halverson

2106.00694

The Pitfalls of Simplicity Bias in Neural Networks

Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, Praneeth Netrapalli

2006.07710

Universal features of price formation in financial markets: perspectives from Deep Learning

Justin Sirignano, Rama Cont

1803.06917

Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks

Shiyu Liang, Yixuan Li, R. Srikant

1706.02690

Practical Obstacles to Deploying Active Learning

David Lowell, Zachary C. Lipton, Byron C. Wallace

1807.04801

Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks

Ruth Fong, Andrea Vedaldi

1801.03454

The Depth-to-Width Interplay in Self-Attention

Yoav Levine, Noam Wies, Or Sharir, Hofit Bata, Amnon Shashua

2006.12467

Gradient Descent for One-Hidden-Layer Neural Networks: Polynomial Convergence and SQ Lower Bounds

Santosh Vempala, John Wilmes

1805.02677

Deep End-to-end Causal Inference

Tomas Geffner, Javier Antoran, Adam Foster, Wenbo Gong, Chao Ma, Emre Kiciman, Amit Sharma, Angus Lamb, Martin Kukla, Nick Pawlowski, Miltiadis Allamanis, Cheng Zhang

2202.02195

How is Distributed ADMM Affected by Network Topology?

Guilherme França, José Bento

1710.00889

Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery

Cun Mu, Bo Huang, John Wright, Donald Goldfarb

1307.5870

Temporal Graph Networks for Deep Learning on Dynamic Graphs

Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, Michael Bronstein

2006.10637

Greedy Layerwise Learning Can Scale to ImageNet

Eugene Belilovsky, Michael Eickenberg, Edouard Oyallon

1812.11446

Improving Variational Auto-Encoders using Householder Flow

Jakub M. Tomczak, Max Welling

1611.09630

Max-value Entropy Search for Efficient Bayesian Optimization

Zi Wang, Stefanie Jegelka

1703.01968

Nonperturbative renormalization for the neural network-QFT correspondence

Harold Erbin, Vincent Lahoche, Dine Ousmane Samary

2108.01403

Correlational Neural Networks

Sarath Chandar, Mitesh M. Khapra, Hugo Larochelle, Balaraman Ravindran

1504.07225

ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector

Shang-Tse Chen, Cory Cornelius, Jason Martin, Duen Horng Chau

1804.05810

Variational Continual Learning

Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, Richard E. Turner

1710.10628

Duality of Graphical Models and Tensor Networks

Elina Robeva, Anna Seigal

1710.01437

Neural Tree Indexers for Text Understanding

Tsendsuren Munkhdalai, Hong Yu

1607.04492

On the Neural Tangent Kernel of Deep Networks with Orthogonal Initialization

Wei Huang, Weitao Du, Richard Yi Da Xu

2004.05867

Variational Inference using Implicit Distributions

Ferenc Huszár

1702.08235

Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning

Shakir Mohamed, Danilo Jimenez Rezende

1509.08731

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Sepp Hochreiter

1706.08500

Some methods for heterogeneous treatment effect estimation in high-dimensions

Scott Powers, Junyang Qian, Kenneth Jung, Alejandro Schuler, Nigam H. Shah, Trevor Hastie, Robert Tibshirani

1707.00102

Dataset Inference: Ownership Resolution in Machine Learning

Pratyush Maini, Mohammad Yaghini, Nicolas Papernot

2104.10706

Preconditioning Kernel Matrices

Kurt Cutajar, Michael A. Osborne, John P. Cunningham, Maurizio Filippone

1602.06693

Provably Efficient Reinforcement Learning with Linear Function Approximation

Chi Jin, Zhuoran Yang, Zhaoran Wang, Michael I. Jordan

1907.05388

Scalable Private Learning with PATE

Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Úlfar Erlingsson

1802.08908

Bayesian Neural Network Priors Revisited

Vincent Fortuin, Adrià Garriga-Alonso, Sebastian W. Ober, Florian Wenzel, Gunnar Rätsch, Richard E. Turner, Mark van der Wilk, Laurence Aitchison

2102.06571

Hierarchical structure and the prediction of missing links in networks

Aaron Clauset, Cristopher Moore, M. E. J. Newman

0811.0484

Sample Complexity of Episodic Fixed-Horizon Reinforcement Learning

Christoph Dann, Emma Brunskill

1510.08906

Learning Invariant Representations for Reinforcement Learning without Reconstruction

Amy Zhang, Rowan McAllister, Roberto Calandra, Yarin Gal, Sergey Levine

2006.10742

Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint

Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, Tong Zhang

2312.11456

Barrier Frank-Wolfe for Marginal Inference

Rahul G. Krishnan, Simon Lacoste-Julien, David Sontag

1511.02124

Efficient and Practical Stochastic Subgradient Descent for Nuclear Norm Regularization

Haim Avron, Satyen Kale, Shiva Kasiviswanathan, Vikas Sindhwani

1206.6384

Semi-Autoregressive Training Improves Mask-Predict Decoding

Marjan Ghazvininejad, Omer Levy, Luke Zettlemoyer

2001.08785

Taking Advantage of Sparsity in Multi-Task Learning

Karim Lounici, Massimiliano Pontil, Alexandre B. Tsybakov, Sara van de Geer

0903.1468

Causal Inference on Discrete Data using Additive Noise Models

Jonas Peters, Dominik Janzing, Bernhard Schölkopf

0911.0280

PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network

Zichao Long, Yiping Lu, Bin Dong

1812.04426

Training independent subnetworks for robust prediction

Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshminarayanan, Andrew M. Dai, Dustin Tran

2010.06610

The limits of min-max optimization algorithms: convergence to spurious non-critical sets

Ya-Ping Hsieh, Panayotis Mertikopoulos, Volkan Cevher

2006.09065

Near-Optimal Joint Object Matching via Convex Relaxation

Yuxin Chen, Leonidas J. Guibas, Qi-Xing Huang

1402.1473

Zero-shot Knowledge Transfer via Adversarial Belief Matching

Paul Micaelli, Amos Storkey

1905.09768

Property Inference Attacks Against GANs

Junhao Zhou, Yufei Chen, Chao Shen, Yang Zhang

2111.07608

Convergence Analysis of Inexact Randomized Iterative Methods

Nicolas Loizou, Peter Richtárik

1903.07971

Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences

Borja Balle, Gilles Barthe, Marco Gaboardi

1807.01647

Learning Parities with Neural Networks

Amit Daniely, Eran Malach

2002.07400

Bayesian nonparametric models for ranked data

Francois Caron, Yee Whye Teh

1211.4321

A Continuous-Time View of Early Stopping for Least Squares

Alnur Ali, J. Zico Kolter, Ryan J. Tibshirani

1810.10082

Last-iterate Convergence of Decentralized Optimistic Gradient Descent/Ascent in Infinite-horizon Competitive Markov Games

Chen-Yu Wei, Chung-Wei Lee, Mengxiao Zhang, Haipeng Luo

2102.04540

Stochastic bandits robust to adversarial corruptions

Thodoris Lykouris, Vahab Mirrokni, Renato Paes Leme

1803.09353

Fundamental Limits of Ridge-Regularized Empirical Risk Minimization in High Dimensions

Hossein Taheri, Ramtin Pedarsani, Christos Thrampoulidis

2006.08917

Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision

Kuan Fang, Yuke Zhu, Animesh Garg, Andrey Kurenkov, Viraj Mehta, Li Fei-Fei, Silvio Savarese

1806.09266

Approximate is Good Enough: Probabilistic Variants of Dimensional and Margin Complexity

Pritish Kamath, Omar Montasser, Nathan Srebro

2003.04180

Monte-Carlo Tree Search as Regularized Policy Optimization

Jean-Bastien Grill, Florent Altché, Yunhao Tang, Thomas Hubert, Michal Valko, Ioannis Antonoglou, Rémi Munos

2007.12509

A Multi-Batch L-BFGS Method for Machine Learning

Albert S. Berahas, Jorge Nocedal, Martin Takáč

1605.06049

Covering Numbers for Convex Functions

Adityanand Guntuboyina, Bodhisattva Sen

1204.0147

NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization

Davood Hajinezhad, Mingyi Hong, Tuo Zhao, Zhaoran Wang

1605.07747

Online Stochastic Gradient Descent with Arbitrary Initialization Solves Non-smooth, Non-convex Phase Retrieval

Yan Shuo Tan, Roman Vershynin

1910.12837

Failures of Gradient-Based Deep Learning

Shai Shalev-Shwartz, Ohad Shamir, Shaked Shammah

1703.07950

Deep Hedging: Learning to Simulate Equity Option Markets

Magnus Wiese, Lianjun Bai, Ben Wood, Hans Buehler

1911.01700

GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series

Edward De Brouwer, Jaak Simm, Adam Arany, Yves Moreau

1905.12374

Entropy Search for Information-Efficient Global Optimization

Philipp Hennig, Christian J. Schuler

1112.1217

Learning To Simulate

Nataniel Ruiz, Samuel Schulter, Manmohan Chandraker

1810.02513

BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning

Abhijit Guha Roy, Shayan Siddiqui, Sebastian Pölsterl, Nassir Navab, Christian Wachinger

1905.06731

Hyperbolic Discounting and Learning over Multiple Horizons

William Fedus, Carles Gelada, Yoshua Bengio, Marc G. Bellemare, Hugo Larochelle

1902.06865

MOReL : Model-Based Offline Reinforcement Learning

Rahul Kidambi, Aravind Rajeswaran, Praneeth Netrapalli, Thorsten Joachims

2005.05951

Optimal Best Arm Identification with Fixed Confidence

Aurélien Garivier, Emilie Kaufmann

1602.04589

Provable Tensor Factorization with Missing Data

Prateek Jain, Sewoong Oh

1406.2784

Universal Planning Networks

Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine, Chelsea Finn

1804.00645

Individualized Rank Aggregation using Nuclear Norm Regularization

Yu Lu, Sahand N. Negahban

1410.0860

Meta-Learning Probabilistic Inference For Prediction

Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, Richard E. Turner

1805.09921

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning

Qing Li, Siyuan Huang, Yining Hong, Yixin Chen, Ying Nian Wu, Song-Chun Zhu

2006.06649

On the Expressive Power of Deep Polynomial Neural Networks

Joe Kileel, Matthew Trager, Joan Bruna

1905.12207

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)

Cedric Gerbelot, Alia Abbara, Florent Krzakala

2006.06581

New insights and perspectives on the natural gradient method

James Martens

1412.1193

How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding

Yuchen Li, Yuanzhi Li, Andrej Risteski

2303.04245

Finite-Sum Smooth Optimization with SARAH

Lam M. Nguyen, Marten van Dijk, Dzung T. Phan, Phuong Ha Nguyen, Tsui-Wei Weng, Jayant R. Kalagnanam

1901.07648

Deep Learning for Sentiment Analysis : A Survey

Lei Zhang, Shuai Wang, Bing Liu

1801.07883

Learning to Compose Visual Relations

Nan Liu, Shuang Li, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba

2111.09297

Efficient and Accurate Gradients for Neural SDEs

Patrick Kidger, James Foster, Xuechen Li, Terry Lyons

2105.13493

Fighting Quantization Bias With Bias

Alexander Finkelstein, Uri Almog, Mark Grobman

1906.03193

Q* Approximation Schemes for Batch Reinforcement Learning: A Theoretical Comparison

Tengyang Xie, Nan Jiang

2003.03924

Mixture Martingales Revisited with Applications to Sequential Tests and Confidence Intervals

Emilie Kaufmann, Wouter Koolen

1811.11419

How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks

Divyansh Kaushik, Zachary C. Lipton

1808.04926

Learning Predictive Representations for Deformable Objects Using Contrastive Estimation

Wilson Yan, Ashwin Vangipuram, Pieter Abbeel, Lerrel Pinto

2003.05436

Algorithms and Theory for Multiple-Source Adaptation

Judy Hoffman, Mehryar Mohri, Ningshan Zhang

1805.08727

Retrosynthetic reaction prediction using neural sequence-to-sequence models

Bowen Liu, Bharath Ramsundar, Prasad Kawthekar, Jade Shi, Joseph Gomes, Quang Luu Nguyen, Stephen Ho, Jack Sloane, Paul Wender, Vijay Pande

1706.01643

Adversarial Perturbations Against Real-Time Video Classification Systems

Shasha Li, Ajaya Neupane, Sujoy Paul, Chengyu Song, Srikanth V. Krishnamurthy, Amit K. Roy Chowdhury, Ananthram Swami

1807.00458

Hyperparameter Ensembles for Robustness and Uncertainty Quantification

Florian Wenzel, Jasper Snoek, Dustin Tran, Rodolphe Jenatton

2006.13570

Never Give Up: Learning Directed Exploration Strategies

Adrià Puigdomènech Badia, Pablo Sprechmann, Alex Vitvitskyi, Daniel Guo, Bilal Piot, Steven Kapturowski, Olivier Tieleman, Martín Arjovsky, Alexander Pritzel, Andew Bolt, Charles Blundell

2002.06038

Regularized Learning for Domain Adaptation under Label Shifts

Kamyar Azizzadenesheli, Anqi Liu, Fanny Yang, Animashree Anandkumar

1903.09734

Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel

Colin Wei, Jason D. Lee, Qiang Liu, Tengyu Ma

1810.05369

Community Detection in Networks using Graph Distance

Sharmodeep Bhattacharyya, Peter J. Bickel

1401.3915

Adaptive Gradient-Based Meta-Learning Methods

Mikhail Khodak, Maria-Florina Balcan, Ameet Talwalkar

1906.02717

Can Adversarially Robust Learning Leverage Computational Hardness?

Saeed Mahloujifar, Mohammad Mahmoody

1810.01407

Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation

Niels Justesen, Ruben Rodriguez Torrado, Philip Bontrager, Ahmed Khalifa, Julian Togelius, Sebastian Risi

1806.10729

Graph Element Networks: adaptive, structured computation and memory

Ferran Alet, Adarsh K. Jeewajee, Maria Bauza, Alberto Rodriguez, Tomas Lozano-Perez, Leslie Pack Kaelbling

1904.09019

Constrained Reinforcement Learning Has Zero Duality Gap

Santiago Paternain, Luiz F. O. Chamon, Miguel Calvo-Fullana, Alejandro Ribeiro

1910.13393

Diffusion Improves Graph Learning

Johannes Gasteiger, Stefan Weißenberger, Stephan Günnemann

1911.05485

Generalized Energy Based Models

Michael Arbel, Liang Zhou, Arthur Gretton

2003.05033

Efficient Pure Exploration for Combinatorial Bandits with Semi-Bandit Feedback

Marc Jourdan, Mojmír Mutný, Johannes Kirschner, Andreas Krause

2101.08534

Multimodal Web Navigation with Instruction-Finetuned Foundation Models

Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur

2305.11854

Concrete Autoencoders for Differentiable Feature Selection and Reconstruction

Abubakar Abid, Muhammad Fatih Balin, James Zou

1901.09346

Stealing Hyperparameters in Machine Learning

Binghui Wang, Neil Zhenqiang Gong

1802.05351

SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training

Gowthami Somepalli, Micah Goldblum, Avi Schwarzschild, C. Bayan Bruss, Tom Goldstein

2106.01342

A Causal Lens for Controllable Text Generation

Zhiting Hu, Li Erran Li

2201.09119

Deep Probabilistic Programming

Dustin Tran, Matthew D. Hoffman, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, David M. Blei

1701.03757

A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method

Simon Lacoste-Julien, Mark Schmidt, Francis Bach

1212.2002

Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't

Weinan E, Chao Ma, Stephan Wojtowytsch, Lei Wu

2009.10713

Understanding Why Neural Networks Generalize Well Through GSNR of Parameters

Jinlong Liu, Guoqing Jiang, Yunzhi Bai, Ting Chen, Huayan Wang

2001.07384

Reducing the variance in online optimization by transporting past gradients

Sébastien M. R. Arnold, Pierre-Antoine Manzagol, Reza Babanezhad, Ioannis Mitliagkas, Nicolas Le Roux

1906.03532

Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers

Colin Wei, Yining Chen, Tengyu Ma

2107.13163

Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction

Aviral Kumar, Justin Fu, George Tucker, Sergey Levine

1906.00949

Statistical Inference for Cluster Trees

Jisu Kim, Yen-Chi Chen, Sivaraman Balakrishnan, Alessandro Rinaldo, Larry Wasserman

1605.06416

Just Train Twice: Improving Group Robustness without Training Group Information

Evan Zheran Liu, Behzad Haghgoo, Annie S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, Chelsea Finn

2107.09044

Posterior Sampling by Combining Diffusion Models with Annealed Langevin Dynamics

Zhiyang Xun, Shivam Gupta, Eric Price

2510.26324

Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning

Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, Yang Zhang

1904.01067

Rényi Fair Inference

Sina Baharlouei, Maher Nouiehed, Ahmad Beirami, Meisam Razaviyayn

1906.12005

Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning

Rishabh Agarwal, Marlos C. Machado, Pablo Samuel Castro, Marc G. Bellemare

2101.05265

Are adversarial examples inevitable?

Ali Shafahi, W. Ronny Huang, Christoph Studer, Soheil Feizi, Tom Goldstein

1809.02104

Quality of Uncertainty Quantification for Bayesian Neural Network Inference

Jiayu Yao, Weiwei Pan, Soumya Ghosh, Finale Doshi-Velez

1906.09686

Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)

Anshumali Shrivastava, Ping Li

1405.5869

On Feature Learning in the Presence of Spurious Correlations

Pavel Izmailov, Polina Kirichenko, Nate Gruver, Andrew Gordon Wilson

2210.11369

NAS evaluation is frustratingly hard

Antoine Yang, Pedro M. Esperança, Fabio M. Carlucci

1912.12522

When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models?

Tengyu Xu, Yi Zhou, Kaiyi Ji, Yingbin Liang

1806.04339

Data augmentation in Bayesian neural networks and the cold posterior effect

Seth Nabarro, Stoil Ganev, Adrià Garriga-Alonso, Vincent Fortuin, Mark van der Wilk, Laurence Aitchison

2106.05586

An Instability in Variational Inference for Topic Models

Behrooz Ghorbani, Hamid Javadi, Andrea Montanari

1802.00568

Integer Discrete Flows and Lossless Compression

Emiel Hoogeboom, Jorn W. T. Peters, Rianne van den Berg, Max Welling

1905.07376

A Theoretical Analysis of Contrastive Unsupervised Representation Learning

Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, Nikunj Saunshi

1902.09229

Box Drawings for Learning with Imbalanced Data

Siong Thye Goh, Cynthia Rudin

1403.3378

Speaker Diarization with LSTM

Quan Wang, Carlton Downey, Li Wan, Philip Andrew Mansfield, Ignacio Lopez Moreno

1710.10468

Real-Time Adaptive Image Compression

Oren Rippel, Lubomir Bourdev

1705.05823

MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment

Hao-Wen Dong, Wen-Yi Hsiao, Li-Chia Yang, Yi-Hsuan Yang

1709.06298

Breaking Reversibility Accelerates Langevin Dynamics for Global Non-Convex Optimization

Xuefeng Gao, Mert Gurbuzbalaban, Lingjiong Zhu

1812.07725

Attacking Binarized Neural Networks

Angus Galloway, Graham W. Taylor, Medhat Moussa

1711.00449

Adversarial Risk and the Dangers of Evaluating Against Weak Attacks

Jonathan Uesato, Brendan O'Donoghue, Aaron van den Oord, Pushmeet Kohli

1802.05666

Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem

Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, Tommi Jaakkola

2206.04119

Can we steal your vocal identity from the Internet?: Initial investigation of cloning Obama's voice using GAN, WaveNet and low-quality found data

Jaime Lorenzo-Trueba, Fuming Fang, Xin Wang, Isao Echizen, Junichi Yamagishi, Tomi Kinnunen

1803.00860

Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit

Belinda Tzen, Maxim Raginsky

1905.09883

Stochastic Controlled Averaging for Federated Learning with Communication Compression

Xinmeng Huang, Ping Li, Xiaoyun Li

2308.08165

Deep Reinforcement Learning For Sequence to Sequence Models

Yaser Keneshloo, Tian Shi, Naren Ramakrishnan, Chandan K. Reddy

1805.09461

Why Normalizing Flows Fail to Detect Out-of-Distribution Data

Polina Kirichenko, Pavel Izmailov, Andrew Gordon Wilson

2006.08545

A Unified Particle-Optimization Framework for Scalable Bayesian Sampling

Changyou Chen, Ruiyi Zhang, Wenlin Wang, Bai Li, Liqun Chen

1805.11659

Constrained Graph Variational Autoencoders for Molecule Design

Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, Alexander L. Gaunt

1805.09076

Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation

Daniel Stoller, Sebastian Ewert, Simon Dixon

1806.03185

Grow and Prune Compact, Fast, and Accurate LSTMs

Xiaoliang Dai, Hongxu Yin, Niraj K. Jha

1805.11797

Learning a Latent Space of Multitrack Measures

Ian Simon, Adam Roberts, Colin Raffel, Jesse Engel, Curtis Hawthorne, Douglas Eck

1806.00195

Scale-Equivariant Steerable Networks

Ivan Sosnovik, Michał Szmaja, Arnold Smeulders

1910.11093

Conditioning Deep Generative Raw Audio Models for Structured Automatic Music

Rachel Manzelli, Vijay Thakkar, Ali Siahkamari, Brian Kulis

1806.09905

Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesis

Daisy Stanton, Yuxuan Wang, RJ Skerry-Ryan

1808.01410

Distributed Low-rank Subspace Segmentation

Ameet Talwalkar, Lester Mackey, Yadong Mu, Shih-Fu Chang, Michael I. Jordan

1304.5583

Wasserstein GAN and Waveform Loss-based Acoustic Model Training for Multi-speaker Text-to-Speech Synthesis Systems Using a WaveNet Vocoder

Yi Zhao, Shinji Takaki, Hieu-Thi Luong, Junichi Yamagishi, Daisuke Saito, Nobuaki Minematsu

1807.11679

Geometric Dataset Distances via Optimal Transport

David Alvarez-Melis, Nicolò Fusi

2002.02923

Outlier-Efficient Hopfield Layers for Large Transformer-Based Models

Jerry Yao-Chieh Hu, Pei-Hsuan Chang, Robin Luo, Hong-Yu Chen, Weijian Li, Wei-Po Wang, Han Liu

2404.03828

Implicit Latent Variable Model for Scene-Consistent Motion Forecasting

Sergio Casas, Cole Gulino, Simon Suo, Katie Luo, Renjie Liao, Raquel Urtasun

2007.12036

Learning how to explain neural networks: PatternNet and PatternAttribution

Pieter-Jan Kindermans, Kristof T. Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Erhan, Been Kim, Sven Dähne

1705.05598

The Discrete Infinite Logistic Normal Distribution

John Paisley, Chong Wang, David Blei

1103.4789

Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysis

Atsushi Nitanda, Denny Wu, Taiji Suzuki

2012.15477

Rotation Equivariant CNNs for Digital Pathology

Bastiaan S. Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, Max Welling

1806.03962

Sub-Gaussian estimators of the mean of a random matrix with heavy-tailed entries

Stanislav Minsker

1605.07129

A Baseline for Few-Shot Image Classification

Guneet S. Dhillon, Pratik Chaudhari, Avinash Ravichandran, Stefano Soatto

1909.02729

A Variational Approach to Privacy and Fairness

Borja Rodríguez-Gálvez, Ragnar Thobaben, Mikael Skoglund

2006.06332

Unsupervised learning of object landmarks by factorized spatial embeddings

James Thewlis, Hakan Bilen, Andrea Vedaldi

1705.02193

From Denoising to Compressed Sensing

Christopher A. Metzler, Arian Maleki, Richard G. Baraniuk

1406.4175

Convergence for score-based generative modeling with polynomial complexity

Holden Lee, Jianfeng Lu, Yixin Tan

2206.06227

Theoretical Analysis of Auto Rate-Tuning by Batch Normalization

Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu

1812.03981

A Latent Variable Recurrent Neural Network for Discourse Relation Language Models

Yangfeng Ji, Gholamreza Haffari, Jacob Eisenstein

1603.01913

Fairness Through Causal Awareness: Learning Latent-Variable Models for Biased Data

David Madras, Elliot Creager, Toniann Pitassi, Richard Zemel

1809.02519

Smooth and Sparse Optimal Transport

Mathieu Blondel, Vivien Seguy, Antoine Rolet

1710.06276

RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data

Yanbo Xu, Siddharth Biswal, Shriprasad R Deshpande, Kevin O Maher, Jimeng Sun

1807.08820

Entropic gradient descent algorithms and wide flat minima

Fabrizio Pittorino, Carlo Lucibello, Christoph Feinauer, Gabriele Perugini, Carlo Baldassi, Elizaveta Demyanenko, Riccardo Zecchina

2006.07897

An Introduction to Probabilistic Programming

Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, Frank Wood

1809.10756

Scalable Discrete Sampling as a Multi-Armed Bandit Problem

Yutian Chen, Zoubin Ghahramani

1506.09039

TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics

Alexander Tong, Jessie Huang, Guy Wolf, David van Dijk, Smita Krishnaswamy

2002.04461

Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs

Benedek Rozemberczki, Oliver Kiss, Rik Sarkar

2003.04819

Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization

Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, Percy Liang

1911.08731

Inference with Deep Generative Priors in High Dimensions

Parthe Pandit, Mojtaba Sahraee-Ardakan, Sundeep Rangan, Philip Schniter, Alyson K. Fletcher

1911.03409

On Learning Fairness and Accuracy on Multiple Subgroups

Changjian Shui, Gezheng Xu, Qi Chen, Jiaqi Li, Charles Ling, Tal Arbel, Boyu Wang, Christian Gagné

2210.10837

A geometric analysis of subspace clustering with outliers

Mahdi Soltanolkotabi, Emmanuel J. Candés

1112.4258

PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos

Paavo Parmas, Carl Edward Rasmussen, Jan Peters, Kenji Doya

1902.01240

On Reward-Free Reinforcement Learning with Linear Function Approximation

Ruosong Wang, Simon S. Du, Lin F. Yang, Ruslan Salakhutdinov

2006.11274

The Effects of Regularization and Data Augmentation are Class Dependent

Randall Balestriero, Leon Bottou, Yann LeCun

2204.03632

Critic Regularized Regression

Ziyu Wang, Alexander Novikov, Konrad Zolna, Jost Tobias Springenberg, Scott Reed, Bobak Shahriari, Noah Siegel, Josh Merel, Caglar Gulcehre, Nicolas Heess, Nando de Freitas

2006.15134

Multi-task Deep Reinforcement Learning with PopArt

Matteo Hessel, Hubert Soyer, Lasse Espeholt, Wojciech Czarnecki, Simon Schmitt, Hado van Hasselt

1809.04474

Robust Low-rank Tensor Recovery: Models and Algorithms

Donald Goldfarb, Zhiwei Qin

1311.6182

Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data

Spencer Frei, Gal Vardi, Peter L. Bartlett, Nathan Srebro, Wei Hu

2210.07082

Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study

Samuel Ritter, David G. T. Barrett, Adam Santoro, Matt M. Botvinick

1706.08606

AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models

Karl Krauth, Edwin V. Bonilla, Kurt Cutajar, Maurizio Filippone

1610.05392

Operator-valued Kernels for Learning from Functional Response Data

Hachem Kadri, Emmanuel Duflos, Philippe Preux, Stéphane Canu, Alain Rakotomamonjy, Julien Audiffren

1510.08231

Inferring deterministic causal relations

Povilas Daniusis, Dominik Janzing, Joris Mooij, Jakob Zscheischler, Bastian Steudel, Kun Zhang, Bernhard Schoelkopf

1203.3475

Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting

Giorgos Bouritsas, Fabrizio Frasca, Stefanos Zafeiriou, Michael M. Bronstein

2006.09252

Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits

Alekh Agarwal, Daniel Hsu, Satyen Kale, John Langford, Lihong Li, Robert E. Schapire

1402.0555

Autoregressive Diffusion Models

Emiel Hoogeboom, Alexey A. Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, Tim Salimans

2110.02037

Reconstructing Training Data from Trained Neural Networks

Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, Michal Irani

2206.07758

Minimax-optimal Inference from Partial Rankings

Bruce Hajek, Sewoong Oh, Jiaming Xu

1406.5638

On the Outsized Importance of Learning Rates in Local Update Methods

Zachary Charles, Jakub Konečný

2007.00878

Model Reduction with Memory and the Machine Learning of Dynamical Systems

Chao Ma, Jianchun Wang, Weinan E

1808.04258

Self-Imitation Learning

Junhyuk Oh, Yijie Guo, Satinder Singh, Honglak Lee

1806.05635

Multiple Descent: Design Your Own Generalization Curve

Lin Chen, Yifei Min, Mikhail Belkin, Amin Karbasi

2008.01036

Minimizing The Misclassification Error Rate Using a Surrogate Convex Loss

Shai Ben-David, David Loker, Nathan Srebro, Karthik Sridharan

1206.6442

Bayesian Optimization for Probabilistic Programs

Tom Rainforth, Tuan Anh Le, Jan-Willem van de Meent, Michael A. Osborne, Frank Wood

1707.04314

Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data

Alexander Robey, Hamed Hassani, George J. Pappas

2005.10247

Expectation-Propagation for Likelihood-Free Inference

Simon Barthelmé, Nicolas Chopin

1107.5959

Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization

Eldad Meller, Alexander Finkelstein, Uri Almog, Mark Grobman

1902.01917

Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspective

Shixia Liu, Xiting Wang, Mengchen Liu, Jun Zhu

1702.01226

A PAC-Bayesian bound for Lifelong Learning

Anastasia Pentina, Christoph H. Lampert

1311.2838

How Interpretable and Trustworthy are GAMs?

Chun-Hao Chang, Sarah Tan, Ben Lengerich, Anna Goldenberg, Rich Caruana

2006.06466

Deep Equals Shallow for ReLU Networks in Kernel Regimes

Alberto Bietti, Francis Bach

2009.14397

Supervised Learning with Quantum-Inspired Tensor Networks

E. Miles Stoudenmire, David J. Schwab

1605.05775

Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination

Qibin Zhao, Liqing Zhang, Andrzej Cichocki

1401.6497

When Can We Learn General-Sum Markov Games with a Large Number of Players Sample-Efficiently?

Ziang Song, Song Mei, Yu Bai

2110.04184

Fairwashing Explanations with Off-Manifold Detergent

Christopher J. Anders, Plamen Pasliev, Ann-Kathrin Dombrowski, Klaus-Robert Müller, Pan Kessel

2007.09969

A Survey of Reinforcement Learning Informed by Natural Language

Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, Tim Rocktäschel

1906.03926

Global Convergence to the Equilibrium of GANs using Variational Inequalities

Ian Gemp, Sridhar Mahadevan

1808.01531

High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity

Po-Ling Loh, Martin J. Wainwright

1109.3714

Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems

Aman Sinha, Matthew O'Kelly, Russ Tedrake, John Duchi

2008.10581

Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyond

Xuechen Li, Denny Wu, Lester Mackey, Murat A. Erdogdu

1906.07868

ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

Minsuk Kahng, Pierre Y. Andrews, Aditya Kalro, Duen Horng Chau

1704.01942

Towards understanding how momentum improves generalization in deep learning

Samy Jelassi, Yuanzhi Li

2207.05931

A Bayesian Approach to Network Modularity

Jake M. Hofman, Chris H. Wiggins

0709.3512

A Survey of Optimization Methods from a Machine Learning Perspective

Shiliang Sun, Zehui Cao, Han Zhu, Jing Zhao

1906.06821

LakhNES: Improving multi-instrumental music generation with cross-domain pre-training

Chris Donahue, Huanru Henry Mao, Yiting Ethan Li, Garrison W. Cottrell, Julian McAuley

1907.04868

Variational Latent Gaussian Process for Recovering Single-Trial Dynamics from Population Spike Trains

Yuan Zhao, Il Memming Park

1604.03053

Conservative Q-Learning for Offline Reinforcement Learning

Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine

2006.04779

Global convergence of neuron birth-death dynamics

Grant Rotskoff, Samy Jelassi, Joan Bruna, Eric Vanden-Eijnden

1902.01843

Grad-CAM: Why did you say that?

Ramprasaath R Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, Dhruv Batra

1611.07450

Improvements to deep convolutional neural networks for LVCSR

Tara N. Sainath, Brian Kingsbury, Abdel-rahman Mohamed, George E. Dahl, George Saon, Hagen Soltau, Tomas Beran, Aleksandr Y. Aravkin, Bhuvana Ramabhadran

1309.1501

BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabularies

Shihao Ji, S. V. N. Vishwanathan, Nadathur Satish, Michael J. Anderson, Pradeep Dubey

1511.06909

Disentanglement of Correlated Factors via Hausdorff Factorized Support

Karsten Roth, Mark Ibrahim, Zeynep Akata, Pascal Vincent, Diane Bouchacourt

2210.07347

Recent Advances in Zero-shot Recognition

Yanwei Fu, Tao Xiang, Yu-Gang Jiang, Xiangyang Xue, Leonid Sigal, Shaogang Gong

1710.04837

SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition

Zhixuan Lin, Yi-Fu Wu, Skand Vishwanath Peri, Weihao Sun, Gautam Singh, Fei Deng, Jindong Jiang, Sungjin Ahn

2001.02407

Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances

Marco Cuturi

1306.0895

Best sources forward: domain generalization through source-specific nets

Massimiliano Mancini, Samuel Rota Bulò, Barbara Caputo, Elisa Ricci

1806.05810

Learning the Parameters of Determinantal Point Process Kernels

Raja Hafiz Affandi, Emily B. Fox, Ryan P. Adams, Ben Taskar

1402.4862

Learning Policies for Contextual Submodular Prediction

Stephane Ross, Jiaji Zhou, Yisong Yue, Debadeepta Dey, J. Andrew Bagnell

1305.2532

Personal VAD: Speaker-Conditioned Voice Activity Detection

Shaojin Ding, Quan Wang, Shuo-yiin Chang, Li Wan, Ignacio Lopez Moreno

1908.04284

Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior

Adam Block, Ali Jadbabaie, Daniel Pfrommer, Max Simchowitz, Russ Tedrake

2307.14619

Understanding Generalization through Visualizations

W. Ronny Huang, Zeyad Emam, Micah Goldblum, Liam Fowl, J. K. Terry, Furong Huang, Tom Goldstein

1906.03291

When to Trust Your Model: Model-Based Policy Optimization

Michael Janner, Justin Fu, Marvin Zhang, Sergey Levine

1906.08253

Deep, Skinny Neural Networks are not Universal Approximators

Jesse Johnson

1810.00393

Black-box Adversarial Attacks with Limited Queries and Information

Andrew Ilyas, Logan Engstrom, Anish Athalye, Jessy Lin

1804.08598

A New Analysis of Differential Privacy's Generalization Guarantees

Christopher Jung, Katrina Ligett, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, Moshe Shenfeld

1909.03577

Graph Convolutional Reinforcement Learning

Jiechuan Jiang, Chen Dun, Tiejun Huang, Zongqing Lu

1810.09202

Norm matters: efficient and accurate normalization schemes in deep networks

Elad Hoffer, Ron Banner, Itay Golan, Daniel Soudry

1803.01814

Fair Resource Allocation in Federated Learning

Tian Li, Maziar Sanjabi, Ahmad Beirami, Virginia Smith

1905.10497

Fault Sneaking Attack: a Stealthy Framework for Misleading Deep Neural Networks

Pu Zhao, Siyue Wang, Cheng Gongye, Yanzhi Wang, Yunsi Fei, Xue Lin

1905.12032

Neural Networks and Quantum Field Theory

James Halverson, Anindita Maiti, Keegan Stoner

2008.08601

Neural Spline Flows

Conor Durkan, Artur Bekasov, Iain Murray, George Papamakarios

1906.04032

Federated Model Distillation with Noise-Free Differential Privacy

Lichao Sun, Lingjuan Lyu

2009.05537

Analysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classification

Maksim Lapin, Matthias Hein, Bernt Schiele

1612.03663

Label Efficient Learning of Transferable Representations across Domains and Tasks

Zelun Luo, Yuliang Zou, Judy Hoffman, Li Fei-Fei

1712.00123

Learning Features of Music from Scratch

John Thickstun, Zaid Harchaoui, Sham Kakade

1611.09827

Invariant Representations for Noisy Speech Recognition

Dmitriy Serdyuk, Kartik Audhkhasi, Philémon Brakel, Bhuvana Ramabhadran, Samuel Thomas, Yoshua Bengio

1612.01928

What game are we playing? End-to-end learning in normal and extensive form games

Chun Kai Ling, Fei Fang, J. Zico Kolter

1805.02777

End-to-End Attention based Text-Dependent Speaker Verification

Shi-Xiong Zhang, Zhuo Chen, Yong Zhao, Jinyu Li, Yifan Gong

1701.00562

Learning to Play with Intrinsically-Motivated Self-Aware Agents

Nick Haber, Damian Mrowca, Li Fei-Fei, Daniel L. K. Yamins

1802.07442

Universal representations:The missing link between faces, text, planktons, and cat breeds

Hakan Bilen, Andrea Vedaldi

1701.07275

Sequence-to-Sequence Models Can Directly Translate Foreign Speech

Ron J. Weiss, Jan Chorowski, Navdeep Jaitly, Yonghui Wu, Zhifeng Chen

1703.08581

Label-Consistent Backdoor Attacks

Alexander Turner, Dimitris Tsipras, Aleksander Madry

1912.02771

Unrestricted Adversarial Examples

Tom B. Brown, Nicholas Carlini, Chiyuan Zhang, Catherine Olsson, Paul Christiano, Ian Goodfellow

1809.08352

Learning Combinatorial Optimization Algorithms over Graphs

Hanjun Dai, Elias B. Khalil, Yuyu Zhang, Bistra Dilkina, Le Song

1704.01665

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, Shin Ishii

1704.03976

Robustness to Adversarial Perturbations in Learning from Incomplete Data

Amir Najafi, Shin-ichi Maeda, Masanori Koyama, Takeru Miyato

1905.13021

Matrix Completion via Max-Norm Constrained Optimization

T. Tony Cai, Wen-Xin Zhou

1303.0341

Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism

Paria Rashidinejad, Banghua Zhu, Cong Ma, Jiantao Jiao, Stuart Russell

2103.12021

Consistent feature attribution for tree ensembles

Scott M. Lundberg, Su-In Lee

1706.06060

Band-limited Training and Inference for Convolutional Neural Networks

Adam Dziedzic, John Paparrizos, Sanjay Krishnan, Aaron Elmore, Michael Franklin

1911.09287

Truncated Power Method for Sparse Eigenvalue Problems

Xiao-Tong Yuan, Tong Zhang

1112.2679

Herding Dynamic Weights for Partially Observed Random Field Models

Max Welling

1205.2605

Distributionally Robust Language Modeling

Yonatan Oren, Shiori Sagawa, Tatsunori B. Hashimoto, Percy Liang

1909.02060

An Optimal Multistage Stochastic Gradient Method for Minimax Problems

Alireza Fallah, Asuman Ozdaglar, Sarath Pattathil

2002.05683

Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals

Surbhi Goel, Sushrut Karmalkar, Adam Klivans

1911.01462

Learning to Self-Train for Semi-Supervised Few-Shot Classification

Xinzhe Li, Qianru Sun, Yaoyao Liu, Shibao Zheng, Qin Zhou, Tat-Seng Chua, Bernt Schiele

1906.00562

Group-Sparse Model Selection: Hardness and Relaxations

Luca Baldassarre, Nirav Bhan, Volkan Cevher, Anastasios Kyrillidis, Siddhartha Satpathi

1303.3207

Particle Gibbs with Ancestor Sampling

Fredrik Lindsten, Michael I. Jordan, Thomas B. Schön

1401.0604

Selfie: Self-supervised Pretraining for Image Embedding

Trieu H. Trinh, Minh-Thang Luong, Quoc V. Le

1906.02940

The CLRS Algorithmic Reasoning Benchmark

Petar Veličković, Adrià Puigdomènech Badia, David Budden, Razvan Pascanu, Andrea Banino, Misha Dashevskiy, Raia Hadsell, Charles Blundell

2205.15659

Bayesian inference for logistic models using Polya-Gamma latent variables

Nicholas G. Polson, James G. Scott, Jesse Windle

1205.0310

The Squared-Error of Generalized LASSO: A Precise Analysis

Samet Oymak, Christos Thrampoulidis, Babak Hassibi

1311.0830

Hypersolvers: Toward Fast Continuous-Depth Models

Michael Poli, Stefano Massaroli, Atsushi Yamashita, Hajime Asama, Jinkyoo Park

2007.09601

Smooth Neighbors on Teacher Graphs for Semi-supervised Learning

Yucen Luo, Jun Zhu, Mengxi Li, Yong Ren, Bo Zhang

1711.00258

Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference Engines

Sean O. Settle, Manasa Bollavaram, Paolo D'Alberto, Elliott Delaye, Oscar Fernandez, Nicholas Fraser, Aaron Ng, Ashish Sirasao, Michael Wu

1805.07941

Variational Federated Multi-Task Learning

Luca Corinzia, Ami Beuret, Joachim M. Buhmann

1906.06268

Complexity Guarantees for Polyak Steps with Momentum

Mathieu Barré, Adrien Taylor, Alexandre d'Aspremont

2002.00915

Reinforced Continual Learning

Ju Xu, Zhanxing Zhu

1805.12369

EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs

Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao B. Schardl, Charles E. Leiserson

1902.10191

Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks

Zhou Fan, Zhichao Wang

2005.11879

Bounding and Approximating Intersectional Fairness through Marginal Fairness

Mathieu Molina, Patrick Loiseau

2206.05828

Text Infilling

Wanrong Zhu, Zhiting Hu, Eric Xing

1901.00158

WHAM!: Extending Speech Separation to Noisy Environments

Gordon Wichern, Joe Antognini, Michael Flynn, Licheng Richard Zhu, Emmett McQuinn, Dwight Crow, Ethan Manilow, Jonathan Le Roux

1907.01160

Convolutional Neural Networks Analyzed via Convolutional Sparse Coding

Vardan Papyan, Yaniv Romano, Michael Elad

1607.08194

InstaHide: Instance-hiding Schemes for Private Distributed Learning

Yangsibo Huang, Zhao Song, Kai Li, Sanjeev Arora

2010.02772

RLHF Workflow: From Reward Modeling to Online RLHF

Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, Tong Zhang

2405.07863

Posterior Differential Regularization with f-divergence for Improving Model Robustness

Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao

2010.12638

Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples

Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, Hugo Larochelle

1903.03096

MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning

Yuan Gao, Haoping Bai, Zequn Jie, Jiayi Ma, Kui Jia, Wei Liu

2003.14058

Convergence of Sparse Variational Inference in Gaussian Processes Regression

David R. Burt, Carl Edward Rasmussen, Mark van der Wilk

2008.00323

Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning

Han-Jia Ye, Hong-You Chen, De-Chuan Zhan, Wei-Lun Chao

2001.01385

Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?

Angelos Filos, Panagiotis Tigas, Rowan McAllister, Nicholas Rhinehart, Sergey Levine, Yarin Gal

2006.14911

Bayesian Structure Learning with Generative Flow Networks

Tristan Deleu, António Góis, Chris Emezue, Mansi Rankawat, Simon Lacoste-Julien, Stefan Bauer, Yoshua Bengio

2202.13903

MOPO: Model-based Offline Policy Optimization

Tianhe Yu, Garrett Thomas, Lantao Yu, Stefano Ermon, James Zou, Sergey Levine, Chelsea Finn, Tengyu Ma

2005.13239

GRAM: Graph-based Attention Model for Healthcare Representation Learning

Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F. Stewart, Jimeng Sun

1611.07012

Flora: Low-Rank Adapters Are Secretly Gradient Compressors

Yongchang Hao, Yanshuai Cao, Lili Mou

2402.03293

Variational Inference in high-dimensional linear regression

Sumit Mukherjee, Subhabrata Sen

2104.12232

Actionable Recourse in Linear Classification

Berk Ustun, Alexander Spangher, Yang Liu

1809.06514

Collapse of Deep and Narrow Neural Nets

Lu Lu, Yanhui Su, George Em Karniadakis

1808.04947

Deep Multi-Output Forecasting: Learning to Accurately Predict Blood Glucose Trajectories

Ian Fox, Lynn Ang, Mamta Jaiswal, Rodica Pop-Busui, Jenna Wiens

1806.05357

GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing

Jian Guo, He He, Tong He, Leonard Lausen, Mu Li, Haibin Lin, Xingjian Shi, Chenguang Wang, Junyuan Xie, Sheng Zha, Aston Zhang, Hang Zhang, Zhi Zhang, Zhongyue Zhang, Shuai Zheng, Yi Zhu

1907.04433

Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees

Yuping Luo, Huazhe Xu, Yuanzhi Li, Yuandong Tian, Trevor Darrell, Tengyu Ma

1807.03858

An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits

Andrea Tirinzoni, Matteo Pirotta, Marcello Restelli, Alessandro Lazaric

2010.12247

Analyzing Differentiable Fuzzy Implications

Emile van Krieken, Erman Acar, Frank van Harmelen

2006.03472

Planning to Explore via Self-Supervised World Models

Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, Deepak Pathak

2005.05960

Distilling portable Generative Adversarial Networks for Image Translation

Hanting Chen, Yunhe Wang, Han Shu, Changyuan Wen, Chunjing Xu, Boxin Shi, Chao Xu, Chang Xu

2003.03519

The asymptotic spectrum of the Hessian of DNN throughout training

Arthur Jacot, Franck Gabriel, Clément Hongler

1910.02875

CycleGAN, a Master of Steganography

Casey Chu, Andrey Zhmoginov, Mark Sandler

1712.02950

Machine Learning for Fluid Mechanics

Steven Brunton, Bernd Noack, Petros Koumoutsakos

1905.11075

Predicting What You Already Know Helps: Provable Self-Supervised Learning

Jason D. Lee, Qi Lei, Nikunj Saunshi, Jiacheng Zhuo

2008.01064

Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learning

Francesco Orabona

1406.3816

Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms

Mathieu Blondel, André F. T. Martins, Vlad Niculae

1805.09717

A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions

Guillaume Staerman, Pavlo Mozharovskyi, Pierre Colombo, Stéphan Clémençon, Florence d'Alché-Buc

2103.12711

Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization

Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, H. Vincent Poor

2007.07481

Animating Arbitrary Objects via Deep Motion Transfer

Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe

1812.08861

TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cut

Yangtao Wang, Xi Shen, Yuan Yuan, Yuming Du, Maomao Li, Shell Xu Hu, James L Crowley, Dominique Vaufreydaz

2209.00383

Deployment-Efficient Reinforcement Learning via Model-Based Offline Optimization

Tatsuya Matsushima, Hiroki Furuta, Yutaka Matsuo, Ofir Nachum, Shixiang Gu

2006.03647

Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning

Francis Bach

0809.1493

Quasi-hyperbolic momentum and Adam for deep learning

Jerry Ma, Denis Yarats

1810.06801

Distral: Robust Multitask Reinforcement Learning

Yee Whye Teh, Victor Bapst, Wojciech Marian Czarnecki, John Quan, James Kirkpatrick, Raia Hadsell, Nicolas Heess, Razvan Pascanu

1707.04175

Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty

Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, Dawn Song

1906.12340

Uncertainty Quantification and Deep Ensembles

Rahul Rahaman, Alexandre H. Thiery

2007.08792

Scalable agent alignment via reward modeling: a research direction

Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, Shane Legg

1811.07871

Neural graphical modelling in continuous-time: consistency guarantees and algorithms

Alexis Bellot, Kim Branson, Mihaela van der Schaar

2105.02522

Differentially Private Fair Learning

Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi-Malvajerdi, Jonathan Ullman

1812.02696

Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering

Li Zhou, Kevin Small

1911.06192

Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances

Bugra Can, Mert Gurbuzbalaban, Lingjiong Zhu

1901.07445

Large Scale Kernel Learning using Block Coordinate Descent

Stephen Tu, Rebecca Roelofs, Shivaram Venkataraman, Benjamin Recht

1602.05310

Differentially Private SGD with Non-Smooth Losses

Puyu Wang, Yunwen Lei, Yiming Ying, Hai Zhang

2101.08925

Robust pricing and hedging via neural SDEs

Patryk Gierjatowicz, Marc Sabate-Vidales, David Šiška, Lukasz Szpruch, Žan Žurič

2007.04154

Automated Scalable Bayesian Inference via Hilbert Coresets

Trevor Campbell, Tamara Broderick

1710.05053

An Empirical Study of Example Forgetting during Deep Neural Network Learning

Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoffrey J. Gordon

1812.05159

Deep Survival Analysis

Rajesh Ranganath, Adler Perotte, Noémie Elhadad, David Blei

1608.02158

Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing

Jingyi Wang, Guoliang Dong, Jun Sun, Xinyu Wang, Peixin Zhang

1812.05793

Off-Policy Deep Reinforcement Learning without Exploration

Scott Fujimoto, David Meger, Doina Precup

1812.02900

On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty

Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, Yarin Gal

2102.11409

A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-$\ell_1$-Norm Interpolated Classifiers

Tengyuan Liang, Pragya Sur

2002.01586

Bridging Convex and Nonconvex Optimization in Robust PCA: Noise, Outliers, and Missing Data

Yuxin Chen, Jianqing Fan, Cong Ma, Yuling Yan

2001.05484

Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees

Johanna Vielhaben, Stefan Blücher, Nils Strodthoff

2301.11911

Explorations in Homeomorphic Variational Auto-Encoding

Luca Falorsi, Pim de Haan, Tim R. Davidson, Nicola De Cao, Maurice Weiler, Patrick Forré, Taco S. Cohen

1807.04689

Generalized Leverage Score Sampling for Neural Networks

Jason D. Lee, Ruoqi Shen, Zhao Song, Mengdi Wang, Zheng Yu

2009.09829

Estimate Sequences for Stochastic Composite Optimization: Variance Reduction, Acceleration, and Robustness to Noise

Andrei Kulunchakov, Julien Mairal

1901.08788

Fast Sketching of Polynomial Kernels of Polynomial Degree

Zhao Song, David P. Woodruff, Zheng Yu, Lichen Zhang

2108.09420

Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation

Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, Paulius Micikevicius

2004.09602

Automatic Rule Extraction from Long Short Term Memory Networks

W. James Murdoch, Arthur Szlam

1702.02540

Tensor Canonical Correlation Analysis for Multi-view Dimension Reduction

Yong Luo, Dacheng Tao, Yonggang Wen, Kotagiri Ramamohanarao, Chao Xu

1502.02330

Recurrent World Models Facilitate Policy Evolution

David Ha, Jürgen Schmidhuber

1809.01999

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion

Tim Dockhorn, Arash Vahdat, Karsten Kreis

2112.07068

Unsupervised State Representation Learning in Atari

Ankesh Anand, Evan Racah, Sherjil Ozair, Yoshua Bengio, Marc-Alexandre Côté, R Devon Hjelm

1906.08226

Flexible Dataset Distillation: Learn Labels Instead of Images

Ondrej Bohdal, Yongxin Yang, Timothy Hospedales

2006.08572

Breaking the Bandwidth Barrier: Geometrical Adaptive Entropy Estimation

Weihao Gao, Sewoong Oh, Pramod Viswanath

1609.02208

Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithms

Chi Jin, Qinghua Liu, Sobhan Miryoosefi

2102.00815

Algorithms and SQ Lower Bounds for PAC Learning One-Hidden-Layer ReLU Networks

Ilias Diakonikolas, Daniel M. Kane, Vasilis Kontonis, Nikos Zarifis

2006.12476

AdaBatch: Efficient Gradient Aggregation Rules for Sequential and Parallel Stochastic Gradient Methods

Alexandre Défossez, Francis Bach

1711.01761

When Does Self-Supervision Help Graph Convolutional Networks?

Yuning You, Tianlong Chen, Zhangyang Wang, Yang Shen

2006.09136

Transferring Knowledge from a RNN to a DNN

William Chan, Nan Rosemary Ke, Ian Lane

1504.01483

Meta-Learning with Implicit Gradients

Aravind Rajeswaran, Chelsea Finn, Sham Kakade, Sergey Levine

1909.04630

Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation

Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, Kaisheng Ma

1905.08094

Crafting Adversarial Examples For Speech Paralinguistics Applications

Yuan Gong, Christian Poellabauer

1711.03280

Adaptive Gradient Quantization for Data-Parallel SGD

Fartash Faghri, Iman Tabrizian, Ilia Markov, Dan Alistarh, Daniel Roy, Ali Ramezani-Kebrya

2010.12460

Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data

Charles H. Martin, Tongsu, Peng, Michael W. Mahoney

2002.06716

Additive Gaussian Processes

David Duvenaud, Hannes Nickisch, Carl Edward Rasmussen

1112.4394

Generalised Wishart Processes

Andrew Gordon Wilson, Zoubin Ghahramani

1101.0240

4+3 Phases of Compute-Optimal Neural Scaling Laws

Elliot Paquette, Courtney Paquette, Lechao Xiao, Jeffrey Pennington

2405.15074

Predicting Training Time Without Training

Luca Zancato, Alessandro Achille, Avinash Ravichandran, Rahul Bhotika, Stefano Soatto

2008.12478

OpenML-Python: an extensible Python API for OpenML

Matthias Feurer, Jan N. van Rijn, Arlind Kadra, Pieter Gijsbers, Neeratyoy Mallik, Sahithya Ravi, Andreas Müller, Joaquin Vanschoren, Frank Hutter

1911.02490

See, Hear, Explore: Curiosity via Audio-Visual Association

Victoria Dean, Shubham Tulsiani, Abhinav Gupta

2007.03669

Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing

Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le

2006.03236

Generative Language Modeling for Automated Theorem Proving

Stanislas Polu, Ilya Sutskever

2009.03393

Revisiting Fundamentals of Experience Replay

William Fedus, Prajit Ramachandran, Rishabh Agarwal, Yoshua Bengio, Hugo Larochelle, Mark Rowland, Will Dabney

2007.06700

On Lower Bounds for Regret in Reinforcement Learning

Ian Osband, Benjamin Van Roy

1608.02732

On Power Laws in Deep Ensembles

Ekaterina Lobacheva, Nadezhda Chirkova, Maxim Kodryan, Dmitry Vetrov

2007.08483

L_DMI: An Information-theoretic Noise-robust Loss Function

Yilun Xu, Peng Cao, Yuqing Kong, Yizhou Wang

1909.03388

Low-Rank Bottleneck in Multi-head Attention Models

Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar

2002.07028

Scalable Transfer Learning with Expert Models

Joan Puigcerver, Carlos Riquelme, Basil Mustafa, Cedric Renggli, André Susano Pinto, Sylvain Gelly, Daniel Keysers, Neil Houlsby

2009.13239

Kullback-Leibler aggregation and misspecified generalized linear models

Philippe Rigollet

0911.2919

Inference in Probabilistic Graphical Models by Graph Neural Networks

KiJung Yoon, Renjie Liao, Yuwen Xiong, Lisa Zhang, Ethan Fetaya, Raquel Urtasun, Richard Zemel, Xaq Pitkow

1803.07710

Fairness for Image Generation with Uncertain Sensitive Attributes

Ajil Jalal, Sushrut Karmalkar, Jessica Hoffmann, Alexandros G. Dimakis, Eric Price

2106.12182

Open Graph Benchmark: Datasets for Machine Learning on Graphs

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, Jure Leskovec

2005.00687

Multitask learning and benchmarking with clinical time series data

Hrayr Harutyunyan, Hrant Khachatrian, David C. Kale, Greg Ver Steeg, Aram Galstyan

1703.07771

VC Classes are Adversarially Robustly Learnable, but Only Improperly

Omar Montasser, Steve Hanneke, Nathan Srebro

1902.04217

Can You Trust This Prediction? Auditing Pointwise Reliability After Learning

Peter Schulam, Suchi Saria

1901.00403

Stochastic gradient variational Bayes for gamma approximating distributions

David A. Knowles

1509.01631

Biologically inspired protection of deep networks from adversarial attacks

Aran Nayebi, Surya Ganguli

1703.09202

Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels

Zhilu Zhang, Mert R. Sabuncu

1805.07836

Optuna: A Next-generation Hyperparameter Optimization Framework

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama

1907.10902

Likelihood Ratios for Out-of-Distribution Detection

Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A. DePristo, Joshua V. Dillon, Balaji Lakshminarayanan

1906.02845

Multimodal Hierarchical Dirichlet Process-based Active Perception

Tadahiro Taniguchi, Toshiaki Takano, Ryo Yoshino

1510.00331

Structured Variable Selection with Sparsity-Inducing Norms

Rodolphe Jenatton, Jean-Yves Audibert, Francis Bach

0904.3523

What Do Compressed Deep Neural Networks Forget?

Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, Andrea Frome

1911.05248

Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale

Atılım Güneş Baydin, Lei Shao, Wahid Bhimji, Lukas Heinrich, Lawrence Meadows, Jialin Liu, Andreas Munk, Saeid Naderiparizi, Bradley Gram-Hansen, Gilles Louppe, Mingfei Ma, Xiaohui Zhao, Philip Torr, Victor Lee, Kyle Cranmer, Prabhat, Frank Wood

1907.03382

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Benjamin Aubin, Florent Krzakala, Yue M. Lu, Lenka Zdeborová

2006.06560

DiBS: Differentiable Bayesian Structure Learning

Lars Lorch, Jonas Rothfuss, Bernhard Schölkopf, Andreas Krause

2105.11839

Observational Overfitting in Reinforcement Learning

Xingyou Song, Yiding Jiang, Stephen Tu, Yilun Du, Behnam Neyshabur

1912.02975

In Defense of Uniform Convergence: Generalization via derandomization with an application to interpolating predictors

Jeffrey Negrea, Gintare Karolina Dziugaite, Daniel M. Roy

1912.04265

How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy

Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin, Zheng Xu, Carson Denison, H. Brendan McMahan, Sergei Vassilvitskii, Steve Chien, Abhradeep Thakurta

2303.00654

Variational Causal Networks: Approximate Bayesian Inference over Causal Structures

Yashas Annadani, Jonas Rothfuss, Alexandre Lacoste, Nino Scherrer, Anirudh Goyal, Yoshua Bengio, Stefan Bauer

2106.07635

Improved generator objectives for GANs

Ben Poole, Alexander A. Alemi, Jascha Sohl-Dickstein, Anelia Angelova

1612.02780

Counterfactual Risk Assessments, Evaluation, and Fairness

Amanda Coston, Alan Mishler, Edward H. Kennedy, Alexandra Chouldechova

1909.00066

Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search

Lars Buesing, Theophane Weber, Yori Zwols, Sebastien Racaniere, Arthur Guez, Jean-Baptiste Lespiau, Nicolas Heess

1811.06272

Poincaré Recurrence, Cycles and Spurious Equilibria in Gradient-Descent-Ascent for Non-Convex Non-Concave Zero-Sum Games

Lampros Flokas, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Georgios Piliouras

1910.13010

The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory

Dan Alistarh, Christopher De Sa, Nikola Konstantinov

1803.08841

Infinite-dimensional reservoir computing

Lukas Gonon, Lyudmila Grigoryeva, Juan-Pablo Ortega

2304.00490

Solving for high dimensional committor functions using artificial neural networks

Yuehaw Khoo, Jianfeng Lu, Lexing Ying

1802.10275

Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensors

Eduardo Arnold, Mehrdad Dianati, Robert de Temple, Saber Fallah

1912.12147

Generalization Error of Generalized Linear Models in High Dimensions

Melikasadat Emami, Mojtaba Sahraee-Ardakan, Parthe Pandit, Sundeep Rangan, Alyson K. Fletcher

2005.00180

Federated Learning via Synthetic Data

Jack Goetz, Ambuj Tewari

2008.04489

A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning

Yehuda Dar, Vidya Muthukumar, Richard G. Baraniuk

2109.02355

Projected Subgradient Methods for Learning Sparse Gaussians

John Duchi, Stephen Gould, Daphne Koller

1206.3249

NetGAN: Generating Graphs via Random Walks

Aleksandar Bojchevski, Oleksandr Shchur, Daniel Zügner, Stephan Günnemann

1803.00816

PAC-Bayesian Inequalities for Martingales

Yevgeny Seldin, François Laviolette, Nicolò Cesa-Bianchi, John Shawe-Taylor, Peter Auer

1110.6886

Robustness Verification for Transformers

Zhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang, Cho-Jui Hsieh

2002.06622

NVAE: A Deep Hierarchical Variational Autoencoder

Arash Vahdat, Jan Kautz

2007.03898

Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning

Jiwoong Park, Minsik Lee, Hyung Jin Chang, Kyuewang Lee, Jin Young Choi

1908.02441

Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies

Paul Vicol, Luke Metz, Jascha Sohl-Dickstein

2112.13835

Adapting Auxiliary Losses Using Gradient Similarity

Yunshu Du, Wojciech M. Czarnecki, Siddhant M. Jayakumar, Mehrdad Farajtabar, Razvan Pascanu, Balaji Lakshminarayanan

1812.02224

Do Residual Neural Networks discretize Neural Ordinary Differential Equations?

Michael E. Sander, Pierre Ablin, Gabriel Peyré

2205.14612

Self-supervised Learning: Generative or Contrastive

Xiao Liu, Fanjin Zhang, Zhenyu Hou, Zhaoyu Wang, Li Mian, Jing Zhang, Jie Tang

2006.08218

Differentially Private Learning Needs Better Features (or Much More Data)

Florian Tramèr, Dan Boneh

2011.11660

Multistep Consistency Models

Jonathan Heek, Emiel Hoogeboom, Tim Salimans

2403.06807

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Stefan Wager, Susan Athey

1510.04342

Relative Entropy Regularized Policy Iteration

Abbas Abdolmaleki, Jost Tobias Springenberg, Jonas Degrave, Steven Bohez, Yuval Tassa, Dan Belov, Nicolas Heess, Martin Riedmiller

1812.02256

Neural Predictor for Neural Architecture Search

Wei Wen, Hanxiao Liu, Hai Li, Yiran Chen, Gabriel Bender, Pieter-Jan Kindermans

1912.00848

Contextual Decision Processes with Low Bellman Rank are PAC-Learnable

Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, John Langford, Robert E. Schapire

1610.09512

Behavior Regularized Offline Reinforcement Learning

Yifan Wu, George Tucker, Ofir Nachum

1911.11361

Counterfactuals uncover the modular structure of deep generative models

Michel Besserve, Arash Mehrjou, Rémy Sun, Bernhard Schölkopf

1812.03253

Towards Best Practice in Explaining Neural Network Decisions with LRP

Maximilian Kohlbrenner, Alexander Bauer, Shinichi Nakajima, Alexander Binder, Wojciech Samek, Sebastian Lapuschkin

1910.09840

Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization

Bryan Wilder, Bistra Dilkina, Milind Tambe

1809.05504

Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask

Hattie Zhou, Janice Lan, Rosanne Liu, Jason Yosinski

1905.01067

Deep Learning for Anomaly Detection: A Review

Guansong Pang, Chunhua Shen, Longbing Cao, Anton van den Hengel

2007.02500

What is the State of Neural Network Pruning?

Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, John Guttag

2003.03033

Scalable Text and Link Analysis with Mixed-Topic Link Models

Yaojia Zhu, Xiaoran Yan, Lise Getoor, Cristopher Moore

1303.7264

Building DNN Acoustic Models for Large Vocabulary Speech Recognition

Andrew L. Maas, Peng Qi, Ziang Xie, Awni Y. Hannun, Christopher T. Lengerich, Daniel Jurafsky, Andrew Y. Ng

1406.7806

Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff

Yochai Blau, Tomer Michaeli

1901.07821

Multi-Head Attention: Collaborate Instead of Concatenate

Jean-Baptiste Cordonnier, Andreas Loukas, Martin Jaggi

2006.16362

Distributed Strongly Convex Optimization

Konstantinos I. Tsianos, Michael G. Rabbat

1207.3031

Towards Deeper Graph Neural Networks

Meng Liu, Hongyang Gao, Shuiwang Ji

2007.09296

Quantization Networks

Jiwei Yang, Xu Shen, Jun Xing, Xinmei Tian, Houqiang Li, Bing Deng, Jianqiang Huang, Xiansheng Hua

1911.09464

How to learn a graph from smooth signals

Vassilis Kalofolias

1601.02513

Patterns of Scalable Bayesian Inference

Elaine Angelino, Matthew James Johnson, Ryan P. Adams

1602.05221

A Signal Propagation Perspective for Pruning Neural Networks at Initialization

Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, Philip H. S. Torr

1906.06307

Hierarchical Autoregressive Image Models with Auxiliary Decoders

Jeffrey De Fauw, Sander Dieleman, Karen Simonyan

1903.04933

Non-Monotonic Sequential Text Generation

Sean Welleck, Kianté Brantley, Hal Daumé, Kyunghyun Cho

1902.02192

Stabilizing GANs with Soft Octave Convolutions

Ricard Durall, Franz-Josef Pfreundt, Janis Keuper

1905.12534

Towards Learning Convolutions from Scratch

Behnam Neyshabur

2007.13657

Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses

Raef Bassily, Vitaly Feldman, Cristóbal Guzmán, Kunal Talwar

2006.06914

Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis

Jingyuan Wang, Ze Wang, Jianfeng Li, Junjie Wu

1806.08946

Weight Poisoning Attacks on Pre-trained Models

Keita Kurita, Paul Michel, Graham Neubig

2004.06660

Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging

Luke Oakden-Rayner, Jared Dunnmon, Gustavo Carneiro, Christopher Ré

1909.12475

Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop

Dmitry Kovalev, Samuel Horvath, Peter Richtarik

1901.08689

Independence Promoted Graph Disentangled Networks

Yanbei Liu, Xiao Wang, Shu Wu, Zhitao Xiao

1911.11430

Stochastic Dual Coordinate Ascent with Adaptive Probabilities

Dominik Csiba, Zheng Qu, Peter Richtárik

1502.08053

Max-MIG: an Information Theoretic Approach for Joint Learning from Crowds

Peng Cao, Yilun Xu, Yuqing Kong, Yizhou Wang

1905.13436

Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach

Minhao Cheng, Thong Le, Pin-Yu Chen, Jinfeng Yi, Huan Zhang, Cho-Jui Hsieh

1807.04457

Learning Visual Reasoning Without Strong Priors

Ethan Perez, Harm de Vries, Florian Strub, Vincent Dumoulin, Aaron Courville

1707.03017

On Theory for BART

Veronika Rockova, Enakshi Saha

1810.00787

Global Non-convex Optimization with Discretized Diffusions

Murat A. Erdogdu, Lester Mackey, Ohad Shamir

1810.12361

Limitations of the Lipschitz constant as a defense against adversarial examples

Todd Huster, Cho-Yu Jason Chiang, Ritu Chadha

1807.09705

What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?

Chi Jin, Praneeth Netrapalli, Michael I. Jordan

1902.00618

Graph Backdoor

Zhaohan Xi, Ren Pang, Shouling Ji, Ting Wang

2006.11890

Optimized Data Pre-Processing for Discrimination Prevention

Flavio P. Calmon, Dennis Wei, Karthikeyan Natesan Ramamurthy, Kush R. Varshney

1704.03354

Unlearnable Examples: Making Personal Data Unexploitable

Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, Yisen Wang

2101.04898

Analyzing biological and artificial neural networks: challenges with opportunities for synergy?

David G. T. Barrett, Ari S. Morcos, Jakob H. Macke

1810.13373

Fast classification using sparse decision DAGs

Djalel Benbouzid, Robert Busa-Fekete, Balazs Kegl

1206.6387

A Fair Comparison of Graph Neural Networks for Graph Classification

Federico Errica, Marco Podda, Davide Bacciu, Alessio Micheli

1912.09893

Quantizing deep convolutional networks for efficient inference: A whitepaper

Raghuraman Krishnamoorthi

1806.08342

Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors

Andrew Ilyas, Logan Engstrom, Aleksander Madry

1807.07978

Generative Flow Networks for Discrete Probabilistic Modeling

Dinghuai Zhang, Nikolay Malkin, Zhen Liu, Alexandra Volokhova, Aaron Courville, Yoshua Bengio

2202.01361

An EM Approach to Non-autoregressive Conditional Sequence Generation

Zhiqing Sun, Yiming Yang

2006.16378

Learning What and Where to Transfer

Yunhun Jang, Hankook Lee, Sung Ju Hwang, Jinwoo Shin

1905.05901

Stochastic Gradient Descent in Continuous Time

Justin Sirignano, Konstantinos Spiliopoulos

1611.05545

Learning Single-Index Models with Shallow Neural Networks

Alberto Bietti, Joan Bruna, Clayton Sanford, Min Jae Song

2210.15651

Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond

Fanghui Liu, Xiaolin Huang, Yudong Chen, Johan A. K. Suykens

2004.11154

Smoothness and Stability in GANs

Casey Chu, Kentaro Minami, Kenji Fukumizu

2002.04185

Ray: A Distributed Framework for Emerging AI Applications

Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, Ion Stoica

1712.05889

ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization

Xiangyi Chen, Sijia Liu, Kaidi Xu, Xingguo Li, Xue Lin, Mingyi Hong, David Cox

1910.06513

Unsupervised Representation Learning by Predicting Random Distances

Hu Wang, Guansong Pang, Chunhua Shen, Congbo Ma

1912.12186

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Zhong-Qiu Wang, Jonathan Le Roux, DeLiang Wang, John R. Hershey

1804.10204

Convex Tensor Decomposition via Structured Schatten Norm Regularization

Ryota Tomioka, Taiji Suzuki

1303.6370

Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models

Giannis Daras, Augustus Odena, Han Zhang, Alexandros G. Dimakis

1911.12287

FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching

Farzin Haddadpour, Belhal Karimi, Ping Li, Xiaoyun Li

2008.04975

Kernel regression in high dimensions: Refined analysis beyond double descent

Fanghui Liu, Zhenyu Liao, Johan A. K. Suykens

2010.02681

GRAND: Graph Neural Diffusion

Benjamin Paul Chamberlain, James Rowbottom, Maria Gorinova, Stefan Webb, Emanuele Rossi, Michael M. Bronstein

2106.10934

Deep learning generalizes because the parameter-function map is biased towards simple functions

Guillermo Valle-Pérez, Chico Q. Camargo, Ard A. Louis

1805.08522

Exponentially Improving the Complexity of Simulating the Weisfeiler-Lehman Test with Graph Neural Networks

Anders Aamand, Justin Y. Chen, Piotr Indyk, Shyam Narayanan, Ronitt Rubinfeld, Nicholas Schiefer, Sandeep Silwal, Tal Wagner

2211.03232

Provable Guarantees for Gradient-Based Meta-Learning

Mikhail Khodak, Maria-Florina Balcan, Ameet Talwalkar

1902.10644

Computing Graph Neural Networks: A Survey from Algorithms to Accelerators

Sergi Abadal, Akshay Jain, Robert Guirado, Jorge López-Alonso, Eduard Alarcón

2010.00130

Three scenarios for continual learning

Gido M. van de Ven, Andreas S. Tolias

1904.07734

Graph Matching Networks for Learning the Similarity of Graph Structured Objects

Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, Pushmeet Kohli

1904.12787

Learning Factorized Multimodal Representations

Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov

1806.06176

Continual Learning via Sequential Function-Space Variational Inference

Tim G. J. Rudner, Freddie Bickford Smith, Qixuan Feng, Yee Whye Teh, Yarin Gal

2312.17210

The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models

Alexander Pan, Kush Bhatia, Jacob Steinhardt

2201.03544

Batch Stationary Distribution Estimation

Junfeng Wen, Bo Dai, Lihong Li, Dale Schuurmans

2003.00722

BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization

Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, Eytan Bakshy

1910.06403

A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning

Amy Zhang, Nicolas Ballas, Joelle Pineau

1806.07937

Explainable Machine Learning for Scientific Insights and Discoveries

Ribana Roscher, Bastian Bohn, Marco F. Duarte, Jochen Garcke

1905.08883

Scattering Networks for Hybrid Representation Learning

Edouard Oyallon, Sergey Zagoruyko, Gabriel Huang, Nikos Komodakis, Simon Lacoste-Julien, Matthew Blaschko, Eugene Belilovsky

1809.06367

Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions

Alekh Agarwal, Sahand N. Negahban, Martin J. Wainwright

1102.4807

Machine Learning on Graphs: A Model and Comprehensive Taxonomy

Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher Ré, Kevin Murphy

2005.03675

Implicit regularization of deep residual networks towards neural ODEs

Pierre Marion, Yu-Han Wu, Michael E. Sander, Gérard Biau

2309.01213

Invariant Rationalization

Shiyu Chang, Yang Zhang, Mo Yu, Tommi S. Jaakkola

2003.09772

Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation

Mikhail Belkin

2105.14368

Signal Recovery on Incoherent Manifolds

Chinmay Hegde, Richard G. Baraniuk

1202.1595

Efficient Bayesian Inference for Generalized Bradley-Terry Models

Francois Caron, Arnaud Doucet

1011.1761

Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning

Johann Brehmer, Siddharth Mishra-Sharma, Joeri Hermans, Gilles Louppe, Kyle Cranmer

1909.02005

Adversarial Neural Machine Translation

Lijun Wu, Yingce Xia, Li Zhao, Fei Tian, Tao Qin, Jianhuang Lai, Tie-Yan Liu

1704.06933

Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis

Bo Jiang, Tianyi Lin, Shiqian Ma, Shuzhong Zhang

1605.02408

Unmasking DeepFakes with simple Features

Ricard Durall, Margret Keuper, Franz-Josef Pfreundt, Janis Keuper

1911.00686

Efficient Formal Safety Analysis of Neural Networks

Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, Suman Jana

1809.08098

PRNet: Self-Supervised Learning for Partial-to-Partial Registration

Yue Wang, Justin M. Solomon

1910.12240

Nonparametric Modern Hopfield Models

Jerry Yao-Chieh Hu, Bo-Yu Chen, Dennis Wu, Feng Ruan, Han Liu

2404.03900

Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs

Jonathan Frankle, David J. Schwab, Ari S. Morcos

2003.00152

Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning

Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, Karol Hausman

1910.11956

SGD: General Analysis and Improved Rates

Robert Mansel Gower, Nicolas Loizou, Xun Qian, Alibek Sailanbayev, Egor Shulgin, Peter Richtarik

1901.09401

DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction

Aviral Kumar, Abhishek Gupta, Sergey Levine

2003.07305

Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs

Dávid Pál, Barnabás Póczos, Csaba Szepesvári

1003.1954

Online Meta-Learning

Chelsea Finn, Aravind Rajeswaran, Sham Kakade, Sergey Levine

1902.08438

Learning from Between-class Examples for Deep Sound Recognition

Yuji Tokozume, Yoshitaka Ushiku, Tatsuya Harada

1711.10282

Function Classes for Identifiable Nonlinear Independent Component Analysis

Simon Buchholz, Michel Besserve, Bernhard Schölkopf

2208.06406

Structure Adaptive Algorithms for Stochastic Bandits

Rémy Degenne, Han Shao, Wouter M. Koolen

2007.00969

Learning Rate Adaptation for Federated and Differentially Private Learning

Antti Koskela, Antti Honkela

1809.03832

FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning

Hong-You Chen, Wei-Lun Chao

2009.01974

Fastfood: Approximate Kernel Expansions in Loglinear Time

Quoc Viet Le, Tamas Sarlos, Alexander Johannes Smola

1408.3060

DLGNet: A Transformer-based Model for Dialogue Response Generation

Oluwatobi Olabiyi, Erik T. Mueller

1908.01841

The Discrete Gaussian for Differential Privacy

Clément L. Canonne, Gautam Kamath, Thomas Steinke

2004.00010

Towards a Neural Statistician

Harrison Edwards, Amos Storkey

1606.02185

On Formalizing Fairness in Prediction with Machine Learning

Pratik Gajane, Mykola Pechenizkiy

1710.03184

Information-theoretic analysis of generalization capability of learning algorithms

Aolin Xu, Maxim Raginsky

1705.07809

Bayesian representation learning with oracle constraints

Theofanis Karaletsos, Serge Belongie, Gunnar Rätsch

1506.05011

Deep Learning Approximation for Stochastic Control Problems

Jiequn Han, Weinan E

1611.07422

Fairness in Recommendation Ranking through Pairwise Comparisons

Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H. Chi, Cristos Goodrow

1903.00780

SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models

Zaccharie Ramzi, Florian Mannel, Shaojie Bai, Jean-Luc Starck, Philippe Ciuciu, Thomas Moreau

2106.00553

Exact Post Model Selection Inference for Marginal Screening

Jason D Lee, Jonathan E Taylor

1402.5596

A Distributed Second-Order Algorithm You Can Trust

Celestine Dünner, Aurelien Lucchi, Matilde Gargiani, An Bian, Thomas Hofmann, Martin Jaggi

1806.07569

word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

Yoav Goldberg, Omer Levy

1402.3722

Normalization Techniques in Training DNNs: Methodology, Analysis and Application

Lei Huang, Jie Qin, Yi Zhou, Fan Zhu, Li Liu, Ling Shao

2009.12836

Graph Neural Ordinary Differential Equations

Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, Jinkyoo Park

1911.07532

Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers

Yukuan Yang, Shuang Wu, Lei Deng, Tianyi Yan, Yuan Xie, Guoqi Li

1909.02384

Asynchronous Anytime Sequential Monte Carlo

Brooks Paige, Frank Wood, Arnaud Doucet, Yee Whye Teh

1407.2864

Deep Generative Adversarial Networks for Compressed Sensing Automates MRI

Morteza Mardani, Enhao Gong, Joseph Y. Cheng, Shreyas Vasanawala, Greg Zaharchuk, Marcus Alley, Neil Thakur, Song Han, William Dally, John M. Pauly, Lei Xing

1706.00051

Efficient Neural Causal Discovery without Acyclicity Constraints

Phillip Lippe, Taco Cohen, Efstratios Gavves

2107.10483

Ridge Regression: Structure, Cross-Validation, and Sketching

Sifan Liu, Edgar Dobriban

1910.02373

Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved

Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, Madeleine Udell

1811.11154

SCAFFOLD: Stochastic Controlled Averaging for Federated Learning

Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh

1910.06378

Ansor: Generating High-Performance Tensor Programs for Deep Learning

Lianmin Zheng, Chengfan Jia, Minmin Sun, Zhao Wu, Cody Hao Yu, Ameer Haj-Ali, Yida Wang, Jun Yang, Danyang Zhuo, Koushik Sen, Joseph E. Gonzalez, Ion Stoica

2006.06762

How to train your neural ODE: the world of Jacobian and kinetic regularization

Chris Finlay, Jörn-Henrik Jacobsen, Levon Nurbekyan, Adam M Oberman

2002.02798

Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Balaji Lakshminarayanan

1906.02994

Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems

Dongkun Zhang, Lu Lu, Ling Guo, George Em Karniadakis

1809.08327

Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders

Nat Dilokthanakul, Pedro A. M. Mediano, Marta Garnelo, Matthew C. H. Lee, Hugh Salimbeni, Kai Arulkumaran, Murray Shanahan

1611.02648

Stop Wasting My Time! Saving Days of ImageNet and BERT Training with Latest Weight Averaging

Jean Kaddour

2209.14981

A General Pipeline for 3D Detection of Vehicles

Xinxin Du, Marcelo H. Ang, Sertac Karaman, Daniela Rus

1803.00387

Towards Stable and Efficient Training of Verifiably Robust Neural Networks

Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, Cho-Jui Hsieh

1906.06316

ASAP: Adaptive Structure Aware Pooling for Learning Hierarchical Graph Representations

Ekagra Ranjan, Soumya Sanyal, Partha Pratim Talukdar

1911.07979

Hypernetwork Knowledge Graph Embeddings

Ivana Balažević, Carl Allen, Timothy M. Hospedales

1808.07018

Streaming Variational Bayes

Tamara Broderick, Nicholas Boyd, Andre Wibisono, Ashia C. Wilson, Michael I. Jordan

1307.6769

Privacy Amplification by Iteration

Vitaly Feldman, Ilya Mironov, Kunal Talwar, Abhradeep Thakurta

1808.06651

Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime

Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli, Florent Krzakala

2003.01054

Differentiable Convex Optimization Layers

Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, Zico Kolter

1910.12430

Mean-field theory of graph neural networks in graph partitioning

Tatsuro Kawamoto, Masashi Tsubaki, Tomoyuki Obuchi

1810.11908

DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks

Xingjian Li, Haoyi Xiong, Hanchao Wang, Yuxuan Rao, Liping Liu, Zeyu Chen, Jun Huan

1901.09229

Near-Optimal Stochastic Approximation for Online Principal Component Estimation

Chris Junchi Li, Mengdi Wang, Han Liu, Tong Zhang

1603.05305

Developing a Recommendation Benchmark for MLPerf Training and Inference

Carole-Jean Wu, Robin Burke, Ed H. Chi, Joseph Konstan, Julian McAuley, Yves Raimond, Hao Zhang

2003.07336

Subsampled Rényi Differential Privacy and Analytical Moments Accountant

Yu-Xiang Wang, Borja Balle, Shiva Kasiviswanathan

1808.00087

KNG: The K-Norm Gradient Mechanism

Matthew Reimherr, Jordan Awan

1905.09436

Controllable Pareto Multi-Task Learning

Xi Lin, Zhiyuan Yang, Qingfu Zhang, Sam Kwong

2010.06313

Generalization and Regularization in DQN

Jesse Farebrother, Marlos C. Machado, Michael Bowling

1810.00123

Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers

Georgios P. Spithourakis, Sebastian Riedel

1805.08154

Semi-Supervised Neural Architecture Search

Renqian Luo, Xu Tan, Rui Wang, Tao Qin, Enhong Chen, Tie-Yan Liu

2002.10389

Normalizing Flows on Tori and Spheres

Danilo Jimenez Rezende, George Papamakarios, Sébastien Racanière, Michael S. Albergo, Gurtej Kanwar, Phiala E. Shanahan, Kyle Cranmer

2002.02428

Attack of the Tails: Yes, You Really Can Backdoor Federated Learning

Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, Dimitris Papailiopoulos

2007.05084

This Looks Like That: Deep Learning for Interpretable Image Recognition

Chaofan Chen, Oscar Li, Chaofan Tao, Alina Jade Barnett, Jonathan Su, Cynthia Rudin

1806.10574

Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules

Sarthak Mittal, Alex Lamb, Anirudh Goyal, Vikram Voleti, Murray Shanahan, Guillaume Lajoie, Michael Mozer, Yoshua Bengio

2006.16981

Intensity-Free Learning of Temporal Point Processes

Oleksandr Shchur, Marin Biloš, Stephan Günnemann

1909.12127

Adaptive norms for deep learning with regularized Newton methods

Jonas Kohler, Leonard Adolphs, Aurelien Lucchi

1905.09201

On the Implicit Bias of Dropout

Poorya Mianjy, Raman Arora, Rene Vidal

1806.09777

Hint-Based Training for Non-Autoregressive Machine Translation

Zhuohan Li, Zi Lin, Di He, Fei Tian, Tao Qin, Liwei Wang, Tie-Yan Liu

1909.06708

Evaluating Compositionality in Sentence Embeddings

Ishita Dasgupta, Demi Guo, Andreas Stuhlmüller, Samuel J. Gershman, Noah D. Goodman

1802.04302

Understanding Self-supervised Learning with Dual Deep Networks

Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli

2010.00578

Tempered Sigmoid Activations for Deep Learning with Differential Privacy

Nicolas Papernot, Abhradeep Thakurta, Shuang Song, Steve Chien, Úlfar Erlingsson

2007.14191

How does Disagreement Help Generalization against Label Corruption?

Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor W. Tsang, Masashi Sugiyama

1901.04215

Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs

Etienne Boursier, Loucas Pillaud-Vivien, Nicolas Flammarion

2206.00939

TiFL: A Tier-based Federated Learning System

Zheng Chai, Ahsan Ali, Syed Zawad, Stacey Truex, Ali Anwar, Nathalie Baracaldo, Yi Zhou, Heiko Ludwig, Feng Yan, Yue Cheng

2001.09249

Asymmetric Valleys: Beyond Sharp and Flat Local Minima

Haowei He, Gao Huang, Yang Yuan

1902.00744

Symmetry, Saddle Points, and Global Optimization Landscape of Nonconvex Matrix Factorization

Xingguo Li, Junwei Lu, Raman Arora, Jarvis Haupt, Han Liu, Zhaoran Wang, Tuo Zhao

1612.09296

The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks

Ryo Karakida, Shotaro Akaho, Shun-ichi Amari

1906.02926

When Does Differentially Private Learning Not Suffer in High Dimensions?

Xuechen Li, Daogao Liu, Tatsunori Hashimoto, Huseyin A. Inan, Janardhan Kulkarni, Yin Tat Lee, Abhradeep Guha Thakurta

2207.00160

Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learning

Christoph Dann, Tor Lattimore, Emma Brunskill

1703.07710

Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement

Benjamin Eysenbach, Xinyang Geng, Sergey Levine, Ruslan Salakhutdinov

2002.11089

Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks

Abdulkadir Canatar, Blake Bordelon, Cengiz Pehlevan

2006.13198

Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques

Kundan Krishna, Sopan Khosla, Jeffrey P. Bigham, Zachary C. Lipton

2005.01795

Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning

Daniel Kuhn, Peyman Mohajerin Esfahani, Viet Anh Nguyen, Soroosh Shafieezadeh-Abadeh

1908.08729

Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space

Jiechao Xiong, Qing Wang, Zhuoran Yang, Peng Sun, Lei Han, Yang Zheng, Haobo Fu, Tong Zhang, Ji Liu, Han Liu

1810.06394

MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing

Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, Aram Galstyan

1905.00067

Nash Learning from Human Feedback

Rémi Munos, Michal Valko, Daniele Calandriello, Mohammad Gheshlaghi Azar, Mark Rowland, Zhaohan Daniel Guo, Yunhao Tang, Matthieu Geist, Thomas Mesnard, Andrea Michi, Marco Selvi, Sertan Girgin, Nikola Momchev, Olivier Bachem, Daniel J. Mankowitz, Doina Precup, Bilal Piot

2312.00886

Mastering Diverse Domains through World Models

Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, Timothy Lillicrap

2301.04104

Fast Conical Hull Algorithms for Near-separable Non-negative Matrix Factorization

Abhishek Kumar, Vikas Sindhwani, Prabhanjan Kambadur

1210.1190

Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search

Zhuwen Li, Qifeng Chen, Vladlen Koltun

1810.10659

DAG-GNN: DAG Structure Learning with Graph Neural Networks

Yue Yu, Jie Chen, Tian Gao, Mo Yu

1904.10098

Learning Adversarially Fair and Transferable Representations

David Madras, Elliot Creager, Toniann Pitassi, Richard Zemel

1802.06309

Adversarial Training for Free!

Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, Tom Goldstein

1904.12843

Optimal whitening and decorrelation

Agnan Kessy, Alex Lewin, Korbinian Strimmer

1512.00809

An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks

Ian J. Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, Yoshua Bengio

1312.6211

GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training

Tianle Cai, Shengjie Luo, Keyulu Xu, Di He, Tie-Yan Liu, Liwei Wang

2009.03294

Byzantine-Tolerant Machine Learning

Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, Julien Stainer

1703.02757

Improved Asymmetric Locality Sensitive Hashing (ALSH) for Maximum Inner Product Search (MIPS)

Anshumali Shrivastava, Ping Li

1410.5410

Efficient Optimal Learning for Contextual Bandits

Miroslav Dudik, Daniel Hsu, Satyen Kale, Nikos Karampatziakis, John Langford, Lev Reyzin, Tong Zhang

1106.2369

Phasebook and Friends: Leveraging Discrete Representations for Source Separation

Jonathan Le Roux, Gordon Wichern, Shinji Watanabe, Andy Sarroff, John R. Hershey

1810.01395

Hypermodels for Exploration

Vikranth Dwaracherla, Xiuyuan Lu, Morteza Ibrahimi, Ian Osband, Zheng Wen, Benjamin Van Roy

2006.07464

Sinkformers: Transformers with Doubly Stochastic Attention

Michael E. Sander, Pierre Ablin, Mathieu Blondel, Gabriel Peyré

2110.11773

Discrete Graph Structure Learning for Forecasting Multiple Time Series

Chao Shang, Jie Chen, Jinbo Bi

2101.06861

Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shuffling

Vitaly Feldman, Audra McMillan, Kunal Talwar

2012.12803

Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate

Mor Shpigel Nacson, Nathan Srebro, Daniel Soudry

1806.01796

A Qualitative Study of the Dynamic Behavior for Adaptive Gradient Algorithms

Chao Ma, Lei Wu, Weinan E

2009.06125

Ask the GRU: Multi-Task Learning for Deep Text Recommendations

Trapit Bansal, David Belanger, Andrew McCallum

1609.02116

Semi-supervised Conditional GANs

Kumar Sricharan, Raja Bala, Matthew Shreve, Hui Ding, Kumar Saketh, Jin Sun

1708.05789

A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning

Samuel Horváth, Peter Richtárik

2006.11077

Network Flow Algorithms for Structured Sparsity

Julien Mairal, Rodolphe Jenatton, Guillaume Obozinski, Francis Bach

1008.5209

Improving Disentangled Text Representation Learning with Information-Theoretic Guidance

Pengyu Cheng, Martin Renqiang Min, Dinghan Shen, Christopher Malon, Yizhe Zhang, Yitong Li, Lawrence Carin

2006.00693

Sum-of-Squares Polynomial Flow

Priyank Jaini, Kira A. Selby, Yaoliang Yu

1905.02325

Reward Constrained Policy Optimization

Chen Tessler, Daniel J. Mankowitz, Shie Mannor

1805.11074

Consistent procedures for cluster tree estimation and pruning

Kamalika Chaudhuri, Sanjoy Dasgupta, Samory Kpotufe, Ulrike von Luxburg

1406.1546

Stronger Privacy Amplification by Shuffling for Rényi and Approximate Differential Privacy

Vitaly Feldman, Audra McMillan, Kunal Talwar

2208.04591

Hopfield Networks is All You Need

Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, Victor Greiff, David Kreil, Michael Kopp, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter

2008.02217

Variational Autoencoders for Learning Latent Representations of Speech Emotion: A Preliminary Study

Siddique Latif, Rajib Rana, Junaid Qadir, Julien Epps

1712.08708

Minimax Weight and Q-Function Learning for Off-Policy Evaluation

Masatoshi Uehara, Jiawei Huang, Nan Jiang

1910.12809

Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations

Polina Kirichenko, Pavel Izmailov, Andrew Gordon Wilson

2204.02937

Exactly Computing the Local Lipschitz Constant of ReLU Networks

Matt Jordan, Alexandros G. Dimakis

2003.01219

ResNet with one-neuron hidden layers is a Universal Approximator

Hongzhou Lin, Stefanie Jegelka

1806.10909

Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Server

Leonard Hasenclever, Stefan Webb, Thibaut Lienart, Sebastian Vollmer, Balaji Lakshminarayanan, Charles Blundell, Yee Whye Teh

1512.09327

Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions

Xuxing Chen, Tesi Xiao, Krishnakumar Balasubramanian

2306.12067

Equivariant Transformer Networks

Kai Sheng Tai, Peter Bailis, Gregory Valiant

1901.11399

Sub-sampled Newton Methods with Non-uniform Sampling

Peng Xu, Jiyan Yang, Farbod Roosta-Khorasani, Christopher Ré, Michael W. Mahoney

1607.00559

Monotone operator equilibrium networks

Ezra Winston, J. Zico Kolter

2006.08591

A Rank-Corrected Procedure for Matrix Completion with Fixed Basis Coefficients

Weimin Miao, Shaohua Pan, Defeng Sun

1210.3709

FedML: A Research Library and Benchmark for Federated Machine Learning

Chaoyang He, Songze Li, Jinhyun So, Xiao Zeng, Mi Zhang, Hongyi Wang, Xiaoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, Xinghua Zhu, Jianzong Wang, Li Shen, Peilin Zhao, Yan Kang, Yang Liu, Ramesh Raskar, Qiang Yang, Murali Annavaram, Salman Avestimehr

2007.13518

On Learning Rates and Schrödinger Operators

Bin Shi, Weijie J. Su, Michael I. Jordan

2004.06977

Efficient Low-rank Multimodal Fusion with Modality-Specific Factors

Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency

1806.00064

Using Natural Language for Reward Shaping in Reinforcement Learning

Prasoon Goyal, Scott Niekum, Raymond J. Mooney

1903.02020

On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data

Nan Lu, Gang Niu, Aditya Krishna Menon, Masashi Sugiyama

1808.10585

Convergence of gradient descent for deep neural networks

Sourav Chatterjee

2203.16462

Hierarchical Graph Pooling with Structure Learning

Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Chengwei Yao, Zhi Yu, Can Wang

1911.05954

Weisfeiler and Leman go Machine Learning: The Story so far

Christopher Morris, Yaron Lipman, Haggai Maron, Bastian Rieck, Nils M. Kriege, Martin Grohe, Matthias Fey, Karsten Borgwardt

2112.09992

Explaining by Removing: A Unified Framework for Model Explanation

Ian Covert, Scott Lundberg, Su-In Lee

2011.14878

Min-Max Kernels

Ping Li

1503.01737

Representation Learning via Invariant Causal Mechanisms

Jovana Mitrovic, Brian McWilliams, Jacob Walker, Lars Buesing, Charles Blundell

2010.07922

Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks

Marwin H. S. Segler, Thierry Kogej, Christian Tyrchan, Mark P. Waller

1701.01329

Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation

Micah J Sheller, G Anthony Reina, Brandon Edwards, Jason Martin, Spyridon Bakas

1810.04304

Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation

Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, Marjan Ghazvininejad

1902.01509

When OT meets MoM: Robust estimation of Wasserstein Distance

Guillaume Staerman, Pierre Laforgue, Pavlo Mozharovskyi, Florence d'Alché-Buc

2006.10325

Beyond variance reduction: Understanding the true impact of baselines on policy optimization

Wesley Chung, Valentin Thomas, Marlos C. Machado, Nicolas Le Roux

2008.13773

Extracting Structured Data from Physician-Patient Conversations By Predicting Noteworthy Utterances

Kundan Krishna, Amy Pavel, Benjamin Schloss, Jeffrey P. Bigham, Zachary C. Lipton

2007.07151

Learning Convolutional Transforms for Lossy Point Cloud Geometry Compression

Maurice Quach, Giuseppe Valenzise, Frederic Dufaux

1903.08548

Interventions, Where and How? Experimental Design for Causal Models at Scale

Panagiotis Tigas, Yashas Annadani, Andrew Jesson, Bernhard Schölkopf, Yarin Gal, Stefan Bauer

2203.02016

High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification

Edgar Dobriban, Stefan Wager

1507.03003

Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs

W. James Murdoch, Peter J. Liu, Bin Yu

1801.05453

Scalable Methods for 8-bit Training of Neural Networks

Ron Banner, Itay Hubara, Elad Hoffer, Daniel Soudry

1805.11046

Estimation of Low-Rank Matrices via Approximate Message Passing

Andrea Montanari, Ramji Venkataramanan

1711.01682

High-dimensional limit theorems for SGD: Effective dynamics and critical scaling

Gerard Ben Arous, Reza Gheissari, Aukosh Jagannath

2206.04030

From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following

Justin Fu, Anoop Korattikara, Sergey Levine, Sergio Guadarrama

1902.07742

Pre-training via Paraphrasing

Mike Lewis, Marjan Ghazvininejad, Gargi Ghosh, Armen Aghajanyan, Sida Wang, Luke Zettlemoyer

2006.15020

Is Local SGD Better than Minibatch SGD?

Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich, Zhen Dai, Brian Bullins, H. Brendan McMahan, Ohad Shamir, Nathan Srebro

2002.07839

Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks

Tianyu Pang, Kun Xu, Jun Zhu

1909.11515

High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm

Wenlong Mou, Yi-An Ma, Martin J. Wainwright, Peter L. Bartlett, Michael I. Jordan

1908.10859

Prevalence of Neural Collapse during the terminal phase of deep learning training

Vardan Papyan, X. Y. Han, David L. Donoho

2008.08186

Neural Rough Differential Equations for Long Time Series

James Morrill, Cristopher Salvi, Patrick Kidger, James Foster, Terry Lyons

2009.08295

Hidden Incentives for Auto-Induced Distributional Shift

David Krueger, Tegan Maharaj, Jan Leike

2009.09153

Understanding the Failure Modes of Out-of-Distribution Generalization

Vaishnavh Nagarajan, Anders Andreassen, Behnam Neyshabur

2010.15775

Markov Determinantal Point Processes

Raja Hafiz Affandi, Alex Kulesza, Emily B. Fox

1210.4850

Support vector machines and linear regression coincide with very high-dimensional features

Navid Ardeshir, Clayton Sanford, Daniel Hsu

2105.14084

A* Sampling

Chris J. Maddison, Daniel Tarlow, Tom Minka

1411.0030

Attention-Based Models for Text-Dependent Speaker Verification

F A Rezaur Rahman Chowdhury, Quan Wang, Ignacio Lopez Moreno, Li Wan

1710.10470

ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies

Bao Wang, Binjie Yuan, Zuoqiang Shi, Stanley J. Osher

1811.10745

Principal Neighbourhood Aggregation for Graph Nets

Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, Petar Veličković

2004.05718

Ensemble Sampling

Xiuyuan Lu, Benjamin Van Roy

1705.07347

The Power of Exploiter: Provable Multi-Agent RL in Large State Spaces

Chi Jin, Qinghua Liu, Tiancheng Yu

2106.03352

Anomaly Detection with Density Estimation

Benjamin Nachman, David Shih

2001.04990

Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Features

Robin Tibor Schirrmeister, Yuxuan Zhou, Tonio Ball, Dan Zhang

2006.10848

Deep Learning for Spatio-Temporal Modeling: Dynamic Traffic Flows and High Frequency Trading

Matthew F. Dixon, Nicholas G. Polson, Vadim O. Sokolov

1705.09851

Markerless tracking of user-defined features with deep learning

Alexander Mathis, Pranav Mamidanna, Taiga Abe, Kevin M. Cury, Venkatesh N. Murthy, Mackenzie W. Mathis, Matthias Bethge

1804.03142

Residual Unfairness in Fair Machine Learning from Prejudiced Data

Nathan Kallus, Angela Zhou

1806.02887

Trajectory balance: Improved credit assignment in GFlowNets

Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, Yoshua Bengio

2201.13259

The Hanabi Challenge: A New Frontier for AI Research

Nolan Bard, Jakob N. Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H. Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, Iain Dunning, Shibl Mourad, Hugo Larochelle, Marc G. Bellemare, Michael Bowling

1902.00506

Decoupled Networks

Weiyang Liu, Zhen Liu, Zhiding Yu, Bo Dai, Rongmei Lin, Yisen Wang, James M. Rehg, Le Song

1804.08071

Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitions

Valentina Ros, Gerard Ben Arous, Giulio Biroli, Chiara Cammarota

1804.02686

Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing

Jinyuan Jia, Xiaoyu Cao, Binghui Wang, Neil Zhenqiang Gong

1912.09899

A Dynamical Central Limit Theorem for Shallow Neural Networks

Zhengdao Chen, Grant M. Rotskoff, Joan Bruna, Eric Vanden-Eijnden

2008.09623

Rényi Divergence Variational Inference

Yingzhen Li, Richard E. Turner

1602.02311

Deep Counterfactual Networks with Propensity-Dropout

Ahmed M. Alaa, Michael Weisz, Mihaela van der Schaar

1706.05966

Theoretical guarantees for sampling and inference in generative models with latent diffusions

Belinda Tzen, Maxim Raginsky

1903.01608

Ensemble Distillation for Robust Model Fusion in Federated Learning

Tao Lin, Lingjing Kong, Sebastian U. Stich, Martin Jaggi

2006.07242

FetchSGD: Communication-Efficient Federated Learning with Sketching

Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph Gonzalez, Raman Arora

2007.07682

Depth Uncertainty in Neural Networks

Javier Antorán, James Urquhart Allingham, José Miguel Hernández-Lobato

2006.08437

Error analysis for denoising smooth modulo signals on a graph

Hemant Tyagi

2009.04859

Bayesian Optimization of Composite Functions

Raul Astudillo, Peter I. Frazier

1906.01537

iNNvestigate neural networks!

Maximilian Alber, Sebastian Lapuschkin, Philipp Seegerer, Miriam Hägele, Kristof T. Schütt, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller, Sven Dähne, Pieter-Jan Kindermans

1808.04260

Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise

Umut Şimşekli, Lingjiong Zhu, Yee Whye Teh, Mert Gürbüzbalaban

2002.05685

AUC Maximization in the Era of Big Data and AI: A Survey

Tianbao Yang, Yiming Ying

2203.15046

Active Learning for Node Classification in Assortative and Disassortative Networks

Cristopher Moore, Xiaoran Yan, Yaojia Zhu, Jean-Baptiste Rouquier, Terran Lane

1109.3240

On Regret-Optimal Learning in Decentralized Multi-player Multi-armed Bandits

Naumaan Nayyar, Dileep Kalathil, Rahul Jain

1505.00553

Generalization bounds for neural ordinary differential equations and deep residual networks

Pierre Marion

2305.06648

On the Complexity of A/B Testing

Emilie Kaufmann, Olivier Cappé, Aurélien Garivier

1405.3224

From Local SGD to Local Fixed-Point Methods for Federated Learning

Grigory Malinovsky, Dmitry Kovalev, Elnur Gasanov, Laurent Condat, Peter Richtárik

2004.01442

Non-Gaussian processes and neural networks at finite widths

Sho Yaida

1910.00019

Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets

Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, Xingjun Ma

2002.05990

Learning Task Grouping and Overlap in Multi-task Learning

Abhishek Kumar, Hal Daume

1206.6417

Generative replay with feedback connections as a general strategy for continual learning

Gido M. van de Ven, Andreas S. Tolias

1809.10635

Evolutionary Reinforcement Learning for Sample-Efficient Multiagent Coordination

Shauharda Khadka, Somdeb Majumdar, Santiago Miret, Stephen McAleer, Kagan Tumer

1906.07315

How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks

Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S. Du, Ken-ichi Kawarabayashi, Stefanie Jegelka

2009.11848

Private Selection from Private Candidates

Jingcheng Liu, Kunal Talwar

1811.07971

On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo

Niladri S. Chatterji, Nicolas Flammarion, Yi-An Ma, Peter L. Bartlett, Michael I. Jordan

1802.05431

Video Compression With Rate-Distortion Autoencoders

Amirhossein Habibian, Ties van Rozendaal, Jakub M. Tomczak, Taco S. Cohen

1908.05717

Deep Interest Evolution Network for Click-Through Rate Prediction

Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, Kun Gai

1809.03672

Analyzing and Improving Representations with the Soft Nearest Neighbor Loss

Nicholas Frosst, Nicolas Papernot, Geoffrey Hinton

1902.01889

Large image datasets: A pyrrhic win for computer vision?

Vinay Uday Prabhu, Abeba Birhane

2006.16923

Multi-scale Mining of fMRI data with Hierarchical Structured Sparsity

Rodolphe Jenatton, Alexandre Gramfort, Vincent Michel, Guillaume Obozinski, Evelyn Eger, Francis Bach, Bertrand Thirion

1105.0363

Provably Efficient Exploration in Policy Optimization

Qi Cai, Zhuoran Yang, Chi Jin, Zhaoran Wang

1912.05830

Optimal estimation of Gaussian DAG models

Ming Gao, Wai Ming Tai, Bryon Aragam

2201.10548

Dark Experience for General Continual Learning: a Strong, Simple Baseline

Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, Simone Calderara

2004.07211

A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks

Owen Marschall, Kyunghyun Cho, Cristina Savin

1907.02649

Learning Neural Causal Models from Unknown Interventions

Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Bernhard Schölkopf, Michael C. Mozer, Chris Pal, Yoshua Bengio

1910.01075

Sponge Examples: Energy-Latency Attacks on Neural Networks

Ilia Shumailov, Yiren Zhao, Daniel Bates, Nicolas Papernot, Robert Mullins, Ross Anderson

2006.03463

OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport

Derek Onken, Samy Wu Fung, Xingjian Li, Lars Ruthotto

2006.00104

Semi-Supervised Learning with Normalizing Flows

Pavel Izmailov, Polina Kirichenko, Marc Finzi, Andrew Gordon Wilson

1912.13025

Equivariance Through Parameter-Sharing

Siamak Ravanbakhsh, Jeff Schneider, Barnabas Poczos

1702.08389

Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks

Tribhuvanesh Orekondy, Bernt Schiele, Mario Fritz

1906.10908

Deepr: A Convolutional Net for Medical Records

Phuoc Nguyen, Truyen Tran, Nilmini Wickramasinghe, Svetha Venkatesh

1607.07519

A Latent Variable Model Approach to PMI-based Word Embeddings

Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, Andrej Risteski

1502.03520

Unbiased estimators for random design regression

Michał Dereziński, Manfred K. Warmuth, Daniel Hsu

1907.03411

Analyzing the Role of Model Uncertainty for Electronic Health Records

Michael W. Dusenberry, Dustin Tran, Edward Choi, Jonas Kemp, Jeremy Nixon, Ghassen Jerfel, Katherine Heller, Andrew M. Dai

1906.03842

Understanding and Improving Knowledge Distillation

Jiaxi Tang, Rakesh Shivanna, Zhe Zhao, Dong Lin, Anima Singh, Ed H. Chi, Sagar Jain

2002.03532

Learning Two-layer Neural Networks with Symmetric Inputs

Rong Ge, Rohith Kuditipudi, Zhize Li, Xiang Wang

1810.06793

Near-Optimal Offline Reinforcement Learning via Double Variance Reduction

Ming Yin, Yu Bai, Yu-Xiang Wang

2102.01748

Reinforcement Learning via Fenchel-Rockafellar Duality

Ofir Nachum, Bo Dai

2001.01866

The Social Cost of Strategic Classification

Smitha Milli, John Miller, Anca D. Dragan, Moritz Hardt

1808.08460

Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming

Saeed Ghadimi, Guanghui Lan

1309.5549

E(n) Equivariant Graph Neural Networks

Victor Garcia Satorras, Emiel Hoogeboom, Max Welling

2102.09844

User Friendly Automatic Construction of Background Knowledge: Mode Construction from ER Diagrams

Alexander L. Hayes, Mayukh Das, Phillip Odom, Sriraam Natarajan

1912.07650

Model-Free Episodic Control

Charles Blundell, Benigno Uria, Alexander Pritzel, Yazhe Li, Avraham Ruderman, Joel Z Leibo, Jack Rae, Daan Wierstra, Demis Hassabis

1606.04460

RODE: Learning Roles to Decompose Multi-Agent Tasks

Tonghan Wang, Tarun Gupta, Anuj Mahajan, Bei Peng, Shimon Whiteson, Chongjie Zhang

2010.01523

Active Learning for Cost-Sensitive Classification

Akshay Krishnamurthy, Alekh Agarwal, Tzu-Kuo Huang, Hal Daume, John Langford

1703.01014

Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation

Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Dvijotham, Pushmeet Kohli

1909.01492

Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Bound

Lin F. Yang, Mengdi Wang

1905.10389

iDLG: Improved Deep Leakage from Gradients

Bo Zhao, Konda Reddy Mopuri, Hakan Bilen

2001.02610

Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting

Hippolyt Ritter, Aleksandar Botev, David Barber

1805.07810

Efficient GAN-Based Anomaly Detection

Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, Vijay Ramaseshan Chandrasekhar

1802.06222

Theory of Graph Neural Networks: Representation and Learning

Stefanie Jegelka

2204.07697

Nearly Optimal Sampling Algorithms for Combinatorial Pure Exploration

Lijie Chen, Anupam Gupta, Jian Li, Mingda Qiao, Ruosong Wang

1706.01081

Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentation

Karl Stelzner, Kristian Kersting, Adam R. Kosiorek

2104.01148

RAB: Provable Robustness Against Backdoor Attacks

Maurice Weber, Xiaojun Xu, Bojan Karlaš, Ce Zhang, Bo Li

2003.08904

SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning

Kimin Lee, Michael Laskin, Aravind Srinivas, Pieter Abbeel

2007.04938

Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties

Benjamin Kurt Miller, Mario Geiger, Tess E. Smidt, Frank Noé

2008.08461

Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditions

Philipp Grohs, Lukas Herrmann

2007.05384

lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits

Kevin Jamieson, Matthew Malloy, Robert Nowak, Sébastien Bubeck

1312.7308

Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model

Atılım Güneş Baydin, Lukas Heinrich, Wahid Bhimji, Lei Shao, Saeid Naderiparizi, Andreas Munk, Jialin Liu, Bradley Gram-Hansen, Gilles Louppe, Lawrence Meadows, Philip Torr, Victor Lee, Prabhat, Kyle Cranmer, Frank Wood

1807.07706

Disentangling the independently controllable factors of variation by interacting with the world

Valentin Thomas, Emmanuel Bengio, William Fedus, Jules Pondard, Philippe Beaudoin, Hugo Larochelle, Joelle Pineau, Doina Precup, Yoshua Bengio

1802.09484

The Linear Representation Hypothesis and the Geometry of Large Language Models

Kiho Park, Yo Joong Choe, Victor Veitch

2311.03658

Private Adaptive Gradient Methods for Convex Optimization

Hilal Asi, John Duchi, Alireza Fallah, Omid Javidbakht, Kunal Talwar

2106.13756

Implicit Graph Neural Networks

Fangda Gu, Heng Chang, Wenwu Zhu, Somayeh Sojoudi, Laurent El Ghaoui

2009.06211

Optimal mini-batch and step sizes for SAGA

Nidham Gazagnadou, Robert M. Gower, Joseph Salmon

1902.00071

Recursive Partitioning for Heterogeneous Causal Effects

Susan Athey, Guido Imbens

1504.01132

Salvaging Federated Learning by Local Adaptation

Tao Yu, Eugene Bagdasaryan, Vitaly Shmatikov

2002.04758

Learning One Convolutional Layer with Overlapping Patches

Surbhi Goel, Adam Klivans, Raghu Meka

1802.02547

Fast Policy Learning through Imitation and Reinforcement

Ching-An Cheng, Xinyan Yan, Nolan Wagener, Byron Boots

1805.10413

Implicit Regularization Towards Rank Minimization in ReLU Networks

Nadav Timor, Gal Vardi, Ohad Shamir

2201.12760

Kernel quadrature with DPPs

Ayoub Belhadji, Rémi Bardenet, Pierre Chainais

1906.07832

Data Poisoning Attacks Against Federated Learning Systems

Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, Ling Liu

2007.08432

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF

Shicong Cen, Jincheng Mei, Katayoon Goshvadi, Hanjun Dai, Tong Yang, Sherry Yang, Dale Schuurmans, Yuejie Chi, Bo Dai

2405.19320

On Correctness of Automatic Differentiation for Non-Differentiable Functions

Wonyeol Lee, Hangyeol Yu, Xavier Rival, Hongseok Yang

2006.06903

M2m: Imbalanced Classification via Major-to-minor Translation

Jaehyung Kim, Jongheon Jeong, Jinwoo Shin

2004.00431

On the Geometric Ergodicity of Hamiltonian Monte Carlo

Samuel Livingstone, Michael Betancourt, Simon Byrne, Mark Girolami

1601.08057

Making Learners (More) Monotone

Tom J. Viering, Alexander Mey, Marco Loog

1911.11030

Does Preprocessing Help Training Over-parameterized Neural Networks?

Zhao Song, Shuo Yang, Ruizhe Zhang

2110.04622

More Adaptive Algorithms for Adversarial Bandits

Chen-Yu Wei, Haipeng Luo

1801.03265

Combining Q-Learning and Search with Amortized Value Estimates

Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Tobias Pfaff, Theophane Weber, Lars Buesing, Peter W. Battaglia

1912.02807

A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes

Zachary Nado, Justin M. Gilmer, Christopher J. Shallue, Rohan Anil, George E. Dahl

2102.06356

Learning Linear-Quadratic Regulators Efficiently with only $\sqrt{T}$ Regret

Alon Cohen, Tomer Koren, Yishay Mansour

1902.06223

Gaussian Probabilities and Expectation Propagation

John P. Cunningham, Philipp Hennig, Simon Lacoste-Julien

1111.6832

Neural-net-induced Gaussian process regression for function approximation and PDE solution

Guofei Pang, Liu Yang, George Em Karniadakis

1806.11187

Tune: A Research Platform for Distributed Model Selection and Training

Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E. Gonzalez, Ion Stoica

1807.05118

SWALP : Stochastic Weight Averaging in Low-Precision Training

Guandao Yang, Tianyi Zhang, Polina Kirichenko, Junwen Bai, Andrew Gordon Wilson, Christopher De Sa

1904.11943

Concolic Testing for Deep Neural Networks

Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska, Daniel Kroening

1805.00089

Information Directed Sampling for Linear Partial Monitoring

Johannes Kirschner, Tor Lattimore, Andreas Krause

2002.11182

An Improved Cutting Plane Method for Convex Optimization, Convex-Concave Games and its Applications

Haotian Jiang, Yin Tat Lee, Zhao Song, Sam Chiu-wai Wong

2004.04250

Learning under selective labels in the presence of expert consistency

Maria De-Arteaga, Artur Dubrawski, Alexandra Chouldechova

1807.00905

FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data

Xinwei Zhang, Mingyi Hong, Sairaj Dhople, Wotao Yin, Yang Liu

2005.11418

Online Adaptive Methods, Universality and Acceleration

Kfir Y. Levy, Alp Yurtsever, Volkan Cevher

1809.02864

Deep Mixture Point Processes: Spatio-temporal Event Prediction with Rich Contextual Information

Maya Okawa, Tomoharu Iwata, Takeshi Kurashima, Yusuke Tanaka, Hiroyuki Toda, Naonori Ueda

1906.08952

Learning Transferable Cooperative Behavior in Multi-Agent Teams

Akshat Agarwal, Sumit Kumar, Katia Sycara

1906.01202

Learning Approximate Inference Networks for Structured Prediction

Lifu Tu, Kevin Gimpel

1803.03376

Nonlinear Kalman Filtering with Divergence Minimization

San Gultekin, John Paisley

1705.00722

Group Whitening: Balancing Learning Efficiency and Representational Capacity

Lei Huang, Yi Zhou, Li Liu, Fan Zhu, Ling Shao

2009.13333

Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis

Zhongkai Sun, Prathusha Sarma, William Sethares, Yingyu Liang

1911.05544

Contrastive Representation Distillation

Yonglong Tian, Dilip Krishnan, Phillip Isola

1910.10699

Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration

Jason Altschuler, Jonathan Weed, Philippe Rigollet

1705.09634

On Symmetric and Asymmetric LSHs for Inner Product Search

Behnam Neyshabur, Nathan Srebro

1410.5518

A mathematical model for automatic differentiation in machine learning

Jerome Bolte, Edouard Pauwels

2006.02080

Coresets for Robust Training of Neural Networks against Noisy Labels

Baharan Mirzasoleiman, Kaidi Cao, Jure Leskovec

2011.07451

End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks

Szu-Wei Fu, Tao-Wei Wang, Yu Tsao, Xugang Lu, Hisashi Kawai

1709.03658

Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent

Pu Zhao, Pin-Yu Chen, Siyue Wang, Xue Lin

2002.07891

On the Identifiability of Nonlinear ICA: Sparsity and Beyond

Yujia Zheng, Ignavier Ng, Kun Zhang

2206.07751

Graph Neural Networks are Dynamic Programmers

Andrew Dudzik, Petar Veličković

2203.15544

CYCLADES: Conflict-free Asynchronous Machine Learning

Xinghao Pan, Maximilian Lam, Stephen Tu, Dimitris Papailiopoulos, Ce Zhang, Michael I. Jordan, Kannan Ramchandran, Chris Re, Benjamin Recht

1605.09721

Regularized Optimal Transport and the Rot Mover's Distance

Arnaud Dessein, Nicolas Papadakis, Jean-Luc Rouas

1610.06447

Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020

Ryan Turner, David Eriksson, Michael McCourt, Juha Kiili, Eero Laaksonen, Zhen Xu, Isabelle Guyon

2104.10201

Injecting Prior Knowledge for Transfer Learning into Reinforcement Learning Algorithms using Logic Tensor Networks

Samy Badreddine, Michael Spranger

1906.06576

Massively scalable Sinkhorn distances via the Nyström method

Jason Altschuler, Francis Bach, Alessandro Rudi, Jonathan Niles-Weed

1812.05189

Degenerate Feedback Loops in Recommender Systems

Ray Jiang, Silvia Chiappa, Tor Lattimore, András György, Pushmeet Kohli

1902.10730

A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark

Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, Neil Houlsby

1910.04867

Distributed Nesterov gradient methods over arbitrary graphs

Ran Xin, Dusan Jakovetic, Usman A. Khan

1901.06995

Training Simplification and Model Simplification for Deep Learning: A Minimal Effort Back Propagation Method

Xu Sun, Xuancheng Ren, Shuming Ma, Bingzhen Wei, Wei Li, Jingjing Xu, Houfeng Wang, Yi Zhang

1711.06528

High Acceleration Reinforcement Learning for Real-World Juggling with Binary Rewards

Kai Ploeger, Michael Lutter, Jan Peters

2010.13483

Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering

Ramakrishna Vedantam, Karan Desai, Stefan Lee, Marcus Rohrbach, Dhruv Batra, Devi Parikh

1902.07864

Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning

Zachary Charles, Jakub Konečný

2103.05032

Meta-Learning Symmetries by Reparameterization

Allan Zhou, Tom Knowles, Chelsea Finn

2007.02933

Bypassing Backdoor Detection Algorithms in Deep Learning

Te Juin Lester Tan, Reza Shokri

1905.13409

Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks

Siyue Wang, Xiao Wang, Pu Zhao, Wujie Wen, David Kaeli, Peter Chin, Xue Lin

1809.05165

Learning Mixtures of Submodular Shells with Application to Document Summarization

Hui Lin, Jeff A. Bilmes

1210.4871

Telescoping Density-Ratio Estimation

Benjamin Rhodes, Kai Xu, Michael U. Gutmann

2006.12204

Supervising strong learners by amplifying weak experts

Paul Christiano, Buck Shlegeris, Dario Amodei

1810.08575

Discovering Phase Transitions with Unsupervised Learning

Lei Wang

1606.00318

BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model

Chenwei Xu, Yu-Chao Huang, Jerry Yao-Chieh Hu, Weijian Li, Ammar Gilani, Hsi-Sheng Goan, Han Liu

2404.03830

SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning

Marvin Zhang, Sharad Vikram, Laura Smith, Pieter Abbeel, Matthew J. Johnson, Sergey Levine

1808.09105

Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning

Yizhu Jiao, Yun Xiong, Jiawei Zhang, Yao Zhang, Tianqi Zhang, Yangyong Zhu

2009.10273

Tradeoffs between Convergence Speed and Reconstruction Accuracy in Inverse Problems

Raja Giryes, Yonina C. Eldar, Alex M. Bronstein, Guillermo Sapiro

1605.09232

Provable limitations of deep learning

Emmanuel Abbe, Colin Sandon

1812.06369

MLPerf Training Benchmark

Peter Mattson, Christine Cheng, Cody Coleman, Greg Diamos, Paulius Micikevicius, David Patterson, Hanlin Tang, Gu-Yeon Wei, Peter Bailis, Victor Bittorf, David Brooks, Dehao Chen, Debojyoti Dutta, Udit Gupta, Kim Hazelwood, Andrew Hock, Xinyuan Huang, Atsushi Ike, Bill Jia, Daniel Kang, David Kanter, Naveen Kumar, Jeffery Liao, Guokai Ma, Deepak Narayanan, Tayo Oguntebi, Gennady Pekhimenko, Lillian Pentecost, Vijay Janapa Reddi, Taylor Robie, Tom St. John, Tsuguchika Tabaru, Carole-Jean Wu, Lingjie Xu, Masafumi Yamazaki, Cliff Young, Matei Zaharia

1910.01500

Retiring Adult: New Datasets for Fair Machine Learning

Frances Ding, Moritz Hardt, John Miller, Ludwig Schmidt

2108.04884

How Fine-Tuning Allows for Effective Meta-Learning

Kurtland Chua, Qi Lei, Jason D. Lee

2105.02221

CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning

Jinhyun So, Basak Guler, A. Salman Avestimehr

1902.00641

On the Apparent Conflict Between Individual and Group Fairness

Reuben Binns

1912.06883

Analysis of k-Nearest Neighbor Distances with Application to Entropy Estimation

Shashank Singh, Barnabás Póczos

1603.08578

Per-Tensor Fixed-Point Quantization of the Back-Propagation Algorithm

Charbel Sakr, Naresh Shanbhag

1812.11732

Personalized Federated Learning: A Meta-Learning Approach

Alireza Fallah, Aryan Mokhtari, Asuman Ozdaglar

2002.07948

Point Cloud GAN

Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabas Poczos, Ruslan Salakhutdinov

1810.05795

Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System

Muhammad Ammad-ud-din, Elena Ivannikova, Suleiman A. Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, Adrian Flanagan

1901.09888

Relax, no need to round: integrality of clustering formulations

Pranjal Awasthi, Afonso S. Bandeira, Moses Charikar, Ravishankar Krishnaswamy, Soledad Villar, Rachel Ward

1408.4045

Certified Robustness to Label-Flipping Attacks via Randomized Smoothing

Elan Rosenfeld, Ezra Winston, Pradeep Ravikumar, J. Zico Kolter

2002.03018

Antisocial Behavior in Online Discussion Communities

Justin Cheng, Cristian Danescu-Niculescu-Mizil, Jure Leskovec

1504.00680

An Improved Parametrization and Analysis of the EXP3++ Algorithm for Stochastic and Adversarial Bandits

Yevgeny Seldin, Gábor Lugosi

1702.06103

Multiple Futures Prediction

Yichuan Charlie Tang, Ruslan Salakhutdinov

1911.00997

Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech

Quan Wang, Yang Yu, Jason Pelecanos, Yiling Huang, Ignacio Lopez Moreno

2202.12163

Generalized Inner Loop Meta-Learning

Edward Grefenstette, Brandon Amos, Denis Yarats, Phu Mon Htut, Artem Molchanov, Franziska Meier, Douwe Kiela, Kyunghyun Cho, Soumith Chintala

1910.01727

Scaling Multidimensional Inference for Structured Gaussian Processes

Elad Gilboa, Yunus Saatçi, John P. Cunningham

1209.4120

Momentum Residual Neural Networks

Michael E. Sander, Pierre Ablin, Mathieu Blondel, Gabriel Peyré

2102.07870

Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning

Georgios Papoudakis, Filippos Christianos, Arrasy Rahman, Stefano V. Albrecht

1906.04737

Perceptual Adversarial Robustness: Defense Against Unseen Threat Models

Cassidy Laidlaw, Sahil Singla, Soheil Feizi

2006.12655

A Primal-Dual Framework for Transformers and Neural Networks

Tan M. Nguyen, Tam Nguyen, Nhat Ho, Andrea L. Bertozzi, Richard G. Baraniuk, Stanley J. Osher

2406.13781

Information-Theoretic Considerations in Batch Reinforcement Learning

Jinglin Chen, Nan Jiang

1905.00360

Data Valuation using Reinforcement Learning

Jinsung Yoon, Sercan O. Arik, Tomas Pfister

1909.11671

Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey

Sicheng Zhao, Bo Li, Colorado Reed, Pengfei Xu, Kurt Keutzer

2002.12169

Jointly Clustering Rows and Columns of Binary Matrices: Algorithms and Trade-offs

Jiaming Xu, Rui Wu, Kai Zhu, Bruce Hajek, R. Srikant, Lei Ying

1310.0512

Fully Neural Network based Model for General Temporal Point Processes

Takahiro Omi, Naonori Ueda, Kazuyuki Aihara

1905.09690

Continual Lifelong Learning with Neural Networks: A Review

German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, Stefan Wermter

1802.07569

Targeted Adversarial Examples for Black Box Audio Systems

Rohan Taori, Amog Kamsetty, Brenton Chu, Nikita Vemuri

1805.07820

Compressed Sensing and Matrix Completion with Constant Proportion of Corruptions

Xiaodong Li

1104.1041

A Factor Graph Approach to Joint OFDM Channel Estimation and Decoding in Impulsive Noise Environments

Marcel Nassar, Philip Schniter, Brian L. Evans

1306.1851

Gradient Starvation: A Learning Proclivity in Neural Networks

Mohammad Pezeshki, Sékou-Oumar Kaba, Yoshua Bengio, Aaron Courville, Doina Precup, Guillaume Lajoie

2011.09468

Deep Learning on Graphs: A Survey

Ziwei Zhang, Peng Cui, Wenwu Zhu

1812.04202

Unifying machine learning and quantum chemistry -- a deep neural network for molecular wavefunctions

K. T. Schütt, M. Gastegger, A. Tkatchenko, K. -R. Müller, R. J. Maurer

1906.10033

Reasoning About Physical Interactions with Object-Oriented Prediction and Planning

Michael Janner, Sergey Levine, William T. Freeman, Joshua B. Tenenbaum, Chelsea Finn, Jiajun Wu

1812.10972

Do Input Gradients Highlight Discriminative Features?

Harshay Shah, Prateek Jain, Praneeth Netrapalli

2102.12781

Tile2Vec: Unsupervised representation learning for spatially distributed data

Neal Jean, Sherrie Wang, Anshul Samar, George Azzari, David Lobell, Stefano Ermon

1805.02855

Properties of the geometry of solutions and capacity of multi-layer neural networks with Rectified Linear Units activations

Carlo Baldassi, Enrico M. Malatesta, Riccardo Zecchina

1907.07578

The Structure of Optimal Private Tests for Simple Hypotheses

Clément L. Canonne, Gautam Kamath, Audra McMillan, Adam Smith, Jonathan Ullman

1811.11148

Is There an Analog of Nesterov Acceleration for MCMC?

Yi-An Ma, Niladri Chatterji, Xiang Cheng, Nicolas Flammarion, Peter Bartlett, Michael I. Jordan

1902.00996

Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?

Nenad Tomasev, Ioana Bica, Brian McWilliams, Lars Buesing, Razvan Pascanu, Charles Blundell, Jovana Mitrovic

2201.05119

Minimizing Trajectory Curvature of ODE-based Generative Models

Sangyun Lee, Beomsu Kim, Jong Chul Ye

2301.12003

Delayed Sampling and Automatic Rao-Blackwellization of Probabilistic Programs

Lawrence M. Murray, Daniel Lundén, Jan Kudlicka, David Broman, Thomas B. Schön

1708.07787

An Introduction to Deep Reinforcement Learning

Vincent Francois-Lavet, Peter Henderson, Riashat Islam, Marc G. Bellemare, Joelle Pineau

1811.12560

Deep & Cross Network for Ad Click Predictions

Ruoxi Wang, Bin Fu, Gang Fu, Mingliang Wang

1708.05123

Inverting Deep Generative models, One layer at a time

Qi Lei, Ajil Jalal, Inderjit S. Dhillon, Alexandros G. Dimakis

1906.07437

Diagnosing Bottlenecks in Deep Q-learning Algorithms

Justin Fu, Aviral Kumar, Matthew Soh, Sergey Levine

1902.10250

Statistical Efficiency of Thompson Sampling for Combinatorial Semi-Bandits

Pierre Perrault, Etienne Boursier, Vianney Perchet, Michal Valko

2006.06613

Towards Deep Learning Models Resistant to Adversarial Attacks

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu

1706.06083

Gotta Learn Fast: A New Benchmark for Generalization in RL

Alex Nichol, Vicki Pfau, Christopher Hesse, Oleg Klimov, John Schulman

1804.03720

RedQueen: An Online Algorithm for Smart Broadcasting in Social Networks

Ali Zarezade, Utkarsh Upadhyay, Hamid Rabiee, Manuel Gomez Rodriguez

1610.05773

Multi-turn Dialogue Response Generation in an Adversarial Learning Framework

Oluwatobi Olabiyi, Alan Salimov, Anish Khazane, Erik T. Mueller

1805.11752

Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC

Roger Frigola, Fredrik Lindsten, Thomas B. Schön, Carl E. Rasmussen

1306.2861

On the Convergence Proof of AMSGrad and a New Version

Tran Thi Phuong, Le Trieu Phong

1904.03590

Bayesian stochastic blockmodeling

Tiago P. Peixoto

1705.10225

To Trust Or Not To Trust A Classifier

Heinrich Jiang, Been Kim, Melody Y. Guan, Maya Gupta

1805.11783

Multi-Armed Bandits with Local Differential Privacy

Wenbo Ren, Xingyu Zhou, Jia Liu, Ness B. Shroff

2007.03121

Practical Lossless Compression with Latent Variables using Bits Back Coding

James Townsend, Tom Bird, David Barber

1901.04866

Bilinear Classes: A Structural Framework for Provable Generalization in RL

Simon S. Du, Sham M. Kakade, Jason D. Lee, Shachar Lovett, Gaurav Mahajan, Wen Sun, Ruosong Wang

2103.10897

Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference

Disi Ji, Padhraic Smyth, Mark Steyvers

2010.09851

Concept Bottleneck Models

Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, Percy Liang

2007.04612

Training Data Attribution for Diffusion Models

Zheng Dai, David K Gifford

2306.02174

Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation

Felix Kreuk, Joseph Keshet, Yossi Adi

2007.13465

The NetHack Learning Environment

Heinrich Küttler, Nantas Nardelli, Alexander H. Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette, Tim Rocktäschel

2006.13760

Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets

Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley Osher, Yingyong Qi, Jack Xin

1903.05662

Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks

Urs Köster, Tristan J. Webb, Xin Wang, Marcel Nassar, Arjun K. Bansal, William H. Constable, Oğuz H. Elibol, Scott Gray, Stewart Hall, Luke Hornof, Amir Khosrowshahi, Carey Kloss, Ruby J. Pai, Naveen Rao

1711.02213

From Poincaré Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularization

Julien Perolat, Remi Munos, Jean-Baptiste Lespiau, Shayegan Omidshafiei, Mark Rowland, Pedro Ortega, Neil Burch, Thomas Anthony, David Balduzzi, Bart De Vylder, Georgios Piliouras, Marc Lanctot, Karl Tuyls

2002.08456

Benign Overfitting in Two-layer Convolutional Neural Networks

Yuan Cao, Zixiang Chen, Mikhail Belkin, Quanquan Gu

2202.06526

Submodular Combinatorial Information Measures with Applications in Machine Learning

Rishabh Iyer, Ninad Khargonkar, Jeff Bilmes, Himanshu Asnani

2006.15412

Deep Reinforcement Learning from Policy-Dependent Human Feedback

Dilip Arumugam, Jun Ki Lee, Sophie Saskin, Michael L. Littman

1902.04257

Auto-Meta: Automated Gradient Based Meta Learner Search

Jaehong Kim, Sangyeul Lee, Sungwan Kim, Moonsu Cha, Jung Kwon Lee, Youngduck Choi, Yongseok Choi, Dong-Yeon Cho, Jiwon Kim

1806.06927

Proximal Methods for Hierarchical Sparse Coding

Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski, Francis Bach

1009.2139

Model Reconstruction from Model Explanations

Smitha Milli, Ludwig Schmidt, Anca D. Dragan, Moritz Hardt

1807.05185

Accounting for Variance in Machine Learning Benchmarks

Xavier Bouthillier, Pierre Delaunay, Mirko Bronzi, Assya Trofimov, Brennan Nichyporuk, Justin Szeto, Naz Sepah, Edward Raff, Kanika Madan, Vikram Voleti, Samira Ebrahimi Kahou, Vincent Michalski, Dmitriy Serdyuk, Tal Arbel, Chris Pal, Gaël Varoquaux, Pascal Vincent

2103.03098

Skew-Fit: State-Covering Self-Supervised Reinforcement Learning

Vitchyr H. Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, Sergey Levine

1903.03698

Risk and parameter convergence of logistic regression

Ziwei Ji, Matus Telgarsky

1803.07300

Topics to Avoid: Demoting Latent Confounds in Text Classification

Sachin Kumar, Shuly Wintner, Noah A. Smith, Yulia Tsvetkov

1909.00453

Multi-consensus Decentralized Accelerated Gradient Descent

Haishan Ye, Luo Luo, Ziang Zhou, Tong Zhang

2005.00797

Mixed membership stochastic blockmodels

Edoardo M Airoldi, David M Blei, Stephen E Fienberg, Eric P Xing

0705.4485

Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks

Priyanka Gupta, Pankaj Malhotra, Lovekesh Vig, Gautam Shroff

1807.01705

Reinforcement Learning with General Value Function Approximation: Provably Efficient Approach via Bounded Eluder Dimension

Ruosong Wang, Ruslan Salakhutdinov, Lin F. Yang

2005.10804

Truncated Back-propagation for Bilevel Optimization

Amirreza Shaban, Ching-An Cheng, Nathan Hatch, Byron Boots

1810.10667

Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data

Sergei Popov, Stanislav Morozov, Artem Babenko

1909.06312

Planning in Markov Decision Processes with Gap-Dependent Sample Complexity

Anders Jonsson, Emilie Kaufmann, Pierre Ménard, Omar Darwiche Domingues, Edouard Leurent, Michal Valko

2006.05879

OpenML Benchmarking Suites

Bernd Bischl, Giuseppe Casalicchio, Matthias Feurer, Pieter Gijsbers, Frank Hutter, Michel Lang, Rafael G. Mantovani, Jan N. van Rijn, Joaquin Vanschoren

1708.03731

Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithms

Mathieu Blondel, Masakazu Ishihata, Akinori Fujino, Naonori Ueda

1607.08810

Circuit Complexity Bounds for Visual Autoregressive Model

Yekun Ke, Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song

2501.04299

Doubly robust off-policy evaluation with shrinkage

Yi Su, Maria Dimakopoulou, Akshay Krishnamurthy, Miroslav Dudík

1907.09623

Regularizing Deep Multi-Task Networks using Orthogonal Gradients

Mihai Suteu, Yike Guo

1912.06844

Rethinking Bias-Variance Trade-off for Generalization of Neural Networks

Zitong Yang, Yaodong Yu, Chong You, Jacob Steinhardt, Yi Ma

2002.11328

Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild

Kibok Lee, Kimin Lee, Jinwoo Shin, Honglak Lee

1903.12648

Task-Free Continual Learning

Rahaf Aljundi, Klaas Kelchtermans, Tinne Tuytelaars

1812.03596

Efficient Estimation of Mutual Information for Strongly Dependent Variables

Shuyang Gao, Greg Ver Steeg, Aram Galstyan

1411.2003

Inductive Graph Neural Networks for Spatiotemporal Kriging

Yuankai Wu, Dingyi Zhuang, Aurelie Labbe, Lijun Sun

2006.07527

bartMachine: Machine Learning with Bayesian Additive Regression Trees

Adam Kapelner, Justin Bleich

1312.2171

Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment

Yifan Wu, Ezra Winston, Divyansh Kaushik, Zachary Lipton

1903.01689

Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck

Maximilian Igl, Kamil Ciosek, Yingzhen Li, Sebastian Tschiatschek, Cheng Zhang, Sam Devlin, Katja Hofmann

1910.12911

Towards Robust, Locally Linear Deep Networks

Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola

1907.03207

Heavy-tailed Representations, Text Polarity Classification & Data Augmentation

Hamid Jalalzai, Pierre Colombo, Chloé Clavel, Eric Gaussier, Giovanna Varni, Emmanuel Vignon, Anne Sabourin

2003.11593

Quantization Algorithms for Random Fourier Features

Xiaoyun Li, Ping Li

2102.13079

First-order Methods for Geodesically Convex Optimization

Hongyi Zhang, Suvrit Sra

1602.06053

Cross-Domain Imitation Learning via Optimal Transport

Arnaud Fickinger, Samuel Cohen, Stuart Russell, Brandon Amos

2110.03684

Conditional Learning of Fair Representations

Han Zhao, Amanda Coston, Tameem Adel, Geoffrey J. Gordon

1910.07162

Building powerful and equivariant graph neural networks with structural message-passing

Clement Vignac, Andreas Loukas, Pascal Frossard

2006.15107

FedSplit: An algorithmic framework for fast federated optimization

Reese Pathak, Martin J. Wainwright

2005.05238

Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research

Bernard Koch, Emily Denton, Alex Hanna, Jacob G. Foster

2112.01716

Corpora Generation for Grammatical Error Correction

Jared Lichtarge, Chris Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, Simon Tong

1904.05780

Sandwiching the marginal likelihood using bidirectional Monte Carlo

Roger B. Grosse, Zoubin Ghahramani, Ryan P. Adams

1511.02543

Pruning Convolutional Neural Networks for Resource Efficient Inference

Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, Jan Kautz

1611.06440

Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs

Dasol Hwang, Jinyoung Park, Sunyoung Kwon, Kyung-Min Kim, Jung-Woo Ha, Hyunwoo J. Kim

2007.08294

Training Deep Neural Networks with 8-bit Floating Point Numbers

Naigang Wang, Jungwook Choi, Daniel Brand, Chia-Yu Chen, Kailash Gopalakrishnan

1812.08011

Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization

H. T. Kung, Bradley McDanel, Sai Qian Zhang

1811.04770

Benchmarking Batch Deep Reinforcement Learning Algorithms

Scott Fujimoto, Edoardo Conti, Mohammad Ghavamzadeh, Joelle Pineau

1910.01708

Topological Recurrent Neural Network for Diffusion Prediction

Jia Wang, Vincent W. Zheng, Zemin Liu, Kevin Chen-Chuan Chang

1711.10162

Robust Hypothesis Testing Using Wasserstein Uncertainty Sets

Rui Gao, Liyan Xie, Yao Xie, Huan Xu

1805.10611

Fast and Deep Graph Neural Networks

Claudio Gallicchio, Alessio Micheli

1911.08941

Relational Forward Models for Multi-Agent Learning

Andrea Tacchetti, H. Francis Song, Pedro A. M. Mediano, Vinicius Zambaldi, Neil C. Rabinowitz, Thore Graepel, Matthew Botvinick, Peter W. Battaglia

1809.11044

Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF

Tengyang Xie, Dylan J. Foster, Akshay Krishnamurthy, Corby Rosset, Ahmed Awadallah, Alexander Rakhlin

2405.21046

Neural Manifold Ordinary Differential Equations

Aaron Lou, Derek Lim, Isay Katsman, Leo Huang, Qingxuan Jiang, Ser-Nam Lim, Christopher De Sa

2006.10254

Model-Based Reinforcement Learning with Value-Targeted Regression

Alex Ayoub, Zeyu Jia, Csaba Szepesvari, Mengdi Wang, Lin F. Yang

2006.01107

High Dimensional Semiparametric Gaussian Copula Graphical Models

Han Liu, Fang Han, Ming Yuan, John Lafferty, Larry Wasserman

1202.2169

Deep kernel processes

Laurence Aitchison, Adam X. Yang, Sebastian W. Ober

2010.01590

DPO Meets PPO: Reinforced Token Optimization for RLHF

Han Zhong, Zikang Shan, Guhao Feng, Wei Xiong, Xinle Cheng, Li Zhao, Di He, Jiang Bian, Liwei Wang

2404.18922

Introduction to Multi-Armed Bandits

Aleksandrs Slivkins

1904.07272

Identifying Generalization Properties in Neural Networks

Huan Wang, Nitish Shirish Keskar, Caiming Xiong, Richard Socher

1809.07402

Optimizing Millions of Hyperparameters by Implicit Differentiation

Jonathan Lorraine, Paul Vicol, David Duvenaud

1911.02590

A Unified Theory of Decentralized SGD with Changing Topology and Local Updates

Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, Sebastian U. Stich

2003.10422

Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image

Ronghang Hu, Nikhila Ravi, Alexander C. Berg, Deepak Pathak

2012.09854

Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lenses

Niharika Jain, Alberto Olmo, Sailik Sengupta, Lydia Manikonda, Subbarao Kambhampati

2001.09528

Mitigating Bias in Adaptive Data Gathering via Differential Privacy

Seth Neel, Aaron Roth

1806.02329

Accelerating Deep Learning by Focusing on the Biggest Losers

Angela H. Jiang, Daniel L. -K. Wong, Giulio Zhou, David G. Andersen, Jeffrey Dean, Gregory R. Ganger, Gauri Joshi, Michael Kaminksy, Michael Kozuch, Zachary C. Lipton, Padmanabhan Pillai

1910.00762

Data-driven Advice for Applying Machine Learning to Bioinformatics Problems

Randal S. Olson, William La Cava, Zairah Mustahsan, Akshay Varik, Jason H. Moore

1708.05070

Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimization

Guodong Zhang, Yuanhao Wang, Laurent Lessard, Roger Grosse

2102.09468

On Feature Learning in Neural Networks with Global Convergence Guarantees

Zhengdao Chen, Eric Vanden-Eijnden, Joan Bruna

2204.10782

Zeroth-order Nonconvex Stochastic Optimization: Handling Constraints, High-Dimensionality and Saddle-Points

Krishnakumar Balasubramanian, Saeed Ghadimi

1809.06474

Learning and Inference on Generative Adversarial Quantum Circuits

Jinfeng Zeng, Yufeng Wu, Jin-Guo Liu, Lei Wang, Jiangping Hu

1808.03425

Group Lasso with Overlaps: the Latent Group Lasso approach

Guillaume Obozinski, Laurent Jacob, Jean-Philippe Vert

1110.0413

Single Trajectory Nonparametric Learning of Nonlinear Dynamics

Ingvar Ziemann, Henrik Sandberg, Nikolai Matni

2202.08311

Fooling Neural Network Interpretations via Adversarial Model Manipulation

Juyeon Heo, Sunghwan Joo, Taesup Moon

1902.02041

Towards Learning a Universal Non-Semantic Representation of Speech

Joel Shor, Aren Jansen, Ronnie Maor, Oran Lang, Omry Tuval, Felix de Chaumont Quitry, Marco Tagliasacchi, Ira Shavitt, Dotan Emanuel, Yinnon Haviv

2002.12764

A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks

Angelos Filos, Sebastian Farquhar, Aidan N. Gomez, Tim G. J. Rudner, Zachary Kenton, Lewis Smith, Milad Alizadeh, Arnoud de Kroon, Yarin Gal

1912.10481

Compatible Natural Gradient Policy Search

Joni Pajarinen, Hong Linh Thai, Riad Akrour, Jan Peters, Gerhard Neumann

1902.02823

Learning Determinantal Point Processes

Alex Kulesza, Ben Taskar

1202.3738

Faster high-accuracy log-concave sampling via algorithmic warm starts

Jason M. Altschuler, Sinho Chewi

2302.10249

Supervised learning with quantum enhanced feature spaces

Vojtech Havlicek, Antonio D. Córcoles, Kristan Temme, Aram W. Harrow, Abhinav Kandala, Jerry M. Chow, Jay M. Gambetta

1804.11326

Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View

Christos Thrampoulidis, Samet Oymak, Mahdi Soltanolkotabi

2011.07729

VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation

Jiyang Gao, Chen Sun, Hang Zhao, Yi Shen, Dragomir Anguelov, Congcong Li, Cordelia Schmid

2005.04259

The Impact of Regularization on High-dimensional Logistic Regression

Fariborz Salehi, Ehsan Abbasi, Babak Hassibi

1906.03761

LatticeNet: Fast Point Cloud Segmentation Using Permutohedral Lattices

Radu Alexandru Rosu, Peer Schütt, Jan Quenzel, Sven Behnke

1912.05905

Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization

Alekh Agarwal, Peter L. Bartlett, Pradeep Ravikumar, Martin J. Wainwright

1009.0571

Deep Imitative Models for Flexible Inference, Planning, and Control

Nicholas Rhinehart, Rowan McAllister, Sergey Levine

1810.06544

Sparse Representation of a Polytope and Recovery of Sparse Signals and Low-rank Matrices

T. Tony Cai, Anru Zhang

1306.1154

Explaining the Unique Nature of Individual Gait Patterns with Deep Learning

Fabian Horst, Sebastian Lapuschkin, Wojciech Samek, Klaus-Robert Müller, Wolfgang I. Schöllhorn

1808.04308

Variational Auto-Regressive Gaussian Processes for Continual Learning

Sanyam Kapoor, Theofanis Karaletsos, Thang D. Bui

2006.05468

Neural Algorithmic Reasoning

Petar Veličković, Charles Blundell

2105.02761

Learning to Draw Samples with Amortized Stein Variational Gradient Descent

Yihao Feng, Dilin Wang, Qiang Liu

1707.06626

Uncertainty in Neural Networks: Approximately Bayesian Ensembling

Tim Pearce, Felix Leibfried, Alexandra Brintrup, Mohamed Zaki, Andy Neely

1810.05546

Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning

Yacine Jernite, Samuel R. Bowman, David Sontag

1705.00557

Deep Partition Aggregation: Provable Defense against General Poisoning Attacks

Alexander Levine, Soheil Feizi

2006.14768

Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications

Jongsoo Park, Maxim Naumov, Protonu Basu, Summer Deng, Aravind Kalaiah, Daya Khudia, James Law, Parth Malani, Andrey Malevich, Satish Nadathur, Juan Pino, Martin Schatz, Alexander Sidorov, Viswanath Sivakumar, Andrew Tulloch, Xiaodong Wang, Yiming Wu, Hector Yuen, Utku Diril, Dmytro Dzhulgakov, Kim Hazelwood, Bill Jia, Yangqing Jia, Lin Qiao, Vijay Rao, Nadav Rotem, Sungjoo Yoo, Mikhail Smelyanskiy

1811.09886

Solving Min-Max Optimization with Hidden Structure via Gradient Descent Ascent

Lampros Flokas, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Georgios Piliouras

2101.05248

On the Implicit Bias in Deep-Learning Algorithms

Gal Vardi

2208.12591

Bolasso: model consistent Lasso estimation through the bootstrap

Francis Bach

0804.1302

Bag of Tricks for Adversarial Training

Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, Jun Zhu

2010.00467

Clamping Improves TRW and Mean Field Approximations

Adrian Weller, Justin Domke

1510.00087

On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

Sitao Xiang, Hao Li

1704.03971

Reverse-Engineering Deep ReLU Networks

David Rolnick, Konrad P. Kording

1910.00744

Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables

Yichi Zhang, Daniel Apley, Wei Chen

1910.01688

Convolutional Tensor-Train LSTM for Spatio-temporal Learning

Jiahao Su, Wonmin Byeon, Jean Kossaifi, Furong Huang, Jan Kautz, Animashree Anandkumar

2002.09131

Probabilistic Forecasting with Stochastic Interpolants and Föllmer Processes

Yifan Chen, Mark Goldstein, Mengjian Hua, Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden

2403.13724

Subpopulation Data Poisoning Attacks

Matthew Jagielski, Giorgio Severi, Niklas Pousette Harger, Alina Oprea

2006.14026

A Theory of Regularized Markov Decision Processes

Matthieu Geist, Bruno Scherrer, Olivier Pietquin

1901.11275

Membership Leakage in Label-Only Exposures

Zheng Li, Yang Zhang

2007.15528

Learning from Noisy Labels with Deep Neural Networks: A Survey

Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, Jae-Gil Lee

2007.08199

Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning

Natalie Dullerud, Karsten Roth, Kimia Hamidieh, Nicolas Papernot, Marzyeh Ghassemi

2203.12748

Conformal Prediction with Large Language Models for Multi-Choice Question Answering

Bhawesh Kumar, Charlie Lu, Gauri Gupta, Anil Palepu, David Bellamy, Ramesh Raskar, Andrew Beam

2305.18404

Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising

Borja Balle, Yu-Xiang Wang

1805.06530

Content preserving text generation with attribute controls

Lajanugen Logeswaran, Honglak Lee, Samy Bengio

1811.01135

GenDICE: Generalized Offline Estimation of Stationary Values

Ruiyi Zhang, Bo Dai, Lihong Li, Dale Schuurmans

2002.09072

MetaPoison: Practical General-purpose Clean-label Data Poisoning

W. Ronny Huang, Jonas Geiping, Liam Fowl, Gavin Taylor, Tom Goldstein

2004.00225

Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks

Alexander Shevchenko, Vyacheslav Kungurtsev, Marco Mondelli

2111.02278

Algorithms and Hardness for Linear Algebra on Geometric Graphs

Josh Alman, Timothy Chu, Aaron Schild, Zhao Song

2011.02466

Asymptotic Analysis of Sampling Estimators for Randomized Numerical Linear Algebra Algorithms

Ping Ma, Xinlian Zhang, Xin Xing, Jingyi Ma, Michael W. Mahoney

2002.10526

Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization

Gang Wang, Georgios B. Giannakis, Jie Chen

1808.04685

Spectral Method and Regularized MLE Are Both Optimal for Top-$K$ Ranking

Yuxin Chen, Jianqing Fan, Cong Ma, Kaizheng Wang

1707.09971

Transferable Clean-Label Poisoning Attacks on Deep Neural Nets

Chen Zhu, W. Ronny Huang, Ali Shafahi, Hengduo Li, Gavin Taylor, Christoph Studer, Tom Goldstein

1905.05897

Adaptively Exploiting d-Separators with Causal Bandits

Blair Bilodeau, Linbo Wang, Daniel M. Roy

2202.05100

KERMIT: Generative Insertion-Based Modeling for Sequences

William Chan, Nikita Kitaev, Kelvin Guu, Mitchell Stern, Jakob Uszkoreit

1906.01604

A Short Tutorial on The Weisfeiler-Lehman Test And Its Variants

Ningyuan Huang, Soledad Villar

2201.07083

Overcoming catastrophic forgetting with hard attention to the task

Joan Serrà, Dídac Surís, Marius Miron, Alexandros Karatzoglou

1801.01423

Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models

Michael U. Gutmann, Jukka Corander

1501.03291

The Disparate Effects of Strategic Manipulation

Lily Hu, Nicole Immorlica, Jennifer Wortman Vaughan

1808.08646

Think Locally, Act Globally: Federated Learning with Local and Global Representations

Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency

2001.01523

Data-Efficient Reinforcement Learning with Self-Predictive Representations

Max Schwarzer, Ankesh Anand, Rishab Goel, R Devon Hjelm, Aaron Courville, Philip Bachman

2007.05929

A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems

Anthony Corso, Robert J. Moss, Mark Koren, Ritchie Lee, Mykel J. Kochenderfer

2005.02979

On the Power of Differentiable Learning versus PAC and SQ Learning

Emmanuel Abbe, Pritish Kamath, Eran Malach, Colin Sandon, Nathan Srebro

2108.04190

A case for new neural network smoothness constraints

Mihaela Rosca, Theophane Weber, Arthur Gretton, Shakir Mohamed

2012.07969

Independent mechanism analysis, a new concept?

Luigi Gresele, Julius von Kügelgen, Vincent Stimper, Bernhard Schölkopf, Michel Besserve

2106.05200

A Max-Norm Constrained Minimization Approach to 1-Bit Matrix Completion

T. Tony Cai, Wen-Xin Zhou

1309.6013

Posterior Concentration for Sparse Deep Learning

Nicholas Polson, Veronika Rockova

1803.09138

Early stopping and non-parametric regression: An optimal data-dependent stopping rule

Garvesh Raskutti, Martin J. Wainwright, Bin Yu

1306.3574

Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse

James Lucas, George Tucker, Roger Grosse, Mohammad Norouzi

1911.02469

Generative networks as inverse problems with Scattering transforms

Tomás Angles, Stéphane Mallat

1805.06621

Multi-View Matrix Completion for Multi-Label Image Classification

Yong Luo, Tongliang Liu, Dacheng Tao, Chao Xu

1904.03901

Query-Efficient Black-box Adversarial Attacks Guided by a Transfer-based Prior

Yinpeng Dong, Shuyu Cheng, Tianyu Pang, Hang Su, Jun Zhu

2203.06560

Composition-based Multi-Relational Graph Convolutional Networks

Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, Partha Talukdar

1911.03082

Compound Probabilistic Context-Free Grammars for Grammar Induction

Yoon Kim, Chris Dyer, Alexander M. Rush

1906.10225

Nonparametric Bayes dynamic modeling of relational data

Daniele Durante, David B. Dunson

1311.4669

Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations

Abbas Sadat, Sergio Casas, Mengye Ren, Xinyu Wu, Pranaab Dhawan, Raquel Urtasun

2008.05930

Deep Learning of Vortex Induced Vibrations

Maziar Raissi, Zhicheng Wang, Michael S. Triantafyllou, George Em Karniadakis

1808.08952

Counterfactual States for Atari Agents via Generative Deep Learning

Matthew L. Olson, Lawrence Neal, Fuxin Li, Weng-Keen Wong

1909.12969

Deep Leakage from Gradients

Ligeng Zhu, Zhijian Liu, Song Han

1906.08935

PolyGen: An Autoregressive Generative Model of 3D Meshes

Charlie Nash, Yaroslav Ganin, S. M. Ali Eslami, Peter W. Battaglia

2002.10880

L$^2$-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks

Yuning You, Tianlong Chen, Zhangyang Wang, Yang Shen

2003.13606

On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift

Alekh Agarwal, Sham M. Kakade, Jason D. Lee, Gaurav Mahajan

1908.00261

PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss

Umut Isik, Ritwik Giri, Neerad Phansalkar, Jean-Marc Valin, Karim Helwani, Arvindh Krishnaswamy

2008.04470

Uniform Convergence of Gradients for Non-Convex Learning and Optimization

Dylan J. Foster, Ayush Sekhari, Karthik Sridharan

1810.11059

Approximation and Convergence Properties of Generative Adversarial Learning

Shuang Liu, Olivier Bousquet, Kamalika Chaudhuri

1705.08991

Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks

Mingchen Li, Mahdi Soltanolkotabi, Samet Oymak

1903.11680

The Marginal Value of Adaptive Gradient Methods in Machine Learning

Ashia C. Wilson, Rebecca Roelofs, Mitchell Stern, Nathan Srebro, Benjamin Recht

1705.08292

Flow Matching on General Geometries

Ricky T. Q. Chen, Yaron Lipman

2302.03660

Nonparametric ridge estimation

Christopher R. Genovese, Marco Perone-Pacifico, Isabella Verdinelli, Larry Wasserman

1212.5156

Variance-reduced $Q$-learning is minimax optimal

Martin J. Wainwright

1906.04697

Deep learning from crowds

Filipe Rodrigues, Francisco Pereira

1709.01779

A General Framework for Constrained Bayesian Optimization using Information-based Search

José Miguel Hernández-Lobato, Michael A. Gelbart, Ryan P. Adams, Matthew W. Hoffman, Zoubin Ghahramani

1511.09422

FedDANE: A Federated Newton-Type Method

Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, Virginia Smith

2001.01920

Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent

Yunwen Lei, Yiming Ying

2006.08157

Exact post-selection inference, with application to the lasso

Jason D. Lee, Dennis L. Sun, Yuekai Sun, Jonathan E. Taylor

1311.6238

Multi-scale Attributed Node Embedding

Benedek Rozemberczki, Carl Allen, Rik Sarkar

1909.13021

Efficient Learning of Generative Models via Finite-Difference Score Matching

Tianyu Pang, Kun Xu, Chongxuan Li, Yang Song, Stefano Ermon, Jun Zhu

2007.03317

Continuous Graph Neural Networks

Louis-Pascal A. C. Xhonneux, Meng Qu, Jian Tang

1912.00967

Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective

Dylan J. Foster, Alexander Rakhlin, David Simchi-Levi, Yunzong Xu

2010.03104

Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks

Stefan Depeweg, José Miguel Hernández-Lobato, Finale Doshi-Velez, Steffen Udluft

1605.07127

Robust Aggregation for Federated Learning

Krishna Pillutla, Sham M. Kakade, Zaid Harchaoui

1912.13445

Low-bit Quantization of Neural Networks for Efficient Inference

Yoni Choukroun, Eli Kravchik, Fan Yang, Pavel Kisilev

1902.06822

Byzantine-Resilient Secure Federated Learning

Jinhyun So, Basak Guler, A. Salman Avestimehr

2007.11115

Linearly convergent stochastic heavy ball method for minimizing generalization error

Nicolas Loizou, Peter Richtárik

1710.10737

Learning Representations of Missing Data for Predicting Patient Outcomes

Brandon Malone, Alberto Garcia-Duran, Mathias Niepert

1811.04752

QEBA: Query-Efficient Boundary-Based Blackbox Attack

Huichen Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, Bo Li

2005.14137

Semi-Supervised Haptic Material Recognition for Robots using Generative Adversarial Networks

Zackory Erickson, Sonia Chernova, Charles C. Kemp

1707.02796

Implicit Weight Uncertainty in Neural Networks

Nick Pawlowski, Andrew Brock, Matthew C. H. Lee, Martin Rajchl, Ben Glocker

1711.01297

DAGs with NO TEARS: Continuous Optimization for Structure Learning

Xun Zheng, Bryon Aragam, Pradeep Ravikumar, Eric P. Xing

1803.01422

Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies

Yae Jee Cho, Jianyu Wang, Gauri Joshi

2010.01243

Inherent Trade-Offs in the Fair Determination of Risk Scores

Jon Kleinberg, Sendhil Mullainathan, Manish Raghavan

1609.05807

The Mythos of Model Interpretability

Zachary C. Lipton

1606.03490

GASC: Genre-Aware Semantic Change for Ancient Greek

Valerio Perrone, Marco Palma, Simon Hengchen, Alessandro Vatri, Jim Q. Smith, Barbara McGillivray

1903.05587

Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding

Deunsol Yoon, Dongbok Lee, SangKeun Lee

1808.07383

Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control

Kendall Lowrey, Aravind Rajeswaran, Sham Kakade, Emanuel Todorov, Igor Mordatch

1811.01848

Gradient descent aligns the layers of deep linear networks

Ziwei Ji, Matus Telgarsky

1810.02032

Rethinking the Value of Network Pruning

Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, Trevor Darrell

1810.05270

SING: Symbol-to-Instrument Neural Generator

Alexandre Défossez, Neil Zeghidour, Nicolas Usunier, Léon Bottou, Francis Bach

1810.09785

Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset

Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, Douglas Eck

1810.12247

Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations

Debraj Basu, Deepesh Data, Can Karakus, Suhas Diggavi

1906.02367

ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learning

Zhixiong Yang, Waheed U. Bajwa

1708.08155

Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples

Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, Pushmeet Kohli

2010.03593

Matrix Completion on Graphs

Vassilis Kalofolias, Xavier Bresson, Michael Bronstein, Pierre Vandergheynst

1408.1717

Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks

Yuan Cao, Quanquan Gu

1902.01384

Knowledge Graph Convolutional Networks for Recommender Systems

Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, Minyi Guo

1904.12575

Correlated-PCA: Principal Components' Analysis when Data and Noise are Correlated

Namrata Vaswani, Han Guo

1610.09307

Early stopping for kernel boosting algorithms: A general analysis with localized complexities

Yuting Wei, Fanny Yang, Martin J. Wainwright

1707.01543

Equivariant Neural Rendering

Emilien Dupont, Miguel Angel Bautista, Alex Colburn, Aditya Sankar, Carlos Guestrin, Josh Susskind, Qi Shan

2006.07630

Goal-conditioned Imitation Learning

Yiming Ding, Carlos Florensa, Mariano Phielipp, Pieter Abbeel

1906.05838

Exact sampling of determinantal point processes with sublinear time preprocessing

Michał Dereziński, Daniele Calandriello, Michal Valko

1905.13476

ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks

Jungmin Kwon, Jeongseop Kim, Hyunseo Park, In Kwon Choi

2102.11600

Go-Explore: a New Approach for Hard-Exploration Problems

Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O. Stanley, Jeff Clune

1901.10995

Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design

Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, Tommi Jaakkola

2402.04997

Gradient Descent-Ascent Provably Converges to Strict Local Minmax Equilibria with a Finite Timescale Separation

Tanner Fiez, Lillian Ratliff

2009.14820

Reward-Free Exploration for Reinforcement Learning

Chi Jin, Akshay Krishnamurthy, Max Simchowitz, Tiancheng Yu

2002.02794

Non-Asymptotic Pure Exploration by Solving Games

Rémy Degenne, Wouter M. Koolen, Pierre Ménard

1906.10431

Asymptotics of Ridge (less) Regression under General Source Condition

Dominic Richards, Jaouad Mourtada, Lorenzo Rosasco

2006.06386

Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis

Qi Chen, Changjian Shui, Mario Marchand

2109.14595

Visualizing Data using GTSNE

Songting Shi

2108.01301

Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search

Chaoyang He, Murali Annavaram, Salman Avestimehr

2004.08546

ORFit: One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares

Youngjae Min, Namhoon Cho, Navid Azizan

2207.13853

Rates of Convergence for Sparse Variational Gaussian Process Regression

David R. Burt, Carl E. Rasmussen, Mark van der Wilk

1903.03571

A Practical Sparse Approximation for Real Time Recurrent Learning

Jacob Menick, Erich Elsen, Utku Evci, Simon Osindero, Karen Simonyan, Alex Graves

2006.07232

Potential Flow Generator with $L_2$ Optimal Transport Regularity for Generative Models

Liu Yang, George Em Karniadakis

1908.11462

Data-Efficient Hierarchical Reinforcement Learning

Ofir Nachum, Shixiang Gu, Honglak Lee, Sergey Levine

1805.08296

Efficient Lifelong Learning with A-GEM

Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, Mohamed Elhoseiny

1812.00420

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges

Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, Chudi Zhong

2103.11251

Universal adversarial perturbations

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard

1610.08401

Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models

Mohammad Emtiyaz Khan, Didrik Nielsen

1807.04489

AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks

Chun-Chen Tu, Paishun Ting, Pin-Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, Shin-Ming Cheng

1805.11770

Handling Missing Data with Graph Representation Learning

Jiaxuan You, Xiaobai Ma, Daisy Yi Ding, Mykel Kochenderfer, Jure Leskovec

2010.16418

Multipole Graph Neural Operator for Parametric Partial Differential Equations

Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar

2006.09535

Policy Learning for Fairness in Ranking

Ashudeep Singh, Thorsten Joachims

1902.04056

SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics

Jonathan Hayase, Weihao Kong, Raghav Somani, Sewoong Oh

2104.11315

SoK: Certified Robustness for Deep Neural Networks

Linyi Li, Tao Xie, Bo Li

2009.04131

PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review

Ivan Stelmakh, Nihar B. Shah, Aarti Singh

1806.06237

Overdispersed Black-Box Variational Inference

Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei

1603.01140

Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction

Maosen Li, Siheng Chen, Yangheng Zhao, Ya Zhang, Yanfeng Wang, Qi Tian

2003.08802

EikoNet: Solving the Eikonal equation with Deep Neural Networks

Jonathan D. Smith, Kamyar Azizzadenesheli, Zachary E. Ross

2004.00361

Model Reduction and Neural Networks for Parametric PDEs

Kaushik Bhattacharya, Bamdad Hosseini, Nikola B. Kovachki, Andrew M. Stuart

2005.03180

Neural Operator: Graph Kernel Network for Partial Differential Equations

Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar

2003.03485

MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

Chiyu Max Jiang, Soheil Esmaeilzadeh, Kamyar Azizzadenesheli, Karthik Kashinath, Mustafa Mustafa, Hamdi A. Tchelepi, Philip Marcus, Prabhat, Anima Anandkumar

2005.01463

Is Pessimism Provably Efficient for Offline RL?

Ying Jin, Zhuoran Yang, Zhaoran Wang

2012.15085

A Review of Multiple Try MCMC algorithms for Signal Processing

Luca Martino

1801.09065

A Convergence Theory for Deep Learning via Over-Parameterization

Zeyuan Allen-Zhu, Yuanzhi Li, Zhao Song

1811.03962

Inductive Matrix Completion Based on Graph Neural Networks

Muhan Zhang, Yixin Chen

1904.12058

Robust and Differentially Private Mean Estimation

Xiyang Liu, Weihao Kong, Sham Kakade, Sewoong Oh

2102.09159

Robust Meta-learning for Mixed Linear Regression with Small Batches

Weihao Kong, Raghav Somani, Sham Kakade, Sewoong Oh

2006.09702

Robust Sub-Gaussian Principal Component Analysis and Width-Independent Schatten Packing

Arun Jambulapati, Jerry Li, Kevin Tian

2006.06980

Federated Optimization in Heterogeneous Networks

Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, Virginia Smith

1812.06127

Budgeted and Non-budgeted Causal Bandits

Vineet Nair, Vishakha Patil, Gaurav Sinha

2012.07058

The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation

Peter Kairouz, Ziyu Liu, Thomas Steinke

2102.06387

Adaptive Federated Learning in Resource Constrained Edge Computing Systems

Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, Kevin Chan

1804.05271

Memory-based Parameter Adaptation

Pablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae, Alexander Pritzel, Adrià Puigdomènech Badia, Benigno Uria, Oriol Vinyals, Demis Hassabis, Razvan Pascanu, Charles Blundell

1802.10542

The Curse of Concentration in Robust Learning: Evasion and Poisoning Attacks from Concentration of Measure

Saeed Mahloujifar, Dimitrios I. Diochnos, Mohammad Mahmoody

1809.03063

In Search of Lost Domain Generalization

Ishaan Gulrajani, David Lopez-Paz

2007.01434

An Investigation of Why Overparameterization Exacerbates Spurious Correlations

Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, Percy Liang

2005.04345

How to Construct Deep Recurrent Neural Networks

Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Yoshua Bengio

1312.6026

Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition

Haşim Sak, Andrew Senior, Françoise Beaufays

1402.1128

Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks

Caglar Gulcehre, Kyunghyun Cho, Razvan Pascanu, Yoshua Bengio

1311.1780

On Fast Dropout and its Applicability to Recurrent Networks

Justin Bayer, Christian Osendorfer, Daniela Korhammer, Nutan Chen, Sebastian Urban, Patrick van der Smagt

1311.0701

Metric-Free Natural Gradient for Joint-Training of Boltzmann Machines

Guillaume Desjardins, Razvan Pascanu, Aaron Courville, Yoshua Bengio

1301.3545

Knowledge Matters: Importance of Prior Information for Optimization

Çağlar Gülçehre, Yoshua Bengio

1301.4083

Training Neural Networks with Stochastic Hessian-Free Optimization

Ryan Kiros

1301.3641

What Regularized Auto-Encoders Learn from the Data Generating Distribution

Guillaume Alain, Yoshua Bengio

1211.4246

Batch Normalized Recurrent Neural Networks

César Laurent, Gabriel Pereyra, Philémon Brakel, Ying Zhang, Yoshua Bengio

1510.01378

Matrix Coherence and the Nystrom Method

Ameet Talwalkar, Afshin Rostamizadeh

1408.2044

Online Robust Subspace Tracking from Partial Information

Jun He, Laura Balzano, John C. S. Lui

1109.3827

Online Identification and Tracking of Subspaces from Highly Incomplete Information

Laura Balzano, Robert Nowak, Benjamin Recht

1006.4046

Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences

Benjamin Recht, Christopher Re

1202.4184

MAP Estimation, Linear Programming and Belief Propagation with Convex Free Energies

Yair Weiss, Chen Yanover, Talya Meltzer

1206.5286

Random Projections for Linear Support Vector Machines

Saurabh Paul, Christos Boutsidis, Malik Magdon-Ismail, Petros Drineas

1211.6085

Learning Bayesian Networks with Local Structure

Nir Friedman, Moises Goldszmidt

1302.3577

Neural Networks for Full Phase-space Reweighting and Parameter Tuning

Anders Andreassen, Benjamin Nachman

1907.08209

Simple And Efficient Architecture Search for Convolutional Neural Networks

Thomas Elsken, Jan-Hendrik Metzen, Frank Hutter

1711.04528

MadMiner: Machine learning-based inference for particle physics

Johann Brehmer, Felix Kling, Irina Espejo, Kyle Cranmer

1907.10621

Unmasking Clever Hans Predictors and Assessing What Machines Really Learn

Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller

1902.10178

Building competitive direct acoustics-to-word models for English conversational speech recognition

Kartik Audhkhasi, Brian Kingsbury, Bhuvana Ramabhadran, George Saon, Michael Picheny

1712.03133

Variational inference for large-scale models of discrete choice

Michael Braun, Jon McAuliffe

0712.2526

Computational Lower Bounds for Sparse PCA

Quentin Berthet, Philippe Rigollet

1304.0828

Using More Data to Speed-up Training Time

Shai Shalev-Shwartz, Ohad Shamir, Eran Tromer

1106.1216

Spectral clustering and the high-dimensional stochastic blockmodel

Karl Rohe, Sourav Chatterjee, Bin Yu

1007.1684

A survey of statistical network models

Anna Goldenberg, Alice X Zheng, Stephen E Fienberg, Edoardo M Airoldi

0912.5410

Piecewise linear regularized solution paths

Saharon Rosset, Ji Zhu

0708.2197

Robust PCA via Outlier Pursuit

Huan Xu, Constantine Caramanis, Sujay Sanghavi

1010.4237

Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study

Peng Xu, Farbod Roosta-Khorasani, Michael W. Mahoney

1708.07827

Semi-Stochastic Gradient Descent Methods

Jakub Konečný, Peter Richtárik

1312.1666

A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights

Weijie Su, Stephen Boyd, Emmanuel J. Candes

1503.01243

Sparse Recovery via Differential Inclusions

Stanley Osher, Feng Ruan, Jiechao Xiong, Yuan Yao, Wotao Yin

1406.7728

An Investigation of Newton-Sketch and Subsampled Newton Methods

Albert S. Berahas, Raghu Bollapragada, Jorge Nocedal

1705.06211

On the Complexity of Best Arm Identification in Multi-Armed Bandit Models

Emilie Kaufmann, Olivier Cappé, Aurélien Garivier

1407.4443

Adversarial Deep Learning for Robust Detection of Binary Encoded Malware

Abdullah Al-Dujaili, Alex Huang, Erik Hemberg, Una-May O'Reilly

1801.02950

Fast global convergence of gradient methods for high-dimensional statistical recovery

Alekh Agarwal, Sahand N. Negahban, Martin J. Wainwright

1104.4824

Asymptotic normality and optimalities in estimation of large Gaussian graphical models

Zhao Ren, Tingni Sun, Cun-Hui Zhang, Harrison H. Zhou

1309.6024

Scaled Sparse Linear Regression

Tingni Sun, Cun-Hui Zhang

1104.4595

Online Bandit Learning against an Adaptive Adversary: from Regret to Policy Regret

Raman Arora, Ofer Dekel, Ambuj Tewari

1206.6400

On the conditions used to prove oracle results for the Lasso

Sara A. van de Geer, Peter Bühlmann

0910.0722

Towards minimax policies for online linear optimization with bandit feedback

Sébastien Bubeck, Nicolò Cesa-Bianchi, Sham M. Kakade

1202.3079

Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theory

Adel Javanmard, Andrea Montanari

1301.4240

Likelihood-free inference with emulator networks

Jan-Matthis Lueckmann, Giacomo Bassetto, Theofanis Karaletsos, Jakob H. Macke

1805.09294

Bias Mitigation Post-processing for Individual and Group Fairness

Pranay K. Lohia, Karthikeyan Natesan Ramamurthy, Manish Bhide, Diptikalyan Saha, Kush R. Varshney, Ruchir Puri

1812.06135

Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking

Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, Mounia Lalmas

1706.06691

Deep Learning with Limited Numerical Precision

Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, Pritish Narayanan

1502.02551

Adversarial vulnerability for any classifier

Alhussein Fawzi, Hamza Fawzi, Omar Fawzi

1802.08686

Learning Optimized Risk Scores

Berk Ustun, Cynthia Rudin

1610.00168

Lower Bounds for Higher-Order Convex Optimization

Naman Agarwal, Elad Hazan

1710.10329

Gradient Adversarial Training of Neural Networks

Ayan Sinha, Zhao Chen, Vijay Badrinarayanan, Andrew Rabinovich

1806.08028

Randomized Prior Functions for Deep Reinforcement Learning

Ian Osband, John Aslanides, Albin Cassirer

1806.03335

Natural Adversarial Examples

Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, Dawn Song

1907.07174

Spherical CNNs

Taco S. Cohen, Mario Geiger, Jonas Koehler, Max Welling

1801.10130

Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks

Yarin Gal, Lewis Smith

1806.00667

Generalisation in humans and deep neural networks

Robert Geirhos, Carlos R. Medina Temme, Jonas Rauber, Heiko H. Schütt, Matthias Bethge, Felix A. Wichmann

1808.08750

Penalizing Unfairness in Binary Classification

Yahav Bechavod, Katrina Ligett

1707.00044

A Tutorial on Learning With Bayesian Networks

David Heckerman

2002.00269

Adaptively Sparse Transformers

Gonçalo M. Correia, Vlad Niculae, André F. T. Martins

1909.00015

Deep EHR: Chronic Disease Prediction Using Medical Notes

Jingshu Liu, Zachariah Zhang, Narges Razavian

1808.04928

The Lottery Ticket Hypothesis for Pre-trained BERT Networks

Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, Michael Carbin

2007.12223

Traversing Knowledge Graphs in Vector Space

Kelvin Guu, John Miller, Percy Liang

1506.01094

Causal inference using the algorithmic Markov condition

Dominik Janzing, Bernhard Schoelkopf

0804.3678

A Linearly Convergent Conditional Gradient Algorithm with Applications to Online and Stochastic Optimization

Dan Garber, Elad Hazan

1301.4666

Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning

Xuan Wang, Yu Zhang, Xiang Ren, Yuhao Zhang, Marinka Zitnik, Jingbo Shang, Curtis Langlotz, Jiawei Han

1801.09851

Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications

Rishabh Iyer, Jeff A. Bilmes

1408.2051

Conditional Gradient Algorithms for Norm-Regularized Smooth Convex Optimization

Zaid Harchaoui, Anatoli Juditsky, Arkadi Nemirovski

1302.2325

On Universal Prediction and Bayesian Confirmation

Marcus Hutter

0709.1516

Three Factors Influencing Minima in SGD

Stanisław Jastrzębski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, Amos Storkey

1711.04623

CoinPress: Practical Private Mean and Covariance Estimation

Sourav Biswas, Yihe Dong, Gautam Kamath, Jonathan Ullman

2006.06618

Near-Optimal Reinforcement Learning with Self-Play

Yu Bai, Chi Jin, Tiancheng Yu

2006.12007

Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning

Nicolas Papernot, Patrick McDaniel

1803.04765

A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time

Yeqi Gao, Zhao Song, Weixin Wang, Junze Yin

2309.07418

How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation

Josh Alman, Zhao Song

2310.04064

Solving Inverse Problems in Medical Imaging with Score-Based Generative Models

Yang Song, Liyue Shen, Lei Xing, Stefano Ermon

2111.08005

Latent Space Oddity: on the Curvature of Deep Generative Models

Georgios Arvanitidis, Lars Kai Hansen, Søren Hauberg

1710.11379

On Coresets for Logistic Regression

Alexander Munteanu, Chris Schwiegelshohn, Christian Sohler, David P. Woodruff

1805.08571

Variance-Based Rewards for Approximate Bayesian Reinforcement Learning

Jonathan Sorg, Satinder Singh, Richard L. Lewis

1203.3518

Learning Mixtures of Linear Regressions in Subexponential Time via Fourier Moments

Sitan Chen, Jerry Li, Zhao Song

1912.07629

The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent

Karthik A. Sankararaman, Soham De, Zheng Xu, W. Ronny Huang, Tom Goldstein

1904.06963

Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Games

Tianyi Lin, Zhengyuan Zhou, Panayotis Mertikopoulos, Michael I. Jordan

2002.09806

On the Sample Complexity of Privately Learning Unbounded High-Dimensional Gaussians

Ishaq Aden-Ali, Hassan Ashtiani, Gautam Kamath

2010.09929

Causal Confusion in Imitation Learning

Pim de Haan, Dinesh Jayaraman, Sergey Levine

1905.11979

Faster Algorithms for High-Dimensional Robust Covariance Estimation

Yu Cheng, Ilias Diakonikolas, Rong Ge, David Woodruff

1906.04661

Quantum Entropy Scoring for Fast Robust Mean Estimation and Improved Outlier Detection

Yihe Dong, Samuel B. Hopkins, Jerry Li

1906.11366

Identifiability of deep generative models without auxiliary information

Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam

2206.10044

An Introduction to Matrix Concentration Inequalities

Joel A. Tropp

1501.01571

Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning

Minghai Chen, Sen Wang, Paul Pu Liang, Tadas Baltrušaitis, Amir Zadeh, Louis-Philippe Morency

1802.00924

Recurrent Attentional Networks for Saliency Detection

Jason Kuen, Zhenhua Wang, Gang Wang

1604.03227

Wasserstein Learning of Deep Generative Point Process Models

Shuai Xiao, Mehrdad Farajtabar, Xiaojing Ye, Junchi Yan, Le Song, Hongyuan Zha

1705.08051

Provably Good Batch Reinforcement Learning Without Great Exploration

Yao Liu, Adith Swaminathan, Alekh Agarwal, Emma Brunskill

2007.08202

Efficiently Searching for Frustrated Cycles in MAP Inference

David Sontag, Do Kook Choe, Yitao Li

1210.4902

Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results

Antti Tarvainen, Harri Valpola

1703.01780

The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems

Robert Krajewski, Julian Bock, Laurent Kloeker, Lutz Eckstein

1810.05642

Momentum-Based Variance Reduction in Non-Convex SGD

Ashok Cutkosky, Francesco Orabona

1905.10018

On Markov chain Monte Carlo methods for tall data

Rémi Bardenet, Arnaud Doucet, Chris Holmes

1505.02827

Batch Policy Learning in Average Reward Markov Decision Processes

Peng Liao, Zhengling Qi, Runzhe Wan, Predrag Klasnja, Susan Murphy

2007.11771

Convergence Rates of Variational Posterior Distributions

Fengshuo Zhang, Chao Gao

1712.02519

Last iterate convergence in no-regret learning: constrained min-max optimization for convex-concave landscapes

Qi Lei, Sai Ganesh Nagarajan, Ioannis Panageas, Xiao Wang

2002.06768

Federated Learning: Challenges, Methods, and Future Directions

Tian Li, Anit Kumar Sahu, Ameet Talwalkar, Virginia Smith

1908.07873

Metropolis Sampling

Luca Martino, Victor Elvira

1704.04629

Error Estimation for Randomized Least-Squares Algorithms via the Bootstrap

Miles E. Lopes, Shusen Wang, Michael W. Mahoney

1803.08021

DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability

Cian Eastwood, Andrei Liviu Nicolicioiu, Julius von Kügelgen, Armin Kekić, Frederik Träuble, Andrea Dittadi, Bernhard Schölkopf

2210.00364

Moment based estimation of stochastic Kronecker graph parameters

David F. Gleich, Art B. Owen

1106.1674

Strong Coresets for Hard and Soft Bregman Clustering with Applications to Exponential Family Mixtures

Mario Lucic, Olivier Bachem, Andreas Krause

1508.05243

Piecewise Deterministic Markov Processes for Continuous-Time Monte Carlo

Paul Fearnhead, Joris Bierkens, Murray Pollock, Gareth O Roberts

1611.07873

Missing Data Imputation with Adversarially-trained Graph Convolutional Networks

Indro Spinelli, Simone Scardapane, Aurelio Uncini

1905.01907

Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings

Jingyan Wang, Nihar B. Shah

1806.05085

Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy

Zuyue Fu, Zhuoran Yang, Zhaoran Wang

2008.00483

Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification

Yinhao Zhu, Nicholas Zabaras

1801.06879

Towards Physics-informed Deep Learning for Turbulent Flow Prediction

Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, Rose Yu

1911.08655

Link prediction for partially observed networks

Yunpeng Zhao, Elizaveta Levina, Ji Zhu

1301.7047

Traditional and Heavy-Tailed Self Regularization in Neural Network Models

Charles H. Martin, Michael W. Mahoney

1901.08276

Simulation Assisted Likelihood-free Anomaly Detection

Anders Andreassen, Benjamin Nachman, David Shih

2001.05001

Inductive Representation Learning on Temporal Graphs

Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, Kannan Achan

2002.07962