The following list considers papers related to neural architecture search. It is by no means a complete list. If you miss a paper on this list, please let us know.
Update (Dec 2018): Since the list is already quite long by now, we will highlight papers accepted at conferences in the future. This should hopefully provide some guidance towards high-quality papers.
- Architecture Search (and Hyperparameter Optimization):
- Learning Implicitly Recurrent CNNs Through Parameter Sharing (Savarese and Maire, accepted at ICLR’19)
https://arxiv.org/abs/1902.09701 - Evaluating the Search Phase of Neural Architecture Searc (Sciuto et al. 2019)
https://arxiv.org/abs/1902.08142 - Random Search and Reproducibility for Neural Architecture Search (Li and Talwalkar 2019)
https://arxiv.org/abs/1902.07638 - Evolutionary Neural AutoML for Deep Learning (Liang et al. 2019)
https://arxiv.org/abs/1902.06827 - Fast Task-Aware Architecture Inference (Kokiopoulou et al. 2019)
https://arxiv.org/abs/1902.05781 - Probabilistic Neural Architecture Search (Casale et al. 2019)
https://arxiv.org/abs/1902.05116 - Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search (Jiang et al. 2019)
https://arxiv.org/abs/1901.11211 - The Evolved Transformer (So et al. 2019)
https://arxiv.org/abs/1901.11117 - Designing neural networks through neuroevolution (Stanley et al. 2019; accepted at Nature Machine Intelligence)
https://www.nature.com/articles/s42256-018-0006-z - NeuNetS: An Automated Synthesis Engine for Neural Network Design (Sood et al. 2019)
https://arxiv.org/abs/1901.06261 - Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search (Chu et al. 2019)
https://arxiv.org/abs/1901.07261 - EAT-NAS: Elastic Architecture Transfer for Accelerating Large-scale Neural Architecture Search (Fang et al. 2019)
https://arxiv.org/abs/1901.05884 - Bayesian Learning of Neural Network Architectures (Dikov et al. 2019)
https://arxiv.org/abs/1901.04436 - Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation (Liu et al. 2019)
https://arxiv.org/abs/1901.02985 - The Art of Getting Deep Neural Networks in Shape (Mammadli et al. 2019; accepted at TACO Journal)
https://dl.acm.org/citation.cfm?id=3291053 - Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search (Chu et al. 2019)
https://arxiv.org/abs/1901.01074 - SNAS: Stochastic Neural Architecture Search (Xie et al. 2018; accepted at ICLR’19)
https://arxiv.org/abs/1812.09926 - Graph Hypernetworks for Neural Architecture Search (Zhang et al. 2018; Accepted at ICLR’19)
https://arxiv.org/abs/1810.05749 - Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution (Elsken et al. 2018; accepted at ICLR’19)
https://arxiv.org/abs/1804.09081 - Macro Neural Architecture Search Revisited (Hu et al. 2018; accepted at Meta-Learn NIPS workshop’18)
http://metalearning.ml/2018/papers/metalearn2018_paper16.pdf - AMLA: an AutoML frAmework for Neural Network Design (Kamath et al. 2018; at ICML AutoML workshop)
http://pkamath.com/publications/papers/amla_automl18.pdf - ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation (Dai et al. 2018)
https://arxiv.org/abs/1812.08934 - Neural Architecture Search Over a Graph Search Space (de Laroussilhe et al. 2018)
https://arxiv.org/abs/1812.10666 - A Review of Meta-Reinforcement Learning for Deep Neural Networks Architecture Search (Jaafra et al. 2018)
https://arxiv.org/abs/1812.07995 - Evolutionary Neural Architecture Search for Image Restoration (van Wyk and Bosman 2018)
https://arxiv.org/abs/1812.05866 - IRLAS: Inverse Reinforcement Learning for Architecture Search (Guo et al. 2018)
https://arxiv.org/abs/1812.05285 - FBNet: Hardware-Aware Efficient ConvNet Designvia Differentiable Neural Architecture Search (Wu et al. 2018)
https://arxiv.org/abs/1812.03443 - ShuffleNASNets: Efficient CNN models throughmodified Efficient Neural Architecture Search (Laube et al. 2018)
https://arxiv.org/abs/1812.02975 - ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware (Cai et al. 2018)
https://arxiv.org/abs/1812.00332 - Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search (Wu et al. 2018)
https://arxiv.org/abs/1812.00090 - Evolving Space-Time Neural Architectures for Videos (Piergiovanni et al. 2018)
https://arxiv.org/abs/1811.10636 - InstaNAS: Instance-aware Neural Architecture Search (Cheng et al. 2018)
https://arxiv.org/abs/1811.10201 - Evolutionary-Neural Hybrid Agents for Architecture Search (Maziarz et al. 2018)
https://arxiv.org/abs/1811.09828 - Joint Neural Architecture Search and Quantization (Chen et al. 2018)
https://arxiv.org/abs/1811.09426 - Transfer Learning with Neural AutoML (Wong et al. 2018)
http://papers.nips.cc/paper/8056-transfer-learning-with-neural-automl.pdf - Evolving Image Classification Architectures with Enhanced Particle Swarm Optimisation (Fielding and Zhang 2018)
https://ieeexplore.ieee.org/document/8533601 - Deep Active Learning with a Neural Architecture Search (Geifman and El-Yaniv 2018)
https://arxiv.org/abs/1811.07579 - Stochastic Adaptive Neural Architecture Search for Keyword Spotting (Véniat et al. 2018)
https://arxiv.org/abs/1811.06753 - You only search once: Single Shot Neural Architecture Search via Direct Sparse Optimization (Zhang et al. 2018)
https://arxiv.org/abs/1811.01567 - Automatically Evolving CNN Architectures Based on Blocks (Sun et al. 2018)
https://arxiv.org/abs/1810.11875 - Training Frankenstein’s Creature to Stack: HyperTree Architecture Search (Hundt et al. 2018)
https://arxiv.org/abs/1810.11714 - Fast Neural Architecture Search of Compact Semantic Segmentation Models
via Auxiliary Cells (Nekrasov et al. 2018)
https://arxiv.org/abs/1810.10804 - Automatic Configuration of Deep Neural Networks with Parallel Efficient Global Optimization (van Stein et al. 2018)
https://arxiv.org/abs/1810.05526 - Gradient Based Evolution to Optimize the Structure of Convolutional Neural Networks (Mitschke et al. 2018)
https://ieeexplore.ieee.org/document/8451394 - Neural Architecture Optimization (Luo et al. 2018)
https://arxiv.org/abs/1808.07233 - Exploring Shared Structures and Hierarchies for Multiple NLP Tasks (Chen et al. 2018)
https://arxiv.org/abs/1808.07658 - Neural Architecture Search: A Survey (Elsken et al. 2018)
https://arxiv.org/abs/1808.05377 - BlockQNN: Efficient Block-wise Neural Network Architecture Generation (Zhong et al. 2018)
https://arxiv.org/abs/1808.05584 - Automatically Designing CNN Architectures Using Genetic Algorithm for Image Classification (Sunet al. 2018)
https://arxiv.org/abs/1808.03818 - Reinforced Evolutionary Neural Architecture Search (Chen et al. 2018)
https://arxiv.org/abs/1808.00193 - Teacher Guided Architecture Search (Bashivan et al. 2018)
https://arxiv.org/abs/1808.01405 - Efficient Progressive Neural Architecture Search (Perez-Rua et al. 2018)
https://arxiv.org/abs/1808.00391 - Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search (Zela et al. 2018)
https://arxiv.org/abs/1807.06906 - Automatically Designing CNN Architectures for Medical Image Segmentation (Mortazi and Bagci 2018)
https://arxiv.org/abs/1807.07663 - Path-Level Network Transformation for Efficient Architecture Search (Cai et al. 2018; accepted at ICML’18)
https://arxiv.org/abs/1806.02639 - Lamarckian Evolution of Convolutional Neural Networks (Prellberg and Kramer, 2018)
https://arxiv.org/abs/1806.08099 - Deep Learning Architecture Search by Neuro-Cell-based Evolution with Function-Preserving Mutations (Wistuba, 2018)
http://www.ecmlpkdd2018.org/wp-content/uploads/2018/09/108.pdf - DARTS: Differentiable Architecture Search (Liu et al. 2018)
https://arxiv.org/abs/1806.09055 - Constructing Deep Neural Networks by Bayesian Network Structure Learning (Rohekar et al. 2018)
https://arxiv.org/abs/1806.09141 - Resource-Efficient Neural Architect (Zhou et al. 2018)
https://arxiv.org/abs/1806.07912 - Efficient Neural Architecture Search with Network Morphism (Jin et al. 2018)
https://arxiv.org/abs/1806.10282 - TAPAS: Train-less Accuracy Predictor for Architecture Search (Istrate et al. 2018)
https://arxiv.org/abs/1806.00250 - AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search (Wang et al 2018)
https://arxiv.org/abs/1805.07440 - GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learning (Huang et al 2018)
https://arxiv.org/abs/1804.06964 - Evolutionary Architecture Search For Deep Multitask Networks (Liang et al. 2018)
https://arxiv.org/abs/1803.03745 - From Nodes to Networks: Evolving Recurrent Neural Networks (Rawal et al. 2018)
https://arxiv.org/abs/1803.04439 - Neural Architecture Construction using EnvelopeNets (Kamath et al. 2018)
https://arxiv.org/abs/1803.06744 - Neural Architecture Search with Bayesian Optimisation and Optimal Transport (Kandasamy et al. 2018)
https://arxiv.org/abs/1802.07191 - Efficient Neural Architecture Search via Parameter Sharing (Pham et al. 2018; accepted at ICML’18)
https://arxiv.org/abs/1802.03268 - Regularized Evolution for Image Classifier Architecture Search (Real et al. 2018)
https://arxiv.org/abs/1802.01548 - Effective Building Block Design for Deep Convolutional Neural Networks using Search (Dutta et al. 2018)
https://arxiv.org/abs/1801.08577 - Combination of Hyperband and Bayesian Optimization for Hyperparameter Optimization in Deep Learning (Wang et al. 2018)
https://arxiv.org/abs/1801.01596 - Memetic Evolution of Deep Neural Networks (Lorenzo and Nalepa 2018)
https://dl.acm.org/citation.cfm?id=3205631 - Understanding and Simplifying One-Shot Architecture Search (Bender et al. 2018; accepted at ICML’18)
http://proceedings.mlr.press/v80/bender18a/bender18a.pdf - Differentiable Neural Network Architecture Search (Shin et al. 2018; accepted at ICLR’18 workshop)
https://openreview.net/pdf?id=BJ-MRKkwG - N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning (Ashok et al. 2017; accepted at ICLR’18)
https://arxiv.org/abs/1709.06030 - Genetic CNN (Xie and Yuille, 2017; accepted at ICCV’17)
https://arxiv.org/abs/1703.01513 - MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks (Gordon et al. 2017)
https://arxiv.org/abs/1711.06798 - MaskConnect: Connectivity Learning by Gradient Descent (Ahmed and Torresani. 2017)
https://arxiv.org/abs/1709.09582 - A Flexible Approach to Automated RNN Architecture Generation (Schrimpf et al. 2017)
https://arxiv.org/abs/1712.07316 - DeepArchitect: Automatically Designing and Training Deep Architectures (Negrinho and Gordon 2017)
https://arxiv.org/abs/1704.08792 - A Genetic Programming Approach to Designing Convolutional Neural Network Architectures (Suganuma et al. 2017; accepted at GECCO’17)
https://arxiv.org/abs/1704.00764 - Practical Block-wise Neural Network Architecture Generation (Zhong et al. 2017; accepted at CVPR’18)
https://arxiv.org/abs/1708.05552 - Accelerating Neural Architecture Search using Performance Prediction (Baker et al. 2017; accepted at NIPS worshop on Meta-Learning 2017)
https://arxiv.org/abs/1705.10823 - Large-Scale Evolution of Image Classifiers (Real et al. 2017; accepted at ICML’17)
https://arxiv.org/abs/1703.01041 - Hierarchical Representations for Efficient Architecture Search (Liu et al. 2017; accepted at ICLR’18)
https://arxiv.org/abs/1711.00436 - Neural Optimizer Search with Reinforcement Learning (Bello et al. 2017)
https://arxiv.org/abs/1709.07417 - Progressive Neural Architecture Search (Liu et al. 2017; accepted at ECCV’18)
https://arxiv.org/abs/1712.00559 - Learning Transferable Architectures for Scalable Image Recognition (Zoph et al. 2017; CVPR’18)
https://arxiv.org/abs/1707.07012 - Simple And Efficient Architecture Search for Convolutional Neural Networks (Elsken et al. 2017; accepted at NIPS workshop on Meta-Learning’17)
https://arxiv.org/abs/1711.04528 - Bayesian Optimization Combined with Incremental Evaluation for Neural Network Architecture Optimization (Wistuba, 2017)
https://www.semanticscholar.org/paper/Bayesian-Optimization-Combined-with-Successive-for-Wistuba/ddb182533c91f0941f088e1e298c52a111253554 - Finding Competitive Network Architectures Within a Day Using UCT (Wistuba 2017)
https://arxiv.org/abs/1712.07420 - Hyperparameter Optimization: A Spectral Approach (Hazan et al. 2017)
https://arxiv.org/abs/1706.00764 - SMASH: One-Shot Model Architecture Search through HyperNetworks (Brock et al. 2017; accepted at NIPS workshop on Meta-Learning’17)
https://arxiv.org/abs/1708.05344 - Efficient Architecture Search by Network Transformation (Cai et al. 2017; accepted at AAAI’18)
https://arxiv.org/abs/1707.04873 - Modularized Morphing of Neural Networks (Wei et al. 2017)
https://arxiv.org/abs/1701.03281 - Towards Automatically-Tuned Neural Networks (Mendoza et al. 2016; accepted at ICML AutoML workshop)
http://proceedings.mlr.press/v64/mendoza_towards_2016.html - AdaNet: Adaptive Structural Learning of Artificial Neural Networks (Cortes et al. 2016)
https://arxiv.org/abs/1607.01097 - Network Morphism (Wei et al. 2016)
https://arxiv.org/abs/1603.01670 - Convolutional Neural Fabrics (Saxena and Verbeek 2016; accepted at NIPS’16)
https://arxiv.org/abs/1606.02492 - CMA-ES for Hyperparameter Optimization of Deep Neural Networks (Loshchilov and Hutter 2016)
https://arxiv.org/abs/1604.07269 - Designing Neural Network Architectures using Reinforcement Learning (Baker et al. 2016; accepted at ICLR’17)
https://arxiv.org/abs/1611.02167 - Neural Architecture Search with Reinforcement Learning (Zoph and Le. 2016; accepted at ICLR’17)
https://arxiv.org/abs/1611.01578 - Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (Li et al. 2016)
https://arxiv.org/abs/1603.06560 - Net2Net: Accelerating Learning via Knowledge Transfer (Chen et al. 2015; accepted at ICLR’16)
https://arxiv.org/abs/1511.05641 - Optimizing deep learning hyper-parameters through an evolutionary algorithm (Young et al. 2015)
https://dl.acm.org/citation.cfm?id=2834896 - Practical Bayesian Optimization of Machine Learning Algorithms (Snoek et al. 2012)
https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf - A Hypercube-based Encoding for Evolving large-scale Neural Networks (Stanley et al. 2009)
https://ieeexplore.ieee.org/document/6792316/ - Neuroevolution: From Architectures to Learning (Floreano et al. 2008; accepted at Evolutionary Intelligence’08)
https://link.springer.com/article/10.1007/s12065-007-0002-4 - Evolving Neural Networks through Augmenting Topologies (Stanley and Miikkulainen, 2002; accepted at Evolutionary Computation’02)
http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf - Evolving Artificial Neural Networks (Yao, 1999; accepted at IEEE)
https://ieeexplore.ieee.org/document/784219/ - An Evolutionary Algorithm that Constructs Recurrent Neural Networks (Angeline et al. 1994)
https://ieeexplore.ieee.org/document/265960/ - Designing Neural Networks Using Genetic Algorithms with Graph Generation System (Kitano, 1990)
http://www.complex-systems.com/abstracts/v04_i04_a06/ - Designing Neural Networks using Genetic Algorithms (Miller et al. 1989; accepted at ICGA’89)
https://dl.acm.org/citation.cfm?id=94034 - The Cascade-Correlation Learning Architecture (Fahlman and Leblere, 1989)
https://papers.nips.cc/paper/207-the-cascade-correlation-learning-architecture - Self Organizing Neural Networks for the Identification Problem (Tenorio and Lee, 1988)
https://papers.nips.cc/paper/149-self-organizing-neural-networks-for-the-identification-problem
- Learning Implicitly Recurrent CNNs Through Parameter Sharing (Savarese and Maire, accepted at ICLR’19)
- Make it more efficient
- Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks (Hinz et al. 2018)
https://www.worldscientific.com/doi/abs/10.1142/S1469026818500086 - Gitgraph – From Computational Subgraphs to Smaller Architecture search spaces (Bennani-Smires et al. 2018)
https://openreview.net/pdf?id=rkiO1_1Pz - Transfer Automatic Machine Learning (Wong et al. 2018)
https://arxiv.org/abs/1803.02780 - Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets (Klein et al. 2017; accepted at AIStat’17)
http://proceedings.mlr.press/v54/klein17a.html - Learning curve prediction with Bayesian Neural Networks (Klein et al. 2017: accepted at ICLR’17)
http://ml.informatik.uni-freiburg.de/papers/17-ICLR-LCNet.pdf
- Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks (Hinz et al. 2018)
- Multi-Objective
- TEA-DNN: the Quest for Time-Energy-Accuracy Co-optimized Deep Neural Networks (Cai et al. 2018)
https://arxiv.org/abs/1811.12065 - NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search (Lu et al. 2018)
https://arxiv.org/abs/1810.03522 - Searching Toward Pareto-Optimal Device-Aware Neural Architectures (Cheng et al. 2018)
https://arxiv.org/abs/1808.09830 - Multi-objective Architecture Search for CNNs (Elsken et al. 2018)
https://arxiv.org/abs/1804.09081 - MnasNet: Platform-Aware Neural Architecture Search for Mobile (Tan et al. 2018)
https://arxiv.org/abs/1807.11626 - MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning (Hsu et al. 2018)
https://arxiv.org/abs/1806.10332 - PPP-Net: Platform-aware progressive search for pareto-optimal neural architectures (Dong et al. 2018)
https://openreview.net/pdf?id=B1NT3TAIM - Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization (Smithson et al. 2016)
https://arxiv.org/abs/1611.02120
- TEA-DNN: the Quest for Time-Energy-Accuracy Co-optimized Deep Neural Networks (Cai et al. 2018)
- Maintained by Marius Lindauer and Arber Zela; Last update: February 28th 2019