Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
5555 |
|
| 1046. | Zheng, X; Ji, R; Chen, Y; Wang, Q; Zhang, B; Ye, Q; Chen, J; Huang, F; Tian, Y MIGO-NAS: Towards Fast and Generalizable Neural Architecture Search Journal Article IEEE Transactions on Pattern Analysis & Machine Intelligence, (01), pp. 1-1, 5555, ISSN: 1939-3539. @article{9377468, title = {MIGO-NAS: Towards Fast and Generalizable Neural Architecture Search}, author = {X Zheng and R Ji and Y Chen and Q Wang and B Zhang and Q Ye and J Chen and F Huang and Y Tian}, url = {https://www.computer.org/csdl/journal/tp/5555/01/09377468/1rUNdbz4LQY}, doi = {10.1109/TPAMI.2021.3065138}, issn = {1939-3539}, year = {5555}, date = {5555-03-01}, journal = {IEEE Transactions on Pattern Analysis & Machine Intelligence}, number = {01}, pages = {1-1}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
2021 |
|
| 1045. | Chen, Wuyang; Gong, Xinyu; Wang, Zhangyang Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective Inproceedings ICLR 2021, 2021. @inproceedings{chen2021neural, title = {Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective}, author = {Wuyang Chen and Xinyu Gong and Zhangyang Wang}, url = {https://arxiv.org/abs/2102.11535}, year = {2021}, date = {2021-05-04}, booktitle = {ICLR 2021}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
| 1044. | Pan, Zheyi; Ke, Songyu; Yang, Xiaodu; Liang, Yuxuan; Yu, Yong; Zhang, Junbo; Zheng, Yu AutoSTG: Neural Architecture Search forPredictions of Spatio-Temporal Graphs Inproceedings WWW 2021, 2021. @inproceedings{PanWWW2021, title = {AutoSTG: Neural Architecture Search forPredictions of Spatio-Temporal Graphs}, author = {Zheyi Pan and Songyu Ke and Xiaodu Yang and Yuxuan Liang and Yong Yu and Junbo Zhang and Yu Zheng}, url = {http://panzheyi.cc/publication/pan2021autostg/paper.pdf}, year = {2021}, date = {2021-04-19}, booktitle = {WWW 2021}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
| 1043. | Zhang, Zhentong ; Shan, Yugang; Yuan, Jie Multi-Level Cell Progressive Differentiable Architecture Search to Improve Image Classification Accuracy Journal Article Journal of Signal Processing Systems, 2021. @article{ZhangJSPS2021, title = {Multi-Level Cell Progressive Differentiable Architecture Search to Improve Image Classification Accuracy}, author = {Zhang, Zhentong and Yugang Shan and Jie Yuan}, url = {https://doi.org/10.1007/s11265-021-01647-1}, year = {2021}, date = {2021-03-08}, journal = {Journal of Signal Processing Systems}, abstract = {In recent years, the neural architecture search has continuously made significant progress in the field of image recognition. Among them, the differentiable method has obvious advantages compared with other search methods in terms of computational cost and accuracy to deal with image classification. However, the differentiable method is usually composed of single cell, which cannot efficiently extract the features of the network. In response to this problem, we propose a multi-level cell progressive differentiable method which allows cells to have different types according to the levels of the network. In differentiable method, the gap between the search network and the evaluation one is large, and the correlation is low. We design an algorithm to improve the distribution of architecture parameters. We also optimize the loss function and use the regularization method of additional action to improve deep network performance. The method achieves good search and classification results on CIFAR10 and ImageNet (mobile setting).}, keywords = {}, pubstate = {published}, tppubtype = {article} } In recent years, the neural architecture search has continuously made significant progress in the field of image recognition. Among them, the differentiable method has obvious advantages compared with other search methods in terms of computational cost and accuracy to deal with image classification. However, the differentiable method is usually composed of single cell, which cannot efficiently extract the features of the network. In response to this problem, we propose a multi-level cell progressive differentiable method which allows cells to have different types according to the levels of the network. In differentiable method, the gap between the search network and the evaluation one is large, and the correlation is low. We design an algorithm to improve the distribution of architecture parameters. We also optimize the loss function and use the regularization method of additional action to improve deep network performance. The method achieves good search and classification results on CIFAR10 and ImageNet (mobile setting). |
| 1042. | Liu, Lanlan; Zhang, Yuting; Deng, Jia; Soatto, Stefano Dynamically Grown Generative Adversarial Networks Inproceedings AAAI 2021, 2021. @inproceedings{LiuAAAI2021, title = {Dynamically Grown Generative Adversarial Networks}, author = {Lanlan Liu and Yuting Zhang and Jia Deng and Stefano Soatto}, url = {https://www.aaai.org/AAAI21Papers/AAAI-1376.LiuL.pdf}, year = {2021}, date = {2021-02-02}, booktitle = {AAAI 2021}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
| 1041. | Xu, Y; Xie, L; Dai, W; Zhang, X; Chen, X; Qi, G; Xiong, H; Tian, Q Partially-Connected Neural Architecture Search for Reduced Computational Redundancy Journal Article IEEE Transactions on Pattern Analysis & Machine Intelligence, (01), pp. 1-1, 2021, ISSN: 1939-3539. @article{9354953, title = {Partially-Connected Neural Architecture Search for Reduced Computational Redundancy}, author = {Y Xu and L Xie and W Dai and X Zhang and X Chen and G Qi and H Xiong and Q Tian}, url = {https://www.computer.org/csdl/journal/tp/5555/01/09354953/1rgCccYlOaQ}, doi = {10.1109/TPAMI.2021.3059510}, issn = {1939-3539}, year = {2021}, date = {2021-02-01}, journal = {IEEE Transactions on Pattern Analysis & Machine Intelligence}, number = {01}, pages = {1-1}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, abstract = {Differentiable architecture search (DARTS) enables effective neural architecture search (NAS) using gradient descent, but suffers from high memory and computational costs. In this paper, we propose a novel approach, namely Partially-Connected DARTS (PC-DARTS), to achieve efficient and stable neural architecture search by reducing the channel and spatial redundancies of the super-network. In the channel level, partial channel connection is presented to randomly sample a small subset of channels for operation selection to accelerate the search process and suppress the over-fitting of the super-network. Side operation is introduced for bypassing (non-sampled) channels to guarantee the performance of searched architectures under extremely low sampling rates. In the spatial level, input features are down-sampled to eliminate spatial redundancy and enhance the efficiency of the mixed computation for operation selection. Furthermore, edge normalization is developed to maintain the consistency of edge selection based on channel sampling with the architectural parameters for edges. Experimental results demonstrate that the proposed approach achieves higher search speed and training stability than DARTS. PC-DARTS obtains a top-1 error rate of 2.55% on CIFAR-10 with 0.07 GPU-days for architecture search, and a state-of-the-art top-1 error rate of 24.1% on ImageNet (under the mobile setting) within 2.8 GPU-day.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Differentiable architecture search (DARTS) enables effective neural architecture search (NAS) using gradient descent, but suffers from high memory and computational costs. In this paper, we propose a novel approach, namely Partially-Connected DARTS (PC-DARTS), to achieve efficient and stable neural architecture search by reducing the channel and spatial redundancies of the super-network. In the channel level, partial channel connection is presented to randomly sample a small subset of channels for operation selection to accelerate the search process and suppress the over-fitting of the super-network. Side operation is introduced for bypassing (non-sampled) channels to guarantee the performance of searched architectures under extremely low sampling rates. In the spatial level, input features are down-sampled to eliminate spatial redundancy and enhance the efficiency of the mixed computation for operation selection. Furthermore, edge normalization is developed to maintain the consistency of edge selection based on channel sampling with the architectural parameters for edges. Experimental results demonstrate that the proposed approach achieves higher search speed and training stability than DARTS. PC-DARTS obtains a top-1 error rate of 2.55% on CIFAR-10 with 0.07 GPU-days for architecture search, and a state-of-the-art top-1 error rate of 24.1% on ImageNet (under the mobile setting) within 2.8 GPU-day. |
| 1040. | Lakhmiri, Dounia; é, S Use of static surrogates in hyperparameter optimization Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-07963, title = {Use of static surrogates in hyperparameter optimization}, author = {Dounia Lakhmiri and S é}, url = {https://arxiv.org/abs/2103.07963}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.07963}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1039. | -, Sian; -, Wei Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-07289, title = {Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator}, author = {Sian - and Wei -}, url = {https://arxiv.org/abs/2103.07289}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.07289}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1038. | Wang, Ruochen; Cheng, Minhao; Chen, Xiangning; Tang, Xiaocheng; Hsieh, Cho-Jui Rethinking Architecture Selection in Differentiable NAS Inproceedings International Conference on Learning Representations, 2021. @inproceedings{<LineBreak>wang2021rethinking, title = {Rethinking Architecture Selection in Differentiable NAS}, author = {Ruochen Wang and Minhao Cheng and Xiangning Chen and Xiaocheng Tang and Cho-Jui Hsieh}, url = {https://openreview.net/forum?id=PKubaeJkw3}, year = {2021}, date = {2021-01-01}, booktitle = {International Conference on Learning Representations}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
| 1037. | Luo, Xiangzhong; Liu, Di; Huai, Shuo; Liu, Weichen HSCoNAS: Hardware-Software Co-Design of Efficient DNNs via Neural Architecture Search Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-08325, title = {HSCoNAS: Hardware-Software Co-Design of Efficient DNNs via Neural Architecture Search}, author = {Xiangzhong Luo and Di Liu and Shuo Huai and Weichen Liu}, url = {https://arxiv.org/abs/2103.08325}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.08325}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1036. | Wu, Xuan; Zhang, Xiuyi; Jia, Linhan; Chen, Liang; Liang, Yanchun; Zhou, You; Wu, Chunguo Neural Architecture Search based on Cartesian Genetic Programming Coding Method Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-07173, title = {Neural Architecture Search based on Cartesian Genetic Programming Coding Method}, author = {Xuan Wu and Xiuyi Zhang and Linhan Jia and Liang Chen and Yanchun Liang and You Zhou and Chunguo Wu}, url = {https://arxiv.org/abs/2103.07173}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.07173}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1035. | Gracheva, Ekaterina Trainless Model Performance Estimation for Neural Architecture Search Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-08312, title = {Trainless Model Performance Estimation for Neural Architecture Search}, author = {Ekaterina Gracheva}, url = {https://arxiv.org/abs/2103.08312}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.08312}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1034. | Choi, Kwanghee; Choe, Minyoung; Lee, Hyelee Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-08157, title = {Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning}, author = {Kwanghee Choi and Minyoung Choe and Hyelee Lee}, url = {https://arxiv.org/abs/2103.08157}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.08157}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1033. | Ban, Hao; Xie, Pengtao Interleaving Learning, with Application to Neural Architecture Search Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-07018, title = {Interleaving Learning, with Application to Neural Architecture Search}, author = {Hao Ban and Pengtao Xie}, url = {https://arxiv.org/abs/2103.07018}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.07018}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1032. | Liu, Xiaobo; Zhang, Chaochao; Cai, Zhihua; Yang, Jianfeng; Zhou, Zhilang; Gong, Xin Continuous Particle Swarm Optimization-Based Deep Learning Architecture Search for Hyperspectral Image Classification Journal Article Remote Sensing, 13 (6), 2021, ISSN: 2072-4292. @article{rs13061082, title = {Continuous Particle Swarm Optimization-Based Deep Learning Architecture Search for Hyperspectral Image Classification}, author = {Xiaobo Liu and Chaochao Zhang and Zhihua Cai and Jianfeng Yang and Zhilang Zhou and Xin Gong}, url = {https://www.mdpi.com/2072-4292/13/6/1082}, doi = {10.3390/rs13061082}, issn = {2072-4292}, year = {2021}, date = {2021-01-01}, journal = {Remote Sensing}, volume = {13}, number = {6}, abstract = {Deep convolutional neural networks (CNNs) are widely used in hyperspectral image (HSI) classification. However, the most successful CNN architectures are handcrafted, which need professional knowledge and consume a very significant amount of time. To automatically design cell-based CNN architectures for HSI classification, we propose an efficient continuous evolutionary method, named CPSO-Net, which can dramatically accelerate optimal architecture generation by the optimization of weight-sharing parameters. First, a SuperNet with all candidate operations is maintained to share the parameters for all individuals and optimized by collecting the gradients of all individuals in the population. Second, a novel direct encoding strategy is devised to encode architectures into particles, which inherit the parameters from the SuperNet. Then, particle swarm optimization is used to search for the optimal deep architecture from the particle swarm. Furthermore, experiments with limited training samples based on four widely used biased and unbiased hyperspectral datasets showed that our proposed method achieves good performance comparable to the state-of-the-art HSI classification methods.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Deep convolutional neural networks (CNNs) are widely used in hyperspectral image (HSI) classification. However, the most successful CNN architectures are handcrafted, which need professional knowledge and consume a very significant amount of time. To automatically design cell-based CNN architectures for HSI classification, we propose an efficient continuous evolutionary method, named CPSO-Net, which can dramatically accelerate optimal architecture generation by the optimization of weight-sharing parameters. First, a SuperNet with all candidate operations is maintained to share the parameters for all individuals and optimized by collecting the gradients of all individuals in the population. Second, a novel direct encoding strategy is devised to encode architectures into particles, which inherit the parameters from the SuperNet. Then, particle swarm optimization is used to search for the optimal deep architecture from the particle swarm. Furthermore, experiments with limited training samples based on four widely used biased and unbiased hyperspectral datasets showed that our proposed method achieves good performance comparable to the state-of-the-art HSI classification methods. |
| 1031. | Zhang, Shi-Xin; Hsieh, Chang-Yu; Zhang, Shengyu; Yao, Hong Neural Predictor based Quantum Architecture Search Technical Report 2021. @techreport{zhang2021neural, title = {Neural Predictor based Quantum Architecture Search}, author = {Shi-Xin Zhang and Chang-Yu Hsieh and Shengyu Zhang and Hong Yao}, url = {https://arxiv.org/abs/2103.06524}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1030. | Cao, Shengcao; Wang, Xiaofang; Kitani, Kris Efficient Model Performance Estimation via Feature Histories Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-04450, title = {Efficient Model Performance Estimation via Feature Histories}, author = {Shengcao Cao and Xiaofang Wang and Kris Kitani}, url = {https://arxiv.org/abs/2103.04450}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.04450}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1029. | Liang, Tingting; Wang, Yongtao; Tang, Zhi; Hu, Guosheng; Ling, Haibin OPANAS: One-Shot Path Aggregation Network Architecture Search for Object Detection Technical Report 2021. @techreport{liang2021opanas, title = {OPANAS: One-Shot Path Aggregation Network Architecture Search for Object Detection}, author = {Tingting Liang and Yongtao Wang and Zhi Tang and Guosheng Hu and Haibin Ling}, url = {https://arxiv.org/abs/2103.04507}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1028. | Zhang, Kaiyu; Chen, Jinglong; He, Shuilong; Xu, Enyong; Li, Fudong; Zhou, Zitong Mechanical Systems and Signal Processing, 158 , pp. 107773, 2021, ISSN: 0888-3270. @article{ZHANG2021107773, title = {Differentiable neural architecture search augmented with pruning and multi-objective optimization for time-efficient intelligent fault diagnosis of machinery}, author = {Kaiyu Zhang and Jinglong Chen and Shuilong He and Enyong Xu and Fudong Li and Zitong Zhou}, url = {https://www.sciencedirect.com/science/article/pii/S0888327021001680}, doi = {https://doi.org/10.1016/j.ymssp.2021.107773}, issn = {0888-3270}, year = {2021}, date = {2021-01-01}, journal = {Mechanical Systems and Signal Processing}, volume = {158}, pages = {107773}, abstract = {Intelligent fault diagnosis, which is mainly based on neural network, has been widely used in machinery monitoring. Although such deep learning methods are effective, the new architectures are mainly handcrafted by series of experiments that require ample time and substantial efforts. To automate process of building neural networks and save designing time, a novel differentiable neural architecture search method is proposed. By gradually reducing candidate operations while retaining trained parameters during pruning, computation consumed by each stage of neural architecture search is decreased, which accelerates search process. To improve inferential efficiency of subnetworks, specially designed penalty terms are introduced into the objective function for searching optimal numbers of layers and nodes, which can reduce complexity of subnetworks and save calculation time of signal analysis. In addition, exclusive competition between candidate operations is broken by changing discretization and selection methods of operations, which provides a basis for channel fusion. Effectiveness of the proposed method is verified by two datasets. Experiments show that this method can generate subnetworks of lower complexity and less computational cost than other state-of-art neural architecture search techniques, while achieving competitive result.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Intelligent fault diagnosis, which is mainly based on neural network, has been widely used in machinery monitoring. Although such deep learning methods are effective, the new architectures are mainly handcrafted by series of experiments that require ample time and substantial efforts. To automate process of building neural networks and save designing time, a novel differentiable neural architecture search method is proposed. By gradually reducing candidate operations while retaining trained parameters during pruning, computation consumed by each stage of neural architecture search is decreased, which accelerates search process. To improve inferential efficiency of subnetworks, specially designed penalty terms are introduced into the objective function for searching optimal numbers of layers and nodes, which can reduce complexity of subnetworks and save calculation time of signal analysis. In addition, exclusive competition between candidate operations is broken by changing discretization and selection methods of operations, which provides a basis for channel fusion. Effectiveness of the proposed method is verified by two datasets. Experiments show that this method can generate subnetworks of lower complexity and less computational cost than other state-of-art neural architecture search techniques, while achieving competitive result. |
| 1027. | Chen, Yaofo; Guo, Yong; Chen, Qi; Li, Minli; Wang, Yaowei; Zeng, Wei; Tan, Mingkui Contrastive Neural Architecture Search with Neural Architecture Comparators Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2103-05471, title = {Contrastive Neural Architecture Search with Neural Architecture Comparators}, author = {Yaofo Chen and Yong Guo and Qi Chen and Minli Li and Yaowei Wang and Wei Zeng and Mingkui Tan}, url = {https://arxiv.org/abs/2103.05471}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2103.05471}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1026. | Javeri, Indrajeet Y; Toutiaee, Mohammadhossein; Arpinar, Ismailcem B; Miller, Tom W; Miller, John A Improving Neural Networks for Time Series Forecasting using Data Augmentation and AutoML Technical Report 2021. @techreport{javeri2021improving, title = {Improving Neural Networks for Time Series Forecasting using Data Augmentation and AutoML}, author = {Indrajeet Y Javeri and Mohammadhossein Toutiaee and Ismailcem B Arpinar and Tom W Miller and John A Miller}, url = {https://arxiv.org/abs/2103.01992}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1025. | Guo, Qingbei; Wu, Xiao-Jun; Kittler, Josef; Feng, Zhiquan Differentiable Neural Architecture Learning for Efficient Neural Network Design Technical Report 2021. @techreport{guo2021differentiable, title = {Differentiable Neural Architecture Learning for Efficient Neural Network Design}, author = {Qingbei Guo and Xiao-Jun Wu and Josef Kittler and Zhiquan Feng}, url = {https://arxiv.org/abs/2103.02126}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1024. | Liu, Luyan; Wen, Zhiwei; Liu, Songwei; Zhou, Hong-Yu; Zhu, Hongwei; Xie, Weicheng; Shen, Linlin; Ma, Kai; Zheng, Yefeng MixSearch: Searching for Domain Generalized Medical Image Segmentation Architectures Technical Report 2021. @techreport{liu2021mixsearch, title = {MixSearch: Searching for Domain Generalized Medical Image Segmentation Architectures}, author = {Luyan Liu and Zhiwei Wen and Songwei Liu and Hong-Yu Zhou and Hongwei Zhu and Weicheng Xie and Linlin Shen and Kai Ma and Yefeng Zheng}, url = {https://arxiv.org/abs/2102.13280}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1023. | Zhang, Ziwei; Wang, Xin; Zhu, Wenwu Automated Machine Learning on Graphs: A Survey Technical Report 2021. @techreport{zhang2021automated, title = {Automated Machine Learning on Graphs: A Survey}, author = {Ziwei Zhang and Xin Wang and Wenwu Zhu}, url = {https://arxiv.org/abs/2103.00742}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1022. | Jeong, Wonyong; Lee, Hayeon; Park, Gun; Hyung, Eunyoung; Baek, Jinheon; Hwang, Sung Ju Task-Adaptive Neural Network Retrieval with Meta-Contrastive Learning Technical Report 2021. @techreport{jeong2021taskadaptive, title = {Task-Adaptive Neural Network Retrieval with Meta-Contrastive Learning}, author = {Wonyong Jeong and Hayeon Lee and Gun Park and Eunyoung Hyung and Jinheon Baek and Sung Ju Hwang}, url = {https://arxiv.org/abs/2103.01495}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1021. | Guo, Yong; Chen, Yaofo; Zheng, Yin; Chen, Qi; Zhao, Peilin; Chen, Jian; Huang, Junzhou; Tan, Mingkui Pareto-Frontier-aware Neural Architecture Generation for Diverse Budgets Technical Report 2021. @techreport{guo2021paretofrontieraware, title = {Pareto-Frontier-aware Neural Architecture Generation for Diverse Budgets}, author = {Yong Guo and Yaofo Chen and Yin Zheng and Qi Chen and Peilin Zhao and Jian Chen and Junzhou Huang and Mingkui Tan}, url = {https://arxiv.org/abs/2103.00219}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1020. | Le, Cat P; Soltani, Mohammadreza; Ravier, Robert; Tarokh, Vahid Neural Architecture Search From Task Similarity Measure Technical Report 2021. @techreport{le2021neural, title = {Neural Architecture Search From Task Similarity Measure}, author = {Cat P Le and Mohammadreza Soltani and Robert Ravier and Vahid Tarokh}, url = {https://arxiv.org/abs/2103.00241}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1019. | Xie, Guoyang; Wang, Jinbao; Yu, Guo; Zheng, Feng; Jin, Yaochu Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search Technical Report 2021. @techreport{xie2021tiny, title = {Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search}, author = {Guoyang Xie and Jinbao Wang and Guo Yu and Feng Zheng and Yaochu Jin}, url = {https://arxiv.org/abs/2103.00363}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1018. | Zhang, T; Lei, C; Zhang, Z; Meng, X -B; Chen, C L P AS-NAS: Adaptive Scalable Neural Architecture Search with Reinforced Evolutionary Algorithm for Deep Learning Journal Article IEEE Transactions on Evolutionary Computation, pp. 1-1, 2021. @article{9360872, title = {AS-NAS: Adaptive Scalable Neural Architecture Search with Reinforced Evolutionary Algorithm for Deep Learning}, author = {T Zhang and C Lei and Z Zhang and X -B Meng and C L P Chen}, url = {https://ieeexplore.ieee.org/abstract/document/9360872}, doi = {10.1109/TEVC.2021.3061466}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Evolutionary Computation}, pages = {1-1}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
| 1017. | Chen, J; Jiang, Y; Huang, Z; Guo, X; Wu, B; Sun, L; Wu, T Fine-Grained Detection of Driver Distraction Based on Neural Architecture Search Journal Article IEEE Transactions on Intelligent Transportation Systems, pp. 1-19, 2021. @article{9352235, title = {Fine-Grained Detection of Driver Distraction Based on Neural Architecture Search}, author = {J Chen and Y Jiang and Z Huang and X Guo and B Wu and L Sun and T Wu}, url = {https://ieeexplore.ieee.org/abstract/document/9352235}, doi = {10.1109/TITS.2021.3055545}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Intelligent Transportation Systems}, pages = {1-19}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
| 1016. | Zhang, Zijun; Cofer, Evan M; Troyanskaya, Olga G AMBIENT: Accelerated Convolutional Neural Network Architecture Search for Regulatory Genomics Inproceedings Machine Learning in Computational Biology (MLCB 2020), 2021. @inproceedings{ZhangMLCB2020, title = {AMBIENT: Accelerated Convolutional Neural Network Architecture Search for Regulatory Genomics}, author = {Zijun Zhang and Evan M. Cofer and Olga G. Troyanskaya}, url = {https://www.biorxiv.org/content/biorxiv/early/2021/02/27/2021.02.25.432960.full.pdf}, year = {2021}, date = {2021-01-01}, booktitle = {Machine Learning in Computational Biology (MLCB 2020)}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
| 1015. | Wu, Junru; Dai, Xiyang; Chen, Dongdong; Chen, Yinpeng; Liu, Mengchen; Yu, Ye; Wang, Zhangyang; Liu, Zicheng; Chen, Mei; Yuan, Lu Weak NAS Predictors Are All You Need Technical Report 2021. @techreport{wu2021weak, title = {Weak NAS Predictors Are All You Need}, author = {Junru Wu and Xiyang Dai and Dongdong Chen and Yinpeng Chen and Mengchen Liu and Ye Yu and Zhangyang Wang and Zicheng Liu and Mei Chen and Lu Yuan}, url = {https://arxiv.org/abs/2102.10490}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1014. | Guo, Yong; Zheng, Yin; Tan, Mingkui; Chen, Qi; Li, Zhipeng; Chen, Jian; Zhao, Peilin; Huang, Junzhou Towards Accurate and Compact Architectures via Neural Architecture Transformer Technical Report 2021. @techreport{guo2021accurate, title = {Towards Accurate and Compact Architectures via Neural Architecture Transformer}, author = {Yong Guo and Yin Zheng and Mingkui Tan and Qi Chen and Zhipeng Li and Jian Chen and Peilin Zhao and Junzhou Huang}, url = {https://arxiv.org/abs/2102.10301}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1013. | Nguyen, Nam; Chang, Morris J Contrastive Self-supervised Neural Architecture Search Miscellaneous 2021. @misc{nguyen2021contrastive, title = {Contrastive Self-supervised Neural Architecture Search}, author = {Nam Nguyen and Morris J Chang}, url = {https://arxiv.org/abs/2102.10557}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {misc} } |
| 1012. | Nayman, Niv; Aflalo, Yonathan; Noy, Asaf; Zelnik-Manor, Lihi HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search Technical Report 2021. @techreport{nayman2021hardcore, title = {HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search}, author = {Niv Nayman and Yonathan Aflalo and Asaf Noy and Lihi Zelnik-Manor}, url = {https://arxiv.org/abs/2102.11646}, year = {2021}, date = {2021-01-01}, journal = {arXiv preprint arXiv:2102.11646}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1011. | Hu, Liangxiao; Liu, Qinglin; Zhang, Jun; Jiang, Feng; Liu, Yang; Zhang, Shengping A-DARTS: attention-guided differentiable architecture search for lung nodule classification Journal Article Journal of Electronic Imaging, 30 (1), pp. 1 – 11, 2021. @article{10.1117/1.JEI.30.1.013012, title = {A-DARTS: attention-guided differentiable architecture search for lung nodule classification}, author = {Liangxiao Hu and Qinglin Liu and Jun Zhang and Feng Jiang and Yang Liu and Shengping Zhang}, url = {https://doi.org/10.1117/1.JEI.30.1.013012}, doi = {10.1117/1.JEI.30.1.013012}, year = {2021}, date = {2021-01-01}, journal = {Journal of Electronic Imaging}, volume = {30}, number = {1}, pages = {1 -- 11}, publisher = {SPIE}, abstract = {Lung cancer has caused the most cancer deaths in the past several years. Benign–malignant lung nodule classification is vital in lung nodule detection, which can help early diagnosis of lung cancer. Most existing works extract the features of chest CT images using the well-designed networks, which require substantial effort of experts. To automate the manual process of network design, we propose an attention-guided differentiable architecture search (A-DARTS) method, which directly searches for the optimal network on chest CT images. In addition, A-DARTS utilizes an attention mechanism to alleviate the effect of the initialization-sensitive nature of the searched network while enhancing the feature presentation ability. Extensive experiments on the Lung Image Database Consortium image collection (LIDC-IDRI) benchmark dataset show that the proposed method achieves a lung nodule classification accuracy of 92.93%, which is superior to the state-of-the-art methods.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Lung cancer has caused the most cancer deaths in the past several years. Benign–malignant lung nodule classification is vital in lung nodule detection, which can help early diagnosis of lung cancer. Most existing works extract the features of chest CT images using the well-designed networks, which require substantial effort of experts. To automate the manual process of network design, we propose an attention-guided differentiable architecture search (A-DARTS) method, which directly searches for the optimal network on chest CT images. In addition, A-DARTS utilizes an attention mechanism to alleviate the effect of the initialization-sensitive nature of the searched network while enhancing the feature presentation ability. Extensive experiments on the Lung Image Database Consortium image collection (LIDC-IDRI) benchmark dataset show that the proposed method achieves a lung nodule classification accuracy of 92.93%, which is superior to the state-of-the-art methods. |
| 1010. | Zhou, Yanqi; Dong, Xuanyi; Akin, Berkin; Tan, Mingxing; Peng, Daiyi; Meng, Tianjian; Yazdanbakhsh, Amir; Huang, Da; Narayanaswami, Ravi; Laudon, James Rethinking Co-design of Neural Architectures and Hardware Accelerators Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2102-08619, title = {Rethinking Co-design of Neural Architectures and Hardware Accelerators}, author = {Yanqi Zhou and Xuanyi Dong and Berkin Akin and Mingxing Tan and Daiyi Peng and Tianjian Meng and Amir Yazdanbakhsh and Da Huang and Ravi Narayanaswami and James Laudon}, url = {https://arxiv.org/abs/2102.08619}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2102.08619}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1009. | Ru, Binxin; Wan, Xingchen; Dong, Xiaowen; Osborne, Michael Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels Inproceedings International Conference on Learning Representations, 2021. @inproceedings{<LineBreak>ru2021interpretable, title = {Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels}, author = {Binxin Ru and Xingchen Wan and Xiaowen Dong and Michael Osborne}, url = {https://openreview.net/forum?id=j9Rv7qdXjd}, year = {2021}, date = {2021-01-01}, booktitle = {International Conference on Learning Representations}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
| 1008. | Roberts, Nicholas Carl; Khodak, Mikhail; Dao, Tri; Li, Liam; Balcan, Nina; Re, Christopher; Talwalkar, Ameet Searching for Convolutions and a More Ambitious NAS Miscellaneous 2021. @misc{<LineBreak>roberts2021searching, title = {Searching for Convolutions and a More Ambitious NAS}, author = {Nicholas Carl Roberts and Mikhail Khodak and Tri Dao and Liam Li and Nina Balcan and Christopher Re and Ameet Talwalkar}, url = {https://openreview.net/forum?id=ascdLuNQY4J}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {misc} } |
| 1007. | Wang, Dilin; Gong, Chengyue; Li, Meng; Liu, Qiang; Chandra, Vikas AlphaNet: Improved Training of Supernet with Alpha-Divergence Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2102-07954, title = {AlphaNet: Improved Training of Supernet with Alpha-Divergence}, author = {Dilin Wang and Chengyue Gong and Meng Li and Qiang Liu and Vikas Chandra}, url = {https://arxiv.org/abs/2102.07954}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2102.07954}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1006. | Robbiano, Luca; Rahman, Muhammad Rameez Ur; Galasso, Fabio; Caputo, Barbara; Carlucci, Fabio Maria Adversarial Branch Architecture Search for Unsupervised Domain Adaptation Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2102-06679, title = {Adversarial Branch Architecture Search for Unsupervised Domain Adaptation}, author = {Luca Robbiano and Muhammad Rameez Ur Rahman and Fabio Galasso and Barbara Caputo and Fabio Maria Carlucci}, url = {https://arxiv.org/abs/2102.06679}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2102.06679}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1005. | Calisto, Maria Baldeon G; Lai-Yuen, Susana K EMONAS: efficient multiobjective neural architecture search framework for 3D medical image segmentation Inproceedings Išgum, Ivana; Landman, Bennett A (Ed.): Medical Imaging 2021: Image Processing, pp. 22 – 34, International Society for Optics and Photonics SPIE, 2021. @inproceedings{10.1117/12.2577088, title = {EMONAS: efficient multiobjective neural architecture search framework for 3D medical image segmentation}, author = {Maria Baldeon G Calisto and Susana K Lai-Yuen}, editor = {Ivana Išgum and Bennett A Landman}, url = {https://doi.org/10.1117/12.2577088}, doi = {10.1117/12.2577088}, year = {2021}, date = {2021-01-01}, booktitle = {Medical Imaging 2021: Image Processing}, volume = {11596}, pages = {22 -- 34}, publisher = {SPIE}, organization = {International Society for Optics and Photonics}, abstract = {Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the macro- or micro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, and do not consider the volumetric nature of medical images. In this work, we propose EMONAS, an Efficient MultiObjective Neural Architecture Search framework for 3D medical image segmentation. EMONAS is composed of a search space that considers both the macro- and micro-structure of the architecture, and a surrogate-assisted multiobjective evolutionary based algorithm that efficiently searches for the best hyperparameters using a Random Forest surrogate and guiding selection probabilities. EMONAS is evaluated on the task of cardiac segmentation from the ACDC MICCAI challenge. The architecture found is ranked within the top 10 submissions in all evaluation metrics, performing better or comparable to other approaches while reducing the search time by more than 50% and having considerably fewer number of parameters.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the macro- or micro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, and do not consider the volumetric nature of medical images. In this work, we propose EMONAS, an Efficient MultiObjective Neural Architecture Search framework for 3D medical image segmentation. EMONAS is composed of a search space that considers both the macro- and micro-structure of the architecture, and a surrogate-assisted multiobjective evolutionary based algorithm that efficiently searches for the best hyperparameters using a Random Forest surrogate and guiding selection probabilities. EMONAS is evaluated on the task of cardiac segmentation from the ACDC MICCAI challenge. The architecture found is ranked within the top 10 submissions in all evaluation metrics, performing better or comparable to other approaches while reducing the search time by more than 50% and having considerably fewer number of parameters. |
| 1004. | Yan, Shen; Song, Kaiqiang; Liu, Fei; Zhang, Mi CATE: Computation-aware Neural Architecture Encoding with Transformers Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2102-07108, title = {CATE: Computation-aware Neural Architecture Encoding with Transformers}, author = {Shen Yan and Kaiqiang Song and Fei Liu and Mi Zhang}, url = {https://arxiv.org/abs/2102.07108}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2102.07108}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1003. | Lopes, Vasco; Alirezazadeh, Saeid; í, Lu EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2102-08099, title = {EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search}, author = {Vasco Lopes and Saeid Alirezazadeh and Lu í}, url = {https://arxiv.org/abs/2102.08099}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2102.08099}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1002. | Turner, Jack; Crowley, Elliot J; O'Boyle, Michael F P Neural Architecture Search as Program Transformation Exploration Technical Report , 2021. @techreport{DBLP:journals/corr/abs-2102-06599, title = {Neural Architecture Search as Program Transformation Exploration}, author = {Jack Turner and Elliot J Crowley and Michael F P O'Boyle}, url = {https://arxiv.org/abs/2102.06599}, year = {2021}, date = {2021-01-01}, journal = {CoRR}, volume = {abs/2102.06599}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 1001. | Sun, Y; Sun, X; Fang, Y; Yen, G G; Liu, Y A Novel Training Protocol for Performance Predictors of Evolutionary Neural Architecture Search Algorithms Journal Article IEEE Transactions on Evolutionary Computation, pp. 1-1, 2021. @article{9336721, title = {A Novel Training Protocol for Performance Predictors of Evolutionary Neural Architecture Search Algorithms}, author = {Y Sun and X Sun and Y Fang and G G Yen and Y Liu}, url = {https://ieeexplore.ieee.org/document/9336721}, doi = {10.1109/TEVC.2021.3055076}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Evolutionary Computation}, pages = {1-1}, abstract = {Evolutionary Neural Architecture Search (ENAS) can automatically design the architectures of Deep Neural Networks (DNNs) using evolutionary computation algorithms. However, most ENAS algorithms require intensive computational resource, which is not necessarily available to the users interested. Performance predictors are a type of regression models which can assist to accomplish the search, while without exerting much computational resource. Despite various performance predictors have been designed, they employ the same training protocol to build the regression models: 1) sampling a set of DNNs with performance as the training dataset, 2) training the model with the mean square error criterion, and 3) predicting the performance of DNNs newly generated during the ENAS. In this paper, we point out that the three steps constituting the training protocol are not well though-out through intuitive and illustrative examples. Furthermore, we propose a new training protocol to address these issues, consisting of designing a pairwise ranking indicator to construct the training target, proposing to use the logistic regression to fit the training samples, and developing a differential method to build the training instances. To verify the effectiveness of the proposed training protocol, four widely used regression models in the field of machine learning have been chosen to perform the comparisons on two benchmark datasets. The experimental results of all the comparisons demonstrate that the proposed training protocol can significantly improve the performance prediction accuracy against the traditional training protocols.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Evolutionary Neural Architecture Search (ENAS) can automatically design the architectures of Deep Neural Networks (DNNs) using evolutionary computation algorithms. However, most ENAS algorithms require intensive computational resource, which is not necessarily available to the users interested. Performance predictors are a type of regression models which can assist to accomplish the search, while without exerting much computational resource. Despite various performance predictors have been designed, they employ the same training protocol to build the regression models: 1) sampling a set of DNNs with performance as the training dataset, 2) training the model with the mean square error criterion, and 3) predicting the performance of DNNs newly generated during the ENAS. In this paper, we point out that the three steps constituting the training protocol are not well though-out through intuitive and illustrative examples. Furthermore, we propose a new training protocol to address these issues, consisting of designing a pairwise ranking indicator to construct the training target, proposing to use the logistic regression to fit the training samples, and developing a differential method to build the training instances. To verify the effectiveness of the proposed training protocol, four widely used regression models in the field of machine learning have been chosen to perform the comparisons on two benchmark datasets. The experimental results of all the comparisons demonstrate that the proposed training protocol can significantly improve the performance prediction accuracy against the traditional training protocols. |
| 1000. | Su, Xiu; You, Shan; Huang, Tao; Wang, Fei; Qian, Chen; Zhang, Changshui; Xu, Chang Locally Free Weight Sharing for Network Width Search Technical Report 2021. @techreport{su2021locally, title = {Locally Free Weight Sharing for Network Width Search}, author = {Xiu Su and Shan You and Tao Huang and Fei Wang and Chen Qian and Changshui Zhang and Chang Xu}, url = {https://arxiv.org/abs/2102.05258}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 999. | Liu, Peidong; Zhang, Gengwei; Wang, Bochao; Xu, Hang; Liang, Xiaodan; Jiang, Yong; Li, Zhenguo Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search Technical Report 2021. @techreport{liu2021loss, title = {Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search}, author = {Peidong Liu and Gengwei Zhang and Bochao Wang and Hang Xu and Xiaodan Liang and Yong Jiang and Zhenguo Li}, url = {https://arxiv.org/abs/2102.04700}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
| 998. | He, Xin; Zhao, Kaiyong; Chu, Xiaowen AutoML: A Survey of the State-of-the-Art Journal Article Knowledge-Based Systems, 212 , pp. 106622, 2021. @article{he2021automl, title = {AutoML: A Survey of the State-of-the-Art}, author = {Xin He and Kaiyong Zhao and Xiaowen Chu}, url = {https://www.sciencedirect.com/science/article/abs/pii/S0950705120307516}, year = {2021}, date = {2021-01-01}, journal = {Knowledge-Based Systems}, volume = {212}, pages = {106622}, publisher = {Elsevier}, abstract = {Deep learning (DL) techniques have obtained remarkable achievements on various tasks, such as image recognition, object detection, and language modeling. However, building a high-quality DL system for a specific task highly relies on human expertise, hindering its wide application. Meanwhile, automated machine learning (AutoML) is a promising solution for building a DL system without human assistance and is being extensively studied. This paper presents a comprehensive and up-to-date review of the state-of-the-art (SOTA) in AutoML. According to the DL pipeline, we introduce AutoML methods – covering data preparation, feature engineering, hyperparameter optimization, and neural architecture search (NAS) – with a particular focus on NAS, as it is currently a hot sub-topic of AutoML. We summarize the representative NAS algorithms’ performance on the CIFAR-10 and ImageNet datasets and further discuss the following subjects of NAS methods: one/two-stage NAS, one-shot NAS, joint hyperparameter and architecture optimization, and resource-aware NAS. Finally, we discuss some open problems related to the existing AutoML methods for future research.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Deep learning (DL) techniques have obtained remarkable achievements on various tasks, such as image recognition, object detection, and language modeling. However, building a high-quality DL system for a specific task highly relies on human expertise, hindering its wide application. Meanwhile, automated machine learning (AutoML) is a promising solution for building a DL system without human assistance and is being extensively studied. This paper presents a comprehensive and up-to-date review of the state-of-the-art (SOTA) in AutoML. According to the DL pipeline, we introduce AutoML methods – covering data preparation, feature engineering, hyperparameter optimization, and neural architecture search (NAS) – with a particular focus on NAS, as it is currently a hot sub-topic of AutoML. We summarize the representative NAS algorithms’ performance on the CIFAR-10 and ImageNet datasets and further discuss the following subjects of NAS methods: one/two-stage NAS, one-shot NAS, joint hyperparameter and architecture optimization, and resource-aware NAS. Finally, we discuss some open problems related to the existing AutoML methods for future research. |
| 997. | Li, Sheng; Tan, Mingxing; Pang, Ruoming; Li, Andrew; Cheng, Liqun; Le, Quoc; Jouppi, Norman P Searching for Fast Model Families on Datacenter Accelerators Technical Report 2021. @techreport{li2021searching, title = {Searching for Fast Model Families on Datacenter Accelerators}, author = {Sheng Li and Mingxing Tan and Ruoming Pang and Andrew Li and Liqun Cheng and Quoc Le and Norman P Jouppi}, url = {https://arxiv.org/abs/2102.05610}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |