Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2022
1370.
Fei, Ke; Li, Qi; Zhu, Congcong
Non-technical losses detection using missing values’ pattern and neural architecture search Journal Article
In: International Journal of Electrical Power & Energy Systems, 134 , pp. 107410, 2022, ISSN: 0142-0615.
@article{FEI2022107410,
title = {Non-technical losses detection using missing values’ pattern and neural architecture search},
author = {Ke Fei and Qi Li and Congcong Zhu},
url = {https://www.sciencedirect.com/science/article/pii/S0142061521006499},
doi = {https://doi.org/10.1016/j.ijepes.2021.107410},
issn = {0142-0615},
year = {2022},
date = {2022-01-01},
journal = {International Journal of Electrical Power & Energy Systems},
volume = {134},
pages = {107410},
abstract = {The fast growth of non-technical loss (NTL) has gradually become one of the main concerns for distribution network operators (DNOs). Electricity theft which constitutes the main part of NTL not only brings losses to the DNOs, but also reduces the quality of the supply. A traditional detection method relies on utility workers’ experience and consumes a large amount of manpower. Thanks to the emerging of advanced metering infrastructure (AMI), utility companies can now collect detailed data reflecting consumers’ electricity usage, which enabled algorithms-based non-technical loss detection. The current data-based methods focus on the characteristics of electricity consumption thereby less efficient when dealing with rapidly changed electricity theft techniques. This article introduced a new data set, the location information of missing values, to improve the accuracy of non-technical loss detection. The relationship between missing values and electricity theft techniques is analyzed and a neural network model is built through neural architecture search (NAS). The improved model achieved an excellent Area Under Curve (AUC) value around 0.926 which verified the close link between missing values and electricity theft techniques. The nature of neural architecture search allows automatic model update which makes it a user-friendly tool even for engineers without any neural network expertise. A case study was carried out in which the missing value pattern was analyzed through Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering algorithm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The fast growth of non-technical loss (NTL) has gradually become one of the main concerns for distribution network operators (DNOs). Electricity theft which constitutes the main part of NTL not only brings losses to the DNOs, but also reduces the quality of the supply. A traditional detection method relies on utility workers’ experience and consumes a large amount of manpower. Thanks to the emerging of advanced metering infrastructure (AMI), utility companies can now collect detailed data reflecting consumers’ electricity usage, which enabled algorithms-based non-technical loss detection. The current data-based methods focus on the characteristics of electricity consumption thereby less efficient when dealing with rapidly changed electricity theft techniques. This article introduced a new data set, the location information of missing values, to improve the accuracy of non-technical loss detection. The relationship between missing values and electricity theft techniques is analyzed and a neural network model is built through neural architecture search (NAS). The improved model achieved an excellent Area Under Curve (AUC) value around 0.926 which verified the close link between missing values and electricity theft techniques. The nature of neural architecture search allows automatic model update which makes it a user-friendly tool even for engineers without any neural network expertise. A case study was carried out in which the missing value pattern was analyzed through Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering algorithm.
2021
1369.
Yan, Kefan; Wang, Shunlong; Guan, Yuwei; Jiang, Ailian
TNAS: Neural Network Architecture Search Sampling Module Based on Variational Autoencoder Journal Article
In: CONVERTER, pp. 288-296, 2021.
@article{Yan2021,
title = { TNAS: Neural Network Architecture Search Sampling Module Based on Variational Autoencoder},
author = {Kefan Yan and Shunlong Wang and Yuwei Guan and Ailian Jiang},
url = {http://converter-magazine.info/index.php/converter/article/view/499},
year = {2021},
date = {2021-09-07},
urldate = {2021-09-07},
journal = {CONVERTER},
pages = {288-296},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1368.
Zhao, Zimin; Kang, Ying; Hou, Aiqin; Gan, Daguang
SP-DARTS: Synchronous Progressive Differentiable Neural Architecture Search for Image Classification Journal Article
In: IEICE TRANSACTIONS on Information and Systems , E104-D (8 ), pp. 1232-1238, 2021, ISBN: 1745-1361.
@article{Zhao2021b,
title = { SP-DARTS: Synchronous Progressive Differentiable Neural Architecture Search for Image Classification},
author = {Zimin Zhao and Ying Kang and Aiqin Hou and Daguang Gan},
url = {https://search.ieice.org/bin/summary.php?id=e104-d_8_1232},
doi = {10.1587/transinf.2020BDP0009},
isbn = {1745-1361},
year = {2021},
date = {2021-08-01},
journal = {IEICE TRANSACTIONS on Information and Systems },
volume = {E104-D },
number = {8 },
pages = {1232-1238},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1367.
Wendlinger, Lorenz; Berndl, Emanuel; Granitzer, Michael
Methods for Automatic Machine-Learning Workflow Analysis Inproceedings
In: ECML PKDD 2021, 2021.
@inproceedings{WEndlingerECML2021,
title = {Methods for Automatic Machine-Learning Workflow Analysis},
author = {Lorenz Wendlinger and Emanuel Berndl and Michael Granitzer},
url = {https://2021.ecmlpkdd.org/wp-content/uploads/2021/07/sub_473.pdf},
year = {2021},
date = {2021-08-01},
booktitle = {ECML PKDD 2021},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1366.
Zhu, Yanming; Meijering, Erik
Automatic Improvement of Deep Learning Based Cell Segmentation in Time-Lapse Microscopy by Neural Architecture Search Journal Article
In: Bioinformatics, 2021.
@article{Zhu2021b,
title = {Automatic Improvement of Deep Learning Based Cell Segmentation in Time-Lapse Microscopy by Neural Architecture Search},
author = {Zhu, Yanming and Erik Meijering },
url = {https://pubmed.ncbi.nlm.nih.gov/34329376/},
doi = {10.1093/bioinformatics/btab556},
year = {2021},
date = {2021-07-30},
journal = {Bioinformatics},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1365.
Zhao, Jiejie; Lv, Weifeng; Du, Bowen; Ye, Junchen; Sun, Leilei; Xiong, Guixi
Deep multi-task learning with flexible and compact architecture search Journal Article
In: International Journal of Data Science and Analytics , 2021.
@article{ZhaoIJDSA2021,
title = {Deep multi-task learning with flexible and compact architecture search},
author = {Jiejie Zhao and Weifeng Lv and Bowen Du and Junchen Ye and Leilei Sun and Guixi Xiong },
url = {https://link.springer.com/article/10.1007/s41060-021-00274-0},
year = {2021},
date = {2021-07-24},
journal = {International Journal of Data Science and Analytics },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1364.
Ünal, Hamit Taner; Başçiftçi, Fatih
Evolutionary design of neural network architectures: a review of three decades of research Journal Article
In: Artificial Intelligence Review, 2021.
@article{Ünal2021,
title = {Evolutionary design of neural network architectures: a review of three decades of research},
author = {Hamit Taner Ünal and Fatih Başçiftçi },
url = {https://link.springer.com/article/10.1007/s10462-021-10049-5},
year = {2021},
date = {2021-07-17},
journal = {Artificial Intelligence Review},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1363.
Coquelin, Daniel; Sedona, Rocco; Riedel, Morris; Götz, Markus
Evolutionary Optimization of Neural Architectures in Remote Sensing Classification Problems Technical Report
2021.
@techreport{Coquelin2021,
title = {Evolutionary Optimization of Neural Architectures in Remote Sensing Classification Problems },
author = {Daniel Coquelin and Rocco Sedona and Morris Riedel and Markus Götz},
url = {https://www.researchgate.net/profile/Morris-Riedel/publication/353352507_EVOLUTIONARY_OPTIMIZATION_OF_NEURAL_ARCHITECTURES_IN_REMOTE_SENSING_CLASSIFICATION_PROBLEMS/links/60f7462b0c2bfa282aeef5cf/EVOLUTIONARY-OPTIMIZATION-OF-NEURAL-ARCHITECTURES-IN-REMOTE-SENSING-CLASSIFICATION-PROBLEMS.pdf},
year = {2021},
date = {2021-07-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1362.
Yu, Kaicheng
From Human-Designed Convolutional Neural Networks Towards Robust Neural Architecture Search PhD Thesis
2021.
@phdthesis{YuPhD2021,
title = {From Human-Designed Convolutional Neural Networks Towards Robust Neural Architecture Search },
author = {Yu, Kaicheng},
url = {https://infoscience.epfl.ch/record/287432},
doi = { 10.5075/epfl-thesis-8035},
year = {2021},
date = {2021-07-01},
abstract = {Artificial intelligence has been an ultimate design goal since the inception of computers decades ago. Among the many attempts towards general artificial intelligence, modern machine learning successfully tackles many complex problems thanks to the progress in deep learning, and in particular in convolutional neural networks (CNN). To design a CNN for a specific task, one common approach consists of adapting the heuristics from the pre-deep-learning era to the CNN domain. In the first part of this thesis, we introduce two methods that follow this approach: i) We build a covariance descriptor, i.e., a local descriptor that is suitable for texture recognition, to replace the first-order fully connected layers in an ordinary CNN, showing that such a descriptor yields state-of-the-art performance on many fine-grained image classification tasks with orders of magnitude fewer feature dimensions; ii) we develop a light-weight recurrent U-Net for image semantic segmentation, inspired by the biological eye saccadic movements, that yields real-time predictions on devices with limited computational resources. As most methods pre-dating automatic machine learning (AutoML), the two above-mentioned CNNs were human-designed. In the past few years, however, neural architecture search~(NAS), which aims to facilitate the design of deep networks for new tasks, has drawn an increasing attention. In this context, the weight-sharing approach, which consists of utilizing a super-net to encompass all possible architectures within a search space, has become a de facto standard in NAS because it enables the search to be done on commodity hardware. In the second part of this thesis, we then provide an in-depth study of recent weight-sharing NAS algorithms. First, we discover a phenomenon in the weight-sharing NAS training pipeline, which we dub multi-model forgetting, that negatively impacts the super-net quality, and propose a statistically motivated approach to address it. Subsequently, we find that (i) on average, many popular NAS algorithms perform similarly to a random architecture sampling policy; (ii) the widely-adopted weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. We then further decouple weight sharing from the NAS sampling policy, and isolate 14 factors that play a key role in the success of super-net training. Finally, to improve the super-net quality, we propose a regularization term that aims to maximize the correlation between the performance rankings of the super-net and of the stand-alone architectures using a small set of landmark architectures.},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Artificial intelligence has been an ultimate design goal since the inception of computers decades ago. Among the many attempts towards general artificial intelligence, modern machine learning successfully tackles many complex problems thanks to the progress in deep learning, and in particular in convolutional neural networks (CNN). To design a CNN for a specific task, one common approach consists of adapting the heuristics from the pre-deep-learning era to the CNN domain. In the first part of this thesis, we introduce two methods that follow this approach: i) We build a covariance descriptor, i.e., a local descriptor that is suitable for texture recognition, to replace the first-order fully connected layers in an ordinary CNN, showing that such a descriptor yields state-of-the-art performance on many fine-grained image classification tasks with orders of magnitude fewer feature dimensions; ii) we develop a light-weight recurrent U-Net for image semantic segmentation, inspired by the biological eye saccadic movements, that yields real-time predictions on devices with limited computational resources. As most methods pre-dating automatic machine learning (AutoML), the two above-mentioned CNNs were human-designed. In the past few years, however, neural architecture search~(NAS), which aims to facilitate the design of deep networks for new tasks, has drawn an increasing attention. In this context, the weight-sharing approach, which consists of utilizing a super-net to encompass all possible architectures within a search space, has become a de facto standard in NAS because it enables the search to be done on commodity hardware. In the second part of this thesis, we then provide an in-depth study of recent weight-sharing NAS algorithms. First, we discover a phenomenon in the weight-sharing NAS training pipeline, which we dub multi-model forgetting, that negatively impacts the super-net quality, and propose a statistically motivated approach to address it. Subsequently, we find that (i) on average, many popular NAS algorithms perform similarly to a random architecture sampling policy; (ii) the widely-adopted weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. We then further decouple weight sharing from the NAS sampling policy, and isolate 14 factors that play a key role in the success of super-net training. Finally, to improve the super-net quality, we propose a regularization term that aims to maximize the correlation between the performance rankings of the super-net and of the stand-alone architectures using a small set of landmark architectures.
1361.
Siems, Julien; Klein, Aaron; Archambeau, Cedric; Mahsereci, Maren
Dynamic Pruning of a Neural Networkvia Gradient Signal-to-Noise Ratio Inproceedings
In: 8th ICML Workshop on Automated Machine Learning (2021), 2021.
@inproceedings{Siems2021icml,
title = {Dynamic Pruning of a Neural Networkvia Gradient Signal-to-Noise Ratio},
author = {Julien Siems and Aaron Klein and Cedric Archambeau and Maren Mahsereci},
url = {https://openreview.net/pdf?id=34awaeWZgya},
year = {2021},
date = {2021-07-01},
booktitle = {8th ICML Workshop on Automated Machine Learning (2021)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1360.
Tian, Ye; Peng, Shichen; Yang, Shangshang; Zhang, Xingyi; Tan, Kay Chen; Jin, Yaochu
Action Command Encoding for SurrogateAssisted Neural Architecture Search Technical Report
2021.
@techreport{Tian2921,
title = {Action Command Encoding for SurrogateAssisted Neural Architecture Search},
author = {Ye Tian and Shichen Peng and Shangshang Yang and Xingyi Zhang and Kay Chen Tan and Yaochu Jin},
url = {https://www.researchgate.net/profile/Ye-Tian-84/publication/353166978_Action_Command_Encoding_for_Surrogate_Assisted_Neural_Architecture_Search/links/60eb072b1c28af34586047a1/Action-Command-Encoding-for-Surrogate-Assisted-Neural-Architecture-Search.pdf},
year = {2021},
date = {2021-07-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1359.
Sääw, Daniel; Nijweide, Frederik
Exploring the feasibility of Weight Agnostic Neural Networks for low-end hardware Technical Report
2021.
@techreport{Sääw2021,
title = {Exploring the feasibility of Weight Agnostic Neural Networks for low-end hardware},
author = {Daniel Sääw and Frederik Nijweide},
url = {https://www.researchgate.net/profile/Fpj_Nijweide/publication/352738002_Exploring_the_feasibility_of_Weight_Agnostic_Neural_Networks_for_low-end_hardware/links/60d630d5a6fdccb745e418ea/Exploring-the-feasibility-of-Weight-Agnostic-Neural-Networks-for-low-end-hardware.pdf},
year = {2021},
date = {2021-07-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1358.
White, Colin; Nolen, Sam; Savani, Yash
Exploring the Loss Landscape in Neural Architecture Search Inproceedings
In: 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021), 2021.
@inproceedings{white2021,
title = {Exploring the Loss Landscape in Neural Architecture Search},
author = {Colin White and Sam Nolen and Yash Savani},
url = {https://auai.org/uai2021/pdf/uai2021.242.pdf},
year = {2021},
date = {2021-07-01},
booktitle = {37th Conference on Uncertainty in Artificial Intelligence (UAI 2021)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1357.
Lin, Yi; Liu, Yanfei; Liu, Jingguang; Liu, Guocai; Ma, Kai; Zheng, Yefeng
LE-NAS: Learning-based Ensemble with Neural Architecture Search for 3D Radiotherapy Dose Prediction Journal Article
In: 2021.
@article{Lin2106,
title = {LE-NAS: Learning-based Ensemble with Neural Architecture Search for 3D Radiotherapy Dose Prediction},
author = {Yi Lin and Yanfei Liu and Jingguang Liu and Guocai Liu and Kai Ma and Yefeng Zheng},
url = {https://arxiv.org/pdf/2106.06733.pdf},
year = {2021},
date = {2021-07-01},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1356.
Dimanov, Daniel; Balaguer-Ballester, Emili; Singleton, Colin; Rostami, Shahin
Moncae: Multi-Objective Neuroevolution of Convolutional Autoencoders Inproceedings
In: 2nd Workshop on Neural Architecture Search at ICLR 2021, 2021.
@inproceedings{Dimanov2021,
title = {Moncae: Multi-Objective Neuroevolution of Convolutional Autoencoders},
author = {Daniel Dimanov and Emili Balaguer-Ballester and Colin Singleton and Shahin Rostami},
url = {https://www.researchgate.net/profile/Daniel-Dimanov-2/publication/352248797_MONCAE_MultiI-Objective_Neuroevolution_of_Convolutional_Autoencoders/links/60c0a43592851ca6f8d2d98b/MONCAE-MultiI-Objective-Neuroevolution-of-Convolutional-Autoencoders.pdf},
year = {2021},
date = {2021-07-01},
booktitle = {2nd Workshop on Neural Architecture Search at ICLR 2021},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1355.
Jin, Yaochu; Wang, Handing; Sun, Chaoli
Surrogate-Assisted Evolutionary Neural Architecture Search Inproceedings
In: Data-Driven Evolutionary Optimization, pp. 373-387, 2021, ISBN: 978-3-030-74640-7.
@inproceedings{Jin2021,
title = {Surrogate-Assisted Evolutionary Neural Architecture Search},
author = {Yaochu Jin and Handing Wang and Chaoli Sun},
url = {https://link.springer.com/chapter/10.1007/978-3-030-74640-7_12},
isbn = {978-3-030-74640-7},
year = {2021},
date = {2021-06-29},
booktitle = {Data-Driven Evolutionary Optimization},
volume = {975},
pages = {373-387},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1354.
Zhu, Wei; Ni, Yuan; Wang, Xiaoling; Xie, Guotong; Zhang, Fang
Discovering Better Model Architectures for Medical Query Understanding Inproceedings
In: Proceedings of NAACL HLT 2021: IndustryTrack Papers, pp. 230-237, 2021.
@inproceedings{ZHU2021,
title = {Discovering Better Model Architectures for Medical Query Understanding},
author = {
Wei Zhu and Yuan Ni and Xiaoling Wang and Guotong Xie and Fang Zhang},
url = {https://www.aclweb.org/anthology/2021.naacl-industry.29.pdf},
year = {2021},
date = {2021-06-06},
booktitle = {Proceedings of NAACL HLT 2021: IndustryTrack Papers},
pages = {230-237},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1353.
Dai, Xiaoliang; Wan, Alvin; Zhang, Peizhao; Wu, Bichen; He, Zijian; Wei, Zhen; Chen, Kan; Tian, Yuandong; Yu, Matthew; Vajda, Peter; Gonzalez, Joseph E.
FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining Incollection
In: CVPR2021, 2021.
@incollection{DaiCVPR2021,
title = {FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining},
author = {Xiaoliang Dai and Alvin Wan and Peizhao Zhang and Bichen Wu and Zijian He and Zhen Wei and Kan Chen and Yuandong Tian and Matthew Yu and Peter Vajda and Joseph E. Gonzalez},
url = {https://openaccess.thecvf.com/content/CVPR2021/papers/Dai_FBNetV3_Joint_Architecture-Recipe_Search_Using_Predictor_Pretraining_CVPR_2021_paper.pdf},
year = {2021},
date = {2021-06-01},
booktitle = {CVPR2021},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
1352.
Ma, Xiaohan; Si, Chang; Wang, Ying; Liu, Cheng; Zhang1, Lei
NASA: Accelerating Neural Network Design with a NAS Processor Inproceedings
In: 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), 2021.
@inproceedings{MaISCA2021,
title = {NASA: Accelerating Neural Network Design with a NAS Processor},
author = {Xiaohan Ma and Chang Si and Ying Wang and Cheng Liu and Lei Zhang1},
url = {https://conferences.computer.org/iscapub/pdfs/ISCA2021-4ghucdBnCWYB7ES2Pe4YdT/333300a790/333300a790.pdf},
year = {2021},
date = {2021-06-01},
booktitle = {2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1351.
Liu, Shaoli; Zheng, Chengjian; Lu, Kaidi; Gao, Si; Wang, Ning; Wang, Bofei; Zhang, Diankai; Zhang, Xiaofeng; Xu, Tianyu
EVSRNet: Efficient Video Super-Resolution With Neural Architecture Search Inproceedings
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 2480-2485, 2021.
@inproceedings{Liu_2021_CVPR,
title = {EVSRNet: Efficient Video Super-Resolution With Neural Architecture Search},
author = {Shaoli Liu and Chengjian Zheng and Kaidi Lu and Si Gao and Ning Wang and Bofei Wang and Diankai Zhang and Xiaofeng Zhang and Tianyu Xu},
url = {https://openaccess.thecvf.com/content/CVPR2021W/MAI/html/Liu_EVSRNet_Efficient_Video_Super-Resolution_With_Neural_Architecture_Search_CVPRW_2021_paper.html},
year = {2021},
date = {2021-06-01},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
pages = {2480-2485},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1350.
Geraeinejad, V.; Seenan, S.; Modarressi, M.; Daneshtalab, M.
RoCo-NAS : Robust and Compact Neural Architecture Search Inproceedings
In: The international joint conference on neural networks IJCNN, 2021.
@inproceedings{Geraeinejad2021,
title = {RoCo-NAS : Robust and Compact Neural Architecture Search},
author = {V. Geraeinejad and S. Seenan and M. Modarressi and M. Daneshtalab},
url = {https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1558970&dswid=-1593},
year = {2021},
date = {2021-06-01},
booktitle = {The international joint conference on neural networks IJCNN},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1349.
Tsokov, Stefan; Lazarova, Milena; Aleksieva-Petrova, Adelina
An Evolutionary Approach to the Design of Convolutional Neural Networks for Human Activity Recognition Journal Article
In: Indian Journal of Computer Science and Engineering, 2021.
@article{Tsokov2021,
title = {An Evolutionary Approach to the Design of Convolutional Neural Networks for Human Activity Recognition},
author = {Tsokov, Stefan and Lazarova, Milena and Aleksieva-Petrova, Adelina},
url = {http://www.ijcse.com/docs/INDJCSE21-12-02-145.pdf},
year = {2021},
date = {2021-06-01},
journal = {Indian Journal of Computer Science and Engineering},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1348.
Huang, Shenyang; Francois-Lavet, Vincent; Rabusseau, Guillaume
Understanding Capacity Saturation in Incremental Learning Inproceedings
In: The 34th Canadian Conference on Artificial Intelligence, 2021.
@inproceedings{Huang2021,
title = {Understanding Capacity Saturation in Incremental Learning},
author = {Shenyang Huang and Vincent Francois-Lavet and Guillaume Rabusseau},
url = {https://assets.pubpub.org/xabw479b/51621564316955.pdf},
year = {2021},
date = {2021-05-25},
booktitle = {The 34th Canadian Conference on Artificial Intelligence},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1347.
Chen, Wuyang; Gong, Xinyu; Wang, Zhangyang
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective Inproceedings
In: ICLR 2021, 2021.
@inproceedings{chen2021neural,
title = {Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective},
author = {Wuyang Chen and Xinyu Gong and Zhangyang Wang},
url = {https://arxiv.org/abs/2102.11535},
year = {2021},
date = {2021-05-04},
booktitle = {ICLR 2021},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1346.
Anwar, Abrar
Evolving Spiking Circuit Motifs Using Weight Agnostic Neural Networks Journal Article
In: Proceedings of the AAAI Conference on Artificial Intelligence, 35 (18), pp. 15956-15957, 2021.
@article{Anwar_2021,
title = {Evolving Spiking Circuit Motifs Using Weight Agnostic Neural Networks},
author = {Abrar Anwar},
url = {https://ojs.aaai.org/index.php/AAAI/article/view/17974},
year = {2021},
date = {2021-05-01},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
volume = {35},
number = {18},
pages = {15956-15957},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1345.
Wang, Xiaobo
Teacher Guided Neural Architecture Search for Face Recognition Journal Article
In: Proceedings of the AAAI Conference on Artificial Intelligence, 35 (4), pp. 2817-2825, 2021.
@article{Wang_2021,
title = {Teacher Guided Neural Architecture Search for Face Recognition},
author = {Xiaobo Wang},
url = {https://ojs.aaai.org/index.php/AAAI/article/view/16387},
year = {2021},
date = {2021-05-01},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
volume = {35},
number = {4},
pages = {2817-2825},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1344.
Perenda, Erma; Rajendran, Sreeraj; Bovet, Gerome; Pollin, Sofie; Zheleva, Mariya
Evolutionary Optimization of Residual Neural Network Architectures for Modulation Classification Miscellaneous
2021.
@misc{perenda_rajendran_bovet_pollin_zheleva_2021,
title = {Evolutionary Optimization of Residual Neural Network Architectures for Modulation Classification},
author = {Erma Perenda and Sreeraj Rajendran and Gerome Bovet and Sofie Pollin and Mariya Zheleva},
url = {https://www.techrxiv.org/articles/preprint/Evolutionary_Optimization_of_Residual_Neural_Network_Architectures_for_Modulation_Classification/14528778/1},
doi = {10.36227/techrxiv.14528778.v1},
year = {2021},
date = {2021-05-01},
publisher = {TechRxiv},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
1343.
Yuan, Zhihang; Liu, Jingze; Li, Xingchen; Yan, Longhao; Chen, Haoxiang; Wu, Bingzhe; Yang, Yuchao; Sun, Guangyu
NAS4RRAM: neural network architecture search for inference on RRAM-based accelerators Journal Article
In: Science China Information Sciences, 2021.
@article{yuan2021,
title = {NAS4RRAM: neural network architecture search for inference on RRAM-based accelerators},
author = {Yuan, Zhihang and Liu, Jingze and Li, Xingchen and Yan, Longhao and Chen, Haoxiang and Wu, Bingzhe and Yang, Yuchao and Sun, Guangyu},
url = {https://doi.org/10.1007/s11432-020-3245-7},
doi = {10.1007/s11432-020-3245-7},
year = {2021},
date = {2021-05-01},
journal = {Science China Information Sciences},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1342.
Zhang, Yi; Liu, Yang; Liu, X. Shirley
Neural network architecture search with AMBER Journal Article
In: Nature Machine Intelligence, pp. 372-373, 2021.
@article{Zhang2021,
title = {Neural network architecture search with AMBER},
author = {Zhang, Yi and Liu, Yang and Liu, X. Shirley},
url = {https://doi.org/10.1038/s42256-021-00350-x},
year = {2021},
date = {2021-05-01},
journal = {Nature Machine Intelligence},
pages = {372-373},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1341.
Schoenherr, Georg P.
The Nonlinearity Coefficient - A Practical Guide to Neural Architecture Design PhD Thesis
2021.
@phdthesis{Schoenherr2021,
title = {The Nonlinearity Coefficient - A Practical Guide to Neural Architecture Design},
author = {Georg P. Schoenherr},
url = {http://reports-archive.adm.cs.cmu.edu/anon/anon/home/ftp/usr/ftp/2021/abstracts/21-110.html},
year = {2021},
date = {2021-05-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
1340.
Yang, Yuxuan; Gao, Zhongke; Li, Yanli; Wang, He
A CNN identified by reinforcement learning-based optimization framework for EEG-based state evaluation Journal Article
In: Journal of Neural Engineering, 18 (4), pp. 046059, 2021.
@article{Yang_2021,
title = {A CNN identified by reinforcement learning-based optimization framework for EEG-based state evaluation},
author = {Yuxuan Yang and Zhongke Gao and Yanli Li and He Wang},
url = {https://doi.org/10.1088/1741-2552/abfa71},
doi = {10.1088/1741-2552/abfa71},
year = {2021},
date = {2021-05-01},
journal = {Journal of Neural Engineering},
volume = {18},
number = {4},
pages = {046059},
publisher = {IOP Publishing},
abstract = {Objective. Electroencephalogram (EEG) data, as a kind of complex time-series, is one of the most widely-used information measurements for evaluating human psychophysiological states. Recently, numerous works applied deep learning techniques, especially the convolutional neural network (CNN), into EEG-based research. The design of the hyper-parameters of the CNN model has a great influence on the performance of the model. Therefore, automatically designing these hyper-parameters can save the time and labor of experts. This leads to the appearance of the neural architecture search technique. In this paper, we propose a reinforcement learning (RL)-based step-by-step framework to efficiently search for CNN models. Approach. Specifically, the deep Q network in RL is first used to determine the depth of convolutional layers and the connection modes among layers. Then particle swarm optimization algorithm is used to fine-tune the number and size of convolution kernels. Through this step-by-step strategy, the search space can be narrowed in each step for saving the overall time cost. This framework is employed for both EEG-based sleep stage classification and driver drowsiness evaluation tasks. Main results. The results show that compared with state-of-the-art methods, the high-performance CNN models identified by the proposed optimization framework, can achieve high overall accuracy and better root mean squared error in the two tasks. Significance. Therefore, the proposed optimization framework has a great potential to provide high-performance results for other kinds of classification and prediction tasks. In this way, it can greatly save researchers’ time cost and promote broader applications of CNNs.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Objective. Electroencephalogram (EEG) data, as a kind of complex time-series, is one of the most widely-used information measurements for evaluating human psychophysiological states. Recently, numerous works applied deep learning techniques, especially the convolutional neural network (CNN), into EEG-based research. The design of the hyper-parameters of the CNN model has a great influence on the performance of the model. Therefore, automatically designing these hyper-parameters can save the time and labor of experts. This leads to the appearance of the neural architecture search technique. In this paper, we propose a reinforcement learning (RL)-based step-by-step framework to efficiently search for CNN models. Approach. Specifically, the deep Q network in RL is first used to determine the depth of convolutional layers and the connection modes among layers. Then particle swarm optimization algorithm is used to fine-tune the number and size of convolution kernels. Through this step-by-step strategy, the search space can be narrowed in each step for saving the overall time cost. This framework is employed for both EEG-based sleep stage classification and driver drowsiness evaluation tasks. Main results. The results show that compared with state-of-the-art methods, the high-performance CNN models identified by the proposed optimization framework, can achieve high overall accuracy and better root mean squared error in the two tasks. Significance. Therefore, the proposed optimization framework has a great potential to provide high-performance results for other kinds of classification and prediction tasks. In this way, it can greatly save researchers’ time cost and promote broader applications of CNNs.
1339.
Pan, Zheyi; Ke, Songyu; Yang, Xiaodu; Liang, Yuxuan; Yu, Yong; Zhang, Junbo; Zheng, Yu
AutoSTG: Neural Architecture Search forPredictions of Spatio-Temporal Graphs Inproceedings
In: WWW 2021, 2021.
@inproceedings{PanWWW2021,
title = {AutoSTG: Neural Architecture Search forPredictions of Spatio-Temporal Graphs},
author = {Zheyi Pan and Songyu Ke and Xiaodu Yang and Yuxuan Liang and Yong Yu and Junbo Zhang and Yu Zheng},
url = {http://panzheyi.cc/publication/pan2021autostg/paper.pdf},
year = {2021},
date = {2021-04-19},
booktitle = {WWW 2021},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1338.
Kyriakides, George; Margaritis, Konstantinos
Evolving graph convolutional networks for neural architecture search Journal Article
In: Neural Computing and Applications, 2021.
@article{Kriakides2021Evolving,
title = {Evolving graph convolutional networks for neural architecture search},
author = {Kyriakides, George and Margaritis, Konstantinos},
url = {https://doi.org/10.1007/s00521-021-05979-8},
doi = {10.1007/s00521-021-05979-8},
year = {2021},
date = {2021-04-15},
journal = {Neural Computing and Applications},
abstract = {As neural architecture search (NAS) becomes an increasingly adopted method to design network architectures, various methods have been proposed to speedup the process. Besides proxy evaluation tasks, weight sharing, and scaling down the evaluated architectures, performance-predicting models exhibit multiple advantages. Eliminating the need to train candidate architectures and enabling transfer learning between datasets, researchers can also utilize them as a surrogate function for Bayesian optimization. On the other hand, graph convolutional networks (GCNs) have also been increasingly adopted for various tasks, enabling deep learning techniques on graphs without feature engineering. In this paper, we employ an evolutionary-based NAS method to evolve GCNs for the problem of predicting the relative performance of various architectures included in the NAS-Bench-101 dataset. By fine-tuning the architecture generated by our methodology, we manage to achieve a Kendall’s tau correlation coefficient of 0.907 between 1050 completely unseen architectures, utilizing only 450 samples, while also outperforming a strong baseline on the same task. Furthermore, we validate our method on custom global search space architectures, generated for the Fashion-MNIST dataset.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
As neural architecture search (NAS) becomes an increasingly adopted method to design network architectures, various methods have been proposed to speedup the process. Besides proxy evaluation tasks, weight sharing, and scaling down the evaluated architectures, performance-predicting models exhibit multiple advantages. Eliminating the need to train candidate architectures and enabling transfer learning between datasets, researchers can also utilize them as a surrogate function for Bayesian optimization. On the other hand, graph convolutional networks (GCNs) have also been increasingly adopted for various tasks, enabling deep learning techniques on graphs without feature engineering. In this paper, we employ an evolutionary-based NAS method to evolve GCNs for the problem of predicting the relative performance of various architectures included in the NAS-Bench-101 dataset. By fine-tuning the architecture generated by our methodology, we manage to achieve a Kendall’s tau correlation coefficient of 0.907 between 1050 completely unseen architectures, utilizing only 450 samples, while also outperforming a strong baseline on the same task. Furthermore, we validate our method on custom global search space architectures, generated for the Fashion-MNIST dataset.
1337.
Wang, Linnan; Xie, Saining; Li, Teng; Fonseca, Rodrigo; Tian, Yuandong
Sample-Efficient Neural Architecture Search by Learning Actions for Monte Carlo Tree Search Journal Article
In: IEEE transactions on pattern analysis and machine intelligence, PP , 2021, ISSN: 0162-8828.
@article{PMID:33826511,
title = {Sample-Efficient Neural Architecture Search by Learning Actions for Monte Carlo Tree Search},
author = {Linnan Wang and Saining Xie and Teng Li and Rodrigo Fonseca and Yuandong Tian},
url = {https://doi.org/10.1109/TPAMI.2021.3071343},
doi = {10.1109/tpami.2021.3071343},
issn = {0162-8828},
year = {2021},
date = {2021-04-01},
journal = {IEEE transactions on pattern analysis and machine intelligence},
volume = {PP},
abstract = {Neural Architecture Search (NAS) has emerged as a promising technique for automatic neural network design. However, existing MCTS based NAS approaches often utilize manually designed action space, which is not directly related to the performance metric to be optimized (e.g., accuracy), leading to sample-inefficient explorations of architectures. To improve the sample efficiency, this paper proposes Latent Action Neural Architecture Search (LaNAS), which learns actions to recursively partition the search space into good or bad regions that contain networks with similar performance metrics. During the search phase, as different action sequences lead to regions with different performance, the search efficiency can be significantly improved by biasing towards the good regions. On three NAS tasks, empirical results demonstrate that LaNAS is at least an order more sample efficient than baseline methods including evolutionary algorithms, Bayesian optimizations, and random search. When applied in practice, both one-shot and regular LaNAS consistently outperform existing results. Particularly, LaNAS achieves 99.0% accuracy on CIFAR-10 and 80.8% top1 accuracy at 600 MFLOPS on ImageNet in only 800 samples, significantly outperforming AmoebaNet with 33x fewer samples. Our code is publicly available at https://github.com/facebookresearch/LaMCTS.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural Architecture Search (NAS) has emerged as a promising technique for automatic neural network design. However, existing MCTS based NAS approaches often utilize manually designed action space, which is not directly related to the performance metric to be optimized (e.g., accuracy), leading to sample-inefficient explorations of architectures. To improve the sample efficiency, this paper proposes Latent Action Neural Architecture Search (LaNAS), which learns actions to recursively partition the search space into good or bad regions that contain networks with similar performance metrics. During the search phase, as different action sequences lead to regions with different performance, the search efficiency can be significantly improved by biasing towards the good regions. On three NAS tasks, empirical results demonstrate that LaNAS is at least an order more sample efficient than baseline methods including evolutionary algorithms, Bayesian optimizations, and random search. When applied in practice, both one-shot and regular LaNAS consistently outperform existing results. Particularly, LaNAS achieves 99.0% accuracy on CIFAR-10 and 80.8% top1 accuracy at 600 MFLOPS on ImageNet in only 800 samples, significantly outperforming AmoebaNet with 33x fewer samples. Our code is publicly available at https://github.com/facebookresearch/LaMCTS.
1336.
Zhang, Zhentong; Shan, Yugang; Yuan, Jie
Multi-Level Cell Progressive Differentiable Architecture Search to Improve Image Classification Accuracy Journal Article
In: Journal of Signal Processing Systems, 2021.
@article{ZhangJSPS2021,
title = {Multi-Level Cell Progressive Differentiable Architecture Search to Improve Image Classification Accuracy},
author = {Zhang, Zhentong and Yugang Shan and Jie Yuan},
url = {https://doi.org/10.1007/s11265-021-01647-1},
year = {2021},
date = {2021-03-08},
journal = {Journal of Signal Processing Systems},
abstract = {In recent years, the neural architecture search has continuously made significant progress in the field of image recognition. Among them, the differentiable method has obvious advantages compared with other search methods in terms of computational cost and accuracy to deal with image classification. However, the differentiable method is usually composed of single cell, which cannot efficiently extract the features of the network. In response to this problem, we propose a multi-level cell progressive differentiable method which allows cells to have different types according to the levels of the network. In differentiable method, the gap between the search network and the evaluation one is large, and the correlation is low. We design an algorithm to improve the distribution of architecture parameters. We also optimize the loss function and use the regularization method of additional action to improve deep network performance. The method achieves good search and classification results on CIFAR10 and ImageNet (mobile setting).},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In recent years, the neural architecture search has continuously made significant progress in the field of image recognition. Among them, the differentiable method has obvious advantages compared with other search methods in terms of computational cost and accuracy to deal with image classification. However, the differentiable method is usually composed of single cell, which cannot efficiently extract the features of the network. In response to this problem, we propose a multi-level cell progressive differentiable method which allows cells to have different types according to the levels of the network. In differentiable method, the gap between the search network and the evaluation one is large, and the correlation is low. We design an algorithm to improve the distribution of architecture parameters. We also optimize the loss function and use the regularization method of additional action to improve deep network performance. The method achieves good search and classification results on CIFAR10 and ImageNet (mobile setting).
1335.
Zimmer, Lucas; Lindauer, Marius; Hutter, Frank
Auto-Pytorch: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL Journal Article
In: IEEE transactions on pattern analysis and machine intelligence, PP , 2021, ISSN: 0162-8828.
@article{PMID:33750687,
title = {Auto-Pytorch: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL},
author = {Lucas Zimmer and Marius Lindauer and Frank Hutter},
url = {https://doi.org/10.1109/TPAMI.2021.3067763},
doi = {10.1109/tpami.2021.3067763},
issn = {0162-8828},
year = {2021},
date = {2021-03-01},
journal = {IEEE transactions on pattern analysis and machine intelligence},
volume = {PP},
abstract = {While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings the best of these two worlds together by jointly and robustly optimizing the architecture of networks and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors on average.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings the best of these two worlds together by jointly and robustly optimizing the architecture of networks and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors on average.
1334.
Zheng, X; Ji, R; Chen, Y; Wang, Q; Zhang, B; Ye, Q; Chen, J; Huang, F; Tian, Y
MIGO-NAS: Towards Fast and Generalizable Neural Architecture Search Journal Article
In: IEEE Transactions on Pattern Analysis & Machine Intelligence, (01), pp. 1-1, 2021, ISSN: 1939-3539.
@article{9377468,
title = {MIGO-NAS: Towards Fast and Generalizable Neural Architecture Search},
author = {X Zheng and R Ji and Y Chen and Q Wang and B Zhang and Q Ye and J Chen and F Huang and Y Tian},
url = {https://www.computer.org/csdl/journal/tp/5555/01/09377468/1rUNdbz4LQY},
doi = {10.1109/TPAMI.2021.3065138},
issn = {1939-3539},
year = {2021},
date = {2021-03-01},
journal = {IEEE Transactions on Pattern Analysis & Machine Intelligence},
number = {01},
pages = {1-1},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1333.
Liu, Lanlan; Zhang, Yuting; Deng, Jia; Soatto, Stefano
Dynamically Grown Generative Adversarial Networks Inproceedings
In: AAAI 2021, 2021.
@inproceedings{LiuAAAI2021,
title = {Dynamically Grown Generative Adversarial Networks},
author = {Lanlan Liu and Yuting Zhang and Jia Deng and Stefano Soatto},
url = {https://www.aaai.org/AAAI21Papers/AAAI-1376.LiuL.pdf},
year = {2021},
date = {2021-02-02},
booktitle = {AAAI 2021},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1332.
Hao, Jie; Zhu, William
Architecture self-attention mechanism: nonlinear optimization for neural architecture search Journal Article
In: Journal of Nonlinear and Variational Analysis, 5 , pp. 119-140, 2021.
@article{Hao2021,
title = { Architecture self-attention mechanism: nonlinear optimization for neural architecture search},
author = {Jie Hao and William Zhu},
url = {http://jnva.biemdas.com/issues/JNVA2021-1-8.pdf},
doi = { 10.23952/jnva.5.2021.1.08},
year = {2021},
date = {2021-02-01},
journal = {Journal of Nonlinear and Variational Analysis},
volume = {5},
pages = {119-140},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1331.
Xu, Y; Xie, L; Dai, W; Zhang, X; Chen, X; Qi, G; Xiong, H; Tian, Q
Partially-Connected Neural Architecture Search for Reduced Computational Redundancy Journal Article
In: IEEE Transactions on Pattern Analysis & Machine Intelligence, (01), pp. 1-1, 2021, ISSN: 1939-3539.
@article{9354953,
title = {Partially-Connected Neural Architecture Search for Reduced Computational Redundancy},
author = {Y Xu and L Xie and W Dai and X Zhang and X Chen and G Qi and H Xiong and Q Tian},
url = {https://www.computer.org/csdl/journal/tp/5555/01/09354953/1rgCccYlOaQ},
doi = {10.1109/TPAMI.2021.3059510},
issn = {1939-3539},
year = {2021},
date = {2021-02-01},
journal = {IEEE Transactions on Pattern Analysis & Machine Intelligence},
number = {01},
pages = {1-1},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Differentiable architecture search (DARTS) enables effective neural architecture search (NAS) using gradient descent, but suffers from high memory and computational costs. In this paper, we propose a novel approach, namely Partially-Connected DARTS (PC-DARTS), to achieve efficient and stable neural architecture search by reducing the channel and spatial redundancies of the super-network. In the channel level, partial channel connection is presented to randomly sample a small subset of channels for operation selection to accelerate the search process and suppress the over-fitting of the super-network. Side operation is introduced for bypassing (non-sampled) channels to guarantee the performance of searched architectures under extremely low sampling rates. In the spatial level, input features are down-sampled to eliminate spatial redundancy and enhance the efficiency of the mixed computation for operation selection. Furthermore, edge normalization is developed to maintain the consistency of edge selection based on channel sampling with the architectural parameters for edges. Experimental results demonstrate that the proposed approach achieves higher search speed and training stability than DARTS. PC-DARTS obtains a top-1 error rate of 2.55% on CIFAR-10 with 0.07 GPU-days for architecture search, and a state-of-the-art top-1 error rate of 24.1% on ImageNet (under the mobile setting) within 2.8 GPU-day.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Differentiable architecture search (DARTS) enables effective neural architecture search (NAS) using gradient descent, but suffers from high memory and computational costs. In this paper, we propose a novel approach, namely Partially-Connected DARTS (PC-DARTS), to achieve efficient and stable neural architecture search by reducing the channel and spatial redundancies of the super-network. In the channel level, partial channel connection is presented to randomly sample a small subset of channels for operation selection to accelerate the search process and suppress the over-fitting of the super-network. Side operation is introduced for bypassing (non-sampled) channels to guarantee the performance of searched architectures under extremely low sampling rates. In the spatial level, input features are down-sampled to eliminate spatial redundancy and enhance the efficiency of the mixed computation for operation selection. Furthermore, edge normalization is developed to maintain the consistency of edge selection based on channel sampling with the architectural parameters for edges. Experimental results demonstrate that the proposed approach achieves higher search speed and training stability than DARTS. PC-DARTS obtains a top-1 error rate of 2.55% on CIFAR-10 with 0.07 GPU-days for architecture search, and a state-of-the-art top-1 error rate of 24.1% on ImageNet (under the mobile setting) within 2.8 GPU-day.
1330.
Liao, Yi-Lun; Karaman, Sertac; Sze, Vivienne
Searching for Efficient Multi-Stage Vision Transformers Technical Report
, 2021.
@techreport{DBLP:journals/corr/abs-2109-00642,
title = {Searching for Efficient Multi-Stage Vision Transformers},
author = {Yi-Lun Liao and Sertac Karaman and Vivienne Sze},
url = {https://arxiv.org/abs/2109.00642},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
journal = {CoRR},
volume = {abs/2109.00642},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1329.
Liu, Jinyuan; Wu, Yuhui; Huang, Zhanbo; Liu, Risheng; Fan, Xin
SMoA: Searching a Modality-Oriented Architecture for Infrared and Visible Image Fusion Journal Article
In: IEEE Signal Processing Letters, 28 , pp. 1818-1822, 2021.
@article{9528046,
title = {SMoA: Searching a Modality-Oriented Architecture for Infrared and Visible Image Fusion},
author = {Jinyuan Liu and Yuhui Wu and Zhanbo Huang and Risheng Liu and Xin Fan},
url = {https://ieeexplore.ieee.org/abstract/document/9528046},
doi = {10.1109/LSP.2021.3109818},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
journal = {IEEE Signal Processing Letters},
volume = {28},
pages = {1818-1822},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1328.
Umamageswari, A.; Bharathiraja, N.; Irene, D. Shiny
In: ICT Express, 2021, ISSN: 2405-9595.
@article{UMAMAGESWARI2021,
title = {A Novel Fuzzy C-Means based Chameleon Swarm Algorithm for Segmentation and Progressive Neural Architecture Search for Plant Disease Classification},
author = {A. Umamageswari and N. Bharathiraja and D. Shiny Irene},
url = {https://www.sciencedirect.com/science/article/pii/S2405959521001090},
doi = {https://doi.org/10.1016/j.icte.2021.08.019},
issn = {2405-9595},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
journal = {ICT Express},
abstract = {This study proposed a novel framework for plant leaf disease identification. The proposed model consists of four steps including pre-processing, segmentation, feature extraction, and classification. At first, the unwanted noise and overfitting are removed, and also image contrast level is enhanced. Secondly, the Fuzzy C-Means (FCM) based Chameleon Swarm Algorithm (CSA) named as (FCM-CSA) is used for plant leaf diseased part segmentation. In the third stage, the feature extraction is performed using a fast GLCM feature extraction model. Finally, the Progressive Neural Architecture Search (PNAS) is used for plant leaf disease identification. The experimental investigations are carried out using MATLAB software with the Mendeley database. From this dataset, we have used Apple Cedar Apple Rust (ACAR), Cherry Powdery Mildew (CPM), Corn Common Rust (CCCR), Apple Healthy (AH), Grape Black Rot (GBR), Pepper Bell Bacterial Spot (PBBS), Potato Late Blight (PLB) and Tomato Leaf Mold (TLM) disease images. Different measures such as precision, recall, sensitivity, specificity, and accuracy results are used to validate the performance of the proposed model.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This study proposed a novel framework for plant leaf disease identification. The proposed model consists of four steps including pre-processing, segmentation, feature extraction, and classification. At first, the unwanted noise and overfitting are removed, and also image contrast level is enhanced. Secondly, the Fuzzy C-Means (FCM) based Chameleon Swarm Algorithm (CSA) named as (FCM-CSA) is used for plant leaf diseased part segmentation. In the third stage, the feature extraction is performed using a fast GLCM feature extraction model. Finally, the Progressive Neural Architecture Search (PNAS) is used for plant leaf disease identification. The experimental investigations are carried out using MATLAB software with the Mendeley database. From this dataset, we have used Apple Cedar Apple Rust (ACAR), Cherry Powdery Mildew (CPM), Corn Common Rust (CCCR), Apple Healthy (AH), Grape Black Rot (GBR), Pepper Bell Bacterial Spot (PBBS), Potato Late Blight (PLB) and Tomato Leaf Mold (TLM) disease images. Different measures such as precision, recall, sensitivity, specificity, and accuracy results are used to validate the performance of the proposed model.
1327.
Shu, Yao; Cai, Shaofeng; Dai, Zhongxiang; Ooi, Beng Chin; Low, Bryan Kian Hsiang
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Technical Report
, 2021.
@techreport{DBLP:journals/corr/abs-2109-00817,
title = {NASI: Label- and Data-agnostic Neural Architecture Search at Initialization},
author = {Yao Shu and Shaofeng Cai and Zhongxiang Dai and Beng Chin Ooi and Bryan Kian Hsiang Low},
url = {https://arxiv.org/abs/2109.00817},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
journal = {CoRR},
volume = {abs/2109.00817},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1326.
Deng, Jiajun; Gutierrez, Fabian Ritter; Hu, Shoukang; Geng, Mengzhe; Xie, Xurong; Ye, Zi; Liu, Shansong; Yu, Jianwei; Liu, Xunying; Meng, Helen
Bayesian Parametric and Architectural Domain Adaptation of LF-MMI Trained TDNNs for Elderly and Dysarthric Speech Recognition Inproceedings
In: Proc. Interspeech 2021, pp. 4818–4822, 2021.
@inproceedings{deng21d_interspeech,
title = {Bayesian Parametric and Architectural Domain Adaptation of LF-MMI Trained TDNNs for Elderly and Dysarthric Speech Recognition},
author = {Jiajun Deng and Fabian Ritter Gutierrez and Shoukang Hu and Mengzhe Geng and Xurong Xie and Zi Ye and Shansong Liu and Jianwei Yu and Xunying Liu and Helen Meng},
url = {https://www.isca-speech.org/archive/interspeech_2021/deng21d_interspeech.html},
doi = {10.21437/Interspeech.2021-289},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
booktitle = {Proc. Interspeech 2021},
pages = {4818--4822},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1325.
Gong, Xinyu; Wang, Heng; Shou, Zheng; Feiszli, Matt; Wang, Zhangyang; Yan, Zhicheng
Searching for Two-Stream Models in Multivariate Space for Video Recognition Technical Report
, 2021.
@techreport{DBLP:journals/corr/abs-2108-12957,
title = {Searching for Two-Stream Models in Multivariate Space for Video Recognition},
author = {Xinyu Gong and Heng Wang and Zheng Shou and Matt Feiszli and Zhangyang Wang and Zhicheng Yan},
url = {https://arxiv.org/abs/2108.12957},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
journal = {CoRR},
volume = {abs/2108.12957},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1324.
Yu, Kaicheng; Ranftl, Rene; Salzmann, Mathieu
An Analysis of Super-Net Heuristics in Weight-Sharing NAS Journal Article
In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021.
@article{9525258,
title = {An Analysis of Super-Net Heuristics in Weight-Sharing NAS},
author = {Kaicheng Yu and Rene Ranftl and Mathieu Salzmann},
url = {https://ieeexplore.ieee.org/abstract/document/9525258},
doi = {10.1109/TPAMI.2021.3108480},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1323.
Xue, Song; Zhao, Bo; Chen, Hanlin; Wang, Runqi; Zhang, Baochang
UCB-ENAS based on Reinforcement Learning Inproceedings
In: 2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA), pp. 2008-2013, 2021.
@inproceedings{9516113,
title = {UCB-ENAS based on Reinforcement Learning},
author = {Song Xue and Bo Zhao and Hanlin Chen and Runqi Wang and Baochang Zhang},
url = {https://ieeexplore.ieee.org/abstract/document/9516113},
doi = {10.1109/ICIEA51954.2021.9516113},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
booktitle = {2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA)},
pages = {2008-2013},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1322.
Nie, Xing; Liu, Yongcheng; Chen, Shaohong; Chang, Jianlong; Huo, Chunlei; Meng, Gaofeng; Tian, Qi; Hu, Weiming; Pan, Chunhong
Differentiable Convolution Search for Point Cloud Processing Technical Report
, 2021.
@techreport{DBLP:journals/corr/abs-2108-12856,
title = {Differentiable Convolution Search for Point Cloud Processing},
author = {Xing Nie and Yongcheng Liu and Shaohong Chen and Jianlong Chang and Chunlei Huo and Gaofeng Meng and Qi Tian and Weiming Hu and Chunhong Pan},
url = {https://arxiv.org/abs/2108.12856},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
journal = {CoRR},
volume = {abs/2108.12856},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1321.
Li, Yuening; Chen, Zhengzhang; Zha, Daochen; Zhou, Kaixiong; Jin, Haifeng; Chen, Haifeng; Hu, Xia
Automated Anomaly Detection via Curiosity-Guided Search and Self-Imitation Learning Journal Article
In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-13, 2021.
@article{9526875,
title = {Automated Anomaly Detection via Curiosity-Guided Search and Self-Imitation Learning},
author = {Yuening Li and Zhengzhang Chen and Daochen Zha and Kaixiong Zhou and Haifeng Jin and Haifeng Chen and Xia Hu},
url = {https://ieeexplore.ieee.org/abstract/document/9526875},
doi = {10.1109/TNNLS.2021.3105636},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}