Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2022
2210.
Ding, Yadong; Wu, Yu; Huang, Chengyue; Tang, Siliang; Wu, Fei; Yang, Yi; Zhu, Wenwu; Zhuang, Yueting
NAP: Neural Architecture search with Pruning Journal Article
In: Neurocomputing, 2022, ISSN: 0925-2312.
@article{DING2022,
title = {NAP: Neural Architecture search with Pruning},
author = {Yadong Ding and Yu Wu and Chengyue Huang and Siliang Tang and Fei Wu and Yi Yang and Wenwu Zhu and Yueting Zhuang},
url = {https://www.sciencedirect.com/science/article/pii/S0925231221018361},
doi = {https://doi.org/10.1016/j.neucom.2021.12.002},
issn = {0925-2312},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Neurocomputing},
abstract = {There has been continuously increasing attention attracted by Neural Architecture Search (NAS). Due to its computational efficiency, gradient-based NAS methods like DARTS have become the most popular framework for NAS tasks. Nevertheless, as the search iterates, the derived model in previous NAS frameworks becomes dominated by skip-connects, causing the performance downfall. In this work, we present a novel approach to alleviate this issue, named Neural Architecture search with Pruning (NAP). Unlike prior differentiable architecture search works, our approach draws the idea from network pruning. We first train an over-parameterized network, including all candidate operations. Then we propose a criterion to prune the network. Based on a newly designed relaxation of architecture representation, NAP can derive the most potent model by removing trivial and redundant edges from the whole network topology. Experiments show the effectiveness of our proposed approach. Specifically, the model searched by NAP achieves state-of-the-art performances (2.48% test error) on CIFAR-10. We transfer the model to ImageNet and obtains a 25.1% test error with only 5.0M parameters, which is on par with modern NAS methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
There has been continuously increasing attention attracted by Neural Architecture Search (NAS). Due to its computational efficiency, gradient-based NAS methods like DARTS have become the most popular framework for NAS tasks. Nevertheless, as the search iterates, the derived model in previous NAS frameworks becomes dominated by skip-connects, causing the performance downfall. In this work, we present a novel approach to alleviate this issue, named Neural Architecture search with Pruning (NAP). Unlike prior differentiable architecture search works, our approach draws the idea from network pruning. We first train an over-parameterized network, including all candidate operations. Then we propose a criterion to prune the network. Based on a newly designed relaxation of architecture representation, NAP can derive the most potent model by removing trivial and redundant edges from the whole network topology. Experiments show the effectiveness of our proposed approach. Specifically, the model searched by NAP achieves state-of-the-art performances (2.48% test error) on CIFAR-10. We transfer the model to ImageNet and obtains a 25.1% test error with only 5.0M parameters, which is on par with modern NAS methods.
2209.
Zhou, Kaixiong; Liu, Zirui; Duan, Keyu; Hu, Xia
Graph Neural Networks: AutoML Book Chapter
In: pp. 371-389, 2022.
@inbook{Zhou2022,
title = {Graph Neural Networks: AutoML},
author = {Kaixiong Zhou and Zirui Liu and Keyu Duan and Xia Hu},
url = {https://link.springer.com/chapter/10.1007/978-981-16-6054-2_17},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
pages = {371-389},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
2208.
Rorabaugh, Ariel Keller; Caíno-Lores, Silvina; Johnston, Travis; Taufer, Michela
High frequency accuracy and loss data of random neural networks trained on image datasets Journal Article
In: Data in Brief, vol. 40, pp. 107780, 2022, ISSN: 2352-3409.
@article{RORABAUGH2022107780,
title = {High frequency accuracy and loss data of random neural networks trained on image datasets},
author = {Ariel Keller Rorabaugh and Silvina Caíno-Lores and Travis Johnston and Michela Taufer},
url = {https://www.sciencedirect.com/science/article/pii/S2352340921010544},
doi = {https://doi.org/10.1016/j.dib.2021.107780},
issn = {2352-3409},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Data in Brief},
volume = {40},
pages = {107780},
abstract = {Neural Networks (NNs) are increasingly used across scientific domains to extract knowledge from experimental or computational data. An NN is composed of natural or artificial neurons that serve as simple processing units and are interconnected into a model architecture; it acquires knowledge from the environment through a learning process and stores this knowledge in its connections. The learning process is conducted by training. During NN training, the learning process can be tracked by periodically validating the NN and calculating its fitness. The resulting sequence of fitness values (i.e., validation accuracy or validation loss) is called the NN learning curve. The development of tools for NN design requires knowledge of diverse NNs and their complete learning curves. Generally, only final fully-trained fitness values for highly accurate NNs are made available to the community, hampering efforts to develop tools for NN design and leaving unaddressed aspects such as explaining the generation of an NN and reproducing its learning process. Our dataset fills this gap by fully recording the structure, metadata, and complete learning curves for a wide variety of random NNs throughout their training. Our dataset captures the lifespan of 6000 NNs throughout generation, training, and validation stages. It consists of a suite of 6000 tables, each table representing the lifespan of one NN. We generate each NN with randomized parameter values and train it for 40 epochs on one of three diverse image datasets (i.e., CIFAR-100, FashionMNIST, SVHN). We calculate and record each NN’s fitness with high frequency—every half epoch—to capture the evolution of the training and validation process. As a result, for each NN, we record the generated parameter values describing the structure of that NN, the image dataset on which the NN trained, and all loss and accuracy values for the NN every half epoch. We put our dataset to the service of researchers studying NN performance and its evolution throughout training and validation. Statistical methods can be applied to our dataset to analyze the shape of learning curves in diverse NNs, and the relationship between an NN’s structure and its fitness. Additionally, the structural data and metadata that we record enable the reconstruction and reproducibility of the associated NN.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural Networks (NNs) are increasingly used across scientific domains to extract knowledge from experimental or computational data. An NN is composed of natural or artificial neurons that serve as simple processing units and are interconnected into a model architecture; it acquires knowledge from the environment through a learning process and stores this knowledge in its connections. The learning process is conducted by training. During NN training, the learning process can be tracked by periodically validating the NN and calculating its fitness. The resulting sequence of fitness values (i.e., validation accuracy or validation loss) is called the NN learning curve. The development of tools for NN design requires knowledge of diverse NNs and their complete learning curves. Generally, only final fully-trained fitness values for highly accurate NNs are made available to the community, hampering efforts to develop tools for NN design and leaving unaddressed aspects such as explaining the generation of an NN and reproducing its learning process. Our dataset fills this gap by fully recording the structure, metadata, and complete learning curves for a wide variety of random NNs throughout their training. Our dataset captures the lifespan of 6000 NNs throughout generation, training, and validation stages. It consists of a suite of 6000 tables, each table representing the lifespan of one NN. We generate each NN with randomized parameter values and train it for 40 epochs on one of three diverse image datasets (i.e., CIFAR-100, FashionMNIST, SVHN). We calculate and record each NN’s fitness with high frequency—every half epoch—to capture the evolution of the training and validation process. As a result, for each NN, we record the generated parameter values describing the structure of that NN, the image dataset on which the NN trained, and all loss and accuracy values for the NN every half epoch. We put our dataset to the service of researchers studying NN performance and its evolution throughout training and validation. Statistical methods can be applied to our dataset to analyze the shape of learning curves in diverse NNs, and the relationship between an NN’s structure and its fitness. Additionally, the structural data and metadata that we record enable the reconstruction and reproducibility of the associated NN.
2207.
Wang, Xin; Zhang, Ziwei; Zhu, Wenwu
Automated Graph Machine Learning: Approaches, Libraries and Directions Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-01288,
title = {Automated Graph Machine Learning: Approaches, Libraries and Directions},
author = {Xin Wang and Ziwei Zhang and Wenwu Zhu},
url = {https://arxiv.org/abs/2201.01288},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.01288},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2206.
Rorabaugh, Ariel Keller; Caino-Lores, Silvina; Johnston, Travis; Taufer, Michela
Building High-throughput Neural Architecture Search Workflows via a Decoupled Fitness Prediction Engine Journal Article
In: IEEE Transactions on Parallel and Distributed Systems, pp. 1-1, 2022.
@article{9674227,
title = {Building High-throughput Neural Architecture Search Workflows via a Decoupled Fitness Prediction Engine},
author = {Ariel Keller Rorabaugh and Silvina Caino-Lores and Travis Johnston and Michela Taufer},
url = {https://ieeexplore.ieee.org/abstract/document/9674227},
doi = {10.1109/TPDS.2022.3140681},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Parallel and Distributed Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2205.
Zhan, Cheng; Zhang, Licheng; Zhao, Xin; Lee, Chang-Chun; Huang, Shujiao
Neural Architecture Search for Inversion Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-01772,
title = {Neural Architecture Search for Inversion},
author = {Cheng Zhan and Licheng Zhang and Xin Zhao and Chang-Chun Lee and Shujiao Huang},
url = {https://arxiv.org/abs/2201.01772},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.01772},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2204.
Zhou, Jian; Zheng, Lianyu; Wang, Yiwei; Wang, Cheng; Gao, Robert X.
Automated model generation for machinery fault diagnosis based on reinforcement learning and neural architecture search Journal Article
In: IEEE Transactions on Instrumentation and Measurement, pp. 1-1, 2022.
@article{9673794,
title = {Automated model generation for machinery fault diagnosis based on reinforcement learning and neural architecture search},
author = {Jian Zhou and Lianyu Zheng and Yiwei Wang and Cheng Wang and Robert X. Gao},
url = {https://ieeexplore.ieee.org/abstract/document/9673794},
doi = {10.1109/TIM.2022.3141166},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Instrumentation and Measurement},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2203.
Zhang, Haoyu; Jin, Yaochu; Jin, Yaochu; Hao, Kuangrong
Evolutionary Search for Complete Neural Network Architectures with Partial Weight Sharing Journal Article
In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2022.
@article{9672175,
title = {Evolutionary Search for Complete Neural Network Architectures with Partial Weight Sharing},
author = {Haoyu Zhang and Yaochu Jin and Yaochu Jin and Kuangrong Hao},
url = {https://ieeexplore.ieee.org/abstract/document/9672175},
doi = {10.1109/TEVC.2022.3140855},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Evolutionary Computation},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2202.
Yipeng, Lv; Wenbing, Lv; Kaixuan, Han; Wentao, Tao; Ling, Zheng; Shizhuang, Weng; Lingsheng, Huang
In: Food Control, pp. 108819, 2022, ISSN: 0956-7135.
@article{YIPENG2022108819,
title = {Determination of wheat kernels damaged by Fusarium head blight using monochromatic images of effective wavelengths from hyperspectral imaging coupled with an architecture self-search deep network},
author = {Lv Yipeng and Lv Wenbing and Han Kaixuan and Tao Wentao and Zheng Ling and Weng Shizhuang and Huang Lingsheng},
url = {https://www.sciencedirect.com/science/article/pii/S0956713522000123},
doi = {https://doi.org/10.1016/j.foodcont.2022.108819},
issn = {0956-7135},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Food Control},
pages = {108819},
abstract = {Wheat kernels damaged by Fusarium head blight (FHB) lose moisture, protein and starch and carry dangerous toxins. Classification of degree of damage can be helpful to customise the use of wheat kernels, reduce financial losses and ensure grain safety. In this study, hyperspectral imaging (HSI) and deep learning network were explored to determine sound, mildly, moderately and severely damaged wheat kernels. Effective wavelengths (EWs) were selected from the reflectance spectroscopy of HSI images by ReliefF, uninformative variable elimination, random frog and shuffled frog leaping algorithm, and the monochromatic images of different combinations of EWs were adopted to develop the classification models combined with an architecture self-search deep network (ASSDN). ASSDN and the images composed of 941, 876 and 732 nm achieved the best determination with average accuracy of 100% and 98.31% in the training and prediction sets, respectively, outperforming other images or methods. And the average area under the curve of 0.9985 indicated its excellent robustness. Using images of sporadic wavelengths, the computation and operation complexity were evidently reduced, and a simple and custom-built instrument can be easily designed for practical recognition of FHB-damaged wheat kernels. Meanwhile, ASSDN can generate and optimise the high-performance classification network by itself, which is user–friendly and largely expands application potential of deep network.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Wheat kernels damaged by Fusarium head blight (FHB) lose moisture, protein and starch and carry dangerous toxins. Classification of degree of damage can be helpful to customise the use of wheat kernels, reduce financial losses and ensure grain safety. In this study, hyperspectral imaging (HSI) and deep learning network were explored to determine sound, mildly, moderately and severely damaged wheat kernels. Effective wavelengths (EWs) were selected from the reflectance spectroscopy of HSI images by ReliefF, uninformative variable elimination, random frog and shuffled frog leaping algorithm, and the monochromatic images of different combinations of EWs were adopted to develop the classification models combined with an architecture self-search deep network (ASSDN). ASSDN and the images composed of 941, 876 and 732 nm achieved the best determination with average accuracy of 100% and 98.31% in the training and prediction sets, respectively, outperforming other images or methods. And the average area under the curve of 0.9985 indicated its excellent robustness. Using images of sporadic wavelengths, the computation and operation complexity were evidently reduced, and a simple and custom-built instrument can be easily designed for practical recognition of FHB-damaged wheat kernels. Meanwhile, ASSDN can generate and optimise the high-performance classification network by itself, which is user–friendly and largely expands application potential of deep network.
2201.
Wu, Meng-Ting; Lin, Hung-I; Tsai, Chun-Wei
A Training-Free Genetic Neural Architecture Search Proceedings Article
In: Proceedings of the 2021 ACM International Conference on Intelligent Computing and Its Emerging Applications, pp. 65–70, Association for Computing Machinery, Jinan, China, 2022, ISBN: 9781450391603.
@inproceedings{10.1145/3491396.3506510,
title = {A Training-Free Genetic Neural Architecture Search},
author = {Meng-Ting Wu and Hung-I Lin and Chun-Wei Tsai},
url = {https://doi.org/10.1145/3491396.3506510},
doi = {10.1145/3491396.3506510},
isbn = {9781450391603},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 2021 ACM International Conference on Intelligent Computing and Its Emerging Applications},
pages = {65–70},
publisher = {Association for Computing Machinery},
address = {Jinan, China},
series = {ACM ICEA '21},
abstract = {The so-called neural architecture search (NAS) provides an alternative way to construct a "good neural architecture," which would normally outperform hand-made architectures, for solving complex problems without domain knowledge. However, a critical issue for most of the NAS techniques is in that it is computationally very expensive because several complete/partial training processes are involved in evaluating the goodness of a neural architecture during the process of NAS. To mitigate this problem for evaluating a single neural architecture found by the search algorithm of NAS, we present an efficient NAS in this study, called genetic algorithm and noise immunity for neural architecture search without training (GA-NINASWOT). The genetic algorithm (GA) in the proposed algorithm is used to search for high potential neural architectures while a modified scoring method based on the neural architecture search without training (NASWOT) is used to replace the training process of each neural architecture found by the GA for measuring its quality. To evaluate the performance of GA-NINASWOT, we compared it with several state-of-the-art NAS techniques, which include weight-sharing methods, non-weight-sharing methods, and NASWOT. Simulation results show that GA-NINASWOT outperforms all the other state-of-the-art weight-sharing methods and NASWOT compared in this study in terms of the accuracy and computational time. Moreover, GA-NINASWOT gives a result that is comparable to those found by the non-weight-sharing methods while reducing 99% of the search time.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The so-called neural architecture search (NAS) provides an alternative way to construct a "good neural architecture," which would normally outperform hand-made architectures, for solving complex problems without domain knowledge. However, a critical issue for most of the NAS techniques is in that it is computationally very expensive because several complete/partial training processes are involved in evaluating the goodness of a neural architecture during the process of NAS. To mitigate this problem for evaluating a single neural architecture found by the search algorithm of NAS, we present an efficient NAS in this study, called genetic algorithm and noise immunity for neural architecture search without training (GA-NINASWOT). The genetic algorithm (GA) in the proposed algorithm is used to search for high potential neural architectures while a modified scoring method based on the neural architecture search without training (NASWOT) is used to replace the training process of each neural architecture found by the GA for measuring its quality. To evaluate the performance of GA-NINASWOT, we compared it with several state-of-the-art NAS techniques, which include weight-sharing methods, non-weight-sharing methods, and NASWOT. Simulation results show that GA-NINASWOT outperforms all the other state-of-the-art weight-sharing methods and NASWOT compared in this study in terms of the accuracy and computational time. Moreover, GA-NINASWOT gives a result that is comparable to those found by the non-weight-sharing methods while reducing 99% of the search time.
2200.
Chen, Yi-Ping; Tsai, Chun-Wei
An Effective Neural Architecture Optimization Algorithm for CNN Based on Search Economics Proceedings Article
In: Proceedings of the 2021 ACM International Conference on Intelligent Computing and Its Emerging Applications, pp. 40–45, Association for Computing Machinery, Jinan, China, 2022, ISBN: 9781450391603.
@inproceedings{10.1145/3491396.3506505,
title = {An Effective Neural Architecture Optimization Algorithm for CNN Based on Search Economics},
author = {Yi-Ping Chen and Chun-Wei Tsai},
url = {https://doi.org/10.1145/3491396.3506505},
doi = {10.1145/3491396.3506505},
isbn = {9781450391603},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 2021 ACM International Conference on Intelligent Computing and Its Emerging Applications},
pages = {40–45},
publisher = {Association for Computing Machinery},
address = {Jinan, China},
series = {ACM ICEA '21},
abstract = {Developing a good convolutional neural network (CNN) architecture manually by trial and error is typically extremely time-consuming and requires a lot of effort. That is why several recent studies have attempted to develop ways to automatically construct a suitable CNN architecture. In this study, an effective neural architecture search (NAS) algorithm based on a novel metaheuristic algorithm, search economics (SE), is presented for CNN to improve the accuracy of image classification. The basic idea of the proposed algorithm is to use the "expected value" instead of the öbjective value" to evaluate a set of searched solutions, i.e., neural architectures in this case. As such, the searches of the proposed algorithm will trend to high potential regions in the solution space. Simulation results show that the proposed algorithm outperforms genetic algorithm-based NAS algorithm in terms of the accuracy, especially for complex image classification problems.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Developing a good convolutional neural network (CNN) architecture manually by trial and error is typically extremely time-consuming and requires a lot of effort. That is why several recent studies have attempted to develop ways to automatically construct a suitable CNN architecture. In this study, an effective neural architecture search (NAS) algorithm based on a novel metaheuristic algorithm, search economics (SE), is presented for CNN to improve the accuracy of image classification. The basic idea of the proposed algorithm is to use the "expected value" instead of the öbjective value" to evaluate a set of searched solutions, i.e., neural architectures in this case. As such, the searches of the proposed algorithm will trend to high potential regions in the solution space. Simulation results show that the proposed algorithm outperforms genetic algorithm-based NAS algorithm in terms of the accuracy, especially for complex image classification problems.
2199.
Pochelu, Pierrick; Petiton, Serge G.; Conche, Bruno
A Deep Neural Networks Ensemble Workflow from Hyperparameter Search to Inference Leveraging GPU Clusters Proceedings Article
In: International Conference on High Performance Computing in Asia-Pacific Region, pp. 61–71, Association for Computing Machinery, Virtual Event, Japan, 2022, ISBN: 9781450384988.
@inproceedings{10.1145/3492805.3492819,
title = {A Deep Neural Networks Ensemble Workflow from Hyperparameter Search to Inference Leveraging GPU Clusters},
author = {Pierrick Pochelu and Serge G. Petiton and Bruno Conche},
url = {https://doi.org/10.1145/3492805.3492819},
doi = {10.1145/3492805.3492819},
isbn = {9781450384988},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {International Conference on High Performance Computing in Asia-Pacific Region},
pages = {61–71},
publisher = {Association for Computing Machinery},
address = {Virtual Event, Japan},
series = {HPCAsia2022},
abstract = {Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid over-fitting but they are memory and time consuming approaches. Therefore, an ideal AutoML would produce in one single run time different ensembles regarding accuracy and inference speed. While previous works on AutoML focus to search for the best model to maximize its generalization ability, we rather propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. First, our extensive benchmarks show asynchronous Hyperband is an efficient and robust way to build a large number of diverse models to combine them. Then, a new ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles by controlling their computing cost. Finally, we propose a novel algorithm to optimize the inference of the DNNs ensemble in a GPU cluster based on allocation optimization. The produced AutoML with ensemble method shows robust results on two datasets using efficiently GPU clusters during both the training phase and the inference phase.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid over-fitting but they are memory and time consuming approaches. Therefore, an ideal AutoML would produce in one single run time different ensembles regarding accuracy and inference speed. While previous works on AutoML focus to search for the best model to maximize its generalization ability, we rather propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. First, our extensive benchmarks show asynchronous Hyperband is an efficient and robust way to build a large number of diverse models to combine them. Then, a new ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles by controlling their computing cost. Finally, we propose a novel algorithm to optimize the inference of the DNNs ensemble in a GPU cluster based on allocation optimization. The produced AutoML with ensemble method shows robust results on two datasets using efficiently GPU clusters during both the training phase and the inference phase.
2198.
Durbha, Krishna Srikar; Amuru, Saidhiraj
AutoML Models for Wireless Signals Classification and their effectiveness against Adversarial Attacks Proceedings Article
In: 2022 14th International Conference on COMmunication Systems NETworkS (COMSNETS), pp. 265-269, 2022.
@inproceedings{9668448,
title = {AutoML Models for Wireless Signals Classification and their effectiveness against Adversarial Attacks},
author = {Krishna Srikar Durbha and Saidhiraj Amuru},
url = {https://ieeexplore.ieee.org/abstract/document/9668448},
doi = {10.1109/COMSNETS53615.2022.9668448},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 14th International Conference on COMmunication Systems NETworkS (COMSNETS)},
pages = {265-269},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2197.
Jiang, Chunheng; Pedapati, Tejaswini; Chen, Pin-Yu; Sun, Yizhou; Gao, Jianxi
Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-04194,
title = {Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics},
author = {Chunheng Jiang and Tejaswini Pedapati and Pin-Yu Chen and Yizhou Sun and Jianxi Gao},
url = {https://arxiv.org/abs/2201.04194},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.04194},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2196.
Cai, Han; Lin, Ji; Han, Song
Chapter 4 - Efficient methods for deep learning Book Section
In: Davies, E. R.; Turk, Matthew A. (Ed.): Advanced Methods and Deep Learning in Computer Vision, pp. 159-190, Academic Press, 2022, ISBN: 978-0-12-822109-9.
@incollection{CAI2022159,
title = {Chapter 4 - Efficient methods for deep learning},
author = {Han Cai and Ji Lin and Song Han},
editor = {E. R. Davies and Matthew A. Turk},
url = {https://www.sciencedirect.com/science/article/pii/B9780128221099000138},
doi = {https://doi.org/10.1016/B978-0-12-822109-9.00013-8},
isbn = {978-0-12-822109-9},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Advanced Methods and Deep Learning in Computer Vision},
pages = {159-190},
publisher = {Academic Press},
series = {Computer Vision and Pattern Recognition},
abstract = {Deep neural networks (DNNs) have achieved unprecedented success in computer vision. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that can lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand to enable numerous edge AI applications. This chapter provides an overview of efficient deep learning methods. We start from introducing popular model compression methods, including pruning, factorization, and quantization. We then describe compact model design techniques including efficient convolution layers and representative efficient CNN architectures. Finally, to reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Deep neural networks (DNNs) have achieved unprecedented success in computer vision. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that can lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand to enable numerous edge AI applications. This chapter provides an overview of efficient deep learning methods. We start from introducing popular model compression methods, including pruning, factorization, and quantization. We then describe compact model design techniques including efficient convolution layers and representative efficient CNN architectures. Finally, to reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization.
2195.
Tan, Huixin; Lai, Jiewei; Liu, Yunbi; Song, Yuzhang; Wang, Jinliang; Chen, Mingyang; Yan, Yong; Zhong, Liming; Feng, Qianjin; Yang, Wei
Neural architecture search for real-time quality assessment of wearable multi-lead ECG on mobile devices Journal Article
In: Biomedical Signal Processing and Control, vol. 74, pp. 103495, 2022, ISSN: 1746-8094.
@article{TAN2022103495,
title = {Neural architecture search for real-time quality assessment of wearable multi-lead ECG on mobile devices},
author = {Huixin Tan and Jiewei Lai and Yunbi Liu and Yuzhang Song and Jinliang Wang and Mingyang Chen and Yong Yan and Liming Zhong and Qianjin Feng and Wei Yang},
url = {https://www.sciencedirect.com/science/article/pii/S1746809422000179},
doi = {https://doi.org/10.1016/j.bspc.2022.103495},
issn = {1746-8094},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Biomedical Signal Processing and Control},
volume = {74},
pages = {103495},
abstract = {Users need to pay for the medical services of the wearable ECG signals. The complicated interferences of wearable ECG may cause invalid medical diagnosis. To save medical expenses and reduce the workload of cardiologists, unacceptable wearable ECG should be filtered out on mobile devices in real time. Therefore, a robust and lightweight quality assessment model is essential. Existing convolutional neural network models are more robust than hand-crafted features based models, but most are too cumbersome to run on mobile devices. In this study, we utilize neural architecture search algorithm to search a network with a few parameters and low latency automatically for real-time quality assessment of wearable ECG data. To reach this goal, we construct a searching space with lightweight network blocks and add the hardware latency loss. We also propose a novel and effective strategy, any-lead, to assess the quality of all leads using a single model. We evaluated our method on a large-scale (10,709 signals) wearable 12-lead ECG dataset and a public dataset named Physionet Cinc Challenge 2011. The parameters and FLOPs of the architecture we searched were about 66.76 K and 36.44 M. Our model achieved excellent performance on the aforementioned datasets, with AUC of 98.32% and 97.64%, F1 scores of 94.36% and 93.52%, MCC of 84.74% and 83.22%, respectively, and with the inference time on an Android emulator of about 78 ms. Extensive experimental results demonstrate the effectiveness of our method in assessing the quality of all leads of wearable ECG data on mobile devices in real time.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Users need to pay for the medical services of the wearable ECG signals. The complicated interferences of wearable ECG may cause invalid medical diagnosis. To save medical expenses and reduce the workload of cardiologists, unacceptable wearable ECG should be filtered out on mobile devices in real time. Therefore, a robust and lightweight quality assessment model is essential. Existing convolutional neural network models are more robust than hand-crafted features based models, but most are too cumbersome to run on mobile devices. In this study, we utilize neural architecture search algorithm to search a network with a few parameters and low latency automatically for real-time quality assessment of wearable ECG data. To reach this goal, we construct a searching space with lightweight network blocks and add the hardware latency loss. We also propose a novel and effective strategy, any-lead, to assess the quality of all leads using a single model. We evaluated our method on a large-scale (10,709 signals) wearable 12-lead ECG dataset and a public dataset named Physionet Cinc Challenge 2011. The parameters and FLOPs of the architecture we searched were about 66.76 K and 36.44 M. Our model achieved excellent performance on the aforementioned datasets, with AUC of 98.32% and 97.64%, F1 scores of 94.36% and 93.52%, MCC of 84.74% and 83.22%, respectively, and with the inference time on an Android emulator of about 78 ms. Extensive experimental results demonstrate the effectiveness of our method in assessing the quality of all leads of wearable ECG data on mobile devices in real time.
2194.
Traoré, Kalifou René; Camero, Andrés; Zhu, Xiao Xiang
Landscape of Neural Architecture Search across sensors: how much do they differ ? Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-06321,
title = {Landscape of Neural Architecture Search across sensors: how much do they differ ?},
author = {Kalifou René Traoré and Andrés Camero and Xiao Xiang Zhu},
url = {https://arxiv.org/abs/2201.06321},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.06321},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2193.
Fedorov, Igor; Navarro, Ramon Matas; Tann, Hokchhay; Zhou, Chuteng; Mattina, Matthew; Whatmough, Paul N.
UDC: Unified DNAS for Compressible TinyML Models Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-05842,
title = {UDC: Unified DNAS for Compressible TinyML Models},
author = {Igor Fedorov and Ramon Matas Navarro and Hokchhay Tann and Chuteng Zhou and Matthew Mattina and Paul N. Whatmough},
url = {https://arxiv.org/abs/2201.05842},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.05842},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2192.
Liu, Xin; Zhao, Jianwei; Li, Jie; Cao, Bin; Lv, Zhihan
Federated Neural Architecture Search for Medical Data Security Journal Article
In: IEEE Transactions on Industrial Informatics, pp. 1-1, 2022.
@article{9684972,
title = {Federated Neural Architecture Search for Medical Data Security},
author = {Xin Liu and Jianwei Zhao and Jie Li and Bin Cao and Zhihan Lv},
url = {https://ieeexplore.ieee.org/abstract/document/9684972},
doi = {10.1109/TII.2022.3144016},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Industrial Informatics},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2191.
Guo, Qingbei; Wu, Xiao-Jun; Kittler, Josef; Feng, Zhiquan
Differentiable neural architecture learning for efficient neural networks Journal Article
In: Pattern Recognition, vol. 126, pp. 108448, 2022, ISSN: 0031-3203.
@article{GUO2022108448,
title = {Differentiable neural architecture learning for efficient neural networks},
author = {Qingbei Guo and Xiao-Jun Wu and Josef Kittler and Zhiquan Feng},
url = {https://www.sciencedirect.com/science/article/pii/S0031320321006245},
doi = {https://doi.org/10.1016/j.patcog.2021.108448},
issn = {0031-3203},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Pattern Recognition},
volume = {126},
pages = {108448},
abstract = {Efficient neural networks has received ever-increasing attention with the evolution of convolutional neural networks (CNNs), especially involving their deployment on embedded and mobile platforms. One of the biggest problems to obtaining such efficient neural networks is efficiency, even recent differentiable neural architecture search (DNAS) requires to sample a small number of candidate neural architectures for the selection of the optimal neural architecture. To address this computational efficiency issue, we introduce a novel architecture parameterization based on scaled sigmoid function, and propose a general Differentiable Neural Architecture Learning (DNAL) method to obtain efficient neural networks without the need to evaluate candidate neural networks. Specifically, for stochastic supernets as well as conventional CNNs, we build a new channel-wise module layer with the architecture components controlled by a scaled sigmoid function. We train these neural network models from scratch. The network optimization is decoupled into the weight optimization and the architecture optimization, which avoids the interaction between the two types of parameters and alleviates the vanishing gradient problem. We address the non-convex optimization problem of efficient neural networks by the continuous scaled sigmoid method instead of the common softmax method. Extensive experiments demonstrate our DNAL method delivers superior performance in terms of efficiency, and adapts to conventional CNNs (e.g., VGG16 and ResNet50), lightweight CNNs (e.g., MobileNetV2) and stochastic supernets (e.g., ProxylessNAS). The optimal neural networks learned by DNAL surpass those produced by the state-of-the-art methods on the benchmark CIFAR-10 and ImageNet-1K dataset in accuracy, model size and computational complexity. Our source code is available at https://github.com/QingbeiGuo/DNAL.git.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Efficient neural networks has received ever-increasing attention with the evolution of convolutional neural networks (CNNs), especially involving their deployment on embedded and mobile platforms. One of the biggest problems to obtaining such efficient neural networks is efficiency, even recent differentiable neural architecture search (DNAS) requires to sample a small number of candidate neural architectures for the selection of the optimal neural architecture. To address this computational efficiency issue, we introduce a novel architecture parameterization based on scaled sigmoid function, and propose a general Differentiable Neural Architecture Learning (DNAL) method to obtain efficient neural networks without the need to evaluate candidate neural networks. Specifically, for stochastic supernets as well as conventional CNNs, we build a new channel-wise module layer with the architecture components controlled by a scaled sigmoid function. We train these neural network models from scratch. The network optimization is decoupled into the weight optimization and the architecture optimization, which avoids the interaction between the two types of parameters and alleviates the vanishing gradient problem. We address the non-convex optimization problem of efficient neural networks by the continuous scaled sigmoid method instead of the common softmax method. Extensive experiments demonstrate our DNAL method delivers superior performance in terms of efficiency, and adapts to conventional CNNs (e.g., VGG16 and ResNet50), lightweight CNNs (e.g., MobileNetV2) and stochastic supernets (e.g., ProxylessNAS). The optimal neural networks learned by DNAL surpass those produced by the state-of-the-art methods on the benchmark CIFAR-10 and ImageNet-1K dataset in accuracy, model size and computational complexity. Our source code is available at https://github.com/QingbeiGuo/DNAL.git.
2190.
Lin, Qiuzhen; Fang, Zhixiong; Chen, Yi; Tan, Kay Chen; Li, Yun
Evolutionary Architectural Search for Generative Adversarial Networks Journal Article
In: IEEE Transactions on Emerging Topics in Computational Intelligence, pp. 1-12, 2022.
@article{9686068,
title = {Evolutionary Architectural Search for Generative Adversarial Networks},
author = {Qiuzhen Lin and Zhixiong Fang and Yi Chen and Kay Chen Tan and Yun Li},
url = {https://ieeexplore.ieee.org/abstract/document/9686068},
doi = {10.1109/TETCI.2021.3137377},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Emerging Topics in Computational Intelligence},
pages = {1-12},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2189.
Wang, Yabin; Ma, Zhiheng; Wei, Xing; Zheng, Shuai; Wang, Yaowei; Hong, Xiaopeng
ECCNAS: Efficient Crowd Counting Neural Architecture Search Journal Article
In: ACM Trans. Multimedia Comput. Commun. Appl., vol. 18, no. 1s, 2022, ISSN: 1551-6857.
@article{10.1145/3465455,
title = {ECCNAS: Efficient Crowd Counting Neural Architecture Search},
author = {Yabin Wang and Zhiheng Ma and Xing Wei and Shuai Zheng and Yaowei Wang and Xiaopeng Hong},
url = {https://doi.org/10.1145/3465455},
doi = {10.1145/3465455},
issn = {1551-6857},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {ACM Trans. Multimedia Comput. Commun. Appl.},
volume = {18},
number = {1s},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
abstract = {Recent solutions to crowd counting problems have already achieved promising performance across various benchmarks. However, applying these approaches to real-world applications is still challenging, because they are computation intensive and lack the flexibility to meet various resource budgets. In this article, we propose an efficient crowd counting neural architecture search (ECCNAS) framework to search efficient crowd counting network structures, which can fill this research gap. A novel search from pre-trained strategy enables our cross-task NAS to explore the significantly large and flexible search space with less search time and get more proper network structures. Moreover, our well-designed search space can intrinsically provide candidate neural network structures with high performance and efficiency. In order to search network structures according to hardwares with different computational performance, we develop a novel latency cost estimation algorithm in our ECCNAS. Experiments show our searched models get an excellent trade-off between computational complexity and accuracy and have the potential to deploy in practical scenarios with various resource budgets. We reduce the computational cost, in terms of multiply-and-accumulate (MACs), by up to 96% with comparable accuracy. And we further designed experiments to validate the efficiency and the stability improvement of our proposed search from pre-trained strategy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recent solutions to crowd counting problems have already achieved promising performance across various benchmarks. However, applying these approaches to real-world applications is still challenging, because they are computation intensive and lack the flexibility to meet various resource budgets. In this article, we propose an efficient crowd counting neural architecture search (ECCNAS) framework to search efficient crowd counting network structures, which can fill this research gap. A novel search from pre-trained strategy enables our cross-task NAS to explore the significantly large and flexible search space with less search time and get more proper network structures. Moreover, our well-designed search space can intrinsically provide candidate neural network structures with high performance and efficiency. In order to search network structures according to hardwares with different computational performance, we develop a novel latency cost estimation algorithm in our ECCNAS. Experiments show our searched models get an excellent trade-off between computational complexity and accuracy and have the potential to deploy in practical scenarios with various resource budgets. We reduce the computational cost, in terms of multiply-and-accumulate (MACs), by up to 96% with comparable accuracy. And we further designed experiments to validate the efficiency and the stability improvement of our proposed search from pre-trained strategy.
2188.
Kim, Youngeun; Li, Yuhang; Park, Hyoungseob; Venkatesha, Yeshwanth; Panda, Priyadarshini
Neural Architecture Search for Spiking Neural Networks Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-10355,
title = {Neural Architecture Search for Spiking Neural Networks},
author = {Youngeun Kim and Yuhang Li and Hyoungseob Park and Yeshwanth Venkatesha and Priyadarshini Panda},
url = {https://arxiv.org/abs/2201.10355},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.10355},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2187.
Rho, Daniel; Park, Jinhyeok; Ko, Jong Hwan
NAS-VAD: Neural Architecture Search for Voice Activity Detection Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-09032,
title = {NAS-VAD: Neural Architecture Search for Voice Activity Detection},
author = {Daniel Rho and Jinhyeok Park and Jong Hwan Ko},
url = {https://arxiv.org/abs/2201.09032},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.09032},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2186.
Shu, Yao; Dai, Zhongxiang; Wu, Zhaoxuan; Low, Bryan Kian Hsiang
Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-09785,
title = {Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search},
author = {Yao Shu and Zhongxiang Dai and Zhaoxuan Wu and Bryan Kian Hsiang Low},
url = {https://arxiv.org/abs/2201.09785},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.09785},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2185.
Chen, Mingzhe; Xiao, Xi; Zhang, Bin; Liu, Xinyu; Lu, Runiu
Neural Architecture Searching for Facial Attributes-based Depression Recognition Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-09799,
title = {Neural Architecture Searching for Facial Attributes-based Depression Recognition},
author = {Mingzhe Chen and Xi Xiao and Bin Zhang and Xinyu Liu and Runiu Lu},
url = {https://arxiv.org/abs/2201.09799},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.09799},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2184.
Mao, Pengli; Lin, Yan; Xue, Song; Zhang, Baochang
Remaining Useful Life Estimation of Aircraft Engines Using Differentiable Architecture Search Journal Article
In: Mathematics, vol. 10, no. 3, 2022, ISSN: 2227-7390.
@article{math10030352,
title = {Remaining Useful Life Estimation of Aircraft Engines Using Differentiable Architecture Search},
author = {Pengli Mao and Yan Lin and Song Xue and Baochang Zhang},
url = {https://www.mdpi.com/2227-7390/10/3/352},
doi = {10.3390/math10030352},
issn = {2227-7390},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Mathematics},
volume = {10},
number = {3},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2183.
Zhang, Xiaofan; Zhou, Zongwei; Chen, Deming; Wang, Yu Emma
AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-08539,
title = {AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models},
author = {Xiaofan Zhang and Zongwei Zhou and Deming Chen and Yu Emma Wang},
url = {https://arxiv.org/abs/2201.08539},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.08539},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2182.
Litany, Or; Maron, Haggai; Acuna, David; Kautz, Jan; Chechik, Gal; Fidler, Sanja
Federated Learning with Heterogeneous Architectures using Graph HyperNetworks Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-08459,
title = {Federated Learning with Heterogeneous Architectures using Graph HyperNetworks},
author = {Or Litany and Haggai Maron and David Acuna and Jan Kautz and Gal Chechik and Sanja Fidler},
url = {https://arxiv.org/abs/2201.08459},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.08459},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2181.
Hong, Zhenhou; Wang, Jianzong; Qu, Xiaoyang; Zhao, Chendong; Liu, Jie; Xiao, Jing
Neural Architecture Search as Self-assessor in Semi-supervised Learning Proceedings Article
In: Liao, Xiangke; Zhao, Wei; Chen, Enhong; Xiao, Nong; Wang, Li; Gao, Yang; Shi, Yinghuan; Wang, Changdong; Huang, Dan (Ed.): Big Data, pp. 95–107, Springer Singapore, Singapore, 2022, ISBN: 978-981-16-9709-8.
@inproceedings{10.1007/978-981-16-9709-8_7,
title = {Neural Architecture Search as Self-assessor in Semi-supervised Learning},
author = {Zhenhou Hong and Jianzong Wang and Xiaoyang Qu and Chendong Zhao and Jie Liu and Jing Xiao},
editor = {Xiangke Liao and Wei Zhao and Enhong Chen and Nong Xiao and Li Wang and Yang Gao and Yinghuan Shi and Changdong Wang and Dan Huang},
url = {https://link.springer.com/chapter/10.1007/978-981-16-9709-8_7},
isbn = {978-981-16-9709-8},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Big Data},
pages = {95--107},
publisher = {Springer Singapore},
address = {Singapore},
abstract = {Neural Architecture Search (NAS) forms powerful automatic learning, which has helped achieve remarkable performance in several applications in recent years. Previous research focused on NAS in standard supervised learning to explore its performance, requiring labeled data. In this paper, our goal is to examine the implementation of NAS with large amounts of unlabeled data. We propose the NAS as a self-assessor, called NAS-SA, by adding the consistency method and prior knowledge. We design an adaptive search strategy, a balanced search space, and a multi-object optimization to generate a robust and efficient small model in NAS-SA. The image and text classification tasks proved that our NAS-SA method had achieved the best performance.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) forms powerful automatic learning, which has helped achieve remarkable performance in several applications in recent years. Previous research focused on NAS in standard supervised learning to explore its performance, requiring labeled data. In this paper, our goal is to examine the implementation of NAS with large amounts of unlabeled data. We propose the NAS as a self-assessor, called NAS-SA, by adding the consistency method and prior knowledge. We design an adaptive search strategy, a balanced search space, and a multi-object optimization to generate a robust and efficient small model in NAS-SA. The image and text classification tasks proved that our NAS-SA method had achieved the best performance.
2180.
Gao, Feng; Song, Bin; Wang, Dan; Qin, Hao
MR-DARTS: Restricted connectivity differentiable architecture search in multi-path search space Journal Article
In: Neurocomputing, vol. 482, pp. 27-39, 2022, ISSN: 0925-2312.
@article{GAO202227,
title = {MR-DARTS: Restricted connectivity differentiable architecture search in multi-path search space},
author = {Feng Gao and Bin Song and Dan Wang and Hao Qin},
url = {https://www.sciencedirect.com/science/article/pii/S0925231222001047},
doi = {https://doi.org/10.1016/j.neucom.2022.01.080},
issn = {0925-2312},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Neurocomputing},
volume = {482},
pages = {27-39},
abstract = {Differentiable search methods can be used to find effective network architectures fast. However, these approaches are accompanied by low accuracy when evaluating a searched architecture, especially evaluating a searched architecture after transferring it to a different dataset. Two reasons can explain this phenomenon. The one is that the networks composed of cells have the depth gap in their structures between the search and evaluate stage. Another is that cells have insufficient ability to extract diverse features. This paper presents the Multi-path Restricted DARTS method to address these critical problems, using a multi-path search space and a restricted connectivity algorithm to perform a more exact search with limited resources. Restricted connectivity algorithm deepens the cells’ structure and makes cells more suitable for deep networks to bridge the depth gap. Multi-path search space enables cells to extract and fuse different-scales features to improve the representation capacity of a network. Our approach achieves state-of-the-art performance on CIFAR10 and CIFAR100 with the smallest parameters (only 2.5 M), demonstrating strong transfer learning ability in complex datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Differentiable search methods can be used to find effective network architectures fast. However, these approaches are accompanied by low accuracy when evaluating a searched architecture, especially evaluating a searched architecture after transferring it to a different dataset. Two reasons can explain this phenomenon. The one is that the networks composed of cells have the depth gap in their structures between the search and evaluate stage. Another is that cells have insufficient ability to extract diverse features. This paper presents the Multi-path Restricted DARTS method to address these critical problems, using a multi-path search space and a restricted connectivity algorithm to perform a more exact search with limited resources. Restricted connectivity algorithm deepens the cells’ structure and makes cells more suitable for deep networks to bridge the depth gap. Multi-path search space enables cells to extract and fuse different-scales features to improve the representation capacity of a network. Our approach achieves state-of-the-art performance on CIFAR10 and CIFAR100 with the smallest parameters (only 2.5 M), demonstrating strong transfer learning ability in complex datasets.
2179.
Ahn, Saehyun; Chang, Jung-Woo; Yoon, Hyeon-Seok; Kang, Suk-Ju
TouchNAS: Efficient Touch Detection Model Design Methodology for Resource-Constrained Devices Journal Article
In: IEEE Sensors Journal, pp. 1-1, 2022.
@article{9695433,
title = {TouchNAS: Efficient Touch Detection Model Design Methodology for Resource-Constrained Devices},
author = {Saehyun Ahn and Jung-Woo Chang and Hyeon-Seok Yoon and Suk-Ju Kang},
url = {https://ieeexplore.ieee.org/abstract/document/9695433},
doi = {10.1109/JSEN.2022.3147469},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Sensors Journal},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2178.
Hassantabar, Shayan; Dai, Xiaoliang; Jha, Niraj K.
CURIOUS: Efficient Neural Architecture Search Based on a Performance Predictor and Evolutionary Search Journal Article
In: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 1-1, 2022.
@article{9698855,
title = {CURIOUS: Efficient Neural Architecture Search Based on a Performance Predictor and Evolutionary Search},
author = {Shayan Hassantabar and Xiaoliang Dai and Niraj K. Jha},
url = {https://ieeexplore.ieee.org/abstract/document/9698855},
doi = {10.1109/TCAD.2022.3148202},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2177.
Li, Wei; Wen, Shiping; Shi, Kaibo; Yang, Yin; Huang, Tingwen
Neural Architecture Search with a Lightweight Transformer for Text-to-Image Synthesis Journal Article
In: IEEE Transactions on Network Science and Engineering, pp. 1-1, 2022.
@article{9699403,
title = {Neural Architecture Search with a Lightweight Transformer for Text-to-Image Synthesis},
author = {Wei Li and Shiping Wen and Kaibo Shi and Yin Yang and Tingwen Huang},
url = {https://ieeexplore.ieee.org/abstract/document/9699403},
doi = {10.1109/TNSE.2022.3147787},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Network Science and Engineering},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2176.
Pauletto, Loïc; Amini, Massih-Reza; Winckler, Nicolas
Self Semi Supervised Neural Architecture Search for Semantic Segmentation Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-12646,
title = {Self Semi Supervised Neural Architecture Search for Semantic Segmentation},
author = {Loïc Pauletto and Massih-Reza Amini and Nicolas Winckler},
url = {https://arxiv.org/abs/2201.12646},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.12646},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2175.
Xu, Dongkuan; Mukherjee, Subhabrata; Liu, Xiaodong; Dey, Debadeepta; Wang, Wenhui; Zhang, Xiang; Awadallah, Ahmed Hassan; Gao, Jianfeng
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-12507,
title = {AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models},
author = {Dongkuan Xu and Subhabrata Mukherjee and Xiaodong Liu and Debadeepta Dey and Wenhui Wang and Xiang Zhang and Ahmed Hassan Awadallah and Jianfeng Gao},
url = {https://arxiv.org/abs/2201.12507},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.12507},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2174.
Wei, Hui; Lee, Feifei; Hu, Chunyan; Chen, Qiu
MOO-DNAS: Efficient Neural Network Design via Differentiable Architecture Search Based on Multi-Objective Optimization Journal Article
In: IEEE Access, vol. 10, pp. 14195-14207, 2022.
@article{9698215,
title = {MOO-DNAS: Efficient Neural Network Design via Differentiable Architecture Search Based on Multi-Objective Optimization},
author = {Hui Wei and Feifei Lee and Chunyan Hu and Qiu Chen},
url = {https://ieeexplore.ieee.org/abstract/document/9698215},
doi = {10.1109/ACCESS.2022.3148323},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Access},
volume = {10},
pages = {14195-14207},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2173.
Guo, Bicheng; He, Shibo; Chen, Tao; Chen, Jiming; Ye, Peng
Neural Architecture Ranker Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-12725,
title = {Neural Architecture Ranker},
author = {Bicheng Guo and Shibo He and Tao Chen and Jiming Chen and Peng Ye},
url = {https://arxiv.org/abs/2201.12725},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.12725},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2172.
Na, Byunggook; Mok, Jisoo; Park, Seongsik; Lee, Dongjin; Choe, Hyeokjun; Yoon, Sungroh
AutoSNN: Towards Energy-Efficient Spiking Neural Networks Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-12738,
title = {AutoSNN: Towards Energy-Efficient Spiking Neural Networks},
author = {Byunggook Na and Jisoo Mok and Seongsik Park and Dongjin Lee and Hyeokjun Choe and Sungroh Yoon},
url = {https://arxiv.org/abs/2201.12738},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.12738},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2171.
Rguibi, Zakaria; Hajami, Abdelmajid; Dya, Zitouni
Äutomatic Searching of Deep Neural Networks for Medical Imaging Diagnostic Proceedings Article
In: Saidi, Rajaa; Bhiri, Brahim El; Maleh, Yassine; Mosallam, Ayman; Essaaidi, Mohammed (Ed.): Ädvanced Technologies for Humanity", pp. 129–140, Springer International Publishing, Cham, 2022, ISBN: 978-3-030-94188-8.
@inproceedings{10.1007/978-3-030-94188-8_13,
title = {Äutomatic Searching of Deep Neural Networks for Medical Imaging Diagnostic},
author = {Zakaria Rguibi and Abdelmajid Hajami and Zitouni Dya},
editor = {Rajaa Saidi and Brahim El Bhiri and Yassine Maleh and Ayman Mosallam and Mohammed Essaaidi},
isbn = {978-3-030-94188-8},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Ädvanced Technologies for Humanity"},
pages = {129--140},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Medical imaging diagnosis is the most assisted method to help physicians diagnose patient diseases using different imaging test modalities. But Imbalanced data is one of the biggest challenges in the field of medical imaging. To advance this field, this work proposes a framework that can be used to find the optimal DNN architectures for a database with the challenge of imbalanced datasets. In our paper, we present a framework for automatic deep neural network search for medical imaging diagnosis.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Medical imaging diagnosis is the most assisted method to help physicians diagnose patient diseases using different imaging test modalities. But Imbalanced data is one of the biggest challenges in the field of medical imaging. To advance this field, this work proposes a framework that can be used to find the optimal DNN architectures for a database with the challenge of imbalanced datasets. In our paper, we present a framework for automatic deep neural network search for medical imaging diagnosis.
2170.
Cardoso, Rui P.; Hart, Emma; Kurka, David Burth; Pitt, Jeremy V.
Augmenting Novelty Search with a Surrogate Model to Engineer Meta-Diversity in Ensembles of Classifiers Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-12896,
title = {Augmenting Novelty Search with a Surrogate Model to Engineer Meta-Diversity in Ensembles of Classifiers},
author = {Rui P. Cardoso and Emma Hart and David Burth Kurka and Jeremy V. Pitt},
url = {https://arxiv.org/abs/2201.12896},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.12896},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2169.
Mehta, Yash; White, Colin; Zela, Arber; Krishnakumar, Arjun; Zabergja, Guri; Moradian, Shakiba; Safari, Mahmoud; Yu, Kaicheng; Hutter, Frank
NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy Proceedings Article
In: ICLR 2022, 2022.
@inproceedings{DBLP:journals/corr/abs-2201-13396,
title = {NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy},
author = {Yash Mehta and Colin White and Arber Zela and Arjun Krishnakumar and Guri Zabergja and Shakiba Moradian and Mahmoud Safari and Kaicheng Yu and Frank Hutter},
url = {https://arxiv.org/abs/2201.13396},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {ICLR 2022},
journal = {CoRR},
volume = {abs/2201.13396},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2168.
Kang, Ziyang; Wang, Shiying; Wang, Lei; Li, Shiming; Qu, Lianhua; Xu, Weixia
Hardware-aware liquid state machine generation for 2D/3D Network-on-Chip platforms Journal Article
In: Journal of Systems Architecture, vol. 124, pp. 102429, 2022, ISSN: 1383-7621.
@article{KANG2022102429,
title = {Hardware-aware liquid state machine generation for 2D/3D Network-on-Chip platforms},
author = {Ziyang Kang and Shiying Wang and Lei Wang and Shiming Li and Lianhua Qu and Weixia Xu},
url = {https://www.sciencedirect.com/science/article/pii/S1383762122000297},
doi = {https://doi.org/10.1016/j.sysarc.2022.102429},
issn = {1383-7621},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Journal of Systems Architecture},
volume = {124},
pages = {102429},
abstract = {The liquid state machine (LSM) is a spiking neural network (SNN) that usually is offline mapped to an NoC-based neuromorphic processor to perform a specific task. The creation of these LSM models does not consider the structure of Network on Chip (NoC) which results in heavy communication pressure on the NoC. This paper proposes a hardware-aware generation framework for the LSM network by considering the spatial distribution of neurons in the NoC. It is the first time for the LSM generation work with combining the characteristics of NoC. This framework also adopts the heuristic algorithm to search the hyperparameter for the LSM networks to achieve state-of-art accuracy. It also reduces the spikes generated by those LSM models. It keeps the communication between neurons within cores as much as possible, which could reduce the communication between cores effectively and improve the performance of NoC, including reducing the traffic flow, reducing the average latency, improving the throughput and reducing the total running time.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The liquid state machine (LSM) is a spiking neural network (SNN) that usually is offline mapped to an NoC-based neuromorphic processor to perform a specific task. The creation of these LSM models does not consider the structure of Network on Chip (NoC) which results in heavy communication pressure on the NoC. This paper proposes a hardware-aware generation framework for the LSM network by considering the spatial distribution of neurons in the NoC. It is the first time for the LSM generation work with combining the characteristics of NoC. This framework also adopts the heuristic algorithm to search the hyperparameter for the LSM networks to achieve state-of-art accuracy. It also reduces the spikes generated by those LSM models. It keeps the communication between neurons within cores as much as possible, which could reduce the communication between cores effectively and improve the performance of NoC, including reducing the traffic flow, reducing the average latency, improving the throughput and reducing the total running time.
2167.
Hu, Xing; Liang, Ling; Chen, Xiaobing; Deng, Lei; Ji, Yu; Ding, Yufei; Du, Zidong; Guo, Qi; Sherwood, Tim; Xie, Yuan
A Systematic View of Model Leakage Risks in Deep Neural Network Systems Journal Article
In: IEEE Transactions on Computers, pp. 1-1, 2022.
@article{9705069,
title = {A Systematic View of Model Leakage Risks in Deep Neural Network Systems},
author = {Xing Hu and Ling Liang and Xiaobing Chen and Lei Deng and Yu Ji and Yufei Ding and Zidong Du and Qi Guo and Tim Sherwood and Yuan Xie},
url = {https://ieeexplore.ieee.org/abstract/document/9705069},
doi = {10.1109/TC.2022.3148235},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Computers},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2166.
Dushatskiy, Arkadiy; Alderliesten, Tanja; Bosman, Peter A. N.
Heed the Noise in Performance Evaluations in Neural Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2202-02078,
title = {Heed the Noise in Performance Evaluations in Neural Architecture Search},
author = {Arkadiy Dushatskiy and Tanja Alderliesten and Peter A. N. Bosman},
url = {https://arxiv.org/abs/2202.02078},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2202.02078},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2165.
Cho, Hyunghun; Shin, Jungwook; Rhee, Wonjong
B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2202-03005,
title = {B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search},
author = {Hyunghun Cho and Jungwook Shin and Wonjong Rhee},
url = {https://arxiv.org/abs/2202.03005},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2202.03005},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2164.
Cho, Minsu; Joshi, Ameya; Garg, Siddharth; Reagen, Brandon; Hegde, Chinmay
Selective Network Linearization for Efficient Private Inference Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2202-02340,
title = {Selective Network Linearization for Efficient Private Inference},
author = {Minsu Cho and Ameya Joshi and Siddharth Garg and Brandon Reagen and Chinmay Hegde},
url = {https://arxiv.org/abs/2202.02340},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2202.02340},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2163.
Mazumder, Arnab Neelim; Mohsenin, Tinoosh
A Fast Network Exploration Strategy to Profile Low Energy Consumption for Keyword Spotting Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2202-02361,
title = {A Fast Network Exploration Strategy to Profile Low Energy Consumption for Keyword Spotting},
author = {Arnab Neelim Mazumder and Tinoosh Mohsenin},
url = {https://arxiv.org/abs/2202.02361},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2202.02361},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2162.
Hassanzadeh, Tahereh; Essam, Daryl; Sarker, Ruhul
EvoDCNN: An Evolutionary Deep Convolutional Neural Network for Image Classification Journal Article
In: Neurocomputing, 2022, ISSN: 0925-2312.
@article{HASSANZADEH2022,
title = {EvoDCNN: An Evolutionary Deep Convolutional Neural Network for Image Classification},
author = {Tahereh Hassanzadeh and Daryl Essam and Ruhul Sarker},
url = {https://www.sciencedirect.com/science/article/pii/S092523122200145X},
doi = {https://doi.org/10.1016/j.neucom.2022.02.003},
issn = {0925-2312},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Neurocomputing},
abstract = {Developing Deep Convolutional Neural Networks (DCNNs) for image classification is a complicated task that needs considerable effort and knowledge. By employing an evolutionary computation approach, one can automatically generate the network models. However, the Neuroevolution is computationally expensive, and in some cases it needs hundreds of GPU days for training. Therefore, there is a need to find optimum Neuroevolutionary models with minimum computation to deal with this problem. In this paper, by utilising a Genetic Algorithm (GA), we introduce EvoDCNN, as a block-based evolutionary model for developing an evolutionary deep convolutional network for image classification. Such that by using the proposed fixed-length encoding model, we can generate variable-length networks with high accuracy while using less computation. The proposed model by utilising a straightforward evolutionary framework is able to establish small networks with high classification accuracy. Eight datasets: CIFAR10, MNIST, and six versions of EMNIST, that include balanced and unbalanced datasets, are used for evaluation of the proposed model. We did a comprehensive evaluation where we compared the results with many previous works, and outperformed the previous state-of-the-art accuracy for classification of five of the datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Developing Deep Convolutional Neural Networks (DCNNs) for image classification is a complicated task that needs considerable effort and knowledge. By employing an evolutionary computation approach, one can automatically generate the network models. However, the Neuroevolution is computationally expensive, and in some cases it needs hundreds of GPU days for training. Therefore, there is a need to find optimum Neuroevolutionary models with minimum computation to deal with this problem. In this paper, by utilising a Genetic Algorithm (GA), we introduce EvoDCNN, as a block-based evolutionary model for developing an evolutionary deep convolutional network for image classification. Such that by using the proposed fixed-length encoding model, we can generate variable-length networks with high accuracy while using less computation. The proposed model by utilising a straightforward evolutionary framework is able to establish small networks with high classification accuracy. Eight datasets: CIFAR10, MNIST, and six versions of EMNIST, that include balanced and unbalanced datasets, are used for evaluation of the proposed model. We did a comprehensive evaluation where we compared the results with many previous works, and outperformed the previous state-of-the-art accuracy for classification of five of the datasets.
2161.
Rabczuk, Timon; Guo, Hongwei; Zhuang, Xiaoying; Chen, Pengwan; Alajlan, Naif
Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media Journal Article
In: Engineering with Computers, vol. 2022, pp. 1 – 26, 2022.
@article{RabczukGuoZhuangetal.2022,
title = {Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media},
author = {Timon Rabczuk and Hongwei Guo and Xiaoying Zhuang and Pengwan Chen and Naif Alajlan},
url = {https://link.springer.com/article/10.1007/s00366-021-01586-2},
doi = {10.1007/s00366-021-01586-2},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Engineering with Computers},
volume = {2022},
pages = {1 -- 26},
keywords = {},
pubstate = {published},
tppubtype = {article}
}