AutoML | Literature on Neural Architecture Search

1660.

Nguyen, N.; Chang, M. J.

CSNAS: Contrastive Self-supervised Learning Neural Architecture Search via Sequential Model-Based Optimization Journal Article

In: IEEE Transactions on Artificial Intelligence, vol. 1, no. 01, pp. 1-1, 2021.

Links | BibTeX

1659.

Chen, Zerui; Huang, Yan; Yu, Hongyuan; Wang, Liang

Learning a Robust Part-Aware Monocular 3D Human Pose Estimator via Neural Architecture Search Journal Article

In: International Journal of Computer Vision (2021), 2021.

Links | BibTeX

1658.

Burrello, Alessio; Pagliari, Daniele Jahier; Risso, Matteo; Benatti, Simone; Macii, Enrico; Benini, Luca; Poncino, Massimo

Q-PPG: Energy-Efficient PPG-based Heart Rate Monitoring on Wearable Devices Journal Article

In: IEEE Trans Biomed Circuits Syst ., 2021.

Links | BibTeX

1657.

Wang, Ning; Gao, Yang; Chen, Hao; Wang, Peng; Tian, Zhi; Shen, Chunhua; Zhang, Yanning

NAS-FCOS: Efficient Search for Object Detection Architectures Journal Article

In: International Journal of Computer Vision , vol. 129, pp. 3299–3312, 2021.

Links | BibTeX

1656.

Oyelade, Olaide N.; Ezugwu, Absalom E.

A bioinspired neural architecture search based convolutional neural network for breast cancer detection using histopathology images Journal Article

In: Scientific Reports, 2021.

Links | BibTeX

1655.

Wang, Yaoming; Liu, Yuchen; Dai, Wenrui; Li, Chenglin; Zou, Junni; Xiong, Hongkai

Learning Latent Architectural Distribution in Differentiable Neural Architecture Search via Variational Information Maximization Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12312-12321, 2021.

Links | BibTeX

1654.

Chen, Ziye; Zhan, Yibing; Yu, Baosheng; Gong, Mingming; Du, Bo

Not All Operations Contribute Equally: Hierarchical Operation-Adaptive Predictor for Neural Architecture Search Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10508-10517, 2021.

Links | BibTeX

1653.

Zeng, Dan; Huang, Yuhang; Bao, Qian; Zhang, Junjie; Su, Chi; Liu, Wu

Neural Architecture Search for Joint Human Parsing and Pose Estimation Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11385-11394, 2021.

Links | BibTeX

1652.

Chatzianastasis, Michail; Dasoulas, George; Siolas, Georgios; Vazirgiannis, Michalis

Graph-Based Neural Architecture Search With Operation Embeddings Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 393-402, 2021.

Links | BibTeX

1651.

Hu, Xiaobin; Ren, Wenqi; Yu, Kaicheng; Zhang, Kaihao; Cao, Xiaochun; Liu, Wei; Menze, Bjoern

Pyramid Architecture Search for Real-Time Image Deblurring Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4298-4307, 2021.

Links | BibTeX

1650.

Chu, Xiangxiang; Zhang, Bo; Li, Qingyuan; Xu, Ruijun; Li, Xudong

SCARLET-NAS: Bridging the Gap Between Stability and Scalability in Weight-Sharing Neural Architecture Search Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 317-325, 2021.

Links | BibTeX

1649.

Xue, Song; Wang, Runqi; Zhang, Baochang; Wang, Tian; Guo, Guodong; Doermann, David

IDARTS: Interactive Differentiable Architecture Search Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1163-1172, 2021.

Links | BibTeX

1648.

Shen, Mingzhu; Liang, Feng; Gong, Ruihao; Li, Yuhang; Li, Chuming; Lin, Chen; Yu, Fengwei; Yan, Junjie; Ouyang, Wanli

Once Quantization-Aware Training: High Performance Extremely Low-Bit Architecture Search Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5340-5349, 2021.

Links | BibTeX

1647.

Lin, Ming; Wang, Pichao; Sun, Zhenhong; Chen, Hesen; Sun, Xiuyu; Qian, Qi; Li, Hao; Jin, Rong

Zen-NAS: A Zero-Shot NAS for High-Performance Image Recognition Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 347-356, 2021.

Links | BibTeX

1646.

Zhou, Qinqin; Zheng, Xiawu; Cao, Liujuan; Zhong, Bineng; Xi, Teng; Zhang, Gang; Ding, Errui; Xu, Mingliang; Ji, Rongrong

EC-DARTS: Inducing Equalized and Consistent Optimization Into DARTS Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11986-11995, 2021.

Links | BibTeX

1645.

Rana, Amrita; Kim, Kyung Ki

Robust architecture search using network adaptation Journal Article

In: Journal of Sensor Science and Technology, vol. 30, no. 5, pp. 290-294, 2021.

Links | BibTeX

1644.

Yan, Kefan; Wang, Shunlong; Guan, Yuwei; Jiang, Ailian

TNAS: Neural Network Architecture Search Sampling Module Based on Variational Autoencoder Journal Article

In: CONVERTER, pp. 288-296, 2021.

Links | BibTeX

1643.

Hao, Jie; Cai, Zhiling; Li, Ruijia; Zhu, William

Saliency: a new selection criterion of important architectures in neural architecture search Journal Article

In: Neural Computing and Applications, 2021.

Links | BibTeX

1642.

Jia, Hong; Hu, Jiawei; Hu, Wen

SwingNet: Ubiquitous Fine-Grained Swing Tracking Framework via Stochastic Neural Architecture Search and Adversarial Learning Journal Article

In: Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 5, no. 3, 2021.

Abstract | Links | BibTeX

@article{10.1145/3478082,

title = {SwingNet: Ubiquitous Fine-Grained Swing Tracking Framework via Stochastic Neural Architecture Search and Adversarial Learning},

author = {Hong Jia and Jiawei Hu and Wen Hu},

url = {https://doi.org/10.1145/3478082},

doi = {10.1145/3478082},

year  = {2021},

date = {2021-09-01},

urldate = {2021-09-01},

journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.},

volume = {5},

number = {3},

publisher = {Association for Computing Machinery},

address = {New York, NY, USA},

abstract = {Sports analytics in the wild (i.e., ubiquitously) is a thriving industry. Swing tracking 

is a key feature in sports analytics. Therefore, a centimeter-level tracking resolution 

solution is required. Recent research has explored deep neural networks for sensor 

fusion to produce consistent swing-tracking performance. This is achieved by combining 

the advantages of two sensor modalities (IMUs and depth sensors) for golf swing tracking. 

Here, the IMUs are not affected by occlusion and can support high sampling rates. 

Meanwhile, depth sensors produce significantly more accurate motion measurements than 

those produced by IMUs. Nevertheless, this method can be further improved in terms 

of accuracy and lacking information for different domains (e.g., subjects, sports, 

and devices). Unfortunately, designing a deep neural network with good performance 

is time consuming and labor intensive, which is challenging when a network model is 

deployed to be used in new settings. To this end, we propose a network based on Neural 

Architecture Search (NAS), called SwingNet, which is a regression-based automatic 

generated deep neural network via stochastic neural network search. The proposed network 

aims to learn the swing tracking feature for better prediction automatically. Furthermore, 

SwingNet features a domain discriminator by using unsupervised learning and adversarial 

learning to ensure that it can be adaptive to unobserved domains. We implemented SwingNet 

prototypes with a smart wristband (IMU) and smartphone (depth sensor), which are ubiquitously 

available. They enable accurate sports analytics (e.g., coaching, tracking, analysis 

and assessment) in the wild.Our comprehensive experiment shows that SwingNet achieves 

less than 10 cm errors of swing tracking with a subject-independent model covering 

multiple sports (e.g., golf and tennis) and depth sensor hardware, which outperforms 

state-of-the-art approaches.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

1641.

Cazasnoves, Anthony; Ganaye, Pierre-Antoine; Sanchis, Kévin; Ceillier, Tugdual

Neural Architecture Search in operational context: a remote sensing case-study Technical Report

Preligens 2021.

Links | BibTeX

1640.

Yang, Xin; Fan, Jiangfeng; Wu, Chenhuan; Zhou, Dake; Li, Tao

NasmamSR: a fast image super-resolution network based on neural architecture search and multiple attention mechanism Journal Article

In: Multimedia Systems (2021), 2021.

Links | BibTeX

1639.

Wendlinger, Lorenz; Berndl, Emanuel; Granitzer, Michael

Methods for Automatic Machine-Learning Workflow Analysis Proceedings Article

In: ECML PKDD 2021, 2021.

Links | BibTeX

1638.

Zhao, Zimin; Kang, Ying; Hou, Aiqin; Gan, Daguang

SP-DARTS: Synchronous Progressive Differentiable Neural Architecture Search for Image Classification Journal Article

In: IEICE TRANSACTIONS on Information and Systems , vol. E104-D , no. 8 , pp. 1232-1238, 2021, ISBN: 1745-1361.

Links | BibTeX

1637.

Ding, Yuhui; Yao, Quanming; Zhao, Huan; Zhang, Tong

DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks Proceedings Article

In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery &amp$mathsemicolon$ Data Mining, ACM, 2021.

Links | BibTeX

1636.

Zhu, Yanming; Meijering, Erik

Automatic Improvement of Deep Learning Based Cell Segmentation in Time-Lapse Microscopy by Neural Architecture Search Journal Article

In: Bioinformatics, 2021.

Links | BibTeX

1635.

Zhao, Jiejie; Lv, Weifeng; Du, Bowen; Ye, Junchen; Sun, Leilei; Xiong, Guixi

Deep multi-task learning with flexible and compact architecture search Journal Article

In: International Journal of Data Science and Analytics , 2021.

Links | BibTeX

1634.

Nguyen, Vu; Le, Tam; Yamada, Makoto; Osborne, Michael A

Optimal Transport Kernels for Sequential and Parallel Neural Architecture Search Proceedings Article

In: Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, 2021.

Links | BibTeX

1633.

Ünal, Hamit Taner; Başçiftçi, Fatih

Evolutionary design of neural network architectures: a review of three decades of research Journal Article

In: Artificial Intelligence Review, 2021.

Links | BibTeX

1632.

Dimanov, Daniel; Balaguer-Ballester, Emili; Singleton, Colin; Rostami, Shahin

Moncae: Multi-Objective Neuroevolution of Convolutional Autoencoders Proceedings Article

In: 2nd Workshop on Neural Architecture Search at ICLR 2021, 2021.

Links | BibTeX

1631.

Lin, Yi; Liu, Yanfei; Liu, Jingguang; Liu, Guocai; Ma, Kai; Zheng, Yefeng

LE-NAS: Learning-based Ensemble with Neural Architecture Search for 3D Radiotherapy Dose Prediction Journal Article

In: 2021.

Links | BibTeX

1630.

White, Colin; Nolen, Sam; Savani, Yash

Exploring the Loss Landscape in Neural Architecture Search Proceedings Article

In: 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021), 2021.

Links | BibTeX

1629.

Sääw, Daniel; Nijweide, Frederik

Exploring the feasibility of Weight Agnostic Neural Networks for low-end hardware Technical Report

2021.

Links | BibTeX

1628.

Tian, Ye; Peng, Shichen; Yang, Shangshang; Zhang, Xingyi; Tan, Kay Chen; Jin, Yaochu

Action Command Encoding for SurrogateAssisted Neural Architecture Search Technical Report

2021.

Links | BibTeX

1627.

Siems, Julien; Klein, Aaron; Archambeau, Cedric; Mahsereci, Maren

Dynamic Pruning of a Neural Networkvia Gradient Signal-to-Noise Ratio Proceedings Article

In: 8th ICML Workshop on Automated Machine Learning (2021), 2021.

Links | BibTeX

1626.

Yu, Kaicheng

From Human-Designed Convolutional Neural Networks Towards Robust Neural Architecture Search PhD Thesis

2021.

Abstract | Links | BibTeX

@phdthesis{YuPhD2021,

title = {From Human-Designed Convolutional Neural Networks Towards Robust Neural Architecture Search },

author = {Yu, Kaicheng},

url = {https://infoscience.epfl.ch/record/287432},

doi = { 10.5075/epfl-thesis-8035},

year  = {2021},

date = {2021-07-01},

abstract = {Artificial intelligence has been an ultimate design goal since the inception of computers decades ago. Among the many attempts towards general artificial intelligence, modern machine learning successfully tackles many complex problems thanks to the progress in deep learning, and in particular in convolutional neural networks (CNN). To design a CNN for a specific task, one common approach consists of adapting the heuristics from the pre-deep-learning era to the CNN domain. In the first part of this thesis, we introduce two methods that follow this approach: i) We build a covariance descriptor, i.e., a local descriptor that is suitable for texture recognition, to replace the first-order fully connected layers in an ordinary CNN, showing that such a descriptor yields state-of-the-art performance on many fine-grained image classification tasks with orders of magnitude fewer feature dimensions; ii) we develop a light-weight recurrent U-Net for image semantic segmentation, inspired by the biological eye saccadic movements, that yields real-time predictions on devices with limited computational resources. As most methods pre-dating automatic machine learning (AutoML), the two above-mentioned CNNs were human-designed. In the past few years, however, neural architecture search~(NAS), which aims to facilitate the design of deep networks for new tasks, has drawn an increasing attention. In this context, the weight-sharing approach, which consists of utilizing a super-net to encompass all possible architectures within a search space, has become a de facto standard in NAS because it enables the search to be done on commodity hardware. In the second part of this thesis, we then provide an in-depth study of recent weight-sharing NAS algorithms. First, we discover a phenomenon in the weight-sharing NAS training pipeline, which we dub multi-model forgetting, that negatively impacts the super-net quality, and propose a statistically motivated approach to address it. Subsequently, we find that (i) on average, many popular NAS algorithms perform similarly to a random architecture sampling policy; (ii) the widely-adopted weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. We then further decouple weight sharing from the NAS sampling policy, and isolate 14 factors that play a key role in the success of super-net training. Finally, to improve the super-net quality, we propose a regularization term that aims to maximize the correlation between the performance rankings of the super-net and of the stand-alone architectures using a small set of landmark architectures.},

keywords = {},

pubstate = {published},

tppubtype = {phdthesis}

}

Close

Artificial intelligence has been an ultimate design goal since the inception of computers decades ago. Among the many attempts towards general artificial intelligence, modern machine learning successfully tackles many complex problems thanks to the progress in deep learning, and in particular in convolutional neural networks (CNN). To design a CNN for a specific task, one common approach consists of adapting the heuristics from the pre-deep-learning era to the CNN domain. In the first part of this thesis, we introduce two methods that follow this approach: i) We build a covariance descriptor, i.e., a local descriptor that is suitable for texture recognition, to replace the first-order fully connected layers in an ordinary CNN, showing that such a descriptor yields state-of-the-art performance on many fine-grained image classification tasks with orders of magnitude fewer feature dimensions; ii) we develop a light-weight recurrent U-Net for image semantic segmentation, inspired by the biological eye saccadic movements, that yields real-time predictions on devices with limited computational resources. As most methods pre-dating automatic machine learning (AutoML), the two above-mentioned CNNs were human-designed. In the past few years, however, neural architecture search~(NAS), which aims to facilitate the design of deep networks for new tasks, has drawn an increasing attention. In this context, the weight-sharing approach, which consists of utilizing a super-net to encompass all possible architectures within a search space, has become a de facto standard in NAS because it enables the search to be done on commodity hardware. In the second part of this thesis, we then provide an in-depth study of recent weight-sharing NAS algorithms. First, we discover a phenomenon in the weight-sharing NAS training pipeline, which we dub multi-model forgetting, that negatively impacts the super-net quality, and propose a statistically motivated approach to address it. Subsequently, we find that (i) on average, many popular NAS algorithms perform similarly to a random architecture sampling policy; (ii) the widely-adopted weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. We then further decouple weight sharing from the NAS sampling policy, and isolate 14 factors that play a key role in the success of super-net training. Finally, to improve the super-net quality, we propose a regularization term that aims to maximize the correlation between the performance rankings of the super-net and of the stand-alone architectures using a small set of landmark architectures.

Close

1625.

Coquelin, Daniel; Sedona, Rocco; Riedel, Morris; Götz, Markus

Evolutionary Optimization of Neural Architectures in Remote Sensing Classification Problems Technical Report

2021.

Links | BibTeX

1624.

Jin, Yaochu; Wang, Handing; Sun, Chaoli

Surrogate-Assisted Evolutionary Neural Architecture Search Proceedings Article

In: Data-Driven Evolutionary Optimization, pp. 373-387, 2021, ISBN: 978-3-030-74640-7.

Links | BibTeX

1623.

Zhu, Wei; Ni, Yuan; Wang, Xiaoling; Xie, Guotong; Zhang, Fang

Discovering Better Model Architectures for Medical Query Understanding Proceedings Article

In: Proceedings of NAACL HLT 2021: IndustryTrack Papers, pp. 230-237, 2021.

Links | BibTeX

1622.

Tsokov, Stefan; Lazarova, Milena; Aleksieva-Petrova, Adelina

An Evolutionary Approach to the Design of Convolutional Neural Networks for Human Activity Recognition Journal Article

In: Indian Journal of Computer Science and Engineering, 2021.

Links | BibTeX

1621.

Geraeinejad, V.; Seenan, S.; Modarressi, M.; Daneshtalab, M.

RoCo-NAS : Robust and Compact Neural Architecture Search Proceedings Article

In: The international joint conference on neural networks IJCNN, 2021.

Links | BibTeX

1620.

Liu, Shaoli; Zheng, Chengjian; Lu, Kaidi; Gao, Si; Wang, Ning; Wang, Bofei; Zhang, Diankai; Zhang, Xiaofeng; Xu, Tianyu

EVSRNet: Efficient Video Super-Resolution With Neural Architecture Search Proceedings Article

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 2480-2485, 2021.

Links | BibTeX

1619.

Ma, Xiaohan; Si, Chang; Wang, Ying; Liu, Cheng; Zhang1, Lei

NASA: Accelerating Neural Network Design with a NAS Processor Proceedings Article

In: 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), 2021.

Links | BibTeX

1618.

Dai, Xiaoliang; Wan, Alvin; Zhang, Peizhao; Wu, Bichen; He, Zijian; Wei, Zhen; Chen, Kan; Tian, Yuandong; Yu, Matthew; Vajda, Peter; Gonzalez, Joseph E.

FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining Book Section

In: CVPR2021, 2021.

Links | BibTeX

1617.

Huang, Shenyang; Francois-Lavet, Vincent; Rabusseau, Guillaume

Understanding Capacity Saturation in Incremental Learning Proceedings Article

In: The 34th Canadian Conference on Artificial Intelligence, 2021.

Links | BibTeX

1616.

Chen, Wuyang; Gong, Xinyu; Wang, Zhangyang

Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective Proceedings Article

In: ICLR 2021, 2021.

Links | BibTeX

1615.

Yang, Yuxuan; Gao, Zhongke; Li, Yanli; Wang, He

A CNN identified by reinforcement learning-based optimization framework for EEG-based state evaluation Journal Article

In: Journal of Neural Engineering, vol. 18, no. 4, pp. 046059, 2021.

Abstract | Links | BibTeX

@article{Yang_2021,

title = {A CNN identified by reinforcement learning-based optimization framework for EEG-based state evaluation},

author = {Yuxuan Yang and Zhongke Gao and Yanli Li and He Wang},

url = {https://doi.org/10.1088/1741-2552/abfa71},

doi = {10.1088/1741-2552/abfa71},

year  = {2021},

date = {2021-05-01},

journal = {Journal of Neural Engineering},

volume = {18},

number = {4},

pages = {046059},

publisher = {IOP Publishing},

abstract = {Objective. Electroencephalogram (EEG) data, as a kind of complex time-series, is one of the most widely-used information measurements for evaluating human psychophysiological states. Recently, numerous works applied deep learning techniques, especially the convolutional neural network (CNN), into EEG-based research. The design of the hyper-parameters of the CNN model has a great influence on the performance of the model. Therefore, automatically designing these hyper-parameters can save the time and labor of experts. This leads to the appearance of the neural architecture search technique. In this paper, we propose a reinforcement learning (RL)-based step-by-step framework to efficiently search for CNN models. Approach. Specifically, the deep Q network in RL is first used to determine the depth of convolutional layers and the connection modes among layers. Then particle swarm optimization algorithm is used to fine-tune the number and size of convolution kernels. Through this step-by-step strategy, the search space can be narrowed in each step for saving the overall time cost. This framework is employed for both EEG-based sleep stage classification and driver drowsiness evaluation tasks. Main results. The results show that compared with state-of-the-art methods, the high-performance CNN models identified by the proposed optimization framework, can achieve high overall accuracy and better root mean squared error in the two tasks. Significance. Therefore, the proposed optimization framework has a great potential to provide high-performance results for other kinds of classification and prediction tasks. In this way, it can greatly save researchers’ time cost and promote broader applications of CNNs.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}