Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2023
2464.
Zhao, Junbo; Ning, Xuefei; Liu, Enshu; Ru, Binxin; Zhou, Zixuan; Zhao, Tianchen; Chen, Chen; Zhang, Jiajin; Liao, Qingmin; Wang, Yu
Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS “Cold-Start” Technical Report
2023.
@techreport{Zhao22,
title = {Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS “Cold-Start”},
author = {Junbo Zhao and Xuefei Ning and Enshu Liu and Binxin Ru and Zixuan Zhou and Tianchen Zhao and Chen Chen and Jiajin Zhang and Qingmin Liao and Yu Wang},
url = {https://nicsefc.ee.tsinghua.edu.cn/nics_file/pdf/4208e529-772e-4977-be31-0b7cc4c7a9fc.pdf},
year = {2023},
date = {2023-12-20},
urldate = {2023-12-20},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2463.
Z, Zhang; I, Joe
CAM-NAS: A Fast Model for Neural Architecture Search based on Class Activation Map Journal Article
In: Research Square, 2023.
@article{ZhangRS3,
title = {CAM-NAS: A Fast Model for Neural Architecture Search based on Class Activation Map},
author = {Zhang Z and Joe I
},
url = {https://europepmc.org/article/ppr/ppr623353},
year = {2023},
date = {2023-02-28},
journal = { Research Square},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2462.
Keisler, Julie; Talbi, El-Ghazali; Claudel, Sandra; Cabriel, Gilles
An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters Technical Manual
2023.
@manual{KeislerHAL23,
title = {An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters},
author = {Julie Keisler and El-Ghazali Talbi and Sandra Claudel and Gilles Cabriel},
url = {https://hal.science/hal-03982852/document},
year = {2023},
date = {2023-02-22},
urldate = {2023-02-22},
keywords = {},
pubstate = {published},
tppubtype = {manual}
}
2461.
Su, Lei; Zhou, Haoyi; Huang, Hua; Zhang, Wancai; Cao, Boyuan
Research on transformer internal defect detection based on large scale model Miscellaneous
2023.
@misc{surtid23,
title = {Research on transformer internal defect detection based on large scale model},
author = {Lei Su and Haoyi Zhou and Hua Huang and Wancai Zhang and Boyuan Cao},
url = {https://iopscience.iop.org/article/10.1088/1742-6596/2425/1/012042/pdf},
year = {2023},
date = {2023-02-22},
urldate = {2023-02-22},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
2460.
Speckhard, Daniel T.; Misiunas, Karolis; Perel, Sagi; Zhu, Tenghui; Carlile, Simon; Slaney, Malcolm
Neural architecture search for energy-efficient always-on audio machine learning Journal Article
In: Neural Computing and Applications , 2023.
@article{SpeckhardNCA23,
title = {Neural architecture search for energy-efficient always-on audio machine learning},
author = {Daniel T. Speckhard and Karolis Misiunas and Sagi Perel and Tenghui Zhu and Simon Carlile and Malcolm Slaney
},
url = {https://link.springer.com/article/10.1007/s00521-023-08345-y},
year = {2023},
date = {2023-02-20},
urldate = {2023-02-20},
journal = {Neural Computing and Applications },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2459.
Mohammadrezaei, Parsa; Aminan, Mohammad; Soltanian, Mohammad; Borna, Keivan
Improving CNN-based solutions for emotion recognition using evolutionary algorithms Journal Article
In: Results in Applied Mathematics, 2023.
@article{MohammadrezaeiRAM22,
title = {Improving CNN-based solutions for emotion recognition using evolutionary algorithms},
author = {Parsa Mohammadrezaei and Mohammad Aminan and Mohammad Soltanian and Keivan Borna },
url = {https://www.researchgate.net/profile/Mohammad-Aminan/publication/368576737_Improving_CNN-based_solutions_for_emotion_recognition_using_evolutionary_algorithms/links/63ef6d7d51d7af0540325e48/Improving-CNN-based-solutions-for-emotion-recognition-using-evolutionary-algorithms.pdf},
year = {2023},
date = {2023-02-13},
urldate = {2023-02-13},
journal = {Results in Applied Mathematics},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2458.
Du, Yangyi; Zhou, Xiaojun; Huang, Tingwen; Yang, Chunhua
A hierarchical evolution of neural architecture search method based on state transition algorithm Bachelor Thesis
2023.
@bachelorthesis{Du-ijmlc23,
title = {A hierarchical evolution of neural architecture search method based on state transition algorithm},
author = {
Yangyi Du and Xiaojun Zhou and Tingwen Huang and Chunhua Yang
},
url = {https://link.springer.com/article/10.1007/s13042-023-01794-w},
year = {2023},
date = {2023-02-13},
urldate = {2023-02-13},
journal = { International Journal of Machine Learning and Cybernetics },
keywords = {},
pubstate = {published},
tppubtype = {bachelorthesis}
}
2457.
Stephen, Okeke; Sain, Mangal
In: Developments in Optimization Algorithms for Smart Healthcare 2022, 2023.
@article{stephen-doafsh22,
title = {Using Deep Learning with Bayesian–Gaussian Inspired Convolutional Neural Architectural Search for Cancer Recognition and Classification from Histopathological Image Frames},
author = {Okeke Stephen and Mangal Sain},
url = {https://www.hindawi.com/journals/jhe/2023/4597445/},
year = {2023},
date = {2023-02-09},
urldate = {2023-02-09},
journal = {Developments in Optimization Algorithms for Smart Healthcare 2022},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2456.
Islama, Usman; Mahuma, Rabbia; AlSalman, AbdulMalik
Facial Emotions Detection using an Efficient Neural Architecture Search Network Technical Report
2023.
@techreport{IslamaRS23,
title = {Facial Emotions Detection using an Efficient Neural Architecture Search Network},
author = {Usman Islama and Rabbia Mahuma and AbdulMalik AlSalman},
url = {https://assets.researchsquare.com/files/rs-2526836/v1_covered.pdf?c=1675443603},
year = {2023},
date = {2023-02-03},
urldate = {2023-02-03},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2455.
Zheng, Xiawu; Yang, Chenyi; Zhang, Shaokun; Wang, Yan; Zhang, Baochang; Wu, Yongjian; Wu, Yunsheng; Shao, Ling; Ji, Rongrong
DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning Journal Article
In: International Journal of Computer Vision , 2023.
@article{Zhengijcv23,
title = {DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning},
author = {Xiawu Zheng and Chenyi Yang and Shaokun Zhang and Yan Wang and Baochang Zhang and Yongjian Wu and Yunsheng Wu and Ling Shao and Rongrong Ji
},
url = {https://link.springer.com/article/10.1007/s11263-023-01753-6},
year = {2023},
date = {2023-02-03},
urldate = {2023-02-03},
journal = {International Journal of Computer Vision },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2454.
Zawad, Syed
Towards Scalable, Private, and Practical Deep Learning PhD Thesis
2023.
@phdthesis{zawad-phd23,
title = {Towards Scalable, Private, and Practical Deep Learning},
author = {Syed Zawad},
url = {https://scholarworks.unr.edu/bitstream/handle/11714/8356/Zawad_unr_0139D_13906.pdf?sequence=1&isAllowed=y},
year = {2023},
date = {2023-02-03},
urldate = {2023-02-03},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2453.
Evolving Deep Neural Networks with Explanations for Image Classification PhD Thesis
2023.
@phdthesis{wang-phd23,
title = {Evolving Deep Neural Networks with Explanations for Image Classification},
url = {https://s3-ap-southeast-2.amazonaws.com/pstorage-wellington-7594921145/38986799/thesis_access.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA3OGA3B5WBO5PUAXV/20230208/ap-southeast-2/s3/aws4_request&X-Amz-Date=20230208T183516Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=598d7d8640153b25c41deadcb126a9cf1656dffaeb6797dc58f70e16638eecc2},
year = {2023},
date = {2023-02-03},
urldate = {2023-02-03},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2452.
Wei, Lanning; Zhao, Huan; He, Zhiqiang; Yao, Quanming
Neural Architecture Search for GNN-Based Graph Classification Journal Article
In: ACM Trans. Inf. Syst., 2023, ISSN: 1046-8188, (Just Accepted).
@article{10.1145/3584945,
title = {Neural Architecture Search for GNN-Based Graph Classification},
author = {Lanning Wei and Huan Zhao and Zhiqiang He and Quanming Yao},
url = {https://doi.org/10.1145/3584945},
doi = {10.1145/3584945},
issn = {1046-8188},
year = {2023},
date = {2023-02-01},
urldate = {2023-02-01},
journal = {ACM Trans. Inf. Syst.},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
abstract = {Graph classification is an important problem with applications across many domains, for which the graph neural networks (GNNs) have been state-of-the-art (SOTA) methods. In the literature, to adopt GNNs for the graph classification task, there are two groups of methods: global pooling and hierarchical pooling. The global pooling methods obtain the graph representation vectors by globally pooling all the node embeddings together at the end of several GNN layers, while the hierarchical pooling methods provide one extra pooling operation between the GNN layers to extract the hierarchical information and improve the graph representations. Both global and hierarchical pooling methods are effective in different scenarios. Due to highly diverse applications, it is challenging to design data-specific pooling methods with human expertise. To address this problem, we propose PAS (Pooling Architecture Search) to design adaptive pooling architectures by using the neural architecture search (NAS). To enable the search space design, we propose a unified pooling framework consisting of four modules: Aggregation, Pooling, Readout, and Merge. Two variants PAS-G and PAS-NE are provided to design the pooling operations in different scales. A set of candidate operations are designed in the search space on top of this framework, and then existing human-designed pooling methods, including global and hierarchical ones, can be incorporated. To enable efficient search, a coarsening strategy is developed to continuously relax the search space, and then a differentiable search method can be adopted. We conduct extensive experiments on six real-world datasets, including the large-scale datasets MR and ogbg-molhiv. Experimental results in this paper demonstrate the effectiveness and efficiency of the proposed PAS in designing the pooling architectures for graph classification. Besides, the Top-1 performance on two Open Graph Benchmark (OGB) datasets further indicates the utility of PAS when facing diverse realistic data. The implementation of PAS is available at: https://github.com/AutoML-Research/PAS.},
note = {Just Accepted},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Graph classification is an important problem with applications across many domains, for which the graph neural networks (GNNs) have been state-of-the-art (SOTA) methods. In the literature, to adopt GNNs for the graph classification task, there are two groups of methods: global pooling and hierarchical pooling. The global pooling methods obtain the graph representation vectors by globally pooling all the node embeddings together at the end of several GNN layers, while the hierarchical pooling methods provide one extra pooling operation between the GNN layers to extract the hierarchical information and improve the graph representations. Both global and hierarchical pooling methods are effective in different scenarios. Due to highly diverse applications, it is challenging to design data-specific pooling methods with human expertise. To address this problem, we propose PAS (Pooling Architecture Search) to design adaptive pooling architectures by using the neural architecture search (NAS). To enable the search space design, we propose a unified pooling framework consisting of four modules: Aggregation, Pooling, Readout, and Merge. Two variants PAS-G and PAS-NE are provided to design the pooling operations in different scales. A set of candidate operations are designed in the search space on top of this framework, and then existing human-designed pooling methods, including global and hierarchical ones, can be incorporated. To enable efficient search, a coarsening strategy is developed to continuously relax the search space, and then a differentiable search method can be adopted. We conduct extensive experiments on six real-world datasets, including the large-scale datasets MR and ogbg-molhiv. Experimental results in this paper demonstrate the effectiveness and efficiency of the proposed PAS in designing the pooling architectures for graph classification. Besides, the Top-1 performance on two Open Graph Benchmark (OGB) datasets further indicates the utility of PAS when facing diverse realistic data. The implementation of PAS is available at: https://github.com/AutoML-Research/PAS.
2451.
Wang, Q.; Zhang, S.
DGL: Device Generic Latency Model for Neural Architecture Search on Mobile Devices Journal Article
In: IEEE Transactions on Mobile Computing, no. 01, pp. 1-14, 2023, ISSN: 1558-0660.
@article{10042973,
title = {DGL: Device Generic Latency Model for Neural Architecture Search on Mobile Devices},
author = {Q. Wang and S. Zhang},
url = {https://www.computer.org/csdl/journal/tm/5555/01/10042973/1KJs8PnAasw},
doi = {10.1109/TMC.2023.3244170},
issn = {1558-0660},
year = {2023},
date = {2023-02-01},
urldate = {5555-02-01},
journal = {IEEE Transactions on Mobile Computing},
number = {01},
pages = {1-14},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {The low-cost Neural Architecture Search (NAS) for lightweight networks working on massive mobile devices is essential for fast-developing ICT technology. Current NAS work can not search on unseen devices without latency sampling, which is a big obstacle to the implementation of NAS on mobile devices. In this paper, we overcome this challenge by proposing the Device Generic Latency (DGL) model. By absorbing processor modeling technology, the proposed DGL formula maps the parameters in the interval theory to the seven static configuration parameters of the device. And to make the formula more practical, we refine it to low-cost form by decreasing the number of configuration parameters to four. Then based on this formula, the DGL model is proposed which introduces the network parameters predictor and accuracy predictor to work with the DGL formula to predict the network latency. We propose the DGL-based NAS framework to enable fast searches without latency sampling. Extensive experiments results validate that the DGL model can achieve more accurate latency predictions than existing NAS latency predictors on unseen mobile devices. When configured with current state-of-the-art predictors, DGL-based NAS can search for architectures with higher accuracy that meet the latency limit than other NAS implementations, while using less training time and prediction time. Our work shed light on how to adopt domain knowledge into NAS topic and play important role in low-cost NAS on mobile devices.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The low-cost Neural Architecture Search (NAS) for lightweight networks working on massive mobile devices is essential for fast-developing ICT technology. Current NAS work can not search on unseen devices without latency sampling, which is a big obstacle to the implementation of NAS on mobile devices. In this paper, we overcome this challenge by proposing the Device Generic Latency (DGL) model. By absorbing processor modeling technology, the proposed DGL formula maps the parameters in the interval theory to the seven static configuration parameters of the device. And to make the formula more practical, we refine it to low-cost form by decreasing the number of configuration parameters to four. Then based on this formula, the DGL model is proposed which introduces the network parameters predictor and accuracy predictor to work with the DGL formula to predict the network latency. We propose the DGL-based NAS framework to enable fast searches without latency sampling. Extensive experiments results validate that the DGL model can achieve more accurate latency predictions than existing NAS latency predictors on unseen mobile devices. When configured with current state-of-the-art predictors, DGL-based NAS can search for architectures with higher accuracy that meet the latency limit than other NAS implementations, while using less training time and prediction time. Our work shed light on how to adopt domain knowledge into NAS topic and play important role in low-cost NAS on mobile devices.
2450.
Jia, Liang; Tian, Ye; Zhang, Junguo
Neural architecture search based on packed samples for identifying animals in camera trap images Journal Article
In: Neural Computing and Applications (2023), 2023.
@article{JiaNCA23,
title = {Neural architecture search based on packed samples for identifying animals in camera trap images},
author = {Liang Jia and Ye Tian and Junguo Zhang },
url = {https://link.springer.com/article/10.1007/s00521-023-08247-z},
year = {2023},
date = {2023-01-29},
urldate = {2023-01-29},
journal = {Neural Computing and Applications (2023)},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2449.
Tchetchenian, Ari; Zhu, Yanming; Zhang, Fan; O’Donnell, Lauren J.; Song, Yang; Meijering, Erik
A comparison of manual and automated neural architecture search for white matter tract segmentation Journal Article
In: Scientific Reports , 2023.
@article{TchetchenianSR23,
title = {A comparison of manual and automated neural architecture search for white matter tract segmentation},
author = {
Ari Tchetchenian and Yanming Zhu and Fan Zhang and Lauren J. O’Donnell and Yang Song and Erik Meijering
},
url = {https://www.nature.com/articles/s41598-023-28210-1},
year = {2023},
date = {2023-01-28},
urldate = {2023-01-28},
journal = {Scientific Reports },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2448.
Yue, Zhixiong; Zhang, Yu; Liang, Jie
Learning Conflict-Noticed Architecture for Multi-Task Learning Miscellaneous
2023.
@misc{Zue23,
title = {Learning Conflict-Noticed Architecture for Multi-Task Learning},
author = {Zhixiong Yue and Yu Zhang and Jie Liang},
url = {https://yuezhixiong.github.io/Papers/CoNAL.pdf},
year = {2023},
date = {2023-01-26},
urldate = {2023-01-26},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
2447.
Chang, Chen-Chia; Pan, Jingyu; Xie, Zhiyao; Li, Yaguang; Lin, Yishuang; Hu, Jiang; Chen, Yiran
Fully Automated Machine Learning Model Development for Analog Placement Quality Prediction Inproceedings
In: 2023 Asia and South Pacific Design Automation Conference (ASP-DAC), 2023.
@inproceedings{ChangASPDAC23,
title = {Fully Automated Machine Learning Model Development for Analog Placement Quality Prediction},
author = {Chen-Chia Chang and Jingyu Pan and Zhiyao Xie and Yaguang Li and Yishuang Lin and Jiang Hu and Yiran Chen},
url = {https://zhiyaoxie.github.io/files/ASPDAC23_NAS_Analog.pdf},
year = {2023},
date = {2023-01-19},
urldate = {2023-01-19},
booktitle = {2023 Asia and South Pacific Design Automation Conference (ASP-DAC)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2446.
Zhang, Jian; Gong, Xuan; Liu, YuXiao; Wang, Wei; Wang, Lei; Zhang, BaoChang
Bandit neural architecture search based on performance evaluation for operation selection Journal Article
In: Science China Technological Sciences 2023, 2023.
@article{ZhangSCTS23b,
title = {Bandit neural architecture search based on performance evaluation for operation selection},
author = {
Jian Zhang and Xuan Gong and YuXiao Liu and Wei Wang and Lei Wang and BaoChang Zhang
},
url = {https://link.springer.com/article/10.1007/s11431-022-2197-y},
year = {2023},
date = {2023-01-16},
urldate = {2023-01-16},
journal = {Science China Technological Sciences 2023},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2445.
Asadi, Mehdi; Poursalim, Fatemeh; Loni, Mohammad; Daneshtalab, Masoud; Sjödin, Mikael; Gharehbaghi, Arash
Accurate Detection of Paroxysmal Atrial Fibrillation with Certified-GAN and Neural Architecture Search Technical Manual
2023.
@manual{nokey,
title = {Accurate Detection of Paroxysmal Atrial Fibrillation with Certified-GAN and Neural Architecture Search},
author = {Mehdi Asadi and Fatemeh Poursalim and Mohammad Loni and Masoud Daneshtalab and Mikael Sjödin and Arash Gharehbaghi},
url = {https://assets.researchsquare.com/files/rs-2485416/v1/03c1c5a579b494a39449a4ee.pdf?c=1674177185},
year = {2023},
date = {2023-01-16},
urldate = {2023-01-16},
keywords = {},
pubstate = {published},
tppubtype = {manual}
}
2444.
Zhang, Jian; Gong, Xuan; Liu, YuXiao; Wang, Wei; Wang, Lei; Zhang, BaoChang
Bandit neural architecture search based on performance evaluation for operation selection Journal Article
In: Science China Technological Sciences, 2023.
@article{ZhangSCTS23,
title = {Bandit neural architecture search based on performance evaluation for operation selection},
author = {Jian Zhang and Xuan Gong and YuXiao Liu and Wei Wang and Lei Wang and BaoChang Zhang
},
url = {https://link.springer.com/article/10.1007/s11431-022-2197-y},
year = {2023},
date = {2023-01-16},
urldate = {2023-01-16},
journal = {Science China Technological Sciences},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2443.
Wendlinger, Lorenz; Granitzer, Michael; Fellicious, Christofer
Pooling Graph Convolutional Networks for Structural Performance Prediction Inproceedings
In: Nicosia, Giuseppe; Ojha, Varun; Malfa, Emanuele La; Malfa, Gabriele La; Pardalos, Panos; Fatta, Giuseppe Di; Giuffrida, Giovanni; Umeton, Renato (Ed.): Machine Learning, Optimization, and Data Science, pp. 1–16, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-25891-6.
@inproceedings{10.1007/978-3-031-25891-6_1,
title = {Pooling Graph Convolutional Networks for Structural Performance Prediction},
author = {Lorenz Wendlinger and Michael Granitzer and Christofer Fellicious},
editor = {Giuseppe Nicosia and Varun Ojha and Emanuele La Malfa and Gabriele La Malfa and Panos Pardalos and Giuseppe Di Fatta and Giovanni Giuffrida and Renato Umeton},
url = {https://link.springer.com/chapter/10.1007/978-3-031-25891-6_1},
isbn = {978-3-031-25891-6},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Machine Learning, Optimization, and Data Science},
pages = {1--16},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Neural Architecture Search can help in finding high-performance task specific neural network architectures. However, the training of architectures that is required for fitness computation can be prohibitively expensive. Employing surrogate models as performance predictors can reduce or remove the need for these costly evaluations. We present a deep graph learning approach that achieves state-of-the-art performance in multiple NAS performance prediction benchmarks. In contrast to other methods, this model is purely supervised, which can be a methodologic advantage, as it does not rely on unlabeled instances sampled from complex search spaces.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search can help in finding high-performance task specific neural network architectures. However, the training of architectures that is required for fitness computation can be prohibitively expensive. Employing surrogate models as performance predictors can reduce or remove the need for these costly evaluations. We present a deep graph learning approach that achieves state-of-the-art performance in multiple NAS performance prediction benchmarks. In contrast to other methods, this model is purely supervised, which can be a methodologic advantage, as it does not rely on unlabeled instances sampled from complex search spaces.
2442.
Mecharbat, Lotfi Abdelkrim; Benmeziane, Hadjer; Ouranoughi, Hamza; Niar, Smaïl
HyT-NAS: Hybrid Transformers Neural Architecture Search for Edge Devices Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-04440b,
title = {HyT-NAS: Hybrid Transformers Neural Architecture Search for Edge Devices},
author = {Lotfi Abdelkrim Mecharbat and Hadjer Benmeziane and Hamza Ouranoughi and Smaïl Niar},
url = {https://doi.org/10.48550/arXiv.2303.04440},
doi = {10.48550/arXiv.2303.04440},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.04440},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2441.
Wang, Zhipeng; Liu, Rengkui; Gao, Yi; Tang, Yuanjie
Metro Track Geometry Defect Identification Model Based on Car-Body Vibration Data and Differentiable Architecture Search Journal Article
In: Applied Sciences, vol. 13, no. 6, 2023, ISSN: 2076-3417.
@article{app13063457,
title = {Metro Track Geometry Defect Identification Model Based on Car-Body Vibration Data and Differentiable Architecture Search},
author = {Zhipeng Wang and Rengkui Liu and Yi Gao and Yuanjie Tang},
url = {https://www.mdpi.com/2076-3417/13/6/3457},
doi = {10.3390/app13063457},
issn = {2076-3417},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Applied Sciences},
volume = {13},
number = {6},
abstract = {Efficient and low-cost modes for detecting metro track geometry defects (TGDs) are essential for condition-prediction-based preventive maintenance, which can help improve the safety of metro operations and reduce the maintenance cost of metro tracks. Compared with the traditional TGD detection method that utilizes the track geometry car, the method that uses a portable detector to acquire the car-body vibration data (CVD) can be used on an ordinary in-service train without occupying the metro schedule line, thereby improving efficiency and reducing the cost. A convolutional neural network-based identification model for TGD, built on a differentiable architecture search, is proposed in this study to employ only the CVD acquired by a portable detector for integrated identification of the type and severity level of TGDs. Second, the random oversampling method is introduced, and a strategy for applying this method is proposed to improve the poor training effect of the model caused by the natural class-imbalance problem arising from the TGD dataset. Subsequently, a comprehensive performance-evaluation metric (track geometry defect F-score) is designed by considering the actual management needs of the metro infrastructure. Finally, a case study is conducted using actual field data collected from Beijing Subway to validate the proposed model.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Efficient and low-cost modes for detecting metro track geometry defects (TGDs) are essential for condition-prediction-based preventive maintenance, which can help improve the safety of metro operations and reduce the maintenance cost of metro tracks. Compared with the traditional TGD detection method that utilizes the track geometry car, the method that uses a portable detector to acquire the car-body vibration data (CVD) can be used on an ordinary in-service train without occupying the metro schedule line, thereby improving efficiency and reducing the cost. A convolutional neural network-based identification model for TGD, built on a differentiable architecture search, is proposed in this study to employ only the CVD acquired by a portable detector for integrated identification of the type and severity level of TGDs. Second, the random oversampling method is introduced, and a strategy for applying this method is proposed to improve the poor training effect of the model caused by the natural class-imbalance problem arising from the TGD dataset. Subsequently, a comprehensive performance-evaluation metric (track geometry defect F-score) is designed by considering the actual management needs of the metro infrastructure. Finally, a case study is conducted using actual field data collected from Beijing Subway to validate the proposed model.
2440.
Alaiad, Ahmad; Migdady, Aya; Al-Khatib, Ra’ed M.; Alzoubi, Omar; Zitar, Raed Abu; Abualigah, Laith
Autokeras Approach: A Robust Automated Deep Learning Network for Diagnosis Disease Cases in Medical Images Journal Article
In: Journal of Imaging, vol. 9, no. 3, 2023, ISSN: 2313-433X.
@article{jimaging9030064,
title = {Autokeras Approach: A Robust Automated Deep Learning Network for Diagnosis Disease Cases in Medical Images},
author = {Ahmad Alaiad and Aya Migdady and Ra’ed M. Al-Khatib and Omar Alzoubi and Raed Abu Zitar and Laith Abualigah},
url = {https://www.mdpi.com/2313-433X/9/3/64},
doi = {10.3390/jimaging9030064},
issn = {2313-433X},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Journal of Imaging},
volume = {9},
number = {3},
abstract = {Automated deep learning is promising in artificial intelligence (AI). However, a few applications of automated deep learning networks have been made in the clinical medical fields. Therefore, we studied the application of an open-source automated deep learning framework, Autokeras, for detecting smear blood images infected with malaria parasites. Autokeras is able to identify the optimal neural network to perform the classification task. Hence, the robustness of the adopted model is due to it not needing any prior knowledge from deep learning. In contrast, the traditional deep neural network methods still require more construction to identify the best convolutional neural network (CNN). The dataset used in this study consisted of 27,558 blood smear images. A comparative process proved the superiority of our proposed approach over other traditional neural networks. The evaluation results of our proposed model achieved high efficiency with impressive accuracy, reaching 95.6% when compared with previous competitive models.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Automated deep learning is promising in artificial intelligence (AI). However, a few applications of automated deep learning networks have been made in the clinical medical fields. Therefore, we studied the application of an open-source automated deep learning framework, Autokeras, for detecting smear blood images infected with malaria parasites. Autokeras is able to identify the optimal neural network to perform the classification task. Hence, the robustness of the adopted model is due to it not needing any prior knowledge from deep learning. In contrast, the traditional deep neural network methods still require more construction to identify the best convolutional neural network (CNN). The dataset used in this study consisted of 27,558 blood smear images. A comparative process proved the superiority of our proposed approach over other traditional neural networks. The evaluation results of our proposed model achieved high efficiency with impressive accuracy, reaching 95.6% when compared with previous competitive models.
2439.
Qin, Dalin; Wang, Chenxi; Wen, Qingsong; Chen, Weiqi; Sun, Liang; Wang, Yi
Personalized Federated DARTS for Electricity Load Forecasting of Individual Buildings Journal Article
In: IEEE Transactions on Smart Grid, pp. 1-1, 2023.
@article{10063999,
title = {Personalized Federated DARTS for Electricity Load Forecasting of Individual Buildings},
author = {Dalin Qin and Chenxi Wang and Qingsong Wen and Weiqi Chen and Liang Sun and Yi Wang},
doi = {10.1109/TSG.2023.3253855},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Smart Grid},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2438.
Chitty-Venkata, Krishna Teja; Emani, Murali; Vishwanath, Venkatram; Somani, Arun K.
Neural Architecture Search Benchmarks: Insights and Survey Journal Article
In: IEEE Access, vol. 11, pp. 25217–25236, 2023.
@article{DBLP:journals/access/ChittyVenkataEVS23,
title = {Neural Architecture Search Benchmarks: Insights and Survey},
author = {Krishna Teja Chitty-Venkata and Murali Emani and Venkatram Vishwanath and Arun K. Somani},
url = {https://doi.org/10.1109/ACCESS.2023.3253818},
doi = {10.1109/ACCESS.2023.3253818},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Access},
volume = {11},
pages = {25217--25236},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2437.
Ali, Mohamed Nabih; Paissan, Francesco; Falavigna, Daniele; Brutti, Alessio
Scaling strategies for on-device low-complexity source separation with Conv-Tasnet Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-03005,
title = {Scaling strategies for on-device low-complexity source separation with Conv-Tasnet},
author = {Mohamed Nabih Ali and Francesco Paissan and Daniele Falavigna and Alessio Brutti},
url = {https://doi.org/10.48550/arXiv.2303.03005},
doi = {10.48550/arXiv.2303.03005},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.03005},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2436.
Shen, Xuan; Wang, Yaohua; Lin, Ming; Huang, Yilun; Tang, Hao; Sun, Xiuyu; Wang, Yanzhi
DeepMAD: Mathematical Architecture Design for Deep Convolutional Neural Network Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-02165,
title = {DeepMAD: Mathematical Architecture Design for Deep Convolutional Neural Network},
author = {Xuan Shen and Yaohua Wang and Ming Lin and Yilun Huang and Hao Tang and Xiuyu Sun and Yanzhi Wang},
url = {https://doi.org/10.48550/arXiv.2303.02165},
doi = {10.48550/arXiv.2303.02165},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.02165},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2435.
Lee, Jong-Ryul; Moon, Yong-Hyuk
Bespoke: A Block-Level Neural Network Optimization Framework for Low-Cost Deployment Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-01913,
title = {Bespoke: A Block-Level Neural Network Optimization Framework for Low-Cost Deployment},
author = {Jong-Ryul Lee and Yong-Hyuk Moon},
url = {https://doi.org/10.48550/arXiv.2303.01913},
doi = {10.48550/arXiv.2303.01913},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.01913},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2434.
Santana, Roberto; Hidalgo-Cenalmor, Ivan; Garciarena, Unai; Mendiburu, Alexander; Lozano, José Antonio
Neuroevolutionary algorithms driven by neuron coverage metrics for semi-supervised classification Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-02801,
title = {Neuroevolutionary algorithms driven by neuron coverage metrics for semi-supervised classification},
author = {Roberto Santana and Ivan Hidalgo-Cenalmor and Unai Garciarena and Alexander Mendiburu and José Antonio Lozano},
url = {https://doi.org/10.48550/arXiv.2303.02801},
doi = {10.48550/arXiv.2303.02801},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.02801},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2433.
Brown, Austin; Gupta, Maanak; Abdelsalam, Mahmoud
Automated Machine Learning for Deep Learning based Malware Detection Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-01679,
title = {Automated Machine Learning for Deep Learning based Malware Detection},
author = {Austin Brown and Maanak Gupta and Mahmoud Abdelsalam},
url = {https://doi.org/10.48550/arXiv.2303.01679},
doi = {10.48550/arXiv.2303.01679},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.01679},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2432.
Ke, Songyu; Pan, Zheyi; He, Tianfu; Liang, Yuxuan; Zhang, Junbo; Zheng, Yu
AutoSTG+: An automatic framework to discover the optimal network for spatio-temporal graph prediction Journal Article
In: Artificial Intelligence, vol. 318, pp. 103899, 2023, ISSN: 0004-3702.
@article{KE2023103899,
title = {AutoSTG+: An automatic framework to discover the optimal network for spatio-temporal graph prediction},
author = {Songyu Ke and Zheyi Pan and Tianfu He and Yuxuan Liang and Junbo Zhang and Yu Zheng},
url = {https://www.sciencedirect.com/science/article/pii/S0004370223000450},
doi = {https://doi.org/10.1016/j.artint.2023.103899},
issn = {0004-3702},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Artificial Intelligence},
volume = {318},
pages = {103899},
abstract = {Spatio-temporal graphs (STGs) are important structures to describe urban sensory data, e.g., traffic speed and air quality. Predicting over spatio-temporal graphs enables many essential applications in intelligent cities, such as traffic management and environment analysis. Recently, many deep learning models have been proposed for spatio-temporal graph prediction and achieved significant results. However, manually designing neural networks requires rich domain knowledge and heavy expert efforts, making it impractical for real-world deployments. Therefore, we study automated neural architecture search for spatio-temporal graphs, which meets three challenges: 1) how to define search space for capturing complex spatio-temporal correlations; 2) how to jointly model the explicit and implicit relationships between nodes of an STG; and 3) how to learn network weight parameters related to meta graphs of STGs. To tackle these challenges, we propose a novel neural architecture search framework, entitled AutoSTG+, for automated spatio-temporal graph prediction. In our AutoSTG+, spatial graph convolution and temporal convolution operations are adopted in the search space of AutoSTG+ to capture complex spatio-temporal correlations. Besides, we propose to employ the meta-learning technique to learn the adjacency matrices of spatial graph convolution layers and kernels of temporal convolution layers from the meta knowledge of meta graphs. And specifically, such meta-knowledge is learned by graph meta-knowledge learners, which iteratively aggregate knowledge on the attributed graphs and the similarity graphs. Finally, extensive experiments have been conducted on multiple real-world datasets to demonstrate that AutoSTG+ can find effective network architectures and achieve up to about 20% relative improvements compared to human-designed networks.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Spatio-temporal graphs (STGs) are important structures to describe urban sensory data, e.g., traffic speed and air quality. Predicting over spatio-temporal graphs enables many essential applications in intelligent cities, such as traffic management and environment analysis. Recently, many deep learning models have been proposed for spatio-temporal graph prediction and achieved significant results. However, manually designing neural networks requires rich domain knowledge and heavy expert efforts, making it impractical for real-world deployments. Therefore, we study automated neural architecture search for spatio-temporal graphs, which meets three challenges: 1) how to define search space for capturing complex spatio-temporal correlations; 2) how to jointly model the explicit and implicit relationships between nodes of an STG; and 3) how to learn network weight parameters related to meta graphs of STGs. To tackle these challenges, we propose a novel neural architecture search framework, entitled AutoSTG+, for automated spatio-temporal graph prediction. In our AutoSTG+, spatial graph convolution and temporal convolution operations are adopted in the search space of AutoSTG+ to capture complex spatio-temporal correlations. Besides, we propose to employ the meta-learning technique to learn the adjacency matrices of spatial graph convolution layers and kernels of temporal convolution layers from the meta knowledge of meta graphs. And specifically, such meta-knowledge is learned by graph meta-knowledge learners, which iteratively aggregate knowledge on the attributed graphs and the similarity graphs. Finally, extensive experiments have been conducted on multiple real-world datasets to demonstrate that AutoSTG+ can find effective network architectures and achieve up to about 20% relative improvements compared to human-designed networks.
2431.
Cereda, Elia; Crupi, Luca; Risso, Matteo; Burrello, Alessio; Benini, Luca; Giusti, Alessandro; Pagliari, Daniele Jahier; Palossi, Daniele
Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs Inproceedings
In: IEEE ICRA 2023, 2023.
@inproceedings{DBLP:journals/corr/abs-2303-01931,
title = {Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs},
author = {Elia Cereda and Luca Crupi and Matteo Risso and Alessio Burrello and Luca Benini and Alessandro Giusti and Daniele Jahier Pagliari and Daniele Palossi},
url = {https://doi.org/10.48550/arXiv.2303.01931},
doi = {10.48550/arXiv.2303.01931},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {IEEE ICRA 2023},
journal = {CoRR},
volume = {abs/2303.01931},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2430.
Kus, Zeki; Akkan, Can; Gülcü, Ayla
Novel Surrogate Measures Based on a Similarity Network for Neural Architecture Search Journal Article
In: IEEE Access, vol. 11, pp. 22596–22613, 2023.
@article{DBLP:journals/access/KusAG23,
title = {Novel Surrogate Measures Based on a Similarity Network for Neural Architecture Search},
author = {Zeki Kus and Can Akkan and Ayla Gülcü},
url = {https://doi.org/10.1109/ACCESS.2023.3252887},
doi = {10.1109/ACCESS.2023.3252887},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Access},
volume = {11},
pages = {22596--22613},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2429.
Cao, Chunhong; Xiang, Han; Song, Wei; Yi, Hongbo; Xiao, Fen; Gao, Xieping
Lightweight Multiscale Neural Architecture Search With Spectral–Spatial Attention for Hyperspectral Image Classification Journal Article
In: IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1-15, 2023.
@article{10061276,
title = {Lightweight Multiscale Neural Architecture Search With Spectral–Spatial Attention for Hyperspectral Image Classification},
author = {Chunhong Cao and Han Xiang and Wei Song and Hongbo Yi and Fen Xiao and Xieping Gao},
doi = {10.1109/TGRS.2023.3253247},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Geoscience and Remote Sensing},
volume = {61},
pages = {1-15},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2428.
Xue, Yu; Chen, Chen; Słowik, Adam
Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack Journal Article
In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2023.
@article{10059145,
title = {Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack},
author = {Yu Xue and Chen Chen and Adam Słowik},
doi = {10.1109/TEVC.2023.3252612},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Evolutionary Computation},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2427.
Wang, Kun; Han, Ling; Li, Liangzhi
A decoupled search deep network framework for high-resolution remote sensing image classification Journal Article
In: Remote Sensing Letters, vol. 14, no. 3, pp. 243-253, 2023.
@article{doi:10.1080/2150704X.2023.2185110,
title = {A decoupled search deep network framework for high-resolution remote sensing image classification},
author = {Kun Wang and Ling Han and Liangzhi Li},
url = {https://doi.org/10.1080/2150704X.2023.2185110},
doi = {10.1080/2150704X.2023.2185110},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Remote Sensing Letters},
volume = {14},
number = {3},
pages = {243-253},
publisher = {Taylor & Francis},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2426.
Njor, Emil; Madsen, Jan; Fafoutis, Xenofon
Data Aware Neural Architecture Search Inproceedings
In: Proceedings of tinyML Research Symposium, 2023, (tinyML Research Symposium’23 ; Conference date: 27-03-2023 Through 27-03-2023).
@inproceedings{40cb879dbb9a4fd9bd92ad27b617056f,
title = {Data Aware Neural Architecture Search},
author = {Emil Njor and Jan Madsen and Xenofon Fafoutis},
url = {https://orbit.dtu.dk/en/publications/data-aware-neural-architecture-search},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of tinyML Research Symposium},
abstract = {Neural Architecture Search (NAS) is a popular tool for automatically generating Neural Network (NN) architectures. In early NAS works, these tools typically optimized NN architectures for a single metric, such as accuracy. However, in the case of resource constrained Machine Learning, one single metric is not enough to evaluate a NN architecture. For example, a NN model achieving a high accuracy is not useful if it does not fit inside the flash memory of a given system. Therefore, recent works on NAS for resource constrained systems have investigated various approaches to optimize for multiple metrics. In this paper, we propose that, on top of these approaches, it could be beneficial for NAS optimization of resource constrained systems to also consider input data granularity. We name such a system “Data Aware NAS”, and we provide experimental evidence of its benefits by comparing it to traditional NAS.},
note = {tinyML Research Symposium’23 ; Conference date: 27-03-2023 Through 27-03-2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) is a popular tool for automatically generating Neural Network (NN) architectures. In early NAS works, these tools typically optimized NN architectures for a single metric, such as accuracy. However, in the case of resource constrained Machine Learning, one single metric is not enough to evaluate a NN architecture. For example, a NN model achieving a high accuracy is not useful if it does not fit inside the flash memory of a given system. Therefore, recent works on NAS for resource constrained systems have investigated various approaches to optimize for multiple metrics. In this paper, we propose that, on top of these approaches, it could be beneficial for NAS optimization of resource constrained systems to also consider input data granularity. We name such a system “Data Aware NAS”, and we provide experimental evidence of its benefits by comparing it to traditional NAS.
2425.
Lu, Shun; Hu, Yu; Yang, Longxing; Sun, Zihao; Mei, Jilin; Tan, Jianchao; Song, Chengru
PA&DA: Jointly Sampling PAth and DAta for Consistent NAS Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-14772,
title = {PA&DA: Jointly Sampling PAth and DAta for Consistent NAS},
author = {Shun Lu and Yu Hu and Longxing Yang and Zihao Sun and Jilin Mei and Jianchao Tan and Chengru Song},
url = {https://doi.org/10.48550/arXiv.2302.14772},
doi = {10.48550/arXiv.2302.14772},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.14772},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2424.
Lin, Shih-Ping; Wang, Sheng-De
SGAS-es: Avoiding Performance Collapse by Sequential Greedy Architecture Search with the Early Stopping Indicator Inproceedings
In: Ärai, Kohei" (Ed.): Ädvances in Information and Communication", pp. 135–154, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-28073-3.
@inproceedings{10.1007/978-3-031-28073-3_10,
title = {SGAS-es: Avoiding Performance Collapse by Sequential Greedy Architecture Search with the Early Stopping Indicator},
author = {Shih-Ping Lin and Sheng-De Wang},
editor = {Kohei" Ärai},
url = {https://link.springer.com/chapter/10.1007/978-3-031-28073-3_10},
isbn = {978-3-031-28073-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ädvances in Information and Communication"},
pages = {135--154},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Sequential Greedy Architecture Search (SGAS) reduces the discretization loss of Differentiable Architecture Search (DARTS). However, we observed that SGAS may lead to unstable searched results as DARTS. We referred to this problem as the cascade performance collapse issue. Therefore, we proposed Sequential Greedy Architecture Search with the Early Stopping Indicator (SGAS-es). We adopted the early stopping mechanism in each phase of SGAS to stabilize searched results and further improve the searching ability. The early stopping mechanism is based on the relation among Flat Minima, the largest eigenvalue of the Hessian matrix of the loss function, and performance collapse. We devised a mathematical derivation to show the relation between Flat Minima and the largest eigenvalue. The moving averaged largest eigenvalue is used as an early stopping indicator. Finally, we used NAS-Bench-201 and Fashion-MNIST to confirm the performance and stability of SGAS-es. Moreover, we used EMNIST-Balanced to verify the transferability of searched results. These experiments show that SGAS-es is a robust method and can derive the architecture with good performance and transferability.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Sequential Greedy Architecture Search (SGAS) reduces the discretization loss of Differentiable Architecture Search (DARTS). However, we observed that SGAS may lead to unstable searched results as DARTS. We referred to this problem as the cascade performance collapse issue. Therefore, we proposed Sequential Greedy Architecture Search with the Early Stopping Indicator (SGAS-es). We adopted the early stopping mechanism in each phase of SGAS to stabilize searched results and further improve the searching ability. The early stopping mechanism is based on the relation among Flat Minima, the largest eigenvalue of the Hessian matrix of the loss function, and performance collapse. We devised a mathematical derivation to show the relation between Flat Minima and the largest eigenvalue. The moving averaged largest eigenvalue is used as an early stopping indicator. Finally, we used NAS-Bench-201 and Fashion-MNIST to confirm the performance and stability of SGAS-es. Moreover, we used EMNIST-Balanced to verify the transferability of searched results. These experiments show that SGAS-es is a robust method and can derive the architecture with good performance and transferability.
2423.
Hasana, Md. Mehedi; Ibrahim, Muhammad; Ali, Md. Sawkat
Speeding Up EfficientNet: Selecting Update Blocks of Convolutional Neural Networks using Genetic Algorithm in Transfer Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-00261,
title = {Speeding Up EfficientNet: Selecting Update Blocks of Convolutional Neural Networks using Genetic Algorithm in Transfer Learning},
author = {Md. Mehedi Hasana and Muhammad Ibrahim and Md. Sawkat Ali},
url = {https://doi.org/10.48550/arXiv.2303.00261},
doi = {10.48550/arXiv.2303.00261},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.00261},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2422.
Eisenbach, Markus; Lübberstedt, Jannik; Aganian, Dustin; Gross, Horst-Michael
A Little Bit Attention Is All You Need for Person Re-Identification Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-14574,
title = {A Little Bit Attention Is All You Need for Person Re-Identification},
author = {Markus Eisenbach and Jannik Lübberstedt and Dustin Aganian and Horst-Michael Gross},
url = {https://doi.org/10.48550/arXiv.2302.14574},
doi = {10.48550/arXiv.2302.14574},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.14574},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2421.
Wang, Chunnan; Chen, Bozhou; Li, Geng; Wang, Hongzhi
Automated Graph Neural Network Search Under Federated Learning Framework Journal Article
In: IEEE Transactions on Knowledge and Data Engineering, pp. 1-13, 2023.
@article{10056291,
title = {Automated Graph Neural Network Search Under Federated Learning Framework},
author = {Chunnan Wang and Bozhou Chen and Geng Li and Hongzhi Wang},
url = {https://ieeexplore.ieee.org/abstract/document/10056291},
doi = {10.1109/TKDE.2023.3250264},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Knowledge and Data Engineering},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2420.
Rajesh, Chilukamari; Kumar, Sushil
Äutomatic Retinal Vessel Segmentation Using BTLBO Inproceedings
In: Thakur, Manoj; Agnihotri, Samar; Rajpurohit, Bharat Singh; Pant, Millie; Deep, Kusum; Nagar, Atulya K. (Ed.): Soft Computing for Problem Solving, pp. 189–200, Springer Nature Singapore, Singapore, 2023, ISBN: 978-981-19-6525-8.
@inproceedings{10.1007/978-981-19-6525-8_15,
title = {Äutomatic Retinal Vessel Segmentation Using BTLBO},
author = {Chilukamari Rajesh and Sushil Kumar},
editor = {Manoj Thakur and Samar Agnihotri and Bharat Singh Rajpurohit and Millie Pant and Kusum Deep and Atulya K. Nagar},
url = {https://link.springer.com/chapter/10.1007/978-981-19-6525-8_15},
isbn = {978-981-19-6525-8},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Soft Computing for Problem Solving},
pages = {189--200},
publisher = {Springer Nature Singapore},
address = {Singapore},
abstract = {The accuracy of retinal vessel segmentation (RVS) is crucial in assisting physicians in the ophthalmology diagnosis or other systemic diseases. However, manual segmentation needs a high level of knowledge, time-consuming, complex, and prone to errors. As a result, automatic vessel segmentation is required, which might be a significant technological breakthrough in the medical field. We proposed a novel strategy in this paper, that uses neural architecture search (NAS) to optimize a U-net architecture using a binary teaching learning-based optimization (BTLBO) evolutionary algorithm for RVS to increase vessel segmentation performance and reduce the workload of manually developing deep networks with limited computing resources. We used a publicly available DRIVE dataset to examine the proposed approach and showed that the discovered model generated by the proposed approach outperforms existing methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The accuracy of retinal vessel segmentation (RVS) is crucial in assisting physicians in the ophthalmology diagnosis or other systemic diseases. However, manual segmentation needs a high level of knowledge, time-consuming, complex, and prone to errors. As a result, automatic vessel segmentation is required, which might be a significant technological breakthrough in the medical field. We proposed a novel strategy in this paper, that uses neural architecture search (NAS) to optimize a U-net architecture using a binary teaching learning-based optimization (BTLBO) evolutionary algorithm for RVS to increase vessel segmentation performance and reduce the workload of manually developing deep networks with limited computing resources. We used a publicly available DRIVE dataset to examine the proposed approach and showed that the discovered model generated by the proposed approach outperforms existing methods.
2419.
He, Yang; Xiao, Lingao
Structured Pruning for Deep Convolutional Neural Networks: A survey Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-00566,
title = {Structured Pruning for Deep Convolutional Neural Networks: A survey},
author = {Yang He and Lingao Xiao},
url = {https://doi.org/10.48550/arXiv.2303.00566},
doi = {10.48550/arXiv.2303.00566},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.00566},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2418.
Chen, Angelica; Dohan, David M.; So, David R.
EvoPrompting: Language Models for Code-Level Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-14838,
title = {EvoPrompting: Language Models for Code-Level Neural Architecture Search},
author = {Angelica Chen and David M. Dohan and David R. So},
url = {https://doi.org/10.48550/arXiv.2302.14838},
doi = {10.48550/arXiv.2302.14838},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.14838},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2417.
Zheng, Xin; Zhang, Miao; Chen, Chunyang; Zhang, Qin; Zhou, Chuan; Pan, Shirui
Auto-HeG: Automated Graph Neural Network on Heterophilic Graphs Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-12357,
title = {Auto-HeG: Automated Graph Neural Network on Heterophilic Graphs},
author = {Xin Zheng and Miao Zhang and Chunyang Chen and Qin Zhang and Chuan Zhou and Shirui Pan},
url = {https://doi.org/10.48550/arXiv.2302.12357},
doi = {10.48550/arXiv.2302.12357},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.12357},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2416.
Zheng, Shenghe; Wang, Hongzhi; Mu, Tianyu
DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-13020,
title = {DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning},
author = {Shenghe Zheng and Hongzhi Wang and Tianyu Mu},
url = {https://doi.org/10.48550/arXiv.2302.13020},
doi = {10.48550/arXiv.2302.13020},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.13020},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2415.
Bataineh, Ali Al; Kaur, Devinder; Al-khassaweneh, Mahmood; Al-sharoa, Esraa
Automated CNN Architectural Design: A Simple and Efficient Methodology for Computer Vision Tasks Journal Article
In: Mathematics, vol. 11, no. 5, pp. 1-17, 2023.
@article{RePEc:gam:jmathe:v:11:y:2023:i:5:p:1141-:d:1079919,
title = {Automated CNN Architectural Design: A Simple and Efficient Methodology for Computer Vision Tasks},
author = {Ali Al Bataineh and Devinder Kaur and Mahmood Al-khassaweneh and Esraa Al-sharoa},
url = {https://www.mdpi.com/2227-7390/11/5/1141},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Mathematics},
volume = {11},
number = {5},
pages = {1-17},
abstract = {Convolutional neural networks (CNN) have transformed the field of computer vision by enabling the automatic extraction of features, obviating the need for manual feature engineering. Despite their success, identifying an optimal architecture for a particular task can be a time-consuming and challenging process due to the vast space of possible network designs. To address this, we propose a novel neural architecture search (NAS) framework that utilizes the clonal selection algorithm (CSA) to automatically design high-quality CNN architectures for image classification problems. Our approach uses an integer vector representation to encode CNN architectures and hyperparameters, combined with a truncated Gaussian mutation scheme that enables efficient exploration of the search space. We evaluated the proposed method on six challenging EMNIST benchmark datasets for handwritten digit recognition, and our results demonstrate that it outperforms nearly all existing approaches. In addition, our approach produces state-of-the-art performance while having fewer trainable parameters than other methods, making it low-cost, simple, and reusable for application to multiple datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Convolutional neural networks (CNN) have transformed the field of computer vision by enabling the automatic extraction of features, obviating the need for manual feature engineering. Despite their success, identifying an optimal architecture for a particular task can be a time-consuming and challenging process due to the vast space of possible network designs. To address this, we propose a novel neural architecture search (NAS) framework that utilizes the clonal selection algorithm (CSA) to automatically design high-quality CNN architectures for image classification problems. Our approach uses an integer vector representation to encode CNN architectures and hyperparameters, combined with a truncated Gaussian mutation scheme that enables efficient exploration of the search space. We evaluated the proposed method on six challenging EMNIST benchmark datasets for handwritten digit recognition, and our results demonstrate that it outperforms nearly all existing approaches. In addition, our approach produces state-of-the-art performance while having fewer trainable parameters than other methods, making it low-cost, simple, and reusable for application to multiple datasets.