AutoML | Literature on Neural Architecture Search

Maintained by Difan Deng and Marius Lindauer.

The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.

Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.

Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.

3425 entries « ‹ 1 of 69 › »

5555

3425.

Chen, X.; Yang, C.

CIMNet: Joint Search for Neural Network and Computing-in-Memory Architecture Journal Article

In: IEEE Micro, no. 01, pp. 1-12, 5555, ISSN: 1937-4143.

Abstract | Links | BibTeX

3424.

Yan, J.; Liu, J.; Xu, H.; Wang, Z.; Qiao, C.

Peaches: Personalized Federated Learning with Neural Architecture Search in Edge Computing Journal Article

In: IEEE Transactions on Mobile Computing, no. 01, pp. 1-17, 5555, ISSN: 1558-0660.

Abstract | Links | BibTeX

@article{10460163,

title = {Peaches: Personalized Federated Learning with Neural Architecture Search in Edge Computing},

author = {J. Yan and J. Liu and H. Xu and Z. Wang and C. Qiao},

doi = {10.1109/TMC.2024.3373506},

issn = {1558-0660},

year  = {5555},

date = {5555-03-01},

urldate = {5555-03-01},

journal = {IEEE Transactions on Mobile Computing},

number = {01},

pages = {1-17},

publisher = {IEEE Computer Society},

address = {Los Alamitos, CA, USA},

abstract = {In edge computing (EC), federated learning (FL) enables numerous distributed devices (or workers) to collaboratively train AI models without exposing their local data. Most works of FL adopt a predefined architecture on all participating workers for model training. However, since workers' local data distributions vary heavily in EC, the predefined architecture may not be the optimal choice for every worker. It is also unrealistic to manually design a high-performance architecture for each worker, which requires intense human expertise and effort. In order to tackle this challenge, neural architecture search (NAS) has been applied in FL to automate the architecture design process. Unfortunately, the existing federated NAS frameworks often suffer from the difficulties of system heterogeneity and resource limitation. To remedy this problem, we present a novel framework, termed Peaches, to achieve efficient searching and training in the resource-constrained EC system. Specifically, the local model of each worker is stacked by base cell and personal cell, where the base cell is shared by all workers to capture the common knowledge and the personal cell is customized for each worker to fit the local data. We determine the number of base cells, shared by all workers, according to the bandwidth budget on the parameters server. Besides, to relieve the data and system heterogeneity, we find the optimal number of personal cells for each worker based on its computing capability. In addition, we gradually prune the search space during training to mitigate the resource consumption. We evaluate the performance of Peaches through extensive experiments, and the results show that Peaches can achieve an average accuracy improvement of about 6.29% and up to 3.97× speed up compared with the baselines.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

2024

3423.

Ouertatani, Houssem; Maxim, Cristian; Niar, Smail; Talbi, El-Ghazali

Accelerated NAS via pretrained ensembles and multi-fidelity Bayesian Optimization Proceedings Article

In: 33rd International Conference on Artificial Neural Networks (ICANN), Lugano, Switzerland, 2024.