Multi-level differentiable architecture search with heuristic entropy
CSTR:
Author:
Affiliation:

(1.Chengdu Institute of Computer Applications, Chinese Academy of Sciences, Chengdu 610041, China; 2.University of Chinese Academy of Sciences, Beijing 100049, China; 3.International Research Institute of Artificial Intelligence, Harbin Institute of Technology, Shenzhen, Shenzhen 518055, Guangdong, China)

Clc Number:

TP183

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Network architecture is an important factor affecting the performance of convolutional neural networks. Due to the low efficiency of the traditional manual design of network architecture, the method of automatically designing network architecture through algorithms has attracted more and more attention. Although the approach of differentiable architecture search (DARTS) has the capacity of designing networks automatically and efficiently, there are still problems owing to its super network construction and derivation strategy. An improved algorithm was proposed to overcome these shortcomings. First, the coupling problem caused by sharing architecture parameters in the super network was disclosed by quantifying the changes in the number of the skip candidate operations during the algorithm search process. Next, aiming at the coupling problem of the super network, a super network with multi-level cells was designed to avoid the mutual influence of cells at different levels. Then, in view of the “gap” between the super network and the derived architecture, the entropy of architecture parameters was introduced as the loss term of the objective function to inspire the training of the super network. Finally, architecture search experiments were conducted on CIFAR-10 dataset, and architecture evaluation experiments were conducted on CIFAR-10 and ImageNet respectively. Experimental results on CIFAR-10 show that the proposed algorithm removed the coupling problem between cells at different levels and improved the performance of the automatically designed architecture, which achieved classification error rate of only 2.69%. The architecture had the classification error rate of 25.9% on ImageNet, which proved its transferability.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:November 20,2020
  • Revised:
  • Adopted:
  • Online: August 10,2021
  • Published:
Article QR Code