Journal of Architectural Engineering Technology
Open Access

Architecture; Landscapes; Concrete

Introduction

Deeper and more intricate neural network designs are being used as a result of recent deep learning accomplishments in handling difficult issues. Millions of weights to be taught are now typical for manually created networks made up of combinations of several layer kinds. Finding a model that excels at a specific activity involves significant knowledge and resources, making the construction of these networks no simple undertaking [1].

Methods

Localized opti networks

Presented local optima networks (LONs) as a graph-based abstraction of the search space representing the global structure. Each LON node is a local optimum, and the edges between nodes show the proximity of the optimum basins (the possibility of search transitioning from one local optimum to another). The reader is directed to the LON website, http://lonmaps.com, for resources, as LONs have been widely used as a landscape study tool (especially for discrete search spaces) [2, 3].

Fault landscapes

In the context of neural networks, training is an optimization problem with the goal of minimising the error on the machine learning task. The search space is the set of all network weights. An error landscape (also known as a loss landscape) is the surface that the training algorithm travels through in weight space and is comparable to a fitness landscape [4].

Since the error to be minimised depends on the data set used to evaluate the error, the examination of neural network error surfaces is made more challenging by the possibility that the same solution can have distinct error values. Theoretically and empirically analysing the error surfaces of multi-layered neural networks, Choromanska et al [5].

The training and testing error grew significantly de-correlated with network size, according to [9]'s theoretical and empirical analysis of the error surfaces of multi-layered neural networks. Finding the global optimum in the training loss landscape may not be very useful as a result of the fact that it is unlikely to be in the same location as the global optimum in the testing loss landscape. We will demonstrate how our study discovered evidence of this behaviour in the instance of the search space of CNN architectures [6, 7].

Landscapes are analyzed by neural architecture

While neural architecture search (NAS) focuses on the optimization of model-related parameters, such as the number and types of layers, the number of neurons in each layer, the choice of activation functions, etc., hyperparameter optimization in neural network training focuses on the search for optimal training-related parameters, such as the batch size and learning rate. Random search, reinforcement learning, gradient-based search techniques, evolutionary search techniques, and Bayesian optimization are some of the broad NAS approaches [8, 9].

Discussion

The paper's insight into the characteristics of CNNs' architectural search landscapes is one of its primary contributions. Fitness distance correlation and local optima networks were used to characterise the global CNN architecture spaces of six classification problems utilising a smaller search space. The analysis revealed that there are only a small number of local optima and that the globally optimal solutions can be easily attained by applying a straightforward perturbation operator, indicating that a hill-climbing approach like ILS might be a useful method for navigating these search landscapes.

Conclusion

The second part of our study tested this theory by experimentally contrasting the performance of ILS with three EA variations while using a more expressive grammar to provide a significantly wider search area. The findings confirm findings from earlier studies by demonstrating that ILS was a more successful method than all other EA variations for scanning the training landscapes. For half of the datasets taken into consideration, ILS was also the best generalisation approach. The examination of the smaller search space revealed a better association between the training and test loss levels for these datasets [10].

Acknowledgement

None.

Conflict of Interest

None.

1. Daniel Roth, Marc Erich L (2020) Construction of the Virtual Embodiment Questionnaire (VEQ). IEEE Trans Vis Comput Graph 26: 3546-3556.

Indexed at, Google Scholar, Crossref

2. Peter DK, Peter EM, Richard B, Anamika K, Markus K, et al. (2021) Pathway Tools version 23.0 update: software for pathway/genome informatics and systems biology. Brief Bioinform 22: 109-126.

 Indexed at, Google Scholar, Crossref

3. Pasi R (2020) Lep-Anchor: automated construction of linkage map anchored haploid genomes. Bioinformatics 36: 2359-2364.

Indexed at, Google Scholar, Crossref

4. Asem A, Joshua LB, Nicholas VH, C David S, Anton SP (2021) The proto-Nucleic Acid Builder: a software tool for constructing nucleic acid analogs. Nucleic Acids Res 49: 79-89.

Indexed at, Google Scholar, Crossref

5. Lior G, Mayrose I (2021) Panoramic: A package for constructing eukaryotic pan-genomes. Mol Ecol Resour 21: 1393-1403.

Indexed at, Google Scholar, Crossref

6. Bahar A, Alan K, Simon JP, Leena S, Christina B (2021) Succinct dynamic de Bruijn graphs. Bioinformatics 37: 1946-1952.

Indexed at, Google Scholar, Crossref

7. Yao L, Giulia A, Jurriaan H (2022) Vesicle-based artificial cells: materials, construction methods and applications. Mater Horiz 9: 892-907.

Indexed at, Google Scholar, Crossref

8. Chen C, Jie H, Xiaowen S, Hua Y, James AB et al. (2021)  GNET2: an R package for constructing gene regulatory networks from transcriptomic data. Bioinformatics 37: 2068-2069.

Indexed at, Google Scholar, Crossref

9. Pasi R (2020) Lep-Anchor: automated construction of linkage map anchored haploid genomes. Bioinformatics 36: 2359-2364.

Indexed at, Google Scholar, Crossref

10. Leah VS, Lissa CA, David SB, Michael RS, Rachel MM, et al. (2021) Construction of Human Proteoform Families from 21 Tesla Fourier Transform Ion Cyclotron Resonance Mass Spectrometry Top-Down Proteomic Data. J Proteome Res 20: 317-325.

Indexed at, Google Scholar, Crossref