Application of DBN and GWO-SVM in analog circuit fault diagnosis

For large-scale integrated electronic equipment, the complex operating mechanisms make fault detection very difficult. Therefore, it is important to accurately identify analog circuit faults in a timely manner. To overcome this problem, this paper proposes a novel fault diagnosis method based on the deep belief network (DBN) and restricted Boltzmann machine (RBM) optimized by the gray wolf optimization (GWO) algorithm. First, DBN is used to extract the deep features of the analog circuit output signal. Then, GWO is used to optimize the penalty factor c and kernel parameter g of support vector machine (SVM). Finally, GWO-SVM is used to diagnose the signal features extracted by the DBN. Fault diagnosis simulation was conducted for the Sallen–Key band-pass filter and a four-opamp biquad highpass filter. The experimental results show that compared with the existing algorithms, the algorithm proposed in this paper improves the accuracy of Sallen–Key bandpass filter circuit to 100% and shortens the fault diagnosis time by about 90%; for four-opamp biquad highpass filter, the accuracy rate has increased to 99.68%, and the fault diagnosis time has been shortened by approximately 75%, and reduce hundreds of iterations. Moreover, the experimental results reveal that the proposed fault diagnosis method greatly improves the accuracy of analog circuit fault diagnosis, which solves a major problem in analog circuitry and has great significance for the future development of relevant applications.

With the rapid development of science and technology, electronic circuits are increasingly being used in communications, industrial control, medical, household appliances, aerospace, military, and many other fields. As the complexity of these electronic circuits increases with circuits becoming more integrated, the requirements and dependency on diagnostic technology has also increased accordingly. For critical electronic equipment used in important fields, the diagnostic index requirements are stricter and more comprehensive and demanding better diagnostic accuracy.
For all electronic circuit equipment types, the size of the analog part accounts for approximately 20% of the device size, but the failure probability of the analog part accounts for more than 80% of the failure probability of the entire circuit equipment. Therefore, the analog part of the electronic circuit is a key part of the electronic circuit and therefore the entire system, and in turn critical to the system's reliability during operation.
Electronic circuit faults are mainly divided into soft faults and hard faults. The hard fault manifests itself as an enormous change in the component parameters. This results in a fundamental change in circuit topology, which greatly damages the overall circuit, such as an open circuit or short circuit. Soft failure refers to the deviation of the relevant parameters of electronic circuit components from typical values. For example, a switching power converter can lose its primary function due to the degradation of the LC filter's electrolytic capacitors (as would be the case in a Sallen-Key bandpass filter and a four-opamp biquad highpass filter). Therefore, it is clear that the timely diagnosis of soft faults is vital for safe system operation 1 .
Many studies have attempted to solve the key problems of circuit fault detection [2][3][4][5][6][7][8][9] . Although these studies have solved the non-linear problems and tolerance effects of the circuit's analog part to some extent, they have various disadvantages. Therefore, it is very important to investigate effective and simple feature extraction methods for extracting the basic features of analog circuit faults. To overcome the limitations of traditional methods, this paper proposes a novel analog circuit fault diagnosis method based on deep learning technology and the support vector machine (SVM) optimized by the gray wolf optimization (GWO) algorithm (GWO-SVM).
Analog circuit fault diagnosis is generally divided into two steps: the original signal feature extraction and fault classification. Notably, the deep belief network (DBN) has many advantages contributing to its superior performance in analog circuit fault detection [10][11][12][13][14][15][16] . First, the DBN has a multi-layer network structure, and its unique network training method allows the DBN to autonomously extract the features of a target and the essential expression of the data. Secondly, the DBN directly extracts the required features from the original signal without any function mapping and transformation, which can significantly improve the identification or distinguishability of the extracted original signal's features. Thirdly, the DBN can solve problems regarding the number of label samples and easy localization in training. The SVM model 17,18 is used in the classification of analog circuit faults. The optimization effect of the SVM largely depends on its penalty factor c, and kernel parameter g. When constructing a diagnostic model, parameters c and g must be set to the optimal values to achieve the best classification effect. The GWO was proposed in 2014 19 and implements the global optimization process by imitating the gray wolf population hierarchy system and hunting mechanism. The gray wolf optimization algorithm has faster convergence, has more stable operation, and can find suitable parameters for complex nonlinear problems.
This paper proposes an analog circuit fault detection method based on a DBN and GWO-optimized SVM for fault diagnosis. First, the different initial fault modes of the analog circuit are set; then, the DBN is used to extract the circuit's output signal as a feature. Finally, the GWO-SVM is used to diagnose the fault mode.

Feature extraction method based on DBN
Deep learning finds the essential expression of data through the layer-by-layer nonlinear mapping expression of the data and then realizes the deep extraction of features. The effective extraction of features is very important for fault diagnosis. The DBN is a typical deep learning method that can effectively extract feature information from the original signal.
Restricted Boltzmann machine. Support vector machines have the advantages of robustness, good ability for learning from small samples, and strong generalization ability. The restricted Boltzmann machine (RBM) solves the problem of feature division by exploiting high-dimensional space 10,20 .
The RBM model is shown in Fig. 1. There is a bidirectional connection between each unit of each layer, but no connection between the units in the layer. The input vector corresponds to the visible vector of the RBM. The hidden vector is the feature extracted from the visible vector. The energy of the RBM unit configuration is defined as follows: where W ij is the connection weight of the visible layer unit i and the hidden layer unit j; a i is the offset of the visible layer unit i; bj is the offset of the hidden layer unit j.
The conditional distribution of the visible layer is expressed as follows: The conditional distribution of the hidden layer is expressed as follows: where σ generally uses the sigmoid function as the activation function. The joint probability distribution between the visible layer and the hidden layer is defined as follows: where Z is obtained by summing all possible visible and hidden vectors, as follows: When inputting v, the hidden layer neuron h can be calculated using (2). Once the hidden layer is obtained, the reconstruction of the visible layer can be obtained using (3). When each v i in (3) is set to 1, the visible layer can be reconstructed.

Feature extraction based on DBN.
The DBN training process is essentially the process of mining deep representation information from the original features. The DBN is formed by stacking several RBMs, and the www.nature.com/scientificreports/ training process is performed layer by layer from low to high. Figure 2 shows a DBN consisting of three hidden layers.
where ε represents the learning rate, and �·� recon represents the expectation of the partial derivative under the reconstructed model distribution.
In the fine-tuning phase, the back-propagation algorithm is used to train the DBN network in reverse by fine-tuning the weight of each connection to improve the feature extraction effect and reduce the diagnostic error. The overall fine-tuning phase is a supervised learning process. The feature extracted by the last RBM after fine tuning is the feature extracted by the DBN.

Classifier based on GWO-SVM
When the pre-training and fine-tuning stages of the DBN are completed, it is necessary to accurately realize the fault division and location of the circuit. This study used the support vector machine algorithm to diagnose the faulty component.

Support vector machine.
The SVM constructs the best separation hyperplane in high-dimensional space to separate the decision categories. If the following conditions are satisfied, the training mode can be separated, as follows: where w is the normal vector of the hyperplane and b is the offset of the hyperplane.
Using the given training set data for training, the optimal solutions w* and b* of the hyperplane separated by the maximum interval can be determined as follows: The classification decision function is expressed as follows: In the case of non-linearly separable samples, a kernel function is typically used to construct the classifier. By using kernel functions, samples can be mapped to high-dimensional space to solve linearly inseparable problems in low-dimensional space. By introducing the relaxation variable ξi and the penalty function C, the problem becomes as follows: where N is the number of samples, i = 1, 2, … N.
In this case, the classification decision function is expressed as follows: www.nature.com/scientificreports/ In the case of linear inseparability, the input is mapped to a high-dimensional space, the kernel function K (x, xi) is introduced, and the linear classification after the nonlinear transformation is realized using the kernel function. The SVM optimal classification decision function is expressed as follows: where K(x, y) represents the kernel function. In this study, the radial basis function (RBF) was adopted.
Because the classification accuracy of the SVM model is greatly influenced by the parameter combination penalty factor c and the value of the kernel function kernel parameter g, this study used the GWO algorithm to optimize the selection of the optimal parameter combination of the SVM model. GWO optimizes SVM. The gray wolf optimization algorithm imitates the species leaderships and hunting mechanisms of gray wolves 19 , who surround, hunt, and attack according to their social rank. The predation tasks are assigned to different gray wolf groups to complete the predation behavior, so as to achieve global optimization. Compared with particle swarm optimization, genetic algorithm, grid search, and so on, the gray wolf algorithm has the advantages of having simple structure, easy operation, and short optimization time. In the GWO algorithm, the wolf with the strongest leadership ability is α, and is mainly responsible for the decision-making part of the optimization process; the remaining gray wolves are sequentially recorded as β, δ, and ω. Alpha is the optimal solution, β and δ are the sub-optimal solutions, and ω is the lowest gray wolf.
The distance between the prey and the wolves is expressed as follows: where t is the number of iterations; Xp(t) is the position of the prey after the tth iteration, that is, the position of the optimal solution; X(t) is the position of the gray wolf after the tth iteration, that is, the position of the potential solution; A and C are coefficient factors, and can be calculated as follows: where the value of a linearly decreases from 2 to 0 as the number of iterations increases; r1 and r2 are random numbers between [0, 1]. The mathematical model of gray wolf rounding behavior is expressed as follows: where D ∂ , D β , D δ and represent the distance between the α, β, δ, and ω wolves (that is, other individuals). Equations (19), (20), and (21) ensure that the gray wolf population obtains a better position than the initial point. When the optimal position appears, Xp is updated to a better position. The flow chart of GWO optimization SVM is shown in Fig. 3: The optimization process of SWO optimized by GWO is: Step 1 Initialize parameters such as population size N, distance D, maximum iteration number t, and randomly generated parameters a, A, and C; Step 2 Calculate the fitness value of each gray wolf individual, record the position of the top three gray wolf individuals as Xa, Xb, Xc, and record the best fitness Xa as the optimal solution at this time; Step 3 Train the SVM model, the best parameters c, g are the output of two Xa; Step 4 Calculate the distance between the remaining individuals ω and Xa, Xb, Xc according to Eq. (19), and update the position of the gray wolf α, β, δ and prey; Step 5 Update the values of parameters a, A, and C according to Eqs. (17) and (18); Step 6 If the algorithm reaches the maximum number of iterations t, then the algorithm ends and outputs the optimal solution Xa; otherwise, update the parameter values of each part and return to step2.

Simulation experiments and analysis
Analog circuit fault definition. Active filters are widely used by in many electronic circuits used for example in power systems, control systems, utility systems and used by precision electronics companies, medical institutions, and so on. Therefore, active filters play a key role in large electronic equipment systems. However, in practical applications, filter faults destroy system functions and even cause catastrophic losses. Therefore, in this study, the Sallen-Key band-pass filter circuit and a four-opamp biquad highpass filter circuit were selected as the test circuits, as shown in Fig. 4. For the Sallen-Key bandpass filter circuit in Fig. 4a, after sensitivity analysis 16 , the center frequency of the circuit is more affected by R2, R3, C1, and C2; therefore, R2, R3, C1, and C2 are used as experimental components. The same reason, C1, C2, R1, R2, R3, R4 are used as experimental components for four-op-amp biquad highpass filter circuit as shown in Fig. 4b. Because the frequency of a single device fault in the filter circuit is higher than the frequency of multiple device failures, this study only investigated the case of a single device fault 21 .
The first case investigated in this study is the Sallen-Key band-pass filter shown in Fig. 4a (hereinafter referred to as Case 1); the nominal values of the components are shown in Fig. 4a. Assuming that only one of the R2, R3, C1, and C2 components fails in each test, the circuit forms the nine different fault categories listed in Table 1 that including no fault mode, where ↑ and ↓ indicate a value higher or lower than the nominal value, respectively, and the tolerance values of the resistance and capacitance are 5% and 10%, respectively. Similarly, the fault categories of the four-op-amp biquad highpass filter circuit (hereinafter referred to as Case 2) are listed in Table 2.
Feature extraction and fault classification. The proposed fault detection method for analog circuits is described as follows: first, use the DBN model to extract the features of the analog circuit's output data. Then, use the GWO-optimized SVM classifier to classify the fault model. The specific method includes the steps shown in Fig. 5.
Step 1 Set the fault circuit mode and tolerance value of the faulty device. Case 1 has nine different fault categories, and Case 2 has 13 different fault categories.
Step 2 Use time-domain transient analysis to collect the signal information output of the simulation in step 1 and save it by category.
Step 3 Use Monte Carlo analysis to obtain 240 instances of each fault category.
Step 4 Divide the data obtained in step 3 into a training group and a test group.
Step 5 use the DBN to extract the features of the training group data.
Step 6 Use the GWO optimization algorithm to optimize the c and g parameters of the SVM.
Step 7 Perform fault diagnosis based on the feature data extracted by the DBN and optimal SVM model.
Step 8 Output the fault diagnosis result. The PSPICE software was used for data generation in the experiment, and the output amplitude-frequency characteristic curve of the circuit was obtained through AC analysis. The peak voltage of the circuit was 10 V, and a sinusoidal voltage source with a frequency of 1 kHz was selected as the excitation. Within the duration of 0 -1.2 ms, 128 data points were output in each group mode of the output node. Then Monte Carlo analysis was carried out to obtain 240 instances of each fault category. The total number for Case 1 was 2160, and the total number of examples for Case 2 was 3120. In the experiment, a DBN with three hidden layers was used. www.nature.com/scientificreports/ For Case 1, the circuit diagram is shown in Fig. 4a, and the DBN structure was set to 128-64-32-16-9. Based on experience 22,23 , the number of pre-training iterations for each RBM was 100, the number of fine-tuning iterations was 500, and the learning rate and momentum were set to 0.11 and 0.9, respectively. To clearly demonstrate the superior performance of the DBN method in feature extraction, this study used the kernel principal component analysis method (KPCA) to reduce the dimension of the original data and feature data extracted by the DBN. Then, the data features in the two cases were visualized in the spatial distribution. Figure 6a is a visual feature distribution diagram of the original data in Case 1 after dimensionality reduction. Figure 6b shows the visual feature distribution diagram of the original data of Case 1 after the DBN extracted the features. Compared with the original data visualization scatter diagram, there is only a slight overlap, and approximately all remaining modes are separated.
In Case 2, the DBN structure was set to 128-64-32-16-13. The remaining parameter settings were the same as in Case 1. Figure 7a shows the visual scatter diagram of the original data in Case 2, and Fig. 7b shows the visual  www.nature.com/scientificreports/ scatter diagram of the original data after the DBN extracted the features. Similar to Case 1, the different failure modes in the original data have severe overlaps. After the DBN extracted the features the overlapping parts were largely separated, as shown in Fig. 7b.
Based on the effective feature information extracted by the DBN, the GWO-SVM diagnostic model is used to classify the fault category. To improve the classification accuracy and shorten the classification time of the SVM, the GWO optimization algorithm is used to optimize the penalty coefficient c and kernel function parameter g of the SVM. In this study, the appropriate population size was set to 10, and the number of iterations was set to 100. The best parameters in Case 1 and Case 2 are listed in Table 3. For Case 1, when c was 93.3724 (retaining four decimal places) and g was 3.7042, the parameter combination of the GWO-SVM achieved the best effect, and the classification accuracy of Case 1 was 100%. For Case 2, when c was 100 and g was 26.9118, the GWO-SVM parameter combination achieved the optimal effect, and the case classification accuracy of Case 2 was 99.68% (retaining two decimal places).

Analysis of fault classification result.
For the experiments described in this section, we propose a diagnostic method for extracting data features from the DBN and classifying the faults using GWO-SVM. The proposed fault diagnosis method achieved diagnosis accuracy of 100% for the classification of circuit faults in Case  www.nature.com/scientificreports/ 1, and diagnosis accuracy of 99.68% for the classification of circuit faults in Case 2. The circuit fault classification result is presented in Fig. 8. In Fig. 8a, it can be seen that the fault classification in Case 1 is entirely correct. In Fig. 8b, it can be seen that, for the fault classification in Case 2, only the first and tenth categories include erroneous data. To verify that the GWO-SVM has faster convergence speed and higher accuracy, the GWO-SVM was compared with the GA-SVM and PSO-SVM. For fault classification, the fault features extracted by the DBN were input into the GWO-SVM, GA-SVM, and PSO-SVM, respectively. The fault diagnosis results of GA-SVM and PSO-SVM are shown in the Fig. 9 and Fig. 10. Obviously, compared with GWO-SVM, the fault diagnosis accuracy of GA-SVM and PSO-SVM for the two circuits is significantly reduced.
Receiver operating characteristic curve (ROC) can represent the trend of the sensitivity and accuracy of the classifier. The closer the curve is to the upper left corner, the better the performance of the classifier is. Generally, the ROC curve of the classifier is drawn to analyze its performance. By comparing GWO-SVM, GA-SVM, PSO-SVM and original SVM, the ROC curve is shown in Fig. 11. It can be seen from the figure that the ROC curve of GWO-SVM is closest to the upper left corner, which indicates that GWO-SVM has better performance.
The diagnostic performance of the three different classifiers in the two cases is presented in Tables 4 and 5. By comparison, it can be seen that the performance of GWO-SVM is superior to that of the other three classifiers. The test accuracy of GWO-SVM has a higher value. In Case 1, the classification accuracy of GWO-SVM was 100%, and the optimization process was completed in 18.48 s, the fault diagnosis time has been shortened by approximately 94%. In Case 2, the classification accuracy of GWO-SVM was 99.68%, and the optimization process was completed in 60.07 s, the fault diagnosis time has been shortened by approximately 91%. Figure 12 shows the fitness curves of the GA-SVM, Fig. 13 shows the fitness curves of the PSO-SVM, and Fig. 14 shows the fitness curves of the GWO-SVM. From the three figures, it can be seen that GWO-SVM classification has the highest accuracy and fastest convergence rate. Compared with other classifier models, the proposed GWO-SVM model only required one iteration in Case 1 to achieve the best fitness. In Case 2, the model only required 10 iterations to achieve the best fitness and stabilized after 35 iterations, compared with other models, the proposed model reduces hundreds of iterations and saves a lot of time. The results reveal that the proposed fault diagnosis method does not only significantly improve the convergence speed, but also has a low prediction error and achieves more robust prediction results. www.nature.com/scientificreports/ Moreover, we compare the proposed method with traditional analog circuit fault diagnosis methods and new technologies 5,14,16,[23][24][25] . As shown in Table 6, using the optimized classification tool can improve the classification performance. In addition, compared with reference 25 , the proposed method not only improves the accuracy of fault diagnosis, but also reduces the fault diagnosis time of the two cases by 89.98% and 74.67% respectively. In a word, the feature extraction method based on Dynamic Bayesian network proposed in this paper can learn deep and intrinsic data features from the original data, besides, GWO-SVM improves the performance of the classifier. Compared with the previous methods, the proposed method obtains the highest diagnostic accuracy and the shortest fault diagnosis time.

Conclusions
This paper proposes a fault diagnosis method based on DBN feature extraction and GWO-SVM classification. Different from traditional feature extraction methods, DBN can perform unsupervised deep feature learning to form hierarchical model. Additionally, the use of the GWO-optimized SVM greatly improves the accuracy of SVM classification and shortens the working time of the SVM. The results reveal that the GWO-SVM method can reduce hundreds of iterations and shorten calculation time by more than 75%. Although the proposed analog circuit fault diagnosis method has higher diagnosis accuracy compared with traditional diagnosis methods, the disadvantage of this method is that significantly large data sets will reduce the diagnostic performance. For large circuits, we guess that the circuit with complex structure can be divided into many different parts, and the output signal of each different module can be used for fault diagnosis, so as to realize the fault diagnosis of the whole circuit. Future work will focus on using analog circuit fault diagnosis theory in more complex circuits and in practical circuit fault diagnosis.    www.nature.com/scientificreports/