Data-fusion for in-situ monitoring and molten state identification during LPBF of NiCoCr medium-entropy alloy

Laser powder bed fusion (LPBF) is an additive manufacturing technology with high practical value. In order to improve the quality of the fabricated parts, process monitoring has become a crucial solution, offering the potential to ensure manufacturing stability and repeatability. However, a cardinal challenge involves discerning a precise correlation between process characteristics and potential defects. This paper elucidates the integration of an off-axis vision monitoring mechanism via a high-speed camera focused on capturing the single-track melting phenomenon. An innovative image processing method was devised to segment the plume and spatters, while Kalman filter was employed for multi-object tracking of the spatters. The features of both the plume and spatters were extracted, and their relationship with molten states was investigated. Finally, the PSO-XGBoost algorithm was utilized to identify five molten states, achieving an accuracy of 92.16%. The novelty of this approach resides in its unique combination of plume characteristics, spatter features, and computationally efficient machine learning models, which collectively address the challenge of limited field of view prevalent in real production scenarios, thereby enhancing process monitoring efficacy. Relative to existing methodologies, the proposed PSO-XGBoost approach offers heightened accuracy, convenience, and appropriateness for the monitoring of the LPBF process. This work provides an effective and novel approach to monitor the LPBF process and evaluate the part fabrication quality for complex and changeable working conditions.

www.nature.com/scientificreports/were inputted as samples into the PSO-XGBoost model for the classification of the molten states.This work offers an alternative to traditional destructive offline inspection methods for monitoring and evaluating LPBF processes.Moreover, this novel approach has the potential to be extended and applied to other laser processing techniques as well.

Materials and experimental setup
In multi-principal element alloys (MPEAs), the NiCoCr medium entropy alloy (MEA) stands out for its remarkable strength and ductility, exhibiting superior performance at both low and high operating temperatures 28 .Owing to its exceptional mechanical properties and corrosion resistance, this alloy has garnered significant attention across industries such as aerospace, automotive, and energy, making it also a suitable candidate for applications in metal additive manufacturing 29 .In this study, we employed NiCoCr medium-entropy alloy powder occurring to its with particle sizes ranging from 15 to 53 μm for our experiments.The elemental composition of the powder is detailed in Table 1. Figure 1 illustrates the schematic diagram and the practical setup of the coaxial system used in LPBF.The LPBF device, DiMetal-100H, is equipped with a 200 W fiber laser having a beam diameter of 50-70 μm.The build chamber has dimensions of 100 mm × 100 mm × 100 mm.The camera was positioned outside the printing chamber at a 35° angle relative to the optical axis of the build platform.Simultaneously, the camera's frame rate was configured at 2100 fps with a resolution of 1280 × 720, and the shutter speed was adjusted to 25 μs to minimize motion blur.Various single tracks were obtained by altering the process parameters (as detailed in Table 2), with each track measuring 70 mm in length and following a horizontal scanning direction.Prior to each experiment, the chamber door was sealed, and Argon gas was introduced to evacuate any air, guaranteeing an oxygen-free environment inside.In this study, over 5000 frame images were captured for each parameter combination.It is worth noting that, for configurations involving higher speeds, additional experiments were conducted to ensure a sufficient sample size and maintain sample balance for subsequent machine learning analysis.In the examination of a single track, our primary emphasis was on understanding how various laser energy inputs affect process characteristics and molten states.We did not delve into the impact of powder layer thickness or laser spot diameter.As a result, our analysis was confined to laser linear energy density 30 , defined as: where P is the laser power, and v is the laser scanning speed.

Melt width measurement and molten state assessment
The morphology of the melt track, both from a top-view and a cross-sectional perspective, was examined using a metallurgical microscope.To observe the cross-section of the melt track, the substrate was first cut into small pieces after single-track printing.Subsequently, these cross-sections underwent a series of polishing steps using 200, 400, 800, 1000, and 1500-grit sandpaper.This was followed by a polishing process using alumina suspension with particle sizes of 1, 0.5, 0.04, and 0.02 μm to achieve a smooth surface finish.Finally, the polished samples were etched using a mixture of 1 g CuSO 4 •5H 2 O, 25 ml HCl, and 1.8 ml H 2 SO 4 .A thorough analysis was carried out on the surfaces of the observed melt tracks under the microscope, which encompassed factors such as continuity, width and height uniformity, and roughness.Various molten states were identified as the resultant categories of interest.Furthermore, measurements of the melt tracks were taken under varying parameters to assess the impact of input energy on both the width and height of the melt tracks.

The overall experimental process
The melting state reflects the defects to some extent and, at the same time, can be used as a basis for classification.
Based on the experimental parameters mentioned above, splashing images of the single-track forming process were collected under different linear energy density inputs.The obtained splash process images were segmented to extract features such as average splash and feather flow pixel area, average particle velocity, and number of splashing particles per frame.Based on these features, a thorough analysis and discussion of the interaction mechanism between laser and metal powder were conducted.Finally, in order to validate the accuracy and reliability of the information reflected by the splashing and feather flow image features, a PSO-XGBoost classification model was constructed.The features and results of the single-track experiment were used as inputs and outputs of the model, respectively.The classification accuracy of the model was then tested.The experimental process is shown in Fig. 2.

Image processing
The original image captured by the camera underwent several processing steps.Firstly, it was cropped to an appropriate size and then converted into a grayscale image.Following this, the resulting grayscale image was (1) www.nature.com/scientificreports/subjected to denoising through Median filtering.Subsequently, the Otsu algorithm 31 was applied to segment the image.This segmentation process effectively differentiates between the plume and spatters due to the substantial disparity in area.For visual reference, Fig. 3 shows the steps of image processing.And the actual and detected plume and spatters can be observed in Fig. 4.

Spatter tracking
The Kalman filter is a technique founded on the fundamental concept of minimizing error covariance.It is employed for prediction-correction type estimation of unknown state variables 32 .In this study, the Kalman filter is utilized to estimate and forecast the trajectory of spatters, subsequently allowing for the calculation of particle velocity.The movements of the particles are considered as combination of uniform rectilinear motion with random disturbance.In the Kalman filter framework, the state vector of a specific target particle at time t is defined as s t = [x t , y t , w t , h t , v t,x , v t,y ] ⊤ , where x t , y t are the coordinates of the target particle's centroid, w t , h t are the width and height of the target particle's bounding rectangle, v t,x , v t,y are the velocities in the x , y direc- tions, respectively.The observation vector is defined as z t = [x t , y t , w t , h t ] ⊤ .The prior estimate system state and www.nature.com/scientificreports/posterior state at t are denoted by s t and ŝt , respectively.P t and Pt represent the corresponding covariances.In this study, the state transition matrix and observation matrix H in the Kalman filter are as follows:   www.nature.com/scientificreports/

Prediction step
The calculation of the a priori estimate s t and covariance P t at time t can be expressed as: where is the state transition matrix, w is the process noise and Q is the process noise covariance.P t is the prediction of the movement of the particle.

Update step
The Kalman gain can be calculated as follows: where H is the observation matrix and R t is the measurement noise covariance, and the calculation of the pos- teriori estimation and covariance is as follows:

Tracking method based on the Kalman filter
The comprehensive diagram of the tracking method is shown in Fig. 5. Within this diagram, "Detections" symbolize the detected target state information, while "Tracks" represent the trajectory details. Figure 6 shows the Schematic of the Kalman-filter-based spatter tracking method.The process of multi spatter tracking is shown in the Supplementary Video 1).The algorithm unfolds as follows: (1) For the first frame, create Tracks corresponding to the detected results.Initialize the motion variables of the Kalman filter and predict the corresponding bounding box through Kalman filtering.At this point, Tracks must be unconfirmed.(2) Match the detected bounding boxes of the current frame with the predicted bounding boxes from the previous frame based on the IOU (Intersection over Union), and find the optimal matching result using the maximum weight matching algorithm.
( www.nature.com/scientificreports/where f represents a regression tree, K is the total number of regression trees, and F represents the space of regression trees.The objective function is represented as follows: where l(y i , ŷi ) is the loss function used to measure the error between the predicted classification value and the true value ŷi is the predicted classification value, and y i represents the true value.�(f k ) represents the regularization term.XGBoost algorithm adopts gradient boosting iterative computation.After each iteration, a new regression tree is added.Therefore, the result of the tth iteration is as follows: and the objective function expression for the tth iteration is given by: To expand Eq. ( 4) using second-order Taylor approximation and incorporate the regularization term �(f k ) to prevent overfitting, which can then be approximated as follows: where γ represents the leaf node penalty factor, T is the number of leaf nodes, ω represents leaf weights, and λ is the weight penalty factor.

Particle swarm optimization
Particle Swarm Optimization (PSO) is a swarm intelligence algorithm inspired by the foraging behavior of bird flocks.The PSO algorithm defines the solution to an optimization problem as the search for particles within a finite-dimensional space.Each particle is characterized by both a position vector and a velocity vector.All particles cooperate to find the optimal solution by continuously updating their positions based on their own best values and the best values found by the entire particle swarm.The fitness function evaluates the fitness of each particle's position, and all particles in the swarm follow the current best particle to explore the solution space.
Assuming a D-dimensional search space with a total of m particles in the swarm, the position of the i-th particle is represented as a vector X i = [x i1 , x i2 , ⋯ , x iD ], the velocity vector is represented as and the best position found by the particle itself is represented as P i = [P i1 , P i2 , ⋯ , P iD ].The overall best position found by the entire particle swarm is denoted as P g = [P g1 , P g2 , ⋯ , P gD ], where g represents the particle index, and g ∈(1,2,3, ⋯ m).After initializing the particle swarm, the PSO algorithm calculates the fitness value for each particle and continuously updates and iterates to search for the optimal solution.In each iteration, the particle X i updates its position and velocity based on its individual best value P i and the swarm's best value P g .The iteration formula is as follows: where k is the number of iterations; ω is the inertia factor, which is used to control the convergence and searching ability of the algorithm; r 1 and r 2 are random numbers between [0, 1]; c 1 and c 2 are the acceleration factors, which denote the weights of the acceleration terms that push the particles to the individual optimum P i and the population optimum P g , respectively.

PSO-XGBoost model
Utilizing the principles of PSO and XGBoost theory, this study employs PSO to optimize the parameters of the XGBoost classifier, ultimately constructing a PSO-XGBoost classification prediction model.PSO-XGBoost algorithm flowchart is shown in Fig. 7.The specific procedure is shown as follows: Figures 8 and 9 show the feature of spatters and plume.The average number of spatters refers to the average number of spatter particles per frame, while other features such as the average spatter velocity and average plume area refer to the average values per particle across all frames.With an increase in laser power, the interaction between the laser and material intensifies, magnifying thermal effects.Consequently, within the same duration, a more extensive melt zone emerges, producing a wider track and a pronounced influence on the powder.Additionally, the increase in laser power enhances the heat transfer effect, resulting in a wider and deeper affected region.The temperature fluctuations on the powder's surface become more drastic, which generates a significant amount of metal vapor.As a result, the melt pool's stability diminishes, and as illustrated in Fig. 8a-c, both the quantity and size of the spatters escalate due to increased recoil pressure and surface tension.However, when the laser power increases from 150 to 200 W, excessive energy input deeply melts the substrate, resulting in an increase in the viscosity of the melt pool and a decrease in the number and size of spatters, as shown in Fig. 8a-c.Although the formation of plumes becomes more prominent (Fig. 8e, f), the spatters are unable to separate from them, resulting in a diminished spatter count.
Between 200 and 250W, the mass of the molten material continues to increase, and the formation of liquid spatters gradually resumes.As the viscosity of the melt pool decreases, the size and quantity of spatters increase once again (Fig. 8a-c).Given the added energy and momentum, the plumes not only expand but the spatters also manage to detach from them, leading to a rise in their quantity and dimensions.
From an initial observation, as the laser scanning speed increases from 50 to 150 mm/s, both the quantity of spatters and the plume area decline (as seen in Fig. 9a, e, f).However, as the laser scanning speed continues to increase, there is a noticeable trend of an upward trajectory in the spatter parameters.This indicates that under www.nature.com/scientificreports/ the same laser power, a rise in scanning speed leads to a longer formed track length within the same timeframe, resulting in a larger region affected by the Bernoulli effect.Specifically, along the direction of laser travel, there are more powder particles on both sides of the weld bead being entrained by metal vapor, forming spatters.Consequently, the number of spatters increases with the improvement of scanning speed.This process may also involve spatter aggregation.Due to the relatively small overall change in input energy, the variation in spatter area is not significantly noticeable (Fig. 9b, c).www.nature.com/scientificreports/Additionally, as the power increases, the volume of evaporated material rises, leading to heightened recoil pressure.This subsequently results in an acceleration of spatter particle velocity, as depicted in Fig. 8d.However, there is no observed decrease in spatter velocity with an increase in laser scanning speed (Fig. 9d).This may be attributed to the fact that changes in the laser scanning speed at this juncture do not bring about a considerable shift in the input energy.
Notably, with parameters set to 50 mm/s-50W and 100 mm/s-100W, even though both maintain identical line energy densities, all process characteristics appear more accentuated in the latter, as illustrated in Figs. 8 and  9.This might be attributed to the simultaneous doubling of speed and power in the 100 mm/s-100W setting, directly correlating to a twofold increase in laser power density (i.e., power/beam area).Concurrently, this could lead to steeper temperature gradients and accelerated melting rates, potentially resulting in faster liquid metal movement and enhanced metal melting, thus impacting the melt pool's stability.

Molten states
Depending on the morphology of the tracks at different energy inputs, it can be categorized into five different molten states: insufficient molten state (Fig. 10a), slight insufficient molten state (Fig. 10b), normal molten state (Fig. 10c), slight excessive molten state (Fig. 10d), and excessive molten state (Fig. 10e).From Fig. 10e, it can be observed that when the energy input is too high, the surface of the melt track is rough.Meanwhile, due to the large fluctuation of the melt pool, the consistency of width and height is poor.The track in the normal molten state is shown in Fig. 10c, with reduced track width, smooth and continuous surface, and good consistency of width and height.The undersupply molten state, as shown in Fig. 10a, has a smaller laser-affected area, resulting in a very thin melt track.At the same time, due to the low input energy, the wettability of the melt track is poor, and the continuity and consistency of width and height are poor, with obvious balling phenomenon.The slight excessive molten state, as shown in Fig. 10d, is between the excessive molten state and the normal molten state.And the slight insufficient molten state, as shown in Fig. 10b, is between the normal molten state and insufficient molten state.Moreover, as shown in Fig. 11, the width and height of the melt track increase with the increase www.nature.com/scientificreports/ of laser power, and decrease with the increase of laser scanning speed.In addition, the fusion channel shows a flat and wide characteristic, and its depth-to-width ratio is in the range of 0.35-0.80.This indicates that the high-energy laser beam shows significant diffusion and scattering of energy when acting on the metal powder.

Results of classification
The relevant parameters were determined by the PSO algorithm respectively, and these parameters were used in the XGBoost.To train the proposed model, 75% of the data was randomly selected, and the remaining 25% was utilized to test and validate.K-fold cross-validation was selected for error estimation, and the classification accuracy was 92.16%.The predicted results of the model can be calculated by applying the model to the test set, as shown in Fig. 12.
The confusion matrix is a matrix where the actual categories form the rows and the predicted categories form the columns.It provides a visual representation of classification results.Each element in the confusion matrix represents a specific classification outcome.The diagonal elements of the matrix represent the number of correct predictions for each category.Therefore, a confusion matrix with larger values on the diagonal indicates better predictive performance.The confusion matrix of the test data is shown in Fig. 13.This indicates the feasibility of the proposed method to capture process characteristics and its potential as a tool for process monitoring.
The training loss and testing accuracy of the PSO-XGBoost classifier are also provided in Fig. 14.As observed in Fig. 14, the training loss of PSO-XGBoost classifier is close to zero after 90 epochs, and the testing accuracy is stable above 90%.It can also be concluded that the PSO-XGBoost classifier trained on the collected signals has high efficiency.
Figure 15 shows the importance of the features.Quantity of the Spatter, Area of Plume, Area of Spatter and Velocity of Spatter are the four features that are key variables when assessing the molten states.www.nature.com/scientificreports/importance, and effective management of high-dimensional data, making it a superior choice that balances practicality with deep insights into the complex melting processes.

Conclusion
In this research, an off-axis vision monitoring system for Laser Powder Bed Fusion was developed using a high-speed camera.Different process parameters were set to obtain single tracks, and the printing process was recorded via the high-speed camera.In order to effectively extract the features of the process phenomenon, a method combined with the Otsu algorithm and Kalman filtering technique was proposed in this study.The area, width, and height of the plume, as well as the different characteristics of spatter such as area, quantity, velocity, width, and height, were studied in different scenarios.By monitoring the feature of spatters and plume in the LPBF process, we found that the linear laser energy density has a significant influence on the behavior of spatters and plume.The statistical data also showed that the width and height of the melt track were positively correlated with the energy input.From the measurements we found that the depth-to-width ratio of the melt channel was in the range of 0.35-0.80,which indicates the phenomenon of laser energy dilution.In addition, we found 5 states of the melt track including the insufficient molten state, the slight insufficient molten state, the normal molten state, the slight excessive molten state and the excessive molten state.In order to accurately evaluate the molten state, a molten state evaluation model based on PSO-XGBoost algorithm is proposed in this study.To overcome the difficulty of XGBoost algorithm parameter adjustment, PSO is used to optimize the parameters of XGBoost algorithm, which improves the efficiency and prediction accuracy.The experimental results show that the accuracy, precision, recall and F1 score of the PSO-XGBoost model are 92.16%,92.16%, 92.14% and 92.15%, respectively, indicating that these eight types of characteristic information points of the spatters and plume have significant application value in reflecting the manufacturing status in the LPBF technology.Compared to other models mentioned in the paper, the obtained results offer enhanced interpretability and exhibit greater robustness and generalization capabilities.Overall, this research offers a novel and promising method for process monitoring in additive manufacturing.
In future research, we will investigate the effectiveness of machine learning in the challenging task of online monitoring of LPBF defects under complex and variable conditions.Improvements will be tailored to the different requirements of LPBF processes for defect monitoring under complex and variable conditions, including the development of a customized online monitoring software system. https://doi.org/10.1038/s41598-024-65545-9

Figure 1 .
Figure 1.In-situ monitoring system for the LPBF process.(a) Schematic of the system.(b) Actual scenes of the system.

Figure 2 .
Figure 2. Flow chart of the experimental scheme.

Figure 3 .
Figure 3. Steps of image processing.

Figure 4 .
Figure 4. (a) The actual spatters and plume.(b) The detected spatters and plume.

Figure 8 .
Figure 8. Analysis of plume and spatter features in different laser power when the laser scanning speed is 50 mm/s.(a) The average quantity of spatter.(b) The average pixel area of spatter.(c) The average pixel width and pixel height of spatter.(d) The average speed of spatter.(e) The average pixel area of plume.(f) The average pixel width and pixel height of plume.

Figure 9 .
Figure 9. Analysis of plume and spatter features in different laser scanning speeds when the laser power is 100W.(a) The average quantity of spatter.(b) The average pixel area of spatter.(c) The average pixel width and pixel height of spatter.(d) The average speed of spatter.(e) The average pixel area of plume.(f) The average pixel width and pixel height of plume.

Figure 11 .
Figure 11.Analysis of melt track widths and lengths.(a) Changing with laser power.(b) Changing with laser scanning speed.

Figure 12 .Figure 15 .
Figure 12.The prediction of the PSO-XGBoost classifier, where C1 to C5 represent insufficient molten state, slight insufficient molten state, normal molten state, slight excessive molten state and excessive molten state, respectively.

Figure 16 .
Figure 16.The ROC curves of all classes.

Table 1 .
Chemical composition of NiCoCr medium-entropy alloy powder.

Table 2 .
Process parameters used in this study.

Table 3 .
The evaluations of different models.