Systematic reduction of hyperspectral images for high-throughput plastic characterization

Hyperspectral Imaging (HSI) combines microscopy and spectroscopy to assess the spatial distribution of spectroscopically active compounds in objects, and has diverse applications in food quality control, pharmaceutical processes, and waste sorting. However, due to the large size of HSI datasets, it can be challenging to analyze and store them within a reasonable digital infrastructure, especially in waste sorting where speed and data storage resources are limited. Additionally, as with most spectroscopic data, there is significant redundancy, making pixel and variable selection crucial for retaining chemical information. Recent high-tech developments in chemometrics enable automated and evidence-based data reduction, which can substantially enhance the speed and performance of Non-Negative Matrix Factorization (NMF), a widely used algorithm for chemical resolution of HSI data. By recovering the pure contribution maps and spectral profiles of distributed compounds, NMF can provide evidence-based sorting decisions for efficient waste management. To improve the quality and efficiency of data analysis on hyperspectral imaging (HSI) data, we apply a convex-hull method to select essential pixels and wavelengths and remove uninformative and redundant information. This process minimizes computational strain and effectively eliminates highly mixed pixels. By reducing data redundancy, data investigation and analysis become more straightforward, as demonstrated in both simulated and real HSI data for plastic sorting.


Introduction
Hyperspectral Imaging (HSI) is a valuable non-destructive technology for online monitoring and high throughput screening, with huge potential to be automated [1][2][3][4].HSI , specifically Near InfraRed (NIR), is both fast and non-destructive, which makes it excellently suitable to perform realtime "online" in identification of polymers [5,6].The combination of HSI as a powerful tool for the optical sorting of plastics, and discriminant analysis, allows automatic sorting in real time under industrial conditions [7] into new well-characterized circular feedstock to replace virgin polymers.
The high throughput required to process sufficient material of the enormous amount of plastic waste puts strong demands on the computational time and storage resources available to support the choice for sorting every waste element into an adequate stream: data analysis, processing, and decisionmaking should be as fast as possible [8].An early implementation of NIR hyperspectral imagingbased cascade detection, exemplified by industry leaders like Steinert, Tomra, and Pellend, was already able to specifically sort post-consumer plastic packaging waste.Innovations in the development of such technology have been continuously ongoing [9][10][11].
The considerable promise of HSI, with its challenging combinations of multivariate spectral and spatially resolved information, have made multivariate methods essential to extract hidden chemical information [12].Chemometrics translates abstract spectroscopic fingerprints into throughput plastic species compositions of each waste element plays an indispensable role in the analysis of plastics during the recycling process.Chemometrics has proven invaluable, by variable selection [13], Multivariate Curve Resolution (MCR) [14], and classification algorithms to mix chemical domain knowledge with data-driven machine learning.It plays a vital role in tackling the complexities presented by challenging objects, particularly multilayers, as it enables the analysis and interpretation of the intricate spectral information embedded within these composite structures.However, the size of Hyperspectral data sets in both the spatial and spectral dimensions may considerably limit the speed of information extraction from such data.The Spectral and spatial redundancy of HSI data however enables a considerable data reduction to reduce resources sufficiently for high-throughput applications in the circular economy [15,16].
Non-negative matrix factorization (NMF) with the inherent non-negativity property originates in signal/image processing with some reports in analytical chemistry yet is largely mathematically equivalent to Multivariate Curve Resolution (MCR-ALS) [17,18] that is a cornerstone of chemometrics.The NMF algorithm has been evaluated in chemistry e.g. for the resolution of overlapped GC-MS spectra, as well as for the deconvolution of GC × GC data set [19].
Besides, NMF has been conducted to classify complex mixtures based on the extracted feature and to analyze time-resolved optical waveguide absorption spectroscopy data [16].NMF recovers pure spectra and their concentration contribution maps of HSI Raman images [20].Another study applied an NMF filter to triboluminescence (TL) data traces of active pharmaceutical ingredients [21], leading to simultaneously recovering both photon arrival times and the instrument impulse response function, i.e. is of considerable value to recover chemical information from HSI data.Zushi proposed an NMF-based spectral deconvolution approach, coupled with a web platform GC mixture [22], that utilizes a faster multiplicative update method instead of the traditional projection step.This approach is highly advantageous for analyzing large mass spectrometry imaging datasets due to its improved speed [23].
Monolayer materials composed of a single polymer species or multilayer materials made up of several polymers are common in plastics.With the help of a single spectral profile, it is possible to identify the polymer composition of monolayer objects [24].When it comes to identifying the polymer composition of waste streams, the objects encountered are often much more complex than monolayer materials.These objects can be multilayered or composed of multiple polymer species, or even coated with labels made of different polymers.Furthermore, an object may be made up of known polymers with an unknown composition.In the case of multilayer plastics, the type and ratio of plastics used can also vary significantly, such as 70/30 or 50/50.This adds significant complexity to the classification process, which is where the importance of data unmixing comes into play.
Data unmixing is a valuable tool that can handle these issues effectively.However, when sorting the objects, the decision to translate a model observation into a specific engineered materials stream may only be made on a small subset of the complete object.Therefore, reducing the size of the data using convex hull is an excellent innovation in curve resolution that can reduce the data to less than one percent of the original data.Handling large HSIs, data compression/size reduction coupled with rapid data analysis tools has top priority.So, unnecessary pixels (data rows) and wavelengths (data columns) should be jointly removed and the rest are essential to characterize.For the study in this research, removing all unnecessary information and the reduced data will be subjected to further decomposition, NMF.Simulated and real experimental data are exemplified.

A brief description of Non-Negative Matrix Factorization
Nonnegative Matrix Factorization (NMF) deconvolutes a matrix,  × , into the product of matrices  × and   × T with an intrinsic non-negativity property as a minimal constraint.I and J represent the number of spatial pixels, while K represents the number of variables in the case of a hyperspectral image.
To this, formalize NMF optimizes the following cost function: Different algorithms are introduced in the literature to calculate component matrices, The multiplicative update rule introduced by Le ad Seung is simple to implement [6].The updating parts are: and Equations 3 and 4 denote element-wise multiplications.These update rules preserve the nonnegativity of  × and  × where  × is element-wise non-negative.

Approach
For any bilinear data set, a minimum number of rows and columns carry the most informative and independent part of the data [25].Consequently, in HSIs, essential pixels, and essential wavelengths are necessary to extract the pure contribution maps and spectral profiles of all components.The main steps of this approach for plastic characterization are visualized in Figure 1 and summarized as: 1) Object detection: Recording a first-order spectrum with K channels for every pixel in an I by 4) Decision making: The matrix  × consists of pure spectral profiles of the components, which can be utilized for qualification by comparing them to a reference library.However, in some cases,  IJ×n has complementary information for identification. IJ×n , contains characteristic information about the composition of unknown objects.The sum of the squares of the elements in each column of WIJ×n represents the variance of the signal contributed by each polymer in an unknown object.This variance can be used to differentiate between monomaterial and multi-material objects using a statistical F-test, which is commonly employed in statistics to compare the standard deviations of two populations.To conduct the F-test, the variance of all columns in WIJ×n is calculated in the first step, and then each value is divided by the noise variance.A significance level of P < 0.05 is used to determine whether the variance is statistically significant.If the object contains only one layer of material, only one type of polymer will pass the F-test.However, if it is a multilayer object, multiple types of polymer will pass the test.Finally, it should be emphasized that all of the computation and reported times in this work are based on a laptop (Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz) which for real industrial purposes can be dramatically improved by using better computers.The utilization of this algorithm enables the exploitation of the computational capabilities of a standard computer system, thereby eliminating the reliance on specialized hardware such as GPUs or high-performance computing clusters.

Simulated data set
A hyperspectral image was simulated to visualize the effect of essential pixels and variables selection using convex polytope and further unmixing by NMF.The concentration contribution maps (refolded concentration profiles) and pure spectral profiles are shown in Figure 2. The simulated hyperspectral data set is of dimensions 253 × 186 pixels by 141 variables and the unfolded two-way data matrix of 47058 pixels and 141 pseudo-spectral channels.Despite the simplicity of the simulation, it should be noted that care was taken to avoid the pure pixels or selective spectral channels, this corresponds to a non-trivial situation for NMF analysis.For this purpose, small random numbers were added to the pure contribution maps.In this case, eight objects are on the hypothetical conveyor belt which are made of polypropylene (PP), polyethylene (PP), and polyethylene terephthalate (PET).Five and three objects are monolayers and multilayers, respectively.Figure 2 presents the pure contribution maps and spectral profiles of PP, PE, and PET.

Experimental Hyperspectral Images of Plastics
A collection of monolayer and multilayer objects made from PP (polypropylene), PS (polystyrene), PET (polyethylene terephthalate), and polypropylene on top of polyethylene (multilayer), all with known compositions, were gathered from packaging waste.The characteristic information of these objects was known in advance.To collect data, the objects were randomly placed on a conveyor belt, and two HSI measurements were recorded in the 900-1700 nm range.The resulting data sets are presented in Figure 3.

Result and Discussions
To illustrate the effect of essential spectral pixels (ESPs) and/or essential spectral variables (ESVs) selection of the plastic sorting using NMF, the results obtained on the simulation and real HSI data set are discussed in detail.
The procedure starts with object detection for the simulated case as it does not need any data perprocessing.Then, the ESPs and ESVs should be selected using a convex hull on the normalized abstract spaces of the data set for each object.First five Principal Components were used to generate the abstract spaces.The selected essential pixels and wavelengths are presented in Figures 4a and  4b.In Figure 4 Later, the reduced data sets were analyzed by NMF.The results of NMF are called W and H.To make a better visualization of the full contribution maps, W changed to binary matrices and visualized in Figure 5.The total calculation time for NMF analysis of the reduced data sets was nearly 0.001 sec, 1% of the time required for the same calculations on all pixels and wavelengths.In addition, strategic data pre-processing methods are essential prior to data analysis to linearize the data, remove artifacts and thereby optimally align the data and the NMF with the Beer-Lambert-Bouguer law. Figure 6 presents the effect of each pre-processing step on the shape of the spectral data for an example object.Figure 6a is the recorded raw HSI of an optional object.Some spectral profiles which reflected the light can be removed first.These profiles can be recognized by the slope of the spectral profiles in the range of 1100-1500 nm.In this wavelength range the slope of the profiles are zero (all saturated).
Figure 6b presents the normalized data after removing the saturated spectral profiles.The next step (6c) is the treated data after baseline correction of the using asymmetric least squares on the minus logarithm of the data set.Finally, the data set is reconstructed by a few principal components for denoising purposes.The corrected data is presented in Figure 6d.Once the data has been preprocessed, it undergoes object detection, as detailed in the theory section.This process yields smaller data sets, each one corresponding to a single object and containing relevant information.In the current scenario, object detection yields five data sets (corresponding to the five objects on the conveyor belt).The crux of the proposed algorithm lies in selecting the appropriate ESPs and ESVs based on the data's inner polygon in abstract spaces.Figures

Fig. 1 .
Fig. 1.A Graphical illustration of the approach based on the essential pixel/wavelength selection and NMF.The hyperspectral data with a 3D structure, needs object detection first.Then the selected parts of the data which correspond to the objects unfold to matrices to calculate the ESPs/ESVs.Finally, the reduced data sets are analyzed by NMF and full contribution maps and spectral profiles are retrieved.In the last step, F-test will help for decision making to monolayer/multilayer plastic sorting.

Fig. 2 .
Fig. 2. The three components simulated the HSI data set.Concentration distribution maps and pure spectral profiles are presented.

Fig. 3 .
Fig. 3.The experimental cases are visualized in a and b.

Fig. 4 .
Fig. 4. The selected ESPs and ESVs for the simulation case are indicated as white crosses and red lines, respectively.

Fig. 5 .
Fig. 5.The full contribution maps of all components in simulated data are visualized.Each color is used for special composition.Blue, green, red, aqua, and purple, brown, are used for, PP, PE, PET, PP/PE (multilayer), PP/PET, and PE/PET, respectively.

Fig. 6 .
Fig. 6.Depicts the data set corresponding to the first unknown object, visualized after undergoing various pre-treatment steps.These steps include object detection and removal of pixels affected by signal saturation, baseline correction of the negative logarithm of the data, and denoising using Singular Value Decomposition.

7 and 8 Fig. 7 .Fig. 8 .Fig. 9 .
Fig. 7. Depicts the mean image and spectral profile of the first experimental case, presented in (a) and (b) respectively.The chosen ESPs and ESVs are denoted by white crosses and red lines.
[25]e into a data cube,  ̃×× .To analyze this cube for plastic characterization, object detection is needed.In this work the corresponding pixels of each object were detected by correlation growing algorithm.The results of object detection on  ̃×× are several cubes (as many as the number of objects) and each of them contains information about one of the objects.Object detection decomposes  ̃×× to some cubes.×=,  ,  , T +  , can be calculate, where  , and  , T are the left and right eigenvectors, respectively., and  , contains singular values and residuals, individually.In addition, n is the number of factors.The number of factors is set up to five.Because most of the objects contain less than five types of polymers/materials."Convhulln"as a MATLAB function can calculate the convex set of  , and  , T and explore the ESPs/ESVs.After join selection of essential pixels and essential variables,  × turn into   ×  .This step is explained in detail in the previous works[25]. × and  IJ×ESVs which are one-mode reduced data in the row and column direction respectively.Finally, it is easy to reshape the columns of  IJ×n to generate the full concentration contribution maps.
2) Essential information extraction: This step will start by unfolding each data cube (resulted from previous step) to a matrix ( × ) with each row a pixel for further analysis.Calculating the most informative pixels (Essential Spectral Pixels, ESPs) and variable/wavelengths (Essential Spatial Variables, ESVs) for each object, which are based on the convexity property in the normalized abstract row and column spaces, is the next step.ESP/ESVs are the smallest set of points needed to generate the whole data in a convex way in the abstract score space.Once ESP/ESVs are identified for all objects separately, all other measured pixels are removed and the reduced data is moved to the next step.Left/right eigenvectors of  × using SVD, 3) Data decomposition: The reduced data sets for all objects,  × can be analyzed by NMF in parallel with multiplicative updates, to calculate the reduced concentration contribution maps,  ×  and reduced spectral profiles  ×  using Eq. 3 and 4. Using least square, full concentration contribution maps,  × , and full spectral profiles,  × can