Background & Summary

Polarization multispectral imaging (PMI) acquires four-dimensional information in polarization, spectral and spatial domains. Benefiting from the rich information acquired, PMI has been attracting widespread attention from researchers in various fields. Concretely, PMI shows great significance for target detection involving specular reflection inpainting1, background segmentation2 and tensor representation3. In remote sensing, PMI is closely related to marsh vegetation classification4, coastal wetland classification5 and leaf nitrogen determination6. PMI also develops rapidly in biological diagnosis, demonstrating compelling effectiveness in revealing skin complications of diabetes mellitus7 and characterizing microstructures of cancerous tissues8. Obviously, the stability, reliability, efficiency and universality of imaging methods determine the popularity of PMI technology in various fields.

Generally, PMI is being improved in both imaging systems and reconstruction algorithms. Snapshot channeled imaging spectropolarimeter usually consists of multiple retarders and polarizers, and then reconstructs information based on Fourier transform9 or neural networks10. Snapshot division imaging spectropolarimeter captures multiple images simultaneously by adopting a four-quadrant retarder array11, a staircase-grid lenslet array12 or a pixelated micro-polarization array13. There is no doubt that optical device division requires advanced technology and prohibitive cost. Furthermore, compressive sensing (CS) theory14 enables conventional optical devices for compressive imaging spectropolarimeter, such as retarder15,16,17,18,19, polarizer15,17, coded aperture15, double Amici prism15, Wollaston prism16, digital micromirror devices18 and liquid crystal tunable filter (LCTF)17,18,19. Recently, significant breakthroughs in PMI are often associated with novel materials. Full-Stokes polarization camera can be implemented by designing diffraction grating metasurfaces20, pixelated polarizer metasurfaces21, or vectorial Fourier metasurfaces22. In addition, spectral and polarization information can be obtained simultaneously by exploiting organic photodetectors23,24. These novel materials greatly simplify imaging systems without standard optics.

Nonetheless, all PMI improvements require simulation validation based on full-Stokes polarization multispectral images (FSPMI). Obviously, it can be found that FSPMI adopted for validation are varied in current studies due to the lack of relevant databases. This phenomenon not only increases the costs and difficulties of measuring FSPMI experimentally, but also prevents researchers from efficiently comparing existing imaging methods based on uniform FSPMI. On the contrary, other publicly available databases25,26,27 are extremely conducive to the multidimensional and extensive development of related technologies.

In this paper, we therefore publicize an FSPMI database, including four Stokes parameters, 18 spectral bands, 512 × 512 spatial pixels and 67 stereoscopic objects. The FSPMI database is carefully measured by establishing an experimental system consisting mainly of a quarter-wave plate (QWP), a linear polarizer (LP), 18 bandpass filters and a complementary metal oxide semiconductor (CMOS) detector. By switching the bandpass filters, the broadband spectral intensities of the object reflected light are modulated and compressed into multiple narrowband spectral intensities to be sequentially detected by the CMOS. Meanwhile, by rotating the fast axis of QWP and the transmission axis of LP, the four Stokes parameters of the object reflected light are modulated and compressed into a new Stokes parameter representing the total light intensity to be also detected by the CMOS. The four Stokes parameters of the object reflected light are then calculated from the light intensities detected under five polarization modulation. Various stereoscopic objects are selected for database measurement because the polarization information can reflect the surface texture of objects. The FSPMI database is likely to enhance PMI technology development in terms of simulation validation and application popularity.

Methods

Experimental system establishment

To measure FSPMI, we first establish an experimental system as shown in Fig. 1. The light emitted from a light source is initially reflected by an object, then passed through a reflector, lens 1, an iris, lens 2, a QWP, an LP, 18 bandpass filters, lens 3 and finally detected by a CMOS. Both lens 1 and lens 3 are imaging lenses, while lens 2 is a collimating lens. The iris is a field stop to prevent overlapping of images. The entire cage system is established in a dark room and connected by several lens tubes to further block stray light. The manufacturers, items and key parameters of each optical component established in the experimental system are clearly listed in Table 1. Obviously, the operating wavelength range (OWR) of the system is restricted by the 18 bandpass filters, and the optical propagation size is limited by the QWP with the minimum diameter of 12.7 mm. Ignoring the fluctuation of only 2 nm, the central wavelength (CWL) of the 18 bandpass filters ranges from 520 nm to 690 nm at 10 nm intervals, and the full width at half maximum (FWHM) is about 10 nm. During the establishment of the system, the object platform is fixed within the optimal working distance range of 50 mm to 140 mm from the ring illuminator. The positions of the three lenses are then carefully adjusted according to their focal length to obtain the clearest object image on the CMOS. To facilitate experimental data measurement, the QWP is mounted in a motorized precision rotation stage, the LP is mounted in a cage rotation mount, and each of the 6 adjacent bandpass filters is mounted in a 6-position filter wheel.

Fig. 1
figure 1

The established experimental system for measuring full-Stokes polarization multispectral images (FSPMI).

Full size image
Table 1 The manufacturers, items and key parameters of each optical component established in the experimental system.
Full size table

Image measurement

System adjustment

Figure 2 illustrates the specific experimental procedure for measuring FSPMI. Firstly, the light source is turned on and preheated for about 10 min. During this period, the object placed on the fixed platform is moved and rotated slowly in the horizontal plane to optimize the object image in the field of view of CMOS. Thus, the object region of interest is imaged within the specified 512 × 512 pixel region near the CMOS center. Meanwhile, based on the pixel format of Mono8 and the fixed exposure time of 10000 μs on CMOS, the intensity of the light source is adjusted carefully so that the maximum value of the images captured under 18 bandpass filters is approximately 230. The light source adjustment is to ensure that the captured images will not be overexposed throughout the experiment. In other words, the object position and the intensity of the light source are fixed by the above operations.

Fig. 2
figure 2

The specific experimental procedure for measuring full-Stokes polarization multispectral images (FSPMI). In the measurement, the step “Capture images by CMOS” is repeated 6 times.

Full size image

Image measurement

Image capture

Then, the transmission axis of LP is rotated to 45° and fixed thereafter. Select a bandpass filter to perform the following polarization measurements. That is, rotate the fast axis of QWP to 0° to capture 6 images consecutively by CMOS for further averaging, and then rotate the fast axis of QWP to 22.5°, 45°, 67.5° and 90° to similarly capture the images respectively. Notably, the 18 bandpass filters are mounted in three 6-position filter wheels. By carefully replacing the filter wheels manually, the 18 bandpass filters are switched to perform the polarization measurements separately to capture a total of 540 images. Finally, the light source is turned off and 6 images are continuously captured by CMOS to reflect the dark noise of the detector. As described above, the entire process of image acquisition for an object takes about 35 min. Generally, the above steps are repeated several hours later to capture images for the next object. In total, the required images are captured for 67 plastic objects in about 19 days. The third object is 3D-printed and glued, while the remaining 66 objects are purchased refrigerator magnets.

Image preprocessing

System fluctuation and dark noise elimination

The captured images are preprocessed separately for each object to minimize the inherent influence of the experimental system. The image preprocessing is divided into four steps: system fluctuation reduction, dark noise elimination, spectral response calibration, image maximum normalization. In order to reduce the influence of system fluctuation, the 6 images captured continuously under the same bandpass filter and QWP angle are averaged to obtain a total of 90 images for each object. The 6 images captured with the light source off are also averaged to obtain dark noise for each experiment. The averaged dark image is further subtracted from each of the averaged object images to eliminate the influence of dark noise. Moreover, the pixel value less than zero in the object image with dark noise eliminated is assigned as zero to satisfy the rationality of the captured image.

Spectral response acquisition

In addition, the spectral response of the system is determined by the combination of adopted optical devices. Figure 3 successively shows the normalized spectral characteristics of the light source, reflector, lens 1, lens 2, QWP, LP, 18 bandpass filters, lens 3, CMOS detector and the entire system. Based on standard measurements provided by the manufacturer, the spectral characteristics of each optical device are obtained by linear interpolation to 196 wavelengths ranging from 488.41 nm to 730.12 nm and then smoothing. The spectral characteristics of all optical devices are multiplied at each wavelength to obtain the spectral response of the entire system. Meanwhile, the spectral maximum is normalized for each optical device. In particular, the spectral response of each spectral band at all 196 wavelengths is integrated by the trapezoidal method. The 18 bandpass filters and the entire system are then normalized to the maximum integral value across all spectral bands, respectively. The integral value of spectral response after normalization of 18 bandpass filters and the entire system is shown in Fig. 4.

Fig. 3
figure 3

The normalized spectral characteristics of adopted optical devices and the entire system according to the manufacturer specifications.

Full size image
Fig. 4
figure 4

The integral value of spectral response after normalization of 18 bandpass filters and the entire system.

Full size image

Spectral response calibration and image normalization

The spectral response calibration is then performed for the object image that eliminates the system fluctuation and dark noise. Specifically, the object image under each spectral band is divided by the integral value of the spectral response of the entire system under the corresponding spectral band. Finally, image normalization is carried out for each object based on the maximum value of all 90 images. Figure 5 comprehensively shows the images captured and preprocessed for an object at 18 spectral bands and five QWP angles. The captured images are displayed well in the grayscale range of 0 to 255, while the preprocessed images are displayed well in the grayscale range of 0 to 1. Obviously, image preprocessing can significantly enhance image brightness at short spectral bands to further compensate for the brightness difference caused by the uneven spectral response of the system. Moreover, image preprocessing can be further improved by experimentally measuring the spectral response of the system instead of the standard measurements provided by the manufacturer.

Fig. 5
figure 5

The images captured and preprocessed for an object at 18 spectral bands and five QWP angles.

Full size image

Full-Stokes polarization modeling

Polarization modulation theory

Full-Stokes images are calculated from the preprocessed images according to the designed polarization modulation strategy. In the experimental system, QWP and LP jointly modulate the polarization information of the light reflected by the object. Let S0, S1, S2, S3 represent the four Stokes parameters of the object light. Assuming that the fast axis of QWP is rotated to an angle θ(0° ≤ θ<180°), then the Mueller matrix of QWP can be expressed as follows:

$${{\bf{M}}}_{1}=\left[\begin{array}{cccc}1 & 0 & 0 & 0\\ 0 & {{\rm{\cos }}}^{2}\left(2\theta \right) & {\rm{\cos }}\left(2\theta \right){\rm{\sin }}\left(2\theta \right) & -{\rm{\sin }}\left(2\theta \right)\\ 0 & {\rm{\cos }}\left(2\theta \right){\rm{\sin }}\left(2\theta \right) & {{\rm{\sin }}}^{2}\left(2\theta \right) & {\rm{\cos }}\left(2\theta \right)\\ 0 & {\rm{\sin }}\left(2\theta \right) & -{\rm{\cos }}\left(2\theta \right) & 0\end{array}\right].$$
(1)

Thus, the four Stokes parameters modulated by QWP, denoted as \({S}_{0}^{1}\), \({S}_{1}^{1}\), \({S}_{2}^{1}\), \({S}_{3}^{1}\), can be calculated from

$$\left[\begin{array}{c}{S}_{0}^{1}\\ {S}_{1}^{1}\\ {S}_{2}^{1}\\ {S}_{3}^{1}\end{array}\right]={{\bf{M}}}_{1}\times \left[\begin{array}{c}{S}_{0}\\ {S}_{1}\\ {S}_{2}\\ {S}_{3}\end{array}\right]=\left[\begin{array}{cccc}1 & 0 & 0 & 0\\ 0 & {{\rm{\cos }}}^{2}\left(2\theta \right) & {\rm{\cos }}\left(2\theta \right){\rm{\sin }}\left(2\theta \right) & -{\rm{\sin }}\left(2\theta \right)\\ 0 & {\rm{\cos }}\left(2\theta \right){\rm{\sin }}\left(2\theta \right) & {{\rm{\sin }}}^{2}\left(2\theta \right) & {\rm{\cos }}\left(2\theta \right)\\ 0 & {\rm{\sin }}\left(2\theta \right) & -{\rm{\cos }}\left(2\theta \right) & 0\end{array}\right]\times \left[\begin{array}{c}{S}_{0}\\ {S}_{1}\\ {S}_{2}\\ {S}_{3}\end{array}\right].$$
(2)

When the transmission axis of LP is rotated to an angle α(0° ≤ α<180°), the Mueller matrix of LP is

$${{\bf{M}}}_{2}=\frac{1}{2}\left[\begin{array}{cccc}1 & {\rm{\cos }}\left(2\alpha \right) & {\rm{\sin }}\left(2\alpha \right) & 0\\ {\rm{\cos }}\left(2\alpha \right) & {{\rm{\cos }}}^{2}\left(2\alpha \right) & {\rm{\cos }}\left(2\alpha \right){\rm{\sin }}\left(2\alpha \right) & 0\\ {\rm{\sin }}\left(2\alpha \right) & {\rm{\cos }}\left(2\alpha \right){\rm{\sin }}\left(2\alpha \right) & {{\rm{\sin }}}^{2}\left(2\alpha \right) & 0\\ 0 & 0 & 0 & 0\end{array}\right].$$
(3)

The four Stokes parameters further modulated by LP are denoted as \({S}_{0}^{2}\), \({S}_{1}^{2}\), \({S}_{2}^{2}\), \({S}_{3}^{2}\), which can be similarly calculated by

$$\begin{array}{c}[\begin{array}{c}{S}_{0}^{2}\\ {S}_{1}^{2}\\ {S}_{2}^{2}\\ {S}_{3}^{2}\end{array}]={{\bf{M}}}_{2}\times [\begin{array}{c}{S}_{0}^{1}\\ {S}_{1}^{1}\\ {S}_{2}^{1}\\ {S}_{3}^{1}\end{array}]={{\bf{M}}}_{2}\times {{\bf{M}}}_{1}\times [\begin{array}{c}{S}_{0}\\ {S}_{1}\\ {S}_{2}\\ {S}_{3}\end{array}]\\ =\frac{1}{2}[\begin{array}{cccc}1 & \cos (2\alpha ) & \sin (2\alpha ) & 0\\ \cos (2\alpha ) & {\cos }^{2}(2\alpha ) & \cos (2\alpha )\sin (2\alpha ) & 0\\ \sin (2\alpha ) & \cos (2\alpha )\sin (2\alpha ) & {\sin }^{2}(2\alpha ) & 0\\ 0 & 0 & 0 & 0\end{array}]\times [\begin{array}{cccc}1 & 0 & 0 & 0\\ 0 & {\cos }^{2}(2\theta ) & \cos (2\theta )\sin (2\theta ) & -\sin (2\theta )\\ 0 & \cos (2\theta )\sin (2\theta ) & {\sin }^{2}(2\theta ) & \cos (2\theta )\\ 0 & \sin (2\theta ) & -\cos (2\theta ) & 0\end{array}]\times [\begin{array}{c}{S}_{0}\\ {S}_{1}\\ {S}_{2}\\ {S}_{3}\end{array}]\\ =\frac{1}{2}[\begin{array}{cccc}1 & \cos (2\alpha ){\cos }^{2}(2\theta )+\sin (2\alpha )\cos (2\theta )\sin (2\theta ) & \cos (2\alpha )\cos (2\theta )\sin (2\theta )+\sin (2\alpha ){\sin }^{2}(2\theta ) & -\cos (2\alpha )\sin (2\theta )+\sin (2\alpha )\cos (2\theta )\\ \cos (2\alpha ) & {\cos }^{2}(2\alpha ){\cos }^{2}(2\theta )+\cos (2\alpha )\sin (2\alpha )\cos (2\theta )\sin (2\theta ) & {\cos }^{2}(2\alpha )\cos (2\theta )\sin (2\theta )+\cos (2\alpha )\sin (2\alpha ){\sin }^{2}(2\theta ) & -{\cos }^{2}(2\alpha )\sin (2\theta )+\cos (2\alpha )\sin (2\alpha )\cos (2\theta )\\ \sin (2\alpha ) & \cos (2\alpha )\sin (2\alpha ){\cos }^{2}(2\theta )+{\sin }^{2}(2\alpha )\cos (2\theta )\sin (2\theta ) & \cos (2\alpha )\sin (2\alpha )\cos (2\theta )\sin (2\theta )+{\sin }^{2}(2\alpha ){\sin }^{2}(2\theta ) & -\cos (2\alpha )\sin (2\alpha )\sin (2\theta )+{\sin }^{2}(2\alpha )\cos (2\theta )\\ 0 & 0 & 0 & 0\end{array}]\times [\begin{array}{c}{S}_{0}\\ {S}_{1}\\ {S}_{2}\\ {S}_{3}\end{array}].\end{array}$$
(4)

Full-Stokes derivation

During image capture, CMOS only detects the total light intensity, which is represented by the modulated first Stokes parameter \({S}_{0}^{2}\). Let Iθ,α represent the light intensity information detected by CMOS when the fast axis of QWP and the transmission axis of LP are rotated to angles θ and α respectively. Therefore, the relationship between the detected information Iθ,α and the four Stokes parameters S0, S1, S2, S3 of the object light can be written as

$${I}_{\theta ,\alpha }={S}_{0}^{2}=\frac{1}{2}\left\{\begin{array}{lcl}{S}_{0} & + & [{\rm{\cos }}(2\alpha ){{\rm{\cos }}}^{2}(2\theta )+{\rm{\sin }}(2\alpha ){\rm{\cos }}(2\theta ){\rm{\sin }}(2\theta )]\\ {S}_{1} & + & [{\rm{\cos }}(2\alpha ){\rm{\cos }}(2\theta ){\rm{\sin }}(2\theta )+{\rm{\sin }}(2\alpha ){{\rm{\sin }}}^{2}(2\theta )]\\ {S}_{2} & + & [{\rm{\sin }}(2\alpha ){\rm{\cos }}(2\theta )-{\rm{\cos }}(2\alpha ){\rm{\sin }}(2\theta )]{S}_{3}\end{array}\right\}.$$
(5)

As described in the image capture, the transmission axis of LP is fixed at 45°, and the fast axis of QWP is rotated successively to 0°, 22.5°, 45°, 67.5° and 90°. By substituting the designed QWP and LP angles into Eq. (5), the intensity information detected by CMOS can be expressed as

$$\left.\begin{array}{l}{I}_{{0}^{\circ },4{5}^{\circ }}={S}_{0}/2+{S}_{3}/2\\ {I}_{22.{5}^{\circ },4{5}^{\circ }}={S}_{0}/2+{S}_{1}/4+{S}_{2}/4+\sqrt{2}{S}_{3}/4\\ {I}_{4{5}^{\circ },4{5}^{\circ }}={S}_{0}/2+{S}_{2}/2\\ {I}_{67.{5}^{\circ },4{5}^{\circ }}={S}_{0}/2-{S}_{1}/4+{S}_{2}/4-\sqrt{2}{S}_{3}/4\\ {I}_{9{0}^{\circ },4{5}^{\circ }}={S}_{0}/2-{S}_{3}/2\end{array}\right\}.$$
(6)

Then, the four Stokes parameters of the object reflected light can be solved through

$$\left.\begin{array}{l}{S}_{0}={I}_{{0}^{\circ },4{5}^{\circ }}+{I}_{9{0}^{\circ },4{5}^{\circ }}\\ {S}_{1}=2\left({I}_{22.{5}^{\circ },4{5}^{\circ }}-{I}_{67.{5}^{\circ },4{5}^{\circ }}\right)-\sqrt{2}\left({I}_{{0}^{\circ },4{5}^{\circ }}-{I}_{9{0}^{\circ },4{5}^{\circ }}\right)\\ {S}_{2}=2{I}_{4{5}^{\circ },4{5}^{\circ }}-\left({I}_{{0}^{\circ },4{5}^{\circ }}+{I}_{9{0}^{\circ },4{5}^{\circ }}\right)\\ {S}_{3}={I}_{{0}^{\circ },4{5}^{\circ }}-{I}_{9{0}^{\circ },4{5}^{\circ }}\end{array}\right\}.$$
(7)

Herein, we further calculate five other well-known polarization parameters, including the angle of linear polarization (AoLP), the angle of circular polarization (AoCP), the degree of linear polarization (DoLP), the degree of circular polarization (DoCP) and the degree of polarization (DoP). The five polarization parameters are calculated as follows:

$$\left.\begin{array}{l}AoLP={\rm{\arctan }}\left({S}_{2}/{S}_{1}\right)/2\\ AoCP={\rm{\arcsin }}\left({S}_{3}/{S}_{0}\right)/2\\ DoLP=\sqrt{{S}_{1}^{2}+{S}_{2}^{2}}/{S}_{0}\\ DoCP=\left|{S}_{3}\right|/{S}_{0}\\ DoP=\sqrt{{S}_{1}^{2}+{S}_{2}^{2}+{S}_{3}^{2}}/{S}_{0}\end{array}\right\}.$$
(8)

Figure 6 shows the images of four Stokes parameters S0, S1, S2, S3 and five polarization parameters AoLP, AoCP, DoLP, DoCP, DoP calculated for an object at 18 spectral bands. Combining the theoretical values of parameters with the actual values of images, the S0 images are displayed in the grayscale range of 0 to 2. The S1, S2 and S3 images are displayed in the grayscale range of −0.34 to 0.34. The AoLP and AoCP images are displayed in the grayscale range of −π/4 to π/4. The DoLP, DoCP and DoP images are displayed in the grayscale range of 0 to 1. It is found that S0 can reflect the overall appearance of the object, and other parameters can reflect the surface texture of the object. Selecting the spectral band of 600 nm, the S0 images of all 67 objects are shown in Fig. 7 to reflect their different appearance.

Fig. 6
figure 6

The images of four Stokes and five polarization parameters calculated for an object at 18 spectral bands.

Full size image
Fig. 7
figure 7

The S0 images of all 67 objects with different appearance at the spectral band of 600 nm.

Full size image

Data Records

The raw experimental images of all 67 objects are packaged into “FSPMI experimental data.zip” and deposited to the figshare28. The folder labeled as “Object_xx” contains images captured for each object. For example, the folder named “Object_01” contains images captured for the first object. The subfolders and image files are described in Table 2. A total of 36582 images are captured for 67 objects, accounting for 571 GB.

Table 2 The raw experimental images in each folder labeled as “Object_xx”.
Full size table

The processed image data of all 67 objects is packaged into “FSPMI dataset.zip” and also deposited to the figshare28. The image data for each object is stored separately in a file labeled as “Object_xx.mat”. For example, the file labeled as “Object_01.mat” stores image data for the first object. Table 3 lists the variable names, sizes and corresponding descriptions of the data in each object file. The data in each file includes a set of spectral bands, five sets of preprocessed images, four sets of Stokes images, two sets of polarization angle images and three sets of polarization degree images.

Table 3 The processed image data in each file labeled as “Object_xx.mat”.
Full size table

Technical Validation

The FSPMI dataset is intercepted by 400×400 spatial pixels and further applied to validate several existing compressive full-Stokes PMI methods17,18,19 to demonstrate data reliability. These compressive full-Stokes PMI methods differ in terms of experimental systems and reconstruction methods. To modulate and compress the four Stokes parameters, the compressive full-Stokes polarization and flexible multispectral four-dimensional imaging (CFPMI) method17 employs a QWP, an LP and an LCTF. Furthermore, the CFPMI method accurately solves S0 and then reconstructs S1, S2, S3 based on CS theory14. The four-dimensional compressed spectropolarimetric imaging (FDCSPI) method18 adopts a QWP and an LCTF to modulate and compress the four Stokes parameters. Moreover, the FDCSPI method firstly reconstructs S1, S2, S3 based on CS theory and then solves S0. The full polarization-compressed multispectral imaging (FPCMI) method19 also modulates and compresses the four Stokes parameters by a QWP and an LCTF. Nevertheless, the FPCMI method first reconstructs S0 and then S1, S2, S3 based on feature scaling and CS theory.

The above methods are validated for all 67 objects in the FSPMI dataset. For each object data, three polarization modulation is performed in the CFPMI, FDCSPI and FPCMI methods. In the CFPMI method, the transmission axis of LP is set to 45°, 135° and 135° respectively, while the fast axis of QWP is randomly set to the same angle in the first two modulation and is changed in the third modulation. In the FDCSPI and FPCMI methods, three angles are randomly set for the fast axis of QWP. In a word, the fast axis angle of QWP is randomly set for both the three methods and 67 object data. Similarly, the transmission axis of linear polarization of the LCTF incidence plane is set to 0° in the above three methods. Meanwhile, all spectral bands of interest are provided for each object by freely switching the central wavelength of the LCTF. In addition, all the reconstruction based on CS theory use the discrete W transform basis29 and the two-step iterative shrinkage/thresholding (TwIST) algorithm30 with iteration accuracy of 0.005.

Figure 8 shows the reconstruction results of the four Stokes parameters for all 67 objects in validating the CFPMI, FDCSPI and FPCMI methods. The reconstruction results are reflected by the peak signal to noise ratio (PSNR) and structural similarity (SSIM) values averaged over 18 spectral bands. For better comparison, the PSNR and SSIM values of each Stokes parameter are averaged over 67 objects. The averaged PSNR and SSIM values of each Stokes parameter for the CFPMI, FDCSPI and FPCMI methods are shown in Fig. 9. The averaged PSNR values are all greater than 20 dB, and the averaged SSIM values are all greater than 0.6. Therefore, the reliability of FSPMI dataset is effectively validated in the CFPMI, FDCSPI and FPCMI methods.

Fig. 8
figure 8

The averaged PSNR and SSIM values of the four Stokes parameters for all 67 objects in validating the CFPMI, FDCSPI and FPCMI methods.

Full size image
Fig. 9
figure 9

The averaged PSNR and SSIM values of each Stokes parameter for the CFPMI, FDCSPI and FPCMI methods.

Full size image

To improve the reconstruction results, the polarization modulation strategy can be further designed based on CS theory. In addition, the diversity of the 67 objects provides flexibility for studying compressive full-Stokes PMI methods in various scenarios. Meanwhile, the richness of the 67 objects contributes to the development of machine learning-based reconstruction methods in compressive full-Stokes PMI.

Usage Notes

The FSPMI dataset can serve as source data for both compressive polarization imaging and compressive multispectral imaging. This dataset significantly settles the problem of inconsistency and time-consuming in data acquisition, including simulation data and experimental data. Simulation and experiment are usually combined to investigate the effectiveness of imaging methods in terms of encoding strategies and reconstruction algorithms. In general, the encoding method is different in simulation and experiment. The numerical calculation performs the simulation encoding, while the imaging system performs the experimental encoding.

Therefore, in polarization imaging, the captured polarization encoded images and the derived full-Stokes parameter images can be involved in different requirements. Less captured polarization encoded images can be used as experimental data to reconstruct full-Stokes parameter images. The derived full-Stokes parameter images can be used as simulation data to numerically calculate less polarization encoded images, and then to reconstruct full-Stokes parameter images. Furthermore, the reconstructed and derived full-Stokes parameter images are compared from various aspects to evaluate the imaging method. Meanwhile, the polarization angle images and polarization degree images can provide additional references for the evaluation.

Multispectral imaging mainly involves captured and derived light intensity images at 18 spectral bands. The central wavelength of 18 spectral bands ranges from 520 nm to 690 nm with 10 nm intervals and 10 nm bandwidths. Thus, the images in each spectral band are regarded as ideal narrowband images. Moreover, multispectral images can be used as simulation data to numerically calculate less spectral encoded images, and then to reconstruct multispectral images. Similarly, the reconstructed and unencoded multispectral images are compared sufficiently to evaluate the imaging method. Notably, image registration may be required to eliminate the slight drift of object images in different spectral bands caused by the position error of multiple filters.

In addition, the diversity of 67 objects ensures that the FSPMI dataset is suitable for developing machine learning-based imaging methods. The 67 objects are plastic cartoon images, including persons, animals, food, vehicles and other things. According to the object’s structural complexity, researchers can test and select appropriate objects to meet the requirements of the imaging method.