Brightness perception under photopic conditions: experiments and modeling with contributions of S-cone and ipRGC

In 1924, the CIE published and standardized the photopic luminous efficiency function. Based on the standardized curve, luminous flux in lumens, luminance in cd/m\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^2$$\end{document}2, and illuminance in lux are determined by an integral of the curve and the incident light spectra in photometers and are considered physical brightness. However, human brightness perception is not only weighted by this simple determination, but is a more complicated combination of all L-cones, M-cones, S-cones, rods and later ipRGCs, which was partly described by the equivalent brightness of Fotios et al. with the correction factor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(S/V)^{0.24}$$\end{document}(S/V)0.24. Recently, new research has demonstrated the role of ipRGCs in human light perception. However, it is still unclear how these signal components of the human visual system are involved in the overall human brightness perception. In this work, human brightness perception under photopic conditions was investigated by visual experiments with 28 subjects under 25 different light spectra. In this way, the contributions of the signal components can be investigated. An optimization process was then performed on the resulting database. The results show that not only the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L+M$$\end{document}L+M component, but also the S-cones and ipRGC play a role, although it is smaller. Thus, the visually scaled brightness model based on the database optimization was constructed using not only illuminance but also S-cones and ipRGC with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R^2$$\end{document}R2 of 0.9554 and RMSE of 4.7802. These results are much better than the brightness model after Fotios et al. using only S-cones (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R^2$$\end{document}R2 = 0.8161, RMSE = 9.7123) and the traditional model without S-cones and ipRGC (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R^2$$\end{document}R2 = 0.8121, RMSE = 9.8171).

The development of artificial light started at the beginning of the 20th century in the era of industrialization and electrification and was intended to provide indoor workplaces with light around the clock, thereby minimizing accidents at work.The research focus was initially based on the criteria of contrast perception, visual acuity, object size and adaptive luminance.From the 1960s, and then increasingly in the 1990s, the psychological component of lighting was intensively investigated with the aspects of homogeneity, luminance distribution, color rendering, color temperature, light direction with indirect and direct light components in the room [1][2][3][4][5] .For this integration of psychological components into lighting research, the chromatic aspects of the radiation components reaching the eye, e.g., the color fidelity, color discrimination, color difference 6,7 , were thus also considered.
Since the beginning of the 21st century, there have been three new focal points in lighting technology.The first focus-following the discovery of intrinsic photosensitive retina ganglion cells (ipRGCs) [8][9][10][11][12][13][14] -is the detection and quantification of the effects of optical radiation of different wavelengths and radiation components on sleep quality, work productivity, alertness, and well-being.The second focus is the research to describe the quality of the illumination of an object and of a scene in the room, substantially driven by the development of the LEDtechnology.Besides the color rendering index, further parameters of other properties such as color gamut, color memory and color saturation shall be added for the assessment of color quality [15][16][17][18][19][20][21] .
The investigations into the general brightness and, in special cases, spatial brightness as an important aspect of room illumination and scene perception, which form the subject of the present paper as the third focus, have been carried out experimentally and theoretically in vision science since the late 1960s [22][23][24][25][26][27] and continued until today [28][29][30][31][32] with an overall evaluation of the experimental methodology and the knowledge gained up to that time.There is a definition for spatial brightness formulated by IESNA (Illuminating Engineering Society of North America) and reproduced in 28 .Generally, spatial brightness describes the visual perception which is evoked by the incident light coming from a large part of the visual field being beyond the fovea area.This should be the case if the light user observes several objects in a room or on a street with different brightness levels.
For an accurate description and modeling of the characteristics of brightness, however, the following research questions are important for vision science: • Is the parameter illuminance or luminance alone decisive for the perceived brightness of neutral (color tone- free) and colored objects?
If the answer to the above question is "No", then: • How can new numerical parameters for the mathematical modelling of perceived brightness be determined?
These questions are of high interest as numerous publications-when describing the test conditions and interpreting the test results-predominantly used only V ( )-weighted parameters, e.g., the illuminance at the object level and the luminance of objects and of the surrounding walls of a room.Important for today's vision science and lighting research for indoor applications, on the other hand, is also the knowledge of whether and to what extent-apart from V ( )-there are other signals or signal combinations of the various photoreceptors of the retina, i.e. the cones (L, M, S), the rods and the intrinsically photosensitive ganglion cells (ipRGCs) that should contribute to the perception of brightness [25][26][27][28] and thus to the perception of the overall atmosphere of an illuminated room and also, which wavelength ranges constitute the main part of electromagnetic radiation for perceived brightness.From many visual experiences, it is known that white light with a higher blue light content, at a certain constant luminance or illuminance level, could evoke a higher brightness perception compared to warm white light 30,33 .The reason for this appearance is that not only the V ( )-weighted signals (L + M), but also the signals of the other retinal mechanisms contribute to brightness perception.The rods responsible for night vision are irrelevant for indoor illumination situations at the daytime with a luminance at a higher level (higher than 10 cd/m 2 ) 27 .From the human eye physiological aspects, it is known, that electrical impulses are formed in the photoreceptors-after photon absorption-and transmitted to the ganglion cell layers (see Fig. 1).The parasol midget and small bistratified ganglion cells accounting for about 75% of all RGCs are responsible for transmitting frequency-coded action potentials to retinorecipient brain nuclei 34 .The ganglion cells' axons project to the lateral geniculate nucleus (LGN) of the thalamus, which undertakes the gateway to the brain's primary visual cortex V 1 in which visual sensory information is processed further (see Fig. 1).
The LGN consists of six different layers with the magnocellular (M) cells which are implemented in the lowest two pieces and the upper four layers contain the parvocellular (P) cells 36 .Between the M-and P-cells of the LGN, six additional layers are found containing the koniocellular (K) cells.The primary visual cortex received the encoded colour and brightness properties of visual stimuli from the parvocellular (PC), magnocellular (MC) and koniocellular (KC) pathways 37 (see Fig. 1).The three different ganglion cell types and the corresponding cell layers in the LGN are responsible for the generation and processing of different visual information.
• The P-and K-cells in the LGN show opposing color responses.Midget ganglion cells provide cone opponent color information to the PC-pathway, derived from a subtraction of L-and M-cones (L-M, red-green opponency) 38 .• Bi-stratified ganglion cells project the middle KC-layers and provide excitatory signals from S-cones with opposed inhibitory information from an additive L-and M-cones signal combination (S − (L + M) , blue- yellow opponency) 36 .In the last five decades, research activities were performed concerning brightness experiments with colored and conventional white light and modeling 27,[40][41][42] , which leaded to a summarizing paper of the CIE (International Commission on Illumination) in 43 .All the models included in this fundamental paper considered the contributions of the opponent channels (L-M) and (S − (L + M) ) indirectly by implementing the chromaticity x and y into a function with the luminance from the achromatic signal (L + M) .In a doctoral thesis on the photopic brightness in indoor lighting, Pepler 33 varied the polychromatic white light source spectra and the luminance on a homogeneous and diffusely reflecting wall in a real room without daylight incidence and found, that under the defined test conditions with white lights, the most consistent model corresponding to the subjective evaluations of the test persons is a simple model of Fotios et al. from 1998 26 in which the so-called equivalent luminance ( L eq ) can be defined according to Eq. (1).
In Eq. ( 1), the exponent of the V ( )-weighted luminance ( L v ) equals 1.This means that the photopic luminance does not experience signal compression.To calculate the signals S or V, the relative spectral radiant flux of the light source must be multiplied by the spectral sensitivity function of the S-cones or by the V ( ) function, respectively, and this product must be integrated in the visible wavelength range.It should be emphasized that, in accordance with Eq. ( 1)-S-cones (see Fig. 2) should contribute to the formation of the signals for the attribute luminance (at least in the photopic region, in which the rod signals are not available, see Figs. 2 and 3).With the discovery of the new type of ganglion cells, the ipRGCs, some research studies have been performed aiming to answer the question if also the ipRGC -signals could contribute to the brightness perception in the photopic vision range.According to the latest studies on the field of neurophysiology, there are some reasons to assume that ipRGCs interact in at least two different ways with the visual channels 44 .In one way, the so-called M 4 -sub- type ipRGCs project to the LGN and might contribute to the human's brightness perception 45 .In another way, a group of M 1 -subtype ipRGCs builds signal connections with upstream dopaminergic amacrine cells and possibly influencing the light adaptation state 46 .In this context, ipRGCs can influence other RGC types and the communication between cones and related bipolar cells.The studies of Zele et al. 44 in 2018 and Yamakawa et al. 47 in 2019 had found several ipRGC -signals in the brightness perception.These findings in the laboratory conditions should be validated due to their small sample size and number of visual stimuli.
In the present work, the attribute brightness is modelled with the S-cone signals and the ipRGC signals based on the results from an empirical study, which was comprehensively conducted in a real room using a higher number of test persons and visual stimuli (luminance of the objects and spectra of the light sources).This work's key outcome is an empirical-based brightness perception model that includes the S-cone and the ipRGC contribution, with which the human's brightness sensation can be predicted more accurately compared to the V( )-based counterparts like the luminance.

Experimental method of the subjective study
The conducted study took place in a real, office-like room without daylight, with homogeneous white and matt painted walls and neutral flooring.The subjects sat on chairs and looked at a homogeneously illuminated table with a white tablecloth (see Fig. 4).The subjects first adapted to the white tablecloth for about 2 min according to the research result of Fairchild et al. 48on the time course of the chromatic adaptation during colour-appearance judgment.The subjects scaled their impression of the brightness of a scene with various 3-dimensional coloured (1) L eq = L v (S/V ) 0.24 www.nature.com/scientificreports/and achromatic objects (doll with long hair, jumper with colourful patterns, artificial water lily, black and white test pattern with fine line structure), which were illuminated in this room with two RGBW LED lights with light diffusing covers causing a uniform and diffuse illumination on the objects.The horizontal illuminance on the table was 45 lx, 90 lx, 470 lx, 1000 lx and 2000 lx respectively, so that the vision condition is photopic.For the condition of 45 lx, the luminance on the table has been 13.1 cd/m 2 .The correlated colour temperature (CCT )-at each illuminance level-was also varied: 2700 K, 3100 K, 4100 K, 5000 K and 10,000 K and covered a range of colour temperatures in private and professional rooms in the evening and daytime.Thus, the subjects were shown 25 different ( E v x CCT ) combinations, waiting about 1.5 min for re-adaptation after adjusting each light combination until the subjective evaluation was made.The illuminating RGBW LED light sources were optimised to ensure high colour rendering index levels (89 ≤ IES TM30-20 R f ≤ 93) for each spectrum (5 × 5 = 25).The illumination spectra are shown in Fig. 5. Table 1 shows the colorimetric and photometric parameters of the 25 spectra which have been presented to the test persons in a randomized order.Table 2 lists the α-opic illuminances (L-cone-opic, M-cone-opic, S-cone-opic, rod-opic and melanopic equivalent daylight D65 illuminance values) according to CIE S 026/E: 2018) 14, 49 .Twenty-eight subjects with normal or corrected visual acuity and without colour vision deficiencies were recruited from a pool of students at the University.After they arrived in the lighting laboratory, they were tested with "The Standard Pseudoisochromatic    50 .This study was approved by the ethics committee of the Technical University of Darmstadt and carried out following the ethical principles of the Declaration of Helsinki.All subjects were informed about the content of the study.Signed consent was obtained from the participants before the experiment took place.Before the main experiment, the test leader explained the procedure of the experiment with questionnaires in about 10 min in which the test person adapted to the lighting condition in the test room.Subsequently, a training was conducted with different light settings and the same questionnaire which contains a continuous brightness scale between 0 and 100 (see Fig. 6).In this training phase of about 10 min, 2 extreme light settings such as very bright, very dark and 3 settings in between were also presented to create so-called anchor stimuli.The training results were not included in the final evaluation of the results.
For a better understanding of the brightness test, it is beneficial to describe the instructions given by the experimenter to the subjects (see also Fig. 6): Brightness: "The term brightness is used in an everyday sense in that you evaluate the whole table with the tablecloth-compared to the reference situation-according to brightness.How bright does the table appear compared to the reference situation?The reference situation is shown several times in the training phase and corresponds to a brightness value of 100.Complete darkness corresponds to a value of 0. Please evaluate your impression of brightness (H) by looking at the table, i.e., the white tablecloth.In the training phase, you can memorise your brightness impression of the reference scene with H=100, which is shown repeatedly.You can tick the scale-according to your brightness impression.After each adjustment of a new light source, please look at the table for 90s first.After this adaptation, you can then evaluate the brightness.Please do not look at your own hand, nor at the faces of the other persons, only at the table, i.e., the white tablecloth.Please first decide in which third the current situation is (top, middle or bottom) and then tick the scale-corresponding to your impression of brightness-within this third".The referent situation is the light SPD14 in Tables 1 and 2.
In the main experiment, the 25 spectra mentioned above were presented randomly.Between each setting in the main experiment, 1.5 min for perception and one additional minute for evaluation were planned.The total experimental duration was therefore about 80-85 min long.The test persons had been paid for their test efforts according to the regulation of the University.

Modelling brightness
The subjects' mean visual scale scores were modeled for brightness and using the quantity M, defined in Eq. ( 2). with S( ) , V ( ) , ipRGC( ) are shown in Fig. 3. E ,abs.( ) is the absolute spectral irradiance distribution on the object plane in W/(nm m 2 ) and E ,rel.( ) is the relative spectral distribution derived from E ,abs.( ).
The symbols in Eq. ( 2) have the following meaning.
• The parameter γ is an exponent for the compression of the illuminance and the parameter δ is an exponent for the compression of the S and the G signals (G denotes in this paper the ipRGC signal).• The parameters a and b are the parameters of a linear transformation to fit the calculated M i values to the mean visual scale values VSB i (for brightness) of the test subjects for the 25 light source spectra (i = 1-25, see Fig. 5) with the smallest error RMSE.The error size root mean square error (RMSE) is defined accordingly in Eq. ( 7).
The parameters α and β of Eq. ( 2) are weighting parameters of the relative signals of the mechanisms S (S-cones) and G(ipRGCs: intrinsically photosensitive retinal ganglion cells).The relative signals were calculated from the 25 relative spectra.To calculate these signals (S and G), the relative spectral radiant flux of the light source must be multiplied by the spectral sensitivity function of the S-cones or the ipRGCs (published in the CIE-publication 14 and this product shall be integrated in the visible wavelength range.Afterwards, these signals are divided by the so-called V-signal.To calculate the V-signal, the relative spectral radiant flux of the light source shall be multiplied by the V ( )-function and this product shall be integrated in the visible wavelength range.
(2) www.nature.com/scientificreports/ The calculation and optimisation of the model parameters according to Eq. ( 2) has several steps (see Table 3): Step 1: Based on Eq. ( 1) of Fotios, the model parameters are suggested as α = 1 , β = 0 , γ = 1 & δ 1 = δ 2 = δ = 0.24 when the brightness weighted by V ( ) is not considered good and should be replaced by an equivalent luminance.The quality of this model without ipRGC (according to Fotios et al. 26 listed in row 1 * of Table 3) as well as the model without contributions of ipRGC-and S-channels (listed in row 18 * * ) has a quality with RMSE of about 9.7-9.8 and with the correlation coefficient R 2 of about 0.81-0.82.
Step 2: In rows 2-11, the process of modeling starts by varying α and β in the 1 range so that a and b follow for a optimal result.Sometimes γ and δ 1 are also changed in the optimization trials.The result is a correlation coefficient R 2 in the range of about 0.94 and RMSE of about 5.5-5.84.The best case in this series occurs in the row 6 with R 2 of 0.9455 and RMSE of 5.287.
The values of the row 6 differ from those of the row 11 if the pair ( α ; β ) is varied from (0.5; 0.5) to (0.92; 0.92) by keeping the values of γ and δ constant.With the new values of the parameters a and b in the row 11, the optimization results of the row 11 are equal those of the row 6.
Step 3: In the next step, from row 12 to row 16, the values of the parameters ( α ; β ) and γ are kept constant and the parameter γ together with the value pair (a, b) are optimized so that the RMSE is minimized and the R 2 is maximized.The best results are found at a value of δ being about 0.0085.
Step 4 (final step): A global optimization is performed with varying all parameters ( α , β , γ , δ 1 for S, δ 2 for G) and correlated parameters (a, b).The result in row 17 shows that it is the best case with R 2 of 0.9554 and RMSE below 5 (4.7802).The results give a lot of valuable information: • Weighting factors α and β are not equal for S-and ipRGC-signal.
• The exponent δ 2 for compressing the output of ipRGC is much lower than that of δ 1 of the S-cone-signal.
• The exponents δ 1 and δ 2 are much lower than the compression exponent γ of the E v -signal.
• The exponent γ of E v is not 1.0 (compression) and lower at 0.2629.Table 3. Parameter values of Eq. ( 2) and the corresponding values of the error (RMSE) of Eq. ( 7) for brightness.Weight of S-cones: α ; weight of ipRGC: β ; exponent for illuminance: γ ; exponent for S and G signals: δ 1 and δ 2 , respectively; a, b: parameters of the linear transformation to approximate the mean visual scale values of the subjective study; *Optimization in row no. 1 * : these parameter values represent Eq. ( 1) according to Fotios and Levermore 26 , where α, β, γ and δ were constant and only a, b were optimized; **Optimization in row no.18 * * : it is in the traditional form without S and G=ipRGC, where α was 1 and only a, b were optimized; Source: Technical University of Darmstadt. No.

Model parameters-synthesis
Model quality parameter Model with S after Fotios:  Table 3 shows that the smallest error ( RMSE = 4.7802 ) results in the row no.17, where the weighting of the S-signal is equal to 1.0 and the weighting of the ipRGC -(G-) signal is equal to 0.5.If the parameters ( α , β ) are being optimized, then the same result is obtained: i.e., there is a global optimum at α = 1.0 , β = 0.5 with RMSE = 4.78 , R 2 = 0.955 , a Fit-Quality = 1 and b Fit-Quality = 1 • 10 −7 .For other fixed ( α , β ) parameter values (other rows in Table 3) there are only slightly larger errors (RMSE higher 5) and only slightly different optimum ( γ , δ ) values.This analysis indicates that, based on the present experimental data set with 28 test persons and with 25 different absolute light source spectra, it is not possible to decide which signal (S or G = ipRGC or both) is decisive for brightness perception.For modeling perceived brightness, the row no.17 of Table 3 with the smallest error RMSE is proposed here in the Eq. 8 which is scaled so that the model value M reflects the same scale used by the subjects.
Figure 7 visualizes the mean, visually scaled brightness values of all observers from the experiment in Section "Experimental method of the subjective study" and their 95% confidence intervals as a function of the parameter M(brightness) of Eq. ( 8), with R 2 = 0.9554 and RMSE = 4.7802.
As mentioned above, the parameters α , β , γ and δ in row no. 1 * of Table 3 remained constant and only the parameters (a, b) were optimized, which linearly fit the scale of the Fotios-Levermore model 26 to the brightness scale of the present experiment.In this case, a significantly larger error results, RMSE = 9.71 (see Fig. 8).The possible reason for this is that the Fotios-Levermore model does not compress the illuminance scale ( E v ), i.e., works with exponent γ = 1.0.
From the comparison of Figs. 7 and 8 it is evident how important it is, on the one hand, to compress the scale of the illuminance (or luminance) with a suitable exponent and, on the other hand, to refine the model with the implementation of "blue-sensitive" signals that represent both components, S-cones and ipRGCs.

Discussion and summary
If the signals of the two blue-light-containing channels S-cone and ipRGC (or one of them, e.g., the S-signal) are considered as in Eq. ( 8), we obtain a better prediction of the mean visually scaled results of lightness perception.According to the results of the present study, either the ipRGC signal (here also called the G signal) or the S signal or both signals play an important role.The question of whether the S-or the G-signal is critical could not be answered from the results of the present study.
In any case, it is very important to include a "blue-sensitive" signal in the model.An implementation of a compression of the scales of the input variables ( E v , S, G) by suitable exponents (< 1) , such as γ and δ in Eq. ( 2), is also relevant because signal compression is generally a significant property of the human perceptual system.

Figure 1 .
Figure 1.Simplified processing pathway of the human's visual perception in response to light.The Figure is reprinted from (Zandi 2022, 35 ) under CC BY-4.0 license.

Figure 2 .
Figure 2. Hypothesis for signal components of the human visual system to form a numerical parameter for brightness perception.Image source: Laboratory of Adaptive Lighting Systems and Visual Processing, Technical University of Darmstadt.

Figure 3 .
Figure 3. Spectral sensitivity of rods, S-cones and photosensitive ganglion cells (ipRGCs) compared to the V ( ) function and the V 10 ( ) function, Image source: Laboratory of Adaptive Lighting Systems and Visual Processing, TU Darmstadt.

Figure 4 .Figure 5 .
Figure 4. Experimental set-up for the subjective assessment of brightness.

Figure 6 .
Figure 6.Rating scale for the subjective assessment of brightness perception ( LS i , i = 1-25)) with the anchor points shown on the right: 0 and 100) 51 .

( 8 )Figure 7 .
Figure 7. Mean visually scaled brightness values (VSB) of all observers from the experiment in Section "Experimental method of the subjective study" and their 95% confidence intervals as a function of the parameter M(brightness) of Eq. (8) with R 2 = 0.955 , RMSE = 4.78.

Figure 8 .
Figure 8. Mean visually scaled brightness values (VSB) of all observers from the experiment in Section "Experimental method of the subjective study" and their 95% confidence intervals as a function of the parameter M(brightness) according to Fotios et al. 26 , M after Fotios = 0.036 • [E v • (S/V ) 0.24 ] + 40.946 with R 2 = 0.8161 and RMSE = 9.7123.

Table 1 .
Photometric and colorimetric data of the illumination spectra (SPD) used in the experiment.