Introduction

The emerging advanced communication technology has dramatically propelled the evolution of the internet of things (IoT) era that tens of billions of devices/sensors1,2,3,4 are involved in a vast data network. For the outburst of redundant data, artificial intelligence (AI) and computer vision (namely computers gaining information from visual inputs) are introduced into IoT devices for data acquisition, desired information extraction and automatic decision-making5,6,7,8,9. However, the traditional Von Neumann architecture are extremely costly in time and power10,11,12 for AI computing, especially artificial neural networks (ANNs), because of the isolated memories and processors (namely the Von Neumann bottleneck). With the capability of memory and processing, ANN hardware based on memristor arrays were recently implemented for in-memory computing13,14,15,16. Intelligent computer vision or visual neural network computing was also proposed by further introducing photosensing functionality into in-memory computing17,18,19,20,21,22,23,24,25,26,27,28,29, realizing the bio-inspired visual information processing, mimicking human eyes and brains (Fig. 1a). With tunable photoresponsivity as ANN weight, sensors themselves can constitute an ANN to sense and process images synchronously and thus in a real-time manner18,20. Nevertheless, it is challenging to store weights locally into a sensor itself and the frequent weight extracting from the have-no-alternative-but external memory suffers additional latency and energy consumption18,19,28. Several tunable photosensors for AI vision have been reported, in which various techniques were introduced to improve the speed18, power23,30, and programmability29 of the photosensors. Most of these works characterize the photoresponse on an individual device and construct array-based networks for AI vision by computer simulation, instead of their hardware implementation.

Fig. 1: Concept diagram and device arrays for in-sensor ANNs.
figure 1

a Comparison between biological vision and artificial vision processes. b The experimental platform with a flexible sample connected to a flexible printed circuit (FPC). c Optical micrograph of a 5 × 5 photomemristor array on a flexible substrate. The colorful patterns can be ascribed to optical interference due to the smooth film with a roughness of 2 nm. The scale bar is 50 µm.

Full size image

To address these challenges on the way to in-sensor ANNs, a qualified candidate should possess uniform array, scalability, and considerable photoresponsivity in response to normal visible light. Inorganic ferroelectric photovoltaic effect has been successfully used for local sensor weighting20,31,32,33 to build sensors serving as neurons integrating the functionalities of sensing, memory and processing. However, heavy metals or toxicity, processing technique issues with a high cost, and the intrinsic wide bandgap of insulating ferroelectrics strongly limit their in-sensor applications. The emergent non-toxic and lead-free molecular ferroelectrics (MFs)34 with high ferroelectricity could provide an alternative choice for such ANNs. Some MFs even show semiconducting properties which can sense visible light35, which implies the feasibility of MF-based in-sensor ANNs with the advantages of low cost, scalability, flexibility and biocompatibility36,37,38, though the exploration for MF-based devices is still on the early stage.

In this paper, we implement the hardware of self-powered in-sensor ANNs using MF-based photomemristor arrays for in-situ optical signal acquisition and recognition. A 5 × 5 array of the photomemristors was fabricated as a reconfigurable artificial retina (as shown in Fig. 1b, c), with the tunable conductivity and photoresponse of a lateral MF/semiconductor heterojunction39. In spite of the insulating feature of the MFs, photoresponsive functionality for white light illumination is achieved simply by attaching a molecular semiconductor layer onto the MF layers. The functionalities of sense-memory-computation (i.e, light sensing, nonvolatile optoelectronic memorization and convolution sum of photocurrents from multiple devices) were integrated into one actual device array. In experiments, the in-sensor ANN successfully performed image classification with a 100% experimental accuracy. Recognition and regression for hand-written Chinese characters were also simulated using a larger scale optical and electrical hybrid in-sensor ANN, exhibiting the capabilities of machine identification and recalling of different optical inputs. The successful in-situ hardware implementation of in-sensor ANNs using an MF-based photomemristor array pave an avenue for in-sensor computing applications with flexibility, biocompatibility, low cost, and so on.

Results and discussion

Electrical characterization of the memristor array

Figure 2a is the device architecture of the interfacial type MF photomemristors. Organic semiconductor layers (copper phthalocyanine, CuPc) were thermally deposited onto various substrates with interdigitated electrode arrays whose channel length is 2 μm. MF Diisopropylammonium bromide (DIPAB)34,36,37,39 films were then blade coated (Supplementary Fig. 1) on the CuPc films. Molecular structures are given in Supplementary Fig. 2a. The thickness of CuPc and DIPAB films is estimated to be 100 and 300 nm (see the cross-sectional scanning electron microscope (SEM) image in Supplementary Fig. 2b), respectively. X-ray diffraction (XRD) patterns were collected for the DIPAB films, which indicate a high crystallinity and crystallographic preferential orientation with the in-plane polarization c-axis (Supplementary Fig. 2c). The phase-voltage hysteresis loop and switchable ferroelectric domain images collected by piezoelectric force microscopy (PFM) (Supplementary Fig. 3) further confirm the ferroelectricity of the DIPAB films with its c-axis parallel to the coating direction. The high ferroelectricity of the DIPAB films with very small coercive fields Ec and large spontaneous polarization Ps can ensure a small operation voltage (<3 V) for the lateral two-terminal devices, even for the devices with a wide electrode gap on a micron scale.

Fig. 2: Electrical characterizations of the MF/CuPc photomemristor array.
figure 2

a Diagram of the MF photomemristor device. b I-V curves collected by continuous positive (red line) and negative (blue line) voltage sweeps. The inset shows a whole I-V curve with an ON/OFF ratio of ~10. Current dependence on (c) pulse amplitude Vp, e period T, and g pulse width W, from which PPRs were respectively extracted d, f, h.

Full size image

Electrical characterizations for the devices with the coating direction perpendicular to electrodes were performed (see the arrow shown in Fig. 1c). The interfacial devices can be in-plane polarized using bias voltages exceeding its coercive voltage (around 2 V for this device, see Supplementary Fig. 4). Symmetrical counterclockwise (both in the first and third quadrant) current-voltage (I-V) hysteresis curves were observed from this DIPAB/CuPc interfacial device (Fig. 2b). ON/OFF ratio of the hysteresis curve can be estimated to be ~10 (see the inset of Fig. 2b). Continuous voltage sweeps with a consistent polarity can gradually increase (decrease) the conductivity of the MF/CuPc interfaces, exhibiting an analog behavior (i.e., gradual resistive switching).

The analog switching behavior shows a high potential for hardware-based multi-state neuromorphic systems40 due to the voltage pulse-induced conductance changes, as experimentally demonstrated in Fig. 2c–h. The amplitude, period and width (or duration) of the voltage pulses can be defined as Vp, T, and W, respectively. The stimulus threshold for the firing of neural signals can be simulated by the coercive voltage Vc of the ferroelectric material. As shown in Fig. 2c, d, when W and T are kept at 50 ms and 200 ms, the current or conductance nearly keeps unchanged for the Vp less than 2 V, even when a large number of pulses are applied. In contrast, the conductance abruptly increases once the Vp values exceed the threshold at which the polarization reversal of the MF layer immediately occurs. At this moment, massive charges can be injected for the screening of polarization, including the electron injection through the p-type CuPc layer due to a magnified huge built-in electric field39. Under thermal equilibrium, the stored screening charges can contribute to the conductance of the MF/CuPc interface, even after withdrawing the applied voltages. The compensation degree of ferroelectric polarization can be tunable by controlling the charge injection via applying voltage pulses. Both multiple intermediate ferroelectric domains and the tunable incomplete compensation ensure the tunable neuromorphic synaptic behavior for ANNs.

Paired-pulse ratio (PPR) for each data set was extracted, which describes the weight changes of ANN neurons depending on the probability of vesicular release in synapses. The parameters W and Vp were fixed to 50 ms and 4 V with a change of T from 60 ms to 2 s, as shown in Fig. 2e, f. It is noted that longer pulse periods lead to a lower possibility of facilitation for the synaptic devices even with constant stimuli. In Fig. 2g, h, T and Vp were fixed to 1 s and 4 V with a change of W from 20 ms to 0.5 s, respectively. A positive correlation between pulse durations and synaptic facilitation can be observed. Hence, continuous pulses lead to continuous changes in conductance, similar to the excitatory and learning behavior of synapses in response to repeated stimuli. The energy consumption is estimated to be as low as 1.87 nJ for one operating event, which can be further improved by optimizing the device architecture, such as reducing the channel length and semiconductor layer thickness.

To investigate the flexibility of the MF-based photomemristor array, conductivity variation with different bending radiuses was also investigated under saturated positive and negative states after poling using ±4-V bias voltages with a 5-s duration. As shown in Supplementary Fig. 5, the array can maintain 70–80% of its original performance before bending untill the bending radius reaches 3 mm. It should be pointed out that better flexibility of the MF-based photomemristor array can be expected if patterned films are adopted due to the release of bending stress, which has been demonstrated in our previous work38.

Optoelectronic characterization and synaptic behaviors

In addition to the tunable conductivity, optoelectronic properties of this memristor are also expected to be programable due to the semiconducting CuPc and ferroelectric polarization. The conductance states obtained after poling with a positive voltage and a negative voltage far exceeding its Vc are defined as ‘S+’ and ‘S-’, respectively. The applied bias voltage for poling is 4 V with a pulse duration of 20 s. Then, the I-V curves under dark and white-light illumination (using a white-light LED, 340–750 nm, 54.55 µW cm−2) conditions were collected from −2 to 2 V for comparison (Fig. 3a). Under dark condition, the interfacial device after poling shows a distinct rectification behavior, which can be ascribed to the incomplete compensation induced built-in electric fields. Note that, without illuminations, the I-V curves under both polarization orientation honestly cross the zero point. Under the white-light illumination, significant changes for the I-V curves were observed with their short-circuit current Isc about −0.28 nA (S+) and 0.31 nA (S-), respectively. Furthermore, combined with the quantized writing of conductance state by the voltage pulses above, multiple self-driven photocurrents after each stimulus can be obtained (Fig. 3b). The writing voltage pulses with an amplitude of 4 V can ensure a wide programmable photoresponsivity ranging from −6 mA W−1 to 6 mA W−1.

Fig. 3: Photoresponse of the MF/CuPc memristor array.
figure 3

a I-V curves of ‘S+’ (blue) and ‘S-’ (red) under light (lines) or dark (dots) state. Inset gives the magnified curves with short-circuit photocurrent at 0 V. b Pulse number dependence of the current (lines) with corresponding photoresponsivity (dots) after each pulse. c The photoresponsivity evolution under the stimuli with alternated 100 negative pulses and 100 positive pulses (4-V amplitude, 100-ms width and 1-s period). d The light intensity dependence of the multi-level photocurrents. The linear fitting coefficient of determination (R2) linearity. e Schematic illustrations of the mechanism for the self-driven photoresponse.

Full size image

Under a series of pulses (4-V amplitude/100-ms width/1-s period), a reversible, quasi-continuous and quasi-linear photoresponse was obtained and shown in Fig. 3c. It demonstrates that the weights of devices can be precisely tuned by pulse stimulation. For each electric stimulus (i.e., a 4-V poling pulse) with a duration of 2 s (Supplementary Fig. 6), the photocurrents were kept essentially constant in seconds, which is able to endure conventional ANN processing events. However, an obvious decline of the photocurrent after a 10-h duration is observed. For a long-term ANN processing, it is essential to inhibit depolarization effect by better charge compensation. As the basis for convolution of light intensity and weight (photoresponsivity), a light intensity-independent photoresponsivity should be required, which asks for a linear relationship between the photocurrent and light intensity. The dependence of these photocurrents with light intensity was obtained and shown in Fig. 3d, which demonstrates a good linearity at different polarization degrees (i.e., from state I to IX). Therefore, this MF photomemristor is applicable for in-sensor computing.

A schematic illustration of the operation mechanisms for the self-driven photoresponse is depicted in Fig. 3e, which can be ascribed to the net electric fields in the MF (Ef) with tunable multi-step remanent polarization (Pr). As shown in Fig. 3e(i), e(ii), the injected charges from the electrodes for screening polarization generate an opposite field Esc (blue arrow) to the depolarization field Ed (red arrow) caused by polarization bound charges. The thick screening length in the CuPc leads to an incomplete compensation (i.e., Esc < Ed), which induces a partially screened Ed and thus a net built-in field (i.e., Ef = Ed - Esc). The Ef naturally results in a potential difference between the two sides of the MF layer.

To address the relations among the Ef, Pr and screening effect of the injected charges, the semiconductor/ferroelectric/semiconductor structure (i.e., a S/F/S physical model in Supplementary Fig. 7a) is employed for analyses41. After solving Maxwell’s equations, the solution of Ef and Pr can be written as

$${E}_{{\rm{f}}}=-\frac{2\alpha {L}_{{\rm{D}}}}{2\alpha {L}_{{\rm{D}}}{\varepsilon }_{0}{\varepsilon }_{{\rm{f}}}+l}{P}_{{\rm{r}}},$$
(1)

where

$$\alpha =\frac{{\rm{cosh }}\frac{L-l}{2{L}_{{\rm{D}}}}-1}{{\varepsilon }_{0}{\varepsilon }_{{\rm{s}}}\,{\rm{sinh }}\frac{L-l}{2{L}_{{\rm{D}}}}},$$
(2)

LD is the Debye length of CuPc and ε0 denotes vacuum permittivity. εf and εs are the relative permittivity of the MF (εf = 85) and CuPc (εs = 3.6), respectively. l is the thickness of the MF in the model, while (L-l)/2 represents the thickness of compensation layer (see the theoretical analyses in Supplementary Fig. 7). Upon light illumination, excitons in the CuPc could be separated by the stray fields in the compensation layer with a significant band bending (Fig. 3e(iii), e(iv)) and the transient net electric fields at the vicinity of the MF dipoles42, as well as the Ef. Photogenerated charges can be collected due to the potential difference. As expected, the polarities of the resultant photocurrents are opposite to the polarization orientation. As given in Supplementary Fig. 7b, Ef and Pr are positively correlated, indicating a degeneration of the screening effect with a lager Pr. Hence increased Pr caused by multiple stimuli with constant polarities can generate a larger Ef in the multi-layer structure and lead to a larger Iph (Fig. 3b), enabling the capability for optoelectronic synaptic behavior. In addition, the device with symmetric Au electrodes naturally exhibits a bipolar performance (Fig. 3e(ii), e(iv)).

MF photomemristor array-based ANN for image recognition

With the capability of fundamental arithmetic calculation for in-sensor ANNs (i.e., summing of multilevel photocurrent, Supplementary Fig. 8), a single-layer ANN hardware of the MF photomemristors was designed and fabricated, with a 5×5patterned optical image (Fig. 4a, b) and a test platform (Fig. 1b and Supplementary Fig. 9). In this ANN, weights of the synapses are defined by different degrees of ferroelectric polarization in prior (as given in Fig. 3b, c). To ensure the availability of the array, the weight (photoresponsivity) mappings throughout all the devices in both ‘S+’ and ‘S-’ states are collected under white-light illumination. No defective unit can be observed (Supplementary Fig. 10). Examination of photocurrent accumulation was conducted by connecting one terminal of each device in the array to a common node (the specific test circuit and procedure are shown in Supplementary Fig. 11). The other terminal of each device is for independent writing. During the collection of photocurrent, the array is illuminated by white light illumination. Figure 4c shows that output photocurrent Iout varies (from −7.11 nA to 7.05 nA) with the number of written devices (i.e., poled devices). The horizontal coordinate of Fig. 4c indicates the number of synapses under ‘S+’ and ‘S-’ states, with the rest synapses at their original state. The linear relationship between the photocurrent values and involved synapse numbers indicates the capability of the array for convolution sum based on Kirchhoff’s current law.

Fig. 4: An ANN hardware for image acquisition and recognition.
figure 4

Schematics of (a) the 5 × 5 single-layer ANN and (b) a mask for optical image. c Iout versus the number of devices in the states of ‘S+’ or ‘S-’. d The photomemristor array with quantized weights (3 steps). e Iout under five independent tests for each letters, i.e., 'F', 'D' and 'U'. f–h Design diagrams (left) of the ‘F’, ‘D’ and ‘U’ letter masks with their resultant image (right). After writing weights to the 5×5 array, 3D images (bottom) of the short-circuit photocurrents from each device are collected, as well as correpsonding Iout values.

Full size image

This ANN was then used to receive and recognize three letter images ‘F’ ‘D’ and ‘U’ from letter-shaped masks (see Fig. 4b). Dataset consisting of 5×5 pixel images for each letter was built in advance. Random pixel absences and noise were then introduced to enlarge the dataset to 100 samples per letter, enhancing the robustness for recognition (Supplementary Figs. 12–14). A single-layer perceptron with 25 input neurons and 1 output neuron (Fig. 4a) was trained using this dataset. Output levels represent different recognition results. The cost function, mean square error (MSE), during training is shown in Supplementary Fig. 15a. After training, the weights were quantized into multiple steps (levels) by minimizing the MSE between the weights before and after quantization. Each step refers to a degree of ferroelectric polarization. Also, recognition MSEs with different quantization levels (all the weight distributions were drawn in Supplementary Fig. 16) are given in Supplementary Fig. 15b. The results show that 3-step quantization is enough to present a reasonable outcome. After writing the 3-step weights (as shown in Fig. 4d) into the array, the optical letter images were casted to obtain accumulated photocurrent (Iout). For repeatability, five independent experiments for each letter were conducted, as shown in Fig. 4e. All the data were written and read within seconds, avoiding state variation from fatigue. Designed and practical photomasks, as well as the corresponding photocurrent for each pixel and Iout after weighting, are given in Fig. 4f–h.

The classification uniformity of the optical letters is evaluated with the quotient (QUO) of between-class covariance and within-class covariance, based on Fisher linear discrimination, as given in below equation:

$${\rm{QUO}}=\frac{\mathop{\sum }\nolimits_{i=1}^{N}{m}_{i}{\left({\mu }_{i}-\mu \right)}^{2}}{\mathop{\sum }\nolimits_{i=1}^{N}\mathop{\sum }\nolimits_{j=1}^{{m}_{i}}{\left({x}_{j,i}-{\mu }_{i}\right)}^{2}},$$
(3)

where N is the number of classes, mi is the number of samples in the ith class, xj,i is the jth sample in the ith class, μi is the mean values of samples in the ith class, and μ is the mean values of all samples. The value QUO ≈ 173 can be calculated with the data in Fig. 4e, which shows a high classification uniformity here with the experimental accuracy reaching 100%. It is worth noting that the ANN itself operates in a short-circuit state with a zero processing energy consumption (i.e., self-powered processing), implying the potential of the MF photomemristor arrays for low-power in-sensor neuromorphic computing.

Simulation of large-scale arrays for Chinese character recognition and regression

To further exhibit the potential of the MF photomemristors for artificial retina applications (Fig. 5a), hand-written Chinese character recognition and regression by simulation were performed based on a larger array. Figure 5b gives the neural network diagram, including two cascaded recognition and regression parts, respectively. Figure 5c depicts the principal schematic of the MF photomemristor array. As a configurable artificial retina, the array integrates the functions of sensing, memory, and computing. It was designed to perceive and process optical patterns of hand-written traditional Chinese characters of digits from 1 to 10 (see the datasets in Supplementary Figs. 17 and 18). The array consisting of 48 × 36 pixels was proposed for perceiving optical patterns with the same size, where each pixel acts as an input neuron. Figure 5c shows a magnified view of the pixel <m, n> with 10 MF photomemristors whose photoresponsivities represent 10 synaptic weights. Hence, the array contains 17,280 devices with 48 row lines and 360 column lines. Unlike the normal strategy with two memristor devices connected in inverse-parallel for one synaptic weight to realize negative weight values43, the photoresponsivity of the proposed photomemristor can be pre-programmed into either positive or negative value. Both the positive and negative synaptic weight can be implemented with only one device, reducing the size of whole systems by a factor of 50%. As shown in Fig. 3b, c this photomemristor is a programmable two-terminal device, hence the topology containing row and column lines throughout the entire array could be realized without requiring any extra configuration pin for individual devices18. This topology guarantees the possibility of implementing an array shown in Fig. 5c on an even larger scale.

Fig. 5: A Large-scale array for Chinese character recognition and regression.
figure 5

Schematics of (a) human vision with feedbacks, and (b) an ANN for artificial vision. c A large-scale photomemristor array for Chinese character recognition and classification. d Data acquisition process for Chinese character image. e Confusion matrixes during the training process from epoch 1 to 100. f Quantized photoresponsivity of the kth devices arranged in the order of pixels. g Accuracy and loss for test set during training. h Output comparison with and without the regression network. i Error histogram and distribution with and without the regression network.

Full size image

Figure 5d, e displays the acquisition procedure of the dataset for the Chinese characters and the examples for confusion matrixes during the ANN training process, respectively. Then, synaptic weights were quantized by minimizing the MSE between the values before and after quantization according to the experimental results of the photoresponsivity in Fig. 3b. Figure 5f draws the quantized photoresponsivity of the kth devices arranged in the order of pixels. It can be clearly observed that the features of each character have been learnt successfully. Accuracy and loss for the test set during the training are also demonstrated in Fig. 5g. The final classification accuracy is 96.9%. The histogram indicating the weight distribution is also given as Supplementary Fig. 19.

Although the classification network exhibited a good outcome, it remains necessary to realize the regression of numbers 1 to 10, so that AI could recall the input digits. Therefore, a regression network composed of 2 hidden layers and 1 output layer was constructed after the classification layer. Detailed network design is presented in Supplementary Fig. 20. If the regression network does not exist and the regression output is obtained simply by convolving the classification output with the vector [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], the result would be unsatisfying, as compared in Fig. 5h. Figure 5i shows the error distribution and its probability density function (PDF) with and without the regression network, which indicates that the result becomes more precise with the regression network. For instances, mean absolute error (MAE) and root mean square error (RMSE) of the regression results are only 0.2895 and 0.5029, respectively, much smaller than those without the regression network.

Compared with traditional ANN strategy for optical pattern processing, the all-in-one reconfigurable photomemristor array in this work can not only avoid data conversion and transmission between separate units, but also possesses the capability of zero energy consumption for sensing. The comparison with other photomemristor arrays for in-sensor ANN18,20,24,25,29,44,45 are summarized in Supplementary Table 1. Our all-in-one organic photomemristor networks show low energy consumption comparable to present inorganic ones, which naturally possess the advantages of organics with low cost, scalability, flexibility and biocompatibility. The organic ferroelectric array with a simple 2-terminal device architecture implies a potential for broader applications in healthcare wearables, interactive robots, smart packagings46.

In conclusion, we successfully demonstrated ANN hardwares for artificial retina using MF-based photomemristor arrays. MF DIPAB/CuPc interfacial photomemristors were employed to construct in-sensor ANNs with continuously reconfigurable conductance and photoresponsivity. The photoresponse of each device is in-situ, multi-level and linear with light intensity, which lay the basis for convolutional computation of in-sensor ANNs. Then, a 5×5 photomemristor array was prepared for optical letter image recognition. Positive/negative synaptic weights can be logically expressed by bi-directional ferroelectric polarization degree, which can reduce the hardware complexity by at least half for conventional in-sensor neural networks. It is demonstrated that the depolarization-driven photoresponse can lead to low latency and even zero processing energy consumption for in-sensor ANNs. We also successfuly simulated a large-scale optical and electrical hybrid ANN for the recognition and regression of hand-written traditional Chinese characters. The realization of artificial vision clearly give the hints of MF-based photomemeristor arrays for in-sensor and neuromorphic computing.

Methods

Device fabrication

All materials and solvents were commercially obtained, except DIPAB which was synthesized as described in ref. 34. Various substrates were employed including silicon, glass and polyimide (PI), which are ultrasonically cleaned with acetone, isopropanol, and deionized water in turn for 30 min, respectively. The residuals on the substrates were blown off using nitrogen. Subsequently, electrode patterns were developed on a spin coated photoresist film by deep ultraviolet lithography. Cr (5 nm)/Au (70 nm) electrodes were then deposited by physical vaper deposition (PVD). After lift-off, Au pair electrodes with a gap of 2 μm was obtained. Semiconductor interlayers of CuPc thin film were thermally evaporated. The thickness of the CuPc films was estimated to be around 100 nm. Finally, a 300-nm DIPAB film was prepared by blade coating on the CuPc layers. To remove residual solvents, the samples are heated at 80 °C in air for 1 h. The substrate of mask is a glass with a thickness of 200 μm. After depositing a thin Ag film with a thickness of 200 nm, deep ultraviolet lithography was employed for patterning as described above.

Characterizations

To characterize the devices, SEM images were recorded using a Zeiss Sigma HD scanning electron microscope. The thicknesses were tested by a step profiler (Alpha-Step D-500 Stylus Profiler) Piezoresponse force microscopy (PFM, Asylum Research, Cypher) was utilized to obtain the surface morphology and ferroelectric properties of the DIPAB films at room temperature in air. Samples were characterized with an AC tip bias of 1 V. For poling, a DC tip bias of 30 V was applied with electrodes disconnected to measurement circuits. XRD patterns were recorded on an X-ray diffractometer (Bruker AXS D8) at room temperature.

Electrical measurements

Electrical characterization was carried out by fixing samples on a probe station with electrodes connected to a source meter (Agilent B1500A). The array sample is held in a specially designed fixture and connected to the external circuit via FPC. A UV-Vis LED of 340 nm-750 nm was employed as the light source for the photocurrent measurements through a microscope system. For electrical writing, inputs are fed to the corresponding electrodes in turn. For optical reading, all the electrodes are grounded to collect the sum of the self-powered photocurrent under illumination.

Chinese characters recognition and regression simulation

The traditional Chinese characters of digits 1 to 10 (Supplementary Figs. 17 and 18) were written on papers in calligraphy by volunteers and scanned into a computer. As illustrated in Fig. 5d, the images were firstly filtered with threshold filters for dust removal. Next, character regions were selected and scaled to a normalized size, approximately 80% of entire page in this case. Then the images were down-sampled into 48 × 36 size, which refers to the shape of a retina array. Finally, the pixels were quantized into 256 levels (8-bit) ranging from 0 to 1, corresponding to the optical power (0 to 54.55 μW cm–2). Totally 20 samples for each character were obtained. For each digit in the dataset, data were randomly divided into two sets, 60% for training and 40% for testing.

Figure 5c draws the principal schematic of the photomemristors array. One end of all the devices in every pixel are shorted together, while the common nodes of all the pixels in each row are connected to a row line. Conversely, only photomemristors with same index in the pixels in each column are connected to a column line. In the pre-programming stage, photoresponsivity of all the 17,280 photomemristors are programmed by applying voltage pulses to the row and column lines. During the sensing stage, the array forms a single-layer perceptron to realize the classification of hand-written Chinese digits. Overall, there are totally 48 row lines and 360 column lines. Each photomemristor yields a photocurrent under illumination, Im,n,k = Pm,nRm,n,k, where Pm,n is the light intensity on pixel <m,n > , Rm,n,k is the pre-programmed photoresponsivity of the kth memristor in the pixel <m,n > , k = 1,2,…,10 is the index of synaptic weights. By shorting the column lines of the same synaptic weight index, the photocurrents sum up as

$${I}_{k}={\sum }_{m=1}^{48}{\sum }_{n=1}^{36}{I}_{m,n,k},$$
(4)

constructing ten output neurons. Among the ten output neurons, the one with the most significant output value represents classification result. The neural network for classification is only composed of photomemristors, and does not employ any bias values and activation function that require complicated circuitries or additional softwares.

The regression network was trained with MATLAB using Levenberg-Marquardt backpropagation. Outputs of the classification network (200 samples) are used as dataset, of which 70% for training, 15% for validation, and 15% for testing.