Cognitive gripping with flexible graphene printed multi-sensor array

Robotics for task simplification of domestic, household, workplace and other assistive activities require efficient robots with decision-making capabilities. Here we report a fully printed graphene-based capacitive multi-sensor array (CAPSENSAR) employed in a cognitive robotic gripper (COGBOT) for decision-making operations. The CAPSENSAR created a contactless capacitive impression of the gripped object surface to determine the optimum gripping pressure. The controlling unit of the COGBOT was associated with an algorithm to address potential breakage. If slippage was detected via the array, the grip pressure was revised to reduce the possibility for damage. This facilitated slippage-free and damage-resistant gripping of the target objects without user interference. Array fabrication was straightforward using a customizable electrode design with cost-effective and biocompatible materials.


Section 1.1: Mathematical Modelling of CAPSENSAR
The CAPSENSAR consists of mutually orthogonal arrangement of TE and BE at separate planes to form an arrangement of 5×4 capacitive proximity and pressure sensor array.The proximity sensor of the CAPSENSAR works on the principle of distortion in fringing electric field lines, which emanates from positively biased TE and terminates at negatively biased BE (Supp.Fig. 1), when an object is introduced in its vicinity.On the other hand, the pressure sensor works on the change in effective dielectric thickness of CAPSENSAR under applied pressure P. The device is mathematically investigated by analyzing the performance of an arbitrary sensor unit in the array.To investigate the performance of an arbitrary ( , ) ijelementary sensor unit in the CAPSENSAR, mathematical modelling was performed on that sensor unit by considering the influences due to the nearest and the second nearest neighboring sensor unit.
Theoretical studies was carried out to determine the change in output capacitance C  experienced by the ( , ) ijsensor unit in presence of an approaching metallic object along the normal to that sensor unit at proximal distance z from it as discussed here.
Here we derive the expression for the change in output capacitance ΔC in a sensor unit due to the presence of an external metallic object.For this we consider an (3 3)  array of sensor units representing a section of the CAPSENSAR as shown in Supp.Fig. 2. Each sensor unit, consisting of a TE and a BE, is schematically shown as square elements of side a and denoted by matrices (i, j), where (i, j) represents the central sensor unit, and ( , 1) ij  , ( 1, ) ij  and ( 1, 1) ij  denotes the nearest and second nearest sensor units respectively.Since the TE is positively biased relative to the BE with an applied voltage V , the former holds a charge TE .QA

+= 
, while the later with the negative equivalent.Here, TE  denotes the surface charge density on TE and A is the effective area of the square shaped sub-electrodes sE.As the CAPSENSAR is designed with non-overlapping sE of TE and BE, the electric field lines emanating from the TE and terminating at the BE of the device are mostly due to fringing field effect of the TE, which are intense in the proximity of the electrodes and decays sharply with distance z.Here each identical sensor unit generates an intrinsic fringing electric field i,j fr () Ez which spatially spreads over the neighboring space around the respective elements.
A metallic object O is introduced at a distance z from the (i, j) sensor unit, it is immersed in the respective electric fields from each of the (3 3)  representative array of sensor units, thereby experiencing a resultant field obj fr E .Thus obj fr E is defined as the non-uniform fringing electric field between the object and the TE of the (i, j) sensor unit due to the charges obj Q induced at the surface of the object by (3 3)  sensor units.The contributions to obj fr E includes (i)   i,j fr () Ez at a distance z from the (i, j) sensor unit, (ii) (i±1,j),(i,j±1 ……………………………….(Supp.Eq. 1), Thus, we first derive the expression of i,j fr () Ez generated due to the contribution of TE  on the TE of (i, j) sensor unit.
Let the area of an elementary charge unit on the TE of (i, j) sensor unit be area dA dx dy =and the distance of the object at O from this elementary charge unit be R as shown in Supp.Fig. 2 (inset).The i,j fr () Ez can be expresses as, ( ) where, the  and k represent the angle between z and R and unit vector in z direction respectively.Substituting the ( ) in Supp.Eq. 2a and using Supp.( ) Where  represents the interelectrode distance of the TE array and  denotes the absolute permittivity of the dielectric medium.Thus substituting Supp.Eq. 2b, c, d in Supp.Eq. (1) we get, The obj fr E acts along the normal to the plane of the (i, j) sensor unit and directed outward from point O.This presence of obj fr E fringing lines of force produces distortion and annihilation of the i,j fr () Ez fringing lines of force terminating at BE due to their interference at the proximity of the (i, j) sensor unit and produces a change in potential drop i,j V  across the TE and BE of the (i, j) sensor unit.The The C  can be theoretically expressed using Supp.Eq. ( 4) and Supp.Eq. ( 6) as: Where, i,j

V
 denotes the change in potential drop across the TE and BE of the ( , ) ijsensor unit, obj fr E is the non-uniform fringing electric field between the object and the TE due to the charges obj Q induced at the surface of the object and has an arctan dependence with z (Supp.Fig. 3a), i,j fr E intrinsic electric field between TE and BE when the object is absent , ( ) where  and  are the absolute dielectric permittivity of the medium (air) and interelectrode distance between two sub electrodes of adjacent TEs respectively.At z=0 (touch) when the metallic object touches the TE of the sensor unit, the capacitance between TE and the object is annulled and the


as the second term in (Supp.Eq. 13) is a constant for a fixed object.

Section 1.2: Effect of dielectric target object on output capacitance
Now we determine the change in output capacitance of the (i, j) sensor unit due to an approaching insulating object at proximal distance z.Here we consider a dielectric object of thickness τ, relative dielectric constant r  approaching the (i, j) sensor unit along its normal direction of (-z).The output capacitance measured by the (i, j) sensor unit is given by i,j out i,j , where Q is the charge on TE and , ij V is the potential drop across TE and BE.In the absence of the object, the output capacitance is given by abs i,j out i,j fr fr .( ) When the dielectric object is exposed to the (i, j) sensor unit of the CAPSENSAR in air of r 1  = , the output capacitance pre i,j out die i,j as the potential drop i,j V across TE and BE decreases.The i,j V between TE and BE in presence of the object is calculated using the relation: , where i,j fr E is the intrinsic fringing field between TE and BE and fr ( ) .

d z z
= is the effective dielectric thickness of the sensor unit for z>0, where  is the linear proportionality function of z. Thus Output capacitance of the (i,j) sensor unit in presence of the object is given by pre Unlike metallic object, when a dielectric material of permittivity r  is introduced at a proximal distance z from the (i,j) sensor unit, the pre i,j out i,j fr r 1 .1 on introduction of the dielectric object.For a fixed object, the pre i,j out C   varies linearly with z.

Section 1.3: Operation of the Cognitive robotic (COGBOT) gripper
The CAPSENSAR (L, R) integrated on the gripper palm (L, R) of the COGBOT is useful for cognitive gripping of the target object and also ensuring slippage and deformation free gripping.Since strong grip is associated with large gripping area, the COGBOT is programmed to determine the pair of faces of the target object with flattest area for effective gripping.The pair of CAPSENSAR using the proximity sensor array creates electronic capacitive impression of different faces of the target object as the arrangement of the pair of robotic palms with CAPSENSARs rotates about the central axis passing through object, thereby scanning the object faces through 360°.The capacitive impressions of different faces of the objects were transformed to their respective proximity distance z-equivalent matrices to estimate the landscape of each of the scanned object faces and their dimensions.The COGBOT performs cognitive computational operations to analyze these z-matrices to determine the fittest pair of opposite gripping faces with largest flat area for effective gripping.The identification of pair of flattest face for gripping allows the COGBOT, to grip the given object with optimum pressure to prevent damage.The optimum pressure was determined using computational steps and contributes to the cognitive operation of the COGBOT.The pressure sensor array of CAPSENSAR initially captures the capacitive impression of the object faces under gripped condition and transform the same into its equivalent P-matrices for the pair of opposite faces of the gripped object.The P-matrices help to ensure that the optimum pressure for gripping is maintained throughout the operation.The P-matrices are also used for the detection of slippage of gripped object and also monitor deformation of the object during gripping.

Supplementary Discussion 2: Simulation Results
As explained in Supp.Disc. 1, the CAPSENSAR operates on the principle of variation of fringing field capacitance  7) and Supp.Eq. ( 8)), the face landscape simulations were investigated in terms of obj fr E generated separately at different ( , ) ijsensor units.The sensor units which are at low z from segmented exposed face of the object generated high obj fr E relative to that which are comparatively at furthest distances.The () ij  array impression of obj fr E generated from the exposed object face facilitated recognition of face landscape and even determination of the dimension of the object.The simulation studies of face landscape recognition were investigated using various geometric shaped objects (with dimensions ~ and ij =20 mm) such as steel sphere, cone and disc kept at z=10 mm from the CAPSENSAR.Supp.shapes-sphere, cone and disc respectively.It is evident from Supp.Fig. 5a-c that the lateral variation in obj fr E in the x- and y-direction is clearly distinguishable for objects of different shapes and that these obj fr E -array impressions bear strong manifestations of 3-dimensional landscape of respective object faces.The stimulations were also performed by exposing different faces of the same custom-made object to the CAPSENSAR as illustrated in Supp.Fig. 5d When Face 1 and Face 2 of this custom-made object were exposed separately to the CAPSENSAR, the obj fr E array impressions for respective faces yielded distinctive patterns as shown in Supp.Fig. 5e and Supp.Fig. 5f respectively.The Face 2 generated a large flatter area as compared to Face 1 where the former face was suitable for reliable gripping.The face landscape estimation ability of the CAPSENSAR is utilized in COGBOT in the identification of flattest pair of opposite faces of the object which is useful in calculating the optimized gripping area and execute successful gripping.

Section 2.3: Pressure sensing
The simulation studies were carried out to investigate the effect of applied pressure P on the effective dielectric

Section 5.3: Characterization of elemenary sensor units
We characterize the elementary sensor units ( , ) ijof the CAPSENSAR as proximity and pressure sensor units.The capacitive outcomes from all the ( , ) ijsensor units in the CAPSENSAR constitute capacitive impressions in the form of matrix which was further transformed into z-matrices and P-matrices when the CAPSENSAR was operated in proximity and pressure sensing mode respectively.The z-matrices and the P-matrices of the capacitive impressions yield the z-contour and the P-contour plots respectively.The z-contour was used for contactless three-dimensional face landscape estimation of the exposed face of the target object while the P-contour was used to ensure reliable gripping of the object.All electrical characterizations were performed by positively biasing TE relative to BE.The experimental set-up for device testing is described in Supp.Discussion 5.

The electrical characterization of all ( , )
ijsensor units of the CAPSENSAR was performed to investigate the response of that sensor unit for (a) an approaching object at a distance z along the normal to that ( , ) ijsensor unit and the (b) pressure exerted on that ( , ) ijsensor unit.Since each sensor unit of the CAPSENSAR spans over an area of 3×3 mm 2 , the unit was tested using a stainless-steel cylinder of diameter ~a=3 mm and height 10 mm to reduce the effects of interfering field lines from large objects.
Section 5.3.1:Proximity sensor units All ( , )  ijsensor units of the CAPSENSAR were investigated for proximity sensing with different proximal distances z in the range of 0-180 mm.The dynamic measurements of output capacitance in present of object pre i,j out C   for the ( , ) ij sensor unit were performed when the target object was kept at different distances z from that sensor unit as shown in the Supp.were calculated using Supp.Eq. ( 9) and the mean of respective data set (of 20 data points) for different z were plotted in Supp.Fig. 10b.The mean i,j abs PROX out ] CC  for the ( , ) ij sensor unit increases as the z decreases when the object approaches the plane of that sensor unit and is attributed to the generation of increased change in intrinsic fringing field where 1 5×10 7 m -2 and 5 M =4×10 10 m -2 for stainless steel object as shown in Supp.Fig. 10b.The Supp.Eq. ( 15) obey theoretically calculated Supp.Eq. ( 9) and validated with simulation results in Supp.Fig. 10b.The sensitivity of the proximity sensor unit was determined from the slope of the abs i,j PROX out C C z  −  curve in Supp.Fig. 10b and found to be 0.012 mm -1 in the range z=0-30 mm.The z-resolution z  of the device within z< 30 mm was obtained from the Supp.Fig. 10a and b to be 0.091 mm.However, at higher distances z> 120 mm the z-resolution was found to be in the order of tens of mm.The high stability and excellent zresolution in the range 0-30 mm make the device suitable for use in surface landscape detection within z=30 mm.
Section 5.3.2:Pressure sensor unit The electrical characterization of the ( , ) ijsensor unit in response to pressure was performed under the same biasing conditions at different pressures (P) in the range 0.1-5 kPa.The dynamic measurement on the ( , ) ijsensor unit was performed with six different P=0.1 kPa, 0.5 kPa, 1 kPa, 2 kPa, 3 kPa, 4 kPa and 5 kPa.The The experiment was repeated for all ( , ) ijsensor units in the array and the respective  -P for the ( , ) ijsensor unit is obtained by fitting the experimental data in Supp.Eq. ( 16) as: where, 1 W =0.0057 kPa -1 and 2 W =0.002 for stainless steel object.The increase in pre .
with P in dynamic range is attributed to reduction in Ecoflex thickness  at increased pressure P. Supp.Eq. ( 16) obey the theoretically derived Supp.Eq. ( 13) and validated with simulation result as shown in Supp.Fig. 10d.The sensitivity of the ( , ) ijsensor unit was obtained from the slope of the calibration curve as 0.006 kPa -1 .Good response time and low instability error of the sensor unit facilitates fast and precise estimation of the optimum gripping force during gripping and also execute reliable and damage free gripping of the target object by COGBOT.
Section 5.4: Detection of face landscape using CAPSENSAR In this section, the proximity sensor array was utilized for the determination of three-dimensional face landscape of the target object in terms of capacitive impression as recorded by respective ( , ) ijelementary sensor units of CAPSENSAR.The pre i,j out C   recorded for all ( , ) ijelementary sensor units of the CAPSENSAR were compounded for determining the capacitive impression of the exposed object face.The demonstration for landscape estimation of the object face was performed using stainless steel objects of various geometrical shapes -sphere, cone and a disc with dimensions as given in Supplementary Table 1.The demonstration was performed by aligning the object at the center of the CAPSENSAR plane and kept stationery at a z=10 mm.The electronically recorded ij sensor unit of the CAPSENSAR.The i,j z elements constituting the z-matrix of a given object face were gridded in 5x4 matrix and illustrated as 3D z-contour plot representation using Origin 8.5 software for sphere, cone and disc as shown in Supp.Fig. 11d, e and f respectively.
Since the elements of the z-matrix denote the segment-wise normal proximal distances z of the object from the respective ( , ) ijsensor units, it provides an estimate about the dimensions of the object face as well as the spatial variation in object landscape.The two-dimensional profile of the exposed faces of different the objects-(i) sphere, (ii) cone and (iii) disc along their (1) x-and (2) y-dimensions are shown in Supp.Fig. 11(g, h), (i, j) and (k, l) respectively.
The accuracy in measurement of object

Fig
Fringing field lines of CAPSENSAR.Cross sectional view of the CAPSENSAR showing the electric fringing field lines emanating from the positively charged TE and converging in the negatively charged BE terminal.Supplementary Figure 2: Mathematical modelling of CAPSENSAR fringing field.Diagrammatic representation of (3×3) sensor unit array of a section of the CAPSENSAR showing the relative locations of sensor elements and the cumulative fringing field generated at the object O at normal distance z from (i×j) element.

.
Since the charge density on the surface of a metallic object (conductor) with finite conductivity depends on the skin depth of the metal at a fixed frequency, the obj  is dependent on the metallic property of the object.Thus the PROX C  varies when the experiment is performed with object of different metal having same shape and size.Although the obj  is a shape dependent quantity, the PROX C  of the single sensor unit is affected when the dimension of the local curvature<< a=3 mm i.e sharp.Thus for objects of dimensions a  , exposed to the sensor unit at a fixed z, may be considered as planar for which PROX C  is independent of shape and size of the object and solely depends on z.However, for an array of sensor units, each sensor unit records their respective PROX C  to construct a capacitive landscape of the segmented face of the object.The dimensions and surface morphology variations of the object can be computed from the calibration curve of the proximity sensor array.Thus, metallic objects (with dimensions 30 × 24 mm) of various shapes and sizes can be distinguished from capacitive impressions generated by the CAPSENSAR.On the contrary, when a dielectric material of permittivity r  is introduced at a proximal distance z from the ( dielectric thickness of the device decreases (see Supp.Disc.1.2).

E
Fig. 4a, b and c respectively.This distortion field obj fr 0 E → at large distances and becomes high obj fr E = 200 Vm -1 as Fig. 5a, b and c show the obj fr E -array impression for various geometric Supplementary Figure 5: Simulation studies on face landscape detection.COMSOL representation of the fringing electric field distribution (Efr array impression) on the elementary sensor units of the CAPSENSAR under exposure to different solid objects -a.sphere, b. cone c. disc and for d.Illustration of different exposure faces-Face 1 and Face 2 of the custom-made test object.COMSOL simulation results for Efr array impression captured by the CAPSENSAR when e. Face 1 and f.Face 2 of the object were placed at distance z=10 mm from the device.
ij elementary sensor unit of CAPAENSAR.When the object was in contact with the device z=0, the effective dielectric thickness fr 0 | z d = incorporates the PI thickness PI d and Ecoflex thickness  .Since the PI d is constant, the application of an external P by the object on the th ( , ) ij sensor unit produced a change   in thickness of the elastomeric Eco-flex dielectric layer.Simulation studies were performed to determine the displacement   in the Ecoflex layer of th ( , ) ij sensor unit under different applied P in the range 0-5 kPa.The   occurs in the -z direction Supplementary Figure 6: Simulation studies with pressure sensor unit.COMSOL representation of Eco-flex thickness variation   for a. P= 0 kPa, b. 2 kPa, c. 5 kPa and d.Variation of Ecoflex thickness and its corresponding change in normalized output capacitance with applied pressure P. The red solid square data points represent simulated Ecoflex thickness and the green solid circle denotes data points obtained from supplementary Eq. 13.in terms of the position of the object on the device (z=0).Supp.Fig.6a, b and cshow the COMSOL representation illustrating the   in Ecoflex thickness under P=0 kPa, 2 kPa and 5 kPa respectively.At increased pressure P, the Ecoflex thickness  was reduced, which led to the increase in PROX C  of the device following Supp.Eq. (13).The linear variation of  and PROX C  with P as obtained from simulation results are plotted in Supp.Fig.6d.The variation in output capacitance due to change in  under applied pressure was utilized to measure the gripping force to be applied on the target object.CAPSENSAR and the data were recorded for respective its ( , ) ijsenor units.The , applied pressure were recorded using the MUX and MC as described earlier.The COGBOT was tested with the pair of CAPSENSAR connected to the MC through separate 8:1 MUX as shown in Supp.Fig. 9b.The rotation and the gripping servos communicate with the MC through Signal 1 (SIG1) and Signal 2 (SIG2) respectively.The Arduino Mega board is powered by the computer.Supplementary Figure 10: Electrical characterization of proximity and pressure sensing unit.an approaching object at different distances z= 0,5,10, 30, 60, 90, 120, and 150 mm, recorded by the ( , ) ijsensor unit of CAPSENSAR, The mean  sets (acquired when the object approaches the device during dynamical measurement) are denoted by green rhombus points with electronic fluctuation represented by error bars.b.Normalized change in distance z calibration curve of a ( , ) ijsensor unit for an object of stainless steel, relative to the results obtained from theoretical and simulation studies, c. different applied pressures P= 0.1, 0.5, 1,2,3,4 and 5 kPa, showing alternative pressure and release cycles when the object was approached onto the ( , ) ij sensor unit from a distance of z=180 mm for each cycle.The green rhombus points denote the mean of the data points acquired when the object undergoes different pressure cycles.The electronic fluctuation is shown by error bars.and d.P calibration curve of ( , ) ijsensor unit showing a dead band in the range 0-0.5 kPa and its comparison with simulation results.The errors in measurements form 20 sensor units were represented by error bars.


Fig. 10a.The data were recorded at a time interval of 200 ms over a total sampling time of 100 s when the CAPSENSAR was kept flat on the testing bed.The mean ( pre i,j out Mean C   ) for different z were calculated from the dynamic data and the corresponding error ( fluc e ) due to electronic fluctuations were represented as error bars as shown in Supp.Fig. 10a.=3.5%.The steady fringing field distribution due to the uniform thickness of printed electrodes and low leakage current generation in the non-porous, highly thermal resistant and electrically insulated PI layer help to achieve the high stability in the performances of the sensor unit.The response time of the proximity sensor unit was graphically determined from Supp.Fig. 10a to be 0.3 s.The low response time may be attributed to electrostatic working mechanism of the sensor and enables high switching rate for rapid response sensors for fast generation of capacitive impressions of the exposed face of the object.Different ( , ) ij sensor units in the CAPSENSAR were investigated and pre i,j out C   were recorded at various z in the range=0-180 mm for all sensor units of sample size 20.The normalized change( at an interval of 200 ms when the object was subjected different pressures under alternative pressure and release cycles of 30 s each, spanning over a total time period of 390 s as shown in Supp.Fig.10c.During dynamic data acquisition, the object was descended from the proximity sensing pF at z=0 mm.The response time and instability error stab  of pressure sensor unit was graphically obtained from Supp.Fig.10cto be 0.4 s and 5.3% respectively. Fig. 10d.The errors in measurements were represented as error bars.The ( , ) ijpressure sensor units suffer a dead band region between 0-0.5 kPa where the sensor unit showed no variation in pre the local stiffness of the 30 µm thick PI sheet bearing the printed electrodes, constituting the proximity sensor array in the device.This stiffness of the PI sheet is unable to produce measurable change in output capacitance pre i,j out C   and thus limits the detection of very low applied pressure in the range 0-0.5 kPa termed as the dead band region.However, beyond P>0.5 kPa, with P in the dynamic range 0.5-5 kPa.The experiment was terminated at P=5 kPa to prevent damage of the graphene printed electrodes.
Origin 8.5 to obtain a capacitive impression of the object face under exposure to CAPSENSAR.These capacitive impressions were used for the estimation of the three-dimensional (3D) face landscape of the different objects such as sphere, cone and disc as shown in Supp.Fig.11a, b and crespectively.The capacitive impression for each object was obtained by the virtue of variation in pre i,j outC   of respective ( , )ijsensor units in response to the 3D profile of that object face exposed to the CAPSENSAR.The 3D profile of the object face produces segment-wise change in the obj fr in Supp.Discussion 2.2.The dimensions and landscape of the object face can be estimated when the ( , ) shown in Supp.Fig.11a,b and c were transformed into its corresponding i,j z to yield the z-matrix using the relation Supp.Eq. (15).The i,j z denotes the perpendicularly projective proximal distance of the object from the th ( , ) particular object, obtained under just contact condition z=0.The effective dielectric layer constitutes the parallelly arranged Eco-flex and PI of thicknesses PI d and  respectively.Under applied pressure P, the decrease in thickness of the eco-flex elastomeric dielectric layer reduces the fr d which linearly increases the z C C C ………………….(Supp.Eq.10), where PROX max C    is a constant for a fixed material and obtained due to roughness of object surface in contact with the TE.Under this condition, the pressure sensor unit records the applied P which commences from the pressure sensing baseline as given by fr d is the effective dielectric thickness of the device in the presence of the object at z=0.The PRES C  increases linearly with pressure P with slope TE i,j fr .A EK