Introduction

Energy harvesting, exploiting ambient energy for our purposes, plays a crucial role in human technological development1. Currently, an important focal area is micro energy harvesters (with output power 10–100 μW). These convert, through various transduction methods, ambient thermal and kinetic energy from the environment to electrical energy. They provide an in-situ power source for remote electronic devices, typically for powering sensor nodes of the Internet of Things. This avoids the problems associated with batteries and/or wiring2,3,4,5,6.

A key challenge for micro harvesters is that ambient energy sources are very often random. For instance, the amplitude and the frequency of a vibrational energy source can be highly variable. This makes it difficult to rectify the generated voltage/current and store the energy in an efficient manner2,3,7.

Interventions by an intelligent agent aids energy harvesting from variable sources in certain contexts, as exemplified by the interventions of a sailor, or a windvane turning a generator into the wind. Such examples serve to remind us that the 2nd law of thermodynamics, as used to prove that a Maxwell’s demon cannot work8, concerns maximum entropy single heat baths, whereas often forces in nature are not maximally random.

Highly sophisticated intelligent interventions in energy harvesting are now practicable, owing to advances in: (i) artificial intelligence software and hardware9,10, (ii) electronic interfacing circuitry11,12,13,14,15, and (iii) experimental and theoretical understanding of the relation between information and energy, such as the fact that reversible computation has no fundamental energy cost16,17,18. Taken together, this gives significant hope that intelligent intervention may be a powerful tool in mitigating the randomness faced by micro-harvesters.

We therefore here aim to identify intelligent interventions that allow micro harvesters to extract more rectified power from variable sources than current state-of-the art methods.

A key paradigm we adapt is to use interventions that are reversible and energy conserving. Moreover, for practical and fundamental thermodynamical reasons, these interventions are pre-programmed, chosen by systematic machine learning methods applied to past data from the harvester, as in Fig. 1. We consider general principles and for concreteness also a case study of a piezo-electric harvester which converts motion to electricity19. In this case study we simulate idealised, pre-programmed, bias-flips and phase-shifts on experimental output data. We find these can indeed replace and outperform the current state-of-the art: the diode bridge.

Figure 1
figure 1

We find that rectifying the voltage with intelligently chosen pre-programmed energy conserving interventions can lead to improved power output. Possible interventions include voltage bias flip with ON/OFF-period τ, and voltage phase shift of time θ. The trainer tunes θ and τ to optimise the voltage output.

We moreover note that the the diode bridge has a thermodynamically fundamental lower bound on power dissipation, whereas the methods use here do not.

We proceed as follows. We briefly describe the harvester being used as a case study. We describe the interventions, and how they can be intelligently chosen. We then give the results, followed by a discussion and conclusion.

Methods

The harvester we use in this paper as a case study is a dual resonant structure energy harvester19, which can harvest energy from random fluctuation sources at low frequencies (typically less than 100 Hz), consistent with motion of everyday objects such as human beings. As shown in Fig. 2, it consists of two piezoelectric devices, each outputting its own voltage time-series, with the voltages finally combined to give one voltage time-series. Both the two cantilevers are made of stainless steel with dimensions of 67 (length) 18 (width) 0.2 mm (thickness). On the surface of each cantilever, there is glued a lead zirconate titanate (PZT-5A) film with dimensions of 52 (length) 17 (width) 0.2 mm (thickness). The measurement is performed when the two piezoelectric devices are mounted on a shaker (Brüel&Kjær, 4810) to mimic the vibration sources from ambient environment.

Figure 2
figure 2

The bi-resonant harvester, originally designed in19. Two piezo-covered cantilevers with masses on their free ends are driven by the same vibrational motion source on the right axis. We consider how much bias flips and phase shifts on the outputs can enhance the power output.

We have used a signal generator (Brüel&Kjær, LAN-XI 3160) and a power amplifier (Brüel&Kjær, 2719), to generate excitation signals with tunable frequency and amplitude. An accelerometer is mounted with the energy harvesters to monitor the real time vibrations. The signal from the piezoelectric devices is connected to a data acquisition unit. It should be noted that even though piezoelectric devices are used for proof of the concept, the proposed method could also be useful for other related energy harvesters. We also remark that since the parasitic capacitance is low relative to load resistance, and the vibration is at low frequency, we may approximate the current and voltage to be fully in phase. Thus the instantaneous power respects \(P=\frac{{V}^{2}}{R}\) for a load resistance R and the average power respects \(\langle P\rangle =\frac{{V}_{{\rm{RMS}}}^{2}}{R}.\)

Diode bridge and its power consumption

A diode bridge, as in Fig. 3, will take any voltage polarity on the inputs to a positive polarity on the output, but at a loss in power.

Figure 3
figure 3

A diode bridge will take any voltage polarity on the inputs to a positive polarity on the output.

The loss in power is necessary given that each diode has a voltage drop. For practical device power dissipation calculations a pn-junction diode’s current vs voltage curve can be approximated as V = V0 + IR for the regime V > 020. The instantaneous power dissipated by a single diode when V > 0 is then P = IV0 + I2R. (The average and rms power dissipations follow immediately). The diode bridge has two diodes in each path and thus twice that dissipation.

Note also that in the case of two sub-harvesters the diode bridge can be applied on each before combining the voltages in order to avoid destructive interference, but again at a power loss.

2nd law mandates diode bridge power consumption

If it were possible to reduce the above power dissipation to 0, a diode bridge could be used to violate the second law of thermodynamics, by turning thermal current fluctuations into rectified current at no work cost. Kelvin’s version of the 2nd law states that no work can be extracted from a single heat bath in a closed cycle. Thermal voltage fluctuations depend on materials and it is beyond the scope of this paper to investigate their values for diode bridges used here, but e.g. the thermal voltage Vth = kT/e is approx 0.03 V at room temperature. (The argument can be modified to other fluctuation sizes).

For the device to be called a diode the current I ≈ 0 for negative voltages beyond the thermal fluctuation range of −Vth (up to some breakdown voltage which is outside of the range currently considered). Then for voltages in the positive thermal fluctuation range we must also have I ≈ 0, or else the diode would generate a current in a circuit embedded in the heat bath, a circuit which could include a load driven by that current, violating the 2nd law. Thus, according to the above argument, there is an inescapable voltage drop of Vth in a diode, and associated power loss. We now turn to the competing approach to turn the AC into DC and to remove destructive interference between voltages.

Two examples of intelligent interventions: sign flip and phase shift

The sign flip, which can also be called voltage inversion, can be written as V → −V where V is the instantaneous voltage. This can switch between being on and off with period τ. τ is a priori a free parameter and will later be set according to optimising based on past data.

The phase shift is simply a delay of the voltage time series by some amount ϕ (so strictly speaking it is a delay rather than a phase shift which should only be between 0 and 2 π times some period). It can be written as V(t) → V(t + ϕ) t where t is time.

Interventions are orthogonal matrices

It is convenient to use bra-ket vector notation here such that a voltage time series Vi(t0), ... Vi(tf) is a vector |Vi〉 with the first entry Vi(t0). (The time series is discrete as it is sampled experimentally at a finite rate). The transpose of the vector is denoted 〈Vi|, such that the dot product of two vectors |Vi〉, |Vj〉 is written as 〈Vi||Vj〉 = 〈Vi|Vj〉. In this notation

$${V}_{{\rm{RMS}}}=\sqrt{\frac{1}{d}{\langle {V}_{i}|V\rangle }_{i}},$$
(1)

where d is the dimension of |Vi〉. Moreover let \(|{V}_{i}^{^{\prime} }\rangle \) denote the transformed |Vi〉. This means the voltage time series one would get instead of |Vi〉 if the intelligent transformations are implemented.

In an idealised case the intelligent transformations preserve VRMS such that

$${\langle {V}_{i}|V\rangle }_{i}=\langle {V^{\prime} }_{i}|{V^{\prime} }_{i}\rangle \forall i\mathrm{.}$$
(2)

Then, if we also assume the interventions can be represented as matrices, the interventions correspond to orthogonal matrices O, meaning OTO = I where I is the identity and T the transpose. The idealised interventions we consider are indeed orthogonal matrices: phase shifting can be represented as a cyclic permutation of elements, a particular permutation matrix, and voltage inversion as a diagonal matrix with diagonal entries all 1 or −1. More generally the interventions \({\mathscr{S}}\) are naturally represented as matrices, since they should respect probabilistic mixtures of different voltages: \({\mathscr{S}}({\sum }_{i}{p}_{i}|{V}_{i}\rangle )={\sum }_{i}{p}_{i}{\mathscr{S}}(|{V}_{i}\rangle )\)21. This together with Eq. 2 implies the idealised interventions, beyond the examples of bias flips and phase shifts, should indeed be represented as orthogonal matrices acting on the voltage vectors.

Optimal interventions when combining two voltages

Now we can compare the VRMS before and after interventions. For example phase shifts can be used to reduce destructive interference, due to individual sub-generators producing voltages out of phase. Given two or more voltage time series, how high can the VRMS of the combined outputs be, if we are allowed to do intelligent interventions on each time series before combining them? For notational convenience let us consider \(d{V}_{{\rm{RMS}}}^{2}=\langle V|V\rangle \). Two time series illustrate the general case: V1 and V2. Suppose these undergo the transforms before being combined, how much can \(d{V}_{{\rm{RMS}}}^{2}\) change? Note that

$$\begin{array}{c}\langle {V^{\prime} }_{1}+{V^{\prime} }_{2}|{V^{\prime} }_{1}+{V^{\prime} }_{2}\rangle -\langle {V}_{1}+{V}_{2}|{V}_{1}+{V}_{2}\rangle =\,\mathrm{2(}\langle {V^{\prime} }_{1}|{V^{\prime} }_{2}\rangle -\langle {V}_{1}|{V}_{2}\rangle \mathrm{).}\end{array}$$
(3)

Thus maximising the VRMS improvement for a given |V1〉, |V2〉 means maximising \(\langle {V^{\prime} }_{1}|{V^{\prime} }_{2}\rangle \). Can we find a closed form expression for how high \(\langle {V^{\prime} }_{1}|{V^{\prime} }_{2}\rangle \) can be? Let us consider maximising it over permutation matrices and sign flips. Note firstly that making the signs the same for all entries, e.g. plus, cannot decrease \(\langle {V^{\prime} }_{1}|{V^{\prime} }_{2}\rangle \). We can assume that in the optimal case the signs are the same, say all positive. Now it is known that the dot product is maximised by ordering the entries of each in descending order: \(\langle {V^{\prime} }_{1}\downarrow |V^{\prime} {\downarrow }_{2}\rangle \). This follows from the rearrangement inequality22. Thus the maximum \(d{V}_{{\rm{RMS}}}^{2}\) one can obtain by signflips and permutations is

$$\begin{array}{c}ma{x}_{{\rm{sgnflip}}+{\rm{perms}}}\langle {V^{\prime} }_{1}+{V^{\prime} }_{2}|{V^{\prime} }_{1}+{V^{\prime} }_{2}\rangle \\ \,=\,\langle {V}_{1}|{V}_{1}\rangle +\langle {V}_{2}|{V}_{2}\rangle +2\langle {V}_{1}\downarrow |{V}_{2}\downarrow \rangle :\,=d{V}_{{\rm{RMS}}}^{(max\mathrm{)2}}\mathrm{.}\end{array}$$
(4)

Physically, phase shifts, which correspond to a restricted set of permutations (cyclic permutations) appear significantly more realistic than arbitrary permutations. We shall therefore in this paper consider phase shifts and sign-flips. This means that the optimum derived in Eq. 4 cannot necessarily be obtained, but one can hope to approach it.

Energy cost of these interventions arbitrarily small

If transformations take individual microstates to other microstates with the same energy they can in principle be performed without an energy cost. Moreover there needs to be a one-to-one mapping between microstates for there not to be a hidden energy cost owing to thermodynamics16. For example compressing a gas to half its volume isothermally does not change the (average) internal energy of the gas but nevertheless costs work, associated with the reduction of the entropy.

The interventions here, in idealised form, satisfy those conditions, whereas the diodes do not. The bias flip does not change the potential energy associated with the voltage difference. The phase shift is a delay, again not changing the potential energy. Moreover both operations are reversible, as can be seen physically, and from the fact that orthogonal matrices are reversible.

In contrast, the diode bridge is logically and thermodynamically irreversible with an inescapable lower power dissipation, as discussed above.

We are investigating what the energetic cost of the interventions will be in practise, taking hope from e.g.11 that it can be made low enough to be practical. We also remark here that the interventions are similar to a feedforward quantum neural net23 wherein the transformations are also reversible (unitary), providing another possible physical platform for these ideas.

Cost function used for the training

We wish to optimise the VRMS of the output, under the restriction that it should be DC, i.e. V > 0. This latter condition is because typically small energy harvesters need to produce DC, e.g. to charge a capacitor.

For a single voltage the cost function quantifying how far we are from only having positive voltage (POS) can be conveniently implemented as

$${C}_{POS}=4\langle |V|||V|\rangle -\langle \tilde{V}|\tilde{V}\rangle ,$$
(5)

where \(|\tilde{V}\rangle =||V|\rangle +|V\rangle \mathrm{.}\) One sees that if all entries are positive, C = 0, and otherwise C > 0.

Moreover we define

$${C}_{{V}_{{\rm{RMS}}}}=d{V}_{{\rm{RMS}}}^{{({\rm{\max }})}^{2}}-d{{V}_{{\rm{RMS}}}}^{2},$$
(6)

where \({V}_{{\rm{RMS}}}^{({\rm{\max }})}\) is the maximal over intelligent interventions of Eq. 4. For simplicity we define the total cost function, taking both desired properties into account, as

$$C={C}_{{V}_{{\rm{RMS}}}}+{C}_{POS}\mathrm{.}$$
(7)

A protocol we shall investigate here is where the phase shift is done on two individual voltages before combining them, followed by a joint sign flip. In this case we accordingly use the cost function

$$\begin{array}{rcl}C(\tau ,\varphi ) & = & {C}_{{V}_{{\rm{RMS}}}}+{C}_{POS}\\ & = & [\langle {V}_{1}\downarrow ||{V}_{2}\downarrow \rangle \,-\,\langle {V^{\prime} }_{1}|{V^{\prime} }_{2}\rangle ]+[4\langle |V^{\prime} ||V^{\prime} |\rangle -\langle \tilde{V^{\prime} }||\tilde{V^{\prime} }\rangle ],\end{array}$$
(8)

where |V′〉 is the sum of the two voltages after the first phase shift and \(|\tilde{V^{\prime} }\rangle =||V^{\prime} |\rangle +|V^{\prime} \rangle .\)

Systematic training methods exist

To have a systematic and scalable approach we consider the well-proven machine learning/optimisation technique of gradient descent on a suitably defined cost function. The gradient descent rule as applied here is that

$$(\begin{array}{c}\tau \\ \theta \end{array})\to (\begin{array}{c}\tau \\ \theta \end{array})-\eta (\begin{array}{c}\frac{C(\theta ,\tau +\delta )-C(\theta ,\tau )}{\delta }\\ \frac{C(\theta +\delta ,\tau )-C(\theta ,\tau )}{\delta }\end{array})$$
(9)

where δ and η are numerical parameters chosen according to what works.

Moreover, when faced with local minima in the cost function landscape, we employ the genetic algorithm, a type of evolutionary algorithm commonly used to find global minima when there are many local minima. In the genetic algorithm, the global minimum (highest fitness generation) can often be found, after operations like mutation, crossover and selection24.

The experimental data used was 16384 voltage readings taken at a very high frequency (8194 Hz) relative to the generator frequency (29 Hz). We used some of the time-series data (80%) for determining the optimal interventions, and then tested those interventions on the remaining data (20%).

The training used here can be classified as reinforcement learning, as the performance is evaluated (rather than the output being compared to a a known correct answer as in supervised learning).

Simulated intelligent intervention beats diode bridge

Our simulation shows that a combination of periodic voltage inversion and phase shift provides DC voltage that is higher than that after the diode bridge.

The diode bridge penalty of about 0.2 V is significant in regimes where VRMS is of the order of 0.2 or less. It is in these regimes it makes sense to consider replacing the diode bridge.

Figure 4 shows how the VRMS is left essentially undiminished and approximately non-negative by an intelligently chosen periodic voltage inversion, whereas the diode bridge loses about half of the VRMS. In regimes of even lower VRMS this advantage will be even greater of course.

Figure 4
figure 4

Intelligently chosen periodic voltage inversion gives better VRMS performance than the diode bridge for the same input. The diode bridge data is fully experimental. The intelligent intervention data is from applying the corresponding orthogonal matrix on the raw data experimental data before the diode bridge. VIF is the voltage after the intelligently chosen flip (experiment + simulated intervention), VRAW (experimental) is the direct output from the harvester, and VDB is that after the diode bridge (experimental).

In the case of two sub-generators we find that the simulated intelligently chosen delay plus intelligently chosen periodic inversion can also in principle significantly outperform the diode bridge, as shown in Fig. 5. An example of the cost function landscape for real experimental data combined with simulated intervention is in Fig. 6, showing local minima, which is why we employed the genetic algorithm for the training. The VRMS improvement here is given in Table 1.

Figure 5
figure 5

Two devices driven by the same source can have very similar frequency but be out of phase, as in this experimental data based on 2 piezo-electric harvesters. The phase difference leads to negative interference. The output of intelligent energy harvesting reduces the consumption of voltage and maximizes the RMS.

Figure 6
figure 6

Cost function landscape. X-axis is period and Y-axis is delay(length of phase shift), Z-axis is the cost function value, which combines a cost for being less than 0 and a cost for having suboptimal Vrms.

Table 1 Comparison of different methods in terms of VRMS and DC character of output achieved.

Signal-to-noise ratio important

We consider adjusting the power of additive Gaussian white noise to see the change of VRMS and C of the intelligent interventions and and diode bridge respectively.

We find that whilst the VRMS of the intelligent interventions is always greater than the diode bridge, when it comes to the total cost function C, it looks like in Fig. 7: there is a threshold signal-to-noise ratio SNR after which the diode bridge wins. This is consistent with the understanding that the intelligent intervention relies on patterns existing.

Figure 7
figure 7

Comparison of diode bridge and intelligent intervention under noise. The signal voltage series are raw experimental data and the noise is added numerically. There are three cases illustrating the corresponding parameter regime: (a) SNR(signal-to-noise ratio) is greater than 5, for which we find that the intelligent harvesting (IEH) has a better cost function performance, (b) SNR is close to 5, wherein the cost of IEH and the diode bridge (DB) are similar, and (c) SNR is less than 5, wherein we find the cost of the diode bridge is lower.

Conclusion

We have simulated interventions on experimental voltage data to investigate whether they could outperform the diode bridge in terms of DC voltage output. We find that in the regime of good signal-to-noise ratio and low voltages this is indeed the case. A reason the low-voltage regime is important is the continuing miniaturisation of e.g. piezo and tribo-electric harvesters. We conjecture that many natural scenarios give rise to good signal-to-noise ratio. This naturally gives rise to follow-on questions: (i) will experimental implementations of the interventions perform comparably to our idealised simulations? (ii) what natural power sources and harvesters do/do not give rise to voltages with good signal-to-noise ratios?