Quantum annealing using vacuum states as effective excited states of driven systems

Quantum annealing, which is particularly useful for combinatorial optimization, becomes more powerful by using excited states, in addition to ground states. However, such excited-state quantum annealing is prone to errors due to dissipation. Here we propose excited-state quantum annealing started with the most stable state, i.e., vacuum states. This counterintuitive approach becomes possible by using effective energy eigenstates of driven quantum systems. To demonstrate this concept, we use a network of Kerr-nonlinear parametric oscillators, where we can start excited-state quantum annealing with the vacuum state of the network by appropriately setting initial detuning frequencies for the oscillators. By numerical simulations of four oscillators, we show that the present approach can solve some hard instances whose optimal solutions cannot be obtained by standard ground-state quantum annealing because of energy-gap closing. In this approach, a nonadiabatic transition at an energy-gap closing point is rather utilized. We also show that this approach is robust against errors due to dissipation, as expected, compared to quantum annealing started with physical excited (i.e., nonvacuum) states. These results open new possibilities for quantum computation and driven quantum systems. For hard Quantum Annealing problems, starting from the ground state might result into high failure probability due to nonadiabatic transitions where the quantum system leaves the ground state. Here, the authors propose an "excited-state" quantum annealing where a driven system starts in its vacuum state set to an effective excited state, and show that this approach is successful in reducing the failure probability.

Q uantum annealing (QA) [1][2][3] , sometimes also called adiabatic quantum computation (AQC) [4][5][6] , is an alternative approach to quantum computation. QA is particularly useful for combinatorial optimization problems, where we have to minimize (or maximize) discrete-variable functions called objective (or cost) functions 7 . The Ising problem (search for ground states of Ising spin models) 8,9 is a typical example of combinatorial optimization problems. There are various situations requiring to solve combinatorial optimization problems which can be mapped to the Ising problem [10][11][12] , and hence QA machines designed for the Ising problem (Ising machines) [13][14][15] are expected to be useful for practical applications. (Another promising applications of QA machines are quantum simulations 16,17 . But here we focus on combinatorial optimization).
The idea of adiabatic QA (i.e., AQC) is simple. We start with the ground state of an initial Hamiltonian, where we know the ground state because the initial Hamiltonian is simple enough. Changing the Hamiltonian slowly to the one corresponding to the objective function for a given problem, we finally obtain the ground state of the final Hamiltonian assuming that the quantum adiabatic theorem 18 holds. The final ground state gives us the solution of the given problem. There is, however, an obvious issue. If the energy gap between the ground and first excited states almost closes during the adiabatic QA, the adiabatic theorem does not hold, and consequently we cannot find the ground state (optimal solution). Thus, the energy gap closing is a fatal issue for adiabatic QA.
An approach to this crucial issue is to use excited states during QA. For instance, we can achieve quantum speedups for certain kinds of problems 19,20 by using excited states via nonadiabatic transitions at energy-gap closing points during QA. It is also known that stoquastic AQC 6,21 , to which we can apply a classical simulation method 2 , becomes as powerful as universal quantum computation by using excited states 6,22 . That is, the use of excited states makes QA more powerful. The positive use of excited states in QA, where the initial state is intentionally set to an excited state, not the ground state, has been proposed 23 . However, this direction of research has not been pursued yet. This may be because this approach is accompanied by the problem that the initial state is prone to errors due to dissipation. (Another example of using excited states in QA is thermal QA 14 , where excited states are populated randomly by thermal noises. In contrast, in the above method 23 , an excited state is fully populated, which can be regarded as a thermal state at a negative temperature, like population inversion in lasers. In this sense, the thermal QA is quite different from the above method 23 ).
In this paper, we propose an approach to QA using excited states, which is referred to as excited-state QA in order to differentiate it from standard QA using ground states (ground-state QA). Our approach is based on the use of driven quantum systems. As a driven system, in this work we use a network of Kerrnonlinear parametric oscillators (KPOs). The concept of the KPO and quantum computations with KPOs, both adiabatic QA (AQC) and gate-based universal quantum computation, were proposed by Goto 24,25 , which were inspired by a more classical approach using optical parametric oscillators [26][27][28][29] . These proposals have been followed by interesting related works, such as superconducting-circuit implementations [30][31][32][33][34][35][36][37][38] , on-demand generation of Schrödinger cat states 39 , and theoretical studies as new driven quantum systems [40][41][42][43][44][45][46][47][48] . Also, Zhang and Dykman 49 proposed an approach to preparing quasienergy excited states of a KPO via quantum adiabatic evolution started with its vacuum state. This inspired us with the essential idea for our approach.
In our approach, excited-state QA is started with the most stable state, i.e., the vacuum state of a KPO network. This counterintuitive approach is possible, because QA with a KPO network uses effective energy eigenstates of the driven system, and we can set the vacuum state to an effective excited state of the network by appropriately setting detuning frequencies for KPOs. Our numerical simulations of four KPOs show that by this approach we can solve some hard instances accompanied by energy-gap closing, where a nonadiabatic transition at an energygap closing point is rather utilized. Our simulations also show that this approach is robust against errors due to dissipation. Thus new possibilities are opened for quantum computation and driven quantum systems.

Results and discussion
Ground-state QA with KPOs. Before presenting our proposed approach to excited-state QA, here we describe standard groundstate QA using a KPO network and show simulation results in order to clarify the energy-gap closing problem mentioned above.
The N-spin Ising problem with coupling coefficients {J i,j } and local fields {h i } is to find a spin configuration minimizing the (dimensionless) Ising energy defined as where s i is the ith spin taking 1 or −1, and the coupling coefficients satisfy J i,j = J j,i and J i,i = 0. The standard ground-state QA with a KPO network is as follows. To solve the Ising problem, we use a KPO network defined by the following Hamiltonian 24,26,42 : where ℏ is the reduced Planck constant, a i and a y i are the annihilation and creation operators, respectively, for the ith KPO, K is the Kerr coefficient, Δ i (t) is the detuning frequency for the ith KPO, p(t) is the parametric pump amplitude, and ξ(t) and A(t) are control parameters (ξ has the dimension of frequency and A is dimensionless). In this work, we assume time-dependent Δ i and ξ, unlike the literature 24,26,42 , for convenience of the extension to excited-state QA. We also assume that all the parameters are positive (if not mentioned). When K is negative, as in superconducting-circuit implementations, we set p, Δ i , and ξ to be negative. Then, we obtain the same results 26 .
The KPO defined above has recently been realized experimentally using superconducting circuits 34,36 . Although multiple coupled KPOs have not been realized yet, we can, in principle, realize a tunable coupling between superconducting KPOs, e.g., using four-wave mixing at a Josephson junction 50,51 . Thus the KPO network model given by Eq. (2) is experimentally feasible.
Note that the Hamiltonian in Eq. (2) is an effective one in a frame rotating at half the pump frequency and in the rotatingwave approximation 24,26 . Consequently, this approach uses an effective ground state of a KPO network for ground-state QA. This is contrast to standard QA using physical ground states [13][14][15] .
We increase p(t) from zero to a sufficiently large value p f (larger than K), decrease Δ i (t) from Δ ð0Þ i to zero, increase ξ(t) from zero to a small value ξ f (smaller than K), and set A(t) as . Then, the initial and final Hamiltonians, H 0 and H f , become where and a constant term, À_Kα 4 f =2, has been dropped in Eq. (4).
From Eq. (3), we find that the initial effective ground state is exactly the vacuum state. On the other hand, the first term of H f , which is dominant for small ξ f , has degenerate effective ground states expressed as tensor products of coherent states, ±α f j iÁ Á Á ±α f j i, with amplitudes of ±α f . (A coherent state α f j i is defined as an eigenstate of an annihilation operator a: a α f j i ¼ α f α f j i 52,53 ). Thus, assuming a sufficiently small ξ f compared to K, the final effective ground state is approximately given by where {s i = ±1} minimizes the following energy: This is proportional to the Ising energy in Eq. (1): Consequently, we can obtain the solution of the Ising problem from the final state of the adiabatic evolution started with the vacuum state, assuming that p(t) varies sufficiently slowly and the quantum adiabatic theorem holds.
To evaluate the ground-state QA with a KPO network, we solved 1000 random instances of the four-spin Ising problem, where {J i,j } and {h i } are randomly chosen from the interval (−1, 1) with uniform distribution, and normalized by the maximum value of their magnitudes. Note that even in such small-size instances, energy gaps can be much smaller than their average value, as mentioned below. (This is also the case for standard QA based on the transverse-field Ising model [1][2][3] . See Supplementary  Fig. 1 in Supplementary Note 1). Hence, by using these instances, we can mimic energy-gap closing situations and show the usefulness of our proposed approach. (Another reason for using small-size instances is the difficulty of simulating systems involving many KPOs).
We numerically solved the Schrödinger equation with the Hamiltonian in Eq. (2). In this work, we set the final pump amplitude as p f = 4K and the initial detuning frequencies as Δ  Figs. 1a, b, where the failure probability is the probability that we fail to obtain the optimal solution (ground state) of the Ising problem and the residual energy is the difference between the ground-state energy of the Ising problem and the expectation value of the Ising energy obtained by the ground-state QA. (The spin-configuration probabilities are calculated by the method in ref. 24 ). It is found that most instances are well solved by this ground-state QA. Here 1000 random instances of the four-spin Ising problem are solved. The coupling coefficients J i,j between the ith and jth spins and the local field h i on the ith spin are randomly chosen from the interval (−1, 1) with uniform distribution, and normalized by the maximum value of their magnitudes. For all 1000 random instances solved by each method, we measure the failure probability, that is, the probability that we fail to obtain the optimal solution (ground state) of the Ising problem, and the residual energy, i.e., the difference between the ground-state energy of the Ising problem and the expectation value of the Ising energy obtained by quantum annealing (QA). a and b show the results obtained by the ground-state QA with a network of Kerr-nonlinear parametric oscillators (KPOs). c and d show the results obtained by the proposed approach based on the excited-state QA with a KPO network.
To magnify bad results, we plot these results in a twodimensional plane, as shown in Fig. 2a. It turns out that the ground-state QA results in high failure probabilities or high residual energies in some instances. To examine the reason, we check the energy levels, i.e., the eigenvalues of the Hamiltonian in Eq. (2), in one of the worst instances indicated by a vertical arrow in Fig. 2a. This instance is the worst in the sense that the residual energy for this instance is the highest among the ten worst instances in terms of failure probability. (The hardness of this instance may come from its energy landscape shown in Fig. 3h, where there is a local minimum far from its global minimum). Figure 3a shows the energy levels in this instance (see "Methods" for calculation of the energies). We find that the energy gap between the effective ground and first excited states almost closes at the point indicated by a vertical arrow. Thus, the reason for the bad results in this instance is attributed to this energy-gap closing. That is, the system is in the effective ground state before the energy-gap closing point. At this point, however, a nonadiabatic transition to the effective first excited state occurs. Consequently, we cannot obtain the effective ground state at the end. This time evolution is depicted by dotted arrows in Fig. 3a. This argument is confirmed by the time evolutions of the populations of the effective ground and first excited states shown in Fig. 3b. Figure 4 shows the cumulative distribution of minimum energy gaps during the ground-state QA normalized by their average value for the 1000 instances. It is found that there are instances with very small energy gaps, as mentioned above. This suggests that the energy-gap closing problem arises not only in the instance discussed above, but also in some other instances (see Supplementary Fig. 1 in Supplementary Note 1 for a similar result for standard QA based on the transverse-field Ising model). Excited-state QA with KPOs. Here we present our proposed approach. To set the vacuum state to the effective first excited state of a KPO network, we set one of the initial detuning frequencies, e.g. Δ 1 , to a negative value. Then, one-photon and two-photon energies for the KPO at the initial time are expressed, respectively, as _Δ ð0Þ 1 and _ðK þ 2Δ ð0Þ 1 Þ. Thus, the vacuum state is the effective first excited state when ÀK=2 < Δ ð0Þ 1 < 0. In the hard instance discussed above, a negative initial detuning, Δ  Fig. 3c, we expect to obtain the effective ground state via a nonadiabatic transition from the effective first excited state to the effective ground state at an energy-gap closing point, as depicted by dotted arrows in Fig. 3c. As shown in Fig 3d, the populations of the effective ground and first excited states actually interchange at the energygap closing point. Our simulation also shows that the failure probability and the residual energy for this instance are improved from 0.963 and 0.171 to 7.10 × 10 −4 and 6.87 × 10 −4 , respectively, by the excited-state QA started with the vacuum state.
Note that the excited-state QA does not always succeed. Our proposal is to try the ground-state QA and the excited-state QA with a negative Δ ð0Þ i (i = 1, …, N) and to select the best result among the (N + 1) cases. By this approach, the results for the 1000 random instances are dramatically improved from Figs. 1a, b and 2a to Figs. 1c, d, and 2b, respectively. In particular, the failure probabilities for the ten worst instances are substantially improved, as shown in Table 1. (The instance discussed above is the tenth one in Table 1 For comparison, we also evaluate two naive approaches using ground-state QA with (N + 1)-time larger computational costs. In the first approach, we iterate ground-state QA (N + 1) times. In the second one, we perform ground-state QA once but with (N + 1) times longer computation time. The results by these approaches for the 1000 instances are shown in Figs. 2c and d, respectively. Unlike the proposed approach, the improvements by these naive approaches are quite modest. In particular, high failure probabilities are almost unimproved. This can be understood as follows. For example, in the case of the worst instance in Table 1, we need log 0:000857=log 0:999959 % 1:7 10 5 times of iteration on average to achieve the low failure probability obtained by the proposed approach. On the other hand, the minimum energy gap in this instance is about 5 × 10 −5 times smaller than the average value of the energy gaps, which suggests that we need 2 × 10 4 times longer computation time to achieve average performance. These results also indicate the advantage of the proposed approach. Note, however, that the proposed approach is useful only when its quantum advantage over classical approaches is over O(N).
The instance indicated by a vertical arrow in Fig. 2b has still high failure probability. To examine this reason, we show the energy levels and the time evolutions of populations in this instance in Figs. 5a and b, respectively. In this case, the minimum energy gap is fairly large, and consequently the final ground-state population reaches 0.864. However, its failure probability is as high as 0.864, which almost exactly corresponds to the groundstate population. This implies that the mapping of the optimal solution of the Ising problem to the ground state of the KPO network may be wrong.
We noticed that final photon numbers in KPOs deviated from our design value, α 2 f ¼ p f =K ¼ 4, as shown in Fig. 5c, and the deviations lead to the wrong mapping through the change of the ratios between the terms in Eq. (5). To resolve this mapping issue, we modulate the pump amplitudes as p i (t) = p(t) × (4/n i ) in order to cancel the deviations, where p is the original pump amplitude, p i is the new pump amplitude for the ith KPO, and n i is the final average photon number for the ith KPO with the original pump amplitude p. The final average photon numbers in this case are shown in Fig. 5f, where the deviations are reduced as expected. The energy levels and the time evolutions of populations in this case are shown in Figs. 5d and e, respectively. We can successfully obtain the ground state. Unlike the original pump amplitude, in this case the Ising mapping is correct and the failure probability is reduced from 0.864 to 1.81 × 10 −4 . Thus the control of individual pump amplitudes to reduce photon-number deviations also improves the performance of QA with a KPO network. (Some methods for reduction of deviations have been proposed for a KPO network 54 and other systems 55,56 ).
We applied this control to other instances and obtained the result shown in Fig. 2e. This is not very different from the original one in Fig. 2a. Selecting the best result from the results in Figs. 2b and e for each instance, however, we obtain the good result shown in Fig. 2f, where all bad results are improved. This suggests that bad results in ground-state QA with a KPO network are due to either energy-gap closing or wrong Ising mapping induced by photon-number deviations. Table 1 Comparison between failure probabilities of groundstate quantum annealing and our proposed approach based on excited-state quantum annealing in terms of the ten worst instances.  Robustness against dissipation. In the proposed excited-state QA, we do not need additional quantum operations, namely, actual excitation to physical excited states. We can start with vacuum states, as in the standard ground-state QA with a KPO network. This is an advantage of our proposed approach. Another advantage of this approach is the robustness of the initial state against errors due to dissipation. To demonstrate it, we evaluated the ground-state and excited-state QAs in the presence of dissipation. For comparison, we also evaluated excited-state QA started with a physical excited state, where one of the initial detuning frequencies is set to K/4 and the KPO is initially set in its one-photon state (see Supplementary Note 2 for the details of these simulations). In the absence of dissipation, this excited-state QA can successfully solve the hard instance in Fig. 3 by utilizing a nonadiabatic transition from the effective first excited state to the effective ground state, as shown in Figs. 3e, f. Figure 3g shows the results in the presence of dissipation. (Success probability is one minus failure probability). Here we assume a field decay rate κ smaller than 0.01K. Such κ is experimentally feasible. (The recently implemented KPO has K/(2π) = 6.6 MHz and a photon lifetime of 15.5 μs 36 , which means that κ/K ≈ 8 × 10 −4 ). Our proposed excited-state QA started with vacuum states is more robust against dissipation than the excited-state QA started with a one-photon state. Interestingly, the performance of the ground-state QA is a little improved by dissipation. This is due to excitation by quantum heating 42 , as in thermal QA 14 . Figure 6 shows the comparison between the results for our proposed excited QA started with the vacuum state and the excited-state QA started with a one-photon state in the presence of dissipation in the ten instances in Table 1. In all the hard instances, our proposed approach is more robust against dissipation than the excited-state QA started with a one-photon state.

Conclusion
We have proposed an approach to excited-state QA, which is started with the most stable state, namely, vacuum states. This is based on the use of effective energy eigenstates of driven quantum systems. As such a driven system, we have used a KPO network, which enables to set the vacuum state to an effective excited state by appropriately setting detuning frequencies for KPOs. Hard instances whose optimal solutions cannot be obtained by groundstate QA because of energy-gap closing can be solved by the excited-state QA, where a nonadiabatic transition from the effective first excited state to the effective ground state at an energy-gap closing point is exploited. Since the excited-state QA is started with vacuum states, this QA is robust against errors due to dissipation, which has been confirmed by numerical simulations. Thus, the proposed approach enhances the power of QA with a KPO network and, in particular, offers a new way for tackling the energy-gap closing problem by harnessing a property of driven quantum systems. In this work, we have used only four KPOs in simulations. Thus it is left for an interesting future work to check whether the proposed approach also has the advantage for more KPOs.

Methods
Numerical calculation of energies. The size of the Hamiltonian matrix in Eq. (2) (15 4 × 15 4 in this work) is too large to directly diagonalize. So instead, we calculated the eigenvalues of the Hamiltonian, namely, the energies as follows.
We first numerically obtain the eigenvectors for each KPO by diagonalizing each term in the first term of H(t) in Eq. (2). Taking N e eigenvectors from low energies as a basis, we obtain a N 4 e N 4 e matrix representation of H(t). Finally, we diagonalize this matrix and obtain the energies. To reduce the computational costs, we set N e = 6 in the present calculations. This approach based on the lowenergy approximation is valid when ξ is small compared to K, as in the present case.

Data availability
The data that support the findings of this study are present in the paper and/or Supplementary Information. Additional data are available from the corresponding author upon reasonable request.  Fig. 6 Success probabilities of excited-state quantum annealing for the ten worst instances of ground-state quantum annealing. The ten instances are characterized in Table 1 and Fig. 3 together with Supplementary Figs. 2-10. Each bar represents the success probability for each of the ten instances without (κ = 0) and with (κ = 0.005K) dissipation, where κ is a field decay rate for each Kerr-nonlinear parametric oscillator. The instances are ordered in descending order with respect to the failure probability of ground-state quantum annealing (QA), as in Table 1, from left. a Results obtained by the proposed excited-state QA started with vacuum states. b Results obtained by excited-state QA started with a one-photon state.