Explainable AI via learning to optimize

Indecipherable black boxes are common in machine learning (ML), but applications increasingly require explainable artificial intelligence (XAI). The core of XAI is to establish transparent and interpretable data-driven algorithms. This work provides concrete tools for XAI in situations where prior knowledge must be encoded and untrustworthy inferences flagged. We use the “learn to optimize” (L2O) methodology wherein each inference solves a data-driven optimization problem. Our L2O models are straightforward to implement, directly encode prior knowledge, and yield theoretical guarantees (e.g. satisfaction of constraints). We also propose use of interpretable certificates to verify whether model inferences are trustworthy. Numerical examples are provided in the applications of dictionary-based signal recovery, CT imaging, and arbitrage trading of cryptoassets. Code and additional documentation can be found at https://xai-l2o.research.typal.academy.

A paradigm shift in machine learning is to construct explainable and transparent models, often called explainable AI (XAI) [65].This is crucial for sensitive applications like medical imaging and finance (e.g.see recent work on the role of explainability [1,9,24,61]).Yet, many commonplace models (e.g.fully connected feed forward) offer limited interpretability.Prior XAI works give explanations via tools like sensitivity analysis [61] and layer-wise propagation [10,49], but these neither quantify trustworthiness nor necessarily shed light on how to correct "bad" behaviours.Our work shows how learning to optimize (L2O) can be used to directly embed explainability into models.
The scope of this work is machine learning (ML) applications where domain experts can create approximate models by hand.In our setting, the inference N Θ (d) of a model N Θ with input d solves an optimization problem.That is, we use where f Θ is a function and C Θ (d) ⊆ R n is a constraint set (e.g.encoding prior information like physical quantities), and each (possibly) includes dependencies on weights Θ.Note the model N Θ is implicit since its output is defined by an optimality condition rather than an explicit computation.thi swork, we adopt the following conventions.We say a model is explainable provided a domain expert can identify the core design elements of a model and how they translated to expected inference properties.We say a particular inference is explainable provided its properties can be linked to the model's design and intended use.
Explainable models and inferences are achieved via L2O with our proposed certificates.
A standard practice in software engineering is to code post-conditions after function calls return.Post-conditions are criteria used to validate what the user expects from the code and ensure code is not executed under the wrong assumptions.[4] We propose use of these for ML model inferences (see Figures 1 and 10).These conditions enable use of certificates with labels -pass, warning or fail -to describe each model inference.We define an inference to be trustworthy provided it satisfies all provided post-conditions.
Two ideas, optimization and certificates, form a concrete notion of XAI.Prior and data-driven knowledge can be encoded via optimization, and this encoding can be verified via certificates (see Figure 5).To illustrate, consider inquiring why a model generated a "bad" inference (e.g. an inference disagrees with observed measurements).
The first diagnostic step is to check certificates.If no fails occurred, the model was not designed to handle the instance encountered.In this case, the model in (1) can be redesigned to encode prior knowledge of the situation.Alternatively, each failed certificate shows a type of error and often corresponds to portions of the model (see Figures 1 and 2).The L2O model allows debugging of algorithmic implementations and assumptions to correct errors.In a sense, this setup enables one to manually backpropagate errors to fix models (similar to training).

Related Works
Closely related to our work is deep unrolling, a subset of L2O wherein models consist of a fixed number of iterations of a data-driven optimization algorithm.Deep unrolling has garnered great success and provides intuitive model design.We refer readers to recent surveys [3,18,48,63] for further L2O background.Downsides of unrolling are growing memory requirements with unrolling depth and a lack of guarantees.
Implicit models circumvent these two shortcomings by defining models using an equation (e.g. as in (1)) rather than prescribe a fixed number of computations as in deep unrolling.This enables inferences to be computed by iterating until convergence, thereby enabling theoretical guarantees.Memory-efficient training techniques were also developed for this class of models, which have been applied successfully in games, [33] music source separation, [40] language modeling, [11], segmentation, [12] and inverse problems.[30,32] The recent work [30] most closely aligns with our L2O methodology.
Related XAI works use labels/cards.Model Cards [47] document intended and appropriate uses of models.Care labels [50,51] are similar, testing properties like expressivity, runtime, and memory usage.FactSheets [8] are modeled after supplier declarations of conformity and aim to identify models' intended use, performance, safety, and security.These works provide statistics at the distribution level, complementing our work for trustworthiness of individual inferences.

Explainability via Optimization
Model Design The design of L2O models is naturally decomposed into two steps: optimization formulation and algorithm choice.The first step is to identify a tentative objective to encode prior knowledge via regularization (e.g.sparsity) or constraints (e.g. unit simplex for classification).We may also add terms that are entirely data-driven.Informally, this step identifies a special case of (1) of the form where the constraints are encoded in the objective using indicator functions, equaling 0 when constraint is satisfied and ∞ otherwise.The second design step is to choose an algorithm for solving the chosen optimization problem (e.g.proximal-gradient or ADMM [23]).We use iterative algorithms, and the update formula for each iteration is given by a model operator T Θ (x; d).Updates are typically composed in terms of gradient and proximal operations.Some parameters (e.g.step sizes) may be included in the weights Θ to be tuned during training.Given data d, computation of the inference N Θ (d) is completed by generating a sequence {x k d } via the relation By design, {x k d } converges to a solution of (1), and we set In our context, each model inference N Θ (d) is defined to be an optimizer as in (1).Hence properties of inferences can be explained via the optimization model (1); note this is unlike blackbox models where one has no way of explaining why a particular inference is made.The iterative algorithm is applied successively until stopping criteria are met (i.e. in practice we choose an iterate K, possibly dependent on d, so that N Θ (d) ≈ x K d ).Because {x k d } converges, we may adjust stopping criteria to approximate the limit to arbitrary precision, which implies we may provide guarantees on model inferences (e.g.satisfying a linear system of equations to a desired precision [30,32,33]).The properties of the implicit L2O model (1) are summarized by Table 1.
Example of Model Design.To make the model design procedure concrete, we illustrate this process on a classic problem: sparse recovery from linear measurements.These problems appear in many applications such as radar imaging [64] and speech recognition [28].Here the task is to estimate a signal x Figure 6) along with certificates.Ground truth was taken from test dataset of implicit dictionary experiment.
The second from left is sparse and inconsistent with measurement data.The second from right complies with measurements but is not sparse.The rightmost is generated using our proposed model (IDM), which approximates the ground truth well and is trustworthy.
Step 1: Choose Model.Since true signals are known to be sparse, we include ℓ 1 regularization.To comply with measurements, we add a fidelity term.Lastly, to capture hidden features of the data distribution, we also add a data-driven regularization.Putting these together gives the problem where τ > 0 and W 1 and W 2 are two tunable matrices.This model encodes a balance of three terms -sparsity, fidelity, data-driven regularization -each quantifiable via (5).
Step 2: Choose Algorithm.The proximal-gradient scheme generates a sequence {z k } converging to a limit which solves (5).By simplifying and combining terms, the proximal-gradient method can be written via the iteration where λ > 0 is a step-size, W is a matrix defined in terms of W 1 , W 2 , and A ⊤ , and η θ is the shrink operator given by From the update on the right hand side of (6), we see the step size λ can be "absorbed" into the tunable matrix W and the shrink function parameter can be set to θ > 0. That is, this example model has weights Θ = (W, θ, τ ) with model operator which resembles the updates of previous L2O works.[19,31,45] Inferences are computed via a sequence {x k d } with updates The model inference is the limit Convergence Evaluation of the model N Θ (d) is well-defined and tractable under a simple assumption.By a classic result [41], it suffices to ensure, for all d, T Θ (•; d) is averaged, i.e. there is α ∈ (0, 1) and Q such that When this property holds, the sequence {x k d } in (3) converges to a solution x ⋆ d .This may appear to be a strong assumption; however, common operations in convex optimization algorithms (e.g.proximals and gradient descent updates) are averaged.For entirely data-driven portions of T Θ , several activation functions are 1-Lipschitz [20,27] (e.g.ReLU and softmax), and libraries like PyTorch [53] include functionality to force affine mappings to be 1-Lipschitz (e.g.spectral normalization).Furthermore, by making T Θ (•; d) a contraction, a unique fixed point is obtained.We emphasize, even without forcing T Θ to be averaged, {x k } is often observed to converge in practice [11,30,32] upon tuning weights Θ.

Concept
Quantity Formula Sparsity Nonzeros ∥x∥ 0 Table 2: Each certificate is tied to a high-level concept, and its corresponding property value is quantified using a formula.For classifier confidence, we assume x is in the unit simplex.The proximal is a data-driven update for f Ω with weights Ω.

Trustworthiness Certificates
Explainable models justify whether each inference is trustworthy.We propose providing justification in the form of certificates, which verify various properties of the inference are consistent with those of the model inferences on training data and/or prior knowledge.Each certificate is a tuple of the form (name, label) with a property name and a corresponding label which has one of three values: pass, warning, or fail (see Figure 3).Each certificate label is generated by two steps.The first is to apply a function that maps inferences (or intermediate states) to a nonnegative scalar value α quantifying a property of interest.The second step is to map this scalar to a label.Labels are generated via the flow: Property Value Functions Several quantities may be used to generate certificates.In the model design example above, a sparsity property can be quantified by counting the number of nonzero entries in a signal, and a fidelity property can use the relative error ∥Ax − d∥/∥d∥ (see Figure 3).To be most effective, property values are chosen to coincide with the optimization problem used to design the L2O model, i.e. to quantify structure of prior and data-driven knowledge.This enables each certificate to clearly validate a portion of the model (see Figure 2).Since various concepts are useful for different types of modeling, we provide a brief (and non-comprehensive) list of concepts and possible corresponding property values in Table 2.
One property concept deserves particular attention: data-driven regularization.This regularization is important for discriminating between inference features that are qualitatively intuitive but difficult to quantify by hand.Rather than approximate a function, implicit L2O models directly approximate gradients/proximals.These provide a way to measure regularization indirectly via gradient norms/residual norms of proximals.Moreover, these norms (e.g.see last row of Table 2) are easy to compute and equal zero only at local minima of regularizers.
To our knowledge, this is the first work to quantify trustworthiness using the quality of inferences with respect to data-driven regularization.
Certificate Labels Typical certificate labels should follow a trend where inferences often obtain a pass label to indicate trustworthiness while warnings occur occasionally and failures are obtained in extreme situations.Let the samples of model inference property values α ∈ [0, ∞) come from distribution P A .We pick property value functions for which small α values are desirable and the distribution tail consists of larger α.Intuitively, smaller property values of α resemble property values of inferences from training and/or test data.Thus, labels are assigned according to the probability of observing a value less than or equal to α, i.e. we evaluate the cumulative distribution function (CDF) defined for probability measure P A by Labels are chosen according to the task at hand.Let p p , p w , and p f = 1 − p p − p w be the probabilities for pass, warning, and fail labels, respectively.Labels are made for α via

Prop Value
The remaining task is to estimate the CDF value for a given α.Recall we assume access is given to property values {α i } N i=1 from ground truths or inferences on training data, where N is the number of data points.To this end, given an α value, we estimate its CDF value via the empirical CDF: where | • | denotes set cardinality.For large N, (13) well approximates the continuous CDF.
Certificate Implementation As noted in the introduction, trustworthiness certificates are evidence an inference satisfies post-conditions (i.e.passes various tests).Thus, they are to be used in code in the same manner as standard software engineering practice.Consider the snippet of code in Figure 10.As usual, an inference is generated by calling the model.However, alongside the inference x, certificates certs are returned that label whether the inference x passes tests that identify consistency with training data and prior knowledge.

Experiments
Each numerical experiment shows an application of novel implicit L2O models, which were designed directly from prior knowledge.Associated certificates of trustworthiness are used to emphasize the explainability of each model and illustrate use-cases of certificates.Experiments were coded using Python with the PyTorch library [53], the Adam optimizer [39], and, for ease of re-use, were run via Google Colab.We emphasize these experiments are for illustration of intuitive and novel model design and trustworthiness and are not benchmarked against state-of-the-art models.The datasets generated and/or analysed during the current study are available in the following repository: github.com/typal-research/xai-l2o.All methods were performed in accordance with the relevant guidelines and regulations.

Algorithms
To illustrate evaluation of L2O model used herein, we begin with an example L2O model and algorithm.Specifically, models used for the first two experiments take the form where K and M are linear operators, δ ≥ 0 is a noise tolerance, and f and h are proximable2 functions.Introducing auxiliary variables w and p and dual variable ν = (ν 1 , ν 2 ), linearized ADMM [60] (L-ADMM) can be used to iteratively update the tuple (p, w , ν, x) of variables via where proj B(d,δ) is the Euclidean projection onto the Euclidean ball of radius δ centered at d, prox f is the proximal operator for a function f , and the scalars α, β, λ > 0 are appropriate step sizes.Further details, definitions, and explanations are available in the appendices.Note the updates are ordered so that x k+1 is the final step to make it easy to backprop through the final x k update.

Implicit Model Training
Standard backpropagation cannot be used for implicit models as it requires memory capacities beyond existing computing devices.Indeed, storing gradient data for each iteration in the forward propagation (see (3)) scales the memory during training linearly with respect to the number of iterations.Since the limit x ∞ solves a fixed point equation, implicit models can be trained by differentiating implicitly through the fixed point to obtain a gradient.This implicit differentiation requires further computations and coding.Instead of using gradients, we utilize Jacobian-Free Backpropagation (JFB) [26] to train models.JFB further simplifies training by only backpropagating through the final iteration, which was proven to yield preconditioned gradients.JFB trains using fixed memory (with respect to the K steps used to estimate N Θ (d)) and avoids numerical issues arising from computing exact gradients, [13] making JFB and its variations [29,35] apt for training implicit models.Model Design All convex regularization approaches are known lead to biased estimators whose expectation does not equal the true signal.[25] However, the seminal work [15] of Candes and Tao shows ℓ 1 minimization (rather than additive regularization) enables exact recovery under suitable assumptions.Thus, we minimize a sparsified signal subject to linear constraints via the implicit dictionary model (IDM)

Implicit Dictionary
The square matrix K is used to leverage the fact x has a low-dimensional representation by transforming x into a sparse vector.Linearized ADMM [60] (L-ADMM) is used to create a sequence {x k d } as in (3).The model N Θ has weights Θ = K.If it exists, the matrix K −1 is known as a dictionary and KN Θ (d) is the corresponding sparse code; hence the name IDM for (16).To this end, we emphasize K is learned during training and is different from M, but these matrices are related since we aim for the product Kx ⋆ d = KMs ⋆ d to be sparse.Note we use L-ADMM to provably solve (16), and N Θ is easy to train.More details can be found in the appendix.
Discussion IDM combines intuition from dictionary learning with a reconstruction algorithm.Two properties are used to identify trustworthy inferences: sparsity and measurement compliance (i.e.fidelity).Sparsity and fidelity are quantified via the ℓ 1 norm of the sparsified inference (i.e.KN Θ (d)) and relative measurement error.Figure 6 shows the training the model yields a sparsifying transformation K. Figure 3 shows the proposed certificates identify "bad" inferences that might, at first glance, appear to be "good" due to their compatibility with constraints.Lastly, observe the utility of learning K, rather than approximating M, is K makes it is easy to check if an inference admits a sparse representation.Using M to check for sparsity is nontrivial.

Model Design
The model for the CT experiment extends the IDM.In practice, it has been helpful to utilize a sparsifying transform.[36,68] We accomplish this via a linear operator K, which is applied and then this product is fed into a data-driven regularizer f Ω with parameters Ω.We additionally ensure compliance with measurements from the Radon transform matrix A, up to a tolerance δ.In our setting, all pixel values are also known to be in the interval [0, 1].Combining our prior knowledge yields the implicit L2O model Here N Θ has weights Θ = (Ω, K, α, β, λ) with α, β and λ step-sizes in L-ADMM.More details can be found in the appendix.
Discussion Comparisons of our method (Implicit L2O) with U-Net, [37] F-FPNs, [32] and total variation (TV) Minimization are given in Figure 7 and Table 3. Table 3 shows the average PSNR and SSIM reconstructions.Our model obtains the highest average PSNR and SSIM values on the test data while using 11% and 62% as many weights as U-Net and F-FFPN, indicating greater efficiency of the implicit L2O framework.Moreover, the L2O model is designed with three features: compliance with measurements (i.e.fidelity), valid pixel values, and data-driven regularization.Table 3 also shows the percentage of "fail" labels for these property values.Here, an inference fails if its property value is larger than 95% of the property values from the training/true data, i.e. we choose p p = 0.95, p w = 0, and p f = 0.05 in (12).For the fidelity, our model never fails (due to incorporating the constraint into the network design).Our network fails 5.7% of the time for the data-driven regularization property.Overall, the L2O model generates the most trustworthy inferences.This is intuitive as this model outperforms the others and was specifically designed to embed all of our knowledge, unlike the others.To provide better intuition of the certificates, we also show the certificate labels for an image from the test dataset in Figure 7.The only image to pass all provided tests is the proposed implicit L2O model.This knowledge can help identify trustworthy inferences.Interestingly, the data-driven regularization enabled certificates to detect and flag "bad" TV Minimization features (e.g.visible staircasing effects [17,58]), which shows novelty of certificates as these features are intuitive, yet prior methods to quantify this were, to our knowledge, unknown.

CFMMs Tokens
Figure 8: Network with 5 CFMMs and 3 tokens; structure replicates an experiment in recent work.[7] Black lines show available tokens for trade in each CFMM.

Optimal Cryptoasset Trading
Setup Ethereum is a blockchain technology anyone can use to deploy permanent and immutable decentralized applications.This technology enables creation of decentralized finance (DeFi) primitives, which can give censorship-resistant participation in digital markets and expand the use of stable assets [38,56] and exchanges [34,66,69] beyond the realm of traditional finance.Popularity of cryptoasset trading (e.g.GRT and Ether) is exploding with the DeFi movement.[62,67] Decentralized exchanges (DEXs) are a popular entity for exchanging cryptoassets (subject to a small transaction fee), where trades are conducted without the need for a trusted intermediary to facilitate the exchange.Popular examples of DEXs are constant function market makers (CFMMs), [5] which use mathematical formulas to govern trades.To ensure CFMMs maintain sufficient net assets, trades within CFMMs maintain constant total reserves (as defined by a function φ).A transaction in a CFMM tendering x assets in return for y assets with reserves assets r is accepted provided with γ ∈ (0, 1] a trade fee parameter.Here r, x, y ∈ R n with each vector nonnegative and i -th entry giving an amount for the i -th cryptoasset type (e.g.Ether, GRT).Typical choices [6] of φ are weighted sums and products, i.e.
where w ∈ R n has positive entries.
This experiment aims to maximize arbitrage.Arbitrage is the simultaneous purchase and sale of equivalent assets in multiple markets to exploit price discrepancies between the markets.This can be a lucrative endeavor with cryptoassets.[46] For a given snapshot in time, our arbitrage goal is to identify a collection of trades that maximize the cryptoassets obtainable by trading between different exchanges, i.e. solve the (informal) optimization problem The set of valid trades is all trades satisfying the transaction rules for CFMMs given by (18) with nonnegative values for tokens tendered and received (i.e.x, y ≥ 0).Prior works [6,7] deal with an idealistic noiseless setting while recognizing executing trades is not without risk (e.g.noisy information, front running, [21] and trade delays).To show implications of trade risk, we incorporate noise in our trade simulations by adding noise ε ∈ R n to CFMM asset observations, which yields noisy observed data d = (1 + ε) ⊙ r .Also, we consider trades with CFMMs where several assets can be traded simultaneously rather than restricting to pairwise swaps.

Model Design
The aim is to create a model that infers a trade (x, y ) maximizing utility.For a nonnegative vector p ∈ R n of reference price valuations, this utility U is the net change in asset values provided by the trade, i.e.Table 4: Averaged results on test data for trades in CFMM network.The analytic method always predicts a profitable trade, but fails to satisfy the constraints (due to noise).This failure is predicted by the certificates "risk" certificate and reflected by the 0% trade execution.Alternatively, the L2O scheme makes conservative predictions regarding constraints, which limits profitability.However, using these certificates, executed L2O trades are always profitable and satisfy constraints.The analytic trade is unable to account for trade risks, causing it to propose large trades that are not executed (giving executed utility of zero).This can be anticipated by the failed trade risk certificate.On the other hand, the L2O scheme is profitable (utility is 0.434) and is executed (consistent with the pass trade risk label).
where A j is a matrix mapping global coordinates of asset vector to the coordinates of the j-th CFMM (see appendix for details).For noisy data d, trade predictions can include a "cost of risk."This is quantified by regularizing the trade utility, i.e. introducing a penalty term.For matrices W j , we model risk by a simple quadratic penalty via risk model (22) The implicit L2O model infers optimal trades via U Θ , i.e.
where C Θ (d) encodes constraints for valid transactions.The essence of N Θ is to output solutions to (20) that account for transaction risks.A formulation of Davis-Yin operator splitting [22] is used for model evaluation.
Further details of the optimization scheme can be found in the appendix.
Discussion The L2O model contains three core features: profit, risk, and trade constraints.The model is designed to output trades that satisfy provided constraints, but note these are noisy and thus cannot be used to a priori determine whether a trade will be executed.For this reason, fail flags identify conditions to warn a trader when a trade should be aborted (due to an "invalid trade").This can avoid wasting transaction fees (i.e.gas costs).Figure 9 shows an example of two trades, where we note the analytic method proposes a large trade that is not executed since it violates the trade constraints (due to noisy observations).The L2O method proposes a small trade that yielded arbitrage profits (i.e.U > 0) and has pass certificates.Comparisons are provided in Table 4 between the analytic and L2O models.Although the analytic method has "ideal" structure, it performs much worse than the L2O scheme.In particular, no trades are executable by the analytic scheme since the present noise always makes the proposed transactions fail to satisfy the actual CFMM constraints.Consistent with this, every proposed trade by the analytic trade is flagged as risky in Table 4.The noise is on the order of 0.2% Gaussian noise of the asset totals.

Conclusions
Explainable ML models can be concretely developed by fusing certificates with the L2O methodology.The implicit L2O methodology enables prior and data-driven knowledge to be directly embedded into models, thereby providing clear and intuitive design.This approach is theoretically sound and compatible with state-of-the-art ML tools.The L2O model also enables construction of our certificate framework with easy-to-read labels, certifying if each inference is trustworthy.In particular, our certificates provide a principled scheme for the detection of inferences with "bad" features via data-driven regularization.Thanks to this optimization-based model design (where inferences can be defined by fixed point conditions), failed certificates can be used to discard untrustworthy inferences and may help debugging the architecture.This reveals the interwoven nature of pairing implicit L2O with certificates.Our experiments illustrate these ideas in three different settings, presenting novel model designs and interpretable results.Future work will study extensions to physics-based applications where PDE-based physics can be integrated into the model [44,57,59].

B Linearized ADMM Formulation
Two of the numerical examples utilize variations of ADMM.This section is dedicated to a derivation of linearized ADMM used to solve problems of the form where K and M are linear operators, δ > 0 is a noise tolerance, and f and h are proximable.First, we define the proximal operator for a closed, convex, and proper function by Letting δ C be the indicator function for a closed and convex set C with value 0 for x ∈ C and ∞ otherwise, the Euclidean projection proj C on C is a special case of the proximal, i.e.

E Supplement for Cryptoasset Trades
This section is broken into three parts.The geometric constraint sets are of particular importance to handle in a decoupled fashion, and so the first subsection is dedicated to handling CFMM constraints for batches of transactions.Note closed form expressions exist for pairwise swaps.The second subsection then identifies the projection operations needed.This is followed by a derivation of a particular operator splitting used to solve the problem, giving explicit lists for updates.

E.1 Constraint Formulation
The dimension of the vector space for each CFMM may differ since some exchanges might not provide access to particular cryptoassets.Consequently, we follow similarly to recent work [7] in using matrices A j ∈ R n j ×n to convert global coordinates into the local coordinates of the j-th CFMM, i.e.
where δ j ≥ 0 is a tolerance, n j is the number of asset types in the j-th CFMM, w j ∈ R n j is a positive weighting, and The set A j identifies a weighted geometric mean inequality that must hold for the j-th CFMM.We include the nonnegative δ j to account for noisy data.Choosing δ j > 0 gives a buffer for ensuring a transaction is still valid for noisy d (at the cost of reducing the achievable utility U).
Remark 1. Ideally, we would directly compute PA j (x) in an algorithm computing optimal trades, which can be derived following an example in Beck's text.[14] However, this projection introduces unscalable coupling since, using δ j = 0 and d = r , where λ > 0 is a solution to As the number of asset types in CFMMs increase, the time of a root finding algorithm to estimate λ also increases.Our alternative approach avoids this scaling issue.We also note JFB would technically require backpropping through the root finding scheme, but we suspect this could be avoided (by not attaching gradients during root finding) without adverse results.
We decouple the constraint A j by defining the hyperplane H j ≜ z : w j , z = ln(α j ) .(46)

Figure 1 :
Figure 1: The L2O model is composed of parts (shown as colored blocks) based on prior knowledge or data.L2O inferences solve the optimization problem for given model inputs.Certificates label if each inference is consistent with training.If so, it is trustworthy; otherwise, the faulty model part errs.

Figure 3 :
Figure 3: Example inferences for test data d.The sparsified version Kx of each inference x is shown (c.f.Figure6) along with certificates.Ground truth was taken from test dataset of implicit dictionary experiment.The second from left is sparse and inconsistent with measurement data.The second from right complies with measurements but is not sparse.The rightmost is generated using our proposed model (IDM), which approximates the ground truth well and is trustworthy.

Figure 4 :Figure 5 :
Figure 4: Probability distribution for values and labels of a particular model property.The majority of samples drawn from this distribution are set to pass while the outliers in the tail fail.

Figure 6 :
Figure 6: Training IDM yields sparse representation of inferences.Diagram shows a sample true data x (left) from test dataset and its sparsified representation Kx (right).

Figure 7 :
Figure 7: Reconstructions on test data computed via U-Net, [37] TV minimization, F-FPNs, [32] and Implicit L2O (left to right).Bottom row shows expansion of region indicated by red box.Pixel values outside [0, 1] are flagged.Fidelity is flagged when images do not comply with measurements, and regularization is flagged when texture features of images are sufficiently inconsistent with true data (e.g.grainy images).Labels are provided beneath each image (n.b. fail is assigned to images that are worse than 95% of L2O inferences on training data).Shown comparison methods fail while the Implicit L2O image passes all tests.

Figure 9 :
Figure9: Example of proposed L2O (left) and analytic (right) trades with noisy data d.Blue and green lines show proposed cryptoassets x and y to tender and receive, respectively (widths show magnitude).The analytic trade is unable to account for trade risks, causing it to propose large trades that are not executed (giving executed utility of zero).This can be anticipated by the failed trade risk certificate.On the other hand, the L2O scheme is profitable (utility is 0.434) and is executed (consistent with the pass trade risk label).
k is in the j-th CFMM's coordinates is token ℓ is in global coordinates 0 otherwise.(39)Note here we use the backwards of the referenced work, mapping global to local rather than local to global.Let d ∈ R n×m be a matrix with the j-th column d j the reserve assets in the j-th CFMM.For weighted geometric CFMMs, setd ≜ (1 + δ) ⊙ d i ) w ji , for all j ∈ [m],

Table 1 :
To clarify the scope of the word explainable in Left shows learning to optimize (L2O) model.Colored blocks denote prior knowledge and data-driven terms.Middle shows an iterative algorithm formed from the blocks (e.g. via proximal/gradient operators) to solve optimization problem.Right shows a trained model's inference N Θ ⋆ (d) and its certificates.Certificates identify if properties of inferences are consistent with training data.Each label is associated with properties of specific blocks (indicated by labels next to blocks in right schematic).Labels take value pass , warning , or fail , and values identify if inference features for model parts are trustworthy.
Summary of design features and corresponding model properties.Design features yield additive properties, as indicated by "+ (above)."Proposed implicit L2O models with certificates have intuitive design, memory efficient training, inferences that satisfy optimality/constraint conditions, certificates of trustworthiness, and explainable errors.

Table 3 :
[43] PSNR Avg.SSIM Box Constraint Fail Fidelity Fail Data Reg.Fail # Params Average PSNR/SSIM for CT reconstructions on the 2,000 image LoDoPab testing dataset.†Reportedfromoriginalwork.[32]U-Netwastrainedwithfilteredbackprojection as in prior work.[37]Threepropertiesareused to check trustworthiness: box constraints, compliance with measurements (i.e.fidelity), and data-driven regularization (via the proximal residual in Table2).Failed sample percentages are numerically estimated via(13).Sample property values "fail" if they perform worse than 95% of the inferences on the training data, i.e. its CDF value exceeds 0.95.Implicit L2O yields the most passes on test data.3.4CTImageReconstructionSetupComparisons are provided for low-dose CT examples derived from the Low-Dose Parallel Beam dataset (LoDoPab) dataset,[43]which has publically available phantoms derived from actual human chest CT scans.CT measurements are simulated with a parallel beam geometry and a sparse-angle setup of only 30 angles and 183 projection beams, giving 5,490 equations and 16,384 unknowns.We add 1.5% Gaussian noise to each individual beam measurement.Images have resolution 128 × 128.To make errors easier to contrast between methods, the linear systems here are under-determined and have more noise than those in some similar works.Image quality is determined using the Peak Signal-To-Noise Ratio (PSNR) and structural similarity index measure (SSIM).The training loss was mean squared error.Training/test datasets have 20,000/2,000 samples.