key: cord-1012675-mskbumd3
authors: Bajgiran, Amirsaman H.; Mardikoraem, Mahsa; Soofi, Ehsan S.
title: Maximum Entropy Distributions with Quantile Information
date: 2020-08-08
journal: Eur J Oper Res
DOI: 10.1016/j.ejor.2020.07.052
sha: a67bc472c6e83bf2b8a866d40211f533defdc239
doc_id: 1012675
cord_uid: mskbumd3

Quantiles are available in various problems for developing probability distributions. In some problems quantiles are elicited from experts and used for fitting parametric models, which induce non-elicited information. In some other problems comparisons are made with a quantile of an assumed model which is noncommittal to the quantile information. The maximum entropy (ME) principle provides models that avoid these issues. However, the information theory literature has been mainly concerned about models based on moment information. This paper explores the ME models that are the minimum elaborations of the uniform and moment-based ME models by quantiles. This property provides diagnostics for the utility of elaboration in terms of the information value of each type of information over the other. The ME model with quantiles and moments is represented as the mixture of truncated distributions on consecutive intervals whose shapes and existence are determined by the moments. Elaborations of several ME distributions by quantiles are presented. The ME model based only on quantiles elicited by the fixed interval method possesses a useful property for pooling information elicited from multiple experts. The elaboration of Laplace distribution is an extension of the information theory connection with minimum risk under symmetric loss functions to the asymmetric linear loss. This extension produces a new Asymmetric Laplace distribution. Application examples compare ME priors with a parametric model fitted to elicited quantiles, illustrate measuring uncertainty and disagreement of economic forecasters based on elicited probabilities, and adjust ME models for a fundamental quantile in an inventory management problem.

Quantile information is available in various decision and statistical problems. Experiments on accuracy of elicitation of probability distributions have shown that subjects can assess median reasonably more accurate than the mean (Garthwaite, et al., 2005) . Variable interval method and fixed interval method are used for eliciting quantiles and probabilities. Central banks such as the Federal Reserve Bank of Philadelphia, European Central Bank, the Bank of England, and other institutions survey economic forecasters for eliciting subjective probabilities for a set of fixed intervals. These probabilities are used to produce economic outlook reports for policy makers. The research area of measuring the uncertainty and disagreement of economic forecasters is primarily based on using these probabilities for developing other aspects of forecast distribution of variables such as the growth of Gross Domestic Product (GDP); Lahiri & Wang (2019) and Shoja & Soofi (2017) provide reviews and latest developments on this topic. In operations and business decision problems a quantile appears as the solution to the minimum risk (expected loss) under the asymmetric linear loss function such as the newsvendor problem, where the optimal order quantity that maximizes the profit is a quantile of the demand distribution (Snyder & Shen, 2011) .

Quantiles provide partial information about a probability distribution. Traditionally, this type of partial information is used to fit parametric probability distributions. The information theory literature primarily have considered developing distributions based on various types of moments (Jaynes, 1957 (Jaynes, , 1968 . The use of probability allocation has been sporadic (Brockett et al. 1995; Ebrahimi et al. 2008) ; quantile is only noted (Asadi et al. 2014 ). This paper aims to fill this void.

Two information measures are used for deriving models. The Shannon entropy of a quantity X with continuous distribution F and a probability density function (PDF) f is defined by

where R is the range of f , provided that the integral is finite. The Kullback-Leibler (KL) information divergence between f and f r is defined by

provided that f is absolutely continuous with respect to f r (f (x) = 0 whenever f r (x) = 0); the inequality becomes equality if and only if f (x) = f r (x) almost everywhere. Unlike the entropy, K(f : f r ) is invariant under one-to-one transformations of X for the continuous case. This measure has been used in operations and decision problems (see, for example, Alwan et al., 1998 , Plischke, Borgonovo, & Smith, 2103 , Saghafian & Tomlin, 2016 , Asadi et al., 2018 . The Minimum Discrimination Information (MDI) principle minimizes K(f : f r ) where f is in a class of distributions Ω and the reference PDF f r / ∈ Ω. When f r is uniform (proper PDF or improper), the MDI coincides with the Maximum Entropy (ME) principle (Jaynes, 1957) . For the continuous case, the ME is defined as minimization of −K(f : f r ), where f r is interpreted as the "invariance measure" function (Jaynes, 1968 ). In the game theory problem of selecting a distribution where the utility is defined in terms of the score function, the MDI and ME models are minimax decisions (Smith, 1974 , Grünwald & Dawid, 2004 . Jaynes (1968) advanced assessing the information value of additional constraints by comparing the entropies of two discrete ME models obtained from information sets of two experts. Soofi et al. (1995) extended this idea through the notion of information distinguishability defined by equality between K(f : f r ) and the entropy difference where f ∈ Ω and f r is the ME model.

We consider the following three classes of partial information about f on continuous support:

Ω α = {f α : E f [1(X ≤ q k )] = α k , k = 1, . . ., m},

Ω θ,α = Ω θ ∪ Ω α .

Maximum entropy

Information Models Updating Information value where T j (X) is integrable with respect to f , θ j sets the value of E f [T j (X)], 1(A) is the indicator function of the event A, and E f [1(X ≤ q k )] = α k sets the value of quantile q k = F −1 (α k ), k = 1, . . . , m; (3) and (4) will be referred to as moment information and quantile information (QI). Figure 1 gives a schematic representation of the modeling approach of this paper. Distributions in (3) and (4) are embedded in the larger family (5). Such larger families are called elaboration of the smaller models (Box & Tiao 1973) indexed by the elaboration parameter (Carota et al., 1996) . In Figure 1 f θ,α ∈ Ω θ,α is elaboration f θ and f α indexed by α and θ, respectively. Carota et al. (1996) proposed a Bayesian KL diagnostic for measuring the utility of elaboration. Thus far, elaborations are chosen arbitrarily for model evaluation. Including additional moments to ME and MDI models is well-known, but is not characterized as being the minimum elaboration. This research develops minimum elaborations of ME models by α and θ according to the MDI principle.

This research develops ME minimal elaborations of ME models and diagnostics (highlighted parts of Figure 1 ). The ME models f * α and f * θ,α are elaborations of the uniform distribution and f * θ by the QI, respectively. We show that the MDI updating of f * θ and f * α relative to each other give the equivalence of f mdi θ,α and f * θ,α and characterizes f * θ,α as the minimum elaboration of the smaller ME models. These MDI adjustments of f * θ and f * α for satisfying the constraints in Ω θ,α provide diagnostics for assessing their information values and the utility of the minimum elaboration. A result represents f * θ,α as the mixture of truncated PDFs on supports formed by consecutive quantiles whose shapes and existence are determined by the type of moments. This representation facilitates computation of the moments and entropy of f * θ,α and simulations from this model. Elaborations of the classical ME models (uniform, exponential, and normal) by QI are given. A result explores a useful property of f * α for aggregating quantiles elicited from multiple experts by the fixed interval method. The elaboration of Laplace distribution extends the notion of minimum risk ME models under symmetric loss (Ardakani et al., 2017) to an asymmetric loss and produces a new Asymmetric Laplace (AL) distribution. The elaboration of well-known AL is also given.

Applications are illustrated for a wide range of problems with a common theme: ME models with QI purely carry the given information into the results because of being "maximally noncommittal with regard to missing information" (Jaynes, 1957) . Three application areas illustrate cases where non-elicited information is included or existing information is excluded. The ME prior, f * α , is piecewise uniform and proper on a bounded range, which is most directly in accord with Laplace's principle of "insufficient reason". The range can be set wide like fitting parametric models such as gamma priors to quantiles elicited for a finite range. However, unlike the parametric families, f * α does not induce non-elicited information. Model fitting also is used for measuring uncertainty and disagreement of forecasters where probabilities are elicited by the fixed interval method. The result for the ME model based on QI with fixed intervals is particularly suitable for this problem. This is illustrated for the Federal Reserve Bank of Philadelphia's Survey of Professional Forecasters.

The case of excluding existing information is seen in the newsvendor (NV) problem of inventory management. The ME principle has been invoked for justifying the classical ME distributions as models for the demand distribution (Perakis & Roels, 2008; Andersson et al., 2013) , where a profit maximizing QI that appears in the analysis is not accounted for. We illustrate updating of these classical ME models in light of this information. The new minimum risk ME model and the elaboration of the known AL model are particularly useful for the NV problem.

The paper is organized as follows. Section 2 gives preliminaries and ME models with moment information. Section 3 presents the minimum elaboration of ME models by the QI. Section 4 illustrates potential applications of QI for developing Bayesian prior, measuring uncertainty and disagreement of economic forecasters, and reliability and inventory problems. Section 5 gives some concluding remarks. Computational details are available in a Supplementary Document.

The ME model in Ω θ , if exists, is unique and has PDF in the following form:

where η 1 , . . . , η J are Lagrange multipliers and

is the normalizing factor for f * θ , and θ j = −∂ log C η /∂η j , see, for example, Soofi et al. (1995) . The existence of the ME model is determined by the finiteness condition (7).

Well-known families of probability distributions are ME models with various types of moment information. On the finite range an ME distribution always exists. The best known examples are the uniform distribution [a, b] with the range information E f [1(a ≤ X ≤ b)] = 1 and the beta distribution [0, 1] with two geometric means: T 1 (x) = log x, T 2 (x) = log(1 − x); hereafter the range information will be omitted. On the nonnegative range, well known examples of ME models are the exponential distribution with T 1 (x) = x, the gamma distribution with T 1 (x) = x and T 2 (x) = log x, and the log-normal distribution with T 1 (x) = log x, T 2 (x) = (log x) 2 . On the unrestricted range, the ME model with T j (x) = x j , j = 1, . . ., J when J is an odd number does not exist. Examples of the ME models with even J are the normal with J = 2 and the quartic exponential with J = 4 (Zellner & Highfield, 1988 ).

If f * θ ∈ Ω θ is the ME model, then for any f ∈ Ω θ with finite entropy,

provided that f is absolutely continuous with respect to f * θ (Soofi et al., 1995) . Due to the uniqueness of f * θ ∈ Ω, f is distinguishable with f * ∈ Ω if and only if it reduces the maximum entropy. The information distinguishability (ID) index of f is defined by normalized KL measure

When f is also an ME in a subset of Ω θ with more constraints, (8) provides the information values of additional constraints. Asadi et al. 2014) have used this information diagnostic for assessing the information value of additional moments in a ME problem. McCulloch (1989) proposed a calibration of the KL divergence based on the KL between the Bernoulli distributions for the outcome of a biased coin with π > 0.5 and a fair coin with π 0 = 0.5. This measure is represented by

The Laplace and normal distributions are ME models consistent with T (X) = |X − ξ| ν = θ for ν = 1, 2, respectively. Ardakani et al. (2017) discussed the ME problem consistent with the minimum decision-theoretic risk of predicting X under the general symmetric loss function L(X,x) = |X − x| ν , ν > 0 on the unrestricted range. In the statistical decision theory, the consequence of a decision d about an unknown quantity X is associated by a loss function L(X, d) ≥ 0 and the risk function of a decision rule is defined by R(X, d) = E f [L(X, d)] (Berger, 1985) . The optimal decision in a set of possible decisions D is defined by

The ME problem consistent with the minimum risk of decision, d, is as follows:

be the minimum risk of the optimal decision. The ME model in Ω θ,d * , if exists, is given by

The optimal decision under the quadratic loss is the mean, d * = E(X) and the risk is the variance, so the minimum risk ME problem includes the first two moments. On the unrestricted demand, the minimum risk ME model is normal, but the solutions on the finite and nonnegative ranges are not so straightforward; these cases will be discussed in Section 2.2. The optimal decision under the absolute loss (symmetric linear loss function ν = 1) is the median, so the minimum risk ME problem includes a moment and a quantile; this case will be extended to the asymmetric linear loss function in Section 3.3. (Fleischhacker & Fok (2015) included the expected loss under demand under the loss function L(p) = x Ap, where p = (p 1 , . . . , p N ) , N i=1 p i = 1 is a probability vector (discrete distribution on N point), A is a nonnegative N × N matrix, and x = (x 1 , . . . , x N ).)

The ME modeling with the first two moments, E f (X j ) = θ j , j = 1, 2 on a restricted range (finite or nonnegative) is intricate. By (6), the PDF of the ME model, if exists, is in the following form:

where the Lagrange multipliers are given by θ 1 = E(X) = µ and σ 2 = θ 2 − θ 2 1 as follows:

However, unlike the unrestricted range, the existence and shape of the ME model depend on the range and the relationship between the moments. Dowson & Wragg (1973) give a rigorous treatment of the problem and provide results that are summarized in our notations as follows.

On the finite range the ME model always exists. The left panel of Figure 2 shows various shapes of the ME PDFs with first two moments on [0, 1], where A r , r = 1, 2, 3 are the regions in the (θ 1 , θ 2 )-plane shown in the right panel of Figure 2 . For the PDF on range [0, 1], (θ 1 , θ 2 ) are bounded in the region defined by θ 2 1 ≤ θ 2 ≤ θ 1 . On the lower boundary θ 2 = θ 2 1 , the distribution is degenerate P (X = θ 1 ) = 1 and on the upper boundary θ 1 = θ 2 , the distribution is a two-point distribution with P (X = 0) = 1 − θ 1 , P (X = 1) = θ 1 . On the curve A 2 , the Lagrange multiplier for the second moment is zero implying that the ME model is the truncated exponential distribution with decreasing PDF when θ 1 < .5, increasing PDF when θ 1 > .5, or uniform for θ 1 = .5. In the interior of the subregion A 1 the Lagrange multiplier for the second moment is negative implying Figure 2 : Shapes of the maximum entropy PDFs with first two moments on [0, 1] and regions of parameters for the shapes of the distributions: A 1 , η 2 > 0 (truncated normal); A 2 , η 2 = 0 (η 1 = 0, truncated exponential, η 1 = η 2 = 0, Uniform); A 3 , η 2 < 0 (U-shaped). that the ME model is the truncated normal distribution. In the interior of the subregion A 3 the Lagrange multiplier for the second moment is positive implying that the ME PDF is U-shaped.

On the nonnegative range, (12) is integrable only when η 2 ≥ 0 implying that with the first two moments an ME distribution exists if and only if θ 2 ≤ 2θ 2 1 . If a = 0 in the finite interval case, then at the limit as b → ∞, the ME PDF T N → T E (Dowson & Wragg, 1973) . Thus, the shape and existence of the ME model in Ω θ with the first two moments on x ≥ 0 are determined by the coefficient of variation cv = θ 1 / θ 2 − θ 2 1 . (The relationships between the parameters µ and σ and the moments of T N (µ, σ) are given in the Supplementary Document by (S.1) and (S.2).

Mixture models will be utilized for representing the ME model with QI and for aggregating ME models when partial information is elicited from multiple individuals. Let

The raw moments of f * θ,α can be calculated using the following decomposition:

where E i (X r ) is the moment of f i . However, the variance of f decomposes as follows:

The information measure of a mixture PDF f is the Jensen-Shannon (JS) divergence,

where W stands for {w 1 , . . ., w n } and K and H are the KL divergences and Shannon entropy as defined in (2) and (1) In the sequel, we will also encounter mixtures where the set of supports of f i is a partition of the support of f . In these cases application of the entropy decomposition formula (Di Crescenzo & Longobardi, 2002) gives the entropy of f as follows:

where H(W ) = − n i=1 w i log w i ≥ 0 denotes the entropy of the discrete distribution that gives probabilities w i to f i , i = 1, . . ., n, provided that H(f i ) is finite for all i.

The elaboration of f * θ ∈ Ω θ by QI is the PDF of the ME distribution in Ω θ,α given by

where the normalizing factor C ζ,λ is determined by the Lagrange multipliers (λ 1 , . . . , λ J , ζ 1 , . . . , ζ m ) for the moment and QI constraints in Ω θ,α . This ME model gives F * θ,α (q k ) = α k , k = 1, . . ., m. Because λ j , j = 1, . . ., J are determined by all constraints in Ω θ,α , in general, they are different from the parameters η j , j = 1, . . . , J in (6).

The ME model (20) generalizes (6). In the absence of the moment information, (20) gives f * α as the elaboration of the uniform distribution by QI. With the moment information, f * θ,α provides the elaboration of f * θ by α and the elaboration of f * α by θ. The KL divergence provides diagnostics for measuring the utility of elaboration (Carota et al., 1996) . Because Ω θ,α ⊆ Ω α and Ω θ,α ⊆ Ω θ , the ME models f * θ does not satisfy the QI constraints and f * α does not satisfy the moment constraints. The MDI principle provides the minimum adjustments for these models to satisfy both types of constraints. The MDI distribution in Ω θ,α , if exists, is unique and has PDF in the following form:

where the Lagrange multipliers are for constraints in Ω θ,α , and C mdi is the normalizing factor for

gives the MDI updates of these ME models for satisfying both types of constraints. The following lemma gives the equivalence between f * θ,α and f mdi θ,α which characterizes the minimum elaboration of f * θ and f * α by α and θ, respectively.

Lemma 1 Let f * θ ∈ Ω θ and f * θ,α ∈ Ω θ,α be the ME models and f mdi θ,α ∈ Ω θ,α denote the MDI model with reference to f * θ or f * α . Then, the MDI updates of f * θ and f * α for satisfying Ω α and Ω θ , respectively, are identical to f * θ,α ∈ Ω θ,α , almost everywhere, where λ j = η j − λ mdi,j , ζ mdi,k = −ζ k for all j and k.

Application of (8) to result of Lemma 1 establishes the following two ID relationships for updating of each type of information by the other type:

where the inequalities become equalities if and only if f * θ,α (x) = f * θ (x) and f * θ,α (x) = f * α (x), respectively, almost everywhere. By the inequalities in (21) and (22), the PDF f * θ,α is more concentrated than f * α and f * θ . Hence, the addition of one type of partial information to the other type leads to an information gain. The KL divergences in (21) and (22) give the minimum utility of the elaboration f * θ,α for f * α and f * θ . The entropy reductions in (21) and (22) quantify the information values of each type of partial information (Jaynes, 1968) .

Consider the partition of the support of f formed by consecutive quantiles in (4):

where q 0 = sup{x : F (x) = 0} and q m+1 = inf{x : F (x) = 1}. The constraints in (4) can be represented in terms of the following expectations:

where α 0 = 0. The following proposition gives a new representation of the ME model (20) in terms of the mixture of m + 1 truncated distributions on the partition of the support (23).

Let Ω θ,α be the class of distributions defined in (5). Then the ME model, if exists, is continuous and its PDF has the following (m + 1)-piece truncated representation:

. ., J, and the Lagrange multipli-

The existence and shape of the ME model f * θ,α is determined by the existence and shapes of f k , k = 1, . . . , m + 1. The kernels of the truncated PDFs f k are in the form of the ME model f * θ ∈ Ω θ , but the Lagrange multipliers of f k and f * θ are different due to the additional QI constraints. Proposition 1 is insightful and facilitates computations of the moments and entropy of f * θ,α via (15). Application of the entropy decomposition (19) facilitates computation of diagnostics (21) and (22) via H(f k ) given by (S.10) of Supplementary Document. Representation (25) facilitates simulations via mixing outcomes generated from f k , k = 1, . . ., m + 1.

The ME model with only QI constraints is in the following mixture of uniform PDFs:

where B k = q k − q k−1 , q 0 = a, q m+1 = b, α 0 = 0, and α m+1 = 1. This piecewise uniform PDF is a density histogram with unequal bins B k . On an unbounded range, additional information such as moments to supplement (24) is needed for the existence of an ME model. In the absence of moment information lower and upper limits for the unbounded bins, X ≤ q 1 and X > q m , should be set (like constructing histograms).

The entropy of (27) is given by (19) with H(f k ) = log B k and H(W ) = H(P α ), where P α is distribution with probabilities p 1 , . . . , p m+1 of the partition sets B k . This measure is sensitive to the choice of end points for unbounded bins. Its sensitivity can be easily assessed using p 1 log B 1 and p m+1 log B m+1 . The diagnostics (21) and (22) are invariant for the choice of endpoints.

When partial information is elicited from multiple experts pooling the data is in order. The arithmetic pool is commonplace. For developing ME forecast models two options may be followed:

(a) develop the ME forecast model that is consistent with arithmetic pool of partial information; or (b) pool the set of ME models that are consistent with the partial information provided by individual respondents. In general, these options provide two different forecast models. For example, the ME model consistent with the arithmetic pools of moments is in the same family as the individual ME models, but is not the mixture of the individual ME models (Shoja & Soofi, 2017) . However, the ME model (27) produces the same model for the two options when probabilities are elicited from multiple experts using fixed interval method. In this approach, the range of X is partitioned into intervals and the expert provides the probability that X will fall in that interval.

The following proposition highlights this interesting mixture property of ME model (27).

denote probabilities assigned by an individual to given

. . , n} be the set of individual ME forecast models (27). Then,

where f * c is the ME model consistent with the average bin probabilitiesp k = n i=1 w i p ik .

In pooling applications, the subscript of f * c stands for its designation as the consensus distribution and the weights are uniform. The divergence and information measures (17) and (18) 

The main moment information used in ME modeling are the mean and the first two moments. The following corollary gives the ME model with the QI and mean constraints.

Then the ME model is the (m + 1)-piece truncated exponential with the following PDF:

and λ is given by

When QI constraint is included in addition to the first two moments, the ME model is given by Proposition 1, where the shapes and existences of the truncated distributions in (26) are given according to results of Dowson & Wragg (1973) , summarized in Section 2. a, b] and the regions A r , r = 1, 2, 3 in Figure 2 . This is an affine transformation which affects the location and scale of a PDF, but does not affect its shape. The following corollary presents the shape and existence of the ME model with QI in addition to the first two moments for the finite, nonnegative, and unrestricted ranges.

Corollary 2 Let f k (x) be as defined in (26) and denote their first two moments by θ jk = E k (X j ), j = 1, 2. Then the shape and existence of the ME model f * θ,α is determined by f k , k = 1, . . . , m + 1 as follows.

(a) On the finite range a ≤ x ≤ b, f * θ,α always exists and its shape is given by

where T N (µ k , σ k ) and T Exp(θ 1k ) are truncated normal and exponential with parent distributions N (µ k , σ 2 k ) and Exp(θ 1k ) on the unrestricted and nonnegative range, respectively.

(b) On the nonnegative range, the shapes of f * θ,α on 0 ≤ x ≤ q k , k = 1, . . ., m are as in (a) and its shape and existence on x > q m is given by

Does not exist if , cv m > 1.

(c) On the unrestricted range, the shapes of f * θ,α on q 1 ≤ x ≤ q k , k = 2, . . . , m are as in (a) and its shape and existence is as in (b) applied to f k , k = 1, m+1, where f 1 (x) = f θ,α (q 1 −x)/α, x ≤ q 1 .

The ME characterization of the Laplace distribution extends to the family of ME models with expected minimum risk under the following asymmetric linear loss function:

where c o and c u are factors for losses of overestimation and underestimation, respectively. The loss function (33) is used in various problems such as quantile regression estimation and inventory problems. The optimal decision that minimizes E f [L(ξ, X)] is given by

The solution is the quantile of the distribution of X corresponding to α = c u /(c o + c u ) and the

risk, E f [L(q, X)] = θ, is the mean absolute deviation from a given quantile q.

The class of distributions that satisfies the minimum risk under loss function (33) is

Proposition 3 The ME model in Ω θ,α on the unrestricted range R = is an Asymmetric Laplace (AL) distribution with the following absolutely continuous PDF:

where the Lagrange multipliers for the risk and QI constraints are λ = 1/θ > 0 and ζ = 0,

Because ζ = 0, the second constraint in (35) is redundant. That is, (36) is the ME model in the following class of distributions:

The entropy of (36) is given by

The entropy expression maps the intuition that the uncertainty increases with the risk θ and confirms that (36) is also the ME model in (37). The mean and variance of (36) are

The mean exceeds q whenever c u < c o . The variance provides the same conclusion as the entropy;

(computations of the entropy and moments are shown in the Supplementary Document).

Proposition 3 establishes the information theoretic link between an AL distribution and the asymmetric linear loss function as the links for the normal and Laplace distributions with the symmetric quadratic and linear loss functions, respectively. The ME model (36) Corollary 3 The ME model in (35) on nonnegative range is the truncated continuous AL distribution with the following piecewise PDF:

where the Lagrange multipliers for the risk and QI constraints are given, respectively, by

Due to the truncation of the PDF in Proposition 3 for the nonnegative range the normalizing factor for the first branch has been adjusted which makes the PDF (38) discontinuous at x = q.

Also the second term in the expression for risk and the negative expression for ζ have been adjusted.

The model for the finite range can be adjusted similarly. including an ME characterization for a version with τ = 0 in terms of the following constraints:

We give the elaboration of this AL distribution by QI as follows.

We first give a new ME characterization of this well-known AL model. Consider the class of PDFs:

where θ 1 = F (τ )(τ − µ 1 ), θ 2 =F (τ )(µ 2 − τ ), and µ = F (τ )µ 1 +F (τ )µ 2 .

Lemma 2 The ME model in (40) is an AL distribution with the following absolutely continuous PDF f * θ 1 ,θ 2 ,τ (x) =

where the Lagrange multipliers for the constraints in (40) are given by η i = 1/(θ i + √ θ 1 θ 2 ), i = 1, 2.

The constraints in Ω θ 1 ,θ 2 ,τ can be written as

Asadi et el. (2014) referred to these as "local moments" and showed that the ME model that satisfies these constraints on the nonnegative range is piecewise exponential. The truncated exponential on 0 ≤ x ≤ τ can be increasing, decreasing, or uniform (see Figure 2 ), but on the unrestricted range f * θ 1 ,θ 2 ,τ must be increasing on 0 ≤ x ≤ τ which gives an AL. Letting η 1 = 1/(σκ) and η 2 = κ/σ in (41) gives a well-known AL distribution, where σ is the scale parameter and κ = (θ 1 /θ 2 ) 1/4 is the asymmetry parameter (Kotz et al.,2001) . Bera et al. (2016) related this formulation to the quantile regression estimation. In general, (41) does not satisfy a given QI, since

Lemma 2 provides an ME formulation suitable for the elaboration of (41) by the QI constraint in the following class of distributions:

The first two constraints in (43) can be represented by the risk constraint in (35) where θ = c o θ 1 + c u θ 2 , and one of the local moments, for example,

The following corollary gives the ME model in this class.

Corollary 4 The ME model in Ω θ 1 ,θ 2 ,α is a continuous AL distribution with the following piecewise PDF:

where the Lagrange multipliers for the risk and QI constraints in (43) are given by

Corollary 4 gives the ME model for unrestricted range. The ME model in (43) Figure 4 : ME prior consistent with the range, the median, three quartiles, and the gamma prior with parameters chosen to satisfy the quartiles. intervals of equal probability for eliciting the expert's median. Then the expert is asked to further bisect each of these intervals for eliciting the lower and upper quartiles. The process resulted in q .25 = 19, q .50 = 24, and q .75 = 30. Faucett used these quartiles for computing the parameters of the gamma conjugate prior (shape r = 9 and scale s = .36) for a Pareto likelihood model. Figure   4 depicts the PDF and CDF of this gamma and the PDF of the ME piecewise uniform priors (27) for the median and for all elicited information. The uniform ME reference prior is also included. Table 1 presents the elicited information provided by the median and the three quartiles, and by the additional non-elicited information induced by the gamma prior. The information value, K ij , is computed by (8) with f * j being the ME prior with less constraint. The information value of the median relative to the uniform reference corresponds to the information provided by experimental results of a biased coin with probability .66 against the fair coin, which is rather informative. The information value of quartiles, given by K 20 , is comparable with a biased coin with probability .86, which is substantial. The non-elicited information, K 32 , is computed by (8) with f * (x) = f * 2 (x); the fitted gamma prior is in Ω α , in an approximate sense (its support is x > 0 but probability over the range [0,70] is greater than .9999 and it approximately satisfies the quartile constraints). This is comparable with the information divergence of a biased coin with probabilities .87 from a fair coin. The entropy reduction of the gamma prior relative to the ME QI prior, given by K 21 , is about the same as the information provided by the quartiles. That is, this fitted prior induces non-elicited Suppose that one also elicits the mean, E(β) = 25, in the above problem; (this is the mean of the fitted gamma prior). The upper row of Figure 5 shows the PDFs and CDFs of the ME priors with the mean, the median, and both. The CDFs of these priors are continuous. The ME prior with the mean is the truncated exponential and the prior with the mean and median is the two-piece truncated exponential. Table 2 presents the information analysis of these priors (rows i = 0, 1, 2, 3). The mean relative to the uniform reference is more informative than the median.

The information value of the mean relative to the ME with the median corresponds to a biased coin with probability .71 against the fair coin, which is not negligible and the information value of the median relative to the ME with the mean corresponds to a biased coin with probability .67.

The lower row of Figure 5 shows the PDFs and CDFs of the ME priors with the mean, the quartiles, and both. Plots for the gamma prior fitted to the quartiles are also shown. Table   2 presents the information analysis of these priors. Comparison of K 10 and K 20 indicates that the mean is more informative than the median. The indices K 31 and K 32 indicate that the mean provides more information over the median than the median over the mean. Comparison of K 30 and K 40 indicates that the three quartiles are substantially more informative than the mean and median jointly. The indices K 53 and K 54 indicate that the mean and median provide more information over the median and quartiles than the latter over the former. Information analysis of the fitted gamma prior indicates that it induces substantial non-elicited information over the elicited information, which corresponds to a biased coin with probability .85; K 65 is calculated using (8).

In (14), with w i = 1/n. A considerable amount of attention is given to the variance decomposition (16) for σ 2 c . Both, σ 2 c and the average of σ 2 i , are used as measures of uncertainty and variance of the means (last term in (16)) is used as a measure of disagreement. Various variance estimates are used, including the variance of the midpoints of the bins x k = (x k + x k−1 )/2, k = 1, . . . , m, this with the Sheppards correction 1/12, the variance of the normal fit to histograms, and the variance of beta distributions fitted to the quantiles.

Information measures also have been used for measuring uncertainty and disagreement. Rich & Tracy (2010) used the discrete entropy of the probabilities H(P α ) ≥ 0, but did not use any information divergence for disagreement. Shoja & Soofi (2017) proposed an information framework for measuring uncertainty and disagreement. This framework requires a set of individual forecast distributions, specifying weights for their arithmetic pooling, and measures of uncertainty and divergence. These authors used the discrete entropy, H(P α ), and the ME models based on the (2019) avoided using discrete entropy and applied the information framework of Shoja & Soofi (2017) using the entropies of beta and triangular distributions fitted to the histograms of forecasters' subjective probabilities and their average; they estimated bounds for the unbounded intervals according to the fit. The use of fitted parametric distributions, such as the normal and beta, induces non-elicited information, so does the use of their variance. In addition, a parametric model fitted to the pooled quantiles is a mixture of the models fitted to the elicited quantiles, which is not a pool of the models fitted for the individuals.

We use the ME model (27) to measure uncertainty, disagreement and information of economic forecasters within the framework of Shoja & Soofi (2017) . The subjective probabilities of each forecaster provides an ME model f * i = f * α i , i, i = 1, . . . , n. By Proposition 2, f * c is the ME forecast model based on the pool of subjective probabilities of forecasters. Then H(f * c ) and (18) with f i = f * i provide two uncertainty measures for the set of forecasters and (17) gives the disagreement measure for the set of forecasters. Following the literature, we use w i = 1/n. We first illustrate the implementation of the information framework with the ME model (27) using data given by four respondents of the SPF in the first and second quarters of 2020 for the U.S. GDP growth in 2020 and then apply to all SPF respondents who provided probabilities for the 2020 U.S. GDP growth. Table 3 gives the probabilities assigned by the four forecasters selected to represent typical forecasters of the first two quarters of 2020 (before and during the coronavirus pandemic). The last row gives the pooled elicited information. Figure 6 shows the plots of the ME forecast distributions for these forecasters, superimposed by the pooled ME forecast model which is derived using the pooled QI constraints according to Proposition 2. The unbounded bins are assigned B 1 = B 10 = (q 1 + q 9 )/2 = 4%. The plots in the first row are for the first quarter of 2020 and the second row shows the corresponding plots for the second quarter. Shifts of all distributions toward the lower side are apparent. Figure 6 : The ME forecast models with quantile information based on subjective probabilities of four SPF forecasters in the first (first row) and second (second row) quarters of 2020 (dashed-red) and the PDFs of the respective pooled ME forecast models (solid blue).

The uncertainty of individual forecasters and disagreement among them are tabulated below the plots of the ME models. These measures are computed using (19) for the individual i and pooled distributions within the information framework of Shoja & Soofi (2017) . For each forecaster,

, maps the extent of its concentration and quantifies the uncertainty of the ME model (solid blue). The KL divergence K i = K(f * i : f * c ) maps the extent to which the individual ME model disagrees with the pooled ME model for the quarter (dashed red); π(K i ) index gives the probability according to the biased coin calibration for the KL divergences K i . These measures indicate that in most cases the uncertainty is decreased in the second quarter and in all cases the disagreement is sharply increased. The pooled uncertainty H c and disagreement JS c are averages

For the quarter pool, H(f * c ) and the average of four individual entropies, H c , are measures of overall uncertainty. The disagreement information, JS c , is the average of four individual divergences, which is also the difference between the two measures of uncertainty for the pool. These measures confirm that, overall, the uncertainty is decreased in the second quarter and the disagreement is sharply increased. The coronavirus pandemic might have played a role in these changes.

Next we present application to all 2019 and 2020 quarterly forecasters of the 2020 U.S. GDP in the SPF database. Table 4 The SPF provides information about the forecaster's industry. 

Two truncated distributions below and above a threshold, τ , on the support of a distribution are prominent in reliability modeling, where τ represents the current age of an item and in inventory management where τ represents the inventory. We consider the case when τ = q is a quantile of the ME PDF f * θ,α on the entire support. Then the truncated distributions in (26) are:

whereF * θ,α (q) = 1−α is the survival function at the optimal order. In reliability modeling, (45) and (46) correspond to the past-life or down-time and the residual distributions at a given time q. The "mean residual" is also used in inventory problems. In inventory analysis, (45) is the distribution of sales (the conditional distribution of the demand, given that demand does not exceed the inventory),

and (46) is the distribution of stock out (the conditional distribution of the demand, given that demand exceeds the inventory). We present ME models useful for inventory problems.

A widely studied topic in inventory management is the so called newsvendor (NV) problem, which is defined by an optimal order quantity for a product with an uncertain demand X and fixed prices. Let c be the cost per unit, r the selling price per unit, and s the salvage value per unit, then c o = c − s and c u = r − c are the overage and underage costs, respectively. Assuming, s < c < r, the expected profit maximizing (loss minimization) solution for the optimal quantity order is the αth quantile of the demand distribution where α = c u /(c o + c u ). This solution is traditionally represented in terms of the βth upper quantile β = 1 − α. This optimization, however, does not offer a model for the unknown demand distribution, F , and without a distribution q remains unknown.

The NV optimal decision theoretic decision under the loss function (33) coincides with NV decision based on the expected profit maximization; see for example, Snyder & Shen (2011, p. 78 ).

Thus, the new ME Asymmetric Laplace model (36) provides a demand distribution consistent with the optimal order for the basic NV problem. Corollary 4 also provides a model for the NV problem. In (39), µ 1 is the average sales which can be assessed based on the sales data. This and the usual assumption of given mean demand, µ, provide the partial information about the demand distribution. In (40), the constraints represent the mean inventory, the mean stock-out, and the profit maximizing quantile.

The NV literature offers moment-based rules for computing optimal order, such as the Scarf's (1958) rule and the minimax regret rule (Perakis & Roels, 2008) . These rules include the cost ratio, α, but do not offer models for the demand distribution. Instead, well-known probability models are then chosen for the demand distribution as a separate task. The classical ME models (uniform, exponential, and normal) are used for the distribution of the demand and justified based on the ME principle (Perakis & Roels, 2008; Andersson et al., 2013) . The moment based ME models are said to be non-robust to various degrees because their αth quantile are different from the optimal solutions of the moment-based rules. Such discrepancies between the optimal order quantities and the αth quantiles of moment-based ME models are due to the lack of inclusion of the QI in the ME calculations. The ME models with moments and quantiles resolve this discrepancy.

We illustrate applications of Corollaries 1 & 2 for two moment-based optimal order rules, one is based on the mean and one is based on the mean and variance. The minimax regret rule, y * r , depends on the partial information. With a given mean on the nonnegative range y * r = µα, α ≤ .5, q r = µ(1 − α), α ≥ .5. For the case of the first two moments on the nonnegative range, an approximate minimax regret rule (Roels, 2006) is

The Scarf's exact rule for the optimal order quantity based on the mean µ and the variance σ 2 is Figure 7 illustrates the lack of robustness of the exponential and truncated normal models against these rules. The PDFs and CDFs of these two moment-based models are shown by dashed red plots and for the corresponding ME models with moments and QI are shown by solid-blue plots. The left two panels of Figure 7 show the plots of the PDF and CDF of the ME models with and without the QI, where E(X) = θ 1 = 5, α = .4 given by the minimum regret rule, q = y * r = 2. Corollary 1 with a = 0 and b = ∞ gives the continuous distribution with the two-piece PDF: a truncated exponential on [0, q] and a shifted exponential on x ≥ q. The CDF plot illustrates the lack of robustness of the exponential model for this rule.

The right two panels of Figure 7 show the plots of the PDF and CDF of the ME models with and without the QI. For case of the ME model with QI θ 1 = 3.77, θ 2 = 15.45 so that the coefficient of variation is cv = 0.3 < 1 and the ME model is truncated normal. We let α = .35 so that the coefficient of variation conditions for the approximate minimum regret and the Scarf's rule hold.

These rules give y * r = 3.49 and y * s = 3.42. With y * r = 3.49, the ME model is T N (3.77, 1.11), where µ and σ are the mean and standard deviation of its parent normal distribution. The first two moments of f k , k = 1, 2 as (θ 11 , θ 21 ) = (1.65, 3.54) ∈ A 1 and (θ 12 , θ 22 ) = (4.91, 25.7) which gives cv 2 = .26; (the moments are computed using (µ k , σ k ) = (3.49, 1.16) in (S.1) and (S.2)) in the Supplementary Document. Part (b) of Corollary 2 with cv < 1 gives a continuous distribution with a two-piece PDF: a truncated normal on [0, q] and a truncated normal on x ≥ q. The CDF plot illustrates the lack of robustness of the truncated normal model for this rule. x ME wiht QI ME without QI Figure 7 : The PDFs and CDFs of the ME exponential and truncated normal models (dashed red) and the corresponding ME models with an additional QI constraint (solid blue). Table 6 : Entropies and information indices for the adjusted classical maximum entropy models. Table 6 gives the information values of QI when the uniform, exponential, and truncated normal distributions are adjusted to satisfy the maximizing profit quantiles as well as the minimax regret or Scarf's rules. The Scarf's rule for the uniform ME model on a finite range, [a, b] , gives y * s = 0 for α ≤ .25 and y * s < F * −1 (α) for α < .5 (for α > .5, y * s > F * −1 (α)). The minimax regret rule when the partial information is a finite range [a, b] gives y * r = (1 − α)a + αb which coincides with F * −1 (α) (Perakis & Roels, 2008) .

In Table 6 the uniform distribution is on [0, 70] and α = .75. The minimax regret rule gives y * r = 75 and there is no information value for the QI of this rule. The Scarf's rule gives y * s = 47.67 and the information gain of QI over the uniform model is comparable to the prediction of outcomes of a coin with probability .83, indicating that the QI of this rule is very informative. Notice that, here we have a single QI constraint and QI is nearly as informative as the four quartiles information in Table 1 . The information values of QI constraint with the minimax regret rule q = y * r and Scarf's rule q = y * s , in addition to the mean (Figure 7) , are comparable to the prediction of outcomes of a coin with probability .59 and .69, respectively. Accordingly, QI is rather informative indicating that these rules are not robust for the exponential model, however, the minimax regret improves over the Scarf's rule. The information values of QI constraint with q = y * r and Scarf's rule q = y * s , in addition to the first two moments (Figure 7) , are comparable to the prediction of outcomes of a coin with probability .62 and .53, respectively. Accordingly, QI with the minimax regret is rather informative, the Scarf's rule is robust under the truncated normal for the demand.

This paper presented several results and an assortment of ME models with given quantiles with or without moments and illustrated their potential applications in various problems. The results explored properties, existence, and shapes of models for the minimum information elaborations of the uniform and moment-based ME models by quantiles, and provided diagnostics for assessment of the utility of the minimal elaborations. The application examples illustrated the merits of the results by explicating problems where non-elicited information is induced in the assumed models

and information that appears in the analysis is not included in the assumed ME models. The results for the ME model with quantile and moments information enabled us to determine the shape and existence of the ME model, compute its entropy rather easily, and assess the information utility of the elaboration. Specific models provided elaborations of the uniform, exponential, normal, Laplace, and a known asymmetric Laplace distributions. The elaboration of the Laplace distribution is along the lines of Ardakani et al. (2018) that explored the link between the information theory and Laplace's first and second laws of error in terms of the minimum risk ME models under the absolute error and quadratic loss functions (Laplace and normal). This extension provides a gateway to further research on developing ME models with minimum risks of other asymmetric loss functions such as Linex loss which is used in various problems.

The ME distributions with quantile information are continuous, but their PDFs are not, so they do not provide the convenience of parametric priors. However, ME models enable assessing the non-elicited information that is induced by assuming parametric models that fit the elicited quantiles. From an example we learned that a gamma distribution fitted to quartiles elicited for a Bayesian prior induces non-elicited information as much as the elicited quartiles. The example also compared information of values of eliciting median, quartiles, and the mean.

Application of the ME model with QI elicited by fixed interval method to Federal Reserve Bank of Philadelphia's SPF revealed the effect of the coronavirus outbreak on the uncertainty and disagreement of the economic forecasters. The results informed that disagreement among forecasters about the U.S. 2020 GDP growth increased as time evolved until the coronavirus outbreak. After the outbreak, the uncertainty and disagreement of forecasters increased sharply above the level of the first quarter of 2020. An industry level analysis indicated that disagreement among forecasters in financial services is lower than those who are in other industries.

Our results shed light on an issue in the inventory management literature. A demand distribution is said to be (perfectly) robust if its profit maximizing quantile is consistent with a moment-based rule for an optimal order quantity. Achieving this property requires inclusion of this important quantile within the partial information set used for developing the rule. This is illustrated using elaborations of the uniform, exponential, and normal distributions by the profit maximizing quantile.

This paper considered only the case of continuous random variables. The discrete case can be illustrated similarly. The biased coin calibration of the KL divergence has been supplied so that the tabulated disagreement measures can be judged sizeable or not. The Bayesian inferences about the entropy and KL divergence can be used for more formal inferences.

Information theoretic framework for statistical process control

A maximum entropy approach to the newsvendor problem with partial information

Ranking forecasts by stochastic error distance, information, and reliability measures

Optimal hazard models based on partial information

New maximum entropy methods for modeling lifetime distributions

Asymmetric Laplace regression: Maximum likelihood, maximum entropy and quantile regression

Statistical Decision Theory and Bayesian Analysis

Bayesian Inference in Statistical Analysis

Information-theoretic approach to unimodal density estimation

Adaptive ordering and pricing for perishable products

Diagnostic measures for model criticism

Entropy-based measure of uncertainty in past lifetime distributions

Maximum-entropy distributions having prescribed first and second moments

Multivariate maximum entropy identification, transformation, and dependence

Comparing the point predictions and subjective probability distribution of professional forecasters

On the relationship between entropy, demand uncertainty, and expected loss

Statistical methods for eliciting probability distributions

Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory

Information theory and statistical mechanics

Prior Probabilities. IEEE Transactions on Systems Science and Cybernetics

On the rationale of maximum-entropy methods

The Laplace Distribution and Generalizations: A Revisit with Applications to Communications

Estimating macroeconomic uncertainty and discord using Infometrics

Bayes estimate and inference for entropy and information index of fit

Local model influence

Regret in the newsvendor model with partial information

Global sensitivity measures from given data

About monotone regression quantiles

The relationships among expected inflation, disagreement, and uncertainty: Evidence from matched point and density forecasts

Information and decentralization in inventory, supply chain, and transportation systems

The newsvendor under demand ambiguity: Combining data with moment and tail information

A min-max solution of an inventory problem

Uncertainty, information, and disagreement of economic forecasters

A derivation of entropy and maximum entropy criterion in the context of decision problems

Fundamentals of supply chain theory

Information distinguishability with application to analysis of failure data

A framework for measuring the importance of variables with applications to management research and decision models

Calculation of maximum entropy distributions and approximation of marginal posterior distributions

The authors are thankful to two reviewers for their comments that led to substantial improvement of the exposition of this paper. Soofi's research was supported by the 2019 Business Advisory Council Faculty Scholar Award.