key: cord-0140707-1hstxx1o
authors: Su, Jiun-Hua
title: No-Regret Forecasting with Egalitarian Committees
date: 2021-09-28
journal: nan
DOI: nan
sha: 0ff757ae741d7808abb7da048fc52363106d8a1e
doc_id: 140707
cord_uid: 1hstxx1o

The forecast combination puzzle is often found in literature: The equal-weight scheme tends to outperform sophisticated methods of combining individual forecasts. Exploiting this finding, we propose a hedge egalitarian committees algorithm (HECA), which can be implemented via mixed integer quadratic programming. Specifically, egalitarian committees are formed by the ridge regression with shrinkage toward equal weights; subsequently, the forecasts provided by these committees are averaged by the hedge algorithm. We establish the no-regret property of HECA. Using data collected from the ECB Survey of Professional Forecasters, we find the superiority of HECA relative to the equal-weight scheme during the COVID-19 recession.

Big data has been a buzzword in social science in recent years, and its popularity is witnessed in surveys such as Varian (2014) and Einav and Levin (2014) in economics, Lazer and Radford (2017) in sociology, and Brady (2019) in political science. The importance of data in empirical studies is self-evident. However, as argued by Lovell (1983) almost forty years ago, "it is by no means obvious that reductions in the costs of data mining have been matched by a proportional increase in our knowledge of how the economy actually works." The advance in theory is also important and indispensable for scientific progress. Data science -thus named perhaps because it emphasizes dialogues between theory and data -suggests an effective way to improve knowledge.

A dialogue between theory and data has been exemplified by the forecast combination puzzle in econometrics. This puzzle refers to the phenomenon that the equal-weight scheme, which is theoretically suboptimal in general, often outperforms the forecast combination with Bates and Granger's (1969) optimal weights as well as other sophisticated combination methods in empirical studies. The early dialogue had sparkled thought-provoking works on the combination of forecasts, as documented in Clemen (1989) . More recent surveys on the forecast combinations are provided by Timmermann (2006) as well as Elliott and Timmermann (2016) . Indeed, the dialogue on such a puzzle continues to this day. For example, acknowledging the equal-weight scheme as a high benchmark, Diebold and Shin (2019) propose the egalitarian ridge regression, which is the ridge regression with shrinkage toward the equal-weight scheme. 1 Their idea has heuristic appeal but leaves open the question of an appropriate computational method.

In this paper, we fortify the theoretical and computational foundations of Diebold and Shin's (2019) shrinkage approach by developing a real-time forecasting algorithm under the decision-making framework given that the growing literature on forwardlooking models in economics highlights the role of forecasting in decision making. 2 A decision maker first organizes egalitarian committees, that is, committees providing their forecasts, respectively, via a ridge regression with shrinkage toward the simple average of individual forecasts selected by mixed integer quadratic programming (MIQP).

The application of MIQP and partition of a parameter space solve Diebold and Shin's 1 Alternatively, Diebold and Pauly (1990) propose empirical Bayes forecasting procedures with shrinkage toward the equal-weight scheme. This Bayesian approach has been applied in, for example, Stock and Watson (2004) and Aiolfi and Timmermann (2006) . 2 As indicated in Clarida et al. (2000) and Mavroeidis (2010) , a forecast-based interest rate rule in response to future macroeconomic conditions can provide a guideline for a monetary policy maker. Forecasting matters in not only a public policy but a private agent's decision as well. Tanaka et al. (2020) build a simple model concerning a firm's decisions on inputs under uncertainty to rationalize the empirical evidence that its GDP forecast accuracy is a predictor of profitability and productivity. computational difficulty in egalitarian ridge regression with simultaneous selection of individual forecasts. Next, the decision maker pools committee forecasts by applying a variant of Freund and Schapire's (1997) hedge algorithm. This variant depends on an estimate of the maximal committee loss for the duration of its implementation. The decision maker's two-stage implementation of real-time forecasting is referred to as hedge egalitarian committees algorithm, henceforth abbreviated to HECA.

We establish non-asymptotic upper bounds on the average regret attained by HECA, which is the decision maker's average forecasting loss in excess of the smallest average forecasting loss accomplished by these egalitarian committees. First, these upper bounds indicate the decision maker's own acumen of business cycles could pay off because given the committee forecasts, a more precise estimate of the maximal committee loss ceteris paribus yields a tighter upper bound on the average regret. Furthermore, these upper bounds show that HECA has no-regret property; that is, the decision maker's long-run performance should be at least as good as the best long-run performance accomplished by the egalitarian committees. This result is in line with the findings in the online learning literature. An excellent overview of this literature is recently provided by Cesa-Bianchi and Orabona (2021) . More importantly, this no-regret property implies the superiority of HECA relative to the equal-weight scheme in the long run. It is such 'theoretical' superiority that makes HECA eligible for the competition with the equal-weight scheme; however, its 'empirical' superiority remains to be examined.

To examine whether HECA outperforms the equal-weight scheme in an empirical study, we focus on the quarterly one-year-ahead forecasts of Euro-area real GDP growth in Survey of Professional Forecasters (SPF), which is conducted by the European Central Bank (ECB). The purpose of selecting this dataset is twofold. On the one hand, it generates the equal-weight scheme that has particularly hard-to-beat forecasting performance, as demonstrated in Genre et al. (2013) , Conflitti et al. (2015) , and Diebold and Shin (2019) . On the other hand, it involves not-so-big data such that theoretical parts of data science (inclusive of domain knowledge, statistical methods, and computational techniques) are crucially important. Our empirical results show that HECA keeps pace with the equal-weight scheme before the outbreak of COVID-19 but wins the competition during the COVID-19 recession. Despite the superiority of HECA relative to the equal-weight scheme, HECA suffers from an upsurge in forecasting loss around the onset of COVID-19 pandemic. This pattern is consistent with previous research, as indicated in Chauvet and Potter (2013) . We also find that the formation of egalitarian committees gives HECA an advantage over Freund and Schapire's hedge algorithm during the COVID-19 recession.

In addition to the pursuit of forecasting performance, we are dedicated to credible forecasting in a spirit that only credible assumptions are maintained, as emphasized in Manski (2013) . To achieve this goal, we treat the data generating process (DGP) of target variables and their individual forecasts as a black box ; that is, no assumption on such DGP is imposed. Instead, the proposed HECA is a data-driven and adaptive approach: At each round, it outputs a combined forecast with more weights on forecasts provided by committees that have performed well in the past; furthermore, the built-in updating mechanism enables HECA to adapt to the environment in the presence of structural breaks that may make forecasters' relative performance unstable over time.

The unstable performance is called model instability in literature, and an excellent survey of this issue is provided by Rossi (2013) . Additionally, the committee forecasts, as inputs of HECA, are obtained by the rolling egalitarian ridge regression scheme. The rolling scheme is used to guard against possible parameter drift, as indicated by West (2006) , whereas the shrinkage toward the simple average of selected individual forecasts is supported by empirical evidence in literature.

HECA embodies interdisciplinary research, which is another marked characteristic of data science. 3 Knowledge in econometric literature, statistical methods, numerical and computational techniques, and online learning modeling are woven into HECA for the decision maker's real-time forecasting. The empirical findings in econometrics treat the equal-weight scheme as a high benchmark, to which the weights should shrink. In spite of alleviating numerical instability, the ridge regression in general fails to achieve the simultaneous selection of individual forecasts, which is implemented by mixed integer optimization. The hedge algorithm further allows for the adaptability to sequential data in real time. Equipped with these designs, HECA complements, but does not replace, existing forecasting methods. It is particularly useful in the situation where the decision maker has limited access to predictors. For surveys of data-rich methods, the reader is referred to Stock and Watson (2006) and Chauvet and Potter (2013) .

Throughout this paper, we write z 1 , z 2 , and z ∞ for the one-norm, two-norm, and infinity-norm of a generic column vector z in an Euclidean space, respectively. We denote the collection of positive integers and the collection of real numbers by N and R, respectively.

The structure of the remaining paper is as follows. Section 2 describes features of data collected from the ECB SPF and Eurostat. Section 3 presents the organization of egalitarian committees, the decision maker's HECA, and the theoretical upper bounds on the average regret of HECA. Section 4 discusses the empirical results of applying HECA to real-time forecasting of the year-on-year growth rate of euro area. Section 5 concludes. Technical proofs are deferred to the appendix.

The forecast target variables in this paper are the year-on-year Euro-area GDP growth estimates collected from Eurostat, the European Statistical Agency. 4 Due to data revisions, several estimates for a given quarter are released by Eurostat. Following Genre et al. (2013) , we focus on the t + 45 flash estimates, which are published about 45 days after the associated quarter, for our empirical study in Section 4. The evaluation sample runs from the first quarter of 2012 to the third quarter of 2020.

As in Genre et al. (2013) , Conflitti et al. (2015) , and Diebold and Shin (2019), we focus on the quarterly one-year-ahead forecasts of Euro-area real GDP growth in the ECB SPF. These one-year-ahead forecasts, however, are actually six to eight months ahead. For example, in the questionnaire for the third quarter of 2018, macroeconomic experts participating in the SPF are asked for the expected year-on-year real GDP growth for the first quarter of 2019 and provided with the GDP growth for the first quarter of 2018 as a reference.

One noticeable feature in the SPF is the frequent entry, exit, and reentry of experts so that an unbalanced panel arises. As pointed out in Genre et al. (2013) , such an unbalanced panel may yield sampling distortions. To lessen the extent of undesirable distortions, we exclude experts who did not reply in two consecutive quarters during the evaluation period spanning from the first quarter of 2012 to the third quarter of 2020. After this removal, there remain 21 experts. Hereafter, we focus on the forecasts provided by this filtered panel of experts. In Figure 1 , we mark a slot by the notation x if a forecast is provided by an associated expert for a specific quarter in the SPF; otherwise, we leave it blank. We further replace each missing value of a forecast with the simple average of the rest of the reported forecasts for the same quarter. For example, Expert 038 provides forecasts throughout the evaluation period except the one for the third quarter of 2015; this missing forecast is filled in with the simple average of forecasts provided by the other 19 experts, as the forecast associated with Expert 110 is also unreported.

Another well-known feature in the SPF is the forecast combination puzzle: It is hard for other sophisticated schemes to improve on the performance of equal-weight scheme, as indicated by Genre et al. (2013) and Conflitti et al. (2015) . Some rationales behind this puzzle are proposed in literature. From a theoretical perspective, as shown in Timmermann (2006) , the equal-weight scheme is optimal if individual forecast errors have the same variance and identical pairwise correlation. From a practical perspective, as noted in Smith and Wallis (2009) and Conflitti et al. (2015) , the finite-sample error and numerical instability may make the estimated optimal weights inferior to the equal-weight scheme in terms of forecasting performance.

Both perspectives are crucial in an empirical study using the evaluation sample from the SPF. First, Table 1 and Figure 2 show the sample variances and pairwise correlation coefficients, respectively, of individual forecast errors for the 21 experts in the filtered panel. Since the sample variances are similar to each other whereas the sample correlation coefficients are centered around 0.995, the hard-to-beat performance of equal-weight scheme is unsurprising. In addition, the asymptotic approximation of estimated weights is arguably imprecise because the overall sample size -35 quartersis obviously small. More importantly, the numerical instability in the estimated optimal weights is severe; for example, the condition number associated with the ordinary least square regression using the evaluation sample is 31, 112. 5 This finding yields a clue as to the application of ridge regression, a classical approach in literature to alleviating numerical instability, to the estimation of optimal weights. Recognizing the remarkable performance of equal-weight scheme, Diebold and Shin (2019) propose the egalitarian ridge regression, which is the ridge regression with shrinkage toward the equal-weight scheme. Following their approach, we further develop an algorithm in the next section that can select experts in each quarter and achieve some satisfactory objective in hindsight.

The fundamental importance of economic forecasting for forward-looking private agents and public policy makers motivates us to propose an algorithm that incorporates features of the SPF forecasts and outperforms the equal-weight scheme under the decision-making framework. Roughly speaking, we consider the situation where a single decision maker 5 The (2-norm) condition number of a matrix A is defined as

Ab 2 and often used to evaluate the stability of a linear system in numerical analysis. As argued in Belsley et al. (1980) , "moderate to strong relations are associated with condition indexes of 30 to 100." is allowed access to forecasts provided by anonymous experts using either quantitative models or model-free judgments, and such forecasts generating processes are unknown to the decision maker. 6 This decision maker is assumed to minimize the cumulative squared loss without discounting. The squared loss can be replaced with other loss functions, for example those documented in Section 2.2 of Elliott and Timmermann (2016) , in the rest of this paper. We refrain from this replacement because the squared loss is used in common with the literature on forecast combination puzzle.

To elaborate on the proposed method, we now introduce notation. Suppose that there are M experts providing a forecast of the target variable y t , respectively, before its realization. These individual forecasts are denoted by f t ≡ (f t,1 , . . . , f t,M ) , where f t,m stands for the forecast of y t provided by expert m ∈ {1, . . . , M }. 7 Accessing the data encompassing current forecasts, realized target variables, and their corresponding forecasts, the decision maker announces his or her own forecast of y t by a two-stage method: At the first stage, the decision maker imagines M committees {C c } M c=1 , where committee C c consists of c members selected among M experts; subsequently, each committee provides a forecastŷ t,c , which is a combination of forecasts provided by its c members. At the second stage, this decision maker applies the hedge algorithm to {ŷ t,c } M c=1 and then yields his or her own forecast of y t . To complete the two-stage method, we explain how these committees {C c } M c=1 are formed at the first stage and how the hedge algorithm works at the second stage in the subsections below.

The imaginary committee C c is organized by solving the following optimization problem for a fixed rolling window r ∈ N and every λ in a pre-specified set Λ of grids: 8 6 These experts' potentially strategic behaviors are also ignored by the decision maker. For the strategic forecasting, we refer the reader to Marinovic et al. (2013) and references cited therein.

7 In our empirical analysis in Section 4, the vector ft of forecasts in the SPF are six to eight months prior to the release of the t + 45 flash estimate yt by Eurostat.

8 As indicated in Elliott and Timmermann (2016, p. 378) , the length of estimation window can be selected by the cross-validation method, which is however rarely done.

where b 0 is the number of nonzero elements in b ≡ (b 1 , . . . , b M ) and 1 is the M dimensional column vector of ones. The rolling scheme is adopted to guard against possible parameter drift. Since all individual forecasts are measured on the same scale, they are not standardized in this ridge-type regression. In addition, the lag term can be set to be either l = 1 or l = 2 for real-time forecasting with the SPF forecasts. Let β t,c (λ) be a solution to problem (P1) associated with λ, and ι m ∈ R M be a unit vector with m-th element equal to one. The tuning parameterλ t,c is selected by settinĝ

where r λ ∈ N denotes the number of periods for validation. The set

is called the egalitarian committee with c members, for problem (P1) can be viewed as a subproblem of partial egalitarian ridge regression in Diebold and Shin (2019) . To see this, let

whereβ t,c (λ) is a minimizer of (P1) for each c ∈ {1, . . . , M } and a fixed λ. Although the objective function is discontinuous due to b 0 , we havẽ

Phrased differently, the partition of a parameter space according to the value of b 0 allows us to recoverβ t (λ). Therefore, Diebold and Shin's (2019) 'one-step' partial egalitarian ridge regression can be equivalently implemented as long as problem (P1) is successfully solved for every c. 9

To solve problem (P1), we recast it as the following MIQP:

If is the smallest machine-representable positive real number, 10 then problems (P1) and (P2) are computationally equivalent. The intuition is that under the constraints

positive; that is, expert j is selected in the committee C c with size b 0 , which is equal to the sum of d j 's. We summarize the discussion in the following proposition.

Proposition 1. Suppose that is the smallest machine-representable positive real number. The optimization problem (P1) and the MIQP (P2) are computationally equivalent in the following sense:

(i) If the machine yields the minimizer b * of (P1), then (b * , d * ) is a minimizer of (P2),

A conceptually simple method of solving problem (P2) is exhaustive enumeration.

To see this, note that there are M c feasible choices of d ≡ (d 1 , . . . , d M ) in problem (P2). For any given feasible d, this optimization problem is essentially the constrained 9 Although Diebold and Shin's (2019) partial egalitarian ridge regression concerns the inclusion of Tibshirani's (1996) one-norm regularization in the objective function rather than in the constraints, the idea of partitioning a parameter space still works mutatis mutandis.

10 As defined in Judd (1998, p. 30) , a machine zero is referred to as a quantity equivalent to zero on a machine. The positive real number is not a machine zero, but every positive real number less than is a machine zero.

ridge regression with the unknown parameter b. Implementing these M c ridge regressions thus suffices to solve problem (P2). We call this approach complete subset ridge regressions by analogy with complete subset regressions in Elliott et al. (2013) .

This exhaustive approach, however, may be computationally inefficient because every egalitarian committee is asked to provide a forecastŷ t,c ≡ f tβ t,c (λ t,c ); consequently, there are M 1 + M 2 + · · · + M M = 2 M − 1 ridge regressions to be carried out for every λ ∈ Λ.

Instead of such an exhaustive search for {β t,c (λ)} M c=1 in the parameter space, the modern solver Gurobi can be used to implement the MIQP in problem (P2). The practical tractability of moderate-size MIQP, though NP-hard in nature, can be attributed to the rapid advances in computation power. According to Bertsimas and Dunn (2019) Remark 1. The constraint on total weight (b 1 = 1) and the range constraints (0 ≤ b j ≤ 1 for each j) are included in problems (P1) and (P2) because they have the following nice properties. First, as pointed out in Granger and Ramanathan (1984) , the combined forecastŷ t,c ≡ f tβ t,c (λ t,c ) provided by the egalitarian committee C c , under the constraint on total weight, remains unbiased if every member in this committee provides an unbiased forecast of y t . As shown in Diebold (1988) , the constraint on total weight also implies that the forecast errors y t −ŷ t,c are serially uncorrelated if the individual forecast errors made by the members in this committee are serially uncorrelated. In addition, Jagannathan and Ma (2003) indicate the shrinkage effect of range constraints on reducing the estimation error of the covariances of experts' forecast errors. Finally, Conflitti et al. (2015) suggest that these constraints together can improve the numerical stability in computation.

In econometrics and statistics, assumptions about the DGP of (y t , f t ) are usually imposed to establish theoretically nice properties of a forecasting method. Elliott and Timmermann (2016, p. 320) , however, put it this way:

Interestingly, combination methods that attempt to explicitly model time variation in the combination weights often fail to perform well, suggesting that regime switching or model "breakdown" can be difficult to predict or even track through time.

Additionally, the knowledge about how individual forecasts are generated by experts is unknown in principle to the decision maker. A practical example given in Diebold (2015) is that forecasts are purchased from a vendor using proprietary models, which are not revealed to the decision maker. Recognizing such limits to knowledge, the decision maker attempts to neither model nor assume the DGP of (y t , f t ) and is dedicated to credible forecasting. 11 This decision maker thus deals with the real-time forecasting problem by the hedge algorithm based on the past performance of the egalitarian committees, as described in the next subsection.

After receiving forecastsŷ t ≡ (ŷ t,1 , . . . ,ŷ t,M ) made by all egalitarian committees, the decision maker announces his or her own forecast, which is a weighted average of {ŷ t,c } M c=1 . Subsequently, the nature announces the realization of the target variable y t . The sequence of target variables can be generated as in statistical models. For example, it can represent business cycles undulating along a trend, either deterministic or stochastic; it can exhibit structural breaks with changing points, either known or unknown; and it can describe switching among different states, either observed or unobservable. Further examples about statistical modeling can be found in Pesaran (2015) and Peña and Tsay (2021) . Alternatively, this sequence of target variables can be adversarially generated as in game-theoretic models, where the nature attempts to maximize the decision maker's forecasting loss. Details about game-theoretic analysis can be found in Cesa-Bianchi and Lugosi (2006) and Schapire and Freund (2012) . Briefly, the following happen in order for each round t:

1. Egalitarian committees announce their forecast combinationŝ y t,c ≡ f tβ t,c (λ t,c ), for c = 1, . . . , M, whereβ t,c andλ t,c are obtained by the method in the previous subsection; 2. Decision maker announces the forecast y t ≡ π tŷ t according to some distribution π t ∈ M ; 11 As argued in Manski (2013) , "[t]he fundamental difficulty of empirical research is to decide what assumptions to maintain." He further argues that "[s]tronger assumptions yield conclusions that are more powerful but less credible." 

log{M } t−1 ; 7: end for 3. Nature announces y t ∈ R;

t,c ≡ (y t −ŷ t,c ) 2 , for c = 1, . . . , M.

Adopting the aforementioned strategy (i.e., announcement ofŷ t in each round) the decision maker ex ante aims to obtain small average regret

by selecting the sequence {π t } T t=1 of distributions. To achieve this goal, this decision maker selects {π t } T t=1 by HECA, whose pseudocode is shown as Algorithm 1. HECA, adapted from the hedge algorithm in Freund and Schapire (1997) , incorporates features of the decision maker's real-time forecasting based on the SPF forecasts. Suppose that the decision maker announces his or her forecastŷ t immediately after receiving the SPF forecasts. In this case, the decision maker does not know the realized forecasting loss { t,c } M c=1 until round t + 2. Hence, in the first two rounds, the decision maker has no information about any committee's performance and thus uses the uniform distributions π 1 and π 2 . In the third and ensuing rounds, the decision maker observes every committee's performance { (t−2),c } M c=1 of two rounds prior, thereby updating the distribution π t ≡ (π t,1 , . . . , π t,M ) . Note that HECA requires an estimate of the maximal committee loss throughout the T rounds. The decision maker assumes B 1 to be this maximal loss, which might be a biased estimate, in the first two rounds, and updates it subsequently round by round.

HECA reflects the presumption embraced by the decision maker: A committee with relatively better performance (i.e., smaller (t−2) ) would maintain the momentum to perform relatively well in the current round; therefore, its weight π t,c in the combined forecastŷ t should relatively increase. The idea of performance-based pooling of forecasts has been used in econometrics, for example forecasts weighted by inverse mean squared error in Stock and Watson (1999) and Capistrán and Timmermann (2009) , and aggregated forecast through exponential reweighting in Yang (2004) and Wei and Yang (2012) , among others. The built-in updating mechanism makes HECA adaptive to the environment. The adaptability of HECA is in sharp contrast to the constancy of equal-weight scheme even in the possibly ever-changing environment.

The following theorem gives upper bounds on the decision maker's average regret.

These upper bounds are non-asymptotic; that is, they hold for every finite T ∈ N.

Theorem 1. LetB T ≡ max{ t,c : t = 1, . . . , T ; c = 1, . . . , M }. HECA guarantees that for all M, T ∈ N,

The non-asymptotic upper bounds in Theorem 1 hold without requirement of any assumption on the DGP. The decision maker may underestimate or overestimate the maximal lossB T . The biased estimation, however, is a contributing factor of an upper bound on the average regret. A sharper upper bound can be obtained if the magnitude of underestimation (overestimation), measured by γ u (γ o ), is smaller. It turns out that given the same committee forecasts and duration of implementing HECA, decision makers having different evaluations of business cycle may achieve different forecasting performances even if they use HECA. Calculation of these upper bounds is also easy ex post. In contrast, upper bounds in Yang (2004) and Wei and Yang (2012) involve nuisance parameters of the underlying DGP and need further evaluation.

Moreover, if the sequence {B T } ∞ T =1 is bounded above, 12 then HECA exhibits no regret because these upper bounds on the average regret all shrink to zero as T tends to infinity. Phrased differently, the performance of HECA is at least close to that of the best egalitarian committee in the long run. The no-regret property is common in the online learning literature. We refer the reader to Cesa-Bianchi and Lugosi's (2006) monograph for early findings and Cesa-Bianchi and Orabona's (2021) survey for recent advances. From a pragmatic standpoint, the assumption about boundedness of {B T } ∞ T =1 could be inconsequential. To see this, note that

by Jensen's inequality; additionally, as indicated by Elliott and Timmermann (2016, p. 17) , "[i]n practice, forecasts are usually bounded and extremely large forecasts typically get trimmed as they are deemed implausible."

More importantly, Theorem 1 establishes the intuition that the decision maker's two-stage method should outperform the equal-weight scheme whenever T is so large that the upper bounds are small. Theorem 1 implies that in the long run, HECA should perform at least as well as the best egalitarian committee. In addition, the best egalitarian committee would dominate the egalitarian committee C M , which should in turn be weakly better than the equal-weight scheme. It follows from these arguments that HECA would outweigh the equal-weight scheme in terms of long-run forecasting performance.

If the decision maker regularly postpones announcingŷ t until the realization of y (t−1) , then the information on { (t−1),c } M c=1 could be exploited. Because of the updated information, the decision maker's forecasting performance is expected to improve. Indeed, HECA with delayed announcements (Algorithm 2) allows for tighter upper bounds on the average regret, as shown in the following theorem.

Theorem 2. HECA with delayed announcements guarantees that for all M, T ∈ N,

The monotonicity of {BT } ∞ T =1 , together with its boundedness, implies that limT →∞BT < ∞.

Input:

ω 1,c ≡ 1 for each c = 1, . . . , M ;

where B 1 is an assumed maximal committee loss;

Output:

A sequence {π t } T t=1 of distributions; Remark 2. Unlike HECA, which updates w t,c according to the latest observed loss (t−2),c , an alternative updating mechanism makes smooth the adjustment to new weights by setting

This updating mechanism relies on the latest observed empirical loss (t − 2) −1 t−2 τ =1 τ,c and parallels Fudenberg and Levine's (1995) exponential fictitious play, which has the no-regret property, also known as Hannan-consistency in the literature on learning in games. For a book length treatment of this topic, see Fudenberg and Levine (1998) and Cesa-Bianchi and Lugosi (2006) . Similarly, we can consider the exponential fictitious play with delayed announcements by replacing (1) with the following updating mechanism

for each c = 1, . . . , M .

(2)

Convinced of the theoretically asymptotic performance of HECA, we are now concerned with its forecasting performance for the evaluation sample mentioned in Section 2.

We first concentrate on the competition among the equal-weight scheme, HECA (Algorithm 1 with l = 2) and HECA with delayed announcements (Algorithm 2 with l = 1). The latter two algorithms involve the first-stage MIQP, which is implemented by the Gurobi Python interface, and the tuning parameters are set by r = 16, r λ = 1, Λ = {0.01g} 200 g=1 , and = 5 × 10 (−324) . 13 We set B 1 to be the maximum of individual forecasting losses (observed by the decision maker) from the first quarter of 2012 to the quarter prior to t = 1. HECA, in comparison to its counterpart with delayed announcements, requires two extra rounds for 'in-sample' estimation ofβ's and validation ofλ's. Thus, we consider t = 1 in HECA to be the fourth quarter of 2016 and t = 1 in HECA with delayed announcements to be the second quarter of 2016. Table 2 reports their forecasting losses and associated differences per round for this competition. As can be seen, HECA and the equal-weight scheme are nearly neck and neck until the first quarter of 2020. Keeping abreast of the equal-weight scheme, which is a well-known high benchmark, HECA also performs well. HECA further outperforms the equal-weight scheme since the second quarter of 2021, in which "the fall in economic activity was unprecedented in depth, speed and scope". 14 HECA with delayed announcements even achieves better forecasting performance by exploiting updated information. Given the small-sample survey data from the SPF, we do not pursue statistical testing for the superiority of HECA because existing tests accounting for in-sample estimation error, for example the tests developed in Diebold and Mariano (1995) and Giacomini and White (2006) , rely on out-of-sample asymptotic approximation to determine the critical value. Moreover, Figure 3 implies that the number of experts in the committee performing best in a single round is not constant but time-varying. Despite this instability, the 13 The number 5 × 10 (−324) is equal to the product of sys.float info.min and sys.float info.epsilon in Python 3.7, and the mixed integer optimization are carried out by Gurobi 9.0.3, which is available at https://www. gurobi.com/.

14 This description is given in the news published on 29th March 2021 by Euro Area Business Cycle Dating Committee. Further details are available at https://eabcn.org/sites/default/files/eabcdc_findings_29_ march_2021.pdf. theoretical results in Section 3 suggest that HECA could perform as well as the best committee over the entire evaluation period in hindsight. As can be seen from Table 3, the average regret is relatively small given the substantial impact of COVID-19 pandemic on the euro area economy. It is worth noting that the best egalitarian committee in the fourth quarter of 2019 and in the third quarter of 2020 are identical, and the average regret is less than 0.03 if HECA with/without delayed announcements terminates in the fourth quarter of 2019.

Finally, we turn the spotlight on the cousin and ancestor of HECA. As shown in Table 4 , the exponential fictitious play with updating mechanism (1) performs almost the same as HECA; similarly, the exponential fictitious play with updating mechanism (2) very much resembles HECA with delayed announcements in terms of forecasting ability.

The close resemblance gives a hint on the no-regret property of exponential fictitious play. Table 5 shows that although neither Freund and Schapire's (1997) hedge algorithm nor HECA dominates each other before the fourth quarter of 2019, HECA wins the competition since the first quarter of 2020. Thus, combining the results in Tables 2 and 5, we find that during the COVID-19 recession, the formation of egalitarian committees gives HECA a competitive edge over the hedge algorithm, which beats the equal-weight scheme by adaptability.

The proposed HECA should be in the data scientist's toolkit for three reasons as follows.

From a theoretical perspective, HECA outputs credible forecasting because it relies on practically convincing assumptions and meanwhile achieves an asymptotically negligible upper bound on the average regret. From an empirical perspective, HECA outweighs the equal-weight scheme after the outbreak of COVID-19 in euro area, whereas the equalweight scheme only outperforms HECA by a margin, if any, before such an outbreak.

From a methodological perspective, HECA differs from other data-rich methods in that it is applicable in the context where no extra predictor of the target variables, except for the forecasts provided by the experts, is available for the decision maker.

We do not deal with the optimal timing of implementing HECA. Our empirical results seem to suggest that compared with the equal-weight scheme, HECA would be suitable for forecasting around business cycle turning points. It is also unclear whether the duration of implementing HECA should be determined at the very beginning. We delegate these fascinating issues for future work.

A.1 Proof of Proposition 1

For ease of notation, we define functions Q 1 : R M → R and Q 2 : R M → R to be

If the machine yields b * j > 0, then

because is the smallest machine-representable positive real number.

Next, we prove that (b * , d * ) is a minimizer of problem (P2) by contradiction.

Suppose that (b,d) is feasible for problem (P2) and Q 2 (b) < Q 2 (b * ). It follows from the constraints in (P2) that

Thus,b is feasible for problem (P1). It turns out that

contradicting the assumption that b * is a minimizer of problem (P1).

(ii) First, we show that b * is feasible for problem (P1). Since (b * , d * ) satisfies the constraints in (P2), we have

Thus, b * satisfies the constraints in (P1).

Next, we prove that b * is a minimizer of (P1) by contradiction. Suppose thatb is computationally feasible for problem (P1) and

Hence, (b,ď) is computationally feasible for problem (P2). It follows that

contradicting the assumption that (b * , d * ) is a minimizer of (P2).

Without loss of generality, let η t = 2

Bt log{M } t and B t+1 = max {B t , max c=1,...,M t,c } for every t ∈ N. We denote the Kullback-Leibler divergence of x = (x 1 , . . . , x M ) and z = (z 1 , . . . , z M ) by

where ri( M ) is the relative interior of probability M -simplex M ⊆ R M . For each t ∈ N, HECA outputs

The Bregman proximal inequality given in Lemma 3.1 of Teboulle (2018) implies that for any π ∈ M and t ∈ N, t , π t − π ≤ 1 η t [D(π π t ) − D(π π t+2 )] + t , π t − π t+2 − 1 η t D(π t+2 π t ).

In addition, by Pinsker's inequality, we have D(π π t ) ≥ 1 2 π − π t 2 1 , for all π ∈ M . It follows from the two inequalities above that for any π ∈ M and t ∈ N,

Since η t is decreasing in t and t ∞ ≤ B t+1 ,

for any T ∈ N. Substituting η t = 2

Bt log{M } t into the last inequality yields

We first establish the upper bound on R T for the case ofB T > B 1 . The right hand side of Inequality (A.1) is bounded above bȳ

Combining Inequalities (A.1)-(A.3), we obtain

Let us now move on to the remaining two cases. In both cases, B t = B 1 for all t. It follows that the right hand side of Inequality (A.1) is bounded above by 

We complete the proof by noting that B 1 = γ oBT and B 1 =B T correspond to the second and third case in the statement of this theorem, respectively.

A simple modification of the proof of Theorem 1 yields the results. 

Persistence in Forecasting Performance and Conditional Combination Strategies

The Combination of Forecasts

Regression Diagnostics: Identifying Influential Data and Sources of Collinearity

Machine Learning under a Modern Optimization Lens, Dynamic Ideas LLC

Best Subset Selection via a Modern Optimization Lens

The Challenge of Big Data and Data Science

Forecast Combination with Entry and Exit of Experts

Prediction, Learning, and Games

Online Learning Algorithms

of Handbook of Economic Forecasting

Best Subset Binary Prediction

Monetary Policy Rules and Macroeconomic Stability: Evidence and Some Theory

Combining Forecasts: A Review and Annotated Bibliography

Optimal Combination of Survey Forecasts

Integer Programming

Serial Correlation and the Combination of Forecasts

Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests

Comparing Predictive Accuracy

The Use of Prior Information in Forecast Combination

Machine Learning for Regularized Survey Forecast Combination: Partially-Egalitarian LASSO and its Derivatives

Economics in the Age of Big Data

Complete Subset Regressions

Economic Forecasting

A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting

Consistency and Cautious Fictitious Play

Combining Expert Forecasts: Can Anything Beat the Simple Average?

Tests of Conditional Predictive Ability

Improved Methods of Combining Forecasts

Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps

Numerical Methods in Economics

50 Years of Integer Programming 1958-2008: From the Early Years to the State-of-the-Art

Data ex Machina: Introduction to Big Data

Data Mining

Public Policy in an Uncertain World

of Handbook of Economic Forecasting

Monetary Policy Rules and Macroeconomic Stability: Some New Evidence

Time Series and Panel Data Econometrics

Statistical Learning for Big Dependent Data

Advances in Forecasting under Instability

Boosting: Foundations and Algorithms

A Simple Explanation of the Forecast Combination Puzzle

A Comparison of Linear and Nonlinear Univariate Models for Forecasting Macroeconomic Time Series

Combination Forecasts of Output Growth in a Seven-Country Data Set

of Handbook of Economic Forecasting, chap

Firm Performance and Macro Forecast Accuracy

A Simplified View of First Order Methods for Optimization

Regression Shrinkage and Selection via the Lasso

of Handbook of Economic Forecasting

Big Data: New Tricks for Econometrics

Robust Forecast Combinations

of Handbook of Economic Forecasting

Combining Forecasting Procedures: Some Theoretical Results

Table 3: Cumulative Forecasting Loss and Average Regret of HECA

Average Regret

Average Regret