Multicriteria Estimation via
Cost-Efficient Frontiers

Last Updated: 15 July 2007

Site Maintained By: Leigh Tesfatsion Professor of Economics and Mathematics Heady Hall 375 Iowa State University Ames, IA 50011-1070 Tel: (515) 294-7318 FAX: (515) 294-0221 http://www.econ.iastate.edu/tesfatsi/ tesfatsi AT iastate.edu

Table of Contents: Overview Research Summary Budget Line Analogy Publications Software & Manual Availability

Overview

Research Summary:

The theoretical relationships postulated for economic processes typically fall into four conceptually distinct categories: cross-sectional (simultaneous) relationships; dynamic relationships; measurement relationships; and probabilistic relationships. Inevitably, an actual economic process will behave in a manner that is incompatible to some degree with each of these postulated theoretical relationships. "All models are wrong, but some are useful." (GEP Box, 1979)

Consequently, the estimation of a model (postulated theory) conditional on a given data set would seem, intrinsically, to be a multicriteria optimization problem. Associated with any set of estimates for the theoretical variables will be a set of discrepancy terms reflecting the extent to which the various types of estimated theoretical relationships are incompatible with the given data. An econometrician undertaking the estimation would presumably prefer each type of discrepancy to be small. However, beyond a particular point a further decrease in one type of discrepancy comes at the cost of an increase in another.

In a series of studies (see below), Robert Kalaba and I address model estimation from a multicriteria perspective. In particular, as summarized in Kalaba and Tesfatsion (Comp. Statistics and Data Analysis, 1996, pdf, 1.6MB), we develop a model estimation framework that encompasses a broad range of views concerning the appropriate interpretation and treatment of theory-data discrepancy terms.

At one end of the range, we develop the Flexible Least Squares (FLS) approach for economic processes modeled in terms of dynamic and measurement relationships. For any given set of process data and any given model (theory) proposed to explain the data, each possible estimated form of the model generates two types of theory-data discrepancy terms, dynamic and measurement, that are separately aggregated into sums of residual squared errors RD and RM. The basic objective of the FLS approach is to constructively determine the "Residual Efficiency Frontier (REF)," the family of all model estimations that are equally efficient with respect to achieving minimal RD and RM. That is, given any model estimation along the REF, by construction there exists no other model estimation for which RD and RM are each at least as small and one is strictly smaller. Our program implementations for FLS and Generalized Flexible Least Squares (GFLS), an extended and more powerful version of FLS, have been incorporated into the statistical packages GAUSS and SHAZAM.

More generally, suppose a model (theory) has been conjectured as a possible explanation for a given set of process data, and suppose the modeler has specified a K-dimensional vector of theory-data incompatibility cost functions for measuring the degree of incompatibility between theory and data. We then propose the constructive determination of the "Cost-Efficient Frontier (CEF)," the family of model estimations that are equally efficient with respect to achieving minimal incompatibility costs.

We derive a recurrence relation for updating the CEF at time t+1 as a function of the CEF at time t together with a K-dimensional vector of incremental incompatibility costs associated with observations obtained during period t. We show that this CEF recurrence relation generalizes a number of other estimation methods. For example, the REF for FLS/GFLS is a special case of the CEF with K=2. Moreover, in the special limiting case in which joint probability assessments can be used to achieve a complete amalgamation of all theory-data discrepancy terms into a single (K=1) real-valued incompatibility cost function in the form of a posterior distribution, the CEF recurrence relation reduces to the Kalman filter.

Budget Line Analogy:

The construction of the Residual Efficiency Frontier (REF) for Flexible Least Squares -- and of the Cost-Efficient Frontier (CEF) for our more general proposed multicriteria estimation procedure -- can be compared with the familiar economic construction of a budget line as follows:

• The irreducible incompatibility between a given model (theory) and a given data set that must be distributed across discrepancy terms through model estimation corresponds to a given real income that must be distributed across the purchase of goods and services.

• Deciding on the precise form of discrepancy terms to be used for the representation of this theory-data incompatibility is akin to deciding on the precise form of goods and services to be used for the representation of real income.

• Researchers can disagree about the best way to go about this representation. Should discrepancy terms (goods and services) be represented in a highly aggregated way, e.g. as either dynamic or measurement misspecification errors (capital or consumption goods)? Or should these categories be further disaggregated? But this representational issue does not affect the objective existence of an irreducible theory-data incompatibility (real income).

• The choice of a particular model estimation along the REF/CEF, conditional on a given representation of the discrepancy terms (with corresponding costs) and on a given irreducible incompatibility between theory and data, is akin to selecting a consumption bundle along a budget line conditional on a given representation of goods and services (with corresponding real prices) and a given real income.

• By construction, starting at any model estimation along a REF/CEF, a decision to lower the measured size of any one type of discrepancy by moving to a different model estimation necessarily implies an increase in at least one other type of discrepancy. Similarly, starting at any point along a budget line, a decision to lower the purchase of any one good or service necessarily implies an increase in the purchase of at least one other good or service in order to remain on the budget line.

• A REF/CEF thus typically represents set-valued estimation in that it corresponds to a family of model estimations that are all equally efficient with regard to the minimalization of the discrepancies between a given model and a given data set. A researcher can use this family of model estimations to determine inferences that are retained along large portions of the frontier, i.e., that are supported by the data largely independently of the precise manner in which the irreducible incompatibility between theory and data is efficiently distributed across the discrepancy terms.

• Objective agreement can reasonably be sought for the construction of a REF/CEF, just as it can be sought for the construction of a budget line.

• But individual preferences will naturally enter into the choice of any one efficient model estimation along a REF/CEF, just as they must enter into the choice of any one efficient consumption bundle along a budget line.

• Selecting any one efficient model estimation along a REF/CEF is akin to a consumer bringing a utility function to bear on a budget line in order to select one "best" efficient consumption bundle. That is, the selection of any one efficient model estimation along a REF/CEF must involve the personal preferences of the researcher for one particular efficient distribution of discrepancies over another for his own particular reasons.

• To the extent that a group of researchers can agree on the relative importance of various types of discrepancies for an application at hand, they might be able to construct a "social welfare function" for picking out and using just one efficient model estimation along a REF/CEF. But this does not change the fact that any such singular choice will rely on the subjective judgements of the researchers concerning which efficient distribution of discrepancies is "best."

NOTE: It is with immense sadness I report that Robert E. Kalaba, a great scholar, mentor, colleague, and friend, died on September 29, 2004.

Publications

• Robert Kalaba and Leigh Tesfatsion (1996), "A Multicriteria Approach to Model Specification and Estimation" (pdf,1.6M), Computational Statistics and Data Analysis, Vol. 21, pp. 193-214. The published article is also available from Science Direct.
  Abstract: This study considers why multicriteria techniques have not been widely adopted in econometrics to date. It then reviews a multicriteria approach to sequential estimation developed earlier by the authors for which the basic objective is state filtering, i.e., sequentially estimating the sequence of states through which a process has passed as time proceeds and new process data is obtained. Suppose a model (theory) for a given process has been proposed, and the modeler has specified a vector of theory-data incompatibility cost functions for measuring the degree of incompatibility between the given model and observed process data. The multicriteria approach then involves the constructive determination and sequential updating over time of the Cost-Efficient Frontier, the family of all estimated forms of the model that are equally efficient with respect to achieving minimal theory-data incompatibility costs conditional on the current set of observed process data. This multicriteria filtering approach is shown to encompass Flexible Least Squares (FLS), Generalized Flexible Least Squares (GFLS), the Larsen-Peschon Filter, the Swerling Filter, the Viterbi Filter, and the Kalman Filter as special cases. Connections with Hendry and Richard's "general-to-specific" econometric methodology and Leamer's "information contract curve" (global sensitivity analysis) are also clarified.

  Robert Kalaba and Leigh Tesfatsion (1993), "A Multicriteria Approach to Dynamic Estimation", Chapter 18, pp. 288-300, in R. Day and P. Chen (eds.), Nonlinear Dynamics and Evolutionary Economics, Oxford University Press.

  Abstract: This chapter reviews work by the authors on multicriteria estimation.

  Robert Kalaba and Leigh Tesfatsion (1991), "A Unified Approach to Dynamic Estimation", Information Sciences Vols. 57-58 (September-December), pp. 159-169. The published article is available from Science Direct.

  Abstract: Discrepancies between assumed dynamical models and observations are often handled by making further probabilistic assumptions, a tactic which has both strengths and weaknesses. A re-examination of filtering and smoothing is conducted, and an alternative multicriteria approach, which is probability free, is advanced. This approach involves vector minimization as a key ingredient, and it specializes to the well-known Kalman, Viterbi, Larson-Peschon, and Swerling filters.

  Robert Kalaba and Leigh Tesfatsion (1990a), "Flexible Least Squares for Approximately Linear Systems (pdf,1.2M), IEEE Transactions on Systems, Man, and Cybernetics, Vol. 20, No. 5, pp. 978-989. The published article is also available from IEEE Xplore.

  Abstract: The problem of filtering and smoothing for a system described by approximately linear dynamic and measurement relations has been studied for many decades. Yet the potential problem of misspecified dynamics, which makes the usual probabilistic assumptions involving normality and independence questionable at best, has not received the attention it merits. A probability-free multicriteria “Generalized Flexible Least Squares (GFLS)” filter that meets this misspecification problem head on is proposed. A Fortran program implementation is provided for this filter, and references to simulation and empirical results are given. Although GFLS has close connections with the standard Kalman filter, it is concretely demonstrated that there are also important conceptual and computational distinctions. The Kalman filter provides a unique estimate for the state sequence, conditional on maintained probability assumptions for discrepancy terms. In contrast, the GFLS filter provides a family of state sequence estimates, each of which is vector-minimally incompatible with the prior dynamical and measurement specifications.

  NOTE: A program implementation for GFLS has been incorporated into the statistical package GAUSS. See the section on Software & Manual Availability below.

  Robert Kalaba and Leigh Tesfatsion (1990b), "A Further Note on Flexible Least Squares and Kalman Filtering", Journal of Economic Dynamics and Control, Vol. 14, No. 1 (February), 183-185. The published article is available from Science Direct.

  Abstract: The logical relationship between the multicriteria Flexible Least Squares (FLS) method and Kalman filtering is clarified.

  Leigh Tesfatsion and John Veitch (1990), "U.S. Money Demand Instability: A Flexible Least Squares Approach" (pdf,1.4M), Journal of Economic Dynamics and Control, Vol. 14, No. 1 (February), pp. 151-173. The published article is also available from Science Direct.

  Abstract: This article uses the Flexible Least Squares (FLS) method to investigate coefficient stability for the Goldfeld U.S. money demand model over the volatile period 1959:Q2 to 1985:Q3. The only constraint imposed on coefficient variation over time is a smoothness prior. Nevertheless, the time paths traced out by the FLS coefficient estimates exhibit systematic idiosyncratic time variations as well as simultaneous shift movements in 1974 during the time of the first oil price shock. Moreover, the FLS estimates also indicate that the "unit root" nonstationarity problem reported by OLS money demand studies disappears if the coefficients are allowed to exhibit even small amounts of time variation.

  Robert Kalaba and Leigh Tesfatsion (1990c), "An Organizing Principle for Dynamic Estimation", Journal of Optimization Theory and Applications, Vol. 64, No. 3 (March 1990), 445-470. The published article is available from SpringerLink.

  Abstract: This paper develops a general multicriteria framework for the sequential estimation of process states. Three well-known state estimation algorithms (the Viterbi, Larson-Peschon, and Kalman filters) are derived as monocriterion specializations. The multicriteria estimation framework is used to clarify both Bayesian and classical statistical procedures for treating potential model specification errors. A recently developed bicriteria specialization (flexible least cost), explicitly designed to take specification errors into account, is also reviewed. The latter application suggests how the multicriteria framework might be used to construct estimation algorithms capable of handling disparate sources of information coherently and systematically, without forced scalarization.

  Robert Kalaba and Leigh Tesfatsion (1989a), "Time-Varying Linear Regression Via Flexible Least Squares" (pdf,2.4M), Computers and Mathematics with Applications, Vol. 17 (1989), pp. 1215-1245. The published article is available from Science Direct.

  Abstract: This article develops in detail a multicriteria Flexible Least Squares (FLS) method for time-varying linear regression. The basic FLS objective is to determine the "residual efficiency frontier," that is, the set of all coefficient trajectory estimates that yield vector-minimal sums of squared residual measurement and dynamic errors conditional on a given set of observations.

  NOTE: A program implementation for FLS has been incorporated into the statistical packages GAUSS and SHAZAM. See the section on Software & Manual Availability below.

  Robert Kalaba and Leigh Tesfatsion (1988a), "The Flexible Least Squares Approach to Time-Varying Linear Regression" (pdf,360K), Journal of Economic Dynamics and Control, Vol. 12, No. 1 (March), 43-48. The published article is available from Science Direct.

  Abstract: This paper proposes a Flexible Least Squares (FLS) method for state estimation when the dynamic equations are unknown but the process state evolves only slowly over time. A smoothness prior is introduced in place of an explicit specification for the unknown dynamic equations governing the evolution of the process state. Simulation experiments illustrating the method are presented.

  Robert Kalaba and Leigh Tesfatsion (1989b), "Sequential Nonlinear Estimation with Nonaugmented Priors", Journal of Optimization Theory and Applications, Vol. 60, No. 3 (March), pp. 421-438. The published article is available from SpringerLink.

  Abstract: A "flexible least cost method" is proposed for investigating the basic compatibility of theory and observations in the absence of valid or known stochastic characterizations for residual error terms.

  Robert Kalaba and Leigh Tesfatsion (1988b), "Exact Sequential Filtering, Smoothing, and Prediction for Nonlinear Systems", Nonlinear Analysis, Vol. 12, Number 6 (June), 599-615. The published article is available from Science Direct.

  Abstract: This article develops two algorithms for the exact sequential updating of the optimal solution for a general discrete-time nonlinear least squares estimation problem as the process length increases and new observations are obtained. One algorithm proceeds via an imbedding on the process length and the final state vector. The second algorithm proceeds via an imbedding on the process length and the final observation vector. Each algorithm generates optimal (least cost) filtered and smoothed state estimates, together with optimal one-step-ahead state predictions.

  Robert Kalaba, Karl Spingarn, and Leigh Tesfatsion (1981), "A Sequential Method for Nonlinear Filtering: Numerical Implementation and Comparisons", Journal of Optimization Theory and Applications, Vol. 34, No. 4 (August), pp. 1144-1149. The published article is available from SpringerLink.

  Abstract: Exact equations are presented for sequentially updating the optimal solution for a discrete-time analog of the basic Sridhar nonlinear filtering problem as the process length increases and new observations are obtained. A tabular method is described for implementing numerically the sequential filtering equations. The accuracy and efficiency of the tabular method are illustrated by means of several numerical examples.

  Robert Kalaba and Leigh Tesfatsion (1981), "An Exact Sequential solution Procedure for a Class of Discrete-Time Nonlinear Estimation Problems" (pdf,595K), IEEE Transactions on Automatic control, Vol. AC-26, pp. 1144-1149. The published article is available from IEEE Xplore.

  Abstract: This article develops an exact procedure for sequentially updating the optimal solution for a general discrete-time nonlinear least squares estimation problem as the process length increases and new observations are obtained.

  Robert Kalaba and Leigh Tesfatsion (1980), "A Least-Squares Model Specification Test for a Class of Dynamic Nonlinear Economic Models with Systematically Varying Parameters", Journal of Optimization Theory and Applications, Vol. 32, No. 4 (December), pp. 537-567. The published article is available from SpringerLink.

  Abstract: This study develops a least-squares measure for simultaneously testing the basic compatibility of prior dynamical, observational, and distributional model specifications against actual data for a class of dynamic nonlinear economic models with parameters explicitly modeled as nonlinear functions of endogenous and exogenous variables. Using invariant imbedding techniques, an algorithm is derived for sequentially updating the optimal least-squares estimates for parameters, endogenous variables, and squared residual modeling error sums as the duration of the process increases and new observations are obtained.

Software & Manual Availability

• FLS Program:

  The Flexible Least Squares (FLS) method is implemented by the FLS Program (html,34K), a Fortran program developed by Kalaba and Tesfatsion (the copyright holders) which is released here as free open-source software under the terms of the Artistic License Agreement (html). The FLS program is also included in the statistical packages SHAZAM and GAUSS - TSM (Time Series Methods).

• FLS Manual:

  Robert Kalaba and Leigh Tesfatsion, "Time-Varying Linear Regression Via Flexible Least Squares" (pdf,2.4M), Computers and Mathematics with Applications Vol. 17 (1989), pp. 1215-1245. The published article is available from Science Direct.

• GFLS Program:

  The Generalized Flexible Least Squares (GFLS) method is implemented by the GFLS Program (html,46K), a Fortran program developed by Kalaba and Tesfatsion (the copyright holders) which is released here as free open-source software under the terms of the Artistic License Agreement. A Help File (html) for the GFLS program is available. The GFLS program is also included in the statistical package GAUSS - TSM (Time Series Methods).

• GFLS Manual:

  Robert Kalaba and Leigh Tesfatsion, "Flexible Least Squares for Approximately Linear Systems" (pdf,1.2MB), IEEE Transactions on Systems, Man, and Cybernetics, Vol. 20, No. 5 (1990), pp. 978-989