The AMES Wholesale Power Market Test Bed

A Free Open-Source Computational Laboratory
for the Agent-Based Modeling of Electricity Systems

Last Updated: 23 June 2009 Site maintained by: Leigh Tesfatsion Professor of Econ, Math, and ECpE Iowa State University Ames, Iowa 50011-1070 http://www.econ.iastate.edu/tesfatsi/ tesfatsi AT iastate.edu Electricity Market Open-Source Software IEEE Task Force on Open-Source Software Agent-Based Computational Economics (ACE) ACE Electricity Research ISU Electric Energy Economics (E3) Group

Software Release Disclaimer:

The AMES Market Package is our software implementation, in Java, of the AMES Wholesale Power Market Test Bed. This software, provided below, is unsupported and provided as-is, without warranty of any kind.

Table of Contents:

• Software Overview
• Software Features
• Software Downloads and Supporting Materials
  • Version Release History
  • Latest Version
    • Version Download
    • Development Software
    • Set-Up Illustration for the NetBeans IDE
    • Manuals and Tutorials
• Licensing Terms
• Publications and References
• Acknowledgements

Software Overview

The wholesale power market design proposed by the U.S. Federal Energy Regulatory Commission (FERC) in an April 2003 white paper [FERC 2003] encompasses the following core features:

• central management by an independent market operator;
• a two-settlement system consisting of a day-ahead market supported by a parallel real-time market to ensure continual balancing of supply and demand for power;
• management of grid congestion by means of locational marginal pricing (LMP), i.e., the pricing of power by the location and timing of its injection into, or withdrawal from, the transmission grid;
• market power oversight and mitigation.

Versions of FERC’s wholesale power market design have been implemented (or scheduled for implementation) in U.S. energy regions in the Midwest (MISO), New England (ISO-NE), New York (NYISO), the mid-Atlantic states (PJM), California (CAISO), the southwest (SPP), and Texas (ERCOT). Nevertheless, strong criticism of the design persists. Part of this criticism stems from the concerns of non-adopters about the suitability of the design for their regions due to distinct local conditions (e.g., hydroelectric power in the northwest). Even in regions adopting the design, however, criticisms continue to be raised about market performance.

One key problem for participants in wholesale power markets restructured in accordance with FERC’s design is a lack of full transparency regarding market operations. Due in great part to the complexity of the market design in its various actual implementations, the business practices manuals and other public documents released by market operators are daunting to read and difficult to comprehend. Moreover, in many energy regions (e.g., MISO), data is only posted in partial and masked form with a significant time delay. The result is that many participants are wary regarding the efficiency, reliability, and fairness of market protocols (e.g., settlement practices and market power mitigation rules). Moreover, outsiders (e.g., university researchers) are hindered from subjecting the design to systematic testing in an open and impartial manner.

As elaborated in Sun and Tesfatsion (2007a), Sun and Tesfatsion (2007b), and Li, Sun, and Tesfatsion (2009), the AMES Wholesale Power Market Test Bed is being developed as a “simple but not too simple” computational laboratory for the systematic experimental study of wholesale power markets restructured in accordance with FERC's market design. AMES is an acronym for Agent-based Modeling of Electricity Systems.

Our objective is the facilitation of research, teaching, and training, not commercial-grade application. The release of AMES as an open-source package is intended to encourage the cumulative development of this test bed by others (as well as ourselves) in directions appropriate for their specific needs.

The release of AMES is also intended to encourage continual dialog with market stakeholders and regulators leading to successive refinements and improvements of the test bed. To further these purposes, AMES has been constructed (in Java) to have an extensible modular architecture and an easily-navigated graphical user interface (GUI).

The following section discusses the core features of AMES in greater detail.

Software Features

The latest version release AMES(2.02) of the AMES Market Package incorporates, in simplified form, core features of FERC's proposed wholesale power market design. Below is a summary description of the logical flow of events in the AMES wholesale power market as currently implemented:

• The AMES wholesale power market operates over an AC transmission grid starting on day 1 and continuing through a user-specified maximum day (unless terminated earlier in accordance with a user-specified stopping rule). Each day D consists of 24 successive hours H = 00,01, ...,23.

• The AMES wholesale power market includes an Independent System Operator (ISO) and a collection of energy traders consisting of Load-Serving Entities (LSEs) and Generation Companies (GenCos) distributed across the buses of the transmission grid. Each of these entities is implemented as a software program encapsulating both methods and data.

• The objective of the ISO is the reliable attainment of appropriately constrained operational efficiency for the wholesale power market, i.e., the maximization of the total net earnings of buyers and sellers subject to generation and transmission constraints. In an attempt to attain this objective, the ISO undertakes the daily operation of a day-ahead market settled by means of locational marginal pricing (LMP), i.e., the determination of prices for electric power in accordance with both the locating and timing of its injection into, or withdrawal from, the transmission grid.

• The objective of each LSE is to secure power for its downstream (retail) customers. During the morning of each day D, each LSE reports a demand bid to the ISO for the day-ahead market for day D+1. Each demand bid consists of two parts: a fixed demand bid (i.e., a 24-hour load profile); and 24 price-sensitive demand bids (one for each hour), each consisting of a linear demand function defined over a purchase capacity interval. LSEs have no learning capabilities; LSE demand bids are user-specified at the beginning of each simulation run.

• The objective of each GenCo i is to secure for itself the highest possible net earnings each day. During the morning of each day D, each GenCo i uses its current action choice probabilities to choose a supply offer from its action domain ADi to report to the ISO for use in all 24 hours of the day-ahead market for day D+1. Each supply offer in ADi consists of a linear marginal cost function defined over an operating capacity interval. GenCo i's ability to vary its choice of a supply offer from ADi permits it to adjust the ordinate/slope of its reported marginal cost function and/or the upper limit of its reported operating capacity interval in an attempt to increase its daily net earnings.

• After receiving demand bids from LSEs and supply offers from GenCos during the morning of day D, the ISO determines and publicly reports hourly power supply commitments and LMPs for the day-ahead market for day D+1 as the solution to hourly bid/offer-based DC optimal power flow (DC-OPF) problems. Transmission grid congestion is managed by the inclusion of congestion cost components in LMPs.

• At the end of each day D, the ISO settles all of the commitments for the day-ahead market for day D+1 on the basis of the LMPs for the day-ahead market for day D+1.

• At the end of each day D, each GenCo i uses stochastic reinforcement learning to upate the action choice probabilities currently assigned to the supply offers in its action domain ADi, taking into account its day-D settlement payment.

• There are no system disturbances (e.g., weather changes) or shocks (e.g., forced generation outages or line outages). Consequently, the binding financial contracts determined in the day-ahead market are carried out as planned and traders have no need to engage in real-time (spot) market trading.

• Each LSE and GenCo has an initial holding of money that changes over time as it accumulates earnings and losses.

• There is no entry of traders into, or exit of traders from, the wholesale power market. LSEs and GenCos are currently allowed to go into debt (negative money holdings) without penalty or forced exit.

The essential idea of stochastic reinforcement learning is that the probability of choosing an action should be increased (reinforced) if the corresponding reward is relatively good and decreased if the corresponding reward is relatively poor. Each AMES GenCo determines its supply offer choices for the day-ahead market by means of VRE reinforcement learning, a variant of a stochastic reinforcement learning algorithm developed by Alvin Roth and Ido Erev on the basis of human subject experiments. The user can select either "daily profit" (revenues minus total cost) or "daily net earnings" (revenues minus total variable cost) as the measure for each GenCo's daily "reward." The user can also tailor the settings of each GenCo's learning parameter values to its situation, in particular to its cost attributes, its operating capacity limits, and its anticipated net earnings. Each GenCo's VRE learning is implemented by means of a free open-source Java Reinforcement Learning Module (JReLM) developed by Gieseler (2005).

At the beginning of each run with learning GenCos, a competitive equilibrium benchmark is first calculated off line in which the GenCos' true cost and capacity attributes are used to solve for LMPs and power commitments. Comparing subsequent market outcomes under learning with competitive equilibrium benchmark outcomes permits the calculation of standard market performance measures such as market efficiency and market power.

The ISO determines hourly power-supply commitments and LMPs for the day-ahead market by solving hourly bid/offer-based DC optimal power flow (DC-OPF) problems that approximate underlying AC-OPF problems. The ISO solves its DC-OPF problems by invoking an accurate and efficient DC-OPF solver, DCOPFJ, incorporated into AMES(V2.02). Developed in Java by Sun and Tesfatsion (2007c), the DCOPFJ package is free open-source software that can be used either as part of a Java application or as a stand-alone DC-OPF solver. Specifically, AMES(V2.02) incorporates DCOPFJ(V2.0) as the ISO's solver for DC optimal power flow problems. DCOPFJ(V2.0) is a generalization of DCOPFJ(V1.0) that permits LSEs to submit price-sensitive as well as fixed demands to the ISO for the day-ahead market.

The ISO is concerned about loss of operational efficiency due to the possible exercise of "market power" by GenCos through strategic reporting of supply offers. Specifically, a GenCo has market power if the GenCo can increase its net earnings (relative to the competitive benchmark) either by reporting a higher-than-true marginal cost function or by reporting a less-than-true upper operating capacity limit. As one possible approach to GenCo market power mitigation, the ISO can impose a supply-offer price cap (PCap). Under such a policy, the maximum sale price reported by any GenCo cannot exceed PCap.

The user can control the length of each simulation run by choosing to set (or not) any combination of the following five stopping rules: (1) stop when a specified maximum day is reached; (2) stop when each GenCo is choosing a single supply offer with a probability that exceeds a user-specified theshold probability; (3) stop when the probability distribution used by each GenCo to select its supply offers has stabilized to within a user-specified threshold for a user-specified number of days; (4) stop when the supply offer selected by each GenCo has stabilized to within a user-specified threshold for a user-specified number of days; and/or (5) stop when the net earnings of each GenCo have stabilized to within a user-specified threshold for a user-specified number of days. When multiple stopping rules are flagged, the simulation run terminates as soon as any one of the flagged stopping rules is satisfied.

AMES(V2.02) has a graphical user interface (GUI) with separate screens for carrying out the following functions: (a) creation, modification, analysis and storage of case studies in either single-run and batched-run mode; (b) initialization and editing of the attributes of the transmission grid; (c) individualized initialization and editing of the attributes of Load-Serving Entities (LSEs) and GenCos; (d) individualized specification of parameter values for the learning method of each GenCo; (e) activation (or not) of a user-specified value for an ISO-imposed supply-offer price cap; (f) specification of simulation controls (e.g., stopping rules); and (f) customizable output reports in the form of both table and chart displays.

Finally, AMES(V2.02) includes two test cases that can be used as templates for new case studies. The first test case is a dynamic extension of a static 5-bus example by John Lally (2002, Section 6) now used extensively in ISO-NE/PJM training manuals. The second test case is a dynamic extension of the modified IEEE 30-node bus system developed by Shahidehpour et al. (2002) in an appendix (Section D.4, pp. 477-478).

It is hoped that the free open-source release of AMES(V2.02) will encourage the cumulative development of future versions with enhanced features critical for determining the performance of real-world restructured electricity markets. Examples of such enhanced features include:

• possibility of shocks to the system leading to differences arising between day D-1 financial contracts and day D required transactions that must be settled in the day-D real-time market at real-time LMPs (active two-settlement system).
• enhanced transmission grid features.
• incorporation of an AC OPF solver to permit DC vs. AC OPF error comparisons.
• enhanced modeling of ISO-managed unit commitment taking into account start-up costs, down-time constraints, and ramping constraints.
• security constraints incorporated into DC/AC OPF problem formulations as a hedge against system disturbances.
• ISO-managed resource adequacy assessment.
• emission constraints and other mandated environmental protection measures.
• upstream fuel markets permitting more empirically-based derivations of cost functions for GenCos.
• incorporation of demand-bid learning capabilities for LSEs.
• additional types of learning methods for potential use by GenCos and LSEs (e.g. anticipatory learning).
• inclusion of bankruptcy rules to handle situations in which one or more traders use up all of their liquid assets.
• a financial transmission rights market to permit hedging against transmission congestion costs.
• bilateral trading to permit longer-term contracting.
• downstream retail markets permitting more empirically-based derivations of LSE demand bids [cf. Widergren, Sun, and Tesfatsion (2006)].

Software Downloads and Supporting Materials

Detailed instructions are provided below for downloading, compiling, and running AMES(V2.02). Explanations of the modifications incorporated into successive versions released to date can be obtained at the Version Release History Site.

AMES Market Package--Version 2.02 (Li, Sun, and Tesfatsion)

• Version Download

  Hongyan Li, Junjie Sun, and Leigh Tesfatsion, AMES Market Package--Version 2.02 (zipfile,3.5M). Release Date: 20 September 2008

• Development Software

  The following free software was used in the development of AMES(V2.02).

  1. Java platform: For Java SE Development Kit (JDK) 6 update 1 (6u1) or higher, visit the Java SE Downloads Page. You can download JDK 6u1 (or higher) either alone or in combination with the NetBeans Integrated Development Environment (IDE) 6.0 (or higher).
  2. Java IDE: For NetBeans IDE 6.0 (or higher), visit the NetBeans Homepage. You can also download the Java SE Development Kit (JDK) 6u1 (or higher) with the NetBeans IDE from the linked download page. The NetBeans IDE is a powerful open-source cross-platform tool for Java programming.
  3. Java Chart Library: JFreeChart (Download Page).
  4. Repast J: A Java Agent-Based Toolkit, Version 3.1 (Download page). For an on-line self-study guide for Repast J and Java, visit here.
  5. JReLM (Java Reinforcement Learning Module): Open-source software for use with Repast J developed by Charles Gieseler. For information, visit here.
  6. DCOPFJ (DC-OPF Solver in Java): A free open-source solver for DC optimal power flow problems. For software downloads and manual materials, visit here.

• Set-Up Illustration for the NetBeans IDE

  • The first step is to install JDK 6u1 (or higher) either separately or in combination with the NetBeans IDE (6.0 or higher). Please note that JDK 6u1 (or higher) is required for the AMES(V2.02) code to run correctly. Error messages will be generated if you attempt to compile the AMES(V2.02) code with any earlier JDK release.

  • The second step is to install the NetBeans IDE (6.0 or higher) if you have not already done so in step one.

  • The final step is then to use the NetBeans IDE to create a standard NetBeans project using the contents of the data ("DATA") directory, required library ("lib") directory, and source code ("src") directory extracted from the above AMES(V2.02) zip file. All Java archive ("jar") files in the lib directory extracted from the AMES(V2.02) zip file must be included in the required library for your AMES(V2.02) project in order for the AMES(V2.02) code to run correctly.

  • Note in particular that a jar file for Repast J 3.1 (repast.jar) is included in the lib directory extracted from the AMES(V2.02) zip file. Consequently, Repast J 3.1 does not have to be separately downloaded and installed unless you are planning to undertake code development for parts of the AMES(V2.02) code involving Repast J and you would like to have access to RePast J debugging facilities.

  • Detailed step-by-step instructions for setting up and running AMES(V2.02) as a standard NetBeans project using the NetBeans IDE (6.0 or higher) can be obtained here (pdf,481K).

  • After your AMES(V2.02) project compiles, you can use entries in appropriate AMES GUI screens to load and run the provided 5-bus and 30-bus test cases, to experiment with changes in the parameter settings for these test cases, and/or to develop and run new cases. Event handlers are now in place to handle problematic parameter settings resulting in "inadequacy events" (supply less than demand) for some hours of the day-ahead market. Nevertheless, care must still be taken because empirically implausible data entries will result in empirically implausible outcomes ("garbage in, garbage out").

• Manuals and Tutorials

  • Basic AMES Manual (pdf,481K)

    Abstract: Topics covered in this basic manual include: Basic Project Set-Up Info; Loading and Viewing AMES Test Cases; Development of New AMES Test Cases; AMES Source Code Modification; Running AMES Experiments in Batch Mode.

  • "The AMES Wholesale Power Market Project" (ppt,991K)

    Abstract: This slide presentation giving a summary overview of the AMES Wholesale Power Market Test Bed together with illustrative experimental results.

  • Hongyan Li, Junjie Sun, and Leigh Tesfatsion, "Separation and Volatility of Locational Marginal Prices in Restructured Wholesale Power Markets" (download site), ISU Economics Working Paper #09009, June 2009.
    Abstract: This study uses the AMES test bed to investigate separation and volatility of locational marginal prices (LMPs) in an ISO-managed restructured wholesale power market operating over an AC transmission grid. Particular attention is focused on the dynamic and cross-sectional response of LMPs to systematic changes in demand-bid price sensitivities and supply-offer price cap levels under varied learning specifications for the generation companies. Also explored is the extent to which the supply offers of the marginal (price-determining) generation companies induce correlations among neighboring LMPs.

    Leigh Tesfatsion (2008), "The AMES Wholesale Power Market Test Bed as a Stochastic Dynamic State-Space Game" (pdf,2.8M), Working Paper, Economics Department, Iowa State University, July 2008.

    Abstract: These notes show how the AMES Wholesale Power Market Test Bed can be recast in more standard state-space equation form. The result is a highly nonlinear and highly coupled system of first-order stochastic difference equations. The AMES state-space equation representation is used to explain how AMES constitutes an open-ended dynamic game among multiple strategically-learning players. It is also used to explain how AMES permits the development and experimental study of a wide variety of test cases.

Licensing Terms

AMES(V2.02) is licensed by the copyright holders (Hongyan Li, Junjie Sun, and Leigh Tesfatsion) as free open-source software under the terms of the GNU General Public License (GPL). Anyone who is interested is allowed to view, modify, and/or improve upon the code used to produce this package, but any software generated using all or part of this code must be released as free open-source software in turn. The GNU GPL can be viewed in its entirety here.

Publications and References

• FERC (2003), Notice of White Paper, U.S. Federal Energy Regulatory Commission, April.

• Charles Gieseler (2005), "A Java Reinforcement Learning Module for the Repast Toolkit: Facilitating Study and Implementation with Reinforcement Learning in Social Science Multi-Agent Simulations" (pdf,1.1M), (ppt,1.1M), Department of Computer Science, Iowa State University, M.S. Thesis.

• Deddy Koesrindartoto and Leigh Tesfatsion (2004), "Testing the Reliability of FERC's Wholesale Power Market Platform: An Agent-Based Computational Economics Approach" (pdf,45K), Energy, Environment, and Economics in a New Era, Proceedings of the 24th USAEE/IAEE North American Conference, Washington, D.C., July 8-10.

• Deddy Koesrindartoto, Junie Sun, and Leigh Tesfatsion (2005), "An Agent-Based Computational Laboratory for Testing the Economic Reliability of Wholesale Power Market Designs" (pdf,112K), Proceedings, Vol. 1, IEEE Power Engineering Society General Meeting, San Francisco, California, June 12-16, pp. 931-936.

• John Lally (2002), "Financial Transmission Rights: Auction Example," in Financial Transmission Rights Draft 01-10-02, m-06 ed., ISO New England, Inc., January.

• Hongyan Li and Leigh Tesfatsion (2009), "Development of Open Source Software for Power Market Research: The AMES Test Bed" (pdf preprint,601K), Journal of Energy Markets, Vol. 2, No. 2, Summer 2009.

  Abstract: This study discusses potential benefits and drawbacks of developing open-source software for power market research, using the AMES Wholesale Power Market Test Bed for concrete illustration.

• Hongyan Li and Leigh Tesfatsion (2009), "Capacity Withholding in Restructured Wholesale Power Markets: An Agent-Based Test Bed Study" (pdf,2.3M), IEEE Power Systems & Exposition Conference, Seattle, WA, March 15-18, 2009.

  Abstract: This study illustrates how AMES GenCos can learn to exercise market power through both economic and physical capacity withholding.

• Hongyan Li, Junjie Sun, and Leigh Tesfatsion (2009),Hongyan Li, Junjie Sun, and Leigh Tesfatsion, "Separation and Volatility of Locational Marginal Prices in Restructured Wholesale Power Markets" (download site), ISU Economics Working Paper #09009, June 2009.
  Abstract: This study uses the AMES test bed to investigate separation and volatility of locational marginal prices (LMPs) in an ISO-managed restructured wholesale power market operating over an AC transmission grid. Particular attention is focused on the dynamic and cross-sectional response of LMPs to systematic changes in demand-bid price sensitivities and supply-offer price cap levels under varied learning specifications for the generation companies. Also explored is the extent to which the supply offers of the marginal (price-determining) generation companies induce correlations among neighboring LMPs.

  Hongyan Li, Junjie Sun, and Leigh Tesfatsion (2008), "Dynamic LMP Response Under Alternative Price-Cap and Price-Sensitive Demand Scenarios" (pdf,465K), Proceedings, IEEE Power Engineering Society General Meetings, Carnegie-Mellon University, Pittsburgh, July 20-24.

  Abstract: This study provides preliminary experiments illustrating how price-sensitive demand and ISO-imposed supply-offer price caps are simulated in AMES.

  Mohammed Shahidehpour, Hatim Yamin, and Zuyi Li (2002), Market Operations in Electric Power Systems, IEEE, Wiley-Interscience.

  Abhishek Somani and Leigh Tesfatsion, "An Agent-Based Test Bed Study of Wholesale Power Market Performance Measures" (pdf,2.8M), IEEE Computational Intelligence Magazine, Volume 3, Number 4, November 2008, pages 56-72.

  Junjie Sun and Leigh Tesfatsion (2007a), "Dynamic Testing of Wholesale Power Market Designs: An Open-Source Agent-Based Framework", Computational Economics, Volume 30, Number 3, pp. 291-327.

  Note: This article is an abridged version of ISU Economics Working Paper No. 06025 (pdf,2.2MB), revised July.

  Abstract: This working paper provides a detailed description of the AMES Wholesale Power Market Test Bed (Version 1.0) together with illustrative experimental findings. An abridged version of this working paper is published in Computational Economics 30(3), 2007, pp. 291-327.

  Junjie Sun and Leigh Tesfatsion (2007b), "An Agent-Based Computational Laboratory for Wholesale Power Market Design" (pdf,724K), Proceedings, IEEE Power Engineering Systems General Meeting, Tampa, Florida, June.

  Note: This proceedings paper is a brief summary of the previous Computational Economics article.

  Junjie Sun and Leigh Tesfatsion (2007c), "DC Optimal Power Flow Formulation and Testing Using QuadProgJ" (pdf,543K), ISU Economics Working Paper No. 06014, Department of Economics, Iowa State University, revised July.

  Abstract: This working paper provides a detailed description of a key AMES(V1.0) module, the DCOPFJ solver used by the AMES ISO to solve DC optimal power flow (OPF) problems. An abridged version of this working paper is published in the Proceedings, IEEE Power Engineering Society General Meeting, Tampa, Florida, June 2007.

  Junjie Sun and Leigh Tesfatsion (2007d), "Open-Source Software for Power Industry Research, Teaching, and Training: A DC-OPF Illustration" (pdf,115K), Proceedings, IEEE Power Engineering Systems General Meeting, Tampa, Florida, June.

  Note: This proceedings paper is a brief summary of ISU Economics Working Paper No. 06014 (see above).

  Steven Widergren, Junjie Sun, and Leigh Tesfatsion (2006), "Market Design Test Environments" (pdf,136K), Proceedings, IEEE Power Engineering Society General Meeting, Montreal, June.

Acknowledgements

The work reported at this site has been supported in part by Grant NSF-0527460 awarded by the National Science Foundation and by grants awarded by the ISU Electric Power Research Center.