A UtiliPoint IssueAlert
Patrick Reames - VP, CommodityPoint

I recently had the opportunity to participate in an industry panel discussion exploring various issues in the energy trading and risk management space. While the questions poised to the panel varied, the majority kept returning to issues surrounding managing the flood of data and information pouring into the trading floor; and more specifically sorting, analyzing, and being able to present it in a more consumable form to those on the floor that needed the information in order to more effectively do their jobs.

On today’s trading floors, information is arriving in greater quantities, from more sources, and at greater velocity. Trading floors are now receiving real-time or near real-time data in the form of price feeds, trading exchange feeds, and operational data from production facilities, pipelines, transmission operators, shipping companies, and even data aggregators, such as Bentek. Wholesale trading in this environment is no longer a function of “dialing for dollars” - making a whole lot of calls and trying to find the best price - it’s about being able to quickly sort through all that data, make a judgment as to what it all means, develop a strategy to take advantage of that judgment, and executing on that strategy with the appropriate counterparties.

This increased availability of data, now within the reach of virtually all traders, is one of (if not THE) biggest changes in wholesale energy trading over the last decade. This increased visibility into market conditions has fundamentally changed the nature of the business in that arbitrage opportunities are smaller in magnitude and last for a fraction of the time that they did in the past. The daily battle is now won or lost based upon how quickly an individual or desk can sort through and consume all that data arriving from the multitude of channels and, ultimately, turn it to their advantage.

New Technologies, but Limitations Continue
On today’s trading floors, the energy trading and risk management (ETRM) systems have become the nexus of all that information, with the leading systems having evolved from simply capturing and sorting deals and transactions after the fact, to being a hub of information necessary to facilitate decision making. Today’s modern ETRM systems are capable of connecting to price feeds, on-line exchanges and energy markets; however, merely connecting to, and capturing, that data is insufficient. Despite the advanced service oriented architectures (SOA) offered by many of the modern ETRM systems, the ability of these systems to make full use of the relevant data and to present it in an easily consumable manner has still not advanced to fully meet the needs of today’s fast paced trading environments.

Despite advances in technology over the last decade, it has proven very difficult for ETRM vendors to blaze new trails in advanced simulations and data visualization. While the hardware available is much more capable, with fast multi-processor computers now standard to all trading shops, the software has lagged behind in taking advantage of those capabilities. ETRM vendor business models do not support revolutionary advances, as most clients demand incremental changes to the software in order to provide manageable upgrade paths from one version to the next, so rather than starting from scratch with an entirely new architecture that better facilities faster processing times and a flexible and rich presentation of information, software developers must generally operate within the constraints of architectures designed to model complex transactions and business processes, and to ensure transactional integrity.

Data servers, a relatively new category of software that sits between many of the data services and an ETRM system, have gained popularity in recent years as traders have become more aware of their capabilities. These systems, from vendors such as ZEMA and Logical Information Machines, provide the ability to compile data from multiple sources (like price feeds, ISOs, and exchanges), create unique pricing curves, and analyze data visually. However, while these systems can provide a sieve for much of the data arriving on the trading floor, making that data more accessible and consumable to traders, and interfacing to ETRM systems in order to pass that information into the system, they are not considered tools for visualizing the impacts of market or price movements on a portfolio of trades, something that can practically be accomplished only within the ETRM system itself. However, experience has shown that ETRM systems are generally not great simulation engines, with stochastic analytics, such as Monte Carlo VAR, requiring several hours to calculate.

CommodityPoint research indicates that more and more traders are starting to look toward these stochastic analytic methods. However, our experience is that traditional ETRM architectures are limited in their ability to perform the massive numbers of calculations necessary--for example, it’s not uncommon for a Monte Carlo VAR calculation to run tens, if not hundreds, of millions of calculations on a multi-year portfolio comprised of thousands of trades; certainly not an unusual trading book in this market. Some ETRM vendors, including Allegro, Triple Point and OpenLink, have turned to grid-computing to help distribute the computation load across the many machines connected to the network. Others, who have recently entered the marketplace, have developed their systems around the ability to rapidly perform portfolio simulations; however, these recently released systems, such as those from Abacus or HyperRig, don’t provide the breadth of operational functionality, such as scheduling and accounting, as do the larger vendors.

Most of the commercial ETRM systems provide some data visualization capability, similar to that available in an Excel spreadsheet, allowing price curve, volumetric information or some standard analytics, such as value at risk (VAR) to be graphically displayed. However, with the exception of these limited capabilities, today’s ETRM systems provide the vast majority of their data compiled in tabular form and displayed in rows and columns, much like a spreadsheet.

What Will the Future Look Like?
Many of the system vendors are constantly working to improve their ability to run multi-variable portfolio simulations, having recognized this area as a solid competitive advantage in the marketplace; and those new market entrants whose systems are built around simulation capabilities are working to strengthen their offerings in terms of operational functionality. Still, addressing the complex issues surrounding data management, data analysis and data visualization, while simultaneously providing “system of record” capabilities, will probably require additional technologies that are not yet in those vendors’ toolkits.

In the area of data simulation, the SOA architectures in place do provide part of the longer term solution in that they are well suited to passing data in and out of the application. However, as previously noted, for calculative intensive operations, they are not ideal. The most likely scenario for enabling real-time or near real-time stochastic analytics will probably involve data marts, smaller databases constantly updated via the primary ETRM system, and containing only the necessary deal data and pricing necessary to perform complex analytics. The computational engines will sit outside the primary ETRM system, running on dedicated high performance machines, architecturally optimized for intensive calculations.

Improvements in data visualization will also play a large role part in facilitating better and more rapid decision making. Humans consume information better when presented in an analog form--that is we are quicker to identify changes in shapes, shades and colors--as opposed to reading columns of numbers. After all, how many of the readers of this article still wear those digital-only watches that were the rage in the 70’s and 80’s, or still drive a car with the speedometer displaying numbers instead of a moving hand? Being able to understand how the value of a complex portfolio of multiple commodities changes under various price or operational assumptions is best accomplished via a graphical view, not by reading through a large matrix of numbers.

At Solarc’s recent ASCENT conference, the company spent a significant amount of time demonstrating some of the research they have underway in the area of data visualization, including a relatively new technology called Surface Computing (from Microsoft), incorporating “multi-touch” capabilities (essentially a giant I-Pod Touch-like device)--allowing multiple users to directly interact with a single screen, manipulating data and documents. While the technology is currently more oriented to the consumer/commercial markets, Solarc did display some prototype capabilities in the area of workflow enablement, including document management.

Scheduling is an area ripe with potential for improved workflow and optimization facilitated by improved data visualization; and this is another area that Solarc displayed some new innovation, using a GIS system overlaid with a pipeline network which, once in full production, would allow schedulers to more quickly balance their services agreements and identify transportation opportunities. This ability to visually display a network of pipelines or transmission lines with additional information, including volumetric (available capacity, interruptions, pool balances, scheduled volumes, demand forecasts, etc.) and pricing data, could allow schedulers or dealmakers to quickly optimize their daily physical portfolios for price or transportation optionality. However, while advances in visualizing this type of data have been made, the technology is still in its infancy. Currently many vendors do provide a schematic view of pipeline or transmission networks; however, they are primarily used to ensure nominations are in balance and are limited in their utility in terms of optimization.

For these types of interfaces to advance and provide the true decision support and optimization capabilities that are required in a real-time environment, a new paradigm in visualization is required. That new paradigm will require equal parts of investment and imagination--investment on the part of the systems vendors and active participation by the market in helping to identify the appropriate data points and envisioning new ways of consuming the mass of information that is currently available in the data flood.

The technology to improve data analysis and visualization does exists now; after all, if we have the ability to model global climate one hundred years into the future, surely we should be able to quickly calculate a multi-variable portfolio simulation five years into the future.

RSS feed for comments on this post.
TrackBack URI