Airlines' Toxic Data
Many airlines are unknowingly using poisoned data – how can they clean it up?
Whizzy airline revenue algorithms use millions of data points to figure out exactly how much they are prepared to sell seats for today on flights that leave in the future.
It is a tricky game. Price the seat too high and some people might not buy, so leaving the seat wasted which airline wonks call “spoil”. Price it too low on the other hand and somebody who does buy could have given the airline more money, known in the trade as “spill”.
To figure out how many seats are likely to be sold at each price point airlines tend to take a view that there is nothing new under the sun. What has gone before will come again.
There are disadvantages to this approach. Post-COVID travel patterns might not be as predictable as before (see article). And emerging data sources like consumer spend and social media chatter might be more forward looking than conventional data (see article).
But a disadvantage that we have not discussed before here on Airline Revenue Economics is the risk that airlines might have poisoned their own data, reducing the extent to which it can be relied on to create good forecasts.
Poisoned data is information that has been allowed to become not reflective of the true state of nature that the data is meant to record. This might have been benign due to neglect and degradation in quality, or malign from a hacker’s attack. But it can also arise from recording things which would not have occurred had the airline not taken bad decisions in the past.
Airlines using data that has been poisoned will be taking poor decisions, leading to a vicious circle where bad decisions lead to poisoned data and worse decisions in. the future. When this has been done as a result of airlines’ own actions things need to change.
There are five ways in which airlines may have poisoned their data. In some cases finding the solution need not be too hard. In other cases putting things right could prove tough. Read on to find out how…
Bid-price collapse is an easy to solve problem
The easiest data poisoning issue to solve is endogenous to airline revenue management (RM) – bid-price collapse in a revenue optimisation system.
The bid price is the lowest expected value that a platform will accept for a seat. For example, if the airline knows from history that fare class L produces $500 and the bid price is $400, it will allow that seat to be sold at whatever the prevailing fare associated with class L might be. L will be open for sale.
Bid price collapse can occur when there is no more demand to come for a flight. From the optimisation platform’s perspective, since no more passengers are expected to buy it will accept anything.
The effect is that many or all fare classes are made available for sale and some passengers or travel agents with flexible bookings can cancel and make a sometimes significant saving. At the same time, rare last-minute bookers with high willingness to pay can have their revenue contribution cannibalised, leaving revenue on the table.
A no more demand to come/bid-price collapse cycle can poison data because the RM system comes to learn that reducing the price at the last minute does sell seats. As explained, this is true but for pernicious reasons.
Fortunately the problem is easy to fix. RM analysts can impose a rule that whatever class is open a short time before travel, the final inventory mix should not involve opening lower classes. Quite when this will be depends on the route. Sometimes it might only be a few hours before departure, other times a few days. In rare instances it might be longer.
Too many fares
Every time industry body ATPCO holds a conference, they announce that airlines now have X billion fares available for sale, up from Y billion last year. This is all well and good, but how many of them are actually sold?
ATPCO is responsible for operating a database of fares that airline selling platforms refer to when pricing a ticket. Loading fares into the database, a process called filing, is time consuming and costly. Up to 32 sets of rules, known as fare categories, have to be completed for each pricing product.
The fare categories range from the well-known charges for changes and refunds through to more obscure points like where exactly the ticket can be sold and if it can be combined with others to price a complex trip.
Loading fares is costly, but helps airlines meet their objectives – undercutting a competitor or raising revenue. Taking them down is costly and has little benefit to the airlines apart from possibly saving a little money on hosting costs in the long term.
As a result many fares are left unused, although industry officials tend to be a little cagey about exactly how many this is. In principle RM systems ought not to care – fares that are unsold make no difference.
But once in a while the systems can be caught out. When I worked at Qatar Airways some ever-so-clever consultants working for the airline made the point by buying some old and obscure fares through their corporate agent that were perhaps only 50% of what the airline might have expected to sell the seat for.
Too many fares is a type of data poisoning because they are superfluous to requirements and can hurt revenue if wily passengers like Qatar Airways’ consultants figure out how to use them to their advantage.
Cleaning them up is a laborious task but easy to achieve. Airlines can identify the fare basis codes that make up most of their income from revenue accounting data. Any fare not in that list and older than, say, a year should be removed.
Inappropriate demand influences
The problems of bid-price collapse and too many fares can arise more due to lack of judgement than poor decision-making. Sometimes though airlines can take actions that actively harm their ability to complete RM in the future.
Sometimes an RM analyst needs to apply a demand influence to tell a system it needs to increase or reduce fares. Special events are one example of a good reason why. A visiting conference will push up demand and will have not been seen before in the historical data. Once the conference is over, things will return to normal.
RM platforms are designed to be influenced. But apply too many influences, especially to achieve price reductions, and the data will be poisoned as the system “learns” that there is less demand than there really is and so will always under-price.
This happened while I was working at Qatar Airways when a PROS O&D system was installed around 2008. The sales department wanted lower fares and RM was instructed to reduce them.
Rather than re-filing fares though, which is costly (see above), RM decided to influence the new tool, which broke under the pressure. Putting things right caused a lot of hassle and the airline lost a good deal of money due to the RM optimiser not working as it should have.
Corporate fares
BA’s fully flexible biz class return fare to New York tomorrow is £11,695.61. I guarantee that nobody on any of their flights is paying that out of their own personal post-tax income. One or two businesses might pay it.
The majority of people using such fares though are business travellers with some kind of corporate incentive. They get money off the headline price or some kind of a rebate at year-end.
Air fares like this are so expensive because every year corporate buyers demand bigger and bigger discounts. And every year airlines put up the headline rates more than they should to accommodate the negotiation.
Fortunately this does not matter for RM as the airline’s tool will recognise that the full fare is not actually received and evaluate bid-prices based on what it really expects.
But it does create a situation where the published market price is not actually what the market pays, another type of data poisoning. Some people may choose not to travel as a result. An interesting solution arises from the world of NFT tickets (see article).
This technology will allow passengers to make offers back to airlines. So when prices are high, members of the public wanting to buy a ticket can make an offer. If their offer is greater than what the airline achieves by selling the seat to a corporate customer then the airline is likely to accept.
Meanwhile the rate-discount cycle keeps going and the corporate sales targets are met. Win-win.
oliver AT ransonpricing DOT com
