2017 was a bad year for Uber. If you’re reading this, you probably don’t need me to tell you why. What you might not have seen though, is how Uber used data science experiments to manipulate drivers. In this New York Times article, Noam Sheiber discusses how Uber uses the results of data-driven experiments to influence drivers, including ways to get drivers to stay on the app and work longer, as well as getting drivers to exhibit certain behaviors (e.g. drive in certain neighborhoods at certain times).
In light of Uber’s widespread bad behavior, it’s been brought up several times that maybe we should have seen this coming. After all, this is a company that has flown in the face of laws and regulations with premeditation and exuberance, operating successfully in cities where by rule their model isn’t allowed. Given this, the question I’ll pursue here is what should we make of Airbnb, a company whose growth to unicorn status has been fueled by similarly brazen ignorance of local laws, pushing into cities where hosts often break the law (or at least the tax code) by listing their homes?
In particular, I’d like to take a look at how Airbnb affects how their hosts price their listings. Why? Well, this is where Airbnb has invested a lot of their data science resources (from what’s known publicly) and it’s one of the key levers where they can influence hosts. The genesis of their pricing model came in 2012, when Airbnb realized they had a problem. In a study, they found that many potential new hosts were going through the entire signup process, just to leave when prompted to price their listing. People didn’t know, or didn’t want to put in the work, to find out how much their listing was worth. So, Airbnb built hosts a tool that would offer pricing “tips”. The inference from Airbnb’s blog posts covering their pricing model is that this addressed the problem, as users happily rely on their tips – though they are careful to point out, repeatedly, that users are free to price at whatever they want.
As someone who is looking at this with the agenda of flagging any potential areas of concerns, this caught my attention. The inference I took from reading several accounts of their pricing model, is that Airbnb believe users lean heavily (or blindly) on their pricing suggestions. I’d buy that. And why that’s concerning is we don’t really know how their model works. Yes, we know that it’s a machine learning classifier model, that extracts features out of a listing, as well as incorporating dynamic market features (season, events, etc) to predict the value of the listing. In their postings about their model, they list features it uses, and many make sense. Wifi, private bathrooms, neighborhood, music festivals, all of these are things we’d expect. And others like “stage of growth for Airbnb” and “factors in demand” seem innocuous at first pass. But wait, what do those really mean?
One of the underlying problems present in Sheiber’s Uber article was that fundamentally, Uber’s and their Driver’s agendas were at odds. And while I wouldn’t say the relationship between Airbnb and their hosts is nearly as fraught as Uber and its drivers, it might not be 100% aligned. For host’s, the agenda is pretty simple: on any given listing, they’re trying to make as much money as possible. But for Airbnb, there’s way more at play. They’re trying to grow, and establish themselves as a reliable, go-to source for short-term housing rentals. They’re competing with the hotel industry as a whole, trying to establish themselves in new markets, and trying to change legislature the world over. Any of these could be a reason why they might include features in their pricing tips model that do not lead it price listings at the maximum potential value.
The potential problem here is that while Airbnb likes to share their data science accomplishments, and even open source tools, they aren’t fully transparent with users and hosts about what factors go into some of the algorithms that effect user decisions. While it would be impossible to share every feature and it’s associated weights, it is entirely possible for them to inform users if their model takes into account factors whose intent is not to maximize user revenue.
Clearly, this is all speculative, as I can’t with any certainty say what is behind the curtain of Airbnb’s pricing model. In writing this, I’m mearly hoping to bring attention to an interaction that is vulnerable to manipulation.