Live from Big Data World Europe 2012 in London
Once, on eBay, a cheese sandwich that someone had in his freezer was auctioned for over 28,000 dollars. The buyer didn’t want to have it for its flavor, but for the simple fact that, during its decay, the face of the Virgin Mary had appeared on it. This example illustrates the fact that the relationship between an item offered on eBay and the price it fetches is not easily converted into digital data. But eBay has some bright data scientists at work who perform all sorts of interesting research to move their business forward.
Every second on eBay a total merchandise value of 1,400 dollars is traded, a vehicle changes owner via eBay every 2 minutes, and 10 million new items are offered on eBay every day. And what’s more, 80% of the items on sale are labeled “one of a kind”. Add to this the more than 30,000 product categories and the fact that both buyers and sellers predominantly move around in just one or two categories, then you have a setting for quite a big ‘Big Data’ challenge.
Blind men studying their part of the elephant
At ‘Big Data World Europe ‒ 2012’ in London Dr. Neal Sundersan, Senior Director and Head of eBay Research Labs, lifted the veil a little on how eBay turns data into intelligent information. His team conducts a wide variety of activities such as machine learning, data mining, economics, user behavior analytics, information retrieval and visualization. And the data they work with is of a wide variety: user, user behavior, transaction, items, feedback, searches…
Sundersan brings up the story of the six blind men studying the elephant. “You need a Big Data platform for everyone to understand the Big Data elephant. And on the very top of all those layers there needs to be an application layer that offers clearly understandable information for the different sort of users in the business.
Some examples of eBay’s data research
eBay is quite busy with search data. Because insight into search data allows its users to broaden or narrow searches, leads buyers to related products and optimizes the overall experience on eBay. In essence, what eBay does is use intelligence from advanced users and apply that to help what they call ‘the naive user’ (a user who’s not good with queries). A lot of effort goes into the first step of cleaning the data. De-duplicating user-associated data provides better suggestions for related searches. After that, eBay goes six years back in time to analyze user behavior. And it does this pretty much in real time.
Another fascinating example of eBay’s data research can be found in the hundreds of thousands of economics experiments its users run every day. Will I sell faster if I offer free shipping? Will I get a higher price if I show more than two images? Is an offering on Thursday a safer bet than one on Friday? Etc. eBay does not have to conduct these economic experiments itself. Its users are doing this continuously and on a large scale. All that remains for eBay to do is gather the data and put it to good use.