Follow Datanami:
April 20, 2015

Experfy: The Uber of Big Data Projects

Starting a big data analytics project can be tough, especially for mid-size firms that can’t justify the cost of a full-time data scientist and don’t want to deal with traditional consulting firms. But a company called Experfy is finding success by emulating the sharing-economy model popularized by Uber, and matching freelance data scientists with analytic projects of clients.

Experfy came out of the Harvard Innovation Lab about a year ago with the goal of creating a marketplace for high-end consulting work revolving around data analytics. To date, the company has attracted a stable of 1,200 data scientists, who create proposals and bid on data projects submitted by clients.

It’s another sign that the sharing economy has come to data science. “We’re the Uber of data science,” says Harpreet Singh, co-founder and co-CEO of Experfy. “If you’re a medium-size business in the U.S. and you want to do data analysis, you can hire somebody or go to a consulting firm. But consulting firms are very expensive, and hiring somebody on the payroll only makes sense if you’re going to have a recurring need.”

So far, Experfy’s data science freelancers have successfully completed about 60 projects. The projects themselves run the gamut from creating equity trading algorithms and building recommendation systems in Hadoop, to translating Excel models into software as a service (SaaS) apps and creating dashboards in Tableau.


A sample of data analytics jobs currently posted on Experfy’s website

In some cases, Experfy’s clients just want an answer to a data-oriented question, while in other cases, they desire the algorithm, expressed in R or Python, which they’ll implement on their own. Some data scientists working for Experfy may serve temporarily as project managers, while teams of data scientists may join a client’s advisory board to provide recommendations and review technical proposals by third party vendors.

“We’re doing a lot of handholding with the clients,” Singh tells Datanami. “Analytics is a complicated thing. You can’t just go post a project and the magic happens. There’s a lot of hard work involved. The clients appreciate that. They want someone who will work with them and help them through the process.”

Experfy works with the clients to craft the description of their project before posting it to the online marketplace. Once it’s online, Experfy’s data scientists can submit their own proposals. “Just reading the proposals is a real treat for the clients because they learn so much about their problem, the different perspectives,” Singh says.

After considering the proposals, Experfy helps the client narrow it down to the two or three best ones, and the parties engage in videoconference calls on the Experfy site. Once the winning bid is selected, Experfy works with the two sides to ensure the work is done on time and meets expectations. Payments are kept in escrow until the client is satisfied. Experfy, which charges a 20 percent commission, guarantees success; no projects have failed, Singh says.

To increase the odds of success, Experfy encourages its clients to post smaller data science projects engagements. “Longer engagements, bigger projects, generally are riskier,” Singh says. The average Experfy engagement lasts two to three weeks and costs $10,000 to $20,000. However, there’s no limit; one recent project to build an IBM Watson-powered application took several months and cost $300,000.

It’s all about making data science accessible to a new class of businesses. “It becomes very affordable for somebody to say, ‘I have this question or need to make this decision’, or ‘I’ve got a business problem and need an algorithm that can solve that problem.’ The data scientists can easily come and solve that problem without a lot of risk,” Singh says. “We’re three to five times cheaper than McKinsey, but we’re three to five times faster as well.”

Experfy closely vets each data scientist who applies to the program. About 80 percent of the data scientists who apply are denied; most of them are rejected by the powerful algorithms that Experfy uses to profile the data scientists from their public profiles available in places like GitHub, Stack Overflow, and Kaggle. But some are rejected by Experfy’s administrators, too.

Considering how difficult it is to find data scientists (i.e. “unicorns”), it’s remarkable how selective Experfy is with its onboarding process. Apparently, data scientists are a sensitive lot who are prone to fits of boredom if left doing the same thing over and over. That could explain why so many are angling to get into Experfy’s system, and why Experfy can afford to be so choosey.experfy logo

“The people who do data science want a degree of intellectual stimulation. If they’re stuck in a 9 to 5 job, they’re not very happy,” Singh says. “What Experfy provides them is an environment where there are a lot of different problems they can solve.”

Singh says there are four main types of data scientists who are attracted to Experfy:

  • Freelancers who are too restless to even consider 9-5 jobs;
  • The 9-5ers who want to spice things up by moonlighting on the side;
  • Academics who desire to see their algorithms used in the real world;
  • Boutique data science firms are trying to build a customer base.

Experfy has been around for just over a year, and is growing at the ridiculous clip that startups do. Singh has about a dozen employees working for him in the Harvard Innovation Lab, and will likely need to hire more as demand for big data science heats up. “This is just the tip of the iceberg,” he says. “Most companies haven’t yet embraced analytics; forget about big data–that’s just for the Fortune 500 perhaps. But everybody can use analytics. Everybody can be data driven.”

Related Items:

9 Must-Have Skills to Land Top Big Data Jobs in 2015

Plugging the Big Data Skills Gap

In Big Data, Jobs Search For You