KX Welcomes New Languages to Speedy Analytics Database

Since its inception, KX has been the exclusive domain of programmers skilled in the language q, and used primarily for the most demanding analytics workloads in the capital markets. While those use cases continue, the company now is seeking a broader audience, as today’s unveiling of support for Python and SQL in KX Insights, its cloud-based offering, shows.

Arthur Whitney’s legend was created at the intersection of high-performance computing and Wall Street. As the co-founder of KX (formerly Kx Systems) and the creator of q and two other languages, A+ and k, Whitney figured out ways to maximize the throughput of the limited compute capacity available back in the 1970s and 1980s.

KX built a reputation for working with big data back before “big data” was a thing. Its customers could not only crunch large amounts of real-time data, but also compare it to big historical datasets too. Time-series data was a KX specialty, particularly among the equity traders who sought an edge, any edge, lurking in the data.

But the q language has limited adoption of Kdb+ database. Customers had to learn to write in q and learn to operate the database to avail themselves of the analytics and modeling capabilities that KX supported. The company has branched out from Wall Street and has customers in telecommunications, manufacturing, and automotive racing. But it hasn’t enjoyed the sort of widespread success that matches its underlying capabilites.

By enabling data analysts and data scientist to use SQL and Python, respectively, to access data stored in the KX environment, KX has significantly widened the pool of developers who will exposed to KX, says Eric Raab, who joined the company as its new CTO seven months ago.

“Because of its esoteric nature, Kdb+ is extremely high performance but also limited to skilled practitioners of Kdb+ and of the q language,” Raab tells Datanami. “What we’re hoping [supporting SQL and Python] will do is open up a universe of developers and data scientists who can work with Kdb+ beyond what they already have.”

The company is offering full ANSI SQL support in the database, which means that any query that can be written to target other databases should work against Kdb+. The new language support, along with an ODBC connector that is under development, immediately puts Kdb+ on the map for millions of SQL-loving data analysts and Python-loving data scientists.

“There’s already a lot of data in Kdb+ in our current customers, so that’ll be where we start. It’ll give these people access to the corporate data without necessarily being trained on q,” Raab says. “The next step will be people who just want to performance and other benefits of Kdb+ and they will learn to use Python with KDB because it’s just going to rock compared to what they’re currently doing.”

KX announced support for Python and SQL in KX Insights, the new cloud-based version of Kdb+ that is supported on all three cloud providers (although Raab says the version on AWS is the most mature). KX Insights has been available since March, and customers are starting to go into production with it, Raab says.

However, Python and SQL support will also available in the on-prem versions of Kdb+. “Yeah it actually works on prem as well, equivalently to how it works in the cloud,” Raab says. “So if existing customers want it, they can get it. It’s an update.”

Exposing Kdb+ to SQL and Python doesn’t necessarily mean that developers will automatically get all the analytics benefits and capabilities that are in KX Insight, Raab says. They will still need to mind their p’s and q’s to leverage the system (well, maybe not the q’s).

“Just like when they learned SQL, you have to learn how to do a proper plan, and how to prune your joins in the most efficient way,” Raab says. “You have to do the same thing, except it’ll be Kdb+ behind it as opposed to PostgreSQL.”

But if users do it correctly, they should get performance that is equal to or even better than what could get with native q development, Raab says. Plus, it’ll be available to a much larger audience, which should help KX spread its mantra and build its customer base and its following.

“That’s one of the reasons why I was hired, to change the culture of the company,” the former Information Builders executive says. “It’s one that the management team and the employees at this point all believe it. But it’s a change and it requires many things to realign themselves as you go along with that. And that’s where in the process of going. Python is a big part of that. SQL is a big part of that. Going to the cloud is a big part of that.”

The days of KX users being members of a secret club are over–or at the very least, they’re numbered.

“When I joined the company, I spoke to customers, engineer, and a lot of them liked that it was a secret. They liked that they had to discover it,” Raab says. “One of the attractions was the treasure-hunt nature of discovering this hidden gem amongst the plethora of modern languages. Only they held the secret key and they were able to unlock all this performance and algorithmic treasure that other’s didn’t have access to.

“That was really cool at the time,” Raab continues. “But you hit a scale point where you need to go beyond. We’ve hit that scale point and we’re opening it up to a wider audience, of course consciously. We don’t want to lose [customer trust]. But we do want more people to recognize the value of what we do.”

Related Items:

Kx Systems: A Historical Need for Speed

Ready to adopt a Microsecond Mindset?

Streaming Analytics: The Silver Bullet for Predictive Business Decisions