Twitter Ranking Tweets With Machine Learning
Machine learning is entering production at Twitter as a way of ranking tweets and boosting engagement.
Twitter engineers this week unveiled the social media platform’s ranking algorithm driven by deep neural networks. In a blog post, company engineers said their approach leverages an in-house artificial intelligence platform that includes new modeling capabilities.
Among the results, wrote Nicolas Koumchatzky, a software engineer with Twitter’s AI team called Cortex, are “more relevant timelines now, and in the future, as this opens the door for us to use more of the many novelties that the deep learning community has to offer, especially in the areas of [natural language processing], conversation understanding and media domains.”
Currently, Twitter (NYSE: TWTR) timelines are arranged chronologically based on a user’s last visit. That alone is a daunting infrastructure task, the company notes. The new ranking algorithm gathers all tweets from accounts being followed by an individual user. It then scores those tweets using a relevance model, with the goal of predicting the most relevant comments. The highest-ranked tweets are then display at the top of a user’s timeline, Koumchatzky explained.
Twitter’s new ranking model takes into account factors such as the number of re-tweets, likes, the inclusion of images and video and other responses to a post. It also attempts to gauge a user’s past interactions with authors along with the “strength” and origin of the follower’s relationship to an author.
“This scoring step imposes an even greater computational demand on timelines serving infrastructure, as we are now scoring thousands of tweets per second to satisfy all of the timeline requests,” the Twitter engineer noted. “The unique challenge is to perform scoring quickly enough to instantly serve tweets back to the people viewing their timelines, yet have powerful enough models to allow for the best possible quality and future improvements.”
Along with prediction models, Twitter said similar requirements are applied to its machine learning frameworks. A set of tools is used to train and launch a prediction model, with particular attention given to training speed and scaling when dealing with very large datasets, much of it unstructured.
Based on the breadth of research into new AI algorithms and model architectures, Koumchatzky argued, “betting on a platform that natively supports deep learning and complex graphs is key to leveraging the promises of that work.”
The Cortex team of data scientists and machine-learning researchers works on Twitter’s deep learning platform. New members were added last year when Twitter acquired London-based AI startup Magic Pony Technology. These and other deals are part of a push by social media and other hyper-scalers to train rather than just program algorithms. It also underscores how AI technology is finding new automation applications required to organize huge datasets.
Having worked out most of the kinks in its deep learning models and platform, Twitter’s Koumchatzky said “online experiments have also shown significant increases in metrics such as tweet engagement and time spent on the platform.”
December 11, 2017
- Kyvos Insights to Host Webinar on Implementing BI on Big Data
- MicroStrategy Announces Availability of MicroStrategy 10.10
- TIBCO Announces Partnership with Change Healthcare
- PSSC Labs Launches PowerWulf HPC Clusters with Pre-Configured Intel Data Center Blocks
December 8, 2017
December 7, 2017
- Splunk Named as a Leader in Gartner’s 2017 Magic Quadrant for Security Information and Event Management
- MicroStrategy Named as a Leader in 2017 Dresner Advisory Services Embedded Business Intelligence Market Study
December 6, 2017
- Kroll Announces AI Partnership with Sigmoidal
- iguazio Debuts the nuclio Serverless Platform for Multi-Cloud and Edge Deployments
December 5, 2017
- Talend Unveils New Metadata Management Solution for Governed Data Lakes
- Overstock.com Selects Snowflake’s Cloud Data Warehouse
December 4, 2017
December 1, 2017
- Striim Recognized as Overall ‘Best Place to Work’ in the US by Comparably
- NVIDIA Expands AI Startup Portfolio with Investments in Three Data Science Companies
November 30, 2017
November 29, 2017
- Domino Data Lab Achieves AWS Machine Learning Competency Status
- MapR Introduces up to 10X Faster, Secure BI/Analytics Experience On Operational Data
- Narrative Science Achieves AWS Machine Learning Competency Status
- AtScale Brings Universal Semantic Layer to the AWS Cloud
- Trifacta Announces Additional Support of Amazon Web Services and Availability on AWS Marketplace
Most Read Features
- Taking the Data Scientist Out of Data Science
- Hadoop 3.0 Likely to Arrive Before Christmas
- 9 Must-Have Skills to Land Top Big Data Jobs in 2015
- A Look at the Graph Database Landscape
- Which Type of SSD is Best: SATA, SAS, or PCIe?
- Spark Streaming: What Is It and Who’s Using It?
- First Look at Scio, a Scala API for Apache Beam
- Machine Learning Education: 3 Paths to Get Started
- Machine Learning, Deep Learning, and AI: What’s the Difference?
- Deep Learning Is About to Revolutionize Sports Analytics. Here’s How
- More Features…
Most Read News In Brief
- AWS Unveils Graph Database, Called Neptune
- Translytical Databases Hit the Ground Running
- What SQL’s Co-Creator Sees in NoSQL
- Machine Learning ‘Arms Race’ Ahead, McAfee Warns
- Data Prep Goes Serverless
- Distributed PostgreSQL Settling Into Cloud
- Google AutoML Claims Machine Vision Advance
- Standards Effort Seeks to Redefine ‘Data Scientist’
- Why Gartner Dropped Big Data Off the Hype Curve
- What AI Taketh, It Could Also Create
- More News In Brief…
Most Read This Just In
- The New Dataiku: A Central Hub for Enterprise Analytics, Machine Learning, and Data Science
- New Dell EMC Solutions Bring Machine and Deep Learning to Mainstream Enterprises
- Gartner Reveals the 2017 Hype Cycle for Data Management
- Cloudera Announces Upcoming Cloudera Altus Analytic DB Beta
- Databricks Becomes Microsoft Partner to Offer Its Unified Analytics Platform as an Integrated Microsoft Azure Service
- Hitachi Develops Open Source Software Based Big Data Analytics Technology to Increase Speed By Up to 100 Times
- Overstock.com Selects Snowflake’s Cloud Data Warehouse
- Sequoia Backs Graphcore as the Future of Artificial Intelligence Processors
- Cambridge Semantics Announces Semantic Layer for Multi-Cloud Environments
- Splunk Inc. Announces Fiscal Third Quarter 2018 Financial Results
- More This Just In…
December 11 - December 13Boston MA United States