{"id":38940,"date":"2023-04-03T14:00:57","date_gmt":"2023-04-03T18:00:57","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=38940"},"modified":"2023-04-24T08:00:39","modified_gmt":"2023-04-24T12:00:39","slug":"twitters-recommendation-algorithm","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2023\/04\/03\/twitters-recommendation-algorithm\/","title":{"rendered":"Twitter’s Recommendation Algorithm"},"content":{"rendered":"

Twitter<\/a> (Hacker News<\/a>, Slashdot<\/a>):<\/p>\n

\n

At Twitter 2.0<\/a>, we believe that we have a responsibility, as the town square of the internet, to make our platform transparent. So today we are taking the first step in a new era of transparency and opening much of our source code to the global community.<\/p>\n

[…]<\/p>\n

We also took additional steps to ensure that user safety and privacy would be protected, including our decision not to release training data or model weights associated with the Twitter algorithm at this point.<\/p>\n<\/blockquote>\n\n

Twitter<\/a> (tweet<\/a>, Hacker News<\/a>):<\/p>\n

\n

Twitter aims to deliver you the best of what’s happening in the world right now. This requires a recommendation algorithm to distill the roughly 500 million Tweets posted daily down to a handful of top Tweets that ultimately show up on your device’s For You<\/b> timeline. This blog is an introduction to how the algorithm selects Tweets for your timeline.<\/p>\n

[…]<\/p>\n

The foundation of Twitter’s recommendations is a set of core models and features that extract latent information from Tweet, user, and engagement data. These models aim to answer important questions about the Twitter network, such as, “What is the probability you will interact with another user in the future?”<\/i> or, “What are the communities on Twitter and what are trending Tweets within them?”<\/i> Answering these questions accurately enables Twitter to deliver more relevant recommendations.<\/p> \n

The recommendation pipeline is made up of three main stages that consume these features: <\/p> \n

    \n
  1. Fetch the best Tweets from different recommendation sources in a process called candidate sourcing<\/b>.<\/li> \n
  2. Rank<\/b> each Tweet using a machine learning model.<\/li> \n
  3. Apply heuristics and filters<\/b>, such as filtering out Tweets from users you’ve blocked, NSFW content, and Tweets you’ve already seen.<\/li> \n<\/ol> \n<\/blockquote>\n\n

    The (mostly Scala) code is here<\/a>.<\/p>\n\n

    Dan Luu<\/a>:<\/p>\n

    The new stuff in the Twitter algorithm is wild. <\/p>

    “author_is_elon”, “author_is_republican”, “author_is_democrat”, etc., are explicit terms that are special cased.<\/p><\/blockquote>\n\n

    Aakash Gupta<\/a>:<\/p>\n

    1. Likes, then retweets, then replies<\/p>

      Here’s the ranking parameters:<\/p>