🐦 Twitter Post Details

Viewing enriched Twitter post

@jerryjliu0

In the past few weeks we’ve created a ton of features/tutorials on “advanced retrieval techniques” for better performing RAG systems. “Okay cool, but how do I know they do better than top-k?” 🤔 Great point. We’re launching an initiative to add retrieval/LLM evals to *all* of our advanced retrieval tutorials to show if they’re better💡: ✅ Recursive retrieval / node references ✅ Auto-merging retriever There’s more coming too: 🗓️ Ensemble Retrieval 🗓️ Hierarchical Retrieval 🗓️ Document Agents The burden of proof is on us to show that these techniques work better than the basic stuff, and also to show you how to create proper benchmarks and evaluate techniques in different settings! 🧪🧑‍🔬 Some super interesting findings so far 🔥: 💡Using node references for retrieval, instead of the raw text, can improve hit-rate / MRR by 10-20%!! 💡The response quality from auto-merging retrieval is better than top-k, and we find that GPT-4 prefers it 65% of the time. Node references notebook: https://t.co/y6XIdSSfhI Auto merging notebook: https://t.co/JT7pE59t30

View on Twitter

🔧 Raw API Response

{
  "user": {
    "created_at": "2011-09-07T22:54:31.000Z",
    "default_profile_image": false,
    "description": "co-founder/CEO @llama_index\n\nEx-ML @robusthq,  AI research @Uber_ATG, ML Eng @Quora, @princeton",
    "fast_followers_count": 0,
    "favourites_count": 3821,
    "followers_count": 22873,
    "friends_count": 1144,
    "has_custom_timelines": true,
    "is_translator": false,
    "listed_count": 597,
    "location": "",
    "media_count": 579,
    "name": "Jerry Liu",
    "normal_followers_count": 22873,
    "possibly_sensitive": false,
    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1283610285031460864/1Q4zYhtb_normal.jpg",
    "screen_name": "jerryjliu0",
    "statuses_count": 2633,
    "translator_type": "none",
    "url": "https://t.co/S7FkTSefQ0",
    "verified": false,
    "withheld_in_countries": [],
    "id_str": "369777416"
  },
  "id": "1706335297079050325",
  "conversation_id": "1706335297079050325",
  "full_text": "In the past few weeks we’ve created a ton of features/tutorials on “advanced retrieval techniques” for better performing RAG systems.\n\n“Okay cool, but how do I know they do better than top-k?” 🤔\n\nGreat point. We’re launching an initiative to add retrieval/LLM evals to *all* of our advanced retrieval tutorials to show if they’re better💡:\n✅ Recursive retrieval / node references\n✅ Auto-merging retriever\nThere’s more coming too:\n🗓️ Ensemble Retrieval\n🗓️ Hierarchical Retrieval\n🗓️ Document Agents\n\nThe burden of proof is on us to show that these techniques work better than the basic stuff, and also to show you how to create proper benchmarks and evaluate techniques in different settings! 🧪🧑‍🔬\n\nSome super interesting findings so far 🔥:\n💡Using node references for retrieval, instead of the raw text, can improve hit-rate / MRR by 10-20%!!\n💡The response quality from auto-merging retrieval is better than top-k, and we find that GPT-4 prefers it 65% of the time.\n\nNode references notebook: https://t.co/y6XIdSSfhI\nAuto merging notebook: https://t.co/JT7pE59t30",
  "reply_count": 8,
  "retweet_count": 35,
  "favorite_count": 252,
  "hashtags": [],
  "symbols": [],
  "user_mentions": [],
  "urls": [],
  "media": [
    {
      "media_url": "https://pbs.twimg.com/media/F64e0XabIAAXHqg.png",
      "type": "photo"
    },
    {
      "media_url": "https://pbs.twimg.com/media/F64e2gibcAAfIn8.jpg",
      "type": "photo"
    }
  ],
  "url": "https://twitter.com/jerryjliu0/status/1706335297079050325",
  "created_at": "2023-09-25T15:50:11.000Z",
  "#sort_index": "1706335297079050325",
  "view_count": 47839,
  "quote_count": 2,
  "is_quote_tweet": false,
  "is_retweet": false,
  "is_pinned": false,
  "is_truncated": true,
  "startUrl": "https://twitter.com/jerryjliu0/status/1706335297079050325"
}