🐦 Twitter Post Details

Viewing enriched Twitter post

@jerryjliu0

grep (and lightweight semantic search) are all you need 🤔 When you have a “medium” sized dataset e.g. 1000 ArXiv PDFs, we found that an extremely strong Q&A baseline is just giving agents access to the CLI, along with some tools for fast semantic search using static embeddings. These agents can answer complex questions, from simple search/filter with keywords, to those that require cross-referencing across docs, to those that require analysis across time. In these cases standard RAG with fixed top-k retrieval is strictly worse. We made file understanding + semantic search very CLI accessible through semtools, come check it out! Blog by @LoganMarkewich : https://t.co/kYr8KkWLYR SemTools: https://t.co/xg1iqbghIr

Media 1
Media 2
Media 3

📊 Media Metadata

{
  "media": [
    {
      "url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1964098215181168732/media_0.jpg?",
      "media_url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1964098215181168732/media_0.jpg?",
      "type": "photo",
      "filename": "media_0.jpg"
    },
    {
      "url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1964098215181168732/media_1.jpg?",
      "media_url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1964098215181168732/media_1.jpg?",
      "type": "photo",
      "filename": "media_1.jpg"
    },
    {
      "url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1964098215181168732/media_2.jpg?",
      "media_url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/1964098215181168732/media_2.jpg?",
      "type": "photo",
      "filename": "media_2.jpg"
    }
  ],
  "processed_at": "2025-09-06T21:16:04.515357",
  "pipeline_version": "2.0"
}

🔧 Raw API Response

{
  "type": "tweet",
  "id": "1964098215181168732",
  "url": "https://x.com/jerryjliu0/status/1964098215181168732",
  "twitterUrl": "https://twitter.com/jerryjliu0/status/1964098215181168732",
  "text": "grep (and lightweight semantic search) are all you need 🤔\n\nWhen you have a “medium” sized dataset e.g. 1000 ArXiv PDFs, we found that an extremely strong Q&A baseline is just giving agents access to the CLI, along with some tools for fast semantic search using static embeddings.\n\nThese agents can answer complex questions, from simple search/filter with keywords, to those that require cross-referencing across docs, to those that require analysis across time.\n\nIn these cases standard RAG with fixed top-k retrieval is strictly worse.\n\nWe made file understanding + semantic search very CLI accessible through semtools, come check it out!\n\nBlog by @LoganMarkewich : https://t.co/kYr8KkWLYR\nSemTools: https://t.co/xg1iqbghIr",
  "source": "Twitter for iPhone",
  "retweetCount": 57,
  "replyCount": 18,
  "likeCount": 422,
  "quoteCount": 5,
  "viewCount": 82123,
  "createdAt": "Fri Sep 05 22:48:00 +0000 2025",
  "lang": "en",
  "bookmarkCount": 487,
  "isReply": false,
  "inReplyToId": null,
  "conversationId": "1964098215181168732",
  "displayTextRange": [
    0,
    284
  ],
  "inReplyToUserId": null,
  "inReplyToUsername": null,
  "author": {
    "type": "user",
    "userName": "jerryjliu0",
    "url": "https://x.com/jerryjliu0",
    "twitterUrl": "https://twitter.com/jerryjliu0",
    "id": "369777416",
    "name": "Jerry Liu",
    "isVerified": false,
    "isBlueVerified": true,
    "verifiedType": null,
    "profilePicture": "https://pbs.twimg.com/profile_images/1283610285031460864/1Q4zYhtb_normal.jpg",
    "coverPicture": "",
    "description": "",
    "location": "",
    "followers": 64986,
    "following": 1380,
    "status": "",
    "canDm": true,
    "canMediaTag": true,
    "createdAt": "Wed Sep 07 22:54:31 +0000 2011",
    "entities": {
      "description": {
        "urls": []
      },
      "url": {}
    },
    "fastFollowersCount": 0,
    "favouritesCount": 7836,
    "hasCustomTimelines": true,
    "isTranslator": false,
    "mediaCount": 1275,
    "statusesCount": 6125,
    "withheldInCountries": [],
    "affiliatesHighlightedLabel": {},
    "possiblySensitive": false,
    "pinnedTweetIds": [],
    "profile_bio": {
      "description": "co-founder/CEO @llama_index\n\nCareers: https://t.co/EUnMNmbCtx\nEnterprise: https://t.co/Ht5jwxSrQB",
      "entities": {
        "description": {
          "urls": [
            {
              "display_url": "llamaindex.ai/careers",
              "expanded_url": "https://www.llamaindex.ai/careers",
              "indices": [
                38,
                61
              ],
              "url": "https://t.co/EUnMNmbCtx"
            },
            {
              "display_url": "llamaindex.ai/contact",
              "expanded_url": "https://www.llamaindex.ai/contact",
              "indices": [
                74,
                97
              ],
              "url": "https://t.co/Ht5jwxSrQB"
            }
          ],
          "user_mentions": [
            {
              "id_str": "0",
              "indices": [
                15,
                27
              ],
              "name": "",
              "screen_name": "llama_index"
            }
          ]
        },
        "url": {
          "urls": [
            {
              "display_url": "llamaindex.ai",
              "expanded_url": "https://www.llamaindex.ai/",
              "indices": [
                0,
                23
              ],
              "url": "https://t.co/YiIfjVlzb6"
            }
          ]
        }
      }
    },
    "isAutomated": false,
    "automatedBy": null
  },
  "extendedEntities": {
    "media": [
      {
        "display_url": "pic.twitter.com/sYr1lWadgh",
        "expanded_url": "https://twitter.com/jerryjliu0/status/1964098215181168732/photo/1",
        "ext_media_availability": {
          "status": "Available"
        },
        "features": {
          "large": {},
          "orig": {}
        },
        "id_str": "1964098191860838400",
        "indices": [
          285,
          308
        ],
        "media_key": "3_1964098191860838400",
        "media_results": {
          "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAARtB4OY12jAACgACG0Hg66PaMFwAAA==",
          "result": {
            "__typename": "ApiMedia",
            "id": "QXBpTWVkaWE6DAABCgABG0Hg5jXaMAAKAAIbQeDro9owXAAA",
            "media_key": "3_1964098191860838400"
          }
        },
        "media_url_https": "https://pbs.twimg.com/media/G0Hg5jXaMAAt-W2.png",
        "original_info": {
          "focus_rects": [
            {
              "h": 551,
              "w": 984,
              "x": 0,
              "y": 179
            },
            {
              "h": 984,
              "w": 984,
              "x": 0,
              "y": 0
            },
            {
              "h": 1122,
              "w": 984,
              "x": 0,
              "y": 0
            },
            {
              "h": 1211,
              "w": 606,
              "x": 90,
              "y": 0
            },
            {
              "h": 1211,
              "w": 984,
              "x": 0,
              "y": 0
            }
          ],
          "height": 1211,
          "width": 984
        },
        "sizes": {
          "large": {
            "h": 1211,
            "w": 984
          }
        },
        "type": "photo",
        "url": "https://t.co/sYr1lWadgh"
      }
    ]
  },
  "card": null,
  "place": {},
  "entities": {
    "urls": [
      {
        "display_url": "llamaindex.ai/blog/semtools-…",
        "expanded_url": "https://www.llamaindex.ai/blog/semtools-are-coding-agents-all-you-need",
        "indices": [
          667,
          690
        ],
        "url": "https://t.co/kYr8KkWLYR"
      },
      {
        "display_url": "github.com/run-llama/semt…",
        "expanded_url": "https://github.com/run-llama/semtools",
        "indices": [
          701,
          724
        ],
        "url": "https://t.co/xg1iqbghIr"
      }
    ],
    "user_mentions": [
      {
        "id_str": "1676011773810360320",
        "indices": [
          649,
          664
        ],
        "name": "Logan Markewich",
        "screen_name": "LoganMarkewich"
      }
    ]
  },
  "quoted_tweet": {
    "type": "tweet",
    "id": "1964009128973783135",
    "url": "https://x.com/llama_index/status/1964009128973783135",
    "twitterUrl": "https://twitter.com/llama_index/status/1964009128973783135",
    "text": "Command-line agents can get you really far in document search and analysis!\n\nWe tested SemTools, our CLI toolkit for parsing and semantic search, with coding agents like @claude_code on 1000 @arxiv papers. The results show that combining Unix tools with semantic search capabilities creates surprisingly capable knowledge workers.\n\n🔍 SemTools adds parse and search commands that let agents handle complex documents with fuzzy semantic keyword search\n📊 Agents with semantic search provided more detailed, accurate answers across search, cross-reference, and temporal analysis tasks\n⚡ CLI access proves incredibly powerful relative to effort - leveraging existing Unix tooling instead of building custom RAG infrastructure\n🛠️ The combination of grep, find, and semantic search handles a wide variety of document tasks at high fidelity\n\nLearn about our SemTools experiment and see the full benchmark results: https://t.co/3LeaejfRWc",
    "source": "Twitter for iPhone",
    "retweetCount": 13,
    "replyCount": 7,
    "likeCount": 96,
    "quoteCount": 4,
    "viewCount": 59644,
    "createdAt": "Fri Sep 05 16:54:00 +0000 2025",
    "lang": "en",
    "bookmarkCount": 107,
    "isReply": false,
    "inReplyToId": null,
    "conversationId": "1964009128973783135",
    "displayTextRange": [
      0,
      270
    ],
    "inReplyToUserId": null,
    "inReplyToUsername": null,
    "author": {
      "type": "user",
      "userName": "llama_index",
      "url": "https://x.com/llama_index",
      "twitterUrl": "https://twitter.com/llama_index",
      "id": "1604278358296055808",
      "name": "LlamaIndex 🦙",
      "isVerified": false,
      "isBlueVerified": true,
      "verifiedType": null,
      "profilePicture": "https://pbs.twimg.com/profile_images/1623505166996742144/n-PNQGgd_normal.jpg",
      "coverPicture": "https://pbs.twimg.com/profile_banners/1604278358296055808/1752258343",
      "description": "",
      "location": "",
      "followers": 101305,
      "following": 28,
      "status": "",
      "canDm": false,
      "canMediaTag": true,
      "createdAt": "Sun Dec 18 00:52:44 +0000 2022",
      "entities": {
        "description": {
          "urls": []
        },
        "url": {}
      },
      "fastFollowersCount": 0,
      "favouritesCount": 1433,
      "hasCustomTimelines": true,
      "isTranslator": false,
      "mediaCount": 1714,
      "statusesCount": 3545,
      "withheldInCountries": [],
      "affiliatesHighlightedLabel": {},
      "possiblySensitive": false,
      "pinnedTweetIds": [
        "1958564769188978822"
      ],
      "profile_bio": {
        "description": "Build AI agents over your documents\n\nGithub: https://t.co/HC19j7vMwc\nDocs: https://t.co/QInqg2zksh\nLlamaCloud: https://t.co/yQGTiRSNvj",
        "entities": {
          "description": {
            "urls": [
              {
                "display_url": "github.com/run-llama/llam…",
                "expanded_url": "http://github.com/run-llama/llama_index",
                "indices": [
                  45,
                  68
                ],
                "url": "https://t.co/HC19j7vMwc"
              },
              {
                "display_url": "docs.llamaindex.ai",
                "expanded_url": "http://docs.llamaindex.ai",
                "indices": [
                  75,
                  98
                ],
                "url": "https://t.co/QInqg2zksh"
              },
              {
                "display_url": "cloud.llamaindex.ai",
                "expanded_url": "https://cloud.llamaindex.ai/",
                "indices": [
                  111,
                  134
                ],
                "url": "https://t.co/yQGTiRSNvj"
              }
            ]
          },
          "url": {
            "urls": [
              {
                "display_url": "llamaindex.ai",
                "expanded_url": "https://www.llamaindex.ai/",
                "indices": [
                  0,
                  23
                ],
                "url": "https://t.co/epzefqQqZx"
              }
            ]
          }
        }
      },
      "isAutomated": false,
      "automatedBy": null
    },
    "extendedEntities": {
      "media": [
        {
          "display_url": "pic.twitter.com/pSPUOWhm3t",
          "expanded_url": "https://twitter.com/llama_index/status/1964009128973783135/photo/1",
          "ext_media_availability": {
            "status": "Available"
          },
          "features": {
            "large": {},
            "orig": {}
          },
          "id_str": "1964009125739974656",
          "indices": [
            271,
            294
          ],
          "media_key": "3_1964009125739974656",
          "media_results": {
            "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAARtBj+TjW1AACgACG0GP5aQbUF8AAA==",
            "result": {
              "__typename": "ApiMedia",
              "id": "QXBpTWVkaWE6DAABCgABG0GP5ONbUAAKAAIbQY/lpBtQXwAA",
              "media_key": "3_1964009125739974656"
            }
          },
          "media_url_https": "https://pbs.twimg.com/media/G0GP5ONbUAAixYg.jpg",
          "original_info": {
            "focus_rects": [
              {
                "h": 605,
                "w": 1080,
                "x": 0,
                "y": 0
              },
              {
                "h": 1080,
                "w": 1080,
                "x": 0,
                "y": 0
              },
              {
                "h": 1080,
                "w": 947,
                "x": 0,
                "y": 0
              },
              {
                "h": 1080,
                "w": 540,
                "x": 27,
                "y": 0
              },
              {
                "h": 1080,
                "w": 1080,
                "x": 0,
                "y": 0
              }
            ],
            "height": 1080,
            "width": 1080
          },
          "sizes": {
            "large": {
              "h": 1080,
              "w": 1080
            }
          },
          "type": "photo",
          "url": "https://t.co/pSPUOWhm3t"
        }
      ]
    },
    "card": null,
    "place": {},
    "entities": {
      "urls": [
        {
          "display_url": "llamaindex.ai/blog/semtools-…",
          "expanded_url": "https://www.llamaindex.ai/blog/semtools-are-coding-agents-all-you-need?utm_source=socials&utm_medium=li_social",
          "indices": [
            906,
            929
          ],
          "url": "https://t.co/3LeaejfRWc"
        }
      ],
      "user_mentions": [
        {
          "id_str": "1903660637177470976",
          "indices": [
            170,
            182
          ],
          "name": "Claude Code Community",
          "screen_name": "claude_code"
        },
        {
          "id_str": "808633423300624384",
          "indices": [
            191,
            197
          ],
          "name": "arXiv.org",
          "screen_name": "arxiv"
        }
      ]
    },
    "quoted_tweet": null,
    "retweeted_tweet": null,
    "isLimitedReply": false,
    "article": null
  },
  "retweeted_tweet": null,
  "isLimitedReply": false,
  "article": null
}