🐦 Twitter Post Details

Viewing enriched Twitter post

@victorialslocum

What is the “Col” in ColBERT and ColPali? ⬇️ Contextualized Late Interaction (Col) is used in models like ColBERT for text retrieval and ColPali for vision-based tasks, and it combines the high performance of cross-encoder models with the computational efficiency of bi-encoders. Instead of performing an all-to-all comparison of query and document tokens at runtime like BERT-based systems, Col pre-computes and stores document token embeddings offline, while still having high accuracy like cross encoders! The MaxSim (Maximum Similarity) operation is the secret sauce of “Col” methods - for each query token, MaxSim calculates its similarity with every token in a document using dot product, then selects the highest score. The final similarity score between the query and document is the sum of these MaxSim scores for all query tokens. Blog with example: https://t.co/VIaY6GXaaL ColBERT: https://t.co/6wiTTYSLFz

🔧 Raw API Response

{
  "user": {
    "created_at": "2021-01-17T17:43:37.000Z",
    "default_profile_image": false,
    "description": "learning cool stuff, machine learning engineer at @weaviate_io 💙",
    "fast_followers_count": 0,
    "favourites_count": 3361,
    "followers_count": 3138,
    "friends_count": 474,
    "has_custom_timelines": true,
    "is_translator": false,
    "listed_count": 41,
    "location": "Berlin, Germany",
    "media_count": 94,
    "name": "Victoria Slocum",
    "normal_followers_count": 3138,
    "possibly_sensitive": false,
    "profile_banner_url": "https://pbs.twimg.com/profile_banners/1350861258539450371/1715929658",
    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1698629814364553217/fRdcw9Hz_normal.jpg",
    "screen_name": "victorialslocum",
    "statuses_count": 452,
    "translator_type": "none",
    "url": "https://t.co/jldayD93ku",
    "verified": true,
    "withheld_in_countries": [],
    "id_str": "1350861258539450371"
  },
  "id": "1848689063315841039",
  "conversation_id": "1848689063315841039",
  "full_text": "What is the “Col” in ColBERT and ColPali? ⬇️\n\nContextualized Late Interaction (Col) is used in models like ColBERT for text retrieval and ColPali for vision-based tasks, and it combines the high performance of cross-encoder models with the computational efficiency of bi-encoders.\n\nInstead of performing an all-to-all comparison of query and document tokens at runtime like BERT-based systems, Col pre-computes and stores document token embeddings offline, while still having high accuracy like cross encoders!\n\nThe MaxSim (Maximum Similarity) operation is the secret sauce of “Col” methods - for each query token, MaxSim calculates its similarity with every token in a document using dot product, then selects the highest score. The final similarity score between the query and document is the sum of these MaxSim scores for all query tokens.\n\nBlog with example: https://t.co/VIaY6GXaaL\nColBERT: https://t.co/6wiTTYSLFz",
  "reply_count": 3,
  "retweet_count": 36,
  "favorite_count": 219,
  "hashtags": [],
  "symbols": [],
  "user_mentions": [],
  "urls": [],
  "media": [
    {
      "media_url": "https://pbs.twimg.com/ext_tw_video_thumb/1848687896351363072/pu/img/d5RGyA6lxJXeJ9tN.jpg",
      "type": "video",
      "video_url": "https://video.twimg.com/ext_tw_video/1848687896351363072/pu/vid/avc1/1280x720/Muytdh1_4JZWYiTr.mp4?tag=12"
    }
  ],
  "url": "https://twitter.com/victorialslocum/status/1848689063315841039",
  "created_at": "2024-10-22T11:33:14.000Z",
  "#sort_index": "1848689063315841039",
  "view_count": 12586,
  "quote_count": 3,
  "is_quote_tweet": false,
  "is_retweet": false,
  "is_pinned": false,
  "is_truncated": true,
  "startUrl": "https://x.com/victorialslocum/status/1848689063315841039"
}