🐦 Twitter Post Details

Viewing enriched Twitter post

@omarsar0

NEW paper worth reading. (bookmark it) The basic idea is to pair a compressive recurrent state with a small exact memory, which helps to recover long-range recall without giving up the efficiency of linear attention. More on it below: Linear-attention and state-space models compress the whole prefix into a fixed-size state. That buys O(1) memory, but when many key-value associations compete, earlier facts get overwritten and needle recall degrades. HOLA gives linear attention a hippocampal complement. It keeps the usual delta-rule state as compressive memory and adds a bounded exact KV cache, forming a semiparametric test-time memory. The state models linearly compressible structure while the cache stores associations that should not be forced through it. The cache writes without a learned eviction module, keeping only tokens whose prediction residual was actually committed to the state. At 340M parameters on 15B SlimPajama tokens, HOLA lowers Wikitext perplexity from 27.32 to 22.92, below a full-attention Transformer++ at 26.88, and stays robust on RULER needle recall out to 32k tokens, 16x its training length. Paper: https://t.co/z1Jzp7qQ6B Learn to build effective AI agents in our academy: https://t.co/1e8RZKs4uX

Media 1
View on Twitter

📊 Media Metadata

{
  "media": [
    {
      "type": "photo",
      "url": "https://crmoxkoizveukayfjuyo.supabase.co/storage/v1/object/public/media/posts/2073068756293869685/media_0.jpg",
      "filename": "media_0.jpg"
    }
  ],
  "processed_at": "2026-07-03T16:01:28.675383",
  "pipeline_version": "2.0"
}

🔧 Raw API Response

{
  "type": "tweet",
  "id": "2073068756293869685",
  "url": "https://x.com/omarsar0/status/2073068756293869685",
  "twitterUrl": "https://twitter.com/omarsar0/status/2073068756293869685",
  "text": "NEW paper worth reading.\n\n(bookmark it)\n\nThe basic idea is to pair a compressive recurrent state with a small exact memory, which helps to recover long-range recall without giving up the efficiency of linear attention.\n\nMore on it below:\n\nLinear-attention and state-space models compress the whole prefix into a fixed-size state. That buys O(1) memory, but when many key-value associations compete, earlier facts get overwritten and needle recall degrades.\n\nHOLA gives linear attention a hippocampal complement. It keeps the usual delta-rule state as compressive memory and adds a bounded exact KV cache, forming a semiparametric test-time memory.\n\nThe state models linearly compressible structure while the cache stores associations that should not be forced through it. The cache writes without a learned eviction module, keeping only tokens whose prediction residual was actually committed to the state.\n\nAt 340M parameters on 15B SlimPajama tokens, HOLA lowers Wikitext perplexity from 27.32 to 22.92, below a full-attention Transformer++ at 26.88, and stays robust on RULER needle recall out to 32k tokens, 16x its training length.\n\nPaper: https://t.co/z1Jzp7qQ6B\n\nLearn to build effective AI agents in our academy: https://t.co/1e8RZKs4uX",
  "source": "Twitter for iPhone",
  "retweetCount": 3,
  "replyCount": 3,
  "likeCount": 20,
  "quoteCount": 0,
  "viewCount": 2324,
  "createdAt": "Fri Jul 03 15:38:01 +0000 2026",
  "lang": "en",
  "bookmarkCount": 21,
  "isReply": false,
  "inReplyToId": null,
  "conversationId": "2073068756293869685",
  "displayTextRange": [
    0,
    278
  ],
  "inReplyToUserId": null,
  "inReplyToUsername": null,
  "author": {
    "type": "user",
    "userName": "omarsar0",
    "url": "https://x.com/omarsar0",
    "twitterUrl": "https://twitter.com/omarsar0",
    "id": "3448284313",
    "name": "elvis",
    "isVerified": false,
    "isBlueVerified": true,
    "verifiedType": null,
    "profilePicture": "https://pbs.twimg.com/profile_images/939313677647282181/vZjFWtAn_normal.jpg",
    "coverPicture": "https://pbs.twimg.com/profile_banners/3448284313/1565974901",
    "description": "",
    "location": "DAIR.AI Academy",
    "followers": 309647,
    "following": 892,
    "status": "",
    "canDm": true,
    "canMediaTag": true,
    "createdAt": "Fri Sep 04 12:59:26 +0000 2015",
    "entities": {
      "description": {
        "urls": []
      },
      "url": {}
    },
    "fastFollowersCount": 0,
    "favouritesCount": 37286,
    "hasCustomTimelines": true,
    "isTranslator": true,
    "mediaCount": 4763,
    "statusesCount": 18588,
    "withheldInCountries": [],
    "affiliatesHighlightedLabel": {},
    "possiblySensitive": false,
    "pinnedTweetIds": [
      "2072735813469905026"
    ],
    "profile_bio": {
      "description": "Building self-improving AI @dair_ai • Prev: Meta AI | PhD • Learn about AI Agents for FREE here: https://t.co/P5SA9u54xO",
      "entities": {
        "description": {
          "urls": [
            {
              "display_url": "academy.dair.ai/courses/elemen…",
              "expanded_url": "https://academy.dair.ai/courses/elements-of-ai-agents",
              "indices": [
                97,
                120
              ],
              "url": "https://t.co/P5SA9u54xO"
            }
          ],
          "user_mentions": [
            {
              "id_str": "",
              "indices": [
                27,
                35
              ],
              "name": "",
              "screen_name": "dair_ai"
            }
          ]
        },
        "url": {
          "urls": [
            {
              "display_url": "dair.ai",
              "expanded_url": "https://www.dair.ai/",
              "indices": [
                0,
                23
              ],
              "url": "https://t.co/XQto5ypkSM"
            }
          ]
        }
      }
    },
    "isAutomated": false,
    "automatedBy": null
  },
  "extendedEntities": {
    "media": [
      {
        "display_url": "pic.twitter.com/P6o9juK2zA",
        "expanded_url": "https://twitter.com/omarsar0/status/2073068756293869685/photo/1",
        "ext_master_playlist_only": [],
        "ext_media_availability": {
          "status": "Available"
        },
        "ext_playlists": [],
        "features": {
          "large": {
            "faces": []
          },
          "orig": {
            "faces": []
          }
        },
        "id_str": "2073068752552558593",
        "indices": [
          279,
          302
        ],
        "media_key": "3_2073068752552558593",
        "media_results": {
          "id": "QXBpTWVkaWFSZXN1bHRzOgwAAQoAARzFBQv+24ABCgACHMUFDN3bYHUAAA==",
          "result": {
            "__typename": "ApiMedia",
            "id": "QXBpTWVkaWE6DAABCgABHMUFC/7bgAEKAAIcxQUM3dtgdQAA",
            "media_key": "3_2073068752552558593"
          }
        },
        "media_url_https": "https://pbs.twimg.com/media/HMUFC_7bgAE9_aa.jpg",
        "original_info": {
          "focus_rects": [
            {
              "h": 960,
              "w": 1714,
              "x": 0,
              "y": 0
            },
            {
              "h": 1714,
              "w": 1714,
              "x": 0,
              "y": 0
            },
            {
              "h": 1902,
              "w": 1668,
              "x": 0,
              "y": 0
            },
            {
              "h": 1902,
              "w": 951,
              "x": 48,
              "y": 0
            },
            {
              "h": 1902,
              "w": 1714,
              "x": 0,
              "y": 0
            }
          ],
          "height": 1902,
          "width": 1714
        },
        "sizes": {
          "large": {
            "h": 1902,
            "w": 1714
          }
        },
        "type": "photo",
        "url": "https://t.co/P6o9juK2zA"
      }
    ]
  },
  "card": null,
  "place": {},
  "entities": {
    "hashtags": [],
    "symbols": [],
    "urls": [
      {
        "display_url": "arxiv.org/abs/2607.02303",
        "expanded_url": "https://arxiv.org/abs/2607.02303",
        "indices": [
          1145,
          1168
        ],
        "url": "https://t.co/z1Jzp7qQ6B"
      },
      {
        "display_url": "academy.dair.ai",
        "expanded_url": "https://academy.dair.ai/",
        "indices": [
          1221,
          1244
        ],
        "url": "https://t.co/1e8RZKs4uX"
      }
    ],
    "user_mentions": []
  },
  "quoted_tweet": null,
  "retweeted_tweet": null,
  "isLimitedReply": false,
  "communityInfo": null,
  "article": null
}