@demian_ai it’s hard to convey this to most people but once people understand that when we hit the “collective” ceiling as to how capable llms can really be-that’s when everything downstream in the stack related to inference becomes more important.
waiting for that that inflection point
undoing positional encoding before compressing the K cache....simple idea that nobody was doing, unlocks 10x KV cache compression with zero accuracy loss
i just beat @GoogleDeepMind's turboquant
introducing Shard. 10x KV cache compression on Llama-3.1-8B. zero quality loss
- 10x @ 8K context, 11.2x @ 32K
- NIAH recall 1.000 across 4K-32K
- LongBench Δ ≈ 0 vs FP16
turboquant tops out at 4-6x at the same quality. we doubled it.
@JamesBondsama Amazing read! Say this a lot but you’re literally at a disadvantage if you’re not in Boardy’s network! He handles and works for you even when you’re not talking to him
69K Followers 6K FollowingWe're building the experiential layer of the Internet of Value - A persistent, shared 3D world where people & brands can own, create, connect & earn 🧬
382 Followers 335 FollowingEntrepreneur. Banker. Investor. Interests - Gadgets, task management apps, F1, AI, IOT, ANU - not necessarily in that order :)
22 Followers 35 FollowingHelping local heroes serve their communities better — by giving every small business owner the same unfair AI advantage that billion-dollar corporations have.
4K Followers 2K FollowingFounder, ChargeRight | IBEW Master Electrician | 70% of EV owners don't need a panel upgrade. I built the tool that proves it. NEC 220.82. @EV_ChargeRight
1K Followers 1K FollowingAI video creator telling African stories with AI. | Building the future of African film @ardaistudios | Teaching creators to do the same.
65 Followers 2K FollowingOptions Flow | Use link below for DISCOUNT on exclusive OWLS flow alerts and expert tips on how to spot flow yourself | SUB to me on X for exclusive information
17K Followers 2K FollowingGTM for @nebiustf @nebiusai // ex @Scaleway // from silicon to token, inference and anything in between. Views are my own - not financial advice
36K Followers 86 FollowingInvesting into the AI buildout ⚡️
Head of AI | 10 years as a Product Manager
Subscribe for trade ideas / build conviction
NFA just fun and vibes.
663K Followers 171 FollowingI only use X, beware of impersonators.
AI/Semi Supply Chain Analyst
ex. RISC-V FDN, AI research scientist; now trading unknown bottlenecks.
9K Followers 584 FollowingOwn your intelligence.
Precision-built inference for AI workloads at scale.
Powered by @nebiusai | Discord: https://t.co/SoJ89Kd4Wh
35K Followers 320 FollowingFounder of Edelbridge Capital. Concentrated public equities fund focused on AI infrastructure. Former pro gamer, YouTuber, and serial entrepreneur.
62K Followers 96 FollowingGiving you all the details about companies earnings
Charts by TradingView: https://t.co/6OyYSIcIN0
To support me: https://t.co/xaI23lGT66
4K Followers 2K FollowingFounder, ChargeRight | IBEW Master Electrician | 70% of EV owners don't need a panel upgrade. I built the tool that proves it. NEC 220.82. @EV_ChargeRight
4K Followers 4K FollowingWalko Systems | For humans that want agents and for agents that need governance | Sift | @WSSignal | https://t.co/nvRK4qckp3 | https://t.co/O0BvYT9JQV
585K Followers 50K FollowingSan Francisco/Silicon Valley AI | Robots, holodecks, BCIs, analysis of new things | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future.
104K Followers 243 FollowingSub to my X for exclusive post | Master flow yourself: Real-time tracking, AI alerts & instant phone notifications. Use link below — code FLOW for 20% off!