All tags
Company: "cursor"
not much happened today
claude-code codex composer-2.5 langchain cognition anthropic openai microsoft cursor agent-automation agent-observability ci-cd prompt-caching remote-execution verification decomposition feedback-loops coding-agents model-efficiency instruction-following krishdpi walden_yan russelljkaplan fchollet gabriberton palashshah shannholmberg
Agent infrastructure is advancing with LangSmith Engine providing CI/CD loops for agents and SmithDB enabling low-latency querying for observability. Cognition's Devin Auto-Triage offers persistent automation for bug triage with memory and subagent structures. Anthropic improves Claude Code for large codebases with prompt cache diagnostics and faster modes, while OpenAI enhances Codex workflows with remote execution and plugins. Microsoft released remote control for GitHub Copilot CLI and VS Code. The community emphasizes verification, decomposition, and feedback loops over prompt cleverness for coding agents. Cursor's Composer 2.5 is highlighted as a strong new coding model, with plans for a larger model trained with SpaceXAI using 10Γ more compute on Colossus 2 hardware, praised for efficiency and collaboration improvements.
not much happened today
claude codex langsmith-engine smithdb duet-agent multi-stream-llm delta-mem star-elastic cline langchain notion cursor nous-research nvidia datology agent-infrastructure developer-platforms observability long-running-state streaming orchestration pretraining-efficiency model-architecture external-memory post-training-compression data-curation vision-language-models jonas_geiping siddharth_joshi pratyush_maini
Cline, LangChain, Notion, and Cursor advanced agent infrastructure and developer platforms with innovations like Cline SDK, LangSmith Engine, SmithDB (offering 12β15Γ faster observability), and Notion's External Agents API integrating third-party agents such as Claude and Codex. Agent UX trends emphasize long-running state, streaming, and orchestration over chat, with tools like Duet Agent and VS Code Agents window enhancing durable execution and inspectable states. Research highlights include Nous Research's Token Superposition Training achieving 2β3Γ speedup in pretraining, a multi-stream LLM architecture for parallel reasoning by Jonas Geiping et al., and Ξ΄-mem external memory improving benchmark scores. NVIDIA's Star Elastic offers post-training model compression at 360Γ lower cost than pretraining, while Datology focuses on data curation for vision-language models.
Anthropic's Claude Opus 4.7
claude-opus-4.7 codex gpt-rosalind anthropic openai cursor replit perplexity-ai microsoft coding agentic-ai tokenization long-context benchmarking image-processing software-engineering computer-use plugin-integration multi-terminal-support ssh-access model-expansion bcherny kimmonismus scaling01 valsai artificialanlys natolambert nrehiew_
Anthropic launched Claude Opus 4.7, its most capable Opus model yet, featuring stronger coding and agentic performance, a new tokenizer, and improved long-context handling with a new xhigh reasoning tier. Benchmarks show substantial gains, including SWE-bench Pro 64.3%, SWE-bench Verified 87.6%, and TerminalBench 69.4%, with top rankings on Vals Index and GDPval-AA. Technical changes include a new tokenizer and increased image input resolution to 3.75MP. Some long-context benchmarks showed mixed results, with a shift in focus from MRCR to Graphwalks. Adoption was rapid across tools like Cursor, VS Code, Replit Agent, and Perplexity. Meanwhile, OpenAI expanded Codex into a broader computer agent with Mac computer use, in-app browser, image generation/editing, 90+ plugins, multi-terminal support, SSH remote devbox access, and richer file previews. A new vertical life-sciences model, GPT-Rosalind, was also introduced.
not much happened today
codex openai github cursor langchain nous-research agent-harnesses multi-agent-systems software-engineering tooling orchestration observability remote-control security-hardening user-experience open-source community-engagement andrew_ng steve_yegge gabrielchua giffmana rhys_sullivan teknium shaun_furman dabit3 robinebers zainanzhou nicoalbanese10 bromann elliothyun tiagonbotelho pierceboggan sydneyrunkle
Harness engineering is emerging as a key discipline in AI agent development, emphasizing components like filesystems, memory, and retries beyond just models. OpenAI's Codex is expanding agentic coding workflows beyond software engineering, including codebase understanding and bug triage. Tooling trends show convergence on multi-agent orchestration, observability, and remote control, with GitHub Copilot, Cursor, and LangChain advancing these capabilities. The Hermes Agent v0.9.0 release introduces a local web dashboard and enhanced security, gaining community traction over OpenClaw for UX and efficiency. The open agent ecosystem is growing with projects like Open Agents and DeepAgent providing modular stacks and runtimes.
not much happened today
gemini gemini-robotics-er-1.6 gpt-5.4-cyber deepagents-0.5 google tencent google-deepmind openai hugging-face cursor langchain agent-infrastructure cuda-optimization visual-reasoning spatial-reasoning gpu-kernels multi-agent-systems memory-management async-systems multimodality prompt-caching software-engineering robotics clementdelangue dylantfwang antoinersx steveschoettler teknium aiqiang888 sydneyrunkle
Google introduced Skills in Chrome, enabling reusable browser workflows with Gemini prompts and a library of ready-made Skills, enhancing end-user agentization. Tencent teased HYWorld 2.0, an open-source 3D world model generating editable scenes from a single image. Google DeepMind released Gemini Robotics-ER 1.6, improving visual/spatial reasoning for robotics with 93% instrument-reading success. OpenAI expanded Trusted Access with GPT-5.4-Cyber, a fine-tuned model for defensive security workflows. Hugging Face launched Kernels on the Hub, offering GPU kernel repos with 1.7xβ2.5x speedups. Cursor showcased a multi-agent CUDA optimization system with a 38% speedup across 235 problems. The Hermes Agent stack advanced to v0.9.0 with enhanced reliability, memory management, and integrations, while LangChain pushed deepagents 0.5 toward deployable, multi-tenant async systems with multimodal support and prompt caching. "Hermesβ key advantage is operational stability, extensibility, and deployability."
not much happened today
kimi-k2.5 claude-code cursor kimi fireworks anthropic langchain model-attribution fine-tuning reinforcement-learning open-source agent-products model-licensing software-integration product-differentiation clementdelangue leerob amanrsanger yuchenj_uw kimmonismus
Cursor's Composer 2, built on Kimi K2.5, sparked discussion over model attribution and licensing, highlighting a shift toward post-trained derivatives of open-source models with domain-specific fine-tuning and reinforcement learning. Claude Code is expanding into third-party tools like T3 Code and communication channels such as Telegram and Discord, while LangChain is evolving from orchestration to multi-agent products with offerings like Deep Agents/Open SWE and LangSmith Fleet. The discourse emphasizes the importance of clear base-model attribution, licensing compliance, and product differentiation through fine-tuning and user experience.
not much happened today
claude-code composer-2 cursor openai anthropic langchain cognition reinforcement-learning developer-tooling agent-systems agent-runtimes security credential-management multi-agent-systems model-training benchmarking software-engineering enterprise-ai kimmonismus mntruell theo ellev3n11 amanrsanger charliermarsh gdb yuchenj_uw neilhtennek simonw yuvalinthedeep lvwerra hrishioa
Cursor launched Composer 2, a frontier-class coding model with major cost reductions and strong benchmark scores like 61.3 on CursorBench and 73.7 on SWE-bench Multilingual. The model was improved via a first continued pretraining run feeding into reinforcement learning, trained across 3β4 clusters worldwide by a ~40-person team. OpenAI acquired Astral, the team behind Python tools uv, ruff, and ty, strengthening its developer platform. Anthropic expanded Claude Code with messaging app channels for persistent developer workflows. The focus in AI agents is shifting from single agents to managed fleets and runtimes, with LangChain launching LangSmith Fleet for enterprise agent management emphasizing agent identity, credential management, and auditability. Other launches include Cognition's teams of Devins, AgentUI by lvwerra, and discussions on agent runtimes with features like checkpointing and rollback. Security and permissions are emerging as critical constraints in agent system design.
Claude Code Anniversary + Launches from: Qwen 3.5, Cursor Demos, Cognition Devin 2.2, Inception Mercury 2
qwen3.5-flash qwen3.5-35b-a3b qwen3.5-122b-a10b qwen3.5-27b qwen3.5-397b-a17b gpt-5.3-codex claude-code alibaba openai anthropic cursor huggingface model-architecture reinforcement-learning quantization context-windows agentic-ai api websockets software-ux enterprise-workflows model-deployment awnihannun andrew_n_carr justinlin610 unslothai terryyuezhuo haihaoshen 0xsero ali_tongyilab scaling01 gdb noahzweben _catwu
Alibaba launched the Qwen 3.5 Medium Model Series featuring models like Qwen3.5-Flash, Qwen3.5-35B-A3B (MoE), and Qwen3.5-122B-A10B (MoE) emphasizing efficiency over scale with innovations like 1M context and INT4 quantization. OpenAI released GPT-5.3-Codex via the Responses API with enhanced file input support and faster web socket-based throughput. Anthropic introduced Claude Code Remote Control enabling terminal session continuation from mobile and expanded enterprise workflow features. Cursor shifted UX to agent demo videos instead of diffs, highlighting new interaction modes.
Claude Sonnet 4.6: clean upgrade of 4.5, mostly better with some caveats
claude-3-sonnet-4.6 claude-3-sonnet-4.5 claude-3-opus-4.5 claude-3-opus-4.6 anthropic cursor microsoft perplexity-ai cognition long-context agent-planning knowledge-work benchmarking tokenization model-integration code-execution model-updates aesthetic-quality alexalbert__ scaling01 rishdotblog claudeai kimmonismus artificialanlys
Anthropic launched Claude Sonnet 4.6, an upgrade over Sonnet 4.5, featuring broad improvements in coding, long-context reasoning, agent planning, knowledge work, and design, plus a 1M-token context window (beta). Benchmarks show Sonnet 4.6 leading on GDPval-AA ELO 1633, with significant token usage increases and improved output aesthetics. Integrations include Cursor, Windsurf, Microsoft Foundry, and Perplexity Pro/Max. Early user feedback noted some regression issues that were later fixed. Pricing remains the same as Sonnet 4.5. Tooling enhancements include code execution for filtering results, improving accuracy and efficiency.
Context Graphs: Hype or actually Trillion-dollar opportunity?
glm-ocr qwen3-coder-next sera-14b zhipu-ai lmsys vllm novita-labs ollama alibaba allenai cognition cursor multimodality ocr long-context mixture-of-experts agentic-coding-models context-graphs benchmarking model-deployment model-optimization model-training jaya_gupta dharmesh_shah
Zhipu AI launched GLM-OCR, a lightweight 0.9B multimodal OCR model excelling in complex document understanding with top benchmark scores and day-0 deployment support from lmsys, vllm, and novita labs. Ollama enabled local-first usage with easy offline operation. Alibaba released Qwen3-Coder-Next, an 80B MoE model with only 3B active parameters, designed for coding agents with a massive 256K context window and trained on 800K verifiable tasks, achieving over 70% SWE-Bench Verified. The open coding ecosystem also saw Allen AI announce SERA-14B, an on-device-friendly coding model with new datasets. The emerging concept of Context Graphs was highlighted as a promising framework for data and agent traceability, with initiatives like Cursor's Agent Trace specifying context graphs for coding agents, emphasizing potential improvements in agent performance and customer-driven adoption. This coverage reflects ongoing innovation in multimodality, long-context, mixture-of-experts, and agentic coding models.
not much happened today
claude-3 codex gemini gpt-5.2-pro anthropic openai google sakana-ai cursor baseten epoch-ai-research deepmind benchmarking reasoning continual-learning reinforcement-learning model-performance agentic-ai security model-training sama fchollet shane_legg demishassabis
Anthropic launches "Claude in Excel Pro" with enhanced features. OpenAI reveals upcoming Codex agent loop and cybersecurity measures. Google boosts Gemini App quotas and partners with Sakana AI for advanced AI Scientist projects in Japan. Cursor introduces Agent Skills for dynamic context focus. GPT-5.2 Pro achieves 31% on FrontierMath Tier 4, showing significant benchmark progress. Baseten raises $300M at a $5B valuation targeting high-performance inference. Discussions highlight math benchmarks as indicators of AI capability, uneven AGI progress, and the importance of reasoning and continual learning as future frontiers. Notable figures include Sam Altman, François Chollet, Shane Legg, and Demis Hassabis.
not much happened today.
gpt-5.2-codex glm-4.7 openai cursor github cerebras modal artificial-analysis vllm long-running-tasks autonomous-agents code-generation inference-speed latency batch-inference gpu-scaling model-evaluation agent-systems operational-scaling swyx kevinweil pierceboggan mntruell scaling01
OpenAI launched GPT-5.2-Codex API, touted as their strongest coding model for long-running tasks and cybersecurity. Cursor integrated GPT-5.2-Codex to autonomously run a browser for a week, producing over 3 million lines of Rust code. GitHub incorporated it into their code tools, easing enterprise adoption. Discussions highlight the importance of review loops in agent systems and debate evaluation metrics for coding models. OpenAI partnered with Cerebras to improve inference speed and latency, with Cerebras serving GLM-4.7 at 1,445 tokens/sec and low latency. Provider benchmarking reveals tradeoffs in throughput, latency, and context window sizes. Modal shared operational scaling insights for self-hosted inference fleets of 20k GPUs, focusing on batch inference optimization with vLLM and FlashInfer backend. This reflects a focus on inference infrastructure, long-horizon autonomous agents, and coding model evaluation.
not much happened today
nouscoder-14b deepseek-r1 langchain cursor huggingface openai weights-biases agent-frameworks context-management reinforcement-learning operational-safety model-transparency trajectory-exploration token-optimization coding-agents integration-platforms karpathy _philschmid omarsar0
AI News for 1/6/2026-1/7/2026 highlights a quiet day with key updates on LangChain DeepAgents introducing Ralph Mode for persistent agent loops, Cursor improving context management by reducing token usage by 46.9%, and operational safety measures for coding agents with allow/deny lists. MCP integration is expanding across assistants and robotics, with Hugging Face embedding assistants via HuggingChat + HF MCP server. The DeepSeek-R1 paper has been expanded to 86 pages, emphasizing trajectory exploration and RL shaping behavior. NousCoder-14B shows a +7% improvement on LiveCodeBench after 4 days of RL training, demonstrating advances in RL for coding with small open models. Top tweets also mention a viral "96GB RAM laptop", ChatGPT Health launch by OpenAI, and Karpathy's nanochat scaling-law miniseries.
not much happened today
nomos-1 axiomprover devstral-2-small deepseek-v3.2 claude-code cursor-2.2 claude-opus-4.5 gpt-5 claude-sonnet-4.5 gemini-3-pro llama qwen mistral gemma nousresearch thinkymachines mistral-ai deepseek anthropic cursor microsoft langchain-ai openai gemini intel vllm_project danielhanchen math formal-reasoning agentic-systems asynchronous-execution multi-agent-systems observability benchmarking quantization post-training-quantization training-speedup kernel-optimization inference-efficiency
NousResearch's Nomos 1 is a 30B open math model achieving a top Putnam score with only ~3B active parameters, enabling consumer Mac inference. AxiomProver also posts top Putnam results using ThinkyMachines' RL stack. Mistral's Devstral 2 Small outperforms DeepSeek v3.2 in 71% of preferences with better speed and cost. Anthropic's Claude Code introduces asynchronous agent execution. Cursor 2.2 adds deep agent primitives like Debug and Plan Modes. VS Code launches unified agent chat sessions improving multi-agent workflows. LangChain releases "Polly" for agent observability. The Stirrup harness leads OpenAI GDPval benchmarks with Claude Opus 4.5, GPT-5, and Gemini 3 Pro following. Advances in quantization include vLLM integrating Intel's AutoRound PTQ for efficient serving. Unsloth achieves up to 3Γ training speedups with new kernels across Llama, Qwen, Mistral, and Gemma models. "Compositional reasoning + specialized post-training under constrained active params can rival frontier closed models on formal math."
not much happened today
kimi-linear kimi-delta-attention minimax-m2 looped-llms aardvark-gpt-5 moonshot-ai minimax bytedance princeton mila openai cursor cognition hkust long-context attention-mechanisms agentic-ai tool-use adaptive-compute coding-agents performance-optimization memory-optimization reinforcement-learning model-architecture kimi_moonshot scaling01 uniartisan omarsar0 aicodeking songlinyang4 iscienceluvr nrehiew_ gdb embeddedsec auchenberg simonw
Moonshot AI released Kimi Linear (KDA) with day-0 infrastructure and strong long-context metrics, achieving up to 75% KV cache reduction and 6x decoding throughput. MiniMax M2 pivoted to full attention for multi-hop reasoning, maintaining strong agentic coding performance with 200k context and ~100 TPS. ByteDance, Princeton, and Mila introduced Looped LLMs showing efficiency gains comparable to larger transformers. OpenAI's Aardvark (GPT-5) entered private beta as an agentic security researcher for scalable vulnerability discovery. Cursor launched faster cloud coding agents, though transparency concerns arose regarding base-model provenance. Cognition released a public beta for a desktop/mobile tool-use agent named Devin. The community discussed advanced attention mechanisms and adaptive compute techniques.
3x in 3 months: Cursor @ $28b, Cognition + Windsurf @ $10b
qwen3-coder chatgpt-agent claude-code mini cursor cognition windsurf alibaba openai anthropic perplexity agentic-ai fundraising software-engineering ai-coding agentic-economy model-integration community-feedback performance-benchmarking bindureddy xikun_zhang_ aravsrinivas gergelyorosz jeremyphoward
Cursor is reportedly fundraising at a $28 billion valuation with $1 billion ARR, while the combined Cognition+Windsurf entity is fundraising at a $10 billion valuation after acquiring Windsurf remainco for $300 million. The competition between AI coding agents intensifies as Cursor focuses on Async SWE Agents and Cognition+Windsurf acquires an agentic IDE. Alibaba's Qwen3-Coder gains widespread adoption for coding tasks and integration into tools like Claude Code and LM Studio. OpenAI rolls out ChatGPT Agent to all Plus, Pro, and Team users, sparking discussions about an "agentic economy" emphasizing AI literacy. Anthropic's Claude Code is praised as a premier development tool with active community feedback. Perplexity's Comet browser assistant receives positive reviews and new feature showcases. The debate continues on whether AI coding tools will replace developers, with critiques highlighting the ongoing human effort required. A new minimalistic software engineering agent, mini, achieves 65% on SWE-bench with just 100 lines of code.
Grok 4: xAI succeeds in going from 0 to new SOTA LLM in 2 years
grok-4 grok-4-heavy claude-4-opus xai perplexity-ai langchain cursor cline model-releases benchmarking long-context model-pricing model-integration voice performance scaling gpu-optimization elonmusk aravsrinivas igor_babuschkin yuchenj_uw
xAI launched Grok 4 and Grok 4 Heavy, large language models rumored to have 2.4 trillion parameters and trained with 100x more compute than Grok 2 on 100k H100 GPUs. Grok 4 achieved new state-of-the-art results on benchmarks like ARC-AGI-2 (15.9%), HLE (50.7%), and Vending-Bench, outperforming models such as Claude 4 Opus. The model supports a 256K context window and is priced at $3.00/M input tokens and $15.00/M output tokens. It is integrated into platforms like Cursor, Cline, LangChain, and Perplexity Pro/Max. The launch was accompanied by a controversial voice mode and sparked industry discussion about xAI's rapid development pace, with endorsements from figures like Elon Musk and Arav Srinivas.
AI Engineer World's Fair Talks Day 1
gemini-2.5 gemma claude-code mistral cursor anthropic openai aie google-deepmind meta-ai-fair agent-based-architecture open-source model-memorization scaling-laws quantization mixture-of-experts language-model-memorization model-generalization langgraph model-architecture
Mistral launched a new Code project, and Cursor released version 1.0. Anthropic improved Claude Code plans, while ChatGPT announced expanded connections. The day was dominated by AIE keynotes and tracks including GraphRAG, RecSys, and Tiny Teams. On Reddit, Google open-sourced the DeepSearch stack for building AI agents with Gemini 2.5 and LangGraph, enabling flexible agent architectures and integration with local LLMs like Gemma. A new Meta paper analyzed language model memorization, showing GPT-style transformers store about 3.5β4 bits/parameter and exploring the transition from memorization to generalization, with implications for Mixture-of-Experts models and quantization effects.
Cursor @ $9b, OpenAI Buys Windsurf @ $3b
llama-nemotron-ultra llama-nemotron-super llama-nemotron-nano qwen3-235b-a22b prover-v2 phi-4-reasoning ernie-4.5-turbo ernie-x1-turbo suno-v4.5 gen-4-references o1-mini openai cursor nvidia alibaba deepseek microsoft baidu suno runway keras reasoning inference-efficiency open-license moe-models math-reasoning theorem-proving model-performance music-generation image-generation recommender-systems tpu-optimization _akhaliq adcock_brett lmarena_ai fchollet
OpenAI is reportedly close to closing a deal with Windsurf, coinciding with Cursor's $900M funding round at a $9B valuation. Nvidia launched the Llama-Nemotron series featuring models from 8B to 253B parameters, praised for reasoning and inference efficiency. Alibaba released the Qwen3 family with MoE and dense models up to 235B parameters, ranking highly in coding and math benchmarks. DeepSeek introduced Prover-V2, an open-source AI for math reasoning with an 88.9% pass rate on MiniF2F-test. Microsoft released reasoning-focused Phi-4 models, outperforming OpenAI's o1-mini. Baidu debuted turbo versions of ERNIE 4.5 and X1 for faster, cheaper inference. Suno v4.5 added advanced AI music generation features, while Runway Gen-4 References enable placing characters into scenes with high consistency. KerasRS, a new recommender system library optimized for TPUs, was released by Fran ois Chollet.
not much happened today
phi-4 phi-4-mini-reasoning qwen3-235b qwen3-moe-235b qwen3-moe-30b qwen3-dense-32b qwen3-dense-14b qwen3-dense-8b qwen3-dense-4b qwen3-dense-0.6b qwen2.5-omni-3b deepseek-prover-v2 llama llama-guard-4 prompt-guard-2 mimo-7b microsoft anthropic cursor alibaba togethercompute deepseek meta-ai-fair xiaomi openrouterai cohere reasoning model-fine-tuning model-evaluation benchmarking model-popularity open-source math model-scaling model-filtering jailbreak-prevention cline reach_vb vipulved akhaliq omarsar0 zhs05232838 huajian_xin mervenoyann karpathy random_walker sarahookr blancheminerva clefourrier
Microsoft released Phi-reasoning 4, a finetuned 14B reasoning model slightly behind QwQ but limited by data transparency and token efficiency issues. Anthropic introduced remote MCP server support and a 45-minute Research mode in Claude. Cursor published a model popularity list. Alibaba launched Qwen3-235B and other Qwen3 variants, highlighting budget-friendly coding and reasoning capabilities, with availability on Together AI API. Microsoft also released Phi-4-Mini-Reasoning with benchmark performance on AIME 2025 and OmniMath. DeepSeek announced DeepSeek-Prover V2 with state-of-the-art math problem solving, scaling to 671B parameters. Meta AI's Llama models hit 1.2 billion downloads, with new Llama Guard 4 and Prompt Guard 2 for input/output filtering and jailbreak prevention. Xiaomi released the open-source reasoning model MiMo-7B trained on 25 trillion tokens. Discussions on AI model evaluation highlighted issues with the LMArena leaderboard, data access biases favoring proprietary models, and challenges in maintaining fair benchmarking, with suggestions for alternatives like OpenRouterAI rankings. "LMArena slop and biased" and "61.3% of all data going to proprietary model providers" were noted concerns.
>$41B raised today (OpenAI @ 300b, Cursor @ 9.5b, Etched @ 1.5b)
deepseek-v3-0324 gemini-2.5-pro claude-3.7-sonnet openai deepseek gemini cursor etched skypilot agent-evals open-models model-releases model-performance coding multimodality model-deployment cost-efficiency agent-evaluation privacy kevinweil sama lmarena_ai scaling01 iscienceluvr stevenheidel lepikhin dzhng raizamrtn karpathy
OpenAI is preparing to release a highly capable open language model, their first since GPT-2, with a focus on reasoning and community feedback, as shared by @kevinweil and @sama. DeepSeek V3 0324 has achieved the #5 spot on the Arena leaderboard, becoming the top open model with an MIT license and cost advantages. Gemini 2.5 Pro is noted for outperforming models like Claude 3.7 Sonnet in coding tasks, with upcoming pricing and improvements expected soon. New startups like Sophont are building open multimodal foundation models for healthcare. Significant fundraises include Cursor closing $625M at a $9.6B valuation and Etched raising $85M at $1.5B. Innovations in AI infrastructure include SkyPilot's cost-efficient cloud provisioning and the launch of AgentEvals, an open-source package for evaluating AI agents. Discussions on smartphone privacy highlight iPhone's stronger user defense compared to Android.
Cerebras Inference: Faster, Better, AND Cheaper
llama-3.1-8b llama-3.1-70b gemini-1.5-flash gemini-1.5-pro cogvideox-5b mamba-2 rene-1.3b llama-3.1 gemini-1.5 claude groq cerebras cursor google-deepmind anthropic inference-speed wafer-scale-chips prompt-caching model-merging benchmarking open-source-models code-editing model-optimization jeremyphoward sam-altman nat-friedman daniel-gross swyx
Groq led early 2024 with superfast LLM inference speeds, achieving ~450 tokens/sec for Mixtral 8x7B and 240 tokens/sec for Llama 2 70B. Cursor introduced a specialized code edit model hitting 1000 tokens/sec. Now, Cerebras claims the fastest inference with their wafer-scale chips, running Llama3.1-8b at 1800 tokens/sec and Llama3.1-70B at 450 tokens/sec at full precision, with competitive pricing and a generous free tier. Google's Gemini 1.5 models showed significant benchmark improvements, especially Gemini-1.5-Flash and Gemini-1.5-Pro. New open-source models like CogVideoX-5B and Mamba-2 (Rene 1.3B) were released, optimized for consumer hardware. Anthropic's Claude now supports prompt caching, improving speed and cost efficiency. "Cerebras Inference runs Llama3.1 20x faster than GPU solutions at 1/5 the price."
Cursor reaches >1000 tok/s finetuning Llama3-70b for fast file editing
gpt-4 gpt-4o gpt-4-turbo gpt-4o-mini llama bloom stable-diffusion cursor openai anthropic google-deepmind huggingface speculative-decoding code-edits multimodality image-generation streaming tool-use fine-tuning benchmarking mmlu model-performance evaluation synthetic-data context-windows sama abacaj imjaredz erhartford alexalbert svpino maximelabonne _philschmid
Cursor, an AI-native IDE, announced a speculative edits algorithm for code editing that surpasses GPT-4 and GPT-4o in accuracy and latency, achieving speeds of over 1000 tokens/s on a 70b model. OpenAI released GPT-4o with multimodal capabilities including audio, vision, and text, noted to be 2x faster and 50% cheaper than GPT-4 turbo, though with mixed coding performance. Anthropic introduced streaming, forced tool use, and vision features for developers. Google DeepMind unveiled Imagen Video and Gemini 1.5 Flash, a small model with a 1M-context window. HuggingFace is distributing $10M in free GPUs for open-source AI models like Llama, BLOOM, and Stable Diffusion. Evaluation insights highlight challenges with LLMs on novel problems and benchmark saturation, with new benchmarks like MMLU-Pro showing significant drops in top model performance.