Our response to the TanStack npm supply chain attack

Fine-tune LLM with Databricks Unity Catalog and Amazon SageMaker AI

Artificial Intelligence Fine-tune LLM with Databricks Unity Catalog and Amazon SageMaker AI When you fine-tune large language models (LLMs) with Amazon SageMaker AI while using Databricks Unity Catalog, you might face unique challenges like how to maintain strict data governance while using best-in-class machine learning (ML) services. Unity Catalog governs metadata and permissions, while the underlying data resides in Amazon Simple Storage Service (Amazon S3) when you choose AWS as the cloud environment for their Databricks Workspace. When SageMaker AI Training job accesses that data, you must preserve and not bypass the Unity Catalog’s fine-grained authorization model. Without a structured integration pattern, you risk inconsistent policy enforcement, audit gaps, and compliance exposure. For example, if SageMaker AI Training jobs bypass Unity Catalog’s authorization model when reading S3 objects, you lose visibility into which data trained which models. This creates critical…

Build financial document processing with Pulse AI and Amazon Bedrock

Artificial Intelligence Build financial document processing with Pulse AI and Amazon Bedrock Financial institutions process thousands of complex documents daily. Optical Character Recognition (OCR) errors in financial data can propagate through interconnected calculations, affecting analytical accuracy. While a single OCR error in a standard legal document might require only a quick manual correction, the same mistake in financial data can cascade through interconnected calculations, leading to systematic errors in analysis and potentially costly to organizations. Traditional OCR tools fall critically short when processing the complex financial documents that institutions handle daily—balance sheets, income statements, SEC filings, research reports, and audit materials. These documents feature intricate table structures with merged cells and hierarchical data, multi-column layouts with interconnected references, and context-dependent information requiring semantic understanding. Traditional OCR approaches treat these documents as images, missing the structural relationships and contextual nuances that…

Navigating EU AI Act requirements for LLM fine-tuning on Amazon SageMaker AI

Artificial Intelligence Navigating EU AI Act requirements for LLM fine-tuning on Amazon SageMaker AI The EU AI Act requires organizations fine-tuning large language models (LLMs) to track computational resources measured in floating-point operations (FLOPs) to determine compliance obligations. As customers increasingly fine-tune LLMs for domain-specific use cases, we hear a common question: how do I know if my training job triggers new regulatory obligations? Amazon SageMaker AI provides a managed machine learning (ML) service for building, training, and deploying models. This solution uses Amazon SageMaker Training jobs to run fine-tuning workloads on fully managed infrastructure. SageMaker Training jobs handle resource provisioning, scaling, and cluster management, with built-in support for distributed training, integration with AWS CloudTrail and Amazon CloudWatch for governance, and automatic decommissioning of compute resources after training completes. The Fine-Tuning FLOPs Meter extends these capabilities with purpose-built compliance tracking…

Automate schema generation for intelligent document processing

Artificial Intelligence Automate schema generation for intelligent document processing Before you can extract information from documents using intelligent document processing (IDP) techniques, you need a schema for each document class that defines what to extract. But how do you create schemas when you have thousands of documents and don’t know what classes exist? Doing this at scale can take substantial manual effort, making downstream IDP initiatives difficult to justify. In this post, we’ll show you how our multi-document discovery feature solves this problem. It serves as an automated pre-processing step, analyzing unknown documents, clustering them by type, and generating schemas ready for the IDP Accelerator. You’ll learn how the new capability uses visual embeddings for automatic clustering and agents for schema generation. We’ll also walk you through running the solution on your own document collections. IDP Accelerator The IDP Accelerator…

Building web search-enabled agents with Strands and Exa

Artificial Intelligence Building web search-enabled agents with Strands and Exa This post is co written by Ishan Goswami and Nitya Sridhar from Exa. If you are building web search-enabled AI agents for research, fact-checking, or competitive intelligence, access to current and reliable information is critical. Most general-purpose search APIs are not designed for agent workflows. They return HTML-heavy pages and short snippets optimized for human browsing, not structured data that an agent can directly consume. As a result, developers often need to build additional layers, custom crawlers, parsers, and ranking logic, to transform this content into something usable within an agent workflow. The Exa integration for the Strands Agents SDK addresses this gap with an AI-native search and retrieval layer built directly into the tool interface. Exa delivers clean, structured content formatted for direct use in LLM context windows, without…

Amazon Quick: Accelerating the path from enterprise data to AI-powered decisions

Artificial Intelligence Amazon Quick: Accelerating the path from enterprise data to AI-powered decisions Enterprise data with tens of millions of rows, row-level and column-level security, and dozens of datasets spanning multiple business domains need AI-generated answers that are trustworthy, reproducible, and fast, while respecting governance rules consistently. With foundation models (FMs), organizations can build systems that work well for small datasets where a business user asks a question about their data and gets an answer in seconds. Amazon Quick can also help turn your large enterprise data into fast and accurate AI-powered decisions. In this post, you will learn about five new capabilities of Amazon Quick that accelerate how data professionals deliver trusted AI-powered insights at enterprise scale. Dataset Q&A: Talk to your data directly When a VP asks, “How is churn trending for this product?“, getting that answer means…

Overcoming reward signal challenges: Verifiable rewards-based reinforcement learning with GRPO on SageMaker AI

Artificial Intelligence Overcoming reward signal challenges: Verifiable rewards-based reinforcement learning with GRPO on SageMaker AI Training large language models requires accurate feedback signals, but traditional reinforcement learning (RL) often struggles with reward signal reliability. The quality of these signals directly influences how models learn and make decisions. However, creating robust feedback mechanisms can be complex and error prone. Real-world training scenarios often introduce hidden biases, unintended incentives, and ambiguous success criteria that can derail the learning process, leading to models that behave unpredictably or fail to meet desired objectives. In this post, you will learn how to implement reinforcement learning with verifiable rewards (RLVR) to introduce verification and transparency into reward signals to improve training performance. This approach works best when outputs can be objectively verified for correctness, such as in mathematical reasoning, code generation, or symbolic manipulation tasks. You…

Secure short-term GPU capacity for ML workloads with EC2 Capacity Blocks for ML and SageMaker training plans

Artificial Intelligence Secure short-term GPU capacity for ML workloads with EC2 Capacity Blocks for ML and SageMaker training plans As companies of various sizes adopt graphic processing units (GPU)-based machine learning (ML) training, fine-tuning and inference workloads, the demand for GPU capacity has outpaced industry-wide supply. This imbalance has made GPUs a scarce resource, creating a challenge for customers who need reliable access to GPU compute resources for their ML workloads. When you encounter GPU capacity limitations, you might consider creating on-demand capacity reservations (ODCRs). ODCRs apply to planned, steady-state workloads with well-understood usage patterns. Short-term ODCR availability for GPU instances, particularly P-type instances, is often limited. Additionally, without a long-term contract, ODCRs are billed at on-demand rates, offering no cost advantage. This makes ODCRs unsuitable for short or exploratory workloads such as testing, evaluations, or events. A guided approach…

Intelligence-driven message defense and insights using Amazon Bedrock

Artificial Intelligence Intelligence-driven message defense and insights using Amazon Bedrock Direct communication between buyers and sellers outside approved channels can result in significant revenue loss annually while severely damaging brand reputation and destroying valuable business relationships. While messaging systems are essential for modern business operations and help provide rich customer insights, they can create significant risks when parties bypass the brokerage system to communicate directly. When buyers and sellers exchange contact information and take their transactions offline, brokerages can not only lose immediate revenue but also suffer long-term damage as their marketplace value diminishes. This challenge is particularly acute in brokerage businesses where the service’s core value lies in facilitating secure, reliable connections between parties. While in-application messaging enables important transaction details, such as delivery placement “leave it by the back door” or specific times “only deliver after 4:00 PM”,…

Introducing Dataset Q&A: Expanding natural language querying for structured datasets in Amazon Quick

Artificial Intelligence Introducing Dataset Q&A: Expanding natural language querying for structured datasets in Amazon Quick Every BI team knows this bottleneck: a business user has a question that falls outside existing dashboards, so they file a ticket. An analyst writes the query, validates the results, and delivers them—hours or days later. Multiply that by hundreds of ad-hoc requests per month, and the backlog becomes the single biggest constraint on data team productivity. Amazon Quick now adds a powerful new natural language query capability, Dataset Q&A, to remove this bottleneck. Your question is translated into SQL, run against the full dataset, and the results are returned in seconds—no row sampling, topic curation, or pre-configured calculated fields required. Quick already offers two natural language querying modes. Dashboard Q&A is intended for questions about data visualized in published dashboards, drawing on the business…

Agent-guided workflows to accelerate model customization in Amazon SageMaker AI

Artificial Intelligence Agent-guided workflows to accelerate model customization in Amazon SageMaker AI Every organization has access to the same foundation models. The real competitive advantage comes from customizing them with your proprietary data and domain expertise. But getting there is complex, even for experienced teams. It requires mastering fine-tuning techniques like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning Verifiable Rewards (RLVR), navigating fragmented APIs and model-specific data formats, designing rigorous evaluations, and managing months-long experiment cycles. Amazon SageMaker AI now offers an agentic experience that changes this. Developers describe their use case using natural language, and the AI coding agent streamlines the entire journey, from use case definition and data preparation through technique selection, evaluation, and deployment. Purpose-built agent skills deliver specialized expertise on fine-tuning applied to your specific use case, data transformation to required formats, quality…

Sun Finance automates ID extraction and fraud detection with generative AI on AWS

Artificial Intelligence Sun Finance automates ID extraction and fraud detection with generative AI on AWS This post was co-authored with Krišjānis Kočāns, Kaspars Magaznieks, Sergei Kiriasov from Sun Finance Group If you process identity documents at scale—loan applications, account openings, compliance checks—you’ve likely hit the same wall: traditional optical character recognition (OCR) gets you partway there, but extraction errors still push a large share of applications into manual review queues. Add fraud detection to the mix, and the manual workload compounds. Sun Finance, a Latvian fintech founded in 2017, operates as a technology-first online lending marketplace across nine countries. The company processes a new loan request every 0.63 seconds and delivers more than 4 million evaluations monthly. In one of their highest-volume industries, with 80,000 monthly applications for microloans, approximately 60% of applications required manual operator review. Sun Finance partnered…

AWS Generative AI Model Agility Solution: A comprehensive guide to migrating LLMs for generative AI production

Artificial Intelligence AWS Generative AI Model Agility Solution: A comprehensive guide to migrating LLMs for generative AI production Maintaining model agility is crucial for organizations to adapt to technological advancements and optimize their artificial intelligence (AI) solutions. Whether transitioning between different large language model (LLM) families or upgrading to newer versions within the same family, a structured migration approach and a standardized process are essential for facilitating continuous performance improvement while minimizing operational disruptions. However, developing such a solution is challenging in both technical and non-technical aspects because the solution needs to: - Be generic to cover a variety of use cases - Be specific so that a new user can apply it to the target use case - Provide comprehensive and fair comparison between LLMs - Be automated and scalable - Incorporate domain- and task-specific knowledge and inputs -…

Run custom MCP proxies serverless on Amazon Bedrock AgentCore Runtime

Artificial Intelligence Run custom MCP proxies serverless on Amazon Bedrock AgentCore Runtime When AI agents connect to tools through the Model Context Protocol (MCP), they gain access to capabilities that range from database queries and API calls to file operations and third-party service integrations. In production, these interactions need proper governance, controls, and observability aligned with an organization’s security policies. This includes sanitizing tool inputs before they reach backend systems, generating audit trails in specific formats, or redacting sensitive data at the protocol layer. These requirements are shaped by internal governance standards, industry regulations, and the specifics of each production environment. This post shows you how to deploy a serverless MCP proxy on Amazon Bedrock AgentCore Runtime that gives you a programmable layer to implement these controls. Amazon Bedrock AgentCore Gateway provides centralized governance and control for agent-tool integration, including…

Building AI-ready data: Vanguard’s Virtual Analyst journey

Artificial Intelligence Building AI-ready data: Vanguard’s Virtual Analyst journey Vanguard is a global investment management firm, offering a broad selection of investments, advice, retirement services, and insights to individual investors, institutions, and financial professionals. We operate under a unique, investor-owned structure and adhere to a straightforward purpose: To take a stand for all investors, to treat them fairly, and to give them the best chance for investing success. When Vanguard’s financial analysts needed to query complex datasets, they faced a frustrating reality: even basic questions required writing intricate SQL queries and sometimes long response times from data teams. This challenge is not unique to Vanguard: conversational AI is a scalable solution, providing analysts immediate responses. However, deploying conversational AI requires more than choosing the right foundation model—it requires AI-ready data infrastructure. In this post, you’ll learn how Vanguard built their…

Organizing Agents’ memory at scale: Namespace design patterns in AgentCore Memory

Artificial Intelligence Organizing Agents’ memory at scale: Namespace design patterns in AgentCore Memory When building AI agents, developers struggle with organizing memory across sessions, which leads to irrelevant context retrieval and security vulnerabilities. AI agents that remember context across sessions need more than only storage. They need organized, retrievable, and secure memory. In Amazon Bedrock AgentCore Memory, namespaces determine how long-term memory records are organized, retrieved, and who can access them. Getting the namespace design right is essential to building an effective memory system. In this post, you will learn how to design namespace hierarchies, choose the right retrieval patterns, and implement AWS Identity and Access Management (IAM)-based access control for AgentCore Memory. If you’re new to AgentCore Memory, we recommend reading our introductory blog post first: Amazon Bedrock AgentCore Memory: Building context-aware agents. What are namespaces? Namespaces are hierarchical…

NVIDIA Nemotron 3 Nano Omni model now available on Amazon SageMaker JumpStart

Artificial Intelligence NVIDIA Nemotron 3 Nano Omni model now available on Amazon SageMaker JumpStart Today, we are excited to announce the day zero availability of NVIDIA Nemotron 3 Nano Omni on Amazon SageMaker JumpStart. This multimodal model from NVIDIA combines video, audio, image, and text understanding into a single, efficient architecture, enabling enterprise customers to build intelligent applications that can see, hear, and reason across modalities in one inference pass. In this post, we walk through the model architecture and key capabilities of Nemotron 3 Nano Omni, explore the enterprise use cases it unlocks, and show you how to deploy and run inference using Amazon SageMaker JumpStart. Overview of NVIDIA Nemotron 3 Nano Omni NVIDIA Nemotron 3 Nano Omni is an open, multimodal large language model with 30 billion total parameters and 3 billion active parameters (30B A3B). It is…

Build Strands Agents with SageMaker AI models and MLflow

Artificial Intelligence Build Strands Agents with SageMaker AI models and MLflow Enterprises building AI agents often require more than what managed foundation model (FM) services can provide. They need precise control over performance tuning, cost optimization at scale, compliance and data residency, model selection, and networking configurations that integrate with existing security architectures. Amazon SageMaker AI endpoints align with these requirements by giving organizations control over compute resources, scaling behavior, and infrastructure placement, while benefiting from the managed operational layer of AWS. These models that are deployed by SageMaker AI, can power AI agents, handle conversational workloads, and integrate with orchestration frameworks like the FMs that are available on Amazon Bedrock. The difference is that the organization retains architectural control over how and where inference happens. In this post, we demonstrate how to build AI agents using Strands Agents SDK…

Automate repetitive tasks with Amazon Quick Flows

Artificial Intelligence Automate repetitive tasks with Amazon Quick Flows Consider a typical Monday morning: you’re manually copying data from several different systems to create a weekly report, then formatting it for different stakeholders. This single task can consume several hours that could be spent on more strategic work. Multiply this across your team, and these repetitive tasks add up quickly. Amazon Quick Flows automates these tasks using AI workflows. With Quick Flows, you create intelligent workflows using natural language—no coding or machine learning (ML) expertise required. You describe what you want automated, and Quick Flows builds it for you. This post shows you how to build your first AI-powered workflow, starting with a financial analysis tool and progressing to an advanced employee onboarding automation. What is Amazon Quick Flows? Amazon Quick Flows is part of Amazon Quick, a collection of…

Company-wise memory in Amazon Bedrock with Amazon Neptune and Mem0

Artificial Intelligence Company-wise memory in Amazon Bedrock with Amazon Neptune and Mem0 This post is cowritten by Shawn Tsai from TrendMicro. Delivering relevant, context-aware responses is important for customer satisfaction. For enterprise-grade AI chatbots, understanding not only the current query but also the organizational context behind it is key. Company-wise memory in Amazon Bedrock, powered by Amazon Neptune and Mem0, provides AI agents with persistent, company-specific context—enabling them to learn, adapt, and respond intelligently across multiple interactions. TrendMicro, one of the largest antivirus software companies in the world, developed the Trend’s Companion chatbot, so their customers can explore information through natural, conversational interactions (learn more). TrendMicro aimed to enhance its AI chatbot service to deliver personalized, context-aware support for enterprise customers. The chatbot needed to retain conversation history for continuity, reference company-specific knowledge at scale, and ensure that memory remained…

Cost-effective multilingual audio transcription at scale with Parakeet-TDT and AWS Batch

Artificial Intelligence Cost-effective multilingual audio transcription at scale with Parakeet-TDT and AWS Batch Many organizations are archiving large media libraries, analyzing contact center recordings, preparing training data for AI, or processing on-demand video for subtitles. When data volumes grow significantly, managed automatic speech recognition (ASR) service costs can quickly become the primary constraint on scalability. To address this cost-scalability challenge, we use the NVIDIA Parakeet-TDT-0.6B-v3 model, deployed through AWS Batch on GPU-accelerated instances. Parakeet-TDT’s Token-and-Duration Transducer architecture simultaneously predicts text tokens and their duration to intelligently skip silence and redundant processing. This helps achieve inference speeds orders of magnitude faster than real-time. By paying only for brief bursts of compute rather than the full length of your audio, you can transcribe at scale for fractions of a cent per hour of audio based on the benchmarks described in this post.…

End-to-end lineage with DVC and Amazon SageMaker AI MLflow apps

Artificial Intelligence End-to-end lineage with DVC and Amazon SageMaker AI MLflow apps Production machine learning (ML) teams struggle to trace the full lineage of a model through the data and the code that trained it, the exact dataset version it consumed, and the experiment metrics that justified its deployment. Without this traceability, questions like “which data trained the model currently in production?” or “can we reproduce the model we deployed six months ago?” become multi-day investigations through scattered logs, notebooks, and Amazon Simple Storage Service (Amazon S3) buckets. This gap is especially acute in regulated industries. For example, healthcare, financial services, autonomous vehicles, where audit requirements demand that you link deployed models to their precise training data, and where individual records might need to be excluded from future training on request. In this post, we show how to combine three…

Omnichannel ordering with Amazon Bedrock AgentCore and Amazon Nova 2 Sonic

Artificial Intelligence Omnichannel ordering with Amazon Bedrock AgentCore and Amazon Nova 2 Sonic Introduction Building a voice-enabled ordering system that works across mobile apps, websites, and voice interfaces (an omnichannel approach) presents real challenges. You need to process bidirectional audio streams, maintain conversation context across multiple turns, integrate backend services without tight coupling, and scale to handle peak traffic. In this post, we’ll show you how to build a complete omnichannel ordering system using Amazon Bedrock AgentCore, an agentic platform, to build, deploy, and operate highly effective AI agents securely at scale using any framework and foundation model and Amazon Nova 2 Sonic. You’ll deploy infrastructure that handles authentication, processes orders, and provides location-based recommendations. The system uses managed services that scale automatically, reducing the operational overhead of building voice AI applications. By the end, you’ll have a working system…

Nova Forge SDK series part 2: Practical guide to fine-tune Nova models using data mixing capabilities

Artificial Intelligence Nova Forge SDK series part 2: Practical guide to fine-tune Nova models using data mixing capabilities This hands-on guide walks through every step of fine-tuning an Amazon Nova model with the Amazon Nova Forge SDK, from data preparation to training with data mixing to evaluation, giving you a repeatable playbook you can adapt to your own use case. This is the second part in our Nova Forge SDK series, building on the SDK introduction and first part, which covered kicking off customization experiments. The focus of this post is data mixing: the technique that lets you fine-tune on domain-specific data without sacrificing a model’s general capabilities. In the previous post, we made the case for why this matters, blending customer data with Amazon-curated datasets preserved near-baseline Massive Multitask Language Understanding (MMLU) scores while delivering a 12-point F1 improvement…

Power video semantic search with Amazon Nova Multimodal Embeddings

Artificial Intelligence Power video semantic search with Amazon Nova Multimodal Embeddings Video semantic search is unlocking new value across industries. The demand for video-first experiences is reshaping how organizations deliver content, and customers expect fast, accurate access to specific moments within video. For example, sports broadcasters need to surface the exact moment a player scored to deliver highlight clips to fans instantly. Studios need to find every scene featuring a specific actor across thousands of hours of archived content to create personalized trailers and promotional content. News organizations need to retrieve footage by mood, location, or event to publish breaking stories faster than competitors. The goal is the same: deliver video content to end users quickly, capture the moment, and monetize the experience. Video is naturally more complex than other modalities like text or image because it amalgamates multiple unstructured…

Optimize video semantic search intent with Amazon Nova Model Distillation on Amazon Bedrock

Artificial Intelligence Optimize video semantic search intent with Amazon Nova Model Distillation on Amazon Bedrock Optimizing models for video semantic search requires balancing accuracy, cost, and latency. Faster, smaller models lack routing intelligence, while larger, accurate models add significant latency overhead. In Part 1 of this series, we showed how to build a multimodal video semantic search system on AWS with intelligent intent routing using the Anthropic Claude Haiku model in Amazon Bedrock. While the Haiku model delivers strong accuracy for user search intent, it increases end-to-end search time to 2-4 seconds. This contributes to 75% of the overall latency. Now consider what happens as the routing logic grows more complex. Enterprise metadata can be far more complex than the five attributes in our example (title, caption, people, genre, and timestamp). Customers may factor in camera angles, mood and sentiment,…

How Automated Reasoning checks in Amazon Bedrock transform generative AI compliance

Artificial Intelligence How Automated Reasoning checks in Amazon Bedrock transform generative AI compliance Compliance teams in regulated industries spend weeks on manual reviews, pay for outside consultants, and still face audit gaps when AI outputs lack formal proof. Automated Reasoning checks in Amazon Bedrock Guardrails address this by replacing probabilistic AI validation with mathematical verification, turning AI-generated decisions into provably correct, auditable results. In this post, you’ll learn why probabilistic AI validation falls short in regulated industries and how Automated Reasoning checks use formal verification to deliver mathematically proven results. You’ll also see how customers across six industries use this technology to produce formally verified, auditable AI outputs, and how to get started. The compliance challenge Regulated industries face high-stakes compliance challenges. Hospitals navigate radiation safety regulations. Financial institutions classify AI risk under the EU AI Act. Insurance carriers answer…

Transform retail with AWS generative AI services

Artificial Intelligence Transform retail with AWS generative AI services Online retailers face a persistent challenge: shoppers struggle to determine the fit and look when ordering online, leading to increased returns and decreased purchase confidence. The cost? Lost revenue, operational overhead, and customer frustration. Meanwhile, consumers increasingly expect immersive, interactive shopping experiences that bridge the gap between online and in-store retail. Retailers implementing virtual try-on technology can improve purchase confidence and reduce return rates, translating directly to improved profitability and customer satisfaction. This post demonstrates how to build a virtual try-on and recommendation solution on AWS using Amazon Nova Canvas, Amazon Rekognition and Amazon OpenSearch Serverless. Whether you’re an AWS Partner developing retail solutions or a retailer exploring generative AI transformation, you’ll learn the architecture, implementation approach, and key considerations for deploying this solution. You can find the code base to…

Generating Beautiful UIs May 08, 2026

With contributions from Sherif Cherfa and Halley Chang There’s an intuitive skepticism we have toward AI-generated work. We see it clearly in writing, where the patterns have gotten familiar and punctuation (the em dash — ) has become a universal signal that AI has been used. Design has lagged behind writing, but it’s catching up. Recent models can produce better UIs, yet it still requires heavy hand-holding and prompt “band-aids.” Overall, AI-generated designs often lack that feeling of deep satisfaction, joy, or whimsy that human designers create. Basic prompts produce boring outputs Media theorist Marshall McLuhan is often credited for his beliefs on the co-evolution of humans and tools: “we shape our tools, and thereafter our tools shape us.” Although AI can create superficially “beautiful” designs, they’re often shallow. When you give a model a generic prompt, you get a…

Introducing Multi-LoRA on Cerebras Inference May 06, 2026

Today, we are launching Multi-LoRA—multi-adapter support for Low-Rank Adaptation—on Cerebras Inference in private preview. Multi-LoRA lets teams use many LoRA adapters with a single shared base model, so they can specialize model behavior for different domains, tasks, customers, and workflows. It advances our mission of making Cerebras Inference the fastest and simplest way to run specialized AI applications. LoRAs are lightweight adapters trained to specialize a base model. Instead of fine-tuning all of the base model’s parameters, teams train a much smaller set of adapter weights that can be applied at inference time. This makes specialization practical and cost efficient without requiring a separate full model for each variant. How Multi-LoRA works on Cerebras Inference Cerebras Inference handles the serving infrastructure behind the endpoint. We manage the base model and adapter serving path, so teams can focus on building the…

MoE at Scale: Making Sparse Models Fast on Real Hardware September 03, 2025

In this video we discuss scaling MoE models on modern hardware and address key optimization challenges. If you can’t open the video displayed above, please use this link to open it on YouTube: https://youtu.be/MXo9LEYzwkg Mixture-of-Experts (MoE) models allow you to increase total parameter count without proportional increase in compute, letting you train bigger and better models efficiently (Soboleva, 2025a). You might wonder if extracting theoretical benefits from MoE models requires significant engineering work. After all, your part 3 implementation (Soboleva and Tiwari, 2025) trained perfectly fine on a small acceleration node (and even your laptop). An important point here is that you used only 4 experts and 124M backbone parameters, but production systems like DeepSeek-V3, Qwen3, etc., use hundreds of experts and huge backbones. Try scaling to their sizes with our previous implementation on the GPU, and you will quickly…

MoE Math Demystified: What Does 8x7B Actually Mean? October 14, 2025

This video breaks down MoE inference arithmetic and deployment bottlenecks across different hardware setups. If you can’t open the video displayed above, please use this link to open it on YouTube: https://youtu.be/gHpDBoyCOrE What does 8x7B actually mean? You probably thought it meant 8 experts with 7B active parameters per token. We did too. Turns out it is actually 13B active parameters. But wait, where does 13B come from? This is exactly the kind of confusion this post clears up (skip to the answer). We'll explain what those numbers actually mean for inference by answering how much memory you need, how many GPUs, and what the commonly hit bottlenecks are in production deployment. We'll show that single-GPU deployment is memory-bound, multi-GPU setups are communication-bound, and specialized hardware like Cerebras WSE is compute-bound. Originally, we set out to write a simple post…

Figma - MultiAgents April 16, 2026

Everything is easier now. I have been toying around with agent orchestration for a while now. I’m currently running 10-20 agents around the clock.AI agents are now capable of bringing my ideas to life. Like many developers, I’ve been feeling the token anxiety. I can do much more now than ever before, and every time I have a spare minute I want to kick off another agent session. - I see a cool product I don’t want to pay for? Codex will build it for me. - I have a silly idea I want to see come to life? Codex will build it for me. - I get mildly annoyed doing the same thing over and over? Codex pls. If you have an army of infinitely patient, intelligent, and helpful agents waiting for your next command, why shouldn’t we take…

Lessons learned from building multi-agent workflows April 16, 2026

I pay my upfront subscription ($200/month), write what I hope is the right prompt (prompt AND context engineer), and wait. 35 minutes later, it’s still 'synthesizing', 'perusing', 'effecting', and 'germinating' (who came up with these). By the end, I have files of bad code, a bloated context window, and I’m counting the remaining tokens on my left hand. Okay, I grab an apple, compact, type some heavy handed verbal abuse, re-explain everything from scratch, and pray the next attempt gets further than the last one…. only to be disappointed by the same result. By now, the spark and joys of AI coding are long dead. Stop being a one-shot Sloperator This is the single-agent ceiling. Every developer building with AI agents hits it the moment their project graduates from a 3D HTML snake game to anything more practical. This happens…

Learn more

We’re joining forces with Aleph Alpha to provide the world with an independent, enterprise-grade sovereign alternative in an era of growing AI concentration. This transatlantic alliance would combine Cohere’s global AI scale with Aleph Alpha’s strong research excellence and deep institutional relationships, forging a globally competitive AI champion backed by Canadian and German ecosystems. By pooling top-tier engineering talent and computational resources across two G7 nations, the partnership aims to significantly accelerate the development of next-generation frontier models and systems while providing a secure alternative to dependence on any single vendor or infrastructure stack. The market for AI services is projected to surpass $1 trillion annually, with sovereign AI needs representing nearly $600B of that total (McKinsey, March 2026). The partnership uniquely bridges the gap between these segments with its sovereign-first approach, capturing the critical intersection where sovereignty requirements meet…

Join the new AI Agents Vibe Coding Course from Google and Kaggle

Join the new AI Agents Vibe Coding Course from Google and Kaggle Last November, we launched our first 5-Day AI Agents Intensive Course with Kaggle, reaching over 1.5 million learners. By popular demand, we’re bringing it back from June 15-19, 2026 — now with updated content, new speakers and a hands-on capstone project, all at no cost to registrants. This five-day online course dives deep into building powerful AI agents from foundational concepts to production-ready systems, especially with vibe coding. You’ll explore vibe coding workflows, where natural language becomes the primary programming interface, and learn how to create “10x agents” by integrating tools and APIs. Each day combines conceptual deep dives with hands-on examples. By the end, you’ll be ready to design, build and deploy robust agent systems — culminating in a capstone project that brings your ideas to life.…

Latest Agent LLM Prompting Context Engineering Kacper Łukawski Lead DevRel at Deepset Context Engineering for Agentic Systems: What Goes Into Your Agent's Mind A practical introduction to context engineering - what fills the LLM context window in agentic systems, why it matters, and how to keep it under control. April 20, 2026

Context Engineering for Agentic Systems: What Goes Into Your Agent's Mind A practical introduction to context engineering - what fills the LLM context window in agentic systems, why it matters, and how to keep it under control. April 20, 2026Every new generation of Large Language Models arrives with a bigger context window - and the temptation to use it fully. If the model can read a million tokens, why not feed it everything? In practice, more context doesn’t reliably mean better answers: it often means higher costs, slower responses, and a model that loses track of what actually matters. Context engineering is the discipline of deciding not just what to put in the context window, but how much, in what form, and when to leave things out - and it’s quickly becoming one of the most important skills in building…

How to build scalable web apps with OpenAI's Privacy Filter

How to build scalable web apps with OpenAI's Privacy Filter - Document Privacy Explorer: drop in a PDF or DOCX, read the document back with every PII span highlighted in place. - Image Anonymizer: upload an image, get it back with redacted black bars over names, emails, and account numbers. The image is also editable on a canvas so you can make your own annotations before downloading. - SmartRedact Paste: paste sensitive text, share a public URL that serves the redacted version, keep a private reveal link for yourself. All three are built on gradio.Server, which lets you pair custom HTML/JS frontends with Gradio's queueing, ZeroGPU allocation, and gradio_client SDK. In all these apps, gradio.Server plays the same backend role, and that consistency is exactly what makes it really powerful. The model Privacy Filter is a 1.5B-parameter model with 50M…

How to Use Transformers.js in a Chrome Extension

How to Use Transformers.js in a Chrome Extension While building it, we ran into several practical observations about Manifest V3 runtimes, model loading, and messaging that are worth sharing. Who this is for This guide is for developers who want to run local AI features in a Chrome extension with Transformers.js under Manifest V3 constraints. By the end, you will have the same architecture used in this project: a background service worker that hosts models, a side panel chat UI, and a content script for page-level actions. What we will build In this guide, we will recreate the core architecture of Transformers.js Gemma 4 Browser Assistant, using the published extension as a reference and the open-source codebase as the implementation map. - Live extension: Chrome Web Store - Source code: github.com/nico-martin/gemma4-browser-extension - End result: a background-hosted Transformers.js engine, a side…

Gemma 4 VLA Demo on Jetson Orin Nano Super

Gemma 4 VLA Demo on Jetson Orin Nano Super You speak → Parakeet STT → Gemma 4 → [Webcam if needed] → Kokoro TTS → Speaker Press SPACE to record, SPACE again to stop. This is a simple VLA: the model decides on its own whether to act based on the context of what you asked, no keyword triggers, no hardcoded logic. If your question needs Gemma to open her eyes, she'll decide to take a photo, interpret it, and answer you with that context in mind. She's not describing the picture, she's answering your actual question using what she saw. And honestly? It's pretty impressive that this runs on a Jetson Orin Nano. :) Get the code The full script for this tutorial lives on GitHub, in my Google_Gemma repo next to the Gemma 2 demos: 👉 github.com/asierarranz/Google_Gemma Grab…

The PR you would have opened yourself

The PR you would have opened yourself TL;DR We provide a Skill and a test harness to help port language models from transformers to mlx-lm, so they become (almost) instantly available the moment they are added to transformers. The Skill is designed to support contributors and reviewers as an aide, not an automation. We explain why we did it, how, and comment about how to meaningfully contribute to open source in the age of agents. The advent of code agents In 2026, code agents started to actually work. What used to be auto-completion at the side of your editor turned into a system that one-shots reasonable solutions from brief specifications. The generated code usually works out of the box, covers what you asked for, and makes reasonable assumptions about details you didn't specify. This is great. As Jensen Huang puts…

Import AI 455: AI systems are about to start building themselves.

Import AI 455: AI systems are about to start building themselves. The first step towards recursive self improvement Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv and feedback from readers. If you’d like to support this, please subscribe. AI systems are about to start building themselves. What does that mean? I’m writing this post because when I look at all the publicly available information I reluctantly come to the view that there’s a likely chance (60%+) that no-human-involved AI R&D - an AI system powerful enough that it could plausibly autonomously build its own successor - happens by the end of 2028. This is a big deal. I don’t know how to wrap my head around it. It’s a reluctant view because the implications are so large that I feel dwarfed by them, and…

GridSFM: A new, small foundation model for the electric grid

Microsoft releases a lightweight foundation model that can predict AC optimal power flow in milliseconds, boosting efficiency and unlocking cost savings in grid analysis. At a glance - Microsoft introduces GridSFM, a small foundation model that approximates AC optimal power flow in milliseconds, unlocking decisions that can directly impact up to $20B/year in congestion losses and 3.4 TWh of renewable curtailment. - Beyond estimating generator dispatch and costs, GridSFM produces full AC system states, giving operators direct visibility into congestion, stability, and overall system health. - It provides a foundation for the community to build advanced power grid simulators and planning tools without recreating data or models from scratch. Microsoft introduces GridSFM, a small foundation model for solving AC optimal power flow (AC-OPF) problems in transmission power grids. This follows our earlier release of a U.S.-based open transmission-topology dataset that…

World Models: 10 Things That Matter in AI Right Now

World Models: 10 Things That Matter in AI Right Now Join a subscriber-only discussion live on Thursday, May 21. World models recently made our list of 10 Things That Matter in AI Right Now. Watch executive editor Niall Firth explain why this emerging area of AI is gaining so much attention. Join MIT Technology Review editors and reporters for a subscriber-only Roundtables discussion, "Can AI Learn to Understand the World?" exploring how AI may evolve to better reason about the real world and what this could mean for the future of AI systems. Related Stories: - How Pokémon Go is giving delivery robots an inch-perfect view of the world - 10 Things That Matter in AI Right Now: World Models - Yann LeCun has a bold new vision for the future of AI Speakers: Keep Reading Most Popular OpenAI is…

Advanced RAG: Data Cleaning and Retrieval Techniques

Retrieval-augmented generation (RAG) makes queries smarter, arming them with proprietary data and contextualized knowledge. But even the best RAG methods produce inaccurate answers, and context windows polluted by noisy data. Advanced RAG emerged to fix that. RAG isn’t a single method — there are several ways to boost the accuracy and reliability of LLM outputs with this framework. This guide covers the advanced LLM RAG techniques teams use in production. Why does basic RAG fall short? Basic RAG is sometimes called Naive RAG because of its simple nature. It indexes a set of documents via a single dense vector, and then embeds them, retrieves the top-K matches, and passes them to an LLM. Simple RAG in LLM systems works well in some scenarios, but they often struggle in real-world use. Here are a few common limitations: - Poor recall: It…

Building Better Agents: LLM Memory Types and Trade-Offs

Engineers often treat large language model (LLM) memory as a simple feature toggle. But in a production environment, memory acts as an agent’s central nervous system, determining whether a system feels like a coherent assistant or a fragmented script. In practice, LLM memory is a high-stakes design challenge. To build resilient agents, you must move beyond basic chat history and navigate a complex decision surface where every choice impacts scalability and reliability and implement best prompt engineering techniques. In this guide, we’ll analyze the trade-offs of architecting persistent memory into your AI systems, examining how to choose the right memory types, implementation layers, and consistency for production-grade performance. What’s LLM memory? An LLM with memory is a stateful system that integrates static training with real-time execution. To understand how LLM memory works, you have to distinguish between parametric knowledge —…

Production AI Playbook: Evaluation and Monitoring

This post is part of a series that explores proven strategies and practical examples for building reliable AI systems. New to n8n? Start with the introduction. Find out when new topics are added to the Production AI Playbook via RSS, LinkedIn or X. The Silent Drift Problem Your AI workflow passed every test. Classifications were accurate. Responses were on-point. You shipped it, and for two weeks, everything looked great. Then support tickets started trickling in. Customers were getting responses that missed the point. Classifications were landing in the wrong buckets. Nothing broke. No errors in the logs. The AI just quietly got worse. This is silent drift, and it's one of the most common failure modes in production AI systems. Unlike traditional software, where a bug either crashes or doesn't, AI outputs degrade gradually. A model update changes behavior slightly.…

ReAct Agent: Architecture, Implementation, and Tradeoffs

Some tasks can't be solved in a single LLM call. When a question requires looking up data, processing it, and making a decision based on the result, a one-shot response will either hallucinate the answer or give a shallow one. ReAct agents solve this with an iterative reasoning loop. Instead of trying to answer everything at once, the agent breaks the problem down step by step: think about what's needed, call a tool, observe the result, and decide what to do next. Each cycle grounds the model's reasoning in real data before moving forward. This Reasoning + Acting pattern turns opaque agent behavior into something you can follow, debug, and audit - every thought and action is visible in the execution trace. Here's how the ReAct pattern works, when to use it over other agent approaches, and how to build…

LLM Tool Calling: How It Works and How To Implement It

Large language models (LLMs) are brilliant reasoners. But without a way to interact with the world, they’re essentially locked behind a glass wall— they have enough knowledge to explain a refund policy in perfect detail but lack the hands to actually trigger one. For developers, this disconnect between reasoning and action is what separates sophisticated chatbots from production-grade agents. LLM tool calling offers an escape from the training-data silo, allowing models to move from passive text generation to active system participation. But the real engineering challenge isn’t just getting the model to output a valid JSON or a tool call — it’s building the orchestration, security, and observability required to ensure those calls don’t fail in a production environment. Here’s a rundown of what LLM tool calling is and how it works at scale. What LLM tool calling means? LLM…

Human-in-the-Loop vs. Human-on-the-Loop: When To Use Each System

There are three main ways people control the quality of AI systems: human-in-the-loop (HITL), human-on-the-loop (HOTL), and hybrid systems using both. These frameworks determine how systems make decisions and where humans intervene. Each approach affects scalability, risk tolerance, and operational expenses. This oversight spectrum gives you a wide range of potential workflows depending on the task, whether your team needs tight human-driven control or occasional check-ins. In this guide, learn the difference between human-in-the-loop versus human-on-the-loop. Plus, discover when to use each approach and how to implement it in your work. What’s human-in-the-Loop (HITL)? HITL is a process where AI performs tasks but humans control final decisions, preventing the system from executing certain actions without approval. This is a synchronous control pattern. The workflow stops at a decision gate until a human provides a required signal. For example, AI processes…

How to evaluate the performance of AI agents?

Traditional software testing is straightforward: you give input X and expect output Y. If the function returns the wrong value, the test fails. LLM-based agents don't work that way. They're non-deterministic which means the same prompt can produce different outputs across runs. They operate over multiple steps, making decisions about which tools to call, what parameters to pass, and how to interpret results. An agent can complete an execution without errors and still hallucinate facts, miss the user's intent, or take unnecessary steps. Classical testing may not catch problematic outputs produced by an AI Agent. When building AI Agents, you face three main evaluation challenges: - You're evaluating trajectories, instead of just outputs. An agent might give the correct final answer but call the wrong tools, use the wrong parameters, or take five steps when one would do. If you…

How to Eliminate Pipeline Friction in AI Model Serving

The path from a trained AI model to production should be smooth, but rarely is. Many teams invest weeks fine-tuning models, only to discover that exporting to a deployment format breaks layers, input shapes cause runtime failures, or version mismatches silently degrade performance. These issues are collectively known as pipeline friction, and they cost organizations time, money, and competitive advantage. This post provides actionable best practices for eliminating the most common sources of friction in AI model serving pipelines. The results are concrete: APIs respond faster under real traffic. Each GPU carries more requests. Scaling up for peak hours is a smooth, low-stress effort. Cost per inference drops. And the deployments themselves stop being the part of every release that breaks. What is pipeline friction in AI model serving? Pipeline friction refers to any obstacle that slows or disrupts the…

Improving Bash Generation in Small Language Models with Grammar-Constrained Decoding

Bash is one of the most flexible and powerful interfaces exposed to AI agents. In the right system, a model that emits grep , curl , tar , or a shell pipeline is producing an executable action that can read files, mutate a workspace, open network connections, and chain tools together. For the NVIDIA AI Red Team, this makes command generation a useful research target. If smaller language models can be guided into valid, policy-aware command structures, they become more reliable components for agentic workflows that can be deployed into a wider range of environments. Constrained decoding is a technique that modifies the sampling process in autoregressive language model generation. At each generation step, the model produces logits as normal, but before a token is selected, a grammar is applied to change the distribution (often by effectively blocking certain tokens).…

How to Build In-Vehicle AI Agents with NVIDIA: From Cloud to Car

The automotive cockpit is undergoing a fundamental shift from rule-based interfaces to agentic, multimodal AI systems capable of reasoning, planning, and acting. In most vehicles on the road today, in-vehicle assistants still rely on fixed command-response patterns: interpret a phrase, trigger an action, reset. While effective for well-defined tasks, this approach doesn’t scale to modern expectations, where drivers and passengers want conversational assistants that can handle ambiguity, manage multi-step tasks, and adapt to context that evolves throughout the journey. Large language models (LLMs), vision-language models (VLMs), and speech models enable a fundamentally new interaction paradigm. Rather than relying on command matching, these models support conversational AI with memory and reasoning, multimodal interaction across voice, vision, and telemetry, and context-aware, proactive assistance that anticipates user needs instead of simply reacting to requests. The range of experiences this unlocks is significant. Intelligent…

How to Build, Run, and Scale High-Quality Creator Workflows in ComfyUI

Creative and visualization teams today produce more assets, in more formats, with leaner teams. Generative AI can accelerate that work – compressing tasks that once took hours of manual effort into automated, repeatable pipelines. ComfyUI is an open-source, node-based creative tool that runs locally on NVIDIA RTX GPUs. It connects image generation, video synthesis, and language models into pipelines that teams can customize and extend — without cloud dependencies or data leaving the client. This guide walks through three production-ready workflows from the NVIDIA GenAI Creator Toolkit, adapted from NVIDIA’s GTC 2026 DLI course Create Generative AI Workflows for Design and Visualization in ComfyUI. Each workflow is standalone and runs locally on NVIDIA RTX. What you’ll accomplish By the end of this guide you will have: - Deconstructed an image into separate layers—foreground, midground, and background, each with a clean…

Build AI-Powered Games with NVIDIA DLSS 4.5, RTX, and Unreal Engine 5

Today, game developers can begin integrating NVIDIA DLSS 4.5 with Dynamic Multi Frame Generation, Multi Frame Generation 6X, and the second-generation transformer model for NVIDIA Super Resolution. In this post, we’ll go over new technologies and resources to share with our game-developer community, including: - A new NVIDIA TensorRT for RTX plugin for Unreal Engine’s Neural Network Engine (NNE) - NVIDIA Kimodo for easier motion generation - A guide to using ComfyUI to help produce pre-production assets - More than a dozen new sessions from GDC and GTC now available on YouTube - Our April “Level Up with NVIDIA” webinar, highlighting path-traced hair in Unreal Engine 5.7 Integrate DLSS 4.5 Dynamic Multi Frame Generation At CES 2026, we introduced DLSS 4.5, extending its AI-driven rendering pipeline with a second-generation transformer model for Super Resolution to deliver another major upgrade to…

Build with DeepSeek V4 Using NVIDIA Blackwell and GPU-Accelerated Endpoints

DeepSeek just launched its fourth generation of flagship models with DeepSeek-V4-Pro and DeepSeek-V4-Flash, both targeted at enabling highly efficient million-token context inference. DeepSeek-V4-Pro is the largest model in the family, with 1.6T total parameters and 49B active parameters. DeepSeek-V4-Flash is a smaller 284B-parameter model with 13B active parameters, designed for higher-speed, higher-efficiency workloads. Both models support up to a 1M-token context window, opening new possibilities for long-context coding, document analysis, retrieval, and agentic AI workflows. Architectural innovations for long-context inference The V4 family builds on the DeepSeek MoE architecture, with an increased focus on optimizing the attention component of the transformer architecture. These innovations are designed to achieve a 73% reduction in per-token inference FLOPs and a 90% reduction in KV cache memory burden compared with DeepSeek-V3.2. That matters because long context is becoming a core requirement for agentic applications.…

Simplify Sparse Deep Learning with Universal Sparse Tensor in nvmath-python

In a previous post, we introduced the Universal Sparse Tensor (UST), enabling developers to decouple a tensor’s sparsity from its memory layout for greater flexibility and performance. We’re excited to announce the integration of the UST into nvmath-python v0.9.0 to accelerate sparse scientific and deep learning applications. This post provides a walkthrough of key UST features, implementation details, and performance overview, including: - Zero-cost interoperability: Data-movement-free conversion with PyTorch, SciPy, and CuPy. - Custom formats: Define novel sparsity schemes. - Polymorphic operations: Sparsity-agnostic functions automatically use optimized kernels or generate custom sparse code—eliminating the need for manual coding of new formats. - PyTorch injection: Easily inject UST performance benefits into existing PyTorch models. - Transparent caching: Avoid JIT/LTO recompilation and replanning—amortizing overhead over subsequent repeated execution of the same operation. Tensor format DSL The UST describes common (e.g., COO, CSR,…

How to Build Vision AI Pipelines Using NVIDIA DeepStream Coding Agents

Developing real-time vision AI applications presents a significant challenge for developers, often demanding intricate data pipelines, countless lines of code, and lengthy development cycles. NVIDIA DeepStream 9 removes these development barriers using coding agents, such as Claude Code or Cursor, to help you easily create deployable, optimized code that brings your vision AI applications to life faster. This new approach simplifies the process of building complex multi-camera pipelines that ingest, process, and analyze massive volumes of real-time video, audio, and sensor data. Built on GStreamer and part of the NVIDIA Metropolis vision AI development platform, DeepStream accelerates a developer’s journey from concept to actionable insight across industries. Video 1. How to use the NVIDIA DeepStream coding agents to generate complete vision AI pipelines from natural language prompts with Claude Code. To watch a recording showing how to build a DeepStream…

Building a safe, effective sandbox to enable Codex on Windows

Building a safe, effective sandbox to enable Codex on Windows By David Wiesen, Member of Technical Staff When I joined the Codex engineering team in September 2025, Codex for Windows didn’t have a sandbox implementation meaning that Windows users were forced to choose between two subpar options when using OpenAI's coding agents: - Approving nearly every command (even reads) that a coding agent wanted to run, which is inefficient and pesky. A major benefit of using Codex is that you don’t have to do all the tedious work yourself. - Enabling Full Access mode: letting Codex run all commands without approval or restrictions, which removes friction at the expense of oversight. Codex, our coding agent, runs on developer laptops—whether that's through the CLI, the IDE extension, or the desktop app. It manages a conversation between a human at a keyboard…

AutoScout24 scales engineering with AI-powered workflows

AutoScout24 scales engineering with AI-powered workflows Codex and ChatGPT accelerate development cycles, improve code quality, and expand AI adoption across 2,000 employees. Results ~10x Faster development cycles (weeks → days). Results ~2,000 Employees enabled with AI tools. Results ~1,000 Builder roles using Codex. AutoScout24 Group(opens in a new window) is the largest pan-European and Canadian online car marketplace, connecting more than 30 million monthly users with over two million vehicle listings. Operating across multiple brands—including AutoScout24 in Europe and AutoTrader.ca in Canada—the company supports a network of 45,000 dealer partners and employs around 2,000 people globally. As product expectations increased and system complexity grew, AutoScout24 Group faced mounting pressure to deliver faster innovation without compromising reliability. This is closely tied to the company’s goal of continuously improving how buyers search, evaluate, and purchase vehicles, and how dealers successfully market and…

How NVIDIA engineers and researchers build with Codex

How NVIDIA engineers and researchers build with Codex Teams use Codex with GPT‑5.5 to ship production systems and turn research ideas into runnable experiments. Results 10x Speed improvement in end-to-end research workflows Results 40k NVIDIANs with access to Codex At NVIDIA, engineers are using Codex as their default tool for complex engineering work, and to run end-to-end machine learning experiments. Codex, built on GPT‑5.5 and running in production on NVIDIA GB200 and GB300 infrastructure, can handle much longer, more autonomous sessions — going beyond execution to surface issues and ideas that weren't part of the original prompt. “Codex is our go-to tool for complex engineering tasks, and with GPT-5.5, it surfaces bugs and gaps in my program that other models weren’t able to find.” NVIDIA’s coding agents team helps engineers across the company adopt and use AI tools effectively in…

How finance teams use Codex

How finance teams use Codex See how finance teams can use Codex to build review-ready assets for monthly business reviews, reporting, variance analysis, and planning. With Codex, finance teams can just build things. Start with the close workbooks, revenue and expense dashboards, forecast updates, prior MBRs, and owner notes you already use. Codex can help turn that context into tangible assets your team can review, refine, and share, no coding required. Use it to spend less time assembling the first pass and more time shaping the story, checking the numbers, and preparing for the decisions ahead. Learn more about using Codex for everyday work in our on-demand webinar(opens in a new window). Ready to try Codex with real finance work? Start with a copy-ready prompt, then use the fully built example to see how that same prompt gets stronger with…

How ChatGPT learns about the world while protecting privacy

How ChatGPT learns about the world while protecting privacy A plain-language guide to model training, privacy safeguards, and the privacy choices available in ChatGPT. Editor's note for Canada: The French text follows the English text (Le texte français suit le texte anglais). ChatGPT is becoming more capable across domains, helping people with complex, real-world work like coding, research, analysis, and multi-step tasks across tools. Those gains in capability are driven by training on a wide variety of data to help our models build broad knowledge of the world and apply it to new tasks. As OpenAI continues to develop frontier models, we work hard to help ensure that our model training process respects privacy. We’ve developed state of the art technologies to help our models learn useful general patterns rather than private information about individuals, and we have a number…

Our commitment to community safety

Our commitment to community safety Mass shootings, threats against public officials, bombing attempts, and attacks on communities and individuals are an unacceptable and grave reality in today’s world. These incidents are a reminder of how real the threat of violence is—and how quickly violent intent can move from words to action. People may also bring these moments and feelings into ChatGPT. They may ask questions about the news, try to understand what happened, express fear or anger, or talk about violence in ways that are fictional, historical, political, personal, or potentially dangerous. We work to train ChatGPT to recognize the difference—and to draw lines when a conversation starts to move toward threats, potential harm to others, or real-world planning. We’re sharing what we do to minimize uses of our services in furtherance of violence or other harm: how our models…

An open-source spec for orchestration: Symphony

An open-source spec for Codex orchestration: Symphony By Alex Kotliarskyi, Victor Zhu, and Zach Brock Six months ago, while working on an internal productivity tool, our team made a controversial (at the time) decision: we’d build our repo with no human-written code. Every line in our project repository had to be generated by Codex. To make that work, we redesigned our engineering workflow from the ground up. We built an agent-friendly repository, invested heavily in automated tests and guardrails, and treated Codex as a full-fledged teammate. We documented that journey in our previous blog post on harness engineering. And it worked, but then we ran into the next bottleneck: context switching. To solve this new problem, we built a system called Symphony. Symphony(opens in a new window) is an agent orchestrator that turns a project-management board like Linear into a…

Our principles

AI has the potential to significantly improve many aspects of society. This technology, like others before, will give people more capability and agency; what people will be able to do with AI will dwarf what people could do with steam engines or electricity. We envision a world with widespread flourishing at a level that is currently difficult to imagine, and a world in which individual potential, agency, and fulfillment significantly increase. A lot of the things we’ve only let ourselves dream about in sci-fi could become reality, and most people could live more meaningful lives than most are able to today. But this outcome is not guaranteed. Power in the future can either be held by a small handful of companies using and controlling superintelligence, or it can be held in a decentralized way by people. We believe the latter…

How to get started with Codex

How to get started with Codex Tips to set up Codex, create your first project, and start completing real tasks. Start by downloading the Codex desktop app and signing in with your ChatGPT account. Once you open Codex, create your first thread. A thread is like a chat in ChatGPT: a space where you go back and forth with Codex to accomplish a task. You can create a standalone thread, but most of the time you’ll want to work inside a project. A project is connected to a folder on your computer: Tip: To keep things simple, create a folder on your computer named Codex. Inside that Codex folder, you can have a separate folder for each project. If you want Codex to work with specific files for a project, just drag them into the folder. If not, you can…

What is Codex?

What is Codex? Understand what Codex is and how it fits into your work Codex is an AI agent that you can delegate real work to. ChatGPT is great for asking questions, brainstorming, and drafting in conversation. Codex is designed for a different kind of task—it can work across files, tools, and repeatable workflows to help move work forward. A simple way to think about it: ChatGPT helps you think through the work, while Codex helps you hand off parts of the work itself. You don’t need to be a developer or working on software to use Codex. It goes beyond coding and is especially useful for tasks that require more than a single answer—like gathering information from multiple sources, creating and updating files, or producing outputs such as documents, slides, and spreadsheets. Codex can connect to tools, take action,…

Codex settings

Codex settings Make Codex work the way you want, with fewer interruptions. You can access settings from the menu in the bottom left corner of Codex. For your first few tasks, focus on a few key settings: personalization, prevent sleep, detail level, and appearance. General > Prevent sleep while running keeps your computer awake while Codex is running. This is useful for longer tasks. If your computer goes to sleep, Codex may stop working. General > Detail level controls how much information Codex shows while it is working. Coding mode shows the specific commands Codex is executing. If this is more information than you need, switch to Default to keep your conversation cleaner. Personalization works a lot like personalization in ChatGPT. You can decide whether you want Codex to speak to you in a friendly tone or a direct tone.…

Working with Codex

Working with Codex Learn how to set up your Codex workspace and start working with threads and projects. When you open Codex, you’ll see a few core elements: a sidebar menu, projects, settings, and a chat window. You don’t need to understand everything right away, but we’ll cover the basics here. The sidebar is where you navigate between threads, projects, and tools. Most of your work will begin by creating a new thread. When you’re using Codex, think of a “thread” the same way you would think of a “chat” in ChatGPT. You can have a thread which stands on its own, or a thread which is nested within a project. Select New thread to begin. You can select an existing project to associate it with, create a new project, or leave it as a standalone conversation. Search to find…

Plugins and skills

Plugins and skills Plugins and skills help Codex do more specific kinds of work. Plugins help Codex connect to other tools and sources of information. For example, a plugin might help Codex reference files in Google Drive, scan your email inbox, or work with information from another tool you use. Plugins can be simple and useful right away. If you already have the information you need in a connected plugin, you can ask Codex to use it instead of copying and pasting everything into the thread. To access plugins, select plugins in the top left corner of Codex. From there, you can see plugins that are recommended or already installed, browse the plugins library, or create a new plugin. Creating a new plugin usually requires more technical expertise than creating a skill. A skill is like a playbook Codex can…

Automations

Automations Run recurring tasks automatically using schedules and triggers in Codex. Codex can automatically run tasks on a schedule. This makes Codex proactive. Instead of waiting for you to come back and ask for an update, Codex can return at the scheduled time, do the work, and surface the result for you to review. This is useful for recurring work, like preparing for the day, reviewing what changed, checking for updates, summarizing recent activity, or creating a weekly report. For example, you might use a thread automation to: - Write a weekly review every Friday - Create a morning brief from yesterday’s work - Summarize new files added to a folder - Clean up a weekly data export - Check for missing or inconsistent information - Create a recurring project status update Some automations can also return to the same…

Workspace agents

Workspace agents Understand, build, and use agents for repeatable work in ChatGPT. Most ChatGPT users already know how to use AI for one-off tasks—like drafting, summarizing, brainstorming, or answering questions. The next phase of AI use is broader and more embedded in day-to-day work. Instead of helping with isolated moments, AI is increasingly being used to support repeatable workflows that depend on shared systems, standard handoffs, consistent outputs, and real-world constraints like timing, accuracy, and process. That’s where workspace agents in ChatGPT fit. They’re designed to be used for repeatable workflows—work you’d otherwise do manually, re-explaining the steps each time, and copying information between tools. Learn more about workspace agents in our blog post. If you’re new to agent building, let’s focus on the core concepts first so when you start building, you’ll know how to set up your workspace…

Writing with ChatGPT

Writing with ChatGPT Draft, revise, and refine written work with clarity and intent. ChatGPT can support many common workplace writing tasks: drafting from scratch, rewriting and tightening, adjusting tone for a specific audience, and turning rough notes into clear communication. It’s especially useful when you’re short on time, staring at a blank page, or trying to land the right level of polish. Tip: ChatGPT can work with uploaded files, or access files via connected apps. Learn more here. Most workplace writing has the same goal: help someone understand something quickly and know what to do next. ChatGPT can speed up the parts that often take the most time—finding a strong opener, organizing ideas, and refining wording—so you can focus on the decisions and details that matter. It is also effective for adapting tone across audiences. You can take the same…

Responsible and safe use of AI

Responsible and safe use of AI Learn best practices for using ChatGPT safely and effectively. AI is a transformative new technology that is reshaping knowledge work. The large language models (LLMs) that power ChatGPT are trained on vast amounts of publicly available text and other data to predict and generate human-like language. This enables them to assist with tasks such as drafting, summarizing, brainstorming, and answering questions, helping people work more efficiently and creatively. As this technology continues to evolve, it is important to use AI responsibly. These models may sometimes produce incorrect information or be misused if their outputs are applied without care. OpenAI’s mission is to ensure that artificial general intelligence (AGI) benefits all of humanity, and achieving this goal requires safe and thoughtful use by everyone. The tips on this page are designed to help anyone using…

Using projects in ChatGPT

Using projects in ChatGPT Organize your work into dedicated spaces with shared context and history. Projects in ChatGPT are dedicated spaces for a specific body of work or area of focus. A project can hold chats, files, instructions, and related context in one place, so you do not need to restate the same background every time you start a new conversation. Projects are especially useful for work that continues over time. Instead of spreading materials across separate chats, you can keep everything together in one place and return to the same context when needed. On some plans, you can also invite other people to collaborate within a project. - Open Projects from the left-hand menu. - Create a new project and give it a name. - You can now add files, set project instructions, or move existing chats into the…

Research with ChatGPT

Research with ChatGPT Use search and deep research to find, analyze, and synthesize information from across the web. ChatGPT can be a helpful research partner because it quickly brings together information from many sources, making it easier to explore ideas, spot patterns, and understand complex topics. By reasoning through context, citing sources, and producing clear, structured summaries, it helps turn open questions into well-defined insights. There are two different ways to search the public internet with ChatGPT—search and deep research. Below is an explanation of both, and when to use each. ChatGPT search allows ChatGPT to pull in the latest information from the internet directly into your conversations. This means you can go beyond ChatGPT’s built-in training knowledge and get up-to-date answers on things like current events, market trends, competitor activity, or niche details not included in its training data.…

ChatGPT for customer success teams

ChatGPT for customer success teams Manage accounts, improve communication, and drive better customer outcomes. Customer success work blends relationship management with operational follow-through—onboarding, adoption, troubleshooting, renewals, and cross-functional coordination. The challenge is often the overhead including pulling context from calls and tickets, turning notes into plans, writing clear follow-ups, and keeping everyone aligned on next steps. ChatGPT helps reduce that overhead by turning scattered inputs into clear, structured outputs so teams can focus more on customers and less on coordination. - Turns scattered customer context into a clear plan. CSMs often have the information—they just don’t have it in one place. ChatGPT can synthesize notes, emails, and product signals into a simple view of goals, current state, risks, and a concrete action plan you can share internally and with the customer. - Makes customer communication clearer and easier to act…

Prompting fundamentals

Prompting fundamentals Learn how to write clear prompts to get better, more useful responses. Prompt engineering is the process of designing and refining your input in a way that helps ChatGPT give the best possible answer. It’s about figuring out how to ask so you get the result you want—whether that’s a clear summary, comprehensive report, or detailed analysis. ChatGPT works best when you give it clear instructions. There’s no single “perfect” way to write a prompt. Think of it as a conversation with a colleague, where you might need to adjust your phrasing or tone to help them understand what you need. Experimentation and iteration are the best ways to discover how AI can be most useful to you. Be clear about what you need ChatGPT to do. Outline what you want, who it’s for, and why it matters.…

How to make remarkable videos with Seedance 2.0

How to make remarkable videos with Seedance 2.0 Run Seedance 2.0 AI video used to be utterly bad. (We’ve all seen Will Smith eat spaghetti more times than we can count, so I’ll spare you.) Last year, however, we really began to see AI video take off with front-runners like Google’s Veo 3 series and Kling from Kuaishou. With each new model release, we inched toward improvements with prompt adherence, audio integration, and solving the “AI look.” Seedance 2.0 is the largest step change we’ve seen in months. You can make movies with this thing. A catastrophic collision between two massive space stations in low Earth orbit. Metal shears apart in slow motion as the stations grind into each other, sending a hailstorm of debris spiraling outward. Entire modules crumple like tin cans. Pressurized compartments blow out in violent bursts…

GPT-5.5 prompting guide

25th April 2026 - Link Blog GPT-5.5 prompting guide. Now that GPT-5.5 is available in the API, OpenAI have released a wealth of useful tips on how best to prompt the new model. Here's a neat trick they recommend for applications that might spend considerable time thinking before returning a user-visible response: Before any tool calls for a multi-step task, send a short user-visible update that acknowledges the request and states the first step. Keep it to one or two sentences. I've already noticed their Codex app doing this, and it does make longer running tasks feel less like the model has crashed. OpenAI suggest running the following in Codex to upgrade your existing code using advice embedded in their openai-docs skill: $openai-docs migrate this project to gpt-5.5 The upgrade guide the coding agent will follow is this one, which…

It's a big one

24th April 2026 This week's edition of my email newsletter (aka content from this blog delivered to your inbox) features 4 pelicans riding bicycles, 1 possum on an e-scooter, up to 5 raccoons with ham radios hiding in crowds, 5 blog posts, 8 links, 3 quotes and a new chapter of my Agentic Engineering Patterns guide. Recent articles - DeepSeek V4 - almost on the frontier, a fraction of the price - 24th April 2026 - Extract PDF text in your browser with LiteParse for the web - 23rd April 2026 - A pelican for GPT-5.5 via the semi-official Codex backdoor API - 23rd April 2026

Millisecond Converter

24th April 2026 LLM reports prompt durations in milliseconds and I got fed up of having to think about how to convert those to seconds and minutes. Recent articles - DeepSeek V4 - almost on the frontier, a fraction of the price - 24th April 2026 - Extract PDF text in your browser with LiteParse for the web - 23rd April 2026 - A pelican for GPT-5.5 via the semi-official Codex backdoor API - 23rd April 2026

Quoting Maggie Appleton

23rd April 2026 [...] if you ever needed another reason to learn in public by digital gardening or podcasting or streaming or whathaveyou, add on that people will assume you’re more competent than you are. This will get you invites to very cool exclusive events filled with high-achieving, interesting people, even though you have no right to be there. A+ side benefit. — Maggie Appleton, Gathering Structures (via) Recent articles - DeepSeek V4 - almost on the frontier, a fraction of the price - 24th April 2026 - Extract PDF text in your browser with LiteParse for the web - 23rd April 2026 - A pelican for GPT-5.5 via the semi-official Codex backdoor API - 23rd April 2026

Join us at PyCon US 2026 in Long Beach - we have new AI and security tracks this year

Join us at PyCon US 2026 in Long Beach—we have new AI and security tracks this year 17th April 2026 This year’s PyCon US is coming up next month from May 13th to May 19th, with the core conference talks from Friday 15th to Sunday 17th and tutorial and sprint days either side. It’s in Long Beach, California this year, the first time PyCon US has come to the West Coast since Portland, Oregon in 2017 and the first time in California since Santa Clara in 2013. If you’re based in California this is a great opportunity to catch up with the Python community, meet a whole lot of interesting people and learn a ton of interesting things. In addition to regular PyCon programming we have two new dedicated tracks at the conference this year: an AI track on Friday…

# kernel-fusion ( 1 )

vLLM Tops the Artificial Analysis LeaderboardMay 11, 2026·15 min readHow vLLM built the leading deployments of DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B.

# benchmarking ( 1 )

vLLM Tops the Artificial Analysis LeaderboardMay 11, 2026·15 min readHow vLLM built the leading deployments of DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B.

# turboquant ( 1 )

A First Comprehensive Study of TurboQuant: Accuracy and Performance ·12 min read TurboQuant, a method for KV-cache quantization, recently gained significant traction in the community due to the large advertised savings in GPU memory from very low bit-width quantization of a...

# agentic ( 1 )

Serving Agentic Workloads at Scale with vLLM x Mooncake ·10 min read TL;DR: Agentic workloads generate massive shared prefixes that are often recomputed across turns. By integrating Mooncake's distributed KV cache store into vLLM, we achieve 3.8x higher throughput,...

DeepSeek V4 in vLLM: Efficient Long-context Attention Apr 24, 2026 · 17 min read A first-principles walkthrough of DeepSeek V4's long-context attention, and how we implemented it in vLLM.

DeepSeek V4 in vLLM: Efficient Long-context Attention We are excited to announce that vLLM now supports the DeepSeek V4 family of models (deepseek-ai/DeepSeek-V4-Pro and deepseek-ai/DeepSeek-V4-Flash ). These models feature an efficient long-context attention mechanism, purpose-built for tasks involving up to one million tokens. While the new attention design may appear intricate on first reading, its underlying principles are straightforward once examined systematically. This blog post is organized into three sections: - Quickstart guide for serving DeepSeek V4 on vLLM - First-principles explanation of DeepSeek V4's new architectural design - Overview of our implementation approach and optimization challenges for this model on vLLM: hybrid KV cache, kernel fusion, and disaggregated serving. This represents our initial release of model support, and further optimizations are actively underway. We hope the technical explanation that follows can help the open-source community understand both the attention…

# fp8 ( 1 )

The State of FP8 KV-Cache and Attention Quantization in vLLM ·21 min read Long-context LLM serving is increasingly memory-bound: for standard full-attention decoders, the KV cache often dominates GPU memory at 128k+ contexts, and each decode step must read a large...

# kv_cache ( 1 )

The State of FP8 KV-Cache and Attention Quantization in vLLM ·21 min read Long-context LLM serving is increasingly memory-bound: for standard full-attention decoders, the KV cache often dominates GPU memory at 128k+ contexts, and each decode step must read a large...

# mamba ( 1 )

Disaggregated Serving for Hybrid SSM Models in vLLM ·15 min read Hybrid architectures that interleave Mamba-style SSM layers with standard full-attention (FA) layers — such as NVIDIA Nemotron-H — are gaining traction as a way to combine the linear-time...

How to Disable Google's Gemini in Chrome

If you use Google's Chrome browser for desktop, there's probably a Gemini Nano AI model running on your computer right now and taking up about 4 GB of space. That's not necessarily a bad thing, but if you didn't know about it and don't want it, there's a way to turn it off. The file started auto-downloading for Chrome users in 2024 after Google built Gemini Nano into the browser. But a report by That Privacy Guy this week and the ensuing reception it received highlighted how unaware many users were—perhaps a result of a flood of AI services and features across the tech industry that have been difficult for users to keep up with. To uninstall the Gemini Nano file, open Chrome on your computer, in the top right corner click the “More” menu represented by three vertical dots,…

These Men Allegedly Profit Off Teaching People How to Make AI Porn

A little over a year ago, MG was leading the relatively normal life of a twentysomething in Scottsdale, Arizona. She worked as a personal assistant and supplemented her income by waiting tables on the weekends. Like most women her age, she had an Instagram account, where she’d occasionally post Stories and photos of herself getting matcha and hanging out by the pool with her friends, or going to Pilates. “I never really cared to pop off and become popular on social media,” says MG (who is cited only as MG in the lawsuit to protect her identity). “I just used it the way most people did when it first came out, to share their lives with the people closest to them.” She has a little more than 9,000 followers—a robust following, but nowhere close to a massive platform. Last summer,…

Elon Musk Testifies That He Started OpenAI to Prevent a ‘Terminator Outcome’

Elon Musk and Sam Altman appeared in a federal courtroom together for the first time on Tuesday as they fight over OpenAI’s decade-long evolution and what it means for the company’s future. The trial in Musk’s lawsuit against Altman could result in financial damages and, more significantly, governance changes at OpenAI that may complicate its plans for an initial public offering as soon as this year. As the first witness on the stand, Musk immediately sought to frame his case as more than just about OpenAI. Siding with Altman “will give license to looting every charity in America” and shake the “entire foundation of charitable giving,” Musk told a panel of nine jurors advising US District Judge Yvonne Gonzalez Rogers on how to rule. Musk has been concerned about computers becoming smarter than people “since he was a young man…

At 'AI Coachella,' Stanford Students Line Up to Learn From Silicon Valley Royalty

As thousands of influencers descended on southern California earlier this month for the annual Coachella Music Festival, a very Silicon Valley program dubbed “AI Coachella” was taking shape a few hundred miles north in Palo Alto. The class, CS 153, is one of Stanford’s buzziest offerings this semester, and like the music festival, it features a star-studded lineup of celebrities—in this case, not pop artists, but Big Tech CEOs. The course is co-taught by Anjney Midha, a former Andreessen Horowitz general partner, and Michael Abbott, Apple’s former VP of engineering for cloud services. The list of guest lecturers reads like a Signal group chat many VCs would pay to join: OpenAI CEO Sam Altman, Nvidia CEO Jensen Huang, Microsoft CEO Satya Nadella, AMD CEO Lisa Su, Anthropic philosopher Amanda Askell, and White House Senior Policy Advisor for AI Sriram Krishnan,…

Apple’s Next Chapter, SpaceX and Cursor Strike a Deal, and Palantir’s Controversial Manifesto

This week on Uncanny Valley, the team discusses what’s next for Apple as Tim Cook steps down from his role as CEO. They also go into the reasoning behind SpaceX and Cursor’s surprising deal, and why Palantir’s self-published manifesto drew a lot of heat online. Also, we discuss why some conspiracy theorists are leaving Trump’s side, and how a scammer created an AI-generated woman to attract and grift MAGA men. Articles mentioned in this episode: - Tim Cook’s Legacy Is Turning Apple Into a Subscription - MAGA Is Starting to Look Beyond Trump - This Scammer Used an AI-Generated MAGA Girl to Grift ‘Super Dumb’ Men You can follow Brian Barrett on Bluesky at @brbarrett, Zoë Schiffer on Bluesky at @zoeschiffer, and Leah Feiger on Bluesky at @leahfeiger. Write to us at [email protected]. How to Listen You can always…