You're offline - Playing from downloaded podcasts
Back to All Episodes
Podcast Episode

AI's Plumbing Week: Enterprise MCP Auth, Persistent Agent Harnesses, Phones Hitting 63 on Benchmarks, and Xiaomi's 1.22TB/s AI Cube

August 25, 2026

0:00
14:59
Podcast Thumbnail

A supposedly quiet few days in AI turned into a masterclass in infrastructure: Anthropic centralised enterprise auth for MCP connectors, open-source "microharnesses" like Headlong and exo made always-on self-repairing agents real, and NVIDIA argued that scanning agent skills predicts almost nothing about usefulness. Meanwhile Liquid AI's Pipette benchmark suite showed phones scoring 63 on quality evals with sub-6-second responses, NVIDIA's Groq 3 LPX claimed 3,400 output tokens/s on Gemma 4 31B, and Xiaomi announced a three-chip AI Cube with 1.22TB/s of memory bandwidth. Plus speculative tool calling, dense-to-MoE conversion, and a 250M-parameter model that deploys in 60MB.

Enterprise MCP Grows Up

Anthropic has rolled out enterprise-managed authentication for MCP (Model Context Protocol) connectors, routing authorisation through an organisation's own identity provider instead of asking every employee to complete per-tool OAuth. Connectors covered include Asana, Atlassian, Canva, Datadog, Figma, Notion, Slack and Supabase. The published MCP roadmap adds long-running workloads with streaming and server push, HTTP for local servers, progressive discovery for very large tool catalogues, and standard delegated identities. This is the gap between an impressive demo and something a security team will actually approve.

The Harness Is the New Model

Several independent threads converged on the same thesis: agent quality is now shaped as much by the harness as the base model. NVIDIA's new evaluation work found that structural "scans" of agent skills correlate with judged quality at a Spearman coefficient of just 0.14, and proposes measuring "Skill Lift" instead: run the identical task with and without the skill and score the delta in completed work. A separate position paper argues enterprises should standardise on one reusable coding-agent harness rather than bespoke orchestration graphs.

Persistent Agents Ship as Open Source

Headlong, an open-source "microharness" from Andy Konwinski, keeps agents thinking continuously rather than only on request, storing trajectories as a graph of JSONL files. It reportedly completed an unattended self-debugging repair in 48 minutes, at a background cost of roughly $1–2 per hour. A complementary architecture, exo, uses an append-only event log, a swappable executor, and a snapshot-and-rollback sandbox so an agent can rewrite its own prompts, tools and memory without corrupting durable state.

Speculative Tool Calling

Speculative Programmatic Tool Calling predicts safe tool calls during code generation and launches them early in a copy of the environment, overlapping execution with token generation. Gains are modest so far at roughly 1.0–1.2x, but the idea shifts optimisation from decoding tricks to workflow pipelining, borrowing directly from CPU speculative execution.

Phones Get a Serious Benchmark

Liquid AI released Pipette, an open-source on-device evaluation suite measuring quality, speed, latency and memory across model, quantisation, runtime and device combinations, with over 10,000 verified results spanning 35 model classes, 7 quants and four devices. Independent phone-scale testing on iPhone 17 Pro and Galaxy S26 Ultra put Nanbeige4.2-3B and LFM2.5-2.6B jointly top at 63, with LFM2.5-2.6B far more efficient on iPhone at 8.0s and 2.3GB versus 21.4s and 4.0GB. Mixture-of-Experts designs like LFM2.5-8B-A1B and Ling 3.0 Tiny activate around 1B parameters per token, enabling sub-6-second replies.

Silicon and Serving

NVIDIA's Groq 3 LPX adds a dedicated token-generation accelerator to Vera Rubin, with a claimed 3,400 output tokens per second on Gemma 4 31B at 100K context; Groq says it will be among the first to deploy it. vLLM's AgentX 1.0 results on real multi-turn coding traces point to KV offload, prefix reuse and prefill/decode disaggregation as the real levers for agentic throughput. And Xiaomi announced a prototype AI Cube built on three in-house chips with a headline 1.22TB/s of memory bandwidth.

Making Models Smaller

The ToMoE paper converts dense LLMs into Mixture-of-Experts models via differentiable dynamic structural pruning, keeping the original weights rather than deleting them. At the extreme end, SHADOW-250M is a from-scratch 250M-parameter model trained on 30B tokens, quantised under 2 bits, deploying in 60MB and running around 400 tok/s on a laptop CPU.

Published August 25, 2026 at 3:32am