AI Agents: Slower Progress, Sharper Tooling

Reality Check: Agents Are Harder Than Expected

Mark Zuckerberg acknowledged to Reuters that AI agent development is proceeding more slowly than anticipated, citing the complexity of building systems that can reliably complete multi-step tasks autonomously. This aligns with field observations published on Hacker News by Dan Luu, whose notes from working with agentic coding loops in a remote environment (Galapagos Island) document persistent failure modes: context loss, tool misuse, and the difficulty of keeping long-running agents on task without human correction.

Together, these two data points—one from a major lab CEO, one from a practitioner in the field—suggest the industry is entering a more honest phase of expectation-setting around agent autonomy. Neither source suggests abandoning the direction; both point to the need for better scaffolding, evaluation, and feedback loops.

Benchmarking Agent Capability: Senior SWE-Bench

Snorkel AI has released Senior SWE-Bench, an open-source benchmark designed to evaluate AI coding agents against tasks representative of senior software engineering work—going beyond the bug-fix and patch-application tasks that characterise the original SWE-Bench. The benchmark aims to surface whether agents can handle design decisions, refactoring at scale, and cross-file reasoning, which are the competencies that matter most in production engineering contexts.

This is a meaningful methodological step: as Zuckerberg's comments highlight the gap between agent promise and delivery, having more granular, role-calibrated benchmarks helps practitioners make evidence-based decisions about where to deploy agents and where human oversight remains essential.

Open-Source Tooling: Quality, Documentation, and Security

Three open-source projects surfaced today that address distinct but related problems in agent development. QUALITY.md (getquality.md) introduces an open format and CLI specification for defining and communicating agent skill quality standards in a machine-readable way—useful for teams that need to codify what "good" looks like for a given agent task. OpenWiki from LangChain (github.com/langchain-ai/openwiki) is a CLI that automatically writes and maintains documentation for agent codebases, addressing the chronic under-documentation problem that makes agent systems hard to audit and extend.

DepTrust (github.com/clidey/deptrust) takes a security angle, offering a CLI that helps AI agents avoid pulling in vulnerable dependencies during code generation—a practical guard against a known supply-chain risk that becomes more acute as agents write and execute code autonomously. Separately, Valmis (github.com/valmishq/valmis) positions itself as an open-source alternative to Claude Cowork, providing a self-hostable collaborative coding environment for agent-assisted development.

Infrastructure and Safety: MCP Cloud, Fable 5, and Claude in Slack

Manufact (YC S25, manufact.com) launched as an MCP (Model Context Protocol) Cloud provider, offering managed infrastructure for teams building on MCP-based agent architectures. As MCP adoption grows, managed hosting and orchestration layers are a logical next step for teams that want to avoid operating their own MCP servers.

On the safety side, Anthropic published additional details on cyber safeguards and a jailbreak framework developed in the context of Fable 5—providing transparency into how the company approaches adversarial robustness for a specific high-stakes deployment. Anthropic also shared how its own teams use Claude via a Tag integration in Slack, offering a concrete internal workflow example that practitioners can reference when evaluating similar integrations. Both disclosures reflect a broader Anthropic pattern of publishing operational and safety details alongside product releases.

Key takeaways

• Zuckerberg (Reuters) publicly acknowledged AI agent development is slower than expected, signalling a broader industry recalibration on timelines.
• Senior SWE-Bench (Snorkel AI) raises the bar for agent evaluation by targeting senior-engineer-level tasks beyond basic bug fixes.
• QUALITY.md, OpenWiki, and DepTrust represent a maturing open-source ecosystem focused on agent quality standards, documentation, and dependency security.
• Manufact (YC S25) enters the market as a managed MCP Cloud provider, reflecting growing demand for hosted agent infrastructure.
• Anthropic's dual disclosures—Fable 5 safety framework and internal Claude-in-Slack usage—continue its pattern of operational transparency.

Sources

• Agentic coding notes from Galapagos Island — Hacker News
• More details on Fable 5’s cyber safeguards and our jailbreak framework — Anthropic
• Zuckerberg says AI agent development going slower than expected — Hacker News
• Show HN: QUALITY.md – open format/specification, agent skill, and CLI — Hacker News
• Launch HN: Manufact (YC S25) – MCP Cloud — Hacker News
• Show HN: I built an open-source alternative to Claude Cowork — Hacker News
• Senior SWE-Bench: open-source benchmark that assesses agents as senior engineers — Hacker News
• How Anthropic works with Claude Tag in Slack — Anthropic
• OpenWiki: CLI that writes and maintains agent documentation for your codebase — Hacker News
• Show HN: CLI that helps AI agents avoid vulnerable dependencies — Hacker News