Snowflake, Google Cloud unite on open lakehouse for AI agents

Snowflake and Google Cloud have converged on a unified approach to building an interoperable, AI-ready data lakehouse. The foundation rests on Apache Iceberg, an open table format originally developed at Netflix for managing petabyte-scale tables and later donated to the Apache Software Foundation. Iceberg is now supported by Spark, Trino, Flink, BigQuery, Snowflake, and dozens of other engines, creating a shared language so each engine can read and write directly to the same table without proprietary adapters.

Beyond format, the architecture requires governance. The Iceberg REST Catalog (IRC) is an open API specification that defines how clients discover namespaces, load table metadata, and commit updates. It solves a coordination problem: when multiple engines can read and write the same files, someone must manage metadata, enforce access policies, coordinate concurrent writers, and ensure no engine sees stale data. Each catalog exposes an IRC endpoint; every engine connects as a client. The catalog also controls storage access through vended credentials—short-lived, narrowly scoped tokens rather than standing bucket credentials—so engines do not gain persistent access to the underlying storage layer. This vended-credential model enables federation: when a Snowflake query reaches into Google Cloud's Lakehouse catalog, or a BigQuery job reads from Snowflake Horizon, each catalog issues only the narrow, time-limited tokens the requesting engine actually needs, enforcing governance regardless of which engine initiates the request.

Snowflake's managed solution is Snowflake Horizon Catalog, which integrates Apache Polaris and embeds standards-compliant IRC endpoints to every Snowflake account with no additional setup. Beyond basic catalog operations, Horizon layers on enterprise governance: RBAC, column-level masking, row access policies, data lineage, and audit logging. It allows Iceberg-compliant engines like Trino, BigQuery, Apache Flink, and others to read and write to Snowflake-managed Iceberg tables. On Google Cloud's side, the Lakehouse runtime catalog is a serverless, scalable metastore serving as a single source of truth for the data lakehouse, allowing multiple engines (BigQuery, Google-managed Spark, Apache Spark, Trino, Snowflake) to access the same data across open formats. Federation allows agents and engines in Google Cloud to access data across borders from catalogs in Snowflake, Databricks, and AWS Glue. When Google Cloud manages a catalog, Snowflake connects through a Catalog-Linked Database (CLD) that automatically discovers and syncs tables from its IRC endpoints. Snowflake users interact with those tables using standard SQL—SELECT, INSERT, UPDATE, DELETE—or via Snowflake's CoCo or CoWork, as if they were native tables, with Snowflake governance layered on top. Similarly, when Snowflake manages a catalog, Google Cloud services reach those tables through Lakehouse catalog federation.

Making the lakehouse AI-ready requires bridging three gaps: programmatic access, semantic grounding, and contextual intelligence. For programmatic access, Snowflake exposes model context protocol (MCP) servers that allow any MCP-compatible client to discover, query, and reason over lakehouse data. MCP is an open standard providing a universal interface between AI applications and data sources, so Gemini Enterprise, code assistants, and custom agent frameworks all connect through the same protocol rather than requiring bespoke integrations. BigQuery also offers an MCP server accessible from both first-party and third-party AI agents. For semantic grounding, business logic belongs in the data layer, not in AI prompts. Snowflake's Semantic View Autopilot, part of Horizon Context, automatically discovers and maintains semantic models by learning from query patterns, user behavior, table relationships, BI tool integrations, and report usage. Semantic Studio's built-in CoCo assistant (in private preview) works alongside a visual and YAML editor with Git integration, allowing semantic definitions to be version-controlled and reviewed like code. Google Cloud's Universal Semantic Layer, leveraging Looker's new support for in-database analytics models and integration with BigQuery Graph and Snowflake Semantic Views, delivers Graph models to business users in agentic journeys. For contextual intelligence, Snowflake Horizon Context creates a contextual layer: table and column descriptions in business terms, data quality signals (freshness, completeness, caveats), usage patterns (which tables are authoritative for which questions), and domain relationships. Google Cloud Lakehouse provides trusted context through Knowledge Catalog, delivering Gemini-powered insights for lakehouse tables that help mitigate hallucinations in AI agents. This context applies to any Iceberg table, whether Snowflake-native, CLD-linked, or federated from Google Cloud. The result is that AI agents operate across structured and unstructured data with high accuracy, and teams can pick the best tool for each job based on their unique preferences while the catalog layer enforces unified governance.

The announcement reflects a fundamental shift in how enterprises handle multi-engine data architectures. Historically, organizations faced a choice: either let each team use its preferred tool and accept data redundancy and inconsistency, or force all teams through a single engine and lose specialization. Apache Iceberg, developed at Netflix and donated to the Apache Software Foundation, solved the first layer of this problem by providing a shared open table format. However, format alone does not ensure governance. The Iceberg REST Catalog standard (an open API specification) became the second pillar, creating a single authority to manage table metadata, enforce access policies, and issue vended credentials that prevent engines from gaining persistent access to underlying storage.

Snowflake and Google Cloud's alignment on this stack is significant because it eliminates the need for custom integrations between their respective catalog systems (Horizon and Lakehouse runtime catalog). Through bidirectional federation, an organization can start with one managed catalog and incrementally add others as needs evolve, all while maintaining unified governance. This is operationally simpler than a DIY self-managed catalog (such as Apache Polaris), which shifts infrastructure and scaling responsibility to the customer's team.

The third layer—making the lakehouse AI-ready—addresses a different problem. Data accessibility does not ensure data usefulness to AI agents. Snowflake's Semantic View Autopilot and Google Cloud's Knowledge Catalog close the gap by automatically discovering and maintaining semantic models (business logic), and by attaching contextual metadata (data quality signals, authoritative source definitions, hierarchies) directly to the data. When an AI agent queries, it receives not just raw data but the institutional knowledge needed to interpret it correctly, which reduces hallucination and aligns answers with business rules.

No comments yet. Be the first to share your thoughts!