Snowflake and Google Cloud have built an open, interoperable data lakehouse using Apache Iceberg as a shared table format that lets multiple compute engines (BigQuery, Spark, Trino, Snowflake, and others) read and write the same data directly from customer storage without duplication or proprietary adapters.
Managed catalogs enforce unified governance across all engines, and semantic models plus contextual metadata help AI agents produce accurate, grounded answers by inheriting consistent business logic and data quality signals rather than hallucinating.
What happened
Snowflake and Google Cloud have architected an interoperable data platform using Apache Iceberg as a shared open table format, allowing multiple engines (BigQuery, Spark, Trino, Flink, Snowflake) to read and write the same data directly from a customer's cloud storage without copying. The setup uses managed catalogs—Google Cloud's Lakehouse runtime catalog and Snowflake's Horizon Catalog—to govern access and enforce security across engines through standardized REST API protocols and time-limited credentials.
Why it matters
Organizations no longer face the false choice between using one engine (sacrificing specialization) or many engines (creating data redundancy and inconsistency). By moving compute to where data lives rather than copying data everywhere, teams reduce costs, improve security, and let each group use its preferred tool while maintaining unified governance, masking, and audit trails. For AI workflows, this means agents can access governed, semantically grounded data across catalogs to produce accurate answers instead of hallucinations.
What to watch
Snowflake's Semantic View Autopilot (part of Horizon Context) automatically discovers and maintains semantic models from query patterns; a natural-language CoCo assistant for semantic definitions is in private preview. Google Cloud offers its Universal Semantic Layer and Knowledge Catalog to add business context to Iceberg tables. Both platforms expose data to AI agents via model context protocol (MCP) servers and REST APIs, enabling agentic workflows across multi-cloud environments.
Apache Iceberg emerged from Netflix's need to manage petabyte-scale tables and was later donated to the Apache Software Foundation. It has since become the industry standard open table format, supported by Spark, Trino, Flink, BigQuery, Snowflake and dozens of other engines. The fundamental insight behind Iceberg is the Data Locality principle: rather than copying massive volumes of data across a network to each compute engine, it is faster, cheaper and more secure to move a small piece of executable code to where the data already resides. With Iceberg as the shared language, all data sits in the customer's own storage bucket, and all engines agree on how it is physically laid out so each can read and write directly to the same table without proprietary adapters.
However, an open format introduces a new governance challenge: when multiple engines can read and write the same files, someone must manage table metadata, enforce access policies, coordinate concurrent writers and ensure no engine sees stale or inconsistent state. This responsibility belongs to the catalog, which serves as the lakehouse's governance layer and is the single authority that knows which tables exist, their schemas, who can access them and where data files physically reside. Without a catalog, the article notes, open data is ungoverned data. The catalog also controls storage access through vended credentials—short-lived, narrowly scoped storage tokens rather than standing bucket credentials, so engines do not gain persistent access to the underlying storage layer. For this model to work across engines built by different vendors, catalogs need a standardized integration protocol. The Iceberg REST Catalog (IRC) is the open API specification that solves this, defining how clients discover namespaces, load table metadata and commit updates. Every catalog exposes an IRC endpoint, and every engine connects as a client.
Snowflake and Google Cloud each provide a managed Iceberg catalog suited to their ecosystems. Google Cloud's Lakehouse runtime catalog is a serverless and scalable metastore that serves as a single source of truth for the data lakehouse, allowing multiple engines—BigQuery, Google-managed Spark, Apache Spark, Trino and Snowflake—to access the same copy of data across open formats like Iceberg. Federation allows agents and engines in Google Cloud to access data across borders from catalogs in Snowflake, Databricks and AWS Glue. Snowflake's Horizon Catalog integrates Apache Polaris, an open source IRC implementation co-created by Snowflake and Dremio and donated to the Apache Software Foundation, and embeds standards-compliant IRC endpoints to every Snowflake account with no additional setup. Beyond basic catalog operations, Horizon layers on enterprise governance: role-based access control (RBAC), column-level masking, row access policies, data lineage and audit logging. It allows Iceberg-compliant engines like Trino, BigQuery, Apache Flink and many others to read and write to Snowflake-managed Iceberg tables. When Google Cloud manages the catalog via Lakehouse runtime catalog, Snowflake connects through a Catalog-Linked Database (CLD) that automatically discovers and syncs tables from its IRC endpoints; Snowflake users interact with those Google Cloud Iceberg tables using standard SQL—SELECT, INSERT, UPDATE, DELETE—or via Snowflake's CoCo or CoWork, as if they were native tables, with Snowflake governance layered on top. Similarly, when Snowflake manages the catalog via Horizon, Google Cloud services reach those tables through Lakehouse catalog federation; BigQuery, Managed Service for Apache Spark and other Iceberg-compatible engines in Google Cloud connect to Horizon's IRC endpoint and read or write to Snowflake-managed Iceberg tables directly.
Beyond interoperability, the article emphasizes that data accessibility alone does not make data useful to AI. Three gaps remain: programmatic access (so AI agents, not just human analysts, can query data), semantic grounding (so AI systems understand what data represents and do not hallucinate), and contextual intelligence (surrounding knowledge that helps agents interpret data correctly). Snowflake addresses these through model context protocol (MCP) servers that allow any MCP-compatible client to discover, query and reason over lakehouse data; MCP is an open standard that provides a universal interface between AI applications and data sources, meaning Gemini Enterprise, code assistants and custom agent frameworks all connect through the same protocol. BigQuery also offers an MCP server accessible from first-party and third-party AI agents, enabling programmatic access for agentic workflows across Google Cloud and multi-cloud environments. For semantic grounding, Snowflake's Semantic View Autopilot, part of Horizon Context, automatically discovers and maintains semantic models by learning from query patterns, user behavior, table relationships, BI tool integrations and report usage; teams that prefer to author models in natural language can use Semantic Studio's built-in CoCo assistant (in private preview), which works alongside a visual and YAML editor with Git integration, so semantic definitions can be version-controlled like code. Google Cloud's Universal Semantic Layer leverages Looker's support for in-database analytics models and integrations with BigQuery Graph and Snowflake Semantic Views. For contextual intelligence, Snowflake's Horizon Context creates and applies a contextual layer including table and column descriptions in business terms, data quality signals (freshness, completeness, caveats), usage patterns and domain relationships; because this context lives in Horizon, it inherits the same governance as the data itself. Google Cloud Lakehouse provides trusted context through Knowledge Catalog, delivering Gemini-powered insights that help mitigate hallucinations and enable users to turn insights into action. The result is that AI agents can move from generic answers ("revenue might be...") to grounded responses ("Q4 revenue was $47.2M from the authoritative finance.revenue table, filtered by fiscal quarter definition").
The core tension Snowflake and Google Cloud address is that modern data teams no longer use a single analytics engine—they combine Spark, BigQuery, Gemini Enterprise Agent Platform, Snowflake's own CoCo and CoWork, and other specialized tools within the same pipeline depending on the job. Historically, this forced a brutal choice: replicate data across systems (creating redundancy and inconsistency) or force everyone through one engine (sacrificing specialization). The industry's solution emerged from the Data Locality principle: it is faster, cheaper and more secure to move executable code to where data lives than to move massive volumes of data across a network.
Apache Iceberg provides the open table format that makes this possible. By standardizing how data is physically laid out, Iceberg lets all engines read and write directly to the same tables without proprietary adapters. But an open format alone is insufficient—multiple engines writing concurrently risk corruption and inconsistency. That is where the catalog enters: it serves as the single governance authority, managing table metadata, enforcing access policies, coordinating writers and ensuring consistency. Snowflake's Horizon Catalog and Google Cloud's Lakehouse runtime catalog both implement the Iceberg REST Catalog (IRC) standard, a universally understood API that allows any engine to discover tables and commit updates. Crucially, both catalogs issue time-limited, scoped credentials rather than standing bucket access, so engines never gain persistent access to underlying storage—a design that makes federation (bidirectional cross-catalog access) secure and practical.
For AI workflows, data accessibility alone falls short. The article identifies three remaining gaps: programmatic access (so agents, not just human analysts, can query lakehouse data), semantic grounding (so AI systems understand what data represents and do not hallucinate), and contextual intelligence (surrounding knowledge—freshness, authority, domain relationships—that helps agents interpret data correctly). Snowflake addresses these through model context protocol (MCP) servers, Semantic View Autopilot (which auto-discovers semantic models from query patterns), and Horizon Context (which governs metadata). Google Cloud offers comparable mechanisms through BigQuery's MCP server, its Universal Semantic Layer, and Knowledge Catalog. Together, these layers allow agents to inherit consistent business logic and data context across the entire lakehouse, reducing drift and hallucination.
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytime
Ask AI anything about this article. Q&As are published on this page for other readers too.
The AI news that matters, in one minute each morning.
Sign up free