Mizuho to Build Secure AI Infrastructure Using NVIDIA Technologies

Mizuho Financial Group announced the start of a collaborative exploration with NVIDIA to advance its AI infrastructure, with the explicit aim of enabling financial institutions to deploy generative AI and AI agents securely and at scale. The initiative was announced by Masahiro Kihara, President and Group CEO of Mizuho.

The exploration encompasses two technical workstreams. First, Mizuho will examine the introduction of NVIDIA DGX B200, a high-performance computing platform equipped with eight NVIDIA Blackwell GPUs connected via fifth-generation NVIDIA NVLink. This hardware will serve as the foundation for the bank's on-premises GPU environment, supporting the training, evaluation, improvement, and inference validation of generative AI models, including the Mizuho LLM currently under development. Beyond the initial DGX B200 deployment, Mizuho plans to study the future construction of a GPU cluster that connects multiple GPU servers, enabling efficient large-scale training and inference. The bank will examine required architecture and operational considerations from perspectives including AI workloads, networking, storage, LLMOps, security, operations management, and scalability, with the goal of creating a hybrid cloud-and-on-premises environment tailored to different use cases based on confidentiality, performance, and cost.

Second, Mizuho will conduct technical validation of NVIDIA NemoClaw, a collection of reference blueprints for securely executing and managing AI agents. NemoClaw is built on NVIDIA Nemotron open models and the NVIDIA OpenShell secure runtime. In this validation, Mizuho will examine critical governance issues including execution environment isolation, data protection, network access control, permission management, and verification of execution history, under the assumption that AI agents are integrated with internal data and business systems. Parallel to this work, Mizuho will validate integration with generative AI models, including the Mizuho LLM, and assess the potential of AI agent use based on internal business context.

The financial services industry faces a fundamental tension that this initiative aims to resolve. Generative AI and AI agents hold significant promise across financial operations—including information gathering, document preparation, inquiry handling, sales support, credit assessment and monitoring, and system development support. Yet financial institutions must simultaneously meet stringent criteria: protection of highly confidential data and business information, compliance with laws and regulations, auditability of usage, and control over agent execution environments. Mizuho's approach separates the infrastructure layer (secure hardware and runtime) from the policy layer (permissions, isolation, audit trails), enabling the bank to expand AI agent autonomy while maintaining governance.

Looking forward, Mizuho intends to securely integrate generative AI models and highly confidential data with internal systems, expanding the scope of AI agent utilization while preserving the governance and control the financial sector demands. The bank frames this as an opportunity for employees to use AI agents as partners in daily work, enabling more efficient information gathering, document preparation, analysis, and development support. By amplifying each employee's expertise and freeing time for customer dialogue and higher-value activities, Mizuho aims to enhance customer value. The bank positions AI not as a tool for efficiency alone, but as a management foundation for deepening employee expertise and increasing customer value, with plans to collaborate with advanced domestic and global technology partners to establish a model case for secure and trusted AI use at financial institutions.

Mizuho's exploration reflects a broader challenge facing financial institutions: how to harness generative AI and autonomous agents without compromising the data security and regulatory compliance that banking demands. The initiative addresses this by grounding the infrastructure in two complementary layers—on-premises GPU computing for controlled model development and training, and a secure agent runtime environment with built-in governance controls.

The choice of NVIDIA DGX B200 as the foundation for on-premises AI workloads signals Mizuho's intention to maintain direct control over sensitive model training and inference validation. This is critical for the bank's proprietary Mizuho LLM, which is designed to embed specialized financial knowledge and internal rules, and cannot safely reside in public cloud environments. By studying a future GPU cluster architecture, Mizuho is also preparing for production-scale inference demands—a sign the bank expects AI agent deployment to grow well beyond proof-of-concept.

The parallel validation of NVIDIA NemoClaw underscores the governance challenge. Financial institutions must ensure that AI agents, despite their autonomy, operate within strict boundaries: they cannot access data beyond their assigned scope, cannot execute transactions without proper authorization, and must leave auditable records of all decisions. Mizuho's focus on permission management, data isolation, and execution history verification indicates this is not a casual experiment but a methodical building of trust in the technology before wider rollout.

No comments yet. Be the first to share your thoughts!