Cohere launches confidential computing for Model Vault AI inference
In short: Cohere has released a public beta of Confidential Computing on Model Vault, running on NVIDIA H100 GPUs via Microsoft Azure. The system encrypts data in use across both CPU and GPU, encrypts the request payload itself using Oblivious HTTP (OHTTP), and uses remote attestation so clients can verify the environment before sending sensitive data. If verification fails, the client refuses to send the protected request. Cohere plans to open-source the confidential inference stack and extend support to NVIDIA Blackwell (B200) GPUs toward general availability.
This summary was generated automatically by AI from Cohere's publication. It is our own text, not a copy of the original — facts, figures and quotes belong to the source, linked above and below.
What changed?
- 1Public beta of Confidential Computing on Model Vault live on NVIDIA H100 GPUs via Microsoft Azure
- 2Confidential virtual machine isolation extended to GPU memory and CPU-GPU communication via NVIDIA Confidential Computing
- 3Request payloads encrypted end-to-end using Oblivious HTTP (OHTTP) to a key tied to the attested environment
- 4Remote attestation verifies hardware/software state before sensitive data is released; verification failure blocks the request
- 5Plans to open-source the confidential inference stack; future support for NVIDIA Blackwell B200 GPUs
Why it matters
Enterprises sending sensitive prompts, documents, and responses to third-party AI inference services gain a mechanism to cryptographically verify that the processing environment cannot be read by the cloud provider, host, or operators, rather than relying solely on contractual trust.
What it means for AI agents and contact centers
If handling sensitive customer data (call transcripts, personal information, CRM data) through third-party AI APIs, this illustrates an emerging standard for verifiable confidential inference — relevant when evaluating cloud AI vendors for compliance-sensitive voice AI and contact-center workloads, though it does not replace prompt injection or application-level security controls.
🧪 Worth Testing
Worth evaluating as a reference architecture for verifying data protection claims of AI vendors handling sensitive call or customer data, particularly for GDPR/compliance-driven contact-center deployments.
Sources
- CohereOfficialPrimary sourceOriginal article →„Inside Cohere Model Vault: Building a verifiable confidential AI inference How Model Vault combines confidential CPU and GPU execution, end-to-end encrypted request contents, and remote attestation to protect sensitive AI workloads while they are being processed. Oct 22, 2026 8 min read“22 Oct 2026, 23:36
- Published by source
- 22 Oct 2026, 23:36
- Found by our system
- 22 Oct 2026, 19:59
- Summary generated
- 22 Oct 2026, 20:31
This article was written by AI from the original source. Facts, numbers and prices come from the source; missing values are marked “Not specified”. Legal notice, copyright and privacy