Cohere adds confidential computing to Model Vault

Cohere launched a public beta of confidential computing for Model Vault, adding controls designed to protect enterprise prompts and responses while its managed inference service processes them. The service runs on NVIDIA H100 GPUs in Microsoft Azure, and customer-side software can verify the computing environment before it sends sensitive data.

Cohere says the service establishes a hardware-backed confidential boundary across the CPU and GPU, protecting GPU memory and encrypting traffic between the two. Microsoft documents an Azure confidential-GPU configuration that combines an AMD SEV-SNP confidential virtual machine with an attached H100 GPU.

Remote attestation provides the central check. Customer-side software compares cryptographically signed evidence about the hardware, firmware and software inside the protected environment with an approved policy. Cohere says the software refuses to send a request if the evidence does not match. NVIDIA describes remote attestation as a way to prove that a trusted execution environment is in an expected state before secrets are released.

A separate layer protects the payload. Cohere’s technical architecture says Model Vault uses Oblivious HTTP to encrypt sensitive application data to a public key associated with the attested environment. Intermediaries can route the ciphertext, but only the OHTTP gateway inside that environment holds the corresponding decryption key. Cohere says responses are protected on the return path, while customers can continue using the existing Model Vault API and request formats.

The design still depends on trusted components. The model operates on plaintext inside the protected environment, while the confidential-computing hardware and firmware, measured software, approved reference measurements and attestation infrastructure remain within the trust boundary. Cohere says attestation provides assurance only when signed evidence is checked against an appropriate policy.

The encrypted application payload does not cover all information. Request timing, message size, traffic volume, routing information, the API credential and model identifier remain visible outside it. Cohere also says the feature does not address prompt injection, insecure tool use, authorization errors or other application-level vulnerabilities.

The company excludes some physical and side-channel attacks from the protection boundary, including potential inference from timing or shared-hardware behavior. NVIDIA’s security white paper similarly places sophisticated physical attacks and denial-of-service attacks outside its confidential-computing threat model. Cohere says operators can still stop or delay a workload, so the feature does not provide an availability guarantee.

Cohere also says confidential computing is not a data-residency mechanism and does not replace compliance obligations, contractual controls, access controls, certifications or security review. The company did not disclose beta pricing, geographic availability, capacity limits, customer eligibility or a general-availability date.