Google showcased its AI ecosystem at Google Cloud Next '24

The Google Cloud Next '24 opening keynote, delivered by Google Cloud CEO Thomas Kurian, was packed with introductions of AI innovations all across the board. From the AI Hypercomputer, Google's AI infrastructure, to the Vertex AI Agent Builder, Gemini Code Assist, new AI-driven cybersecurity, and Workspace-specific features, the company left no stone unturned. Below, we recapitulate some of the most noteworthy announcements featured in the Google Cloud Next '24 keynote.

AI Hypercomputer

Google seems well aware that an industry-leading needs an equally innovative foundation, leading to key advancements in all the company's AI stack layers.

• The H100 Tensor Core GPU-based A3 Mega will be generally available next month, delivering twice as much GPU bandwidth as the current A3 GPU instances. Confidential A3 instances will be introduced to protect customers' privacy and sensitive data.
• In addition, the new NVIDIA Blackwell architecture's expected arrival at Google Cloud in early 2025 will bring HGX B200 devices to support demanding AI and data analytics workloads, and GB200 NVL72 devices to power real-time LLM inference and trillion-parameter model training performance.
• Google's custom AI accelerator, the TPU v5p is now generally available along with Google Kubernetes Engine support for TPU v5p.
• Other infrastructure launches include storage options geared towards AI, new Dynamic Workload Scheduler options, and a slew of features for the Google Distributed Cloud.

General workload advancements were introduced alongside AI announcements, the most relevant being the availability of Google Axion, a performant and energy efficient custom Arm-based CPU.

Vertex AI Agent Builder

Google's enterprise AI platform, Vertex AI, currently homes over 130 models, ranging from popular open source models like Gemma, Llama and Mistral to proprietary partner models such as Claude. Vertex AI is the starting point for a host of AI applications, so it makes sense that the first measure to ensure Vertex AI remains a comprehensive AI application platform is the expansion of its offerings. Starting immediately, state-of-the art models such as Gemini 1.5 Pro, CodeGemma, Claude 3 and Imagen 2 (with Digital Watermarking).

Every model hosted in Vertex AI can be fine-tuned using techniques including Reinforcement Learning with Human Feedback (RLHF), distilling, and supervised adapter-based tuning techniques such as Low Rank Adaption (LORA). Gemini models can now be finetuned using supervised adapter-based tuning techniques for cost-reducing, efficient training. Additionally, Vertex AI now offers two new model grounding capabilities: Google Search, and private data from applications such as Workday or Salesforce, and Google Cloud database offerings including AlloyDB and BigQuery. For the final stage of model deployment, Google announced new ML Ops features such as Prompt Management Tools, Automatic side-by-side and Rapid Evaluation.

Gemini Code Assist

The coding assistant has been tested in-house by developers and by some of Google's customers, and the reports of increased productivity abound: Google reports that Gemini Code Assist users complete common development tasks 40% faster, and spend 55% less time writing new code. Google's customer Quantiphi reportedly saw a 30% productivity increase in its dev team. Gemini Code Assist plugs into private codebases wherever they are (on premises, GitHub, GitLab, Bitbucket, or a combination), has been upgraded with Gemini 1.5 Pro, and now provides end-to-end support across the application lifecycle with Gemini Cloud Assist.

Google's AI-ready Data Cloud

The preview of Gemini in BigQuery assists users in the data preparation process, including discovery, analysis, and governance. Additionally, the new BigQuery data canvas (also in preview) is a notebook-like experience that enables users to build and execute data pipelines using natural language and embedded visualizations. A further preview, the BigQuery integration to Vertex AI will enable customers to extract insights from unstructured data and build multimodal analysis scenarios. Gemini in Looker lets users talk to their business data thanks to the introduction of Conversational Analytics, which offer a dedicated space in Looker where users can initiate a chat on any topic related to the organization's business data as if it were a conversation with an expert.

On the operational database front, Gemini in Databases assists customers in the database migration process, so they can swiftly transform legacy system databases into modern cloud databases using tools like AlloyDB. Vector indexing is also available in AlloyDB, meaning that here too users can leverage AI over their data without moving it to increase the accuracy of the responses. Gemini in Databases offers a full range of tools, which also include bringing generation and summarization capabilities to Database Studio.

AI-driven cybersecurity

Gemini in Threat Intelligence and Gemini in Security Operations are tools poised to become an essential member in any organization's security team. Gemini in Threat Intelligence uses natural language to deliver useful insights about malicious behavior. By leveraging Gemini's expanded context window, the tool is capable of analizing substantial chunks of code and the relationship between its modules, making it easier to provide accurate insights on the code's true intent. Similarly, Gemini in Security Operations is a tool that translates natural language inputs into detections, event data summaries, and action recommendations. This tool also assists users in navigating the platform through its conversational chat interface.

Gemini for Workspace

Google reports that Workspace, its productivity suite, is the world's most popular at 3 billion users and over 1o million paying customers. "Help me write", one of Workplace's first AI-powered features, has 70% of users following its suggestions in Docs and Gmail. Additionally, over 75% of users that create images in Slides also end up using them. Building on the success of these initial features, Google is introducing Google Vids. Sitting among Docs, Sheets, and Slides, Vids is an AI-powered video editor that assists users from the storyboard stage up to the final product. Google Vids will launch in the Workspace Labs in June.

Other notable announcements include an AI Messaging and Meetings add-on that can take notes, summarize and translate in 69 languages (4,600 language pairs). The add-on will be available for $10 per user, monthly.

More generally, Gemini for Google Cloud features Shared Fate, where Google commits to take responsibility for copyright violation claims against its customers, gives users total control over their data, and enables them to track sources and connect responses to their data sources to increase accountability.