Google transforms its data center architecture for agent era
Google adjusted the Google Kubernetes Engine into an agent-native environment, where agents could be quickly spun up in sandboxes and containers. “From an infrastructure perspective, you need to spin up a bunch of TPUs or GPUs very rapidly. Then you need to be able to run them and spin them back down,” Lohmeyer said.
Google also made drastic improvements to its silicon to support its middleware changes. It recently introduced new AI chips, with the TPU-8t for training, and TPU-8i for inference. The 8t chip has three times more computing power than the previous-generation Ironwood chip. The 8i chip has 384 megabytes of SRAM and 288GB of HBM3e memory, which is 50% more than the previous-generation chip.
The platform is optimized for KV cache (key-value cache), which stores important contextual information needed by agents to make decisions, which reduces the round trips to other memory and storage systems. “Being able to store more of the KV cache directly on the chip allows you to respond much more rapidly and cost-effectively,” Lohmeyer said.
