Most people who use AI and invest in AI Infrastructure operate at a high abstraction level and chase trends. However, developing a deep and broad understanding of AI Infrastructure is difficult because domain-specific technical data lives in isolated silos and isn't translated well to a broad audience. As a result, engineers often don't have system context of what they are designing and investors often don't have a decent mental model of what they are investing in. My goal is for investors and engineers to think more like system architects, because architecture decisions have financial implications and influence day-to-day execution of engineering teams. Over the past year, I've travelled to the semiconductor industry's flagship conferences and learned how these subsystems worked from the industry's foremost experts. I've written a complete suite of mental models on the subsystems that comprise AI Infrastructure: - Mixed-Signal ICs (SerDes, ADCs, PLLs, Bandgaps) - High Speed Optical/CPO - High-Power PMICs - High-Speed Signal Integrity - Advanced Packaging - AI for Chip Design / EDA - AI Accelerators - Memory (HBM and SRAM) Link to the website is in my bio and in the comment below 👇

The Halo effect:

Ken Griffin of Citadel questions the economics of the AI infrastructure boom: ›What happens when GPU supply exceeds demand & pricing goes into free fall? ›Are today's semiconductor margins sustainable? Are investors overestimating the useful life of GPUs? “Look, I think this market is actually experiencing people doing research. For the couple of years, if you were on the thematic program of we're gonna spend hundreds of billions or a trillion dollars in data centers, we're gonna build compute at a scale never seen before, which companies in that story are choke points in supply chains that will have abnormally high margins? And who will be the big winners in this transformation? Will it be the Microsofts & AWSs who will be the gatekeepers to the foundational models? Like, who's going to earn the economic rents? But in some sense, like, 'I'm just gonna buy them all. I'm gonna buy the hyperscalers, I'm gonna buy the chip, I'm gonna buy businesses that make things I've never even heard of 5 years ago in Korea and Taiwan & in Japan.' And now we're reaching a point where people are becoming more discerning. We do realize that we are seeing extremely high margins in DRAM & MLCCs in areas where there are real supply constraints, but people are now starting to look out 2 years, 3 years. How long is this going to last? And in particular, I don't wanna be doom & gloom, but GPUs, like airlines, have a very low marginal cost to run. The incremental seat on an airline that's not full, if you can sell that ticket for $20, you're making money in that seat. And that's one of the reasons that the airline industry is such a nightmare, is because the moment you have overcapacity, pricing goes into free fall. We saw this actually with the telecom build-out back in 2000. We built more fiber than we had demand for, & the price of telecommunications just went straight down. If we reach a point where the provisioning of GPUs exceeds demand for GPUs, marginal pricing will actually go into free fall. And that really upends all the investment assumptions that people are working with today. That's issue number one. And issue number 2 is: I think people are taking a step back & saying, 'What is the useful life of a GPU?' Right here, right now, there's people that are earning extraordinary pricing on GPUs that are 5 or 6 years old. And so there's a growing set of concerns that I can't give the same capitalization rate or the same revenue models to year 7, 8, 9 of a chip in the future that maybe I was using 12 months ago. So I think we're really grappling with just how far will this demand carry us in the marketplace.”

The six materials that shape and power AI datacenters - sand, salt(?), iron, copper, oil, and lithium

Agreed

How JEDEC standards are an impediment to pushing memory BW and the technical fundamentals needed to co-design HBM for $MU : siliconcodesign.com/p/the-sy…

The key processing steps of hybrid bonding, how thermo-mechanical stress makes it hard to hybrid bond multiple layers of HBM die, and key suppliers (besides BESI): siliconcodesign.com/p/why-hy…

AT GTC Berlin, @nvidia announced DSX Pods, modular pods of Vera Rubin Racks for fast deployment. Modular pods are going to play a key role in getting AI compute out to market quickly and mitigating the heavy CapEx from the building infrastructure.

Why AI is having a hard time automating chip design - for now siliconcodesign.com/p/design…

Congestion Signaling (CSIG) is potentially one of the most significant, yet underdiscussed advances in AI data center networking. The UEC released spec 1.1 on October 12, incorporating CSIG. Traditional congestion control often relies on indirect signals such as ECN and RTT. CSIG goes a step further by providing more detailed congestion information from switches along the data packet's path that are updated in a compact tag in the packet. CSIG can potentially enable more balanced network traffic and reduce congestion that significantly slow down AI workloads. ultraethernet.org/ultra-ethe…

He's going to have a lot of tough questions at @TomorrowXSummit

Yep, Jensen is right. Europe has great researchers and engineers, but are often constrained by the surrounding system that stifles their potential.

The new rule in AI Infra racks: "Standardize where you can, tightly integrate where you must."

What is this chaos going on in Europe

For best thermal performance of AI Accelerators, the cold plate should be co-designed with the physical design of the silicon and the package. On-chip physical designers should be cognizant of potential hotspots and place power-hungry blocks in such a way to minimize hotspots and uniformly distribute heat throughout the package. Source: Y. Hang "Pushing the Single-Phase Cold Plate Limit" - piped.video/watch?v=niepkE3b…

College football might be one of the most successful marketing campaigns for the “college experience” in American history.

My upcoming post will explore AI rack trends in @OpenComputePrj through Clayton Christensen's framework of modular versus interdependent architectures. Christensen's work has reportedly influenced @JensenHuang who has praised "The Innovator's Dilemma" for explaining how technology industries evolve. His framework offers a useful lens for understanding where economic value might shift across the industry. At @OpenComputePrj, I focused on analyzing three interconnected areas - power, cooling, and networking - while paying attention to which interfaces are becoming standardized. What's interesting is that while some parts of AI infrastructure are becoming increasingly modular, others still require tighter system-level integration to achieve maximum performance. I'm really looking forward to connecting the technical topics I've written about with the broader business strategy!

Why optical testing could become one of the biggest bottlenecks in CPO manufacturing Unlike traditional semiconductor testing, CPO requires coordination between electrical ATE, optical instrumentation, and precision mechanical alignment. Any imperfections in fully assembled units become costly to scrap, making known-good die screening critical early on. siliconcodesign.com/p/why-op…