While up in Calgary in Canada a few weeks ago, we had the chance to do two pieces. The first was the How Liquid Cooling Servers Works with Gigabyte and CoolIT piece. While there, something that struck me was that it would be interesting to show how CoolIT is addressing the dreaded “L-word” in liquid cooling: leaks. CoolIT has an entire Liquid Lab dedicated to prototyping and testing designs as well as doing integration and compatibility testing to ensure that solutions perform well and are reliable in the field. In this piece, we are going to show off the lab.

This is one of those pieces that probably makes more sense to watch via the video that you can find here:

Also, if you want to see a system being assembled, you can see our How Liquid Cooling Servers Works with Gigabyte and CoolIT here:

As always, we suggest opening these in a YouTube window, tab, or app for a better viewing experience.

The CoolIT Liquid Lab has a number of machines for making prototypes. That includes CNC machines as well as skiving machines and 3D printers. We could show the CNC machine but the other machines we could not show.

In the video, you can see the CNC machine at work. This is important because it allows for rapid prototyping of designs and then being able to put them into test platforms quickly. CoolIT also gets parts from third party vendors, but having in-house prototyping is important. These machines were busy during our visit.

Once parts are machined, printed, or arrive in the lab, the next step is putting them together into larger assemblies. That is the assembly floor.

Here engineers can take various parts and ensure proper fit. There are also bins of commonly used parts to aid in the quick assembly of prototypes.

CoolIT can also do things like run initial batches of liquid as well as gas like nitrogen through loops to ensure everything is properly sealed and seated.

Something that was interesting, that also makes a lot of sense is that I saw these acrylic carriers. These solutions often need to be installed across several cold plates with precisely cut tubing to fit. As a result, these carriers ensure that the entire assemblies can be transported without putting too much stress on the tubing. This makes sense, but I had no idea prior to visiting the lab.

CoolIT has a number of different nodes, fittings, and components from different vendors.

One of them I immediately recognized from our SC21 coverage as a HPE Cray Shasta node that will power some of the US’s first exascale supercomputers (more on exascale later this week on STH.)

Once the smaller assemblies are built, the next step is often the temperature chambers.

In the lab, there are a number of different temperature chambers.

Each of these chambers has different capabilities.

The reason these test chambers are used is for things like simulating rapid heating and cooling cycles to ensure the parts and assemblies can handle rated loads. Often, the parts are taken beyond ratings to see failure points.

This is important because of the different environments that liquid cooling is deployed in.

Beyond that, there are non-operational environments like being shipped over air freight that need to be simulated to ensure problems do not arise during shipping. These chambers all run accelerated testing cycles to get data faster than simply putting units in the field and observing.

The integration area is where loops, CDUs, manifolds and other components come together. Here the parts are assembled. Then the secondary cooling loop (the one that goes to the racks) can be filled with different liquids and additives. This allows testing to ensure that one additive does not create corrosion, or that there are no adverse effects using a liquid and additives along with all of the components in a loop.

There is also a container-based data center simulator to test larger environments. You can see those in the video.

While a CDU (Coolant Distribution Unit) may seem like it only has a few primary components, they have evolved. Here is a 40kW CoolIT CHx40 CDU that is an old model.

Here is a CHx200, a 200kW unit that is a more modern example. One can see just how much more is going on.

Now, CDUs are much larger, and can be in the Megawatt scale. This unit is around 1MW, but CoolIT has larger units as well.

In order to achieve this scaling, CoolIT needs to test every part, and that is why the Liquid Lab is important.

One of the biggest questions we get on liquid cooling stories is around reliability and leaks. Hopefully this tour of the Liquid Lab helps explain what is being done to test and address those challenges. The next generation of high-performance and high-density solutions will require liquid cooling, and so this is important work.

I also just wanted to say thank you to the CoolIT folks for hosting STH. There were a few places we could not show because of the prototyping going on in a working lab, but this was a great look at a lot of the process.

For those wondering, these nodes are HEAVY!

This seems like some impressive level of engineering…But I just can’t get over the amount of what must be the cost versus the old forced-air method of racks in a data hall, or the horizontal vats of mineral oil (immersion cooling) that I experienced starting in 2017/2018.

Besides the high upfront costs, years of hoping 1,000s of connections never leak, seems like an enormous risk to take on.

…Let’s hope the “wave” of the future is a metaphor and not part of a data center incident report.

Please use the Shasta pic for your profile. Looks good.

Save my name, email, and website in this browser for the next time I comment.

Sign me up for the STH newsletter!

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Get the best of STH delivered weekly to your inbox. We are going to curate a selection of the best posts from STH each week and deliver them directly to you.

By opting-in you agree to have us send you our newsletter. We are using a third party service to manage subscriptions so you can unsubscribe at any time.