التبريد بالغمر

Immersion cooling, also known as "direct liquid cooling", is a technique used for computer cooling, battery cooling [1], and motor cooling in which electrical and electronic components, including complete servers and storage devices,[1] are mostly or fully submerged in a thermally conductive but electrically insulating liquid coolant. Heat is removed from a system by putting the coolant in direct contact with hot components, and circulating the heated liquid through heat exchangers. This practice is highly effective because liquid coolants can absorb more heat from the system, and are more easily circulated through the system, than air.

Unlike many other devices, computers cannot use water cooling, as water is electrically conductive and will break electronic components. Therefore, the fluids used in immersion cooling are dielectric liquids to ensure that they can safely come into contact with energized electronic components.

Example of an "immersed computing" system
An immersion-cooled system
Example of server immersion cooling of one server

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Dielectric liquids

In general, the dielectric liquids used for immersion cooling fall into two categories: hydrocarbons (i.e. mineral, synthetic, or biological oils) and fluorocarbons (fully engineered liquids). Dielectric liquids are divided into single- and two-phase applications, which differ in whether or not the cooling fluid turns into a gas during the cooling cycle.

  • Single-phase immersion uses a circulation method for the dielectric liquid across hot electronic components and to a heat exchanging approach.
  • Two-phase immersion uses fluorocarbons which boil at low temperatures, transferring heat from the components as a gas. This gas is recovered, condensed by a heat exchanger, and returned to the components.


Forms

An enclosed chassis require dripless connectors to interface to the individual chassis. These chassis are usually based on traditional rack style implementations. The dripless connectors usually require a small closed circuit cooling loop with a coolant to protect the flow integrity through relatively small pipes and connectors. The closed circuit is facilitated by a CDU or Coolant Distribution Unit, which usually can facilitate multiple racks at once.

An open bath refers to the “open” liquid-air interface and thus surface tension between the liquid and the air being a distinctive element. Open bath systems are usually tanks which contain a larger body of dielectric liquid where electronics are immersed into the bath. Multiple electronic assemblies are sharing the same liquid. This liquid can be based on single- or two-phase technology. Regardless of the term, open bath systems can be fully sealed, but are always opened from the top to service IT equipment. The coolant tank for open bath immersion systems is either connected to a CDU which circulates the dielectric liquid, or to an integrated heat exchanging device which is part of the tank. For a facility interface, CDU's are usually designed for 100 kW or more, whereas an integrated heat exchanging device is usually designed for 10-100 kW cooling capacity.

Hybrid cooling refers to combinations of enclosed and open bath apparatus.[2]

Evolution

Immersion cooling reduces energy consumption through the elimination of the air cooling infrastructure including on-board server fans, CRACs, A/C compressors, air-circulation fans, necessary duct work, air handlers, and other active ancillary systems such as dehumidifiers. These systems are replaced with liquid circulation pumps and heat exchanger and/or dry cooler systems.

Power use at data centers is often measured in terms of Power Usage Effectiveness (PUE). The definitions of PUE for air-cooled devices and liquid immersion cooled devices are different which makes such direct comparisons inaccurate. The PUE for air-cooled data centers includes the power used by the fans and other active cooling components found in the servers. The PUE for liquid immersion cooling excludes these values from the IT Equipment Energy component because these system elements (in particular on board fans) are generally removed from the IT equipment as they are not necessary to circulate the dielectric coolants. This discrepancy in the definition of PUE for the different cooling methods results in the PUE of air-cooled data centers generally being overstated when compared against the PUE of a liquid immersion cooled facility of the same power usage.[3]

Servers and other IT hardware cooled by immersion cooling do not require fans to circulate the dielectric liquid, thus they are removed from the system prior to immersion. Thermal pastes which are typically used on heat spreaders for CPUs and other chips may require replacement with a different compound in order to avoid the thermal degradation within the dielectric liquid.[4] Depending on the type of application, solder, Indium foil and thermally conductive epoxies may be used as a replacement materials.

Network router and smart-phone immersed in synthetic single-phase, liquid immersion coolant
Network router and smart-phone immersed in synthetic single-phase, liquid immersion coolant

The temperatures used in immersion cooling are determined by the highest temperature at which the devices being immersed can reliably operate. For servers this temperature range is typically between 15 and 65 °C (59 and 149 °F);[5] however, in ASIC-based crypto mining devices, this range is often extended up to 75C.[6] This increase in the high end of the temperature range allows data centers operators to use entirely passive dry coolers, or much more efficient evaporative or adiabatic cooling towers[7] instead of chiller-based air cooling or water chillers. This increase in the temperature range also allows operators using single-phase immersion coolants to more effectively use the change in outdoor temperatures to get more efficient cooling from their systems because the single-phase systems are not limited in their effectiveness by the boiling point of the coolant as is the case with two-phase coolants.[8]

Multiple relevant brands like Intel and Facebook have already validated the advantages of submerging servers.[9][10]

Current commercial applications for immersion cooling range from datacenter oriented solutions for commodity server cooling,[11][12] server clusters, HPCC applications[13] and Cryptocurrency mining.[14] and mainstream cloud-based and web hosting architectures. Electric vehicle and battery manufacturers also employ liquid immersion cooling in batteries, drive-train, Kinetic energy recovery systems, electric motors, electric motor controllers, and other on-board electronic sub systems[بحاجة لمصدر]. Liquid immersion cooling is also used in the thermal management of LEDs, Lasers, X-Ray machines, and Magnetic Resonance Imaging devices[بحاجة لمصدر].

Immersion cooling is applied to electronic components in deep sea research where Remotely Operated Underwater Vehicles with electronic equipment are filled with Single-phase Liquid Dielectrics to both protect them from corrosion in seawater and as a pressure compensating fluid to avoid being crushed by the extreme pressure exerted on the ROV while working in the deep sea[بحاجة لمصدر]. This application also includes the cooling of the electric motors used for under sea propulsion.

Until about 2014, the technology was typically only utilized in special very intensive supercomputing projects, like the Cray Computer Applications.[15] Even though the expected increase in global energy consumption by data centers has remained steady,[16] there is an increased focus on energy efficiency which has driven the utilizing of liquid immersion cooling in both data centers and crypto mining operations to reevaluate its application. The advent of new very high density CPUs and GPUs for use in real-time processing, Artificial intelligence, machine learning, and data mining operations is resulting in users and data center operators to evaluate liquid immersion cooling for ability to cool high density racks as well as reduce the total mechanical footprint of data centers.

History

19th and 20th century immersion milestones:

  • Immersing electric systems (specifically transformers) in dielectric fluids for thermal management was used before 1887.[17]
  • The first patent to explicitly mention the use of oil as a coolant and insulator is in the patent submitted for a Constant Current Transformer in 1899 by Richard Fleming of Lynn, Massachusetts, assignor to the General Electric Company of New York[18]
  • The first reference to the specific use of dielectric fluids being used to cool "computers" is in 1966 by Oktay Sevgin of IBM.[19]
  • In 1968, Richard C. Chu and John H. Seely, working for IBM patented an "Immersion cooling system for modularly packaged components."[20]
  • Seymour R. Cray Jr. founder of Cray Research, LLC patented a "Immersion cooled high density electronic assembly" in 1982.[21]
  • The Cray T90 (released in 1995) used large liquid-to-chilled liquid heat exchangers and single or two-phase immersion cooling liquids for heat removal[22]
  • Due to the arrival of CMOS, significant energy savings were achieved in CPU's which rapidly reduced the cooling challenges of HPC systems. It was not until the second decade that immersion regained traction due to increasing thermal properties of chips.

21st century immersion milestones:

  • In 2006, Hardcore Computer Inc was founded on the concept of introducing enclosed chassis style PC's for gaming.
  • In 2009, Green Revolution Cooling rebooted the open bath immersion cooling concept by bringing a commercial open bath immersion system to the HPC industry
  • In 2010, Midas Green Technologies ran and operated the first Immersion Cooling Data Center
  • In 2011, Iceotope launched the first commercial rack-style enclosed chassis based technology, specifically designed for datacenter deployments.
  • In 2016, Asperitas created the first pumpless natural convection circulated high density, single phase open bath immersion system.
  • Starting in 2016, the rise of crypto currency becomes a main and significant driving force behind immersion. This is due to the high TCO advantages which are highly valued in crypto mining. This period has allowed many immersion technologies to gain essential experience and mature their technologies.
  • 2017 shows a large volume of start-ups in the immersion cooling domain. Mostly related to cryptocurrency and the increasing power and cooling challenges in the datacenter industry.
  • In 2018, the Open Compute Project officially embraces immersion in a new project under Rack & Power as part of ACS (Advanced Cooling Solutions).
  • In 2019, the first documented industry standards for immersion are presented at the OCP summit in San Jose.
  • In 2020, Telecommunications Industry Association publishes their first mention of Immersion Cooling as a viable cooling option.
  • In 2021, as chips thermal properties have gone beyond air-cooling capabilities, various hyperscale cloud companies, chip manufacturers and server OEMs have announced their adoption of immersion cooling.
  • In 2022, Intel announces investment of 700 million USD[23] in a mega lab focused on immersion cooling and publishes a fluid specification for immersion cooling which will allow the support of warranty[24] of Intel products.

Server immersion cooling techniques

Open bath immersion cooling

Open bath immersion cooling is a data center cooling technique that implies fully submerging IT equipment in dielectric liquid. The “open” aspect does not refer to an open or sealed system, but refers to the “open” liquid-air interface and thus surface tension between the liquid and the air being a distinctive element.[25]

These baths allow the coolant fluid to be moved through the hardware components or servers submerged in it.[26]

Single-phase-immersion-cooling-diagram.png

Dual loop Single phase immersion requires circulation of the dielectric liquids by pumps or by natural convection flow.[27] These liquids always remain in liquid state while operating. They never boil or freeze. The dielectric coolant is either pumped through an external heat exchanger where it is cooled with any facility coolant, or the facility coolant is pumped through an immersed heat exchanger, which facilitates heat transfer within the dielectric liquid.

Natural convection circulation

In two-phase systems, fluorocarbons[28] are used as heat transfer fluids. Heat is removed in a two-phase system, where the liquid boils when it comes in contact with hot components due to its low boiling point.[29] The system takes advantage of a concept known as “latent heat” which is the heat (thermal energy) required to change the phase of a fluid, this occurs when the two-phase coolant comes in contact with the heated electronics in the bath that are above the coolants boiling point. Once the two-phase coolant enters its gas phase it must be cooled or condensed, typically through the use of water cooled coils placed in the top of the tank. Once condensed the two-phase coolant drips back into the primary cooling tank. The two-phase coolant in the tank generally remains at it “saturation temperature”. Energy transferred from the servers into the two-phase coolant will cause a portion of it to boil off into a gas. The gas rises above the liquid level where it contacts a condenser which is cooler than the saturation temperature. This causes the gaseous state coolant to condense back into a liquid form and fall back into the bath.[30]


. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Enclosed chassis immersion cooling

Sealed server immersion cooling encloses servers in liquid-tight casings. The dielectric coolant is circulated inside or pumped through each server to collect heat from the components. The heated fluid is circulated to a heat exchanger in the rack where it is either circulated directly outside the building to a cooling tower or to a heat exchanger or cooled directly at the rack with a facility coolant infrastructure.[31] The main advantage of this approach is that servers are mounted in self contained vessels that can be replaced in the rack without accessing the fluid. A disadvantage is that not all hardware can be used as the vendor defines the hardware specs of the sealed servers.

Fire hazards

Some hydrocarbon based immersion cooling fluids provide a fire hazard as they have a fire point.[32]

Other uses

Domestic or process heating

In the last few years[when?], immersion cooling in particular for bitcoin mining has become a popular method to generate usable heat. In cold climates a single ASIC miner can provide ultra high efficiency electric heat conversion sufficient to heat an entire home. Immersion cooling offered a means to silently convert the waste heat from the mining operation to heat water, melt snow, power in floor heating, heat hot tubs and pools, shops, outbuildings, sheds and greenhouses. There is a compelling case to combine bitcoin mining operations with indoor vertical farms and traditional greenhouses to offset or eliminate the heating cost of the facilities. Indoor and outdoor recreation facilities both public and private can also benefit from the free waste heat. There are some companies which provide some computing based heating for residential and commercial operations.[بحاجة لمصدر]

See also

References

  1. ^ "Five Reasons Data Center Liquid Cooling is on the Rise". 23 July 2018.
  2. ^ "OCP Immersion Requirements". Open Computer Project. Open Compute Project. Retrieved 22 January 2023.
  3. ^ "A Review on efficient thermal management of air- and liquid-cooled data centers: From chip to the cooling system". Applied Energy.
  4. ^ Pambudi, Nugroho Agung; Sarifudin, Alfan; Firdaus, Ridho Alfan; Ulfa, Desita Kamila; Gandidi, Indra Mamad; Romadhon, Rahmat (2022-12-01). "The immersion cooling technology: Current and future development in energy saving". Alexandria Engineering Journal (in الإنجليزية). 61 (12): 9509–9527. doi:10.1016/j.aej.2022.02.059. ISSN 1110-0168. S2CID 247823719.
  5. ^ Pambudi, Nugroho Agung; Sarifudin, Alfan; Firdaus, Ridho Alfan; Ulfa, Desita Kamila; Gandidi, Indra Mamad; Romadhon, Rahmat (2022-12-01). "The immersion cooling technology: Current and future development in energy saving". Alexandria Engineering Journal (in الإنجليزية). 61 (12): 9509–9527. doi:10.1016/j.aej.2022.02.059. ISSN 1110-0168. S2CID 247823719.
  6. ^ "Application-Specific Integrated Circuit (ASIC) Miner". Investopedia (in الإنجليزية). Retrieved 2022-06-21.
  7. ^ "Data center liquid immersion cooling with adiabatic cooling towers". Submer Technologies. January 2, 2016.
  8. ^ "The immersion cooling technology: Current and future development in energy saving". AEJ - Alexandria Engineering Journal.
  9. ^ "Ice X: Intel and SGI test full-immersion cooling for servers". Computerworld, Inc. April 8, 2014.
  10. ^ "Facebook throws servers on their back in HOT TUBS of OIL". The Register. October 14, 2013.
  11. ^ "Liquid immersion cooling relief for ultra-dense data centers". TechTarget. October 5, 2014.
  12. ^ "What is liquid immersion cooling? - Definition from WhatIs.com". WhatIs.com (in الإنجليزية الأمريكية). Retrieved 2017-07-25.
  13. ^ "Immersion Cooling Steps Up for HPC Clusters". insideHPC. May 7, 2014.
  14. ^ "BitFury to Launch Energy Efficient Immersion Cooling Data Center". Business Wire. December 11, 2015.
  15. ^ "Cray-2 Machines". 7 August 2021.
  16. ^ "Energy demand data centers globally by type 2021".
  17. ^ "Transformer's History and its Insulating Oil" (PDF).
  18. ^ "Constant Current Transformer" (PDF).
  19. ^ "Multi-liquid heat transfer".
  20. ^ "Immersion cooling system for modularly packaged components".
  21. ^ "Immersion cooled high density electronic assembly".
  22. ^ "Fluid Selection and Property Effects in Single and Two-Phase Immersion Cooling" (PDF). JOHN R. SAYLOR, AVRAM BAR-COHEN, SENIOR MEMBER, IEEE, TIEN-YU LEE, TERRY W. SIMON, WE1 TONG, AND PEY-SHEY WU. November 4, 1988.
  23. ^ "Intel Makes Key Investments to Advance Data Center Sustainability".
  24. ^ "Intel Technology Innovations and OCP". YouTube.
  25. ^ "Rack & Power/Advanced Cooling Solutions - OpenCompute". www.opencompute.org. Retrieved 2019-05-07.
  26. ^ "Electronics Take a Bath" (PDF). Lawrence Berkeley National Laboratory. November 5, 2014.
  27. ^ "Asperitas Immersed Computing". www.asperitas.com (in الإنجليزية الأمريكية). Rolf Brink. 2017-01-30. Retrieved 2017-07-24.
  28. ^ "3M™ Novec™ 7100 Engineered Fluid". © 3M.
  29. ^ "Immersion Cooling with 3M(TM) Novec(TM) Engineered Fluids". 3M. April 8, 2014.
  30. ^ "Immersion-2 Rack Platform (PUE 1.01)". AlliedControl. January 22, 2014.
  31. ^ "Targeted liquid cooling for a system". Rackspace. March 23, 2011.
  32. ^ NFPA 30 Flammable and Combustible Liquids Code. 2018. pp. A.4.2.4.