[ExI] [Beowulf] oil immersion cooled blades

[Eugen Leitl]

----- Forwarded message from "Lux, Jim (337C)" <james.p.lux at jpl.nasa.gov> -----

From: "Lux, Jim (337C)" <james.p.lux at jpl.nasa.gov>
Date: Wed, 14 Mar 2012 20:12:47 -0700
To: Mark Hahn <hahn at mcmaster.ca>,
	"beowulf at beowulf.org" <Beowulf at beowulf.org>
Subject: Re: [Beowulf] oil immersion cooled blades
user-agent: Microsoft-MacOutlook/14.12.0.110505

Immersing motherboards in oil has been done for years (probably even
mentioned on this list).  There's countless hacker versions using various
and sundry off the shelf mobos and power supplies etc in various and
sundry containers (ice chests, aquariums, purpose built acrylic boxes)
using a variety of coolants.



The vast majority of these have no "thermal engineering".. They're more
"we tried it and it seemed to work".  There's not a lot of content at the
hardcore computer website. It could be ok or it might not.  They're not
real clear on how they actually get the heat out of the enclosures. The
MSDS for their coolant says that it's basically synthetic mineral oil with
a antioxidant (you can look up the CAS number to get more info). I'd be a
bit nervous about the fire hazard aspects of a machine room full of the
stuff.

So here's my experience using oil as an insulator/coolant.
1) it wicks up insulated wire, particularly stranded.  Put the mobo in oil
and the power supply outside, and pretty soon your power supply will be
full of oil.
2) oil leaks.  There is *nothing* that is oil insulated that doesn't have
a fine film of oil on its surface eventually, unless it is in a
hermetically sealed can with welded/crimped seals.
3) oil is a mess when you need to fix something



The Cray-2 used Fluorinert(tm) FC-74 as the coolant, which is very nice to
work with, although expensive.
It doesn't wet things very well, so when you pull something out of the
bath, it doesn't bring much fluid with it. The Cray used it as a heat
transfer medium to water coolant.  I think they had a way to drain it into
a tank quickly for servicing.


 It can be used for ebullient (boiling) cooling by picking the right vapor
pressure/BP grade (the Cray didn't use this). Ebullient cooling is quite
efficient at moving the heat away because it's a phase change, and the
bubbling causes good circulation, but it does require careful design so
you don't get film boiling/Leidenfrost effect (the phenomenon that
protects your feet when walking across burning coals barefoot)



On 3/14/12 3:02 PM, "Mark Hahn" <hahn at mcmaster.ca> wrote:

>> Server blades in oil
>
>sounds like some brutalist take on molecular gastronomy!
>
>> Hardcore Computer LSS 200
>
>http://www.hardcorecomputer.com/Resources/assets/Documents/LSS-specs.pdf
>
>seems kind of uninspiring, in hardware specs.  1366 socket, not 2011,
>optional fast network, gpu, etc, builtin ipmi.  8x in 5U, so not really
>a density play.
>
>I guess I'm a bit skeptical about the utility of this approach -
>would be nice if they had some technical literature.  something about
>thermal resistance.  define how the oil bath dumps the heat (water
>hookups in the back?)  comparison to modern heatpipe-based solutions, etc.
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>To change your subscription (digest mode or unsubscribe) visit
>http://www.beowulf.org/mailman/listinfo/beowulf

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE