DEC750 NVMe thermal pad?

[foxxx0]

Hey folks,

I recently received my DEC750 and noticed the NVMe temperature is a bit high for my taste.

Depending on the room/ambient temperature and the I/O load, the NVMe sits around 63-67°C in idle and easily goes up to 75°C+ under load.

The CPU on the other hand stays comparatively cool, sitting around 48-52°C at idle, with an enclosure surface temperature of ~39°C.

I wasn't able to find teardown pictures of the DEC750, only the DEC850 on reddit, which is based on a completely different PCB and enclosure. However, there it looked like the NVMe had no "direct" thermal contact with the enclosure. Apparently there is a recess milled into the enclosure top to make space for the NVMe but I didn't see any thermal pads there.

Comparing the NVMe temperature and the enclosure surface temperature leads me to believe that there is no direct thermal transfer between the two, only passive thermal radiation.

So before I open the DEC750 myself, I thought I'd just post this question here.
I'm kinda hoping I could just purchase a thermal pad of the necessary thickness (guessing somewhere between 0.5-3mm) to improve the NVMe temperature.


Thanks in advance!


Cheers,
foxxx0

[pfry]

[...]I'm kinda hoping I could just purchase a thermal pad of the necessary thickness (guessing somewhere between 0.5-3mm) to improve the NVMe temperature.[...]

Some things to consider:

• Thermal pad will be better than air;
• ...But contact/pressure is important, and M.2 boards are worthless/weak and generally unsupported;
• The controller is the major power consumer, so your focus should be on it;
• Be aware of component height - I have M.2 SSDs where the controller is .25mm lower than surrounding components, a critical but hard-to-see issue.

You'll want a dead soft pad. I'd recommend getting a few and seeing what you can work with. Ideally you can trial the thing (such that you can directly view contact) or take an impression (via, say, very soft clay) to try to see how applying pressure to the controller will flex the board. Some motherboards use medium foam supports under the board, but sourcing something like that might be tough. On one of my tiny ARMs I had to trial several pads and thicknesses to find one that didn't simply flex the board such that the controller surface was no longer parallel to the opposite surface, resulting in limited contact. Foam pads are garbage from a thermal standpoint, but may be better from a physical standpoint.

Good luck.

[patient0]

the NVMe sits around 63-67°C in idle

Are you using a RJ45 module in one of the 10G SFP+? The NVME sits behind the SFP+ ports and got warm/hot when I did test RJ45 modules. But it was only for testing in my case so I didn't install any pads and can't help with that. I think the gap between the NVMe and the enclosure is quite substantial but maybe that changed with later models. There was a cutout for the RAM stick and like a pedestal (whatever the correct english word would be) for the CPU, nothing else.

Motherboard on the shop website:
https://shop.opnsense.com/dec700-series-opnsense-desktop-security-appliance/
https://shop.opnsense.com/wp-content/uploads/2021/11/NetBoard-A10_Gen.3_P1.0-CN_2021-Nov-08_03-52-59PM-000_CustomizedView15531474162.png

I'll upload a few of the pictures I took (some time ago) and update this post.

Update: four photos under https://kupper.org/OPNsense-DEC740/ . Open the images in a separate tab/window for the bigger size. And I'm very much not a web guy, so sorry about that.

[foxxx0]

Are you using a RJ45 module in one of the 10G SFP+? The NVME sits behind the SFP+ ports and got warm/hot when I did test RJ45 modules.

Nope, just a passive DAC SFP+ cable, so pretty much as low-power as you can go.
I've managed to completely stay away from those SFP+ RJ45 modules, precisely because of that.

I think the gap between the NVMe and the enclosure is quite substantial but maybe that changed with later models. There was a cutout for the RAM stick and like a pedestal (whatever the correct english word would be) for the CPU, nothing else.

Motherboard on the shop website:
https://shop.opnsense.com/dec700-series-opnsense-desktop-security-appliance/
https://shop.opnsense.com/wp-content/uploads/2021/11/NetBoard-A10_Gen.3_P1.0-CN_2021-Nov-08_03-52-59PM-000_CustomizedView15531474162.png

Update: four photos under https://kupper.org/OPNsense-DEC740/ . Open the images in a separate tab/window for the bigger size. And I'm very much not a web guy, so sorry about that.

Thank you so much for the detailed high-res photos. That finally gives some insight.

Based on these I might try to fit a thin m.2 heatsink onto the NVMe, I've got a couple different ones lying around somewhere^tm. The gap does indeed look way to big for just a thermal pad.

At least I have something to go off of and if I do indeed find a different case-heatsink-design in my DEC750 I'll just have to adapt.

[...]I'm kinda hoping I could just purchase a thermal pad of the necessary thickness (guessing somewhere between 0.5-3mm) to improve the NVMe temperature.[...]

Some things to consider:

• Thermal pad will be better than air;
• ...But contact/pressure is important, and M.2 boards are worthless/weak and generally unsupported;
• The controller is the major power consumer, so your focus should be on it;
• Be aware of component height - I have M.2 SSDs where the controller is .25mm lower than surrounding components, a critical but hard-to-see issue.

You'll want a dead soft pad. I'd recommend getting a few and seeing what you can work with. Ideally you can trial the thing (such that you can directly view contact) or take an impression (via, say, very soft clay) to try to see how applying pressure to the controller will flex the board. Some motherboards use medium foam supports under the board, but sourcing something like that might be tough. On one of my tiny ARMs I had to trial several pads and thicknesses to find one that didn't simply flex the board such that the controller surface was no longer parallel to the opposite surface, resulting in limited contact. Foam pads are garbage from a thermal standpoint, but may be better from a physical standpoint.

Good hints, thanks for that. If necessary I'll try to fit some (foam) padding underneath the NVMe to prevent its PCB from bending (i.e. between the NVMe and the mainboard).
Given the pictures above, I might have to just go for a low-profile off the shelf m.2 heatsink instead though.

[OPNenthu]

These videos show a DEC740 taken apart (same channel):

https://www.youtube.com/watch?v=v86V-E70RKA
https://www.youtube.com/watch?v=jl23_mrM7cQ

Not sure if it's the same dimensions exactly as the 750 but at least some more angles you can look at.

[patient0]

Not sure if it's the same dimensions exactly as the 750 but at least some more angles you can look at.

The DEC750 is identical but with 8GB RAM instead of 4 and 256GB NVME instead of 128GB. At some point the DEC750 had 3 2.5GbE ports while the DEC740 still got 3 1GbE ports (but wasn't available in the store). But now they both got 3 2.5GbE ports.

[foxxx0]

Alright, I finally managed to tackle this project.

There is quite a bit of space between the NVMe and the chassis-heatsink, so I was able to successfully fit a Aqua Computer kryoM.2 micro heatsink onto the NVMe with about ~2mm space left.
This heatsink including the thermal pad adds around 6-7mm of height, so in total you probably have about 9mm "usable" space between the NVMe and the chassis-heatsink.

Unfortunately the chassis-heatsink only partially covers the SFP+ cages, but I still added some thermal pads while I had the unit open anyways.
For this you'll want 3mm soft thermal pads. I had some 1.5mm Arctic TP-3 lying around, which you are allowed to stack, according to its manual.
The space between the SFP+ cages and the chassis-heatsink is a tiny bit less than 3mm, hence the need for soft thermal pads, that squish easily.
I can neither provide you with measurements nor tell you, if this made any positive or negative impact whats-o-ever, since I'm currently using only one of the SFP+ ports with a passive DAC cable, which is among the lowest power/heat connections you can possibly plug into there.

The factory thermal paste on the CPU was rather dry and didn't even fully cover the whole CPU-Die.
Since I didn't want to open this unit again for the foreseeable future, I went with a Thermal Grizzly PhaseSheet PTM (based on PTM7950) phase-change-pad in the hopes that this would pretty much last for a lifetime with no pump-out, drying up or anything of this nature.

The pictures below only show the NVMe heatsink as I did not take additional ones after applying the thermal pads to the CPU and SFP+ cages.

After booting up the unit I was greeted with improved temperatures for both the NVMe and the CPU, but I'm still trying to "burn-in" the PTM7950 phase-change-pad on the CPU, which apparently only reaches peak performance after 10 heating cycles above 60°C.
However, I'm really struggling to achieve this now.

I've been running

Code Select Expand

stress -c 8for around 30 minutes non-stop and have just reached a mere 56°C for Tctl according to hwmon/k10temp.

Before the swap, my CPU idle temperature was between 47°C - 51°C depending on my room temperature.
The NVMe was sitting typically anywhere between 59°C - 65°C and quickly peaked to 80-85°C load e.g. during system/package updates, which I found a bit too warm for comfort as mentioned in my opening post.

Now, even with the CPU running at full tilt for over 30 minutes and heat-soaking the chassis, the NVMe is at 56°C.

I'll post an update in a couple of days after a longer observation window.

[BrandyWine]

But contact/pressure is important, and M.2 boards are worthless/weak and generally unsupported;

In some setups I have done, not specifically for an M.2 board, but to gain extra support I would build a 1-2 "towers" of 100% silicone under the board in area that needed supporting. Let that cure for ~12-24hrs, then proceed with the contact/pressure. The silicone is stiff enough to give some support, yet still yields enough.

foxx0 points out what I would call design flaws. Doing some DIY as-needed to make it better, is the right way.

As for temps when using stress, cat the temps in /sys/class/thermal/[zone]/temp , i find this to be more accurate.