Yes, that makes sense. Thanks for your explanation. I tried with 96MB DVMT, but I continue to have issues with HDMI. The TB3 -> DP works fine at 4K@60 and I couldn't ask for more.
When I connect the HDMI cable, the display still shows the default resolution as 2560x1440, though I can see the 4K resolution in the scaled resolution list. Once I select 4K from there, the external monitor goes black and does not come back, unless I reboot.