My only experience of the memory foam was at a B & B - and you are correct, it was uncomfortably hot, even when the room was not heated.
My friend bought one with the 'topper' and finds it very uncomfortable.
Another hint - make sure whatever you buy can be turned as some these days are only one-sided.
After our B & B experience I remembered our Mum putting a layer of foam under our sheets to keep us warm (we had a poor childhood) so it must all hold the body heat.
Look forward to comments from others.