Yes, fleece. Perhaps being an older demographic, CT posters are wool devotees. It's what we had in the day. Or perhaps it is because they are wiser--that comes with age, too. But fleece is great and I prefer it, because it drains and dries very quickly, and provides warmth even when wet. I mentioned my January tip-out in another thread. My pants that day were nylon shell with poly lining and I was very pleased at how warm I stayed while wearing those pants, that had been entirely submerged, for about forty minutes.
I became a fleece devotee before I became a regular paddler. Used to do a lot of skiing, cycling and running, often in the rain. I found a rain coat caused sweat to accumulate inside the jacket (I can sweat faster than goretex can breath), so I'd just wear fleece as an outer layer. My polar-tec fleece (is there really a difference?) seemed to keep the water on the outside of the garment, kept me warm when wet, and was dry almost as soon as the rain let up. I think it is also easier to launder than wool, and moths don't eat holes in it.
Presently have a pant from REI, probably the forerunner of
their current fleece pant (which does not claim to be polar-tec). If it was just medium cold, I wore the fleece pant under goretex rain pant or drysuit. If colder, I'd wear some poly long johns under the fleece pant. I find the REI fleece pant awkward when peeing from the drysuit, because it doesn't have an opening on the front. So at the end of last winter, I went and shopped for fleece lounge pants/pajama bottoms. Careful, most are cotton flannel, but you can find versions in wool and poly. These pants are now my standard cool weather pant for under the drysuit, keep me warm and are much easier when nature calls. I haven't been in them when they are wet, but expect they'd perform like the poly lining of the pants I recently swam in. So, keep your eye peeled for end-of-winter close outs. Nice fluffy lounge pants will be very reasonably priced.