You'll probably be feeling the cold when you stop from the sweat. Keeping warm is all about breathability. I've been wearing my mountainbiking gear under my textiles, in particular this top
http://www.wiggle.co.uk/p/Cycle/7/dhb_Compton_Zip_Neck_L~S_Base_Layer/5360031535/