[embee]

Once upon a time I rode all year round (not any more, thanks). Fundamental thing is to keep the core warm, as said by others. If your body senses it's getting cold the first thing it will do is cut blood flow to the extremities. Arms are next, I'd suggest, you won't keep hands warm if your arms are getting cold.
I found a waterproof outer over a decent jacket helped a lot, just keeping the direct wind off works wonders. A good quality merino wool undergarment is a great help too.