MYC, Ive used the muffs alone for the last two winters and ridden only in my summer gloves. I only remember one occasion when my hands got too cold, but it was a seriously cold day with ice on the road. After a couple of trips id forget the muffs were there and never had a problem with them interfering with the controls.
This winter I have heated grips too!!