This four pages Ars Technica article explains why "Why YouTube buffers: The secret deals that make--and break--online video -- When internet service providers (ISPs) and video providers fight over money, Internet users suffer.
It is not just YouTube too! :(