I clear mine at least once a month and have found the system, with other tools, running quite smooth... Highly recomend NOT selecting "Passwords" as you'll loose them!!
From the Authors:
Every web browser stores a cache of the websites you visit (e.g. HTML pages, images) to reduce bandwidth usage and server load. This is called the browser's cache. Clearing it occasionally can make some websites run smoother and free up space on your computers hard drive if it's running low. Since every browser is different, the following sections will outline the steps involved in clearing the cache for several popular web browsers. February 27th is National Clear your Browser's Cache day; although you should do it more than once per year.