I have a table that is over 400 MB in size and has about 150,000 records. Queries are getting steadily slower as the table grows which is slowing the site down.

The server itself has 4 GB of RAM available to it.

Anyone have any suggestions for setting the following variables?
sort_buffer_size
read_buffer_size
read_rnd_buffer_size
join_buffer_size

I have heard that larger sizes on these is not always better and can even slow the queries down so I am trying for figure out what would be the best values based upon this very large table as it takes the bulk of the queries from the site.