In your httpd.conf file you should have a Directory block for your root web directory. Mine looks like this after I removed a lot of comments (your's might be slightly different):
<Directory "C:/Program Files/Apache Group/Apache2/htdocs">
Options Indexes Includes
Order allow,deny
Allow from all
</Directory>
You want to delete "Indexes" from the first line within the block so you now have:
<Directory "C:/Program Files/Apache Group/Apache2/htdocs">
Options Includes
Order allow,deny
Allow from all
</Directory>
Then I think you have to restart apache.
If there are other directories that you want to browse such as the /icons then do the same for those blocks.