I think what you are looking for is in this tutorial:
Basically, those pages are just using IFRAMES. These are just little "boxes" that act as frames. Essentially, you just make this IFRAME using code something like this:
<iframe src="yourpage.htm" width="500" height="200" border="0" frameborder="0" name="yourframe"></iframe>
Of course you need to costumize that code to your needs. (i.e. your width and height and all that good stuff.) To make a link open a page in that "box", you just target the link there like so:
<a href="here.html" target="yourframename">LINK</a>
That should do it for the basics. Of course it is more complex, but I am not into complex things too much!!!