Probably the easiest way to force IE to use the doctype that you set, or to prevent IE from defaulting to "quirks" mode would be to simply make your doctype Strict instead of Transitional.
If you use a Strict doctype then a.) your complying with current web standards b.) IE7 and FF should render pages almost identically. c.) it will prevent IE from reverting to quirks mode which could otherwise mess up layouts.
If you would like an example I am in the process of switching over to valid Strict doctype for my pro-bono website. Try to view it in both FF and IE7 and it should not faulter to quirks and the layouts stay the same.
Here is the site test page HERE
hope that helps some,