This has stumped me for well over 2 days, and I can't find a solution, and it's become very frustrating.
The MP4 file is about 6MBs in size, WEBM is about 9MBs, and OGV is about 13MBs. It seems to load fine for the most part in browsers, but on my iPhone, it loads incredibly slow, to the point where you have continuously hit the play button ever 0.5 seconds or wait 5-10 minutes for the whole minute and a half video to load. This is not right, obviously, but I don't know what to do. I've converted it many times with different settings and downsized it to the max which leads me to believe the problem isn't the size, but something else.
The current MP4 settings are:
Mime: video/mp4 (MPEG-4)
Video Codec: H.264 (x264)
v264 Tone: none
H.264 Profile: high
H.264 Level: 3.1
(I just copied them from HandBrake, the converter I'm using; I don't know what half that means).
Nothing seems to fix the problem though. The settings I'm using are the program's iPhone/iPod Touch compatible preset.