According to
this your machine should be good for up to 1080p, even with H.264 (though not necessarily all H.264 files). What was in the MP4 container? Were you running other apps at the same time (such as a web browser with Flash)? Driving a very high-resolution screen (or two)? What media player did you use to play it?