That's because he's so passionate about the subject matter.
I gave up studying physics in secondary school because my teacher was a disaster. Instead of trying to impart a deeper understanding of the subject matter in question, he would focus on getting student to learn off scientific definitions word for word (i.e. having all the "ands", "ats", "is'", "the's" in the correct place). So you would have a textbook answer memorized for the test (perhaps), but sweet fuck all actual knowledge of physics.
To see this kind of passion is refreshing, and the way things should be. (If you've ever seen 'The Wire', seasons 4 & 5 have 'Mr. Prezbo' teaching maths. He struggles to connect with his student until he teaches them how to apply probability theory to dice games, allowing them to gamble more effectively. Unfortunately for him, he also has to 'teach to the test' so that the school can say standardised test scores are going up).
When I was studying for my degree, my favourite course was "Digital Signal Processing" and I'm pretty sure it was because the lecturer was genuinely interested and enthusiastic about the subject. Enthusiasm is very contagious and hard to resist. I think everyone in that course liked it and found it easy to learn, just because we had a teacher who was really enthusiastic about what he was teaching. If only everyone was like this...
I agree it's the passion. I had a chemistry teacher in highschool that was equally as passionate about chemistry as Feynman is here and it really helped me learn the subject.