I said, 'I think you have the wrong person,'" Morgan tells PEOPLE exclusively. "I live on a farm, and I kill zombies, and I'm a dad, and I'm kind of a pain in the a--."Throughout his decades-long career as a successful actor,

Any Walking Dead fan will tell you, if there is one guy you don’t want to piss off, it is the dude known for carrying around the barbed wire-covered baseball bat. But that's exactly what the cast of Destination X did when they got on host Jeffrey Dean Morgan ’s bad side at one point during filming.

The Walking Dead star had reservations about hosting NBC's new competition series, but he had a little help from someone close to him.