My husband always said he didn't mind where my appetite came from, as long as I came home for dinner - in regards to flirtation and admiration. Humans were not originally wired to be with one person (thanks discovery channel shows on cavemen and tribe societies), so it's healthy and it's natural to be attracted to others. Remind yourself that they only see in you what your wife does. I don't think I need to go into whether or not you should act on it.
