I'd be tempted to agree that Button needs to win the WDC to silence some of his doubters. I'd also say his poor qualifying record does not work in his favour. He is certainly a great driver though, and deserves to be remembered as such, not as the Villeneuve of the modern era some claim him to be.
However, the other thing I would say is people putting Hamilton in this "elite" category is contentious. Certainly his attitude since a disappointing 2009 season has been poor; and his lack of discipline this season alarming. He definitely has the pace, that's not in dispute, but a truly elite driver, in my view, requires discipline and attitude about such things. (Note that Alonso, Germany last year aside, has been a lot less outspoken these last couple of years)