I agree with just about every one of the comments and, furthermore, as a guy who wears flannel shirts about 4 out of 5 days when I'm not at work, it would suck to be judged so harshly for that.

I could tell you stories all day about long-gone relatives who became extremely wealthy and successful and had long, happy marriages that were not based whatsoever upon what either one was wearing.

That said, I'd probably wear a polo or a button-down cotton shirt if I ever went on another date.