(LINKS BELOW) I usually dress kind of formal. In the winter I wear a button-up shirt, tie, wrist watch, dark colored jeans, dark brown leather shoes, and an argil sweater vest, or a plain/argil sweater. In summer, a wrist watch, a button-up shirt, a tie(sometimes, and not with shorts), dark colored jeans or kaki shorts, brown leather shoes or brown leather driving moccasins,( and when outside aviator sunglasses). The most casual I dress in jeans or shorts, with leather shoes/driving moccasins, and a button-up shirt. link (with tie) link
If it helps I'm 6 feet 2 inches tall, dark brown hair, blue eyes, fair skin, and am quite muscular.
From your description, you sound basically like the ideal guy as far as looks go, so keep it up! Just make sure all your clothes fit well, because a classy formal style only looks sexy if all your clothes fit your body exactly.
It's a nice look and it sounds like you can pull it off from your build.
It's not my favourite way to dress though - I quite like the ripped jeans and t-shirt look personally. Also, my ex dresses like this so it brings back a lot of memories!
I wouldn't ever make a guy think that he has to dress differently for that reason though. Dress how it makes you feel comfortable and if it suits you go with it! No point dressing a different way if you don't like it!