I think that's perfectly fine. From a style stand point it can be viewed different ways though. If it is clean, tailored well, and looks adult then you'll look suave but it can look a little rag tag or bland.
I don't think dressing like that is bad though because I'd hope someone doesn't spend all their time on clothing anyway..and think like Dr. Ian Malcolm..life is too short to wonder what you're going to wear
My boyfriend wears clothes like this and I think it makes him look more attractive cause doesn't ditract from his natural looks, rather brings them out. Especially black or white tops look best. As long as they are good quality and clean, brings an air that you look after yourself but aren't flashy about it :)