As in little/no makeup, t-shirts & jeans, hoodies?
Most Helpful Girl
Guys don't really notice whether you look stylish or not. (Except for guys who probably wouldn't be getting with you anyway.) What they notice mainly is how good your face and body look.
If you want to dress casually, then finding flattering clothes should be easy. Just figure out what colors look good with your natural coloring (there are 5 million places talking about this on the internet) and the cuts and shapes and necklines or whatever that look good on your figure (again, 5 million places). You will look attractive without much more than careful shopping.
As for makeup, guys like less makeup, as you've probably seen. You don't need foundation and stuff. Eye makeup, if put on well, can help, but you don't need it.
So you can look great casual, if it really is your style. If you don't put thought into finding stuff, even shirts and hoodies, that look good on you and sort of express who you are, you'll find out that you don't really feel confident about yourself. And some people don't have that style at all, and think they do. Besides, it's not like the only options are casual and boob-tube-high-heel--you could find out that you really like boho skirts, or fluffy romantic dresses.
So, basically, no guy is going to mind, as long as you can 'pull it off,' as Bruticus Maximus says. But that's largely a matter of finding colors and cuts that look good on you, and being sure that casual is your style.1