No, that's fine. Preference is legit, everyone is entitled to that. Racist would be more glorifying the preference while belittling and seriously insulting other races in the process.
I know sometimes white people are afraid to speak about something like this due to fear of being labeled a racist. One of my really good friends is a blonde Norwegian-American girl... one of the most racially accepting, tolerant, amazing people I know. She says she would prefer to be with her own race and preferably a blond like herself, because she likes how it is in her family and finds it beautiful and familiar... though she finds people of all races attractive. Totally fair.
I would say it's racist. It's no different from the "No Blacks" signs that used to be all over town, discriminating against a particular group of people. You can prefer broccoli to asparagus because you know for sure that no matter how asparagus is seasoned or cooked it will taste the same. Human beings, distinctive individuals who's physical features and minds vary incredibly, are nowhere near the same. People from a certain race don't just come out the same every time.
I think sheeple trying to claim that "it's just your preference" isn't racist probably have discriminatory, prejudice thoughts of others deep down that they're too prideful and lazy to work on so instead they try to encourage the idea that 'preference' is okay and should be accepted by all.
Technically it's both. Your preferences are more or less inherent. Racism can be internalized through exposure to certain media and this informs some of your innate preferences. While I won't vilify someone who prefers to only date a certain race (or their own), I will suggest to them that they have some internalized racism to deal with.