Do whatever you prefer, there's probably an equal split. I'd say trim because you've probably altered yourself, subconsciously, to suit your beard, so shaving it might be a little bit of a shock to your appearance. But that could work, dynamics and stuff.
You could do what my father does with his hair. He gets it cut real short, and lets it grow long, and then cuts it again. This cycle lasts about a year. Says "End of the year, your a totally different person. Then you go right back again".
I like beards but am not a fan of a hairy body. My fiancee has a long mustache goatee thing (all grey lol) that I think is incredibly sexy. He got a lot of negative feedback from a lot of his woman friends though, which might mean I am alone in my opinion. But who cares! I'm the one with him!