Rhythm and harmony are two entirely different things.
Rhythm is the "accent of the beat" so to speak, and is applied to the timing structure of the song, ie., shuffle, swing, march, waltz, etc. These give various rhythmic "flavors" to the style of the song, in a given time signature.
Harmony is the blending of tones, usually from different instruments or voices to produce "Chords", that have a tonal structure comrprised of several fundamental tones. These give tonal "flavors" and "body" to the song.
As far as importance goes, from a music theory viewpoint, there must be a rhythm, or it is not a song. So that makes rhythm quite important. On the other hand, harmony is not required in a song, and is simply part of the tone pallette. Simple one-note melodies like nursery rhymes do not have any harmony. They are simplistic. Choral works are masterpieces of harmony.
It is all a personal thing about what "flavors" you like in your music, and I like most of them.
I think that the Beach Boys had some beautiful pop harmonies. But I don't really like Barbershop Quartet harmonies, even though the Barbershop Quartets are usually perfectly constructed. Personal taste.
I agree with Detlof that the rhythm is something that most people just "feel" in their body. They don't necessarily identify it as a particulart style, because most people wouldn't be able to tell the difference between a "swing" and a "shuffle" anyway. Most people can't even identify a 3/4 time from a 4/4 time, but that doesn't stop them from enjoying it.
Music theory is something I study because I like music, and want to know more about it, and how it is made. I learned to play instruments, and studied theory, because I wanted to be able to construct my own music(DIY again!) instead of just playing what others had written. It is a fairly deep subject and mathematically complex, at times.
Long live music!