Ray Charles - "Rap is not music"

I listened to a Rap station this morning on ride in and I apologize for the no harmony comment in my first post. Most did have some form of harmony,  even if it was only a one cord song with no changes.

Rap to me is poetry to rhythms with little if any harmony, so IMO, I agree with Ray, but does it matter what category it's in? 

Honest Question. Why is it insulting to consider Rap poetry instesd of music? As others have posted, many non Rap artist have used spoken word, I considered most Dylan to be poetry.