Use the frequency response graph as a preliminary filter (kind of like tossing away candidates that didn’t go to college when reviewing résumés), but don’t ever buy speakers from a store or seller that doesn’t have a decent return policy and/or allow at-home auditioning of equipment because the room you set the speakers up in is equally important as the natural response curve of the speakers because the acoustics of the room will heavily dictate which speakers work out for you and which ones won’t.
-Ed
PS On a related, but separate note, I wouldn’t really hesitate to rely heavily on a frequency response curve if researching in-ear-monitors or headphones. Totally different situation there, and part of why I consider headphones a lot easier to pin down.