Does the first reflection point actually matter??

yes

Okay now you are questioning the holy grail. 1st reflection points.
I built 12each of  2' x 4' x 2" Owens Corning 703 absorbing panels.

In a room of 15' x 18' open to other rooms I deployed 2 panels on the ceiling, 2 behind the speakers, 2 for early reflections and 2 behind the listeners. Floor is carpeted.

The difference was so profound I nearly cried.

Now your post makes me question which panels are the most critical.

My sound was harsh and staging was poor. GIK 244 full range absorbers at my first reflection points made such a large improvement I could not believe my ears. I then bought numerous diffusers for the rear wall and directly to my left and right.  More absorbers are in the works. The first reflection points are key. 

Hi Erik,

So I have also had problems believing in the first reflection point.  Whatever I tried seemed to make no difference at all. I've tried many times in many different ways but mostly using thick fabric panels.

For years I've always had a nice thick rug in front of the speakers and a diffuser panel on the wall behind the speakers with a panel on either side.
The speakers were usually along the long wall of my rooms...as they were also part of the entertainment system (TV).

In my last house (the one before my current house), the chosen room would not support the diffuser panel because there was a pony wall (low wall) behind it and I decided to use the rug elsewhere instead.  The left speaker (Salk Songtower) was 10" from the pony wall and about 20" from the left wall.  The right speaker was also 10" from the back wall but there was no right wall.  The right wall didn't start until about 9' into the space.

Clearly not an ideal setup...but for the first time I had a dedicated room for the audio with the speakers on the short wall of a bigger room.

Sound was OK...not great...but sounded good enough for me.

My girlfriend at the time suggest I move the stack of records (3 stacks high) from the right wall (near the middle of the space) to the left wall nearer the audio rack.  Only problem was that it was very close to the left speaker (way too near the speaker to be a first reflection point) but I decided it made sense form a practical POV.  

That one change fueled my obsession with the room acoustics. Even though the right speaker had no wall the stacks of vinyl refocused the sound on the left speaker to a point that everything was more coherent...locked in.  It's as if the sound had more clarity, focus and precision.  BIG, HUGE improvement in creating a 'reality' to the music if that makes sense. 

Over the next 6 months the tweaks resulted in the 180 degree change to the other short wall, nothing between the speakers but a rug and the audio rack along the right wall. I finally understood what imaging/soundstage was about.

You've got to try everything...every position sounds different.  I tried so many things but now have a sense of how a rooms work acoustically. 
That experience has helped me set up my current home.  But it will never be like that room.  That room was perfect.  BTW, it also had an acoustic ceiling...not the soft kind.  It was the early concrete acoustic ceiling which has no asbestos.  Acoustic ceilings get a bad rap.  They're the best thing going.

The first reflection point results from a basic flaw in most speaker designs: flat baffles with front firing speakers. The real solution to your problem are omnidirectional speakers which much more closely emulate the natural propagation of sound and obviously obviate the whole topic.