Maineiac, here are some recommendations to pick through as starting points for beginning your exploratation of classical music. Ive tried to list music that is both accessible and mainstream, and to recommend recordings that are both readily available and have excellent sound quality. This is not a definitive list by any stretch, but it can help you get started and it covers a wide variety musical genres and styles.
Sources/Starting Places...As Lvk47 suggests, I also recommend that you start your exploration of classical music by borrowing some CDs from your local library, if they have some. Select some of the works recommended below or by others and try them out to see what you think of the music. You probably wont find the specific performance recommended, but youre likely to find **a** performance of any of these recommended pieces of music because they are all basic repertoire. Just remember that in classical music, different performances of the same music can sound very different and, in some cases, can either make or break the music for you.
Reference material for futher exploration...
The Grammophone Recommended Recordings list (as suggested above by Gaudio_eek) once you get some sense of which composers you like
www.gramophone.co.uk/recrecordings.asp
The NPR Guide to Building a Classical CD Collection (as suggested above by Sdcampbell)
Orchestral recommendations...Haydn: Symphony No. 94 Surprise and others, Colin Davis/Concertgebouw Orch, Philips
Mozart: Symphony No. 41 Jupiter Trevor Pinnock/ English Concert, Archiv
Mozart: Serenata notturna / Eine kleine Nachtmusik, Orpheus Chamber Orch, Deutsche Grammophon
Beethoven: Symphony No. 5, Carlos Kleiber/Berlin Philharmonic, Deutsche Grammophon
Rachmaninov: Symphony No. 2, Andre Previn/London Symphony Orch, RCA (there is a later EMI recording by Previn which is very good, but I somewhat prefer the earlier RCA)
Holst: The Planets, Andre Previn/London Symphony Orchestra, EMI
Vaughan Williams: Fantasia on a Theme by Thomas Tallis, Marriner/ASMF, Brown -violin, Decca/Argo
Mussorgsky: Pictures at an Exhibition, Reiner/Chicago Symphony Orch, RCA
Sibelius, Sym 2, Barbirolli/RPO, Chesky #3
Stravinsky, Firebird, Dorati/LSO, Mercury 432012
.
Chamber music recommendations...The following are a diverse offering of chamber music that would introduce you to a range of periods and musical idioms. Any listing could go on for pages and pages, but these are some selections that I enjoy and return to, and they are all accessible for someone just beginning to explore classical music.
Arturo Delmoni, Songs My Mother Taught Me (multiple works pieces for violin and piano, beautifully played) John Marks Records
Arturo Delmoni and Nathaniel Rosen, Music for a Glass Bead Game, (complimentary selections for violin and cello by Bach and Kodaly, very accessible, beautifully played and recorded) John Marks Records
Bach: Brandenburg Concertos, Pinnock/English Consort, Archiv (for period instruments); or Britten/English Chamber Orch, Decca (for modern instruments)
Handel: Water Music, Pinnock/English Concert, Archiv
Telemann: Tafelmusik (Music for Table) Brueggen/Leohnardt/Bylsma is my preferred set but there are other good ones. This one unfortunately is available only in a box set today.
Haydn: String Quartets (choose virtually any of them) The performances by the Kodaly Quartet on Naxos have been well received, but I havent heard them.
Schubert: Trout Quintet, C.Curzon (piano) and Members of the Vienna Octet, Decca; or The Festival Quartet, RCA (lots of other choices as well, but I like these two)
Beethoven: Archduke Trio, Kempff/Szeryng/Fournier vc, Deutsche Gramophone (Polygram)
Debussy: Sonata for Violin and Piano; or Sonata for Cello and Piano
Franck: Sonata in A for Violin and Piano
Stravinsky: L'Histoire du Soldat Suite, Chicago Pro Musica, Reference Recordings #17
Malcolm Arnolds chamber music performed by the Nash Ensemble on Hyperion (for a change of pace with something more contemporary and with humor I particularly enjoy the various fantasies for oboe and for bassoon.)
.