Every time I clean an LP that’s not new, I also clean the jacket before putting it in a new outer sleeve. Lately, I’ve been using Method All Purpose Cleaner (because it’s very mild and usually around the house) on a microfiber cloth and it works great. You wouldn’t believe how dirty the microfiber is after a quick wipe. I then dampen a clean part of the towel and give it another quick wipe to remove whatever cleaner is left on the jacket. The whole process takes about a minute and is completely necessary for used vinyl in my opinion.
You can actually feel the difference. Rub your hand on a jacket before cleaning and note how rough it feels. After cleaning, do the same thing. It will be silky smooth. That's the lack of dirt.