Might just be the O-ring, but I'd replace the whole cartridge - they're generally cheap enough. Delta cartridges can be found at any of the Big 3 but I've had the best luck at Menard's for plumbing kit. Yank it out (or replace both while you've got the tools out and handy), bring it in to the store to make sure it matches, then pop it back in easy peasy. Make sure to put some silicone grease on the O-ring and the insert-end of the cartridge.
If that doesn't work you've got bigger problems and might have to replace the whole assembly. I had to do just that last weekend for a downstairs shower. Pain.
