I carry mine in my left front pants pocket (because I always have change or other stuff in my right). No case, sleeve, etc. I like to carry it in my shirt pocket (if I have one), but I don't want it slipping out and crashing to the floor if I stoop over (I've done that with my old phone about 6,000 times), so I don't do it much (unless it's a shirt with a button flap). I often wear cargo pants or shorts, and in those scenarios I'll put the phone in the little side/thigh pocket (aka the Han Solo holster position).
When I'm in my car, I put it in my little console area, where I can glance down to see the screen if a call comes through. Depending on who it is, I'll pull off and answer it. Otherwise, I just kick it to voicemail and deal with it when I get where I'm going (I don't "phone while driving"...not looking to be part of the problem).
I'd be afraid to keep it in my back pocket. That just seems to be asking for trouble...bent metal, broken/cracked glass, etc.
Why run the risk on such an expensive item? It's not your typical "free P.O.S. with a contract" phone you often get with a cellular plan. I don't baby my iPhone necessarily, but I also don't go out my way to be careless with it either. $499 doesn't grow on trees, you know...