There's been some speculation on manipulation on the follower counts, but I'm guessing it's as simple as it looks:
- some people no longer feel welcome and are deleting and/or deactivating (I know some folks who have at least considering it)
- the flipside of that is that some people who previously didn't feel welcome now do again