i would try finding a ph balanced cream cleanser, preferably all natural for combination skin, at a health store like whole foods.

but if you cant find one, alot of people have luck with cetaphil gentle skin cleanser. its basically a cream cleanser. its ph balanced.

If you're using actual soap then yes it can cause potential dryness. As hopeless suggested look for a gentle PH cleanser which won't strip away the oils on your face when cleansing. I've found oils to help with tight face and peeling, look for jojoba, emu or coconut. They should stop the dry feeling you experience. Good luck