Swimming shouldn't be too much of a problem, but I do understand your concern. Apply moisturizer as soon as possible after finishing your swim, preferably while your skin is still damp. This will help lock in moisture to prevent excess drying. And yes, it's totally fine to moisturize 3 times a day! 

As an FYI, many people find the drying effects of the chlorine beneficial to acne, as it helps dry out acne and promote faster healing.

I swim about 16 hours a week because I swim for a high school team and my face peels and flakes all the time but not as much if i rinse of the chlorine of right after I get out of the pool .

Thanks for your help