If you have pretty oily skin then you should be okay when it comes to irritation. If you're someone that deals with oiliness in certain areas more than others then I would do some targeted application to avoid irritating the areas that don't require as much oil control.

The only thing I would caution you against is mixing the Benzoyl Peroxide. I've personally never used it with witch hazel but I worry the combination might be irritating to your skin. I waited until going off of my acne ointments in order to avoid counteraction, but obviously I'm not an expert on the subject. If you decide to continue using the BP in combination with the witch hazel try testing it out on a spot for a couple of days to see how your skin reacts in that area.

Also regarding WHEN you should apply it, witch hazel is better off used as a toner/or targeted spot corrector. If you're doing with oiliness on your nose, use it on that area after you wash your face in the morning and eventually the oiliness should even itself out within a week or so.

Good luck!

I have oily skin too and I also bought Dickinson's witch hazel. I feel like it actually made me more oily, despite all the good reviews of witch hazel out there.

I think this is because of the alcohol content of Dickinson's. I haven't tried an alcohol-free witch hazel, but I've seen them in stores.