Hey i_ckone,
I'm going to make some personal observations based on my experiences, so take them for that and do your research as well if any of it intrigues you
Many people have had success with supplements, but it's important to look at all the options out there. I haven't come across anyone on here who has had success with B5 alone, and taking it in high amounts can lead to hair loss. 2 grams is the "norm", if you will as far as safe amounts. You could also be producing a lot of oil because your skin is trying to compensate for being overly treated with topicals that are stripping, washing too much, etc. One of the best ways to remedy this is to find a moisturizer that doesn't break you out, and always apply it after cleansing. Eventually, your skin will start to normalize it's oil production.
As far as self medicating, it's important to look into side effects, and realize that not all of the possible side effects will be realized. Now with some clinical, peer reviewed studies that I have read, EPA (an omega 3 fat) and GLA (an omega 6 fat) are both known improve the quality of one's sebum, decrease the amounts of active lesions. Of course many mainstream doctors do not advise taking supplements, but there is a lot of empirical evidence out there that it is effective, and it's much easier to prevent a disease than it is to treat it. There are of course doctors who believe in supplementation/dietary changes (or at least possess a curiosity of it), hence the peer reviewed studies, and shows like The Doctors and Dr. Oz.
In short, I hope the best for you in finding what works for you!