It would be great to have an idea of what causes your acne. Whether it is hormones, hygiene, sensitive skin issues,...this way you have an idea how to go about fixing the problem. This will probably take some possible tests with a physician, washing your pillow cases and hand more often. Then you should be able to gauge if these changes are lessening your acne.
I would say that is still safe to use just from my own experience the jojoba oil I used with the build up at the bottom didn't break me out when I mixed a few drops in my moisturizer or when I used it as makeup remover.