It's not you - it's the fragrance oil. Some of them will do that. I have one that will do it to me every time if I don't baby it a little.
In my experience, the soap itself will be fine, just not very pretty.
There are several things you can do next time to avoid the problem:
1) Let your lye water cool down completely until it's cold. It's a myth that your lye water and oils have to be the same temperature. I've even soaped oils that were so hot that my plastic stick blender melted. (I should have saved that thing!) The soap was fine in the end.
In my experience, higher soaping temperatures are what causes a lot of problems. I started out using the oven hot process method, and always added my FOs after the cook, so I never had to worry about what temperatures my oils and lye were at. Once I switched to the more traditional cold process methods, I had to pay more attention.
Just make sure that you keep your lye water away from small children/pets/other humans while it's cooling down!!
2) Add your FO to your oils before you add the lye water. This will give you a much higher degree of control, and you'll be able to make sure that your FO is evenly distributed in your soap.
3) Increase the water portion of your lye water. I usually use a 2:1 water to lye ratio, which is pretty low. Increasing your water portion will help you control the saponification rate - but it will also make your bars shrink more during the curing process. I use the other methods because shrinking soap bars annoy me.
4) Ditch the FO and use a different one, lol! Which, really, you don't have to do. Some of them are just more finicky than others.
Hope that helps!