The Complete Metalsmith, by Tim McCreight, was my manual during my goldsmithing degree, and is still my 'go to' reference.
My advice is to buy the tools as you need them. Someone may tell you that you 'need' a tool, but you end up barely using them.
I started with 4 types of pliers: round nose, needle nose, flat nose, half round nose
You'll need a pair of wire cutters
Get yourself a nice saw frame with 2/0 saw blades (they do break easily. It's not you, just practice)
Slightly more expensive, I use a set of parallel pliers, a pin vise, and a ring mandrel.
My torch is simply a butane culinary torch, or for bigger projects, a propane hardware torch. I've never had a need for a specialized 'jewelry' torch. (Soldering is a skill that takes practice. Watch lots of youtube videos, and don't get discouraged)
These are honestly the tools I use the most often at my bench. I do have some tools that are more specialized, but they've been aquired over the years and I only purchased them as I needed them.