I moved from Stavanger about a year ago, so things might have changed, but here's a few places that might be worth your while...off the top of my head:
For second hand --check out Fretex (Salvation Army). You can come across fabric and buttons, and books/magazines and, well, anything, really. There's one downtown (in Breigata,the store might be overprised on 'vintage/retro' things, which is annoying, but you could get lucky!)
-- across the street lies Harry Camping, a good store for inspiration and funny findings.
-- there's also a Fretex store in Mariero (and right next to it is a Panduro-store/craft supply store).
There's also a decent craft supply store next to Stavanger Aftenblad's building downtown, can't remember the name, it's across the street from Magasin Blaa. And there's a bead store and a scrap book store, also downtown (around the area of Øvre Holmegate, the street with all the colourful houses).
Except for the two places in Mariero, all these stores are within five-ten minutes of each other, all downtown. Good luck
