I just want to chime in... I bought a really cheap little machine. It was not a hand-held one. But it was about 30 USD, it was a mini-model. It only did a straight stitch. And I was so excited about it, and I sewed together two lovely cloth dolls. BUT BUT... it suddenly stopped working... could not keep tension, and was having weird back-looping problems.
Turns out I had inserted the wrong bobbin. And in the meantime, I went and bought a brand new (but floor model) sewing machine. Now I have two perfectly working machines. Whoops! Anyway, you might be able to get a good deal on mini-sewing machines, and as with any other, check the size of the bobbin, and use metal ones!
This is what the mini sewing machine is like:
It's great if you can get a gift of one, or afford a fulll-size machine. But my little mini machine is GREAT. and it takes batteries! (I imagine hand ones do too?)
gratuitous picture of Nina the doll sewn on a mini-machine: