I understand what you mean. That's what I started doing but I found it hard to figure out whether to do the pattern stitches on the "extra" edge stitches. Let's say you find the tip of the first diamond, and there are 4 stitches before that stitch, then deciding what to do with those 4 stitches also depends on the last "extra" stitches, ykwim?
I frogged the whole body (all 12 rows of it LOL). I delimited the number of stitches I need on the chart and before starting a row, I check if my yos and k2togs balance. So far so good, I have 16 rows done, but only 1 decrease row (I'm doing the vest). I'm crossing my fingers that it will be smooth sailing from here.
Here's my progress so far. I'm using Elann Silken Kydd in Clove, and I'm knitting with 3.0 mm (size 3) (!) needles.