Daniel, you are going through what most of us went through when we first started. The advantage you have over some of us, is that when we started, there weren't any forums like this and there weren't any videos on-line to help.
First off, I would practice on some scrap wood, instead of trying to cut an actual project. Draw some lines, curves, straight, sharp angles, etc on the wood and just focus on following the lines. This way you don't get the added frustration of feeling like you've spoiled a project.
You may be experiencing some blade drift. Some scrollsaw blades tend to drift to one side or the other (mostly to the right, in my experience). It's fairly common, but a source of great frustration until it's understood and compensated for. Draw a straight line on a piece of wood. Line it up with the blade and feed it straight into the cut, being careful not to twist or push the wood to one side or the other. If the blade wants to wander off to the side, despite you not feeding it there, this is blade drift. It can be compensated for by moving slightly to the side and feeding the wood into the blade from a slight angle. If the blade drifts off to the line to the right, then stand/sit slightly off center to the right and then feed the wood straight ahead. You will be pushing the wood at a slight angle across the table, but the blade should cut a straighter line. It might take some trial & error to find the sweet spot, but once you do, it should be easier to follow the line.
Good luck and have fun making sawdust!