The actual solution is probably some control surfaces for the atmosphere, reaction wheels for space (both for steering) and upscaling to the biggest rocket available with no more than 2 largest size fuel tanks for each rocket. SRBs are your friend until they aren't. Stabilizers save more fuel than imagined.
I think it's the thumper booster that I determined to be the cheapest one. So I got over 20 of those, run like six of them at first, when they run out of fuel, dump them and activate four of the others, dump them when they run out, etc. It's 100% cheap boosters until I get into space.