It isn't just yoga - any exercise wears me out quickly, and it's very easy to over do it. I personally don't do yoga anymore because my wrists have damage and can't bend all the way. The closest I'll get is regularly doing gymnastics stretches I learned from when I was a gymnast, as many of them don't put weight on my wrists. I really need exercise. I feel better when I do it regularly. My doctor even ordered 45 minutes of cardio a day! However, when I over do it, I'm over fatigued the next day. I always have to recover from exerting myself too much. I've also had mild flares from over doing a work out. It isn't just being sore - I've been somewhat athletic most of my life until recently (despite being bad at athletic things, I just loved to move,) so I'm familiar with being sore.
So I personally like to swim, as it's no-impact, works my entire body, and gets my heart rate up quickly. I won't swim for more than 10 minutes at a time, unless I'm doing very light easy laps. I always take a break between each lap.
Otherwise, I like doing Xbox Fitness. It's just workouts from popular DVD series, but the Xbox's Kinect reads your body and tells you how well you're doing and offers further instructions. It also features a point and achievement system, which shouldn't be motivational, but for some reason it works really well for me, lol. I usually stick to the 10 minute low-impact work outs. They work me hard, but they're short so I can handle them. I've toned up a lot from doing these, and I'm honestly feeling pretty strong (despite being very weak with fatigue.)
Each day I make sure I do a few chores and go for a walk if I can in addition to my work outs.
I've been doing this for 6 months. I thought I'd be getting better at working out by now. I'm really not. So I don't know if "needing to adjust" is the right way of putting it. I think your body gets used to working out, and even starts to expect it. However, I should be making progress with my cardio ability, and I'm not. I think fatigue is very limiting.
