What you have described sounds like you are perhaps gripping too much with your upper leg or knee. I had the same problem. When you are riding think about sinking your knees downward and grounding your feet in the stirrups, allowing your toe to align with your knee. You should be able to easily go into your two point at any gait and stay there effortlessly. Instead of gripping with your leg, allow it to drape the horse with a light, even pressure and remember to breathe deeply. You might have developed some tension.
Any time spent with horses doing anything is a good time!