When you say you do liberty work with him is he saddle? it seems that his resistance is when you are riding him, maybe is a tack fit issue, his he an OTTB? in general before I make any judgments about personality or behavior and start “training away” I check medical issues (he may have ulcers, back pain, etc) and tack fit. If all of that checks ok then again look at clues from his former “life” horses at the track work out in well…tracks 🙂 the idea of an arena is very strange to them, same thing for a trail horse that is asked to be in an arena….not saying you cannot do it but it requires understanding. Lastly you are right not every horses loves jumping, working cows, the trail, etc it takes a special combination of having a conformation suitable to make the job “pleasant” for them (as in pain free) and also a personality fit.
I love that you are thinking about what your horse enjoys and trying to find out if you can find a common interest to pursue.