First, ask yourself what you want out of riding. If you want to do hunters/jumpers, maybe this horse just isn’t the right one for you. I know it’s hard, but maybe he would be better in a different situation and you could get a horse that better fits your goals.
If you don’t have your heart set on a particular discipline, and you want to keep this horse, is doing ground work for the rest of his life going to be enough for you? Probably not. It can be a great foundation to start from. I would recommend following a Parelli type program for a while. It gives you set goals and will help your relationship. Your horse is young and you might both really benefit from the ground work.
However, eventually you need to ride your horse. Regardless of whether you do jumping, dressage, reining, or trail riding, you horse needs to be able to calmly walk, trot, and canter both directions around the arena, be engaged behind, bend, and have clean transitions on your command. With a young horse, this is easier said than done. This will take a lot of work and consistency. Is there a trainer that can ride your horse and give you lessons while you build his foundation under-saddle? Doing patterns, transitions, and poles can help keep him entertained. I understand wanting to be his partner, but he can’t be aloud to push you into corners. Do the ground work in the “problem” areas of the arena first, then work your way up to riding him in those areas. He might need a swift kick in the rear to teach him that behavior isn’t acceptable.
Lastly, if he is so much better on the ground than under-saddle, are you sure his tack fits him correctly? Is his behavior being caused by pain? An ill-fitting saddle could cause him to be lazy, buck, and turn sour. Consider finding a saddle fitter in your area.
I hope this helps.