Straight oats should be okay, so long as you are careful to make sure he gets all the minerals and other nutritional needs that are not be found in plain oats. another option might be to buy dry molasses (which you can wet if you wish) to make whatever you are giving him more palatable. There is no real reason of which I am aware to take him off the sweet feed just because it is spring.
Why do you have him on a deworming schedule? Is he wormy? My preference is to have a fecal count done, and only worm when necessary. So far, my horses have not needed to be wormed but once in the last three and a half years.
It is never the horse's fault