I used to have a J-180. I swamped it plenty of times. I never did get used to it. The part that drove me nuts is that I never felt unstable. The primary stability feels pretty good. It's just that there seems to be no secondary stability after you dip the wing. I'd be paddling along just fine until some light boat wakes would catch me from the rear and next thing I knew I was swimming.
I spent a little time intentionally swamping in warm weather and shallow water to try and get a better feel for it. I think more of that would have helped.
I think seat time is the key. Plenty of people paddle/race J-boats without issue so it must be something you get used it. I used to have a racing kayak as well. It seemed almost unpaddleable when I first sat in it. But I put in the hours and eventually felt very comfortable, even in chop.
What's your weight? As I recall it was made for lighter paddlers. 180lbs used to be tossed around as the max weight.
I assume you already have a foot brace in it?
Alan