This is my personal view after nearly a decade of kayak and jetski fishing. It works for me, but may not suit everyone. Two points which are somewhat related.
1) When I moved from kayak to ski, I thought I'd get out in most conditions. While the ski is safe and stable in rough water, it's a bloody wet ride in a decent chop, so I simply stay at home. I don't enjoy fishing in those conditions anyway.
2) Wet = cold. I've tried everything from wetsuits to Sharkskin and the only way for me to stay warm is to stay dry. I achieve this with dry pants with dry socks attached (Lovig Kayak Fishing Pants) and a semi-dry top (Rooster), and vary what I wear underneath according to air temperature. This works well and I come home dry 100% of the time, but see point 1.
If I was determined to go out in the worst of conditions, then I would buy a surface drysuit.