That is what you think is happening just like i did when i extend my rocket launcher and had clears put down the side,

the old wise guy said, the wind first blows the boat over, but you alter the steering wheel to counter it, which in turn brings the boat back and now its leaning into the wind, 

its true believe me that is exactly what happens, 

take note the next time your out.

trim tabs are awesome, better ride in any direction less gas, why more ways to alter the boat angle in the water,  

 

Or you have not forgotten you were towing a skier !

if you have a hardtop this will be your problem for sure,

when driving and the wind is slightly off to the side the wind can effect the boat handling,

its not to hard to understand, wind from say right hand side will make the boat lean away from the wind which is to the left, 

so you make an adjustment to counter it by turning the steering wheel to the right bring it back to upright and going in the direction you wont,

but in the mean time the boat feels like its running away from you,

the best and only solution is trim tabs, you can now adjust it so your running true.