Sounds like a case of the bigger kahawai holding further out. Finding out where-abouts the bigger kahawai are in the 'close' water you can reach fly casting should solve the problem. I have struck the same problem at a local rivermouth. I find though that the bigger fish come in close along the edges of the outgoing river current (but not in the river water itself) and can reach them there with the fly rod. Also in the channel of the main river towards the top half of the incoming tide as the tide pushes in.

 

Always lots of baby ones in the shallow surf..