A search of Google images confirmed my recollection that it's pretty shallow the whole way around what is a very small coral islet.
And it's pretty busy around most of the island with people splashing in the shallows etc.
You might be able to use a kayak or other craft to get out to a bit of a drop off and flick some lures around.
Another option would be to use some brownie points and catch the ferry that takes you to the island back to one of the other islands where there is easier access to deep water.
You'll see what I mean on the way out to Treasure - as it's one of the last stops - and there are several ferries in each direction every day.
Just be aware that fishing is not a right in some places in the Pacific Islands, as villages may own a particular area as their traditional fishing turf. It's good to ask rather than end up causing offence.