Good on you for asking the question, I'm continually amazed at the level of ignorance displayed by anglers when it comes to identifying the fish that swim in our seas. Many species have inappropriately inherited northern hemisphere names which muddies the waters as they are not close relations, or even in the same family. Few species that you are likely to catch are bad to eat, but a lot comes down to treatment of the fish from point of capture and cooking technique. 

Quick reference:

https://www.fishspecies.nz/" rel="nofollow - https://www.fishspecies.nz/

https://www.oceanhunter.co.nz/resources/species.html" rel="nofollow - https://www.oceanhunter.co.nz/resources/species.html

Comprehensive:

https://docs.niwa.co.nz/library/public/NZAEBR-208.pdf" rel="nofollow - https://docs.niwa.co.nz/library/public/NZAEBR-208.pdf

If you want a printed book, then I would recommend:  https://www.mcleodsbooks.co.nz/p/field-guides-photographic-guide-to-sea-fishes-of-new-zealand" rel="nofollow - https://www.mcleodsbooks.co.nz/p/field-guides-photographic-guide-to-sea-fishes-of-new-zealand

Hook, Line & Sinker by Daryl Crimp. 

the sub-title 'What they look like, where to find them, and how to catch them' says it all.

It was first published 20 years ago, but the info re the fish species and basic fishing methods for each is still quite applicable for a newbie.

If you live in the upper North Island and bait or lure fish in the in-shore waters you'll typically encounter the classic common catches of snapper, kahawai, jack mackerel, trevally, kingfish, gurnard and various sharks/rays. John dory are another occasional catch. That's dependent on time of year, fishing zone and type of fishing, obviously. 

The good thing about that book i mentioned is that if you live in another part of the country, it includes the common species there.

There is also a lot of good info on this forum in the advice section.

NZers tend to turn their noses up at so-called lesser table fish, often just based on what they've been told, which is a bit sad as we miss out on some delicious eating. But as MB said, correctly treating and cooking fish is the key. e.g. kahawai is stunning eating in my view (raw fish dishes, Thai curries, fish cakes), but biffed back by many.

Last thing - I'd encourage you to familiarise yourself with the regulations about minimum sizes and bag limits.

"Hook, Line & Sinker" very much looks like what I'm looking for. The photographic guide looks nice, but perhaps more for a marine biologist than a fisher.

I'm in Wellington, in case that makes any difference.

Another I just found is called Marine Fishes of New Zealand by Heath/Moreland Eric Heath and John Moreland Old book 1967 (my print is 1976) all colour drawings but identifiable just 100 photos/descriptions. 

I've ordered "Hook, Line & Sinker" and will see how it goes. Looking forward to trying some real life IDing.

I think you will find there are two books titled 'Hook, Line and Sinker'.  The earlier version was written by R.B. Doogue and dates back to the 1960s.

Another reference book is titled 'New Zealand Sea Anglers Guide' by Raymond Doogue and John Moreland.  This was first published around 1960.  This was my bible when I was a kid.

We had the Moreland one when i was a kid, Krow - a classic.

This is the book i recommended, by Daryl Crimp.