For that time of the year you could enjoy some excellent striped bass fishing from the North Fork of Long Island which, is a quick ferry ride from Rhode Island.  If you are arriving by plane to New York and driving to the Cape this is a quicker way to get there and you can stop for an evening fish on one of the charter boats (all are excellent and quite inexpensive maybe $100) and head up north to the Cape in the morning.