pillaging far away inactive that has a fleet

trying to pillage an inactive, they have one paddle ship as a reserve (both saw in soying and actually fighting / combat report), still getting the message that my troops are getting turned around when trying to pillage since they would lose a sea battle.
since the ship is in reserve i cant destroy it, nor occupy port since its an inactive. is there a way to successfully pillage the city without destroying one of my cities and temporarily settling in the same island just to pillage?
thanks in advance

UPDATE
pillaging *right after* "occupy port" worked, pretty sure last time ive waited less than 4 hours after attacking the fleet to pillage, maybe by the time the pillaging army came 4 hours have passed already and im just misremembering (4 hours for units to scatter)