2021 IECC C403.12.1 ... Exception 2 says the duct does not require insulation if the "design temperature difference between the interior and exterior of the duct or plenum is not greater than 15°F". I error on the side of insulation. Especially in moist climate zones.