There is no real limit imposed by B31.3. I've directed repairs to refinery piping up to 60" at refineries and I have seen larger. I can vouch for the 72" size indicated by pennpiper.
Our fabrication shop built some gas ducting just this year for the sulfuric acid industry that was 80"ID per B31.3. It was for low pressure (~45psi) gas service, and included 5" thick refractory on the inside surface.
You'll often find this really big B31.3 piping on furnace discharge piping, where you have a large volume of low-density gas and pressure drop is at a premium because the whole system is driven by a fan of some sort.
At some stage, dont you require a very thick wal to satisfactie the D/T ratio requirements? Or am I mixing things up and does that only have to do with pipe stress analysis, and branch calcs?
The only limits with respect to D/t are that the Appendix D formulae provide values for k and i. Outside of those limits, analysis is permitted, but there are no codified values.