Port part of cache key

I have written my own DNS resolver and have stumbled open something I dont understand.
Why is the port an argument of the resolve method of AbstractResolver and used as part of the DNS Cache key? Wouldn’t it be more effcient to only use IP and Family as key to avoid another DNS request on rewrite?