Your ISP could be doing CGNAT in a way that causes this - you can see if you're behind CGNAT by checking what your router reports your WAN address as being and what a site like https://ipinfo.io says; if these two addresses differ, you're likely behind CGNAT.