A port identifies an application or a service. It has been set up this way. There are TCP ports and UDP ports.
Servers can accept multiple connections on a single port because of active and passive sockets.
However, the important thing to remember is that there are two types of ports: well known and ephemeral.