Making good use of available computing resources sometimes means finding a way through the network that keeps them out of reach.
Why I needed an internal-network tunnel
Recently, my school allocated several servers to me. Their combined market value would be roughly CNY 10,000 per month, and the hardware is genuinely capable. The catch is that none of them has a public IP address. To use the machines normally, I would have to go to the server room every time.
The room itself is not the real problem. It is cold and noisy, but that is manageable. The inconvenient part is that the servers can effectively only be used from there, while I obviously cannot spend all my time in the server room. The practical solution is to use network tunneling, commonly referred to as internal-network penetration or NAT traversal.
There are many tools and techniques available today, but most of them are built around the same basic idea: creating a tunnel between a machine inside the network and a machine that can be reached from the public internet.
Why an internal machine cannot simply be reached from outside
In a typical setup, the local network is created by a router. If every machine were to connect directly to the public internet, the router could theoretically be replaced with a switch. In practice, that is not a realistic solution. Public IPv4 addresses are limited, and even with a switch, each machine would still need an address assigned by the ISP.
Communication between a local network and the wider internet generally relies on NAT. When an internal device sends data outward, the router can present itself to the remote server as if it were an ordinary computer. At the same time, it records which internal device initiated the connection. When a response comes back, the router uses that record to forward the data to the original machine.
This mechanism works well for connections initiated from inside the network, but it creates a problem in the opposite direction. If a computer on the public internet wants to contact a machine inside the local network, it can normally reach only the router's public IP address. Since the internal machine never sent a request outward, the router has no connection record telling it where the incoming data should go. The packet is therefore discarded.
One straightforward way around this is for the internal machine to establish a connection first. Once that outbound connection exists, the external machine can use it to reach the internal host. This is the approach used by most internal-network tunneling solutions.
UPnP can provide another route in some environments. It appears to be used by many peer-to-peer applications and can automatically create port mappings on a compatible router. In effect, it allows an internal machine to ask the router to forward a port to it, even when the machine has not first initiated an ordinary connection. The exact details depend on the protocol and router, and I have not investigated that mechanism deeply.
If you have administrative access to the router connecting the local network to the internet, the problem is even simpler: configure port forwarding directly. Unfortunately, ordinary users generally do not have that level of access. If they did, there would be little need to look for alternative tunneling methods in the first place.
Common ways to build the tunnel
There are now many implementations, but the following approaches mostly rely on the internal machine making an outbound connection to the internet.
Using a VPN
A VPN creates a tunnel from the host to a VPN server and uses virtual routing to form a shared network, or VLAN-like environment. Because the connection is initiated by the internal host, a computer outside the original network can connect to the VPN and then reach machines inside the local network. It may also be able to access other networks available to those machines.
This is relatively simple and can be secure. The drawback is reliability: many ISPs in China block or interfere with VPN traffic, so using a VPN is not always dependable.
SSH reverse tunneling
SSH is a remarkably versatile tool. If VPN traffic is blocked, perhaps port 22 will remain available—otherwise it would interfere with a great deal of system administration work.
An SSH tunnel is also fairly easy to set up. Enable the GatewayPorts option in sshd on a public server, then run a command like the following on the internal machine:
ssh -fN -R 10022:localhost:22 远程用户@远程主机
The tunnel allows other computers to connect to the internal device through the public server. This works well, but it has an unavoidable requirement: a publicly reachable server is needed. The same is true for the VPN approach, which makes it worth considering whether a more convenient solution is available.
FRP services from third-party providers
FRP is, as its name suggests, a kind of proxy and reverse-proxy solution. A number of providers offer ready-made FRP services, including SakuraFrp and Peanut Shell. Anyone with their own public server can also deploy FRP independently.
Compared with the previous methods, a useful feature of this approach is that the reverse-proxied services can be made directly available to other users. That makes it convenient when a service needs to be shared rather than accessed only by the person who created the tunnel.
There is still a technical limit: one server can reverse-proxy at most 64,551 services, ignoring the distinction between TCP and UDP. For ordinary use, that limit is more than sufficient.
Direct port forwarding
Direct port mapping is arguably the best option. It avoids the bandwidth limitations that may come with an intermediary and does not require a separate public server. The difficulty is obtaining administrative access to the router. In most real-world networks, that is the part that cannot easily be solved.
What I ended up using
Since direct port forwarding is the most attractive option, that was ultimately what I used. The problem was that I did not have administrative access to the router.
Fortunately, someone had previously configured port forwarding on this router, although the mapping was no longer being used. I followed the existing configuration and assigned my server the same IP address as the host that had previously been mapped. With that arrangement, the forwarding worked without requiring me to change the router settings.
There is still the question of what to do with all those resources. I have not yet thought of a particularly good use for everything, and I have even considered whether running something like an IDC might make sense. Until a more concrete plan appears, the servers can at least contribute their computing power to the wider world. Their performance is impressive: while participating in WCGrid, my current daily ranking has reached the top 200 worldwide.