Evolving the BSD Cafe Network Setup: From Bridging to Routing with FreeBSD

The article discusses the evolution of a network setup using FreeBSD, transitioning from a simple VPS configuration to a more complex multi-node network. Initially, the setup involved a single VPS (VPSSmall) with an internal bridge and jails for service management. As the number of jails increased, a second VPS (VPSBig) was added, utilizing NAT and ZeroTier for inter-VPS communication. However, this bridged setup faced limitations such as performance overhead, scalability issues, and security concerns.

To address these challenges, the author implemented a new configuration using Wireguard and VXLAN, which improved security and performance while reducing dependency on third-party services. Despite these improvements, the reliance on bridging remained a concern, prompting a final shift to a routed setup. This new configuration allowed each VPS to operate on distinct subnets, enhancing scalability, traffic control, and security.

The use of FreeBSD's Forwarding Information Base (FIB) feature enabled separate routing tables for jails, allowing for more efficient traffic management. The article concludes by emphasizing the importance of adaptability in network architecture, understanding trade-offs in networking approaches, leveraging advanced features of FreeBSD, and maintaining thorough documentation throughout the evolution of network setups. The techniques shared can assist in building robust and adaptable networks, highlighting that network design is an iterative process that should evolve with changing needs.