cross-posted from: https://lemmy.world/post/2357075
It seems that self hosting, for oneself, a federated service, like Lemmy, would only serve to increase the traffic in the network, and not actually serve the purpose of load balancing between servers.
As far as I understand it, the way federation is supposed to work is that the servers cache all the content locally to then serve to the people that are registered to that server. In doing so, the servers only have to transmit a minimal amount of data between themselves which lowers the overhead for small servers – this then means that a small server doesn’t get overwhelmed by a ton of people requesting from it. Now, if, instead, you have everyone self hosting their own server, you go right back to having everyone sending a ton of requests to small servers, thereby overwhelming them. It seems that it’s really only beneficial to the network if you have, say, hundreds of medium sized servers instead of, say, thousands, of very small servers. While there is the resilience factor, the overhead of the network would be rather overwhelming.
Perhaps one possibility of fixing this is to use some form of load balancer like IPFS to distribute the requests more evenly, but I am no where even remotely close to being knowledgeable enough in that to say anything definitively.
I think you got it backwards. When you federate with an instance, you don’t send request to them (unless you’re submitting a post/comments/vote). It’s the other way around, the instance will send you activity messages to your /inbox endpoint. This means even if there are thousands upon thousands of instances federating with them, the instance won’t get overwhelmed because they can send out the activity messages at their own leisure (the amount of workers is configurable by the instance admin).
Now imagine what this mean if everyone run their own instance. The instance where the community hosted will basically act like a mailing list, receiving user activities on their /inbox endpoint from federated instances, and rebroadcasting them again to those instances. Even if the server load is high, it’s actually more reliable because the instance will automatically retry failed message delivery with exponential back off compared to real users traffics where users will keep hitting refresh if the server is not responsive.