ELI5: What is a "Server Load"?

You have a corner restaurant that can handle 50 ppl daily. Service is always good and prompt and waitress shows right when needed one day due to a busier restaurant changing service or maybe just a food critic raves about the location, and this is a national food critic. So instead of 50 the restaurant need to handle 500.

Just the logistics of trying to serve that many ppl is server load, and will remain until the restaurant can improve functionality or increase the building or staff.

A server is just a computer, right? It runs some specialized software to host a website. Anytime you go to visit a webpage, you’re making a request to the server that hosts that webpage. In response, the server will dig through its files, find the webpage you’re requesting, and send that page back to you. Some requests might be more complicated than that. All of the comments you see on a thread, for example, may be contained in a database somewhere. So the request could be; “hey, sort through ALL of the thousands of comments in your database, find the ones that are supposed to be on THIS thread, and send those back to my computer so my browser can show them.” Computers are fast, but they can only do so much at a time. Think about how your phone or computer works totally fine if you’re just playing a game; but how it may slow down if you have tons and tons of games and programs open. This is kind of the same thing. A server may work fine if it’s getting 100 requests a day. But then if you suddenly start sending 100,000 requests to it a day, that may be to much for it. It’s trying to do too much at once, and it will start to slow way down. That’s what people mean by “server load”. It’s basically “how much the server is trying to do all at once.”

It's like a queue at the checkout of a grocery store: the server is the grocery store clerk, and the load is the people in the queue.

Some people in the queue only buy one thing and it doesn't cause much wait time for the others in the queue, while others buy a lot and slow the queue down.

Bigger websites are run like bigger grocery stores and have more checkouts to serve more people at a time.

Most modern websites are multiple servers with servers being a specialized computer with no screen that has all the software needed to run the website. Each of these individual computers can each show you the website but it's somewhat random as to which you will connect to when you connect with your web browser (i.e, a request). Having multiple computers able to show you the website is one way software like lemmy is designed to help reduce its load -- often termed load balancing.

To make your website be able to be split between multiple servers you must make it so that they can all access the same information (otherwise with 5 servers a user could only see the content posted to the server they are currently connected to or approximately 20% of the content.) To achieve the data sharing, all of the computers with the software for the website will connect to the same database on a different server. This makes it so all the content is shared across all servers. But now we've introduced a new bottleneck where the database must be updated or read from on every request. This is where this can get slow or even stop when the database is not able to handle all the requests. Typically, a write operation like posting or upvoting is considered more computationally "expensive" than a read operation (e.g. browsing posts or comments). We tend to measure a server's load by its slowest component which is typically data access. Database software is built by some very smart people over time and have a lot of math involved to make them efficient but sometimes software like lemmy can write data in a way that the database algorithm isn't well designed for, making it harder to read the database if you add too much information. Alternatively database software can have inefficiencies appear once enough data is entered in their database.

There is some work that can be done to make load less impactful like: having separate databases for writing and reading and/or cloning data into multiple copies of the database, or designing the way that the data is entered into the database that makes it easy to retrieve.

The load of a server is like the load of a truck - if it is too big, the truck first becomes slower and then breaks down. Server load on Lemmy means many many "clients" - apps like Mlem, Jerboa, Liftoff, wewef and the regular Lemmy website itself (lemmy-ui) send many many "requests" to the server, which must be processed on the CPU (the "engine" of the computer). At the moment upvoting, downvoting, posting and browsing threads all cause requests to be sent to the server and therefore cause load. All those upvote requests must sooner or later be written to a database for bookeeping and the ability to save and read posts in the first place (more load). Lastly, the upvotes and posts must be distributed to all interested parties (other servers, "instances" in fedispeak) causing even more load. This allows the other instances to mirror the content and upvotes/downvotes of the post. This idea of "federation" is very powerful but probably also hard to get right when the number of interested ("federated") servers rises from a dozen to several hundreds and the number of clients to several hundred thousands. Federation is said to provide benefits like censorship resistance, independence of single points of failure and is meant to act as means of load balancing in itself. Federation is a feature which is unique to mostly opensource fediverse software like lemmy/kbin but has precedent in technologies like email and USENET (discussion boards for computer nerds of the 80s).

Similar to when your computer has to do a lot of things at the same time it slows down the machine. A server under load is just a machine that has to process a lot of information at the same time. It does not really matter which operation is performed, but rather the amount of computing power needed to perform it. Sometimes the sheer amount of simultaneous operations can cause load, for example when many users use a service at the same time.

Servers have physical limitations and when those are reached the operating system of that server will try to stay on top of it by scheduling the operations. This will make the server respond slower, since it will first have to fulfill other requests before reaching the newest one. At some point this method will fail, due to physical limitations and timeouts. When a server takes too long to respond because it's overwhelmed, other services will react with "I've waited for x amount of time for a response, I didn't get one, I will cancel this request."

To boil it down to one simple statement: The more work has to be done in any given time, the longer each individual piece of work will take to complete on average.

You are the server and you have a wheelbarrow as your resources. When there is nothing in the barrow you can run around fast with it. The more you load in the barrow, the slower you are able to move with it. It's possible to overload the wheelbarrow which will make you crash and stop.

Many apps work on a "client-server" model, where you (via a website or an app, the client) ask for things from some other machine on the internet - the server. These things the client asks for could be to get some information about something, or to make some sorta change to something

In general, these "things" the client asks for are called "requests", and Server Load is a sorta measure of how much effort the server needs to spend to handle these requests

Different requests could have different levels of effort, and even different types of effort (ex. A file upload would depend on how fast the storage is. Complex calculations like trying to find the fastest route on a map would depend on how fast the processor is). In general though, the more requests per second that you have to deal with, the more load it puts on the server

I've heard of this concept multiple times throughout my time on the internet, but I never understood what causes a “server load”. Now that I’m in one of the biggest instances on Lemmy, this is one of the significant issues that we face. So what is a “server load”, why does it slow down websites instead of stopping them?

If your computer is running two programs at the same time that have to perform computation, it creates the illusion that they are operating concurrently by giving little slices of time on the CPU to one, then to the other. Maybe a thousandth of a second to one, then to the other.

Problem is that then each program appears to run at half speed, since they have half as many CPU cycles per second.

If you are downloading two files from the Internet at the same time and you are downloading as quickly as your Internet connection can handle, same thing. Each file is broken up into little packets of information and if two are coming down at once, each will appear to come down at about half speed.

The same is true for various other resources that your computer has. Bandwidth to hard drives (usually less of an issue now that solid state memory is common, which has higher rates of throughout). Amount of RAM available, if two programs both need a lot of memory.

"Server load" tends to be used to just refer to all of this in aggregate. If a server is overloaded, some resource is less-available than one would like relative to the amount of work being done.

Server load doesn't normally stop websites because, like your computer, software is designed to just slow down concurrent work rather than just stop working. But sometimes it does stop working. I'm using kbin.social, and in some cases it has returned an HTTP 503 error to my web browser. That usually means that some component cannot keep up and has hit a maximum of concurrent operations, and is causing this one to fail to limit load.

Does the load on a server increase by the amount of information in it, the browsing of information, or both?

Normally browsing information.

Does upvoting and downvoting cause a load too?

Yes. Though it's a pretty good bet that that probably isn't what's producing most of the load. There are people looking to identify the problems, but I would guess that a more-significant cause is doing things like refreshing lists of threads and communities.

Does saving posts and comments to your profile also cause a server load?

Yes.

Just to be clear, the fact that server load would be an issue is not a surprise. The idea behind lemmy and kbin was that there would be many servers and the load would be spread out across them as people used different servers.

Some of what people will be working on is figuring out what the issues are that are causing the most slowdown, and then the developers will fix those. This is common in software development -- scaling up means that one gets more users until something starts to go wrong, fixes that, and so forth. There will be a limit where one server cannot scale up further, though.

Most large websites like Reddit have many servers, but they're rigged up to split work up so that an individual user can't really see which server is being used -- a server that isn't heavily loaded is automatically used. For a number of reasons, mostly centering around the fact that kbin and lemmy instances aren't all identical and serving up the same information, kbin and lemmy don't do that (or at least, mostly don't do that).