Scaling an App
I was recently asked a question about how to scale an app. When I work on my own personal React projects, I don’t really think about scaling…I mainly focus on playing around with code to understand all its nuances. That’s probably the case with a lot of new developers…if you’re just working on a small project, thinking about scaling can actually get in the way of the creative work. In fact, a number of solutions to scaling introduce complexity and abstraction, which in turn can make testing code harder, fixing bugs and introducing new features becomes more difficult and and frustration increases when trying to match production versus local environments. These complications can be too “expensive” to deal with if your app isn’t actually large enough to warrant some scaling solution.
Ok, so let’s say you are actually starting to see some slowing down in your app. The very first thing that is needed is a clear definition of exactly what the problem is. In other words, you need to determine what the limiting resource is…what is your app running out of?
The first step would most likely be to check resource monitoring to see if some resource was at capacity. What you use for resource monitoring depends on how your servers are being managed. If you’re using Amazon, AWS offers the service for your, and if you’re using Heroku, New Relic seems to be preferred. If you manage your own servers, you should install something like Munin. If the resource monitoring doesn’t show anything being overwhelmed, it would probably be a good idea to do some logging to check if there is something that is taking a long time to load.
It is highly recommended that you host your web servers on a different machine than your database…and so resource monitoring should be done on both machines. The consensus seems to be that scaling issues usually occur first in the database.
Another recommendation is that it’s always a good idea to make sure the tech stack is doing as little work as possible for requests that occur most frequently. This could involve things like storing results of common operations (to avoid repeating work), reusing data that has already been looked up, and avoiding complex operations in the request-response cycle. Some form of caching, in other words.
It’s also a good idea for a domain to point to a load balancer, which can then route requests between multiple web servers. this makes it easier to handle increases in loads…and also gives the added bonus of being able to deal with failures much more easily. Requests can be spread across more machines, thereby allowing each individual one to do less work. Using a load balancer allows apps to grow with fewer hiccups over time and can deal with temporary traffic surges. It’s definitely something that I’ve seen recommended from the start, as it doesn’t add much complexity and it only needs to be set up once.
One thing that happens with large databases is that lookups can take quite a bit of time, with queries having to examine every single row in thousands of rows to find the right condition. If a database index was added for particular queries (say username lookups), the right row could be found immediately, thereby saving a significant amount of time.
It might also be a good idea to consider where you store your session data; if you are storing the actual key/value data in a table, your database may experience quite a lot of delay from the load. In order to alleviate this load, one option is to move session storage to an in-memory caching tool, like redis or memcached.
Summary
Let’s sum up the scaling solutions I’ve touched on here:
- resource monitoring and logging (to actually figure out what the problem is)
- host your web servers on a different machine than your database
- make sure the tech stack is doing as little work as possible for requests that occur most frequently
- use a load balancer
- database indexing
- in-memory caching tool
This is by no means an exhaustive list, as there are a number of other solutions out there. The solution (or solutions) you’ll need will obviously be dependent on what specific scaling issue you’re experiencing with your app. The thing I realized, though, is that it’s never too early to at least start thinking about what it would mean to have to scale an app! Happy coding!