I Like Self-Hosting

As I mentioned in my previous posts, I love open-source software. How do I prefer my open-source projects? Of course, self-hosted. I love self-hosting not because I save a ton of money on hosting or SaaS fees, but because it offers a fun and educational experience. However, in my professional life, I usually avoid self-hosting due to the responsibilities it entails and the total cost that accumulates over time. I understand it depends on individual circumstances, so it’s not right to generalize, but this has been my general experience.

Where It All Began

So, where did it all start? Probably, many of us have gone through a similar journey. I was just learning how to code and created a basic website using Bootstrap (thank you for teaching me what responsive design is). I had the webpage ready and needed to share it with my friends and family."OK, here you go C:/Users/Dogac Eldenk/Desktop/awesome_website.html". You're saying it doesn't work? I installed something called Apache, now try localhost:8000. It still doesn't work? Oh, there are internal and external IP addresses. I found mine; here you go: 100.0.0.255. It doesn't work either. Let's learn about NAT, firewalls and port forwarding. Oh, I have to pay my ISP just to forward my port? Fine, a couple bucks per month must be worth it (it was really hard to convince my dad back then). Finally, my website was online!

But who could remember those digits to visit my website? I wanted a cool domain name, so I paid for one. OK, I finally got everything in place, I thought. My server goes offline when I turn off my computer? Do people really keep their computers open all day? I don't think so. I thought the magical answer to this question was CPanel hosting. It was super cheap to host my own website on CPanel, however it had several limitations. It was constrained to PHP, HTML and MySQL in my case. So, what is the alternative? I found Digital Ocean. This website allowed me to host a VPS (Virtual Private Server) for $5 per month. As you might have guessed, I was broke because I paid yearly for hosting and a domain name and could never afforded a proper VPS for more than a month.

Of course, things have changed quite fast in the last 10 years. When I went through this, I knew basically nothing about how servers work. I was routing myself towards the shortest path to achieve hosting a website online to get only 3 clicks per month. Why am I telling this story? Because I have learned so many things just to showcase my website to my friends. I know this could have been a screenshot or screen recording, but where is the fun in that?

What About My Own Server?

So, at this point, I had a brief idea about how websites work. I also questioned how game servers worked when I wanted to spin-up my own Minecraft server locally to play with my friends. Then I found out about SBCs, Single Board Computers! I have bought my first Raspberry Pi, plugged it in and started tumbling. I learned so much when I first used it: Linux, python, hardware, networking etc... I previously had an Arduino, ESP-8266; I wrote C code to blink some LEDs, display some text on an old-school display. However none of them were as capable as real computers. So, when I met the Raspberry Pi, I realized I could connect everything together and connect it to the internet.

My first significant project was creating a weather station. I had some electronic components that measured temperature and humidity. I also had an RF radio module which communicated with the Raspberry Pi I set up in my room. Using this setup, I was running my weather station on an AA battery, which I prematurely optimized to run for years on a single battery even though I was going to use it for only a week and leave it. The station was reporting to my Raspberry Pi using RF and the Pi was hosting the data on my website online!

Of course, this was a fun little project that thought me a lot. I was still unable to do more complex tasks, for example run a Minecraft server. Also. the ARM architecture was not so popular back then, so occasionally I hit a hard wall of "x86 only" applications. So, I never actually had a server that I could use generally; it was always limited in some way.

When the COVID pandemic began, I had to suspend school and go back to live with my parents. I was bored for months; the only fun I had was playing Counter-Strike with my friends. So as a fun project, I spun up my own game server to play with my friends. To do this, I used my old laptop, which wasn't being used. I kept it open all day. However, it only lasted until summer because my room would get super hot, and the fan of that computer was giving me a hard time sleeping.

Half a year later, I returned to the College and started working part-time. With my first month's salary, I upgraded my desktop computer. During my internship in the UK, one of my colleagues, Aaron, showed me his home setup. He had a separate computer running Unraid. So I got inspired and thought I could use the remaining parts to create a home server like he did. I bought a wooden crate to put my computer parts in because I did not have an extra case.

I did not use Unraid for some reason. I installed Ubuntu on my computer (Pop OS!) and occasionally used the HDMI output to connect to my TV. For the first time ever, I had a computer at home that was capable of almost anything I could imagine, except for machine learning training for my coursework. I used this computer to host our final year project, game server, media station and even a personal cloud. I was so happy to be using this computer for my daily life.

What About Now?

Rolling forward for two and a half years, I moved to the US in the meantime. I had to leave a lot of stuff I owned back in my home in Turkey. During this period, I learned a lot about the cloud. However, I was still pretty much excited about having my own hardware. My first months in the US were rough, so I bought an Orange Pi 5 Pro and Raspberry Pi 5 to spend some time with. I probably should have only bought the Raspberry Pi; however I wanted the 16GB of RAM on my server and thought 8GB was not enough.

It felt pretty nostalgic to be working with a SBC again after 10 years. The community was much more bigger than before, and I knew much more about computers and software. I have used those two SBCs to learn about Kubernetes and and created my own Kubernetes cluster using those two. I still have some fun projects in my mind involving Pi Hole, personal Grafana dashboards, CasaOS and so on.

Later on, I was frustrated by the fact that the tutorials I followed still did not publish Docker images on the ARM architecture. I was frustrated and I wanted to do something more professional with x86, something I can use more generally, something with more memory and more importantly, disk space.

I surfed through eBay to find refurbished computers. Then I found the Dell OptiPlex 3060. This computer offers so much for only $135: specs are i5 8500T, 32GB RAM and 512GB SSD. It had twice the performance, double the memory, much much more disk capacity and speed.

This time, I won't be running barebones Linux on my server. So I installed Proxmox. Because I had so much memory and disk space, I could create multiple VMs for various tasks. First, I created 3 Ubuntu Server virtual machines. I used those virtual machines to learn about kubernetes and created a cluster using microk8s.

One day at work, I had to do some testing with an external integration and I needed a public HTTP server. Unfortunately, I am not able to port-forward devices based on IP addresses because the Xfinity app sucks. I went ahead and created a lightweight container using Proxmox without any VM and set up HAProxy. This was an opportunity for me to learn about HAProxy. I put this container behind DMZ and let it handle the traffic and forward it to my laptop. I also configured a Minecraft server and put it behind the load balancer.

Moving forward, I decided to start a blog, the one that you are reading right now. I created this blog using Ghost, an open source blogging platform. I am self-hosting this using the Dell machine inside a virtual machine. As always, it is a fun experience to go through the hassle of initial setup with software. You always learn something during the process.

Getting Serious

Now I want to get serious with my blog. As I post more stuff, I hope to get more traffic (more than 2 people per day). As I do this, availability is a concern. For example, yesterday, the electricity went off and my server shut down. The website was inaccessible for a whole day, and I did not realize it.

I was at home, so I was able to recover my blog. However, I am planning to travel this summer to visit home. So I won't be able to do any emergency recovery in that case. Also, I am not backing up my blog yet. So I am thinking about moving my blog temporarily during summer and observing how it goes.

So I have a couple of alternatives; either to keep self-hosting on the cloud with higher availability or use a hosted version of Ghost. And I chose, of course, to keep self hosting. I have done some cost analysis. To keep things simple, I have used AWS as the baseline. The server I need should be fairly minimal with a couple of gigs of RAM.

• ECS: Too expensive, not even near EC2 for the smallest instance. 1 vCPU + 1 GB RAM for $30 per month.
• EC2: My choice of instance is t4g.micro with 2 vCPU and 1GB RAM. It costs about $7 for a public IP, and I choose spot instances ,so $3, adding up to about $10 per month.
• Amazon Lightsail: This is the most traditional approach, a VPS. The pricing for this class is also much more predictable as there aren't many moving pieces around. The same setup with 2 vCPU and 1GB RAM is only $5 per month using IPv6.

Currently, my choice is Amazon Lightsail. My website is super lightweight; I only need a couple of features. I am not even thinking about using AWS managed MySQL to manage my data. I am hosting Ghost and MySQL under the same instance. Note that MySQL can run on instances with 512MB RAM out of the box.

For backing up and alerting, I am planning to set up some alerts that would send me an email regarding downtime on my server. I am also planning to back up my MySQL database to S3 daily or weekly. This seems like the cheapest and easiest option.

Conclusion

Self-hosting is fun; it teaches you a lot. It doesn't matter if the thing you do is the most efficient or productive way possible. What matters is how you get there and what values it brings. I feel like trying to do stuff on my own is an important part of my early life that taught me lots of stuff that I currently know. In this post, I have focused on servers; however, it doesn't have to be just servers. It can be writing your own X, where X is already a solved problem such as writing your own serialization format, implementing compression with Huffman Encoding, writing a Chess Engine, implementing Neural Networks from scratch, a custom JSON parser, and so on.