There's no clear smoking gun, but what I did discover digging around is that we saw a huge jump in site access starting on Saturday. By Monday, there was 20 times as much activity...
I suspect the real issue was the access logs. When you have increased activity, logs grow accordingly.... As a log file size increases, the amount of time needed to insert each new transaction also increases, and the transactions start to stack up. That latency continues to snowball until the system is taking 20-30 seconds to respond.
I took a couple steps on Monday which I'd hoped would work, but apparently didn't.
First off, I have banned a dozen IP ranges that were accounting for over 90% of the traffic, which all appeared to be crawling each and every topic, image, and attachment on the site.
That caused some immediate relief, but as has happened in the past, they'll just move on to a new set of IP addresses.
The second step I took was to implement a "velocity check" which will temporarily ban the IP address for anyone trying to access more than ten different pages in a ten second timeframe.
If you're a fast clicker, you might wind up getting locked out for a few minutes. My advice... slow down and enjoy the ride a bit.
Some other steps I've taken are to limit what Guest users (i.e. not logged in) are able to access. I want to balance out the common good access that the site offers to new users who haven't quite committed to creating a user account here, but at the same time, this same guest access is how the Bots are hitting and scraping the site. I may have to come up with an alternative way of letting guests browse the forum.
Hopefully the combination of steps will turn down the heat a bit and let everyone enjoy the site as expected.
If you're a FastPass or First Class subscriber, I've extended the expiration date by five days (assuming I did the math correctly....) to help make up for anyone who didn't get the level of uptime and server responses they expected. Please PM me if that extension didn't get applied to your account.
If you are finding it difficult to access the site because of a "forbidden" message, let me know or use the Trainsim.com Discord page as an alternate way to contact me.
