Forum host upgrade - proposal

[Rich Brown]

Ted: Thanks for:

a) doing so much heavy lifting on the forum. 
b) thinking about ways to keep the server responsive.

I'm glad you are paying attention.

I submit that it's entirely reasonable to spend an additional $50-$60/month to support this critical piece of our infrastructure. It's one of the most important places that people get information about OpenWrt. If the alternative is to spend our administrative efforts making the server "less useful", I vote for spending the money.

Last time I looked, we're spending something like $250€ per month on servers. Increasing this to 300€ would not make much of a difference. As long as we have the funding - which I believe we do - from SFC, other resources, or donations (https://openwrt.org/donate) it makes sense to keep the forum server responsive. This is a good use of money.

Thanks again.

Rich

> On Aug 10, 2024, at 4:19 PM, Ted Hess <thess at kitschensync.net> wrote:
> 
> Johnathan - Not sure what is your point here. Do you want to do some work on the forum performance? All help is appreciated. I'm not sure the areas you suggest would make a significant reduction in use given the shear size and popularity of the site.
> 
> With-respect-to (wrt) Crawlers - A typical bad day breaks down like this:
>    15K Logged in users
>    110K Anonymous users
>    47K Crawlers
> 
> Biggest offenders are the major indexers - Bing, Google, Yandex &  Baidu. In general it is not a good idea to block these. However, we do slow them down by rate-limiting based on user-agent. People masquerade as other user-agents to try and bypass this. We also block a number of known offenders entirely. It's a moving target.
> 
> Wrt email - This one is difficult. Especially, since we allow users to interact with the forum via email. Folks (like myself and others) get summaries mailed by option - you can turn yours off. Others prefer to receive notices of new posts on their favorite topics. I haven't really broken down the details of how many of what type of emails are being sent. I do remember a while back I looked at the number of users getting summaries and it was a reasonable number given the size of the forum.
> 
> 
> On 8/10/2024 3:22:43 PM, "Jonathan lancett" <lancett01 at googlemail.com> wrote:
> 
>> 25% attributed to crawlers is a lot. Can we do something about that?
>> Is there a way of cutting down on the mail messages?
>> 
>> 
>> On 10/08/2024 16:53, Ted Hess wrote:
>>> Hi all -
>>> 
>>> Those of you who visit the forum have probably noticed its, sometimes extreme, sluggishness over the last several months. Since April, I have worked on and applied a number of attempts at tuning, cleanup and storage re-arrangement to try and alleviate these issues. As best as I can determine, we have have just simply exceeded the capacity of the current Droplet CPU and RAM. To upgrade to a system with 2X CPU/RAM (4vCPU/8GB -> 8vCPU/16GB) will increase cost from $48/mo to $96-112/mo. The difference in cost is a "Regular Intel" vs "Premium Intel" processors. The Premium (latest Xeon) option will provide enhanced NVMe SSDs which would improve DB performance (a bit). I think it would be worth $112/mo for maximal impact on the responsiveness of the forum. AMD vs Intel - cost is the same. AMD may be slightly higher GHz.
>>> 
>>> Are there any objections / reasons why we shouldn't go ahead and do the upgrade. There are a couple of methods available however, the down-time will probably be 1-3hrs. if no problems. This isn't the first system upgrade we have done, so I expect no issues.
>>> 
>>> If any of you wish to see some of the forum performance stats, I can post another message with numbers and personal observations. A quick overview of what our daily forum activity looks like:
>>> 
>>> * Serving 120-160K page views. (about 25% attributed to crawlers - a different discussion)
>>> * 6-8K mail messages generated
>>> * ~1100 User visits (signed-in)
>>> * ~200 engaged users (liked or posted)
>>> * ~500 new posts
>>> 
>>> Footnote on bandwidth usage... (DO charges bandwidth on aggregate Droplet use)
>>> Proposed Droplet upgrade will add 3TB to our total.
>>> I don't have per-Droplet stats, but last month we transmitted ~29TB of our 25TB allowance. A $40 overage cost.
>>> 
>>> 
>>> _______________________________________________
>>> openwrt-adm mailing list
>>> openwrt-adm at lists.openwrt.org
>>> https://lists.openwrt.org/mailman/listinfo/openwrt-adm
>> 
>> 
>> 
>> _______________________________________________
>> openwrt-adm mailing list
>> openwrt-adm at lists.openwrt.org
>> https://lists.openwrt.org/mailman/listinfo/openwrt-adm
> 
> 
> _______________________________________________
> openwrt-adm mailing list
> openwrt-adm at lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-adm