Forum host upgrade - proposal

Sat Aug 17 01:13:18 PDT 2024

Hi,

> On 16. Aug 2024, at 21:49, Stijn Tintel <stijn at linux-ipv6.be> wrote:
> 
> Resend because someone decided to send HTML email which tricked my email client to sending HTML email as well, which resulted in my response being rejected.
> 
> Please people, stop sending HTML email, and while at it, stop top-posting.
> 
>> On 16/08/2024 21:37, Koen Vandeputte wrote:
>> 
>> 
>> Op do 15 aug. 2024 01:06 schreef Hauke Mehrtens <hauke at hauke-m.de>:
>> 
>>    On 8/12/24 22:11, Stijn Tintel wrote:
>>    >
>>    > On 8/12/24 15:33, Robert Marko wrote:
>>    >> On Sat, 10 Aug 2024 at 17:54, Ted Hess <thess at kitschensync.net>
>>    wrote:
>>    >>> Hi all -
>>    >>>
>>    >>> Those of you who visit the forum have probably noticed its,
>>    sometimes
>>    >>> extreme, sluggishness over the last several months. Since
>>    April, I have
>>    >>> worked on and applied a number of attempts at tuning, cleanup and
>>    >>> storage re-arrangement to try and alleviate these issues. As
>>    best as I
>>    >>> can determine, we have have just simply exceeded the capacity
>>    of the
>>    >>> current Droplet CPU and RAM. To upgrade to a system with 2X
>>    CPU/RAM
>>    >>> (4vCPU/8GB -> 8vCPU/16GB) will increase cost from $48/mo to
>>    $96-112/mo.
>>    >>> The difference in cost is a "Regular Intel" vs "Premium Intel"
>>    >>> processors. The Premium (latest Xeon) option will provide
>>    enhanced NVMe
>>    >>> SSDs which would improve DB performance (a bit). I think it
>>    would be
>>    >>> worth $112/mo for maximal impact on the responsiveness of the
>>    forum. AMD
>>    >>> vs Intel - cost is the same. AMD may be slightly higher GHz.
>>    >>>
>>    >>> Are there any objections / reasons why we shouldn't go ahead
>>    and do the
>>    >>> upgrade. There are a couple of methods available however, the
>>    down-time
>>    >>> will probably be 1-3hrs. if no problems. This isn't the first
>>    system
>>    >>> upgrade we have done, so I expect no issues.
>>    >>>
>>    >>> If any of you wish to see some of the forum performance stats,
>>    I can
>>    >>> post another message with numbers and personal observations. A
>>    quick
>>    >>> overview of what our daily forum activity looks like:
>>    >>>
>>    >>> * Serving 120-160K page views. (about 25% attributed to
>>    crawlers - a
>>    >>> different discussion)
>>    >>> * 6-8K mail messages generated
>>    >>> * ~1100 User visits (signed-in)
>>    >>> * ~200 engaged users (liked or posted)
>>    >>> * ~500 new posts
>>    >>>
>>    >>> Footnote on bandwidth usage... (DO charges bandwidth on aggregate
>>    >>> Droplet use)
>>    >>> Proposed Droplet upgrade will add 3TB to our total.
>>    >>> I don't have per-Droplet stats, but last month we transmitted
>>    ~29TB of
>>    >>> our 25TB allowance. A $40 overage cost.
>>    >> Hi Ted,
>>    >> I am for upgrading the droplet as forum is probably the main
>>    point of
>>    >> interaction
>>    >> in the OpenWrt community.
>>    >>
>>    >> But, like others pointed out, we maybe should look for a more
>>    >> affordable host
>>    >> then DO as they are quite expensive.
>>    >
>>    > We use DO because they sponsor us USD 5000 to use their
>>    services. If you
>>    > look at the current costs, upgrading another droplet will cause
>>    us to
>>    > use up the USD 5000 before they expire. Please keep that in mind
>>    before
>>    > doing anything that will increase cost.
>>    >
>>    > Stijn
>> 
>>    Hi,
>> 
>>    I think it is fine if we invest some money into the server for the
>>    forum. From my point of view it is fine to upgrade the server from
>>    4vCPU/8GB -> 8vCPU/16GB for an additional ~70$ per month.
>> 
>>    We should consider to move some resources to other providers like
>>    Hetzner, but we can do this later.
>> 
>> 
>>    Hauke
>> 
>>    _______________________________________________
>>    openwrt-adm mailing list
>>    openwrt-adm at lists.openwrt.org
>>    https://lists.openwrt.org/mailman/listinfo/openwrt-adm
>> 
>> 
>> Hi all,
>> 
>> Is there any interest in my proposal for hosting the forum at Citymesh? (Free of charge)
>> Datacenter team has given a Green Light internally.
>> 
>> Thanks,
>> 
>> Koen (Mobile)
>> 
>> 
> Hi Koen,
> 
> While I appreciate the offer, I am hesitant to accept it. I remember one of the main driving factors of the LEDE fork all too well. One of them was the downtime of key OpenWrt infrastructure, with the inability of all but one OpenWrt team members to solve said downtime, due to the rest not having access. This is something we absolutely have to avoid from happening again in the future. That said, I would be open to accepting the offer, if the following conditions are met:
> 
> At least 3 (preferably 5) OpenWrt team members who are currently involved in maintaining the OpenWrt infrastructure should have full access. As to full access, this includes the ability to call someone at any given time of day to recover something that is broken.
> 
> Keeping in mind that cyber attacks are currently happening everywhere, phishing-proof 2FA must be mandatory. In my opinion, the only standard that provides this, is FIDO2 hardware keys.
> 
> A signed statement from a major stakeholder at Citymesh, that if a decision is made to retract the offer, we are guaranteed to have at least six months to migrate everything elsewhere.
> 
> I know I am harsh on this, but we should learn from the past.

I did not follow the whole thread but when it comes to having a more powerful machine at DO, I could migrate away the ASU server to another provider. It’s less critical infrastructure and rather costly due to CPU and storage requirements.

It’s rather trivial to setup so I could move it within days. Please let me know if that would be of help.

Best,
Paul