Help untangling CAKE settings for FTTH

[Sebastian Moeller]

Hi Thibaut,


> On Nov 17, 2022, at 16:00, Thibaut <hacks at slashdirt.org> wrote:
> 
> 
> 
>> Le 17 nov. 2022 à 15:42, Sebastian Moeller <moeller0 at gmx.de> a écrit :
>> 
>> Hi Thibaut,
>> 
>>> On Nov 17, 2022, at 15:22, Thibaut <hacks at slashdirt.org> wrote:
>>> 
>>> Hi Sebastian,
>>> 
>>>> Le 17 nov. 2022 à 10:50, Sebastian Moeller <moeller0 at gmx.de> a écrit :
>>>> 
>>>> Hi T.
>>>> 
>>>> 
>>>> so taking your proposa under consideration I canged the section that threw you off course to read:
>>>> 
>>>> 
>>>> 	• Ethernet with Overhead: SQM can also account for the overhead imposed by VDSL2 links - add 22 bytes of overhead (mpu 68). Cable Modems (DOCSIS) set both up- and downstream overhead to 18 bytes (6 bytes source MAC, 6 bytes destination MAC, 2 bytes ether-type, 4 bytes FCS), to allow for a possible 4 byte VLAN tag it is recommended to set the overhead to 18 + 4 = 22 (mpu 64). For FTTH the answer is less clear cut, since different underlaying technologies have different relevant per-packet-overheads; however underestimating the per-packet-overhead is considerably worse for responsiveness than (gently) overestimating it, so for FTTH set the overhead to 44 (mpu 84) unless there is more detailed information about the true overhead on a link available.
>>>> 	• None: All shaping below the physical gross-rate of a link requires correct per-packet overhead accounting to be precise, so None is only useful if approximate shaping is sufficient, say if you want to clamp a guest network to at best ~50% of the available capacity or similar tasks, but even then configuring an approximate correct per-packet-overhead is recommended (overhead 44 (mpu 84) is a decent default to pick).
>>>> 
>>>> 
>>>> I hope this is explicit enough.
>>> 
>>> Yes this looks a lot better, thank you.
>>> 
>>> Although I must confess that it certainly feels counter-intuitive that for ethernet (and FTTH) we suggest a higher overhead than e.g. VDSL2/cable (which themselves run off an ethernet interface).
>> 
>> 	That is simply because for (ADSL) DOCSIS VDSL2 we have a much clear picture about what we need to account for. And AON can be essentially using true ethernet encapsulation so we can expect an unspecified "FTTH" class to encompass a broad set of different encapsulations, if we want to recommend a single number that should also cover AON (the PONs are much less clear*). Why do you assume that FTTH necessarily would have a smaller per-packet-overhead than other link technologies? 
> 
> I’m not (necessarily) making that assumption for FTTH (although I would expect it to be the case, from my limited understanding of FTTH technologies),

	Well, all I can say about encapsulations and per-packet-overhead is: it's complicated.

> I am however certainly making that assumption for plain ethernet, which is included in the 44-byte documentation case. I think it’s a reasonable one?

	Yes, the goal is to give one number that works everywhere so 44 is what we recommend. As I said overestimating the overhead is benign, underestimating it is not. True ethernet has an effective* per-packet-overhead of:
7 Byte Preamble + 1 Byte start of frame delimiter (SFD) + 12 Byte inter frame gap (IFG) + 4 Byte Frame Check Sequence (FCS) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype)
7 + 1 + 12 + 4 + 6 + 6 + 2 = 38 bytes

Now if a VLAN tag is added you end up with 38+4 = 42 so both pure ethernet and ethernet+VLAN have a er-packet-overhead close to 44 bytes.

*) The IFG is not real bytes, but transmission silence for the time it would take to transmit 12 octets.

> 
>> 	Now, if you have reliable information for say GPON-overhead, by all means add it (but to be useful this also should contain an easy identifier for other users to figure out whether they use GPON in the first place).
>> 
>> 	The bigger point however is IMHO, from the perspective of minimizing bufferbloat/latency-under-load-increase using a slightly too large per-packet-overhead is fully benign, while specifying to small an overhead can easily result in measurable bufferbloat increase. So IMHO it is fine to err on the side of too large when estimating the per-packet-overhead.
> 
> OK. Although I would think people reading the detailed explanation are looking for precise info and explanations, not one-stop-shop approximations.

	Sure, but the problem is we can not give them that "precise information" them, you , me would like to have, so we do the best we can.

> Not to mention the kind of users who want to squeeze the maximum performance out of their setup.

	Just read the section where I explain how per-packet-overhead and shaper-rate are not orthogonal variables to understand how problematic the whoe thing is, add to it that ISPs often employ traffic shapers with potentially independent overhead-assumptions making the whole problem even harder.

> 
>> *) The core problem is that we have no straight forward way to empirically deduce the effective per-packet-overhead over an arbitrary network path, so if a link technology defines/documents the overhead well and conclusively (like docsis) we are in luck, but if the details a vague or leave many options to the ISP we have to make an educated guess.
>> 
>> 
>>> I would also like to see some info about ppp vs ethernet interface in there (matching your previous email): unless you beat me to it I will add it.
>> 
>> 	I will not beat you to it, because for users of cake it does not matter
> 
> You said the exact opposite in your previous email (persistence of statistics) :)

	I misunderstood your argument here and thought you wanted a section explaining the issues with getting the overhead numbers for pppoe** and ethernet interfaces set for HTB+fq_codel, sorry. If you want to explain differences between cake on pppoe or on ethN go for it.


**) ppp or pppoe somewhat matters, PPPoE has 6 Bytes of overhead on top of PPP's 2 byte resulting in a total of 
PPPoE: 2 Byte PPP + 6 Byte (PPP)oE = 8 Byte

> 
>> and we do recommend to use cake in the first place... heck even for folks wanting to use HTB+fq_codel I would recommend to start with cake and then look at the output of `tc -s qdisc` to figure out the overhead that is already accounted for on an interface.
>> 
>>> I also think the « details » page needs to be reformatted a bit, it’s very dense and relevant info is all over the place and not very well organized.
>> 
>> 	Might well be true (the page evolved over time and might need a full editorial pass), but it covers quite some territory and hence always will be a bit unwieldy.
>> 
>> 
>>> I’ll try to get around improving that.
>> 
>> 	Please try to keep all correct information around in the document.
> 
> Sure.
> 
> Cheers

Regards