[LEDE-DEV] Test results of OpenWrt 15.05.1 according to BSI test concept for home routers

[Hauke Mehrtens]

The German Bundesamt für Sicherheit in der Informationstechnik (short:
BSI, English: Federal Office for Information Security) published a
"Testkonzept für Breitband-Router (DSL-, Kabel-, SOHO-, CE-, CPE-Router,
IADs)" (English: Test concept for broadband routers). This test concept
is only available in German and most chapters are published in the
public by the BSI, chapter 4 and 5 are only available after signing a
NDA (Traffic Light Protocol) with the BSI:
https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Cyber-Sicherheit/Themen/Testkonzept-Breitbandrouter.pdf?__blob=publicationFile&v=5

Some unnamed organization tested OpenWrt 15.05.1 on a TP-Link  TL-WR841N
V10.0 according to this test concept. Because I commented on the first
public draft of the test concept and said that I am active in the
OpenWrt project, the organization contacted me to check if their test
results are correct and provided me with the full test report under NDA.


OpenWrt 15.05.1 failed this test and LEDE will probably also fail this
test, because we failed on section 4.5.1 "Firewall-Bypass", which is a
criterion for exclusion (Ausschlusskriterium), see details below.

The test concept focus on "normal" routers and only tests the web GUI
and also looks mostly on features normal home routers have. We are
missing some functionality like individual default password, for the web
GUI and the Wifi, the logging is not very good and so on. The tests
regarding DNS are interesting and more advanced, if someone wants to
look into that it would be very nice.

The main problem is in section 4.5.1 "Firewall-Bypass".
OpenWrt and LEDE implement RFC4890 section 4.3.1:
-------------------------------------------------------------------
4.3.1.  Traffic That Must Not Be Dropped

   Error messages that are essential to the establishment and
   maintenance of communications:

   o  Destination Unreachable (Type 1) - All codes
   o  Packet Too Big (Type 2)
   o  Time Exceeded (Type 3) - Code 0 only
   o  Parameter Problem (Type 4) - Codes 1 and 2 only

   Appendix A.4 suggests some more specific checks that could be
   performed on Parameter Problem messages if a firewall has the
   necessary packet inspection capabilities.

   Connectivity checking messages:

   o  Echo Request (Type 128)
   o  Echo Response (Type 129)
-------------------------------------------------------------------

The BSI used RFC6092 (Recommended Simple Security Capabilities in
Customer Premises Equipment (CPE) for Providing Residential IPv6
Internet Service) with this section as the base for the test:
-------------------------------------------------------------------
3.2.1.  Internet Control and Management

   Recommendations for filtering ICMPv6 messages in firewall devices are
   described separately in [RFC4890] and apply to residential gateways,
   with the additional recommendation that incoming "Destination
   Unreachable" and "Packet Too Big" error messages that don't match any
   filtering state should be dropped.

   REC-10: IPv6 gateways SHOULD NOT forward ICMPv6 "Destination
   Unreachable" and "Packet Too Big" messages containing IP headers that
   do not match generic upper-layer transport state records.
-------------------------------------------------------------------


Attached are the results of this test of OpenWrt 15.05.1. The
information on how the tests from chapter 4 and 5 are done is redacted
from the document, if you want to work on these problems and would like
to get more details about the problems from chapter 4 and 5, please
contact me. I can also help you with translating the problem from German
to English. ;-)

The "sensitive" informations are under the Traffic Light Protocol
classification "TLP AMBER", see these German information about the NDA:
https://mip.bsi.bund.de/Anlage_1_TLP-Merkblatt.pdf


I commented on the tests itself, because they are missing many important
stuff to test, most of the security problem of IoT devices and home
routers one hears about in the media are not covered here at all.


History:
20.10.2015: I read this article https://heise.de/-2851354 and wrote some
comments to the BSI based on the first public draft. In this mail I
mentioned that I am activate in the OpenWrt project.
23.10.2015: The BSI answered me and offered me the full draft when I
would sign an NDA, I did so and got the full document, but did not
comment on it again.
23.2.2016: I got the full final draft from the BSI
22.11.2016: I was told by the unnamed organization that they tested a
TP-Link device running OpenWrt 15.05.1 and if I could comment on their
results. I got the results under the NDA and commented in them.
31.3.2017: I asked if I can publish at lest some parts of the results
again and got the OK that I am allowed to publish the redacted results.


Hauke
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenWRT_Ergebnisse.pdf
Type: application/pdf
Size: 166574 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/lede-dev/attachments/20170408/b7ef96ad/attachment-0001.pdf>