|
|
Post by mjgraves on Dec 29, 2017 21:48:02 GMT
I had a curious thing happen this week. I thought I'd pose a question about it to the dev & user community. First - I've used m0n0wall and SmallWall for 15 years (I think.) For the past several years I've run it on an old HP T5700 thin client with a dual-ported Soekris NIC. This was connected to Comcast Business Class (50/10) using an Arris SB6141 cable modem. Everything had been working fine for literally years. On Wednesday, December 27th I started suffering packet loss connecting to my employer's phone system. Using WinMTR on my desktop, I tracerouted conf.zipdx.com. This showed that the problem was somewhere in the *.comcast.net space. It was on a hop between two servers in their network that the packet loss appeared. I called Comcast support and was routed to the Business Class team. The tier 2 tech I spoke with was NOT a dummy. He reported zero packet loss using tracert to access the far side of the Comcast realm. He asked me to take the SmallWall out of the circuit, connecting directly to the cable modem. When I did this the amount of packet loss was dramatically reduced. Not eliminated, but reduced a lot. Given that experimental evidence, I took SmallWall offline, switching to a Buffalo WZR-600DHP running DD-WRT. I just happened to have it on-hand and nearly ready to go. It bugged me to think that a traceroute issue that appeared to be in the middle to a path might be in some way caused by the router in my office, at one end of the path. But SmallWall was running on a old platform, so I thought I'd run another experiment. I took my SmallWall installation to a utility PC. - 3.3 GHz AMD FX6100 3-core (6-thread) CPU - 10 GB RAM - booting from USB key - Onboard Realtek NIC for WAN - USB2-to-Gigabit adapter for LAN When I use this host to run my existing instance of SmallWall I still see massive packet loss accumulate at the second hop inside comcast.net.  It appears to me that even using this much more powerful PC for SmallWall, there's packet loss accumulating that does NOT appear using the Buffalo router. This sort of packet loss doesn't impact web surfing or email, but it can kill any streaming media, most especially VoIP telephony. There's my mystery...any idea why SmallWall causes a difference? Michael Graves |
|
|
|
Post by Lee Sharp on Dec 30, 2017 2:02:01 GMT
My first thought was overload, but the second computer eliminates that! My next is MTU. MTU detection is black magic voodoo, and no one does it the same. On that Windows box, set the MTU to 1000 and see if it still drops.
|
|
|
|
Post by mjgraves on Dec 30, 2017 21:15:37 GMT
Tried the following:
netsh interface ipv4 set subinterface "Ethernet 2" mtu=1000 store=persistent
Where Ethernet 2 is the only active interface. Didn't seem to make a difference.
I follow that there's is ICMP de-prioritization. That would cause one host to stop responding altogether. This does not seem to be that.
|
|
|
|
Post by Lee Sharp on Dec 30, 2017 21:35:31 GMT
Try it on the firewall, just for goggles... But now I think that may not be it. Do you get the same result when it is not icmp? Like with ltf or nping? It seems that a lot of network types block, filter, or deprioritize icmp. Note that de-prioritization can make it drop ICMP only when heavily loaded, not just drop it all the time. There are as many implementations as there are coders...  |
|
|
|
Post by Lee Sharp on Dec 30, 2017 21:38:28 GMT
Also, I have a similar path, but am not seeing the same results...
traceroute to conf.zipdx.com (166.88.23.109), 30 hops max, 60 byte packets
1 fw-sharp.no-ip.org (192.168.64.1) 0.346 ms 0.396 ms 0.479 ms
2 96.120.118.101 (96.120.118.101) 11.800 ms 11.867 ms 11.945 ms
3 ae-105-rur02.royalton.tx.houston.comcast.net (68.85.249.69) 13.930 ms 13.964 ms 12.015 ms
4 ae-2-rur01.royalton.tx.houston.comcast.net (162.151.134.65) 13.770 ms 13.736 ms 13.810 ms
5 ae-29-ar01.bearcreek.tx.houston.comcast.net (68.85.245.85) 12.345 ms 12.735 ms 12.624 ms
6 be-33662-cr02.dallas.tx.ibone.comcast.net (68.86.92.61) 20.729 ms 17.511 ms 22.343 ms
7 be-12441-pe01.1950stemmons.tx.ibone.comcast.net (68.86.89.206) 20.285 ms 15.537 ms 16.146 ms
8 dls-b21-link.telia.net (62.115.150.6) 15.996 ms 15.998 ms 16.025 ms
9 las-b21-link.telia.net (62.115.123.137) 48.178 ms 48.169 ms 48.126 ms
10 sjo-b21-link.telia.net (62.115.116.40) 54.920 ms 54.923 ms 59.899 ms
11 egi-ic-318658-sjo-b21.c.telia.net (213.248.75.141) 63.963 ms 60.593 ms 60.519 ms
12 sjc-e-109.zipdx.com (166.88.23.109) 54.969 ms !X 55.718 ms !X 54.977 ms !X
|
|
|
|
Post by mjgraves on Jan 5, 2018 23:10:11 GMT
For the past week I've been operating two networks; (1) SmallWall on the desktop utility PC and (2) Buffalo N600 (DDWRT) - both connected to the cable modem using a small switch.
I had another issue today. Using SmallWall when I connected to a Google Hangout the video quality I was sending was VERY bad. Poor resolution. Low frame-rate.
When I patched my desktop to the Buffalo router it was much improved.
I'm perplexed. I've ordered a dual-ported 1GB PCIe NIC to upgrade the utility PC running SmallWall. It has Intel NICs.
I'd rather order new hardware and continue to run SmallWall than move to something else, but I need it to be reliable. I need to get this resolved. It's so puzzling that it just started to happen a week ago.
|
|
|
|
Post by Lee Sharp on Jan 6, 2018 3:58:51 GMT
I have some piles of nics and other computers to try swapping with to insure it is not hardware. Seeing as how we are close... I too want to figure this out. |
|
|
|
Post by mjgraves on Jan 10, 2018 16:30:21 GMT
Argh! The dual-ported Intel NIC requires a PCIe 4x slot. My utility PC only has 1x slots. Returned.
Ordered a single-port Intel NIC with a PCIe 1x form factor. Delivery expected today.
In the mean time, swapped out the utility PC for an old PCEngines ALIX (AMD Geode LX800) that I had in the closet. It still has the last m0n0wall install on it. I had to abandon it back when our Comcast service was upgraded to 50/10. It could not cope with that throughput.
Will try SmallWall on the ALIX as I await the Intel NIC.
Any opinions on the PCEngines APU2C4? Their hardware seems harder to get than it was in the paste, at least in the US. But at Under $200 for a kit I'd consider it as an upgrade option.
|
|
|
|
Post by Lee Sharp on Jan 10, 2018 18:18:19 GMT
For that price, you can get Atom based motherboards that will out perform it. Or 2 versions back terminal servers that will also out perform it... I just can not justify the price so I have never owned one.
As for the nic... I have a 2 port intel nic that is PCI. (Actually PCI 64 but will work in PCI) Yes, it will slam the buss at full bandwidth, but might be worth a try for testing.
|
|
|
|
Post by mjgraves on May 11, 2019 13:06:48 GMT
For that price, you can get Atom based motherboards that will out perform it. Or 2 versions back terminal servers that will also out perform it... I just can not justify the price so I have never owned one. As for the nic... I have a 2 port intel nic that is PCI. (Actually PCI 64 but will work in PCI) Yes, it will slam the buss at full bandwidth, but might be worth a try for testing. Since that old utility PC was a hulking beast, I bought an HP T620 Plus on Ebay and dropped a dual-ported Intel NIC into it. Around $150 all in. It's got an AMD GX-420CA CPU, 4GB RAM, 16 GB Sandisk msata SSD. Right now it boots to a USB key. I tried installing Smallwall to the internal flash drive (16 GB) but the install routine in the ISO failed. Ordered an msata to USB adapter so I can try to flash the module on another PC. I note that Smallwall does not recognize the USB 3 ports if a USB 3 device is connected. The UEFI BIOS need to be set to use legacy booting. |
|
|
|
Post by Lee Sharp on May 11, 2019 14:32:46 GMT
I have noticed a lot of FreeBSD booting issues from CDs on modern systems. I needed to se up some Xigmanas systems and I failed to boot on 6 different computers. Either a read error on the CD or a boot loop. I am not sure why this is becoming such a problem with the newer UEFI systems as the older ones worked fine. Ended up writing the sticks with an old BIOS type laptop.
With the price I am seeing, I may just try and grab one and troubleshoot it. They are quite cheap for what they are.
|
|
|
|
Post by mjgraves on May 13, 2019 15:48:02 GMT
I have noticed a lot of FreeBSD booting issues from CDs on modern systems. I needed to se up some Xigmanas systems and I failed to boot on 6 different computers. Either a read error on the CD or a boot loop. I am not sure why this is becoming such a problem with the newer UEFI systems as the older ones worked fine. Ended up writing the sticks with an old BIOS type laptop. With the price I am seeing, I may just try and grab one and troubleshoot it. They are quite cheap for what they are. Yeah, crazy capable little box. The SSD is a SanDisk mSata. I was able to flash it using an external adapter. Although, in the end I may not use it. The T620 Plus has two internally available USB 2 ports. For now I just installed the USB key I was already using to one of those. That USB key is literally 8 years old! I created it back when I was using a Net4801. I can't even recall how I created it with a second partition to receive the config file. Is that in the docs somewhere? |
|
|
|
Post by Lee Sharp on May 13, 2019 16:40:07 GMT
|
|
