Hi all.  I've been running White Russian on my Linksys WRT54G for some time now and decided it was time for a hardware upgrade (my Linksys has been notoriously unreliable).

So I picked up a Gateworks Avila GW2348-4 and installed Kamakiaze 7.09 on it.

The network performance has been horrible.  I've tried all sorts of things but really don't know what the deal is.  I rarely see throughput above 5kbyte/s on it.  My Linksys could push over 300kbyte/s so I'm really at a loss, here.  Also, for some reason the ethernet ports lose their links randomly.  The lights are still on on the ports themselves, but OpenWRT reports that the links are down.  It takes cycling the power to bring them back up.  If someone could steer me in the right direction as to what's up here then I'd be grateful. 

Honestly, I'm not entirely sure where to start with this one...

(Last edited by sadpengu on 29 Mar 2008, 22:05)

I'm experiencing the same problem here.  You can use ethtool to check your settings: ethtool eth0

You can configure for maximum throughput by typing the followings:

ethtool -s eth0 speed 100 duplex full autoneg off

Sometimes the throughput improves immediately but sometimes I would have to do ifconfig eth0 down and ifconfig eth0 up several times before bandwidth reaches around 60Mbps.

This is not a fix but it will help until someone comes up with a better solution.

Thanks for the idea with ethtool.  It's been a long time since I've had to do any networking in Linux and couldn't remember what the link tool was.

But now that I do, I've discovered something very disturbing.  Here's the output of ethtool -S eth1 after a few minutes of activity:

root@cerberus:/etc/dnscache# ethtool -S eth1
NIC statistics:
     RX ready to use queue len     : 63
     RX received queue len         : 0
     TX to be send queue len       : 32
     TX done queue len             : 0
     StatsAlignmentErrors          : 131280
     StatsFCSErrors                : 3225368512
     StatsInternalMacReceiveErrors : 3276733816
     RxOverrunDiscards             : 3276733756
     RxLearnedEntryDiscards        : 3221580572
     RxLargeFramesDiscards         : 3221577828
     RxSTPBlockedDiscards          : 131280
     RxVLANTypeFilterDiscards      : 16
     RxVLANIdFilterDiscards        : 3223797152
     RxInvalidSourceDiscards       : 3276726272
     RxBlackListDiscards           : 4294967295
     RxWhiteListDiscards           : 3223231264
     RxUnderflowEntryDiscards      : 0
     StatsSingleCollisionFrames    : 0
     StatsMultipleCollisionFrames  : 131280
     StatsDeferredTransmissions    : 3223361024
     StatsLateCollisions           : 131280
     StatsExcessiveCollsions       : 3223231264
     StatsInternalMacTransmitErrors: 3276733872
     StatsCarrierSenseErrors       : 3276733820
     TxLargeFrameDiscards          : 3221662408
     TxVLANIdFilterDiscards        : 3221661420

The eth1 interface is my WAN interface and is connected to an ADSL modem.  The output for eth0 (LAN) looks normal.

I just tried taking down eth1 and bringing it back up (ifconfig eth1 down followed by ifconfig eth1 up).  When I tried to bring it back up it gave me this:

eth1: NPE-C not responding
ifconfig: SIOCSIFFLAGS: Input/output error

This does not occur on eth0.  I can bring that one up and down without an issue.

I'm leaning towards there being some sort of nasty bug in the microcode that the IxP4xx boxes use to control the ethernet ports.  At present I'm downloading the Gateworks BSP source to see if I can pull the microcode that they're using.

Further help/suggestions/gotchas will be much appreciated.  I'd love to solve this issue.  Otherwise this is the most expensive brick (and the only blue one) I've ever purchased.

(Last edited by sadpengu on 29 Mar 2008, 23:59)

Also, ksoftirqd is now using over 99% of the CPU time according to top.  I did a little reading on that and apparently that has something to do with the interfaces needing to poll something or other repeatedly?  Don't recall exactly...

try trunk

I'm using the 7.09 trunk and my ethernet (eth0) performs poorly.  Upon booting up, eth0 defaults to 100Mbps half-duplex autonegotiation on.  After typing "ethtool -s eth0 speed 100 duplex full autoneg off", ethtool claims that it's running at 100Mbps full-duplex and autoneg off.  However, running iperf -r, the throughput for one direction is around ~60kbps while the other direction is at ~60Mbps.  After typing ifconfig eth0 down/up several times, both directions are able to transmit at ~60Mbps.

I also tried to rebuild the image w/ and w/o microcode but it doesn't seems to have any effect.

Shane

You should compile and try trunk (https://svn.openwrt.org/openwrt/trunk/)...

Until you compile your own trunk it's useless to add more posts...

(Last edited by forum2008 on 30 Mar 2008, 21:29)

Revision 10690 of trunk errors out on make menuconfig.  Which revision should I use?

I'm working on seeing if I can fix it.  I'm probably missing a utility.  We'll see.

(Last edited by sadpengu on 31 Mar 2008, 01:16)

Something goofy was going on with the check-lxdiag.sh.  Reporting errors on empty lines.  My keen sense of intuition tells me that the line endings are wrong since I had to download it on a Windows box (my router being down and all, I'm leaching from my neighbor's wireless).

I've hooked my server up to my DSL to remove the taint of Windows, we'll see how it works this time.

I'm able to build and test the new Avila image from r10690 but it kept rebooting itself (redboot > booting kernel > redboot > booting kernel...).  I've attached the boot log for reference:

Uncompressing Linux................................................................... done, booting the kernel.
Linux version 2.6.24.2 (schao@eris) (gcc version 4.1.2) #2 Sun Mar 30 18:09:49 PST 2008
CPU: XScale-IXP42x Family [690541f1] revision 1 (ARMv5TE), cr=000039ff
Machine: Gateworks Avila Network Platform
Warning: bad configuration page, trying to continue
Memory policy: ECC disabled, Data cache writeback
CPU0: D VIVT undefined 5 cache
CPU0: I cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets
CPU0: D cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 4064
Kernel command line: rtc-ds1672.probe=0,0x68 root=/dev/mtdblock2 rootfstype=squashfs,jffs2 noinitrd console=ttyS0,115200 init=/etc/preinit
PID hash table entries: 64 (order: 6, 256 bytes)
Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)
Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)
Memory: 16MB = 16MB total
Memory: 14076KB available (1844K code, 180K data, 96K init)
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
net_namespace: 64 bytes
NET: Registered protocol family 16
IXP4xx: Using 16MiB expansion bus window size
PCI: IXP4xx is host
PCI: IXP4xx Using direct access for memory space
PCI: bus0: Fast back to back transfers enabled
NET: Registered protocol family 2
Time: OSTS clocksource has been installed.
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 512 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 512 bind 512)
TCP reno registered
IXP4xx Queue Manager initialized.
squashfs: version 3.0 (2006/03/15) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler deadline registered (default)
Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing disabled
serial8250.0: ttyS0 at MMIO 0xc8000000 (irq = 15) is a XScale
console [ttyS0] enabled
serial8250.0: ttyS1 at MMIO 0xc8001000 (irq = 13) is a XScale
eth0: MII PHY 0 on NPE-B
eth1: MII PHY 1 on NPE-C
eth1: MII read failed
eth1: MII read failed
IXP4XX-Flash.0: Found 1 x16 devices at 0x0 in 16-bit bank
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Using buffer write method
cfi_cmdset_0001: Erase suspend on write enabled
Searching for RedBoot partition table in IXP4XX-Flash.0 at offset 0xfe0000
5 RedBoot partitions found on MTD device IXP4XX-Flash.0
Creating 5 MTD partitions on "IXP4XX-Flash.0":
0x00000000-0x00060000 : "RedBoot"
0x00060000-0x00160000 : "kernel"
0x00160000-0x00c60000 : "rootfs"
split_squashfs: no squashfs found in "IXP4XX-Flash.0"
0x00fc0000-0x00fc1000 : "RedBoot config"
0x00fe0000-0x01000000 : "FIS directory"
i2c /dev entries driver
i2c-gpio i2c-gpio.0: using pins 7 (SDA) and 6 (SCL)
+
Trying NPE-B...success. Using NPE-B with PHY 0.
Ethernet eth0: MAC address 00:03:47:df:32:a8
IP: 172.16.0.99/255.255.255.0, Gateway: 0.0.0.0
Default server: 172.16.0.199
Change WDT to 'boot mode' timeout (60 seconds)

RedBoot(tm) bootstrap and debug environment [ROM]
Non-certified release, version UNKNOWN - built 11:27:43, Feb 19 2008

Platform: UWP01 (IXP42X 266MHz) BE
Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.
Copyright (C) 2003, 2004, 2005, 2006 eCosCentric Limited

RAM: 0x00000000-0x02000000, [0x00025b60-0x01fc1000] available
FLASH: 0x50000000 - 0x51000000, 128 blocks of 0x00020000 bytes each.
== Executing boot script in 2.000 seconds - enter ^C to abort
RedBoot> fis load kernel
RedBoot> go -n 0x25c00
Uncompressing Linux................................................................... done, booting the kernel.
Linux version 2.6.24.2 (schao@eris) (gcc version 4.1.2) #2 Sun Mar 30 18:09:49 PST 2008
CPU: XScale-IXP42x Family [690541f1] revision 1 (ARMv5TE), cr=000039ff
Machine: Gateworks Avila Network Platform
Warning: bad configuration page, trying to continue
Memory policy: ECC disabled, Data cache writeback
CPU0: D VIVT undefined 5 cache
CPU0: I cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets
CPU0: D cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 4064
Kernel command line: rtc-ds1672.probe=0,0x68 root=/dev/mtdblock2 rootfstype=squashfs,jffs2 noinitrd console=ttyS0,115200 init=/etc/preinit
PID hash table entries: 64 (order: 6, 256 bytes)
Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)
Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)
Memory: 16MB = 16MB total
Memory: 14076KB available (1844K code, 180K data, 96K init)
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
net_namespace: 64 bytes
NET: Registered protocol family 16
IXP4xx: Using 16MiB expansion bus window size
PCI: IXP4xx is host
PCI: IXP4xx Using direct access for memory space
PCI: bus0: Fast back to back transfers enabled
NET: Registered protocol family 2
Time: OSTS clocksource has been installed.
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 512 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 512 bind 512)
TCP reno registered
IXP4xx Queue Manager initialized.
squashfs: version 3.0 (2006/03/15) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler deadline registered (default)
Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing disabled
serial8250.0: ttyS0 at MMIO 0xc8000000 (irq = 15) is a XScale
console [ttyS0] enabled
serial8250.0: ttyS1 at MMIO 0xc8001000 (irq = 13) is a XScale
eth0: MII PHY 0 on NPE-B
eth1: MII PHY 1 on NPE-C
eth1: MII read failed
eth1: MII read failed
IXP4XX-Flash.0: Found 1 x16 devices at 0x0 in 16-bit bank
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Using buffer write method
cfi_cmdset_0001: Erase suspend on write enabled
Searching for RedBoot partition table in IXP4XX-Flash.0 at offset 0xfe0000
5 RedBoot partitions found on MTD device IXP4XX-Flash.0
Creating 5 MTD partitions on "IXP4XX-Flash.0":
0x00000000-0x00060000 : "RedBoot"
0x00060000-0x00160000 : "kernel"
0x00160000-0x00c60000 : "rootfs"
split_squashfs: no squashfs found in "IXP4XX-Flash.0"
0x00fc0000-0x00fc1000 : "RedBoot config"
0x00fe0000-0x01000000 : "FIS directory"
i2c /dev entries driver
i2c-gpio i2c-gpio.0: using pins 7 (SDA) and 6 (SCL)
+
Trying NPE-B...success. Using NPE-B with PHY 0.
Ethernet eth0: MAC address 00:03:47:df:32:a8
IP: 172.16.0.99/255.255.255.0, Gateway: 0.0.0.0
Default server: 172.16.0.199
Change WDT to 'boot mode' timeout (60 seconds)

RedBoot(tm) bootstrap and debug environment [ROM]
Non-certified release, version UNKNOWN - built 11:27:43, Feb 19 2008

Platform: UWP01 (IXP42X 266MHz) BE
Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.
Copyright (C) 2003, 2004, 2005, 2006 eCosCentric Limited

RAM: 0x00000000-0x02000000, [0x00025b60-0x01fc1000] available
FLASH: 0x50000000 - 0x51000000, 128 blocks of 0x00020000 bytes each.
== Executing boot script in 2.000 seconds - enter ^C to abort
RedBoot> fis load kernel
RedBoot> go -n 0x25c00
Uncompressing Linux................................................................... done, booting the kernel.
Linux version 2.6.24.2 (schao@eris) (gcc version 4.1.2) #2 Sun Mar 30 18:09:49 PST 2008
CPU: XScale-IXP42x Family [690541f1] revision 1 (ARMv5TE), cr=000039ff
Machine: Gateworks Avila Network Platform
Warning: bad configuration page, trying to continue
Memory policy: ECC disabled, Data cache writeback
CPU0: D VIVT undefined 5 cache
CPU0: I cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets
CPU0: D cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 4064
Kernel command line: rtc-ds1672.probe=0,0x68 root=/dev/mtdblock2 rootfstype=squashfs,jffs2 noinitrd console=ttyS0,115200 init=/etc/preinit
PID hash table entries: 64 (order: 6, 256 bytes)
Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)
Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)
Memory: 16MB = 16MB total
Memory: 14076KB available (1844K code, 180K data, 96K init)
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
net_namespace: 64 bytes
NET: Registered protocol family 16
IXP4xx: Using 16MiB expansion bus window size
PCI: IXP4xx is host
PCI: IXP4xx Using direct access for memory space
PCI: bus0: Fast back to back transfers enabled
NET: Registered protocol family 2
Time: OSTS clocksource has been installed.
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 512 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 512 bind 512)
TCP reno registered
IXP4xx Queue Manager initialized.
squashfs: version 3.0 (2006/03/15) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler deadline registered (default)
Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing disabled
serial8250.0: ttyS0 at MMIO 0xc8000000 (irq = 15) is a XScale
console [ttyS0] enabled
serial8250.0: ttyS1 at MMIO 0xc8001000 (irq = 13) is a XScale
eth0: MII PHY 0 on NPE-B
eth1: MII PHY 1 on NPE-C
eth1: MII read failed
eth1: MII read failed
IXP4XX-Flash.0: Found 1 x16 devices at 0x0 in 16-bit bank
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Using buffer write method
cfi_cmdset_0001: Erase suspend on write enabled
Searching for RedBoot partition table in IXP4XX-Flash.0 at offset 0xfe0000
5 RedBoot partitions found on MTD device IXP4XX-Flash.0
Creating 5 MTD partitions on "IXP4XX-Flash.0":
0x00000000-0x00060000 : "RedBoot"
0x00060000-0x00160000 : "kernel"
0x00160000-0x00c60000 : "rootfs"
split_squashfs: no squashfs found in "IXP4XX-Flash.0"
0x00fc0000-0x00fc1000 : "RedBoot config"
0x00fe0000-0x01000000 : "FIS directory"
i2c /dev entries driver
i2c-gpio i2c-gpio.0: using pins 7 (SDA) and 6 (SCL)
+
Trying NPE-B...success. Using NPE-B with PHY 0.

shao, this line in your output log shows what is causing your reboot problem:

split_squashfs: no squashfs found in "IXP4XX-Flash.0"

It can't find your root filesystem.  You probably skipped/mistyped a step during the flash process (assuming you flashed it from RedBoot).  I am unfamiliar with the process from inside OpenWRT as I've never done it before, but these are the commands I execute to flash and subsequently boot my Avila from RedBoot:

ip_address -l <local IP> -h <server IP>
fis init
load -r -v -b 0x00800000 openwrt-avila-zImage
fis create linux
load -r -v -b 0x00800000 openwrt-ixp4xx-squashfs.img
fis create -l 0xe60000 rootfs
fis load linux
exec

Substituting the correct IP addresses for your Avila and TFTP server, of course.

But back to my issues.  I checked out trunk 10690 to my server (the Linux box responsible for compiling OpenWRT) and built a set of images.  After getting my Avila re-flashed it is now performing to my expectations.  All after I added my own software, configuration, etc, etc...

Thanks Kaloz and forum2008 for the suggestion.  To be honest I had discounted trunk as I was under the impression that it would not be a working release.  I suppose I was wrongly projecting my experiences from other projects onto OpenWRT.  My apologies, I'll know better next time, and won't have to bother the rest of you

(Last edited by sadpengu on 31 Mar 2008, 05:11)