OpenWrt Forum Archive

Topic: [HOWTO] WNDR3700 Openwrt Backfire 10.3 - Rootfs on External Storage

The content of this topic has been archived on 7 Jul 2014. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hello this is my first contribution to this project. I just bought this router, Netgear wndr3700, and have been trying to work with openwrt backfire 10.3 on it. It was not easy, many information is scattered around the wiki and forum. It’s my sincere hope this helps to at least avoid the initial information gathering that I had to do to get this setup. Comments, suggestions are most welcome.  I'll be posting a some more about 3G/UMTS usb modem and this router buttons. Thanks, and here it goes.

1 - Install jffs2 rom image, openwrt-ar71xx-wndr3700-jffs2-factory.img with tftp.
Squashfs works but only if custom built with the usb stuff and rootfs.


2 - Base packages for this router:

#USB Led for wndr3700
kmod-leds-wndr3700-usb

#for Wireless lan
wpad-mini kmod-ath9k

#for usb
kmod-usb2 kmod-ohci kmod-usb-storage

#for rootfs
kmod-ext2 kmod-ext3 e2fsprogs block-mount block-extroot

Install them with:

opkg update
opkg install kmod-leds-wndr3700-usb wpad-mini kmod-ath9k kmod-usb2 kmod-ohci kmod-usb-storage kmod-ext2 kmod-ext3 e2fsprogs block-mount block-extroot

Reboot (reboot) so that all modules get loaded in the correct order.

Optional:
Configure this environment (internal router flash) with some sane defaults for your setup, so that if external usb fails, minimal services will be available, like wireless, dhcp lan and wan access.


3 - Prepare the external usb storage for use.

Create partitions for root, tmp and swap. Use fdisk
Format root and tmp partitions with mkfs.ext3
Prepare the swap partition with mkswap

Note: If you don’t know how to use this tools then you probably need to do some more reading before continuing.


4 - Create / Edit your  /etc/config/fstab as:

config global automount
      option from_fstab 1
      option anon_mount 0

config global autoswap
      option from_fstab 1
      option anon_swap 0

config mount
      option target   /mnt/usbroot
      option device   uuid
      option uuid     5c72eb36-d9d8-439d-b91b-c0ab12b686f9
      option fstype   ext3
      option options  rw,sync,relatime,errors=remount-ro
      option enabled  1
      option enabled_fsck 1
      option is_rootfs 1

config mount
      option target   /mnt/usbtmp
      option device   uuid
      option uuid     02d4d4ef-b1ca-463b-ab1b-e7325614517c
      option fstype   ext3
      option options  rw,noatime
      option enabled  1
      option enabled_fsck 1

config swap
      option device   uuid
      option uuid     ecaf5815-fbb7-4cd9-8ed2-e8583b7e76a3
      option enabled  1

The key here is option is_rootfs 1. That is what makes the selected device your rootfs.
uuid can be checked with blkid command. Create the folders /mnt/usbroot and /mnt/usbtmp.
I chose ext3 because this router, for home use, can and will be turned off at any moment, since ext3 is more resilient to this kind of behavior it is better suited for this. USB pen drives are cheap now so flash wearing is not an issue, it should last a very long time anyway. Just remember to backup if you make any configuration changes so that you don’t have to do it all over again.

More details on fstab and rootfs see:
Fstab Configuration http://wiki.openwrt.org/doc/uci/fstab
Rootfs on External Storage http://wiki.openwrt.org/doc/howto/rootfsonexternalstorage


5 - Reboot (reboot) to get your new and shinny external rootfs.

Check with mount if you are now on external storage, you should have this:

rootfs on / type rootfs (rw)
/dev/root on /rom type jffs2 (ro,relatime)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
tmpfs on /dev type tmpfs (rw,relatime,size=512k)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,size=30988k)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
/dev/sda1 on /overlay type ext3 (rw,sync,relatime,errors=remount-ro)
mini_fo:/overlay on / type mini_fo (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda2 on /mnt/usbtmp type ext2 (rw,noatime,errors=continue)
none on /proc/bus/usb type usbfs (rw,relatime)

So now /rom will be jffs2 and mounted ro. Your external device should be mounted at /overlay, see the line /dev/sda1 on /overlay type ext2 (rw,relatime,errors=remount-ro).

At this point you are now working on the usb external drive. No changes you make from here onward will be done one the router internal flash.


6 - Let’s install some more stuff and replace /tmp on ramdisk (tmpfs) with /tmp on our external drive.

Now create tmp2usb in /etc/init.d/tmp2usb as:

#!/bin/sh /etc/rc.common

START=21
STOP=99

. /lib/functions/mount.sh

start() {
      rm -rf /mnt/usbtmp/*
      cp -a /tmp/* /mnt/usbtmp/
      umount /tmp       
      [ $? -ne 0 ] && {
                  umount -l /tmp
                  echo "tmpfs lazy removed!"
      } || echo "tmpfs removed!"
      mount --move /mnt/usbtmp/ /tmp
      echo "/tmp on usb drive!"
}

stop() {
      rm -rf /tmp/*
      sync
      umount -r /tmp
}

Do /etc/init.d/tmp2usb enable, to create symblinks where needed.
Check that the file was actually created at /overlay//etc/init.d/tmp2usb as expected.

Reboot and check mount again:

rootfs on / type rootfs (rw)
/dev/root on /rom type jffs2 (ro,relatime)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
tmpfs on /dev type tmpfs (rw,relatime,size=512k)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
/dev/sda1 on /overlay type ext3 (rw,sync,relatime,errors=remount-ro)
mini_fo:/overlay on / type mini_fo (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda2 on /tmp type ext3 (rw,noatime,errors=continue)
none on /proc/bus/usb type usbfs (rw,relatime)

See that /tmp is now on your external device


7 - I also enabled umount init script, /etc/init.d/umount, just to be on the safe side with manual reboots.


8 - Optional, modify /etc/banner to show that you are running on external rootfs. In my case I just added this to the existing file.

-------------------- On Usb -----------------------

9 - Want a more permanent storage of syslog messages? Remove symbolic link /var to /tmp and mkdir /var. Then uci set system.@system[0].log_type=file.

(Last edited by DavidMorgado on 10 Aug 2010, 10:50)

Thanks for the info.

So is using an external and jffs mandatory with your setup?? I have a WNDR3700 and I have a older prebuilt image that I want to upgrade.

Why can't I just install the packages you describe into a squashfs installation of OpenWRT?

Also, maybe not a question for you, but why do I need to install all these packages in what is supposed to be a tailored disk image for the wndr3700?

(Last edited by napierzaza on 2 Aug 2010, 14:41)

napierzaza wrote:

Thanks for the info.

So is using an external and jffs mandatory with your setup?? I have a WNDR3700 and I have a older prebuilt image that I want to upgrade.

Why can't I just install the packages you describe into a squashfs installation of OpenWRT?

I tried just that and it didn't work. My guess is that the necessary modules are not available until jffs overlay is mounted and at that point it's to late. For this to work with a squash image it must be custom build, and I didn't want to do that myself just yet.

napierzaza wrote:

Also, maybe not a question for you, but why do I need to install all these packages in what is supposed to be a tailored disk image for the wndr3700?

kmod-usb2, kmod-ohci, kmod-usb-storage, block-mount and block-extroot are minimal for this setup, someone correct me if I'm wrong. And you need some file system support so kmod-ext3 or any other you might want to use is necessary also.

All other are optional, but you'll loose some functionality, like wireless without them.

Edit: I misunderstood that, ok some devs might reply to that. Maybe next backfire update everything will just work, I checked trunk and it does except the router's switch witch stops accepting configuration options like changing led colors and behaviour.

Good luck

(Last edited by DavidMorgado on 2 Aug 2010, 18:22)

Sounds like I still want to wait before I leap into this.

When you say it didn't work under jffs, did you mean the external drive setup? Or the router's network/wireless/leds etc??

Some confusion here. I was referring to external root, on a usb stik, doesn't work with squashfs pre-made image, backfire 10.3, because it is missing some kernel modules and the actual block-extroot package.

Thank you! This is exactly what I was looking for! I have had the WNDR3700 for a couple of weeks now, used OpenWrt on my old Asus WL-500gP which died a month ago after having served me for ~4 years, it was rock solid till the end smile

I played around with 10.03 just when I got the router, and experienced some problems configuring the wireless for optimal performance - the stock firmware gives me around 13-16MiB/s WiFi<->LAN while OpenWrt 10.03 gave me 5-7MiB/s (on the 2.4GHz radio). I also experienced not being able to boot after having restarted a couple of times, so I had to save the router by TFTP'ing the stock firmware image back to it.

How long time has your setup been running? What speeds are you getting WiFi<->LAN wise?

I am planning to give OpenWrt another go on the WNDR3700 once 10.03.1 is released, will this howto, to your knowledge, also apply to 10.03.1?

Best Regards
Martin

(Last edited by clemme on 9 Sep 2010, 21:18)

DavidMorgado wrote:

mount --move /mnt/usbtmp/ /tmp

This should be:

      mount /mnt/usbtmp/ /tmp -o move

Also, when the script  /etc/init.d/tmp2usb is executed at startup the partitions may not be ready yet (it happened to me), causing a failure to mount the partitions. As I was not interested in a rootfs on USB, I have restructured the script as follows (also removing the need for /etc/config/fstab ):

#!/bin/sh /etc/rc.common

START=40
STOP=99

# start after USB (39) but before other apps possibly installed on external disk

. /lib/functions/mount.sh

start() {
    i=0
    while [ $i -lt 10 ] ; do
        swapon /dev/sda3 && break
        sleep 1
        i=$((i + 1))
    done
    if [ $i -ge 10 ]; then return; fi
    # here we are sure the flashdisk is ready
    logger -p info "swap set to ON after $i seconds of retries"
    umount /mnt/sda1 2>/dev/null # optional
    mount /dev/sda1 /opt
    # quit if /dev/sda2 is already mounted on /tmp
    mount | grep -q '/dev/sda2 on /tmp' && return
    mkdir -p /mnt/usbtmp
    umount /mnt/sda2 2>/dev/null # if /dev/sda2 was automounted there
    # quit if mount of /dev/sda2 on /mnt/usbtmp fails
    mount /dev/sda2 /mnt/usbtmp || return
    # replace last image of /tmp on flashdisk with current /tmp on tmpfs
    # WARNING: this erases the content of /dev/sda2 on the flashdisk!!!
    rm -rf /mnt/usbtmp/*
    cp -a /tmp/* /mnt/usbtmp/
    umount -l /tmp
    # now move the mount point to /tmp, which will lazily release /dev/tmpfs
    #mount --move /mnt/usbtmp/ /tmp
    mount /mnt/usbtmp/ /tmp -o move
}

stop() {
    swapoff /dev/sda3
    mkdir -p /mnt/usbtmp # just in case it didn't exist
    umount /mnt/usbtmp 2>/dev/null # just in case soething was mounted there
    mount -t tmpfs -o size=14M,mode=0777 tmpfs /mnt/usbtmp
    cp -a /tmp/* /mnt/usbtmp/
    umount -l /tmp
    # now move the mount point to /tmp, which will lazily release /dev/sda2
    mount /mnt/usbtmp/ /tmp -o move
    umount /dev/sda1
    sync
}

We are assuming that the disk installed on USB is partitioned as:

Partition    type           mount
======  ======= =====
/dev/sda1 Linux          /opt
/dev/sda2 Linux          /tmp
/dev/sda3 Linux Swap (swap)

Running "/etc/init.d/tmp2usb start" will mount /dev/sda1 under /opt. In order to be able to install packages there, this is the recommended procedure:

- Add to /etc/opkg.conf the line:
dest opt /opt

- Edit /etc/ld.so.conf adding:
/opt/lib
/opt/usr/lib

- Install ldconfig:
opkg install ldconfig

- Add to /etc/profile the line:
export PATH=/opt/bin:/opt/sbin:/opt/usr/bin:/opt/usr/sbin:$PATH

Reboot; from now on optionally install packages under /opt with:

opkg --dest opt install ...
ldconfig

Executing "/etc/init.d/tmp2usb stop" brings the configuration bach to the "no-usb" state: no swap active on USB, nothing mounted under /opt, and a 14Kb tmpfs under /tmp .

YMMV...

Enzo

(Last edited by enzo on 19 Sep 2010, 15:34)

I'm trying to get this to work, but it keeps on failing... I've followed all the steps (both from this start-post as also from the extroot wiki page), but it seems the "option is_root 1" is being ignored...

My /etc/config/fstab:

config global automount
        option from_fstab 1
        option anon_mount 0

config global autoswap
        option from_fstab 1
        option anon_swap 0

config mount
      option target   /mnt/usb/root
      option device   uuid
      option uuid     fd7a22e9-73d3-4897-afce-dedaebb39247
      option fstype   ext4
      option options  rw,sync,relatime,errors=remount-ro
      option enabled  1
      option enabled_fsck 1
      option is_rootfs 1

config mount
      option target   /mnt/usb/tmp
      option device   uuid
      option uuid     66610291-b4a4-460e-8e39-a064e0143907
      option fstype   ext4
      option options  rw,noatime
      option enabled  1
      option enabled_fsck 1

config mount
      option target   /mnt/usb/data
      option device   uuid
      option uuid     32b3039f-0314-41fa-b49a-0e7bfd78a758
      option fstype   ext4
      option options  rw,sync
      option enabled  1
      option enabled_fsck 1

config swap
      option device   uuid
      option uuid     3b088da8-7838-4e7b-b5b7-310748dba844
      option enabled  1

The output of mount after a reboot:

rootfs on / type rootfs (rw)
/dev/root on / type jffs2 (rw,relatime)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,size=30988k)
tmpfs on /dev type tmpfs (rw,relatime,size=512k)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda2 on /mnt/usb/root type ext4 (rw,sync,relatime,errors=remount-ro,barrier=1,data=writeback)
/dev/sda3 on /mnt/usb/tmp type ext4 (rw,noatime,barrier=1,data=writeback)
/dev/sda4 on /mnt/usb/data type ext4 (rw,sync,relatime,barrier=1,data=writeback)
none on /proc/bus/usb type usbfs (rw,relatime)

As you can see, my rootfs is still mounted as rw and
/dev/sda2 on /overlay type ext4 (rw,sync,relatime,errors=remount-ro)
is missing...

Could this be because I'm using ext4? This shouldn't make any difference... I've got kmod-fs-ext4 installed (as is obvious from the fact that the partitions are mounted and accessible...)

Can anyone please help me, I'm stuck here, but I need the extra space for additional software...

Ps.
There are a few typo's in the startingpost for the essential packages.
kmod-ext2 => kmod-fs-ext2
kmod-ext3 => kmod-fs-ext3
kmod-ohci => kmod-usb-ohci.

Furthermore, after changing /etc/config/fstab, you need to enable it, since is disabled by default:

/etc/init.d/fstab enable

(Last edited by deepbass909 on 2 May 2011, 13:23)

It is possible that your versions of kernel and/or mount require a syntax different from the one I was using at the time. For instance, I'm pretty sure that the names of the modules you mention were correct on my system. Unfortunately, I can't check because I decommissioned the router a while ago...

(Last edited by enzo on 2 May 2011, 14:48)

Fixed smile
Apparently the extroot partition can't be ext4... I've reformatted the partition to ext3 and behold:

/dev/root on /rom type jffs2 (ro,relatime)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,size=30988k)
tmpfs on /dev type tmpfs (rw,relatime,size=512k)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
/dev/sda2 on /overlay type ext3 (rw,sync,relatime,errors=remount-ro,data=writeback)
mini_fo:/overlay on / type mini_fo (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda3 on /mnt/usb/tmp type ext4 (rw,noatime,barrier=1,data=writeback)
/dev/sda4 on /mnt/usb/data type ext4 (rw,sync,relatime,barrier=1,data=writeback)
none on /proc/bus/usb type usbfs (rw,relatime)

/dev/root is ro and /overlay is mounted on /dev/sda2 smile

(Last edited by deepbass909 on 2 May 2011, 14:59)

The discussion might have continued from here.