Hello,

I'm using openwrt for quite some years and in order to take most benefit of it, I'm using the root file system on an USB stick (my routers are tp-1043nd's).

I have loads of logging (lots of foregin port scans, etc.) constantly being written, which means that from time to time, the USB stick gets damaged for so much use (wearing) and needs to be replaced.

In the past, I tried to implement a OpenWrt RAID1 solution (two USB sticks in mirror), where I could replace a damaged USB stick without loosing the information (and with no downtime), but had no success...

Recently in another personal project I implemented a DIY NAS based on OpenMediaVault (OMV). I'm using an old PC and the NAS OS is running on an USB stick as well. However, OMV seem to be sensitive to the USB wearing issue and use a plugin called "OpenMediaVault flashmemory" (AKA Folder2RAM), which I believe it creates some virtual folders in the device's RAM in order to avoid excessive writing on the USB stick.

I realised that a similar solution would also be useful for OpenWRT, although I know that typically OpenWrt's Router RAM is much more humble than a PC (even though old).

I imagine lots of OpenWRT users also use a USB stick in their routers and would also have similar problems and would therefore be interested in something like that.

Would it be easy to somehow port this functionality to OpenWRT? Would it make sense? I mean, how many people would actually benefit from it?

Any thoughts?

Thanks,
Jabss

mount ext4 with  noatime,nodiratime,commit=600

large commit time prevent filesystem from flushing buffers too often, but creates the risk of data loss in case of power failure.
Logs are not vital data, aren't they ?
Filesystem has journal and it will not die if powered off. Enable fsck in fstab.
In your scripts use "sync" command to make sure something important is written to disk.
Backup root filesystem with tar on important configuration changes. Put tar to another partition with few writes. Copy tar from there to another system. This way you can always restore the system to a new usb stick.

mkdir /tmp/root
mount --bind / /tmp/root
tar c /tmp/root >root.tar
sync
umount /tmp/root /tmp/root

Also may be move to f2fs ?
Note that block-mount support f2fs for non-extroot only in recent trunk. Chaos calmer will not work.
Even the most recent version is limited to ubifs and ext* as extroot.
So, probably will need more than one partition.
This restriction is pure artificial.  It can be removed by commenting the following code in 'block.c' of package fstools/block-mount.

    /*
        if (strncmp(pr->id->name, "ext", 3) &&
            strncmp(pr->id->name, "ubifs", 5)) {
            ULOG_ERR("extroot: unsupported filesystem %s, try ext4\n", pr->id->name);
            return -1;
        }
    */

I tested , f2fs extroot worked

(Last edited by bolvan on 14 Feb 2016, 13:24)

f2fs looks promissing

How can I have it installed in my CC systems?

Thanks,
Jabss

(Last edited by jabss on 29 Feb 2016, 18:44)

Here's a precompiled package (for Chaos Calmer) made by me. Use at your own risk!

Install instructions:
Execute the following commands remotely (ssh) on your router:

cd /tmp
wget http://dl.dropbox.com/s/qci8zn7lq0egago/block-mount_2016-01-10-96415afecef35766332067f4205ef3b2c7561d21_ar71xx.ipk
opkg install block-mount_2016-01-10-96415afecef35766332067f4205ef3b2c7561d21_ar71xx.ipk
reboot

Credits for this go also to this thread, the thread over here and this post on the mailing list!

(Last edited by v10lator on 24 Mar 2016, 16:59)

Thank you v10lator, I will check that out.

For the record, this is what I did to reduce flash wear on a battery-backed TP-LINK MR3040.

I booted into a Linux recovery CD, and used GParted to create a partition table on /dev/sdb (the card), with two MiB-aligned entries: a 500 MB swap, and a non-formatted one using the remaining space. Then I opened a terminal and ran the following commands.

# mkfs.ext4 -O ^has_journal /dev/sdb2
# tune2fs -o journal_data_writeback /dev/sdb2
# e2fsck -f /dev/sdb2
# dumpe2fs /dev/sdb2 | grep features

This creates an ext4 file system on the second partition, with no journal. You can check the absence of "has_journal" in the last command's output to make sure everything went okay. Some people go the extra mile and tweak RAID stripe options to align clusters with erase block sizes, which provides extra performance at no cost, but I haven't tried that yet.

In OpenWRT, I added the following mount options to my fstab. noatime implies nodiratime, so you don't need to add them both.

async,commit=30,noatime,data=writeback,barrier=0,nobh

These give the best performance, at the cost of greatest data loss should there be a power failure (but with a 2000 mAh battery, you'd need a ~4 hour blackout for that). Whenever I need to turn the router off, I SSH into it, stop all processes writing to disk, sync and only then flip the power switch. The rOOter firmware has a "Power Off" page in LuCI; it'd be nice to see how they did it and if it can be turned into a package.

More ideas:

• jabss could compress his logs by piping them through gzip.

• tweak the swappiness value if you also use a swap partition, I set mine to 10.

Furthermore, you can change the I/O scheduler to noop to avoid wasting CPU time in reordering requests, which has no benefit on USB thumb drives or memory cards. Ideally, this should be done by adding elevator=noop to your kernel parameters, but on OpenWRT that requires recompiling the image, which some may not be able or willing to do. A workaround is adding the following at the top of your /etc/rc.local.

sync
echo noop > /sys/block/sda/queue/scheduler
# Add lines for sdb, sdc, ... if you have them

Could you please create that utility for RAMIPS/RT305X target under 15.05.1 (and 15.05 - just in case)?
Thank you in advance!

(Last edited by booBot on 25 Mar 2016, 22:45)

Battery backed APs? That's a topic I'm really interested in (esp. when combined with solar power to re-fill the batteries) but maybe not something for that forum so... If anyone has some tips and/or tricks please PM me.

While syncing greatly reduces negative effects from a sudden power-off it won't eliminate them completely. What I do to shut down down my router is: SSH into it, exec the "reboot" command, wait till the LEDs go off for a split second and pull the electrical cable then. The reason for that is that the router won't reset fully before everything is golden but when you miss the split second of power down it's also no tragedy to pull the cable shortly after as the router will need some time before mounting filesystems writable.

Link me to that firmware and I'll see what I can observe.

Sorry to say but that's not correct: http://www.phoronix.com/scan.php?page=a … &num=2 - https://www.reddit.com/r/archlinux/comm … _for_ssds/ and so on... But on the other hand all this has been measured on desktop PCs, so maybe you're correct. Want to share your sources?

What utility are you talking about? If you mean f2fs: That's not a utility but a file system. I would love to handle you precompiled packages for it but I don't own the hardware and I will never ever handle out completely untested software. Anyway, doing it yourself is not as hard as you think it is. For a first step follow the links in my previous post, they'll give you a great starting point (this f2fs kernel module was also my first "contribution" to OpenWrt). Also google for "openwrt SDK" and if you're stuck: PM me.

II'm asking for the thus patched block utility:

--- fstools-2015-09-15/block.c.orig    2016-03-26 01:24:33.975386509 +0300
+++ fstools-2015-09-15/block.c    2016-03-26 01:24:02.810383306 +0300
@@ -1084,6 +1084,7 @@
     }
     if (pr) {
         if (strncmp(pr->id->name, "ext", 3) &&
+            strncmp(pr->id->name, "f2fs", 4) &&
             strncmp(pr->id->name, "ubifs", 5)) {
             ULOG_ERR("extroot: unsupported filesystem %s, try ext4\n", pr->id->name);
             return -1;

I'm on ramips/rt305x Asus WL-330N3G.

The MR3040 is a portable router and supports battery power out of the box. It's pretty useful if you need to keep your mobility while capturing packets The TL-MR13U in particular looks brutal, with its 10400 mAh battery. Sadly it's not sold in my area and I can't find it on eBay or Amazon either. I do have a TL-PB10400 powerbank, which has the same capacity. If the 90% efficiency claim is true, a small router like the TL-WR703N could run for three days and a half with nothing on the USB port.

Normally, I wouldn't be too keen to unplug the router while it's rebooting, but your method makes sense. I'll keep it in mind.

http://www.ofmodemsandmen.com/downloadsp.html

Hope it can be backported!

My sources all come from desktop systems too. I did some research the first time I set up extroot, and all Linux SSD optimization guides and benchmarks agreed noop was better for SSDs (which USB sticks technically are) and virtual machines. That made sense based on what I knew, although it would be great if I'm wrong and deadline is better, as that's the default in OpenWrt. Guess I need to find some time to do a proper test...

folder2ram is simply a self-contained shell script. You can probably "port" it easily as long as you have all the commands it uses (sed, gawk, mount/umount, grep).

It was made to keep some OpenMediaVault things in RAM, because it was writing and rewriting the same rrd databases very often, amounting to multiple GB of writes per day (the same file is erased/written thousands of times).

OpenWrt does not usually feature high writing packages.

Also, for the same reason, changing schedulers won't be terribly useful regardless of what you choose. The OS of embedded devices lives in RAM.

Now if you were using the router as a NAS or something like that, things would also change.

Sorry, don't know where my head was. Ofc. the kernel module is in official repos and you just need the patched utility. Anyway, my point about the links + google helping you to patch it for yourself still stands. Either that or you wait till somebody with the same CPU arch does the work, but that may take time.

BTW: For some reason the patch from the mailing list didn't work for me, so I rebased it: http://pastie.org/10775085

Yea, benchmarking this would be great. Please don't forget to share the result.

Thanks for explaining, no need for me to look at the firmware then.

//EDIT: Damn, messed something up... Ofc. I still had to look at the ROOter firmware, so I did. A first look shows that this line of LUA code is what produces the shutdown:

    os.execute("poweroff")

and indeed the command poweroff brings the router into a weird state with USB powered down. So now we know how to shut down from console, next thing would be to extract the LUA and LuCI logic and put it into a package.

//EDIT²: Porting was as simple as copying two files. Transforming this into a package should be extremely simple, so expect a package, soon:

//EDIT³: https://www.dropbox.com/s/xsttlygyi7fyb … x.ipk?dl=0

root@OpenWrt:~# wget http://dl.dropbox.com/s/xsttlygyi7fyby4/luci-app-systemstop_1_ar71xx.ipk && opkg
 install luci-app-systemstop_1_ar71xx.ipk && rm luci-app-systemstop_1_ar71xx.ipk

//EDIT⁴:

"Greatest data loss" doesn't tell in the slightest how this completely destroyed my extroot after ~12 hours with a single power failure... Ofc. my "porting" of the poweroff luci app was there, too (I'm trying to adapt it more to OpenWrt by making it just like the reboot app is done). So now I'm sad + don't know what to run as extroot: f2fs won't run for long with just 32 MB RAM (OOM kills even when there's much swap free) and your tuned ext4 is a data bomb.

(Last edited by v10lator on 2 Apr 2016, 08:15)

Didn't notice your edit until now! Thanks for the luci-app-systemstop package.

Also, sorry to hear about your data loss. I've been running extroot on the same USB stick for some months now, without any problems (always taking care to sync before rebooting or shutting down as I mentioned). To my credit, I did warn that these are "extreme" settings that were being used with a battery backup...

This fix landed recently in LEDE trunk. (actually it was done a bit more properly. f2fs was added to the whitelist you just commented out whole, and block is now able to auto-check the f2fs overlay partition on boot, and can also reformat it if the proper tools are installed, the same as it does with ext4).