After spending nearly a day "configuring" OpenWRT (stable RC4) by altering
most init scripts, effectively forking my own private distro, I've decided to
post here my reasons for doing so. Hopefully this post will be perceived as
constructive criticism and not merely flaimbait.

My reason for choosing OpenWRT was that I could use the full power of a shell
based distro and not be limited by some web interface (Linksys or some other
<cool name>WRT). Unfortunately, I've discovered quickly that the limits of the
web interface were preserved in a lower layer, i.e. the miscrosoftish nvram
registry, which OpenWRT unfortunately adopted.

Perusing some of the posts here, I've discovered that using the nvram as
master config was a design decisions of the OpenWRT team. I'm trying to
argue two points: (1) that keeping the nvram as master config is wrong for a
shell focused distro, and (2) that similar Linksys compatibility can be
achieved by a less annoying import/export procedure.

(1) The nvram registry makes perfect sense for a firmware that lacks a
    writable file system, and is web focused, i.e. Linksys and clones.
    The "param=value" paradigm fits web applications like a glove.

    When a writable file system is available, keeping the nvram as master
    config is a bad decision. Many important kernel features (e.g. iptables
    firewall) and applications (e.g. dnsmasq) require more powerful languages
    than "param=value" offers. The OpenWRT compatibility with Linksys firmware
    is fairly limited anyway, as as illustrated by OpenWRT ignoring the
    important wk_mode parameter. Having the init scripts rewrite the firewall
    rules on every boot to fit wk_mode, which thankfully is not done,
    illustrates the limitations of the nvram as master config. OpenWRT has to
    play a constant catchup game if it really wants to keep up with the
    plethora of Linksys parameters.

    I found the master nvram approach very annoying in some specific cases. An
    example is the deletion at boot time of /etc/resolv.conf if
    lan_dns is not set, even if lan_proto=static. An even more annoying one
    was the dnsmasq init script: it added wrong DHCP ranges, which were
    impossible to fix using the nvram. I have multiple subnets connected to
    the WRT, so a single nvram "lan" setting is not feasible, whereas I can
    easily specify them in dnsmasq.conf.

(2) So what's the solution? Mine was to effectively import the settings into
    /etc files, sometimes manually, e.g. for wk_mode, and _prevent_ the boot
    scripts from subsequently changing them. The import could easily be
    automated in the firstboot script. The import approach (as opposed to
    master) has the added benefit that one can use config files from most
    other LSB (Linux Standards Base) disk based distro's. E.g, I've imported
    quagga rip config from a disk based box; quagga init script thankfully
    didn't read the nvram at all.

    I personally don't care at all about exporting to nvram. Most people
    choosing to use OpenWRT would only care about import anyway, as long as
    the nvram is preserved, which will allow them to back out of OpenWRT if a
    shell based router is too rich for them. Exporting the config of features
    not present in Linksys firmware (e.g. wifi_*) is pointless anyway.

    Most people using a web based WRT distro are probably unaware of the nvram
    language because they don't use it directly, while most admins would
    probably find (like me) that using the nvram registry means learning new
    config settings, which don't have any advantages but plenty of
    disadvantages in a shell router.

It is very difficult for me, not to flame you now. Because you spent only a day and did not understand anything about the reasons behind the scene.

There are some good reasons to have some backward compatibility.
1. Your router would not boot without nvram. You can't just remove it and use it for something else.
CFE, the bootloader on most of the systems we support in whiterussian, checks nvram for some stuff. (boot_wait as example).
2. Users which like to try OpenWrt are happy that their old configuration is used to setup the new system.
(just network configuration)
3. With 2.4 kernel and the binary only wl.o driver for the wireless chip, we need nvram to configure some of the wireless settings. et.o, the driver for the switch use nvramto setup vlan. 

What do you think we are trying todo in the future? If it would be such a simple task, we already had it working.
Please feel free to submit patches. In Kamikaze our base configuration will be saved in /etc/config.
So please make a better research next time. You can see our efforts in our subversion repository.
Next release will have support for a lot more systems without nvram. (AR7/Soekris/Netgear WGT634U/..)

The boot scripts never change NVRAM permanently (one exception boot_wait, which is needed for recovery of a broken system). There are only some overwrites in RAM.

Feel free to submit patches and try Kamikaze. http://dev.openwrt.org

the problem is that some of the non-free binary-only tools (eg nas) read the nvram for parameters.

i agree the nvram is rather limiting and arcane compared to normal scripts and config files though. redhat's /etc/sysconfig/network-scripts is pretty good for configuring and controlling interfaces.

Keeping the interface network config in nvram is not the issue. After all, the
Redhat ifcfg scripts are nothing more than param=value pairs. The location is
not that relevant except to network init tool/scripts. If it has to be the
nvram for compatibility reasons, that's not really a problem. I didn't change
the location of those settings in my setup, simply because I didn't care where
they were.

The problems start when one tries to extend the param=value approach to more
complex apps: dns, dhcp, firewall. AFAICT, all the indispensable binary-only
tools (CFE included) deal only with the hardware and network interface
config. There's no good reason to shoehorn the other apps to use the nvram.

We're in different circles (and probably different countries) with
different philosophies. Most IT people I know will not spend a day
configuring something, they'll just buy/use another box/software which
can do what they want without having to bend over backwards. I consider
spending a day on this a reasonable time investment on my part.

I agree with all 3 points, none of which support using the nvram for
anything beyond hardware and (layer 2) network interface setup. I'm
not convinced the drivers need to know any layer 3 stuff (IP address
etc.), but I've seen my fair share of silly closed source software
before, so I'll assume you are right.

I'm sorry to have to say this, but the OpenWRT teams in not doing a
good job at letting people outside where White Russian is headed, and
even less so for Kamikaze. I couldn't find any mention of it in any
sticky posts or site news. There's no TODO file or even a changelog in
the source tree. If one has to build a mental picture from the svn
commit messages, then it's really more than most would be developers
would normally do. I'm not the only one to think this:
http://forum.openwrt.org/viewtopic.php?id=3482. So, don't blame me for
not guessing what you're planning for the future.

I agree that making a unified config for all the hardware platforms
will be a nasty problem if various platforms need config bits in
different places for the binary-only stuff. Presumably you'd have to
export enough stuff from /etc/config to the right places on each
platform. BTW, the search feature on this board ignores '/'
characters, so I couldn't find the precise posts which mention
/etc/config.

I'll be the first to admit that there are flaws in the development model but please refrain from flaming the developers based on your initial reaction. I appoligize if it wasn't actually ment as a flame but it certainly came across that way.

Using nvram:
- nvram is required by the bootloader and various proprietary drivers and utils
- nvram is a fixed size and location, you cannot reclaim the space
- nvram is preserved across all upgrades

Creating a config file on the filesystem:
- duplicates many values alreay in nvram
- may require an additional parser
- may be difficult for a script or web interface to modify the configuration
- is not preserved across upgrades; the filesystem gets wiped out when you reflash with a newer release

EDIT -

OpenWrt's policy is to only use nvram when the configuration calls for a simple <key>=<value> already in nvram; it will not try to create anything more complicated in nvram. As an example, SSH keys and firewall rules are kept on the filesystem.

The /etc/resolv.conf is a link to /tmp (ramdisk) to avoid writing to flash each time the router renews it's dhcp/ppp connection or a lan_dns variable which can be used when lan_proto=static. It's simply us trying to avoid wearing out the flash with continuous writes; you can just delete the /etc/resolv.conf symlink and create your own.

(Last edited by mbm on 11 Dec 2005, 07:51)

how does openwrt handle platforms without nvram?

(Last edited by bani on 11 Dec 2005, 10:38)

I had similar thoughts about an nvram free configuration after my previous nightmares with other (DD-WRT, Sveasoft) releases due largely to nvram confusion. But after using OpenWRT for a few weeks I came to not only appreciate OW's minimal dependence on nvram, but also the utility of nvram for some purposes.

I routinely clear nvram after installing OW (not recommended on some models!) to rid all the linksys garbage, but then set only a few vars for minimal config instead installing an init script to basically customize the configuration. Virtually anything that can be specified in nvram can be overridden in a startup script, but even so I've even added a few nvram vars myself (that my startup scripts reference) because these are easier to reference via a web page or step a customer thru changing via telnet. For obscure reasons my clients are not currently remotely accessible (tunneling, vlan, etc) so if I need to step a customer thru changing xmit antenna the last thing I want to do is to step them through vi to edit the startup script .

The lack of a documented "path", true, but being a one man developer of a major software package myself (content management system) I fully appreciate that documenting, providing roadpaths's etc is a massive amount of work, equal to or greater than the sw development itself, and with limited resources I'd prefer that development take precedence. From what I've seen so far I have trust in the sanity of the roadpath for openwrt.

- DL

(Last edited by dl on 11 Dec 2005, 10:34)

A set of fles in /etc/config/ which use the same key pairs as nvram. The files are named after the set of variables they contain, in other words there's a /etc/config/network which contains the various lan_* and wan_* variables used by networking components like the ifup script.

i dont think he was suggesting platform-specific nvram settings be removed. but i don't think _openwrt_ should rely on platform-specific nvram settings. boot_wait should matter only to CFE, not openwrt (as an example).

i'm not sure this is a good argument for using nvram for all settings. i'm not sure backwards compatibility with linksys (dd-wrt, etc) firmware is all that good of an idea, as it's rather limiting. and on some platforms (routerboard, soekris, gumstix, etc) it doesn't apply at all.

i think this is a good reason for openwrt to adopt opensource driver when they become usable

(Last edited by bani on 11 Dec 2005, 10:42)

well imagine a setup with 30, 50, 100 ethernets/subnets/vlans. at some point nvram will be insufficient to store the basic system config, even if the hardware is easily capable of it

Which will happen when they become usable ...

So what's the problem? You just break the bridge in nvram (so you don't have to tear it down in a script) and setup everything with a script.

- DL

...which defeats the purpose of storing such things in nvram.

...only for this particular case. This is not a common case, isn't it ?

well, i switched from official linksys firmware to dd-wrt to openwrt, and was astonished to find the nvram was about 90% full.

so i think 'keeping compatibility across firmware versions' is not a good reason to keep settings in nvram. only things in nvram should be what's required by the bootloader (CFE) to basically boot the system (like pc bios cmos settings) and nothing more.

(Last edited by bani on 11 Dec 2005, 13:27)

You trashed out your nvram by installing various firmwares that abused it; why are you complaining about OpenWrt's use of nvram? OpenWrt doesn't create any nvram variables and only really uses a handful of the variables in nvram, the rest is just trash from the other firmwares.

i'm saying the claim for using nvram as some sort of compatibility system across different firmwares is a bad justification "they can just upgrade from (svea|linksys|ddwrt|etc) and it will work", especially since some firmwares use different variables names and are in fact not compatible.

but if you erase nvram and reboot openwrt, you can see it does in fact create nvram variables. what you probably meant to say (but worded badly) is that openwrt doesn't create any new custom nvram variables.

but even openwrt has changed variable names a few times (eg obsoleted variables / changed names) so openwrt itself may be leaving trash too

i think it's a good idea to keep nvram clean as possible. just because linksys or other firmware uses block_java=1 i don't think this is a good justification for openwrt to do the same thing

Wrong, because OpenWrt doesn't create any nvram variables by itself
EDIT: at least it doesn't commit any variables automatically. The small amount of default variables that are created are also used by the original firmware.

(Last edited by nbd on 11 Dec 2005, 14:38)

Actually nvram gets initialized with a default set of variables from the bootloader; we're not responsible for what values CFE writes to the nvram and recommend against erasing nvram because some devices have some nasty defaults. OpenWrt will attempt to fix a few hardware bugs by changing the values with the S05nvram script, but to say OpenWrt creates any variables is wrong.

There might have been a few variables which changed due to wireless driver upgrades, but the reset of the system (which is to say the ifup script and maybe one or two of the init.d scripts) still uses the same set of variables they always did.

I'm sorry but you're really restating the obvious here -- Did you really mean to imply that OpenWrt is hell bent on destroying nvram by creating all sorts of trivial, meaningless, random and obscufiated variables ... ?

Nobody ever said that we were going to maintain full backwards compatibility with every other firmware in existance, only that we happen to have some backwards compatibility because we use some of the same variables.

whats the nvram commit doing in S05nvram then

(Last edited by bani on 11 Dec 2005, 15:22)

It's there because we're forced to. Some routers have a nasty tendancy to crash and won't work properly until you commit a few changes and reboot.

Seriously, I'm getting sick of this thread and the conspiracy theories ...

Then please buy another box.

Use IRC and try to get in contact with us. I personally have no fun in writing down what we want todo.
I'll better spent the time to actually do something. Shut up and hack is my philosophy.

Purist: This response from the developers may seem harsh but it is fair.  They work tirelessly on OpenWRT and provide it to you "as is", free of charge.  For OpenWRT to be as useful as it is and be free is the wonder of Open Source Software, and the people who write it.  On the other hand, with OSS you generally don't get 24/7 customer service and thorough documentation unless you pay for it.  That said, OpenWRTs documentation has improved in leaps and bounds over the past 6 months and is continues to do so - there are useful additions to the documentation wiki every day.

If you want to play with bleeding edge software, be prepared to do a bit of work yourself.  If you want a product that works, right here, right now, then expect to pay for it, and don't expect it to have bleeding edge features.  If you think that OpenWRT should be better documented, then contribute some yourself for the benefit of everyone.  If you want to know where OpenWRT is headed - ask the developers!  I can imagine that the future of OpenWRT is not set in stone - it's difficult to outline a "roadmap" for OpenWRT when there isn't even a road to travel on.