This patch gives 320KiB more freespace (kernel:128 + rootfs:192) on a WRT54G v2.0 w/buildroot as of 2005-02-06.

Before:

@wrt54g:/# df -k
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/root                  896       896         0 100% /rom
/dev/mtdblock/4           2240       352      1888  16% /
@wrt54g:/# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00010000 "pmon"
mtd1: 003b0000 00010000 "linux"
mtd2: 000d27f1 00010000 "rootfs"
mtd3: 00010000 00010000 "nvram"
mtd4: 00230000 00010000 "OpenWrt"

After:

@wrt54g:/# df -k
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/root                  704       704         0 100% /rom
/dev/mtdblock/4           2560       352      2208  14% /
@wrt54g:/# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00010000 "pmon"
mtd1: 003b0000 00010000 "linux"
mtd2: 000aa400 00010000 "rootfs"
mtd3: 00010000 00010000 "nvram"
mtd4: 00280000 00010000 "OpenWrt"

Nothing new on the rootfs part, of course, thanks to Oleg's work.
The kernel loader by Oleg, too, which is slightly based on mjn3's idea and using the decompression code in Igor Pavlov's LZMA SDK.

This includes my CRC error fix.

Edit: The patch has been updated - Now this should also work on the units with 8MiB RAM/16-bit flash.

WoW! I'am (as usual) very interested in this, beause it makes the opportunity for my firmware project to run on the old/very new WAP54g devices also. (V1.0 has 2 Mb flash, V2.0-de has 4 Mb and the actual version selled here in DE obviously only has 2 Mb again). That patch may shrink the firmware below the threshold of 2 Mb...

Anyway - do you see any drawbacks, if I make the general assumption that only 8 Mb of RAM are available? Gunzipped kernel size is 1642496/0x191000, this leaves 454656/0x6f000 bytes on top of expanded kernel.

TIA, Sven-Ola

Changing the top address to 8MiB should be pretty safe, perhaps even 4MiB is ok but I can't tell how much it would go in reality with the kernel features people out there might enable/add, so it might not.

I don't have any of those units, but I can try the BZ_MEM_TOP:=0x80800000 version on my WRT at least, maybe later tonight or tomorrow.  This doen't guarantee it working on WAP, but it'll never work otherwise.

If you are going to try it yourself, tripple check the boot_wait is enabled before trying.

Thanks for the tipps. Boot_wait? If something's really wrong, I have a serial console and even a JTAG cable to revive that thing. Especially the JTAG will help you too one day - because you fiddle a lot with that flash/boot/kernel stuff. So the time will come you need that JTAG badly Ask google for "jtag hairydairymaid" and heat up your soldering gun...

Sorry about underestimating you. (lol) I do have a serial console and JTAG already.  (Actually, I am switching from *that* firmware to OpenWRT.)

Then, please try it and post the results.  I'll update the patch after hearing it works on WAP.

fyi - BZ_MEM_TOP is just for the bzip2 decompression

Just to warn everyone - mjn3 code will not work on units other than wrt54g - it uses byte access to flash, which does not work with 16-bit configurations like wl500g has. Also, there is a problems with caches - both data and instruction - mjn3 code never worked for me (also, it was never used by openwrt - probably due to mistake in the build scripts)...
In fact, I've already made lzma kernel decompressor, which works fine with wl500g/wl500g deluxe boxes and sent it to Kaloz a month ago...
I'm attaching it just in case. The code is intended to be compiled outside kernel tree and does not require recompilation when kernel changes. It's just occupies first trx partition, leaving second and third for kernel and filesystem. This makes it very easy to use. Also, it should work with 8MB of RAM units. The kernel image should be compressed using lzma stream mode. Anyway, check README and let me know if you've questions.

Also, usage sample (this is not openwrt, so for reference only; loader is built in the separate directory called loader)

    $(MAKE) -C $(SRCBASE)/lzma/SRC/7zip/Compress/LZMA_Alone/
    cat $(LINUXDIR)/arch/mips/brcm-boards/bcm947xx/compressed/piggy | 
        $(SRCBASE)/lzma/SRC/7zip/Compress/LZMA_Alone/lzma e -si -so > $(PLATFORMDIR)/vmlinuz
    trx -o $(PLATFORMDIR)/linux.trx 
        loader/loader.gz $(PLATFORMDIR)/vmlinuz $(PLATFORMDIR)/target.cramfs

Wow, Oleg, you are so fast!

I was wondering why mjn3's code hasn't been used.  Searched and asked in the forum but didn't get a clear answer.  I noticed the decompression code loading address issue and omission of proper cache initializations, suspected there might be something in the flash access, but it didn't think about the fetching unit.

I initially had the cache problem, too.  I figured out that a piece of the code zero-clearing the bzip control structure was what's making the cache to work.  I tried the cache_init(), invalidation, flush, etc. from misc.c/bcm4710_cache.h, but I coulnd't make it work, so I put a klugde of just reading a block of RAM sequencially.

Ah, good idea.  I knew there was one unused partition in the trx header, but couldn't think about using it for this.  I was thinking of cleaning it up some time by object header/sections manipulations so the decompression code can see the address directly.

Thanks a lot, Oleg.  I'll update my patch for the old buildroot.  Hopefully Kaloz or someone can get a chance to put it in the new buildroot soon. (Even better if the new one becomes official soon.)

I might still try the object header fiddlings some time, though. (I know it doesn't have much of a real value in interms of space savings.)

The reason for that is what actual decompressor code should be reallocated first, and during this phase dcache is used. It needs to be flashed before calling decompressor, as well as icache should be invalidated (to remove stale data).

Edit: The change mentioned below is unnecessary with the compressed image created with either -si or -eos option.

I finally got a chance to give Oleg's loader a try, however, I had to make a minor modification to make it work on my WRT.  It wasn't really a surprise to me as I had to do the same in the patch I posted first, that is to pass down the actual uncompressed size rather than the max-uint32 to the decoding function.

I'll put together a buildroot patch and update the first post in this thread in a few days.

Hm... This patch is NOT needed if you've compressed your kernel (and you should do) using stream mode. It works fine then... Could you please clarify this?

Ah, OK, I missed nuance of the -si option in your example. (and realized that -so didn't add the marker as -si does.)

  -eos:   write End Of Stream marker
  -si:    Read data from stdin
  -so:    Write data to stdout

Worked fine WITHOUT the patch after creating the compressed image with either -si or -eos.

Sorry about the confusion.

Anyway, I've applied your patch (with minor changes) cause it allows to relax restrictions and to use lzma stream regardless of presence of end of stream marker.

BTW, there is a bug in the lzma compressor command line parsing, so it treats leading slashes as an option prefixes. This results in the inability to compress images using absolute path names, that's why I've ended up using cat and pipe.

Thank you. The updated package is here: http://wl500g.dyndns.org/loader/loader-0.03.tar.gz

Hey - your patch seems to work O.K. The WAP54G (2 MB version) now boots my firmware. There are only 128k free for jffs2 - that's eventually not enough for my patched firstboot to run. But it boots failsafe now... Thank You!

@mtakahar/oleg: now it seems to work on a WAP. There where several problems to solve, because the WAP only has 8 Mb of RAM. This causes a lot of trouble with wl.o, because the driver needs -gt 1 Mb of RAM (unfragmented!) in order to load. Not to speak about a firmware upgrade using the mtd tool.

But: With the actual/old buildroot and some further space savings I've seen this line in my dmesg today:

"0b9a54 - 19fe54 linux". Web-UI, OLSR-daemon and some other stuff already included in squasfs. Great. Many thanks again...