OpenWrt Forum Archive

Hi everyone, and welcome to the tutorial to make and flash a ubifs image of openwrt into your currently not-supported kirkwood device.

Why?
I was looking to add Openwrt support to Zyxel nsa310 and nsa325v2 (devices I own), since kirkwood-based devices are very similar I thought why not *cough*save someone else the hours of time that I needed to find all this stuff on my own*cough* helping a bit anyone else that wanted to do the same with his own?

If you want to see your device eventually included in official openwrt releases, please post the info I'm requesting during the tutorial, so I (or others) can add your device-specific settings to the automatic build system. Or do so yourself if you feel brave enough.

DISCLAIMER: I'd like to remind you that ONLY YOU are responsible of any damage caused by following this guide, so BE VERY CAUTIOUS, ASK FIRST IF YOU ARE UNSURE. I repeat, PROCEED AT YOUR OWN RISK.

DISCLAIMER2: This tutorial has not yet been confirmed by anyone else but myself, so be VERY wary. (If you know the way, please read this and point out any mistakes you find before some device gets hurt)

0. Prerequisites:
-Debian or debian-based distro (or a linux/unix system with the right tools, I used Debian jessie, it's easy to figure out yourself the ones for yours)
-serial connecton to the device, as we need to interrogate and setup the bootloader for ubifs booting, most kirkwood devices have a ttl serial debug header somewhere.
-the tools we will use are inside a debian package called "mtd-utils", so install it in your workstation/virtual machine before we start.
-general knowledge of how the u-boot (most common bootloader in kirkwood devices) works and how to configure it
-knowledge of how to connect serial connection to your device

In case you were wondering, yes, I'm not giving the very basic info to reduce the amount of reckless newbies that brick their own device.
This tutorial is for real men only.

The post after this detains a relatively easier way, boot into debian ARM first as instructed in the forums linked below, then follow the commands written in the post.

1. Collecting necessary info:
connect the TTL/serial to your box (sorry, but you need to search how to do so on your own), power it up and stop the booting to get in the uboot console (press some keys as soon as you see text on screen).

1.1 informations about nand flash
write

nand info

in the uboot console, to get informations about the NAND flash chip.

It should answer something like this (this is from a zyxel nsa310):

  Page size       2048 b
  OOB size          64 b
  Erase size    131072 b
  subpagesize      512 b
  options     0x4000101c
  bbt options 0x    8000

these are onboard flash storage technical informations needed to prepare the ubifs, save them for later. The "b" stands for bytes.

1.2 informations about uboot
write

help

in the uboot console

It will answer with a list of all possible commands, post them here too.

Look for the ones like these (again from my zyxel nsa310)

ubi     - ubi commands
ubifsload- load file from an UBIFS filesystem
ubifsls - list files in a directory
ubifsmount- mount UBIFS volume
ubifsumount- unmount UBIFS volume

these are the ubi tools and commands needed to read and boot from a ubifs partition.
If you have these, you won't need to flash the zimage (with dtb embedded) on raw nand.

The tutorial does only hint at flashing the zimage into NAND, as most bootloaders I've seen in kirkwoods can work with ubifs, might add that in the future (also welcoming additions).

If your device lacks these tools, please ask before going further.

1.3 informations about usable space in the NAND
many networking devices use nand partitions to store firmwares or data needed to initialize their hardware, and the bootloader and its envs (settings) need some space too.

So, write

printenv

in the uboot console

and save the output for later. Post it here too (remove the ethaddr line as that is your mac address)

look into the output for a line called "mtdparts"
in my device (a zyxel nsa310) it was inside the bootargs env

bootargs=console=ttyS0,115200 mtdparts=nand_mtd:0x100000(uboot),0x80000(uboot_env),0x80000(key_store),0x80000(info),0xA00000(etc),0xA00000(kernel_1),0x2FC0000(rootfs1),0xA00000(kernel_2),0x2FC0000(rootfs2) root=/dev/nfs rw init=/init

To get the "amount of bytes used" in hexadecimal plus a "0x" that says "the actual number has a long amount of zeroes before this" that we ignore, if you take uboot partition is "100000", converting it it to decimal it's 1048576 bytes, or 1 MB.

uboot_env is "80000" which becomes 0.5 MB, and so on.

Here the only ones that should truly matter to us are uboot and uboot_envs, which means that the first 1.5 MB of space is taken, the rest is space we can use.

In case you are unsure, ask for assistance, or google about hackings of your device to see if that particular partition is useful or not.

Or look up methods to backup such partitions from uboot (using tftp to send them to your PC, or whatever) or how to unbrick your device.

2. finding a dtb for your device:
a dtb is "device tree blob", a binary file the kernel reads on boot to work correctly on your device. Saves a ton of annoyances as it allows devs to compile a kernel ONCE and use it for all boards of the same architecture, by just swapping the dtb file.

A project that has a good quantity of dtb files you can use for your kirkwood device is bodhi's "DebianARM on Kirkwood" project (it has no official name actually).
If you own a GoFlex Home/Net, Pogoplug E02/Mobile/V4, iConnect, Dockstar, Sheevaplug, NSA320, NSA320S, NSA325, NSA310S, NSA310, Topkick, Netgear Stora, ASUS-M25, Iomega ix4-200d, QiZhiTong 501mv2, HP Thin Client T5325, LG N1T1, head here, http://forum.doozan.com/read.php?2,12096
download the latest kernel build, extract it, and in the /boot/dts folder you will find all the dtb files.
Take the one you need, and don't forget to thank him if this works.

2. preparing the rootfs:
We need a openwrt root file system and a zimage, use the imagebuilder to make a "generic kirkwood" target with the packages that you *MUST* have in the image (some devices like say my own nsa310 needed a ethernet driver that wasn't included by default)

If you have no clue of what packages you can/should install, look at the available packages from the /packages folder in the download page you got your imagebuilder from.

you will get a something-rootfs.tar.gz and a something-zimage, among other things.

2.1 embedding the dtb in the zimage
it's unlikely that the bootloader in your box can boot a kernel with dtb files, so we need to use the fallback way.
Take the dtb and integrate it in the zimage, the kernel will figure it out by itself.

cp openwrt-something-zimage zImage_dtb
cat my_device.dtb >> zImage_dtb

this copies the standard zimage into a zImage_dtb and then appends the dtb to the zimage.

3. building the ubifs:

Let's extract the something-rootfs.tar.gz into a folder. This is the root file system.

first, we need to decide if we can leave the zimage inside or outside the ubifs. if your uboot did show the ubi tools as explained above, you can place the zimage in the root of the extracted filesystem and it will boot fine.

If your bootloader cannot work with ubifs, the zimage must stay out of the rootfs and will be flashed to NAND raw since all bootloaders can read raw nand.

Now, making the ubifs is a two-step process.

mkfs.ubifs -r </path/to/the/rootfs> -m <min io size> -e <LEB size> -c <Eraseblocks count> -o </path/to/output/ubifs.img>

generates a ubifs image that cannot be flashed directly by uboot

ubinize -vv -o </path/to/output/flashable_ubifs.img> -m <min io size> -p <PEB size>KiB -s <sub-page-size> <configuration file>

prepares the image for flashing with the bootloader

here the explanation of all the variables:

<min io size> is the "page size" reported above in step 1, in my case 2048.

<LEB size> is the logical erase block size. It has to be calculated.
as of here http://processors.wiki.ti.com/index.php/UBIFS_Support (much more clear than the official docs)
LEB Size Calculations
UBIFS adds two headers (erase-header and volume-id-header) at the start of each NAND block for block identification purposes.

When sub-page feature is not supported then

erase-header is written to '1st page' and
volume-id-header is flashed in '2nd page' of every block,
so remaining 'n-2' pages are available for user-data, Hence LEB_SIZE = $BLOCK_SIZE - (2 x $PAGE_SIZE)

When sub-page feature is supported then

both these headers are packed into '1st page' of the block and
remaining 'n-1' pages are available to store user-data. Hence LEB_SIZE = $BLOCK_SIZE - (1 x $PAGE_SIZE)

in my case you see that there is a "subpagesize 512 b" reported by the uboot.
So my LEB is = 2048 - 512 = 1536 bytes

<Eraseblock count> has to be calculated by dividing total NAND size by the "erase size" In my case the NAND is 128MB -> 134217728 bytes, and 134217728 / 131072 yelds 1024 erase blocks . Debian ARM's mtdinfo -a I ran from the nsa310's debian agrees with this.

<PEB size> is the physical erase block size, this is the "erase size", but in kilobytes, 128 in my case.

<sub-page-size> is the "subpagesize" in the data above.

<configuration file> is a text file you must create that contains ubifs specific settings about what volumes and other things you want to add to your image.
this is an example

[the volume name]
mode=ubi
image=</path/to/output/ubifs.img>
vol_id=1
vol_type=dynamic
vol_name=<volname>
vol_alignment=1
vol_flags=autoresize

vol_type can also be "static" for read-only volumes
vol_flags=autoresize means that at initialization this volume will fill the whole partition space it is set into.
You can also specify a vol_size=200MiB or something instead of autoresize.

4. pre-flashing
preparations for flashing

4.1 changing mtdparts
remember the mtdparts I talked about in point 1.3 above? let's modify them a bit:

mtdparts=mtdparts=nand_mtd:0x100000(uboot),0x80000(uboot_env),0x7E80000(rootfs)

See, now I have a 7E00000 partition, which becomes 126,5 MB, called rootfs.

In case your uboot is ancient and cannot read ubifs, we need another partition for the zImage.

mtdparts=mtdparts=nand_mtd:0x100000(uboot),0x80000(uboot_env),0x200000(zImage),0x7C80000(rootfs)

don't change anything for now, we will do this later.

4.2 loading the image from a tftp server
If your device does has USB or Sata ports (and its bootloader can initialize them), you don't need to use tftp.
The overwhelming majority of kirkwood devices can use USB from stock u-boot, so you may skip to section 4.4.

run a tftp server in your LAN (look up a tutorial)

set uboot environment in a way that your tftp server is recognized later.

setenv ipaddr  203.0.113.32
setenv serverip 203.0.113.1
setenv netmask 255.255.255.0

no need to save these envs permanently, we will use this only once, all future upgrades will be done from inside openwrt which does not need this trickery.

then load the image into ram beginning at RAM offset 0x800000 (that's a place at 8MB into the RAM, most kirkwood devices have plenty, for very low-ram devices adjust accordingly if the image does not fit)

tftpboot <RAM offset> flashable_ubifs.img

4.4 loading from usb or Sata
since most kirkwood devices have usb ports or sata, we can use them instead of playing games with tftp.

place the files you want to flash in the usb drive (or hard drive), with a single partition, formatted as FAT (16 or 32)
write "sata" instead of usb if you are using a device on a Sata port.

usb reset

wait a bit for it to detect the drive
then load the image into ram beginning at RAM offset 0x800000 (that's a place at 8MB into the RAM, most kirkwood devices have plenty, for very low-ram devices adjust accordingly if the image does not fit)

fatload usb 0:1 0x800000 flashable_ubifs.img

5. flashing
Flashing your stuff requires you to know where to flash it. u-boot cannot (afaik) simply flash things into mtd partitions.

So, we need to figure out at what eraseblock we can flash our images.

let's add the size in hex of the partitions before it and at the end add also a erase size (do the calculation in hexadecimal, not decimal or you will screw up the place)

100000 (uboot) + 80000 (uboot_envs) + 20000 (the 131072 bytes erase size in hexadecimal) = 1A0000 --> 0x1A0000

This is the place where we can flash the first thing.

these are the commands, again to be done while inside the uboot console. The erase command is very important, don't skip it.

nand erase <nand offset> <size>

nand write <RAM offset> <nand offset> <size>

so we need to calculate (again in hex) the size of the stuff we are flashing, and take into account the block size.

Let's say flashable_ubifs.img is 5242450 bytes (a bit less than 5 MB, you need to look at the actual size in bytes of that file from the Properties panel or from commandline, not at the rounded-up value you see in the folder view).

Next we need to divide 5242450 by 131072 (the erase block size, from above) to make sure we are telling u-boot to erase an integer number of erase blocks.

In our example the result is 39.99-something in decimal. We need that to be an integer as nand erase can only erase full blocks, not "0.something" of a block, so we bump it to 40, multiply it by 131072 again and finally we can convert into hexadecimal.

So, the new size, rounded up to the nearest block is 5242880 (decimal) or 500000 (hexadecimal) bytes.

Now we can finally give the order (this is again my example)

nand erase 0x1A0000 500000

nand write 0x800000 0x0180800 500000

6. adjusting the boot settings
these set up the mtdparts to be what we wanted and the boot command to what we want (boot command = what happens when booting).

setenv mtdparts 'mtdparts=nand_mtd:0x100000(uboot),0x80000(uboot_env),0x7E80000(rootfs)'
setenv bootcmd_ubi 'setenv bootargs console=ttyS0,115200 ${mtdparts} ${bootargs_ubi}; ubi part rootfs; ubifsmount ubi:<volname>; ubifsload 0x800000 zImage_dtb; ubifsumount; bootz 0x800000'

this sets up the kernel command line passed by uboot to the linux kernel.

setenv bootargs_ubi  'ubi.mtd=2 root=1f02 rootfstype=ubifs rw'

then you need to alter the current boot line to run your new bootcmd_ubi

setenv bootcmd 'run bootcmd_ubi'

saving the settings and then rebooting

saveenv
reset

7. end! Does it work?
Now if everything was done right, you should see the uboot finding and loading the zimage, then the linux kernel loading things, and at the end you should see a "press enter to activate this console" message, pressing enter will activate OpenWrt console.

Now please post here to show off and provide info about what you made and the specs of your device.

(Last edited by starshipeleven on 19 May 2016, 08:53)