OpenWrt Forum Archive

Topic: [Howto] build your own iPKG packages

The content of this topic has been archived on 27 Apr 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Maybe this information is of use to someone, so I'll write it down. Maybe it has been mentioned elsewhere already, but you'll have to forgive me then. Better an abundance of information than a shortage, eh?

- Download the latest version of the iPKG tools from ftp://ftp.handhelds.org/packages/ipkg-utils/ (version 1.7 seems to be the latest);
- untar the archive, cd into the just untarred directory, and type 'make install' (make sure you're root);
- open /usr/local/bin/ipkg-build, find the line where it says "outer=ar" (line 159 in version 1.7), and change "ar" into "tar";

That sets up your build environment. From here on it's pretty straightforward. To create a package copy everything you want in it into a directory, in the directorystructure it needs to be on the target device. Also, you need to create a control file with some meta data about the package, and it has to live in the CONTROL directory. e.g. Say I have a binary, foo, and I need it to be available as /bin/foo on the target device. The  procedure would be something like this:

mkdir -p /tmp/foo/bin
cp foo /tmp/foo/bin
mkdir -p /tmp/foo/CONTROL
cat >/tmp/foo/CONTROL/control <<EOF
Package: foo
Version: 0.1
Architecture: mipsel
Maintainer: foo bar <foo@bar.com>
Section: base
Priority: optional
Description: This package provides foo, which does nothing.
Source: http://foo.bar/foo
EOF
ipkg-build -o root -g root /tmp/foo /tmp

The result will be a package /tmp/foo-0.1_mipsel.ipk, which is ready to install on a standard openwrt distribution.

If modifying your ipkg environment is not an option, there's the possibility to modify your openwrt installation.
A standard ipkg-build produces packages which are basically two tarballs put together with "ar". In a standard openwrt, busybox is compiled without support for "ar", so to use standard iPKG packages in openwrt this has to be compiled back in. Modify buildroot/sources/busybox.config to include "CONFIG_AR=y" (line 39). Then modify buildroot/build_mipsel/root/bin/ipkg to use "ar" for extracting ipkg files (line 729, change "tar" into "ar -p", just like 2 lines below it). Possibly also a symlink has to be made to busybox for ar, so while you're in buildroot/build_mipsel/root/bin/ anyway, type "ln -s busybox ar". Rebuild your openwrt image, and it *should* support standard iPKG packages now...well, except for the ones which were already created for openwrt :-)

Chicken, egg, time to make a decision on whether or not to support standard iPKGs?

That's all for now, more to follow as I progress. Happy packaging.

Replying to my own post, as seen in this post
http://openwrt.ksilebo.net/forum/viewtopic.php?t=108
it is indeed possible to utilize ipkg-buildpackage to create an iPKG package. I've been looking into this too, yesterday, and I'm in the process of building a standard ipkg/rules template which developers can use to create packages with. I was thinking to grab the main Makefile from buildroot as base,  then add some stuff from a standard make/*.mk file for the details. I'd introduce a macro to point to the openwrt buildroot, so little effort is needed to set up a crosscompile environment to build packages when you already built openwrt at least once :-)
Of course this will all be documented, and maybe we can use that to fill in the "how to build packages" in the WiKi.

The fun part about this: If all this gets standardized, you can potentially crosscompile a lot of small packages straight from the Debian repository to MIPS32 without too much effort. This is because Debian source packages are already prepared to be packaged into a deb. At most things might require small patches to work in a relatively stripped environment like openwrt, but all that can be added as we go along.

Another thing that needs to be done is define some simple package guidelines. Use /usr/bin, not /usr/local, you get the idea.

I wrote this down so we can prevent effort being duplicated. I plan on going through with this unless someone steps up and mentions it's already been done. So please direct questions, comments and suggestions my way through this thread so we can coordinate. There's no need to have people reinvent the wheel twice and thrice, or duplicate effort in porting packages which are already ported by someone else.

Yeah, I was actually going to try and make a simple howto for creating more ipkg's I just haven't gotten to it. A nice thing to have for any of the repositories up is to provide to source/patches/rules/etc for how they built the package. This way we can do some Q/A and possibly fix small errors that may occur in packages. My standard was just creating a directory at my top level called src/ which includes all the files I used to create a package.

Any comments on providing the source for the packages we create?

I've created an ipkg/ directory skeleton for use with ipkg-buildpackage. You can download it from http://www.linuxops.net/ipkg/utils/ipkg.skel.tar.gz
To use it, just unpack your package source, cd into its directory, and untar this skeleton as ipkg/
Then modify ipkg/control for package information, and ipkg/rules so it can find your openwrt buildroot. You'll likely need to dig into the build process itself too, and modify some targets like build-stamp and install. Its hard to generalize that stuff. I've prepared some example code which you can easily modify to match your packages build process.

For an example, download e.g. my openswan ipkg/ directory from http://www.linuxops.net/ipkg/openswan-2.1.2-ipkg.tar.gz

I can't get the ipkg utils running. sad
It fails at 'make':

phil@aurora:~/dev/wrt54g/ipkg-utils-1.7$ make
python setup.py build
Traceback (most recent call last):
  File "setup.py", line 1, in ?
    import distutils.core
ImportError: No module named distutils.core
make: *** [build] Error 1
phil@aurora:~/dev/wrt54g/ipkg-utils-1.7$ 

But right before, I installed python1.5-disutils and I'm still getting this error.

Is there anything special I have to do?

I can't get the ipkg utils running. sad
...
Is there anything special I have to do?

Oh yeah, apparently you need to install some python stuffies. I dunno what distribution you use, but apt-get install python-dev  did the trick for me.

I've stuggled a little with naming conventions for the packages I've been creating.  I think I've settled on using something like this:
name_major.minor-wrt<revision>_mipsel.ipk

For example, I just repackaged the unik-olsr daemon, version 0.4.4.  So, my package is olsrd_0.4.4-wrt1_mipsel.ipk.  The package name is olsrd, the version is 0.4.4-wrt1, arch is mipsel.  Unfortunately, I forgot the init.d entry for it, so I'm going to create another package with -wrt2 as the version extension.
  Anyone have any thoughts on this?  I think it's important that we stick to the original authors' versioning system, but have a method for tracking internal additions as well.  Anyway, if you get your packages from http://www.wildcatwireless.net/wrt54g this is something you'll be seeing.

John
AB0OO

I'd suggest a system simliar to what portage uses. Something like:

name_major.minor-r<revision #>_mipsel.ipk

So the packages would look something like:

dropbear_0.41-r1_mipsel.ipk

Adding wrt to the end just seems a bit ackward to me. But I do agree we need to come up with a standardized way of naming so we don't get confused.

Thoughts?

Actually, adding "wrt" to the package name isn't a bad idea because what we're building are not standard ipkg packages. We're building with "tar", which is different from the normally used "ar". So to make sure people don't try to use packages for openwrt on some other mipsel device (can't think of any, but it never hurts to take future developments into account), hardwiring the packages to the platform doesn't hurt.

Also adding the version might be a good idea, so for b4 you get e.g.

openswan-2.1.2wrtb4_mipsel.ipk or somesuch.

Thoughts?

allright, python-dev did the job.
thanks smile

trying to use PolarWolf's rules to compile dropbear. 

Added the --disable-zlib option but other than that pretty much his basic rules.

compiling goes fine up until linking  where i get the following:

...buildroot/build_mipsel/staging_dir/bin/mipsel-linux-uclibc-ld: warning: cannot find entry symbol __start; defaulting to 00000000004033a0
libtommath/libtommath.a(bn_mp_div.o)(.text.mp_div+0x3a4): In function `mp_div':
: undefined reference to `__udivdi3'

mjn3 pointed out (on irc) that thats a function in libgcc and suggested adding -lgcc as and LDFLAG.  I did that but then I get an error that it can't find lgcc.

running mipsel-linux-uclibc-gcc --print-libgcc-file-name shows that libgcc is in

...staging_dir/lib/gcc-lib/mipsel-linux-uclibc/3.3.3/libgcc.a

so any ideas what's up?  how to make the compiler find the right libs?

trying to use PolarWolf's rules to compile dropbear. 

so any ideas what's up?  how to make the compiler find the right libs?

I'm in no way an expert in developing stuff, nor in gcc, but I *think* you have to add a -L flag too, pointing to the libraries you wanna include. In this case: -L$(STAGING_DIR)/lib/gcc-lib/mipsel-linux-uclibc/3.3.3, so it finds the right libraries to link to.

Just a long shot, like I said, I'm no expert :-)

You might make sure you have all the stuff you need in your path.

This is the path I have in my .bash_profile:

PATH=~/local/bin:~/src/buildroot/build_mipsel/staging_dir/bin:"${PATH}"

I also make sure I have the proper gcc/linker included in my rules file like this:

    ./configure 
    CC=mipsel-linux-gcc 
    AR=mipsel-linux-ar 
    RANLIB=mipsel-linux-ranlib 
    LDFLAGS="-Wl,--gc-sections" 
    CFLAGS="-Os -ffunction-sections -fdata-sections" 
    --disable-zlib 
    --disable-lastlog 
    --disable-utmp 
    --disable-utmpx 
    --disable-wtmp 
    --disable-wtmpx 
    --target=mipsel-linux 
    --host=mipsel-linux 
    --prefix=$(DESTDIR) 
    --exec-prefix=$(DESTDIR)/usr

    make CC=mipsel-linux-gcc AR=mipsel-linux-ar RANLIB=mipsel-linux-ranlib

I hope that helps, but it just sounds like you're not finding the right linker possibly.

Cheers!

heres my rules files -- it's basically PolarWolf's with some cleanup and Ramereth's dropbear specific options (as well as some defines to make adding pacakges specific options a little more extensible).

but....

it fails with the error above.  i really can't see what i'm missing.

#!/usr/bin/make -f

PACKAGE=dropbear
OPENWRTBR=~/wrt/buildroot
PKG_LDFLAGS=
PKG_CFLAGS=-ffunction-sections -fdata-sections
PKG_CONFIG=--disable-zlib 
        --disable-lastlog 
        --disable-utmp 
        --disable-utmpx 
        --disable-wtmp 
        --disable-wtmpx

BASE_DIR:=${shell pwd}
BUILD_DIR:=$(BASE_DIR)/ipkg
DESTDIR:=/tmp/$(PACKAGE)
GNU_TARGET_NAME=mipsel-linux
STAGING_DIR=$(OPENWRTBR)/build_mipsel/staging_dir
TARGET_PATH=$(STAGING_DIR)/bin:/bin:/sbin:/usr/bin:/usr/sbin
TARGET_CROSS=$(STAGING_DIR)/bin/mipsel-linux-uclibc-
TARGET_CC=$(TARGET_CROSS)gcc

HOSTCC=gcc
HOST_ARCH:=$(shell $(HOSTCC) -dumpmachine | sed -e s'/-.*//' 
        -e 's/sparc.*/sparc/' 
        -e 's/arm.*/arm/g' 
        -e 's/m68k.*/m68k/' 
        -e 's/ppc/powerpc/g' 
        -e 's/v850.*/v850/g' 
        -e 's/sh[234]/sh/' 
        -e 's/mips-.*/mips/' 
        -e 's/mipsel-.*/mipsel/' 
        -e 's/cris.*/cris/' 
        -e 's/i[3-9]86/i386/' 
        )
GNU_HOST_NAME:=$(HOST_ARCH)-pc-linux-gnu

TARGET_CONFIGURE_OPTS= 
                AR=$(TARGET_CROSS)ar 
                AS=$(TARGET_CROSS)as 
                LD=$(TARGET_CROSS)ld 
                NM=$(TARGET_CROSS)nm 
                CC=$(TARGET_CROSS)gcc 
                GCC=$(TARGET_CROSS)gcc 
                CXX=$(TARGET_CROSS)g++ 
                RANLIB=$(TARGET_CROSS)ranlib


STRIP=$(TARGET_CROSS)strip --remove-section=.comment --remove-section=.note

.PHONY: build install clean

all: install

build: build-stamp

build-stamp: $(PACKAGE)/.configured
        # Example make target
        $(MAKE) $(TARGET_CONFIGURE_OPTS) DESTDIR=$(DESTDIR)

$(PACKAGE)/.configured:

        rm -rf config.cache; 
        $(TARGET_CONFIGURE_OPTS) 
        CFLAGS="-Os -mips2 $(PKG_CFLAGS)" 
        LDFLAGS=$(PKG_LDFLAGS)
        ./configure 
        --target=$(GNU_TARGET_NAME) 
        --host=$(GNU_TARGET_NAME) 
        --build=$(GNU_HOST_NAME) 
        --prefix=/usr 
        --exec-prefix=/usr 
        $(PKG_CONFIG)
        touch  .configured

install: build
        # Example make install target
        $(MAKE) $(TARGET_CONFIGURE_OPTS) DESTDIR=$(DESTDIR) STRIP=$(STRIP) install

clean:
        rm -f build-stamp
        rm -rf $(DESTDIR)
        $(MAKE) clean

I just built a dropbear package.

I did indeed run into the same problems you did, and it took me a while to debug. It dawned me when I just did a 'make' in the source directory after I let the package configure using 'ipkg/rules'. A regular 'make' built the package just fine, so it had to be something screwey in ipkg/rules. And indeed. Changing two targets solved the problem for me:

build-stamp: $(PACKAGE)/.configured
  # Example make target
  $(MAKE)
  $(MAKE) strip STRIP=$(STRIP)

install: build
  # Example make install target
  $(MAKE)
  $(MAKE) strip STRIP=$(STRIP)
  $(MAKE) install DESTDIR=$(DESTDIR)

package build, stripped and all. Have fun reproducing :-)

You should include some default configuration and all inside the package, I guess, to make it a real package, fit for redistribution. What above procedure will leave you is just a package with three stripped binaries. But since I don't need an sshd on my router, I'll leave that as excersise for the reader.

open /usr/local/bin/ipkg-build, find the line where it says "outer=ar" (line 159 in version 1.7), and change "ar" into "tar";

It seems that ipkg-build accepts a command line option '-c' to tell it to use tar instead of ar.  Using this I have successfully built and installed packages without editing ipkg-build.

Graham

"Graham Cobb wrote:

It seems that ipkg-build accepts a command line option '-c' to tell it to use tar instead of ar.  Using this I have successfully built and installed packages without editing ipkg-build.

Aaah, good to know. I'll have a looksee at that, and see if ipkg-buildpackage (what I use these days) also supports this. If not, I'll make it tongue

Thanks.

As a learning excercise, I've been trying to make an ipkg of IPTABLES-SAVE and IPTABLES-RESTORE.  Yes, I'm probably biting off a lot more than I can chew at this point.  Anyway, I've been going about this by trying to compile ipatables-save.c with this command

mipsel-linux-uclibc-gcc -Wall -g -o iptables-save iptables-save.c

but I get a pageful of errors.  Looking through the Makefile that comes with the iptables 1.2.9 tarball I see that a lot of flags are passed to the compiler - it's all quite complicated.  I suppose I could work my way through the makefile to see exactly what's going on, but I'd rather just work out how to use this makefile with the mipsel compiler.
Is this possible?  The compiler is specified with the variable $(CC) but I can't work out where this variable is defined.  Does make just insert a default compiler here?  I don't understand how make knows which compiler to use.

Given the complexity of iptables, and the likely dependency of the required tools on libipt.so, I would suggest to not just build the binaries you need, but do a complete compile of the entire iptables source tree. After you do a `make install` (into a temporary location, set $DESTDIR) strip the installation of stuff you *don't* need, and create a package from what you have left (binaries, and maybe the libipt library). Don't forget to strip.

This way you're using the "intelligence" built into the build process of iptables to get everything in working order, instead of having to reinvent the wheel for getting the required binaries to build.

Cool, thanks for the advice.  I should probably read up a bit more about compiling in general, and cross-compiling as well.  I still don't understand - if I do a 'make install' how do I get the makefile that came with iptables to cross-compile for mipsel?  How do I tell it to use the mipsel compiler and utilities?

Also, I've noticed that in some of the 'rules' files for ipkg building there are routines that 'strip'.  What exactly is this strip process?

Cool, thanks for the advice.  I should probably read up a bit more about compiling in general, and cross-compiling as well.  I still don't understand - if I do a 'make install' how do I get the makefile that came with iptables to cross-compile for mipsel?  How do I tell it to use the mipsel compiler and utilities?

Depends on how the build process works. If you're in luck the developers use autotools (autoconf, etc, you know, the "./configure" script) which you simply feed the proper variables and it'll pick all of it up itself. If not, well, then you get to dig into Makefiles and such. Fun, but not for the faint of heart.

Also, I've noticed that in some of the 'rules' files for ipkg building there are routines that 'strip'.  What exactly is this strip process?

strip(1) removes stuff from binaries which are not really required for functioning. Like symbols and the likes. It shrinks the binaries in size. Which is a good thing for embedded systems.

Hi,

How can I install OLSRd into OpenWRT?

Thanks for attention,

The discussion might have continued from here.