Topic: Makefiles for MD and mdadm

The content of this topic has been archived on 2 Apr 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Page 1 of 1

Post #1

beikeland

19 Oct 2007, 00:42

I'm trying to add Makefiles to enabe MD/Raid and mdadm. (As I find the idea of running my tiny fileserver of the same build system as my routers rather appealing)

mdadm I got right (I think):

include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk

PKG_NAME:=mdadm
PKG_VERSION:=2.6.3
PKG_RELEASE:=1

PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tgz
PKG_SOURCE_URL:=http://www.cse.unsw.edu.au/~neilb/source/mdadm
PKG_MD5SUM:=0797caab07dce50cb71d6dd99d8da487

include $(INCLUDE_DIR)/package.mk

define Package/$(PKG_NAME)
  SECTION:=utils
  CATEGORY:=Utilities
  TITLE:=MD admin tools
  URL:=http://neil.brown.name/blog/mdadm
endef

define Package/$(PKG_NAME)/description
  Manages Mulitple Devices (sofware raid)
endef

define Package/$(PKG_NAME)/install
    $(INSTALL_DIR) $(1)/usr/sbin
    $(INSTALL_BIN) $(PKG_BUILD_DIR)/mdadm $(1)/usr/sbin/
endef

$(eval $(call BuildPackage,$(PKG_NAME)))

But I think I might need a bit of input on how to get the MD and Raid stuff to work as you guys would expect?

so far I have

define KernelPackage/md
  SUBMENU:=$(BLOCK_MENU)
  TITLE:=Multiple devices driver support (RAID and LVM)
  KCONFIG:=CONFIG_MD
  FILES:=\
   $(LINUX_DIR)/drivers/md/md.$(LINUX_KMOD_SUFFIX) \
   $(LINUX_DIR)/drivers/md/bitmap.$(LINUX_KMOD_SUFFIX) \
  AUTOLOAD:=$(call AutoLoad,30,md)
endef

define KernelPackage/md/description
 Support multiple ...
endef

$(eval $(call KernelPackage,md))


define KernelPackage/md-raid
  SUBMENU:=$(BLOCK_MENU)
  TITLE:=RAID support
  KCONFIG:=CONFIG_BLK_DEV_MD
  DEPENDS:=kmod-md
endef

define KernelPackage/md-raid/description
 This driver lets ...
endef

$(eval $(call KernelPackage,md-raid))

define KernelPackage/md-linear
  SUBMENU:=$(BLOCK_MENU)
  DEPENDS:=kmod-md-raid
  TITLE:=Linear (append) mode
  KCONFIG:=CONFIG_MD_LINEAR
  FILES:=$(LINUX_DIR)/drivers/md/linear.$(LINUX_KMOD_SUFFIX)
  AUTOLOAD:=$(call AutoLoad,30,linear)
endef

define KernelPackage/md-linear/description
 If you say Y here ...
endef

$(eval $(call KernelPackage,md-linear))

define KernelPackage/md-raid0
  SUBMENU:=$(BLOCK_MENU)
  DEPENDS:=kmod-md-raid
  TITLE:=RAID-0 (striping) mode
  KCONFIG:=CONFIG_MD_RAID0
  FILES:=$(LINUX_DIR)/drivers/md/raid0.$(LINUX_KMOD_SUFFIX)
  AUTOLOAD:=$(call AutoLoad,30,raid0)
endef

define KernelPackage/md-raid0/description
 If you say Y here ..
endef

$(eval $(call KernelPackage,md-raid0))

define KernelPackage/md-raid1
  SUBMENU:=$(BLOCK_MENU)
  DEPENDS:=kmod-md-raid
  TITLE:=RAID-1 (mirroring) mode
  KCONFIG:=CONFIG_MD_RAID1
  FILES:=$(LINUX_DIR)/drivers/md/raid1.$(LINUX_KMOD_SUFFIX)
  AUTOLOAD:=$(call AutoLoad,30,raid1)
endef

define KernelPackage/md-raid1/description
 A RAID-1 set ...
endef

$(eval $(call KernelPackage,md-raid1))

define KernelPackage/md-raid456
  SUBMENU:=$(BLOCK_MENU)
  DEPENDS:=kmod-md-raid
  TITLE:=RAID-4/RAID-5/RAID-6 mode
  KCONFIG:=CONFIG_MD_RAID456            
  FILES:=$(LINUX_DIR)/drivers/md/raid456.$(LINUX_KMOD_SUFFIX)
  AUTOLOAD:=$(call AutoLoad,30,raid456)
endef

define KernelPackage/md-raid456/description
 A RAID-5 set of ...
endef

$(eval $(call KernelPackage,md-raid456))

define KernelPackage/md-raid5rs
  SUBMENU:=$(BLOCK_MENU)
  DEPENDS:=kmod-md-raid456
  TITLE:=Support adding drives to a raid-5 array
  KCONFIG:=CONFIG_MD_RAID5_RESHAPE            
endef

define KernelPackage/md-raid5rs/description
 A RAID-5 set c...
endef

$(eval $(call KernelPackage,md-raid5rs))

Prefixing the packages with 'md-' was to get to group up with the main MD entry. Is there another way to achieve grouping without calling it md-raid5?

The option to enable reshaping on raid5 - is there another way of adding this? As it doesnt generate any files, having it as a module seems kind of wrong?

And what kind of dependencies should I set for MD? In the kernel config it depends on the block layer - but it isnt an option in OpenWRT. And using ide-core or scsi-core makes both a necessity witch isnt true?

Lastly, regarding autoloading - when would it be apropriate to load the modules? I went for the same as NBD, but is this the best option?

Thanks for your input guys!

Post #2

alexsamad

19 Oct 2007, 01:14

Sounds like a great idea, any thoughts of including lvm in this as well ?

Post #3

beikeland

19 Oct 2007, 12:43

Eventually I'm going to see if I cant get LVM to work too, however I'm no master of linux, more a master of disaster.. But keeping a constructive thread here should be enough to get me the pointers i need.. (I hope)

Post #4

beikeland

20 Oct 2007, 20:46

What would be the best way of adding raid=autodetect to the menu.lst in grub? Or is it better to run autodetection as a start up script?

Post #5

beikeland

22 Oct 2007, 05:23

I've finally worked out the new bits in this version of the kernel, and it compiles under x86 and 2.6, and runs under qemu. I'm hoping someone more knowledable can have a look and spot any stupidities before I submit it as an enhancement ticket

define KernelPackage/md-raid456
  SUBMENU:=$(BLOCK_MENU)
  DEPENDS:=kmod-md-raid
  TITLE:=RAID-4/RAID-5/RAID-6 mode
  KCONFIG:=\
   CONFIG_MD_RAID456\
   CONFIG_ASYNC_MEMCPY\
   ASYNC_XOR
  FILES:=\
   $(LINUX_DIR)/drivers/md/raid456.$(LINUX_KMOD_SUFFIX)\
   $(LINUX_DIR)/crypto/xor.$(LINUX_KMOD_SUFFIX)\
   $(LINUX_DIR)/crypto/async_tx/async_xor.$(LINUX_KMOD_SUFFIX)\
   $(LINUX_DIR)/crypto/async_tx/async_memcpy.$(LINUX_KMOD_SUFFIX)\
   $(LINUX_DIR)/crypto/async_tx/async_tx.$(LINUX_KMOD_SUFFIX)
  AUTOLOAD:=\
   $(call AutoLoad,30,xor)\
   $(call AutoLoad,30,async_xor)\
   $(call AutoLoad,30,async_memcpy)\
   $(call AutoLoad,30,async_tx)\
   $(call AutoLoad,30,raid456)
endef

define KernelPackage/md-raid456/description
 A RAID-5 set of ...
endef

$(eval $(call KernelPackage,md-raid456))

Post #6

beikeland

8 Nov 2007, 02:58

This is a copy from the WR forums, I didnt realise the old thread was that old, so I'm sticking this in here:

Finally got around to sorting out the Makefile for lvm2, but it now complains about libraries and symbols. Full build log @ http://pastebin.com/m83e6057 , and the last few lines for are included below.

I've searched around and found some references to this problem, but nothing that made me any wiser; how about you guys?
http://www.opencascade.org/org/forum/thread_9224/
http://www.raditha.com/blog/archives/001119.html

/home/bjorn/dev/OpenWRT/trunk/staging_dir/toolchain-i386_gcc4.1.2/lib/gcc/i386-linux-uclibc/4.1.2/../../../../i386-linux-uclibc/bin/ld: errno@@GLIBC_PRIVATE: TLS definition in /lib/libc.so.6 section .tbss mismatches non-TLS definition in /home/bjorn/dev/OpenWRT/trunk/staging_dir/toolchain-i386_gcc4.1.2/i386-linux-uclibc/bin/../lib/libc.so.0 section .bss
#
/home/bjorn/dev/OpenWRT/trunk/staging_dir/toolchain-i386_gcc4.1.2/i386-linux-uclibc/bin/../lib/libc.so.0: could not read symbols: Bad value
#
collect2: ld returned 1 exit status
#
make[4]: *** [lvm] Error 1
#
make[4]: Leaving directory `/home/bjorn/dev/OpenWRT/trunk/build_dir/i386/LVM2.2.02.28/tools'
#
make[3]: *** [tools] Error 2
#
make[3]: Leaving directory `/home/bjorn/dev/OpenWRT/trunk/build_dir/i386/LVM2.2.02.28'
#
make[2]: *** [/home/bjorn/dev/OpenWRT/trunk/build_dir/i386/LVM2.2.02.28/.built] Error 2
#
make[2]: Leaving directory `/home/bjorn/dev/OpenWRT/trunk/feeds/http_svn_.assembla.com_svn_test_openwrt_package_/lvm2'
#
make[1]: *** [package/lvm2/compile] Error 2
#
make[1]: Leaving directory `/home/bjorn/dev/OpenWRT/trunk'
#
make: *** [package/lvm2-compile] Error 2
#
 
#
Script done on Wed 07 Nov 2007 09:04:11 PM CET

Makefile for lvm2 is as follows

include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk

PKG_NAME:=LVM2
PKG_VERSION:=2.02.28
PKG_RELEASE:=1
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME).$(PKG_VERSION)

PKG_SOURCE:=$(PKG_NAME).$(PKG_VERSION).tgz
PKG_SOURCE_URL:=ftp://sources.redhat.com/pub/lvm2
PKG_MD5SUM:=940a882a71cae84f6befc8069a260ad1

include $(INCLUDE_DIR)/package.mk

define Package/lvm2
  SECTION:=utils
  CATEGORY:=Utilities
  DEPENDS:=device-mapper
  TITLE:=LVM2
  URL:=http://sourceware.org/lvm2/
endef

define Package/lvm2/description
  LVM2 userspace toolset
endef

define Build/Configure
  $(call Build/Configure/Default, \
     --with-dmdir=$(BUILD_DIR)/device-mapper.1.02.22 \
  )
endef 

define Package/lvm2/install
    $(INSTALL_DIR) $(1)/usr/sbin
    $(INSTALL_BIN) $(PKG_BUILD_DIR)/lvm2 $(1)/usr/sbin/
endef

$(eval $(call BuildPackage,lvm2))

edit: bad link

Post #7

beikeland

8 Nov 2007, 03:11

I've tried building lvm2 again from a completely clean trunk, without ccache, but it throws the same error. And I'm no closer to sorting it out I'm affraid. I've searched for the make files used in nslu2-linux and every other place I could think - but this is a bit out of my leauge i think.

If anyones interested in the (other) packages until til goes in trunk you can add this to the package feed in menuconfig.
http://svn2.assembla.com/svn/test101/openwrt/package/

the .mk files for all the kernel modules can be found here (test101.mk contains everything in one file)
http://svn2.assembla.com/svn/test101/op … l/modules/

Post #8

beikeland

8 Nov 2007, 22:24

.. well, i made lvm compile in the end - but it weights in at 600k so I think I've gotten something wrong as its more than on my ubuntu box.. Though I've got plenty of room so for now it stays that way..

I would really appreciate some feedback on how to improve things!

include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk

PKG_NAME:=LVM2
PKG_VERSION:=2.02.28
PKG_RELEASE:=1
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME).$(PKG_VERSION)

PKG_SOURCE:=$(PKG_NAME).$(PKG_VERSION).tgz
PKG_SOURCE_URL:=ftp://sources.redhat.com/pub/lvm2
PKG_MD5SUM:=940a882a71cae84f6befc8069a260ad1

include $(INCLUDE_DIR)/package.mk

define Package/lvm2
  SECTION:=utils
  CATEGORY:=Utilities
  DEPENDS:=device-mapper
  TITLE:=LVM2
  URL:=http://sourceware.org/lvm2/
endef

define Package/lvm2/description
  LVM2 userspace toolset
endef

define Build/Configure
  $(call Build/Configure/Default, \
     --libdir="$(STAGING_DIR)/usr/include" \
     --includedir="$(STAGING_DIR)/usr/include" \
     --with-dmdir=$(BUILD_DIR)/device-mapper.1.02.22 \
  )
endef

define Package/lvm2/install
    $(INSTALL_DIR) $(1)/sbin
    $(INSTALL_BIN) $(PKG_BUILD_DIR)/tools/lvm $(1)/sbin/
    ln -s /sbin/lvm $(1)/sbin/dumpconfig
    ln -s /sbin/lvm $(1)/sbin/formats
    ln -s /sbin/lvm $(1)/sbin/help
    ln -s /sbin/lvm $(1)/sbin/lvchange
    ln -s /sbin/lvm $(1)/sbin/lvconvert
    ln -s /sbin/lvm $(1)/sbin/lvcreate
    ln -s /sbin/lvm $(1)/sbin/lvdisplay
    ln -s /sbin/lvm $(1)/sbin/lvextend
    ln -s /sbin/lvm $(1)/sbin/lvmchange
    ln -s /sbin/lvm $(1)/sbin/lvmdiskscan
    ln -s /sbin/lvm $(1)/sbin/lvmsadc
    ln -s /sbin/lvm $(1)/sbin/lvmsar
    ln -s /sbin/lvm $(1)/sbin/lvreduce
    ln -s /sbin/lvm $(1)/sbin/lvremove
    ln -s /sbin/lvm $(1)/sbin/lvrename
    ln -s /sbin/lvm $(1)/sbin/lvresize
    ln -s /sbin/lvm $(1)/sbin/lvs
    ln -s /sbin/lvm $(1)/sbin/lvscan
    ln -s /sbin/lvm $(1)/sbin/pvchange
    ln -s /sbin/lvm $(1)/sbin/pvresize
    ln -s /sbin/lvm $(1)/sbin/pvck
    ln -s /sbin/lvm $(1)/sbin/pvcreate
    ln -s /sbin/lvm $(1)/sbin/pvdata
    ln -s /sbin/lvm $(1)/sbin/pvdisplay
    ln -s /sbin/lvm $(1)/sbin/pvmove
    ln -s /sbin/lvm $(1)/sbin/pvremove
    ln -s /sbin/lvm $(1)/sbin/pvs
    ln -s /sbin/lvm $(1)/sbin/pvscan
    ln -s /sbin/lvm $(1)/sbin/segtypes
    ln -s /sbin/lvm $(1)/sbin/vgcfgbackup
    ln -s /sbin/lvm $(1)/sbin/vgcfgrestore
    ln -s /sbin/lvm $(1)/sbin/vgchange
    ln -s /sbin/lvm $(1)/sbin/vgck
    ln -s /sbin/lvm $(1)/sbin/vgconvert
    ln -s /sbin/lvm $(1)/sbin/vgcreate
    ln -s /sbin/lvm $(1)/sbin/vgdisplay
    ln -s /sbin/lvm $(1)/sbin/vgexport
    ln -s /sbin/lvm $(1)/sbin/vgextend
    ln -s /sbin/lvm $(1)/sbin/vgimport
    ln -s /sbin/lvm $(1)/sbin/vgmerge
    ln -s /sbin/lvm $(1)/sbin/vgmknodes
    ln -s /sbin/lvm $(1)/sbin/vgreduce
    ln -s /sbin/lvm $(1)/sbin/vgremove
    ln -s /sbin/lvm $(1)/sbin/vgrename
    ln -s /sbin/lvm $(1)/sbin/vgs
    ln -s /sbin/lvm $(1)/sbin/vgscan
    ln -s /sbin/lvm $(1)/sbin/vgsplit
    ln -s /sbin/lvm $(1)/sbin/version
endef

$(eval $(call BuildPackage,lvm2))

Post #9

forum2006

9 Nov 2007, 00:17

In the Package/lvm2/install template you can use foreach loop to create the symlinks. Below is an example for the $(CP) command. If you modify it you can use it for the ln command:

$(CP) $(foreach lib,com_err e2p ext2fs,$(PKG_INSTALL_DIR)/usr/lib/lib$(lib).so.*) $(1)/usr/lib/

(Last edited by forum2006 on 9 Nov 2007, 00:28)

Post #10

beikeland

9 Nov 2007, 00:47

so I put in

LVM2_LINK_NAMES:= \
dumpconfig formats help lvchange lvconvert lvcreate lvdisplay lvextend \
lvmchange lvmdiskscan lvmsadc lvmsar lvreduce lvremove lvrename lvresize \
lvs lvscan pvchange pvresize pvck pvcreate pvdata pvdisplay pvmove pvremove \
pvs pvscan segtypes vgcfgbackup vgcfgrestore vgchange vgck vgconvert \
vgcreate vgdisplay vgexport vgextend vgimport vgmerge vgmknodes vgreduce \
vgremove vgrename vgs vgscan vgsplit version

and

$(foreach link, $(LVM2_LINK_NAMES), ln -s /sbin/lvm $(1)/sbin/$(link) ; )

wich cleaned things up a bit! cheers

edit, damn smileys

(Last edited by beikeland on 9 Nov 2007, 00:52)

Post #11

buzzy

24 Mar 2008, 01:54

Is there any howto on making a openwrt kernel with raid-support? I've managed to create a kernel that can boot debian, but I cant find any option for raid in "make menuconfig". Please help.

Post #12

Wodin

24 Mar 2008, 02:24

See if "make kernel_menuconfig" gets you anywhere useful.

Post #13

buzzy

24 Mar 2008, 02:40

that seem to have downloaded a complete debian 2.6 kernel for mips-systems. Shouldn't I use openwrt?
but the downloaded system had raid-option in it. Where do I go from here?

The discussion might have continued from here.

Page 1 of 1