JFFS file system mount time is proportional to filesystem size. JFFS should read some data at mount time and unfortunately, amount of these data to process is directly proportional to filesystem size. So, you WILL have this issue with JFFS.
Do not use JFFS2 for huge partitions, ever. And take into consideration that CF cards are coming with their own built-in controller which manages wear leveling, addresses transations, physical properties handling (like erase boundaries) on it's own. So any general-purpose filesystem should be fine (with some notes - see below if interested).
JFFS has been designed for use cases where quite small "raw" flash device (usually just single IC) attached to CPU bus without any sophisticated controllers between CPU and flash, so main system CPU have to bother itself with some things which are not visible otherwise (when using flash cards with integrated controllers hiding these things, etc).
So JFFS bothers itself with wear leveling, writes alignment on erase boundaries, addresses translations, etc while this only needed when flash ICs are directly attached to CPU so there is nobody except CPU itself to bother with such low-level tasks.
So, for example, small boot flash of CPU (where usually system is stored) is a good place to use JFFS and JFFS really useful since in such case filesystem MUST take flash physical properties into account. Most huge JFFS real-world use I know is a 256Mb NAND flash of Nokia N8x0 devices. Anything more than that is, uhm, not recommended. Due to mount times 
Huge cards which are supplied with FAT formatting are not a right place for JFFS at all since at very most JFFS will do the same job as internal card controller already does, there is no gain doing so but extra overhead happens.
So, summary is: to speed things up, just use some another filesystem on your card. I'm personally would prefer EXT2/3 or XFS in your scenario but it's matter of preferences, tastes, etc.
P.S. Actually, "sophisticated" cards (SD, MMC, CF, ...) are hiding their true geometry from the world and are trying to look like traditional disk-based devices. So there is no need to have special-purpose filesystem on such card. But please note: erasing "default" filesystem (usually, FAT) from card is not a great idea at all and can lead to sub-optimal results. But you already did it so you have a freedom of choice which FS to use. Because you lack true geometry knowledge of ICs anyway, as well as knowledge how it's built-in controller works. So you will have a equally hard times putting ANY filesystem into proper alignments and sizes so you do not have to stick to fat and can use some better filesystem since there is nothing to lose left already but maybe something to gain vs (still sub-optimally placed) FAT
. Default vendor's FAT formatting done with a geometry knowledge and knowledge how controller works, so FAT's structures are aligned in a very tricky manner to match physical boundaries, etc. What is the penalty for misplacement? Up to 2x times loss of write speed + 2x more writes (and hence 2x faster wearing) if it has turned out that filesystem clusters (blocks or whatever) are clashing with physical location of blocks on ICs (technically, when FS block crosses NAND page boundaries, it is a "poor" scenario since to overwrite one FS block, two pages should be erased while it could be possible to satisfy write with single page rewrite if FS alignment is "proper" to flash ICs physics).
(Last edited by Drone on 22 Jun 2009, 20:17)