I have an ASUS RT-N16 too and never got the brcmsmac driver to work properly. That meant I was stuck on the b43 driver which is limited to 802.11g, no support for 802.11n unfortunately.
However, I have finally cracked the Broadcom proprietary driver using Chaos Calmer rc3. I spent two days struggling with all the options, as some seemed to cancel each other out or wouldn’t survive reboots.
The first crucial discovery I made (after the first day!) is that the Broadcom-wl config utility is useless. It appears to be outdated and not support WPA2 for instance. When I used wlc instead of wl I started to make progress.
These are the steps I took (while being connected with a cable of course):
1) Remove any remnants of previous drivers such as b43 and brcmsmac (kmod-b43, kmod-b43legacy, kmod-brcmsmac, kmod-brcmutil etc.) and you might as well remove the official wl utility to prevent confusion.
2) Reboot the router
3) Delete etc/config/wireless
* kmod-brcm-wl (the driver itself)
* nas (broadcom’s WPA supplicant. Without this you won’t get WPA or WP2 to work)
* wlc (the open source version of the official wl tool)
5) Create a new wireless config file containing the following:
config wifi-device wl0
option type broadcom
option channel 3 (change the channel to what works best for you)
option country GB (replace GB with your country’s ISO 3166 country code)
option htmode HT20
option device wl0
option network lan
option mode ap
option ssid XXXXXXXXXX (replace with your ssid)
option encryption psk2+ccmp (this forces AES encryption over WPA2)
option key XXXXXXXXXX (replace with your WPA2 passphrase)
6) Use the wlc utility to set a few additional settings that appear to be only work using wlc
In this order:
wlc ap 1 (I wasn’t able to get the AP mode without this, it seemed to default to client mode until I used wlc to force it)
wlc wpa_auth 128 (this sets the key type to WPA2-PSK. Mixed mode WPA/WPA2 appears to be 132)
There seems to be no manual for wlc. It is largely compatible with wl so you can use that manual and list of commands for the basics however while wl requires to specify the interface for each command, wlc seems to be fine without it. Furthermore, wlc appears to support more than wl so for more advanced/modern settings it takes a bit of trial and error. I also used the manual for nas to get some of the settings from (the correct code for wpa_auth for instance). If your requirements are different from mine (using RADIUS authentication for instance) I suggest you dive deeper into the nas utility and its manual.
7) Test whether everything works OK by rebooting the router. If it comes up without any interventions you’ve done it right.
(Last edited by MorrisMajor on 7 Sep 2015, 12:02)