Pi3 wifi+virtual ap bridged that really works (for me). Worth trying on pcp?

Hi guys,

This is a recipe for Pi3B only (not 3B+, not 0w), to get the built-in wlan0 interface working as a WPA2 STA and the "virtual AP" interface working at the same time. For now I use an open AP, IIRC a WPA2 AP works as well, I haven't tried other combinations.
It seems to work so far for me on a buildroot, 32-bit, busybox init with ifupdown, Pi 3B (Linux rpi3 4.14.50-v7 #1 SMP Mon Oct 15 14:26:41 CEST 2018 armv7l GNU/Linux)

I'm posting this because:

• I've spent a long time trying to make STA+AP work on brcm 43430 in a reliable and repeatable way, and according to my stress-test it seems I've achieved that
• I suppose some here might want to test that with pCP, which is not too different from my buildroot system, and I have a hunch the wifi virtual AP could sustain streaming to a couple of wifi speakers.
  Unfortunately I don't have time to test this recipe myself on pCP but I think it should mostly apply and work.

I can't really answer further questions:

• I expose here the gist of my recipe and fundamentally I do not understand why it works...
• For sure you can't expect it to work with something like Raspbian or a systemd-based OS: dhcpcd and systemd have their way of managing interfaces, and this changes everything.
• Along my quest I've had systems where removing/reinserting the brcmfmac module would help getting STA+AP work, and others where doing that just makes the OS crash...

Folks running Raspbian or a systemd OS might want to refer to this excellent post on StackExchange. I know for a fact the author Ingo has been researching on the question for a while, too.

Ok, now the recipe:

1. Bring up the wlan0 interface with the normal wpa_supplicant daemon (in "pre-up") and next, add a wpa_cli instance with something like this...
   Code:

   ---

   [ -f /var/run/wpastate.wlan0.pid ] || /usr/sbin/wpa_cli -B -i wlan0 -a /usr/local/bin/hostapd_helper.sh -P /var/run/wpastate.wlan0.pid

   ---

   ...having hostapd_helper.sh as this:
   Code:

   ---

   #!/bin/sh
freq_to_chan(){
    F="$1"
    [ $F -eq 2484 ] && echo 14 && return
    [ $F -lt 2484 ] && echo $(( (F-2407) / 5)) && return
    echo $(( F / 5 - 1000))
}
case $2 in
    "CONNECTED")
    F=$(/usr/sbin/wpa_cli -i ${1} status | grep freq | cut -d '=' -f 2 | xargs)
    [ -n "$F" ] && C=$(freq_to_chan ${F})
    [ -n "$C" ] && echo ${F} ${C} > /var/run/network/wpastate.${1} 
    ;;
    "DISCONNECTED")
    rm -f /var/run/network/wpastate.${1}
    ;;
esac
exit 0

   ---

   This makes wpa_supplicant report its current frequency and channel in file /var/run/network/wpastate.wlan0 when it is connected. hostapd needs that to launch the AP successfully.
   
   
2. Tear wlan0 down like that (in "post-down"):
   Code:

   ---

   # Use the cli to stop the backend. Our cli exits in turn
if [ -e /var/run/wpastate.wlan0.pid ]; then
    /usr/sbin/wpa_cli -i wlan0 terminate
else
    pkill -f "/usr/sbin/wpa_supplicant -B -i wlan0"
    pkill -f "/usr/sbin/wpa_cli -B -i wlan0"
fi
rm -f /var/run/wpastate.wlan0.pid
rm -f /var/run/network/wpastate.wlan0

   ---

3. Bring up the ap0 interface this way (in "pre-up"):
   Code:

   ---

   /usr/sbin/iw phy0 interface add ap0 type __ap
/usr/sbin/iw ap0 set power_save off
/sbin/ip link set ap0 address 02:96:ea:ca:ff:f4

   ---

   I suggest you set the MAC to "02:something" (see MAC addr classes and linux bridge behaviour as to why.) power_save off really seems to make a difference, make sure to apply it to both ap0 and wlan0.
   
   
4. Tear ap0 down like that (in "post-down"):
   Code:

   ---

   /usr/sbin/iw ap0 del

   ---

5. You also need a bridge interface (I called it local0), there is nothing special about it. Recommend not shutting down STP but using a short forward delay to speed up DHCP lease acquisition.
   
   
6. You need a DHCP server listening on the bridge, as usual dnsmasq does just fine.
   
   
7. You need a hostapd config, nothing special about it. For the record here is my open AP test config in /etc/hostapd/hostapd-ap0.conf:
   Code:

   ---

   ctrl_interface=/var/run/hostapd
ctrl_interface_group=netdev
interface=ap0
bridge=local0
driver=nl80211
country_code=FR
ssid=test
wpa=0
hw_mode=g
channel=11
wmm_enabled=1
ieee80211n=1
ieee80211d=1
max_num_sta=5
logger_syslog=0
logger_stdout=0

   ---

8. The last piece of the puzzle is the start/stop hostapd script. The stop action is simple, in essence it does:
   Code:

   ---

   kill $(cat /var/run/hostapd.ap0.pid)
rm -f /tmp/hostapd-ap0.conf
ifdown ap0
ifdown local0

   ---

9. The start action is a little bit more involved...
   Code:

   ---

   cp -f /etc/hostapd/hostapd-ap0.conf /tmp/hostapd-ap0.conf
# If wlan0 is active we need to know
STA_IS_UP=0
[ -s /var/run/network/wpastate.wlan0 ] && STA_IS_UP=1
if [ ${STA_IS_UP} -eq 1 ]; then
    echo "wlan0 is in use, reset..."
    CHAN=$(cat /var/run/network/wpastate.wlan0 | cut -d' ' -f 2)
    echo "set AP channel to ${CHAN}..."
    sed -e "s/channel=.*/channel=${CHAN}/" -i /tmp/hostapd-ap0.conf
    ifdown wlan0
fi
echo "set AP interfaces up..."
ifup ap0
ifup local0
echo "start hostapd and bridge helper..."
/usr/sbin/hostapd -B -P /var/run/hostapd.ap0.pid /tmp/hostapd-ap0.conf
/usr/bin/hostapd_cli -B -i ap0 -a /usr/local/bin/bridge_helper.sh
if [ $STA_IS_UP -eq 1 ]; then
    echo "reconfigure wlan0..."
    ifup wlan0
fi

   ---

   ...having bridge_helper.sh as this:
   Code:

   ---

   #!/bin/sh
[ "$#" -eq 3 ] || exit 1
[ "${2}" == 'AP-STA-CONNECTED' ] && /sbin/ip link set ${1} up
exit 0

   ---

   Basically, if wpa_supplicant has engaged with the bcrmfmac driver, it is impossible to start the AP. So we need to stop wlan0 (which stops wpa_supplicant). Catch-22, the wlan0 interface won't be able to reconnect if the virtual AP isn't initially operating on the channel the STA needs to use to connect with its AP. So we get the channel used by wlan0 before stopping it, and fudge the hostapd conf with it. Then restarting wlan0 will succeed.
   Curiously, once the AP+STA couple is active, if the STA changes channel, hostapd will have no problem following. There is no need to repeatedly change hostapd.conf and restart hostapd. Only once at startup.
   
   Why ifupdown wlan0 when killing/restarting wpa_supplicant would do? Because I thought it would make the state machine simpler, and I don't expect many AP restarts. Same goes with systematically creating/deleting the virtual interface with iw when bringing up/down ap0.
   
   By uselessly setting ap0 up each time a new client connects to the AP, the bridge_helper script makes bridging actually work. Without it, the AP client takes a long time obtaining an IP address over DHCP and after that it isn't able to connect beyond the bridge itself, it doesn't see any other MACs. This is pure magic to me, but reasserting link up works.
   
   One last thing, in case the uplink is eth0 and wlan0 is not used in STA mode: according to my tests there is no penalty to using ap0 instead of wlan0 as the AP interface. In other words, you don't need both an hostapd-wlan0.conf and an hostpad-ap0.conf, you can always run the AP off ap0.

That's it. Caveat emptor and good luck :)