Improve AAC encoding: disable band limit filter

[ValdikSS]

FDK-AAC software encoder in Android limits the bandwidth up to 17 kHz max for bitrates >= 192 kbit/s stereo.
Lower bitrates get capped more:

• <144 kb/s: 16120 Hz
• <112 kb/s: 15500 Hz
• <80 kb/s: 14260 Hz

It's unclear why the bandwidth is set to 17 kHz max for CBR encoding, as the best VBR mode sets it to 20 kHz (maximum possible value, the filter could not be disabled completely). However VBR mode is not used for Bluetooth in Android.

FDK-AAC could be patched to set 20000 Hz for bitrates higher than 192 kbit/s with just a search&replace binary patch of the table linked above (no code patching required).

This is applicable only to software-only encoding (not "bluetooth offloading" with DSP).

[anonymix007]

This is applicable only to software-only encoding (not "bluetooth offloading" with DSP).

Yeah, and that's the issue. Sure, I can patch that library (diff would be appreciated though, I'll probably apply it to the upcoming A15 version of the module), but it doesn't matter because there's literally no point in disabling hw offload. And Qualcomm limits their encoder to 165 kbps anyway. And my OnePlus Buds 2 Pro are limited even further, that is, to 131 kbps.

If anything, one should go for XPAN (aka Bluetooth-over-IP) if the best audio quality is required.

[ValdikSS]

Apparently, starting from Android 11.0.0_r7, there's persist.bluetooth.a2dp_aac.vbr_supported prop which enables VBR 5 mode (the best one), which in turn sets band filter to 20 kHz of FDK-AAC.
So the fix could be as easy as setting this prop.

Also, Android sets peak bitrate to the maximum which could fit in a single packet according to the MTU, so the audio quality should be higher in VBR mode anyway (average bitrate would be as negotiated with A2DP capabilities).

Here's the log from Pixel 8a running GrapheneOS, connecting CMF Buds 2 Plus (128 kbit/s bitrate limit):

06-07 23:37:22.523  2077  2554 I bluetooth-a2dp: system/stack/a2dp/a2dp_aac_encoder.cc:189 a2dp_aac_encoder_update: sample_rate=44100 bits_per_sample=16 channel_count=2
06-07 23:37:22.523  2077  2554 I bluetooth-a2dp: system/stack/a2dp/a2dp_aac_encoder.cc:198 a2dp_aac_encoder_update: MTU=663, peer_mtu=667
06-07 23:37:22.523  2077  2554 I bluetooth-a2dp: system/stack/a2dp/a2dp_aac_encoder.cc:199 a2dp_aac_encoder_update: sample_rate: 44100 channel_mode: 4
06-07 23:37:22.523  2077  2554 I bluetooth-a2dp: system/stack/a2dp/a2dp_aac_encoder.cc:271 a2dp_aac_encoder_update: MTU = 663 Sampling Frequency = 44100 Bit Rate = 131072
06-07 23:37:22.523  2077  2554 I bluetooth-a2dp: system/stack/a2dp/a2dp_aac_encoder.cc:353 a2dp_aac_encoder_update: AACENC_BITRATEMODE: 5
06-07 23:37:22.537  2077  2554 I bluetooth-a2dp: system/stack/a2dp/a2dp_aac_encoder.cc:379 a2dp_aac_encoder_update: AAC frame_length = 1024 input_channels_n = 2 max_encoded_buffer_bytes = 1536

Peak bitrate is set to 228424 bit/s, which could be seen in Mediatek-based Android 13 device with obvious modifications to the stock aac encoding stack (note negotiation of VBR 4 mode and peak bitrate printf):

06-07 23:35:08.827  4912  8397 I a2dp_aac_encoder: a2dp_aac_encoder_update: a2dp_aac_encoder_update: sample_rate=44100 bits_per_sample=16 channel_count=2
06-07 23:35:08.827  4912  8397 I a2dp_aac_encoder: a2dp_aac_encoder_update: a2dp_aac_encoder_update: MTU=663, peer_mtu=667
06-07 23:35:08.827  4912  8397 I a2dp_aac_encoder: a2dp_aac_encoder_update: a2dp_aac_encoder_update: sample_rate: 44100 channel_mode: 4 
06-07 23:35:08.827  4912  8397 I a2dp_aac_encoder: a2dp_aac_encoder_update: a2dp_aac_encoder_update: MTU = 663 Sampling Frequency = 44100 Bit Rate = 131072 Peak Bit Rate = 228424
06-07 23:35:08.827  4912  8397 I a2dp_aac_encoder: a2dp_aac_encoder_update: a2dp_aac_encoder_update: AAC VBR is enable, maximum vbr mode 4
06-07 23:35:08.827  4912  8397 I a2dp_aac_encoder: a2dp_aac_encoder_update: a2dp_aac_encoder_update: AACENC_BITRATEMODE: 4
06-07 23:35:08.828  4912  8397 I a2dp_aac_encoder: a2dp_aac_encoder_update: a2dp_aac_encoder_update: AAC frame_length = 1024 input_channels_n = 2 max_encoded_buffer_bytes = 1536

[ValdikSS]

Yep, if both the smartphone and the earphones support announce VBR support, Android just enables VBR 5 with peak bitrate calculated from MTU, completely ignoring the negotiated A2DP bitrate (it doesn't do anything in fdk-aac in VBR mode, only peak bitrate matters).

On Pixel 8a, with hardware encoding, it takes bitrate into consideration. With FDK-AAC, it does not.