Just configured stubby on my server, but UDP replies seem to cope from one of the other configured addresses than the query was actually sent to:
18:23:04.133483 IP 192.168.60.157.39923 > 192.168.128.60.53: 40098+ A? slashdot.org. (30)
18:23:04.211034 IP 192.168.128.54.53 > 192.168.60.157.39923: 40098 2/0/1 A 104.18.4.215, A 104.18.5.215 (73)
Perhaps a bug in getdns? I will have a poke around.