Dave, after some digging, I also realized that some syscalls are being handled as vdso calls, such as gettimeofday. However, if they are vdso calls, I shouldn't be able to see them using ptrace when collecting only syscalls. The weird thing is I see a lot of gettimeofday syscalls being made. Then I stumbled upon this article here:
https://stackoverflow.com/questions/42622427/gettimeofday-not-using-vdso, which describes why some vdso calls on Xen VMs are actually real syscalls.
I guess the problem I'm really having is that for some syscalls being handled as vdso calls, but due to specific virtualization platform (Xen), are eventually handled as real syscalls, their backtrace seems to be screwed up. Is there something special in those couple of vdso pages that kernel sets up that libunwind will need some kind of workaround to get a proper backtrace?