Re: [PATCH v1 01/12] configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext

[Alex Bennée]

Robert Foley <robert.foley@linaro.org> writes:

> From: Lingfeng Yang <lfy@google.com>
>
> We tried running QEMU under tsan in 2016, but tsan's lack of support for
> longjmp-based fibers was a blocker:
>   https://groups.google.com/forum/#!topic/thread-sanitizer/se0YuzfWazw
>
> Fortunately, thread sanitizer gained fiber support in early 2019:
>   https://reviews.llvm.org/D54889
>
> This patch brings tsan support upstream by importing the patch that annotated
> QEMU's coroutines as tsan fibers in Android's QEMU fork:
>   https://android-review.googlesource.com/c/platform/external/qemu/+/844675
>
> Tested with '--enable-tsan --cc=clang-9 --cxx=clang++-9 --disable-werror'
> configure flags.
>
> Signed-off-by: Lingfeng Yang <lfy@google.com>
> Signed-off-by: Emilio G. Cota <cota@braap.org>
> [cota: minor modifications + configure changes]
> Signed-off-by: Robert Foley <robert.foley@linaro.org>
> [RF: Error out in configure if tsan not available, fix checkpatch warnings]
> ---
>  configure                 | 41 +++++++++++++++++
>  util/coroutine-ucontext.c | 97 +++++++++++++++++++++++++++++++++++----
>  2 files changed, 129 insertions(+), 9 deletions(-)
>
> diff --git a/configure b/configure
> index b969dee675..c18efae65e 100755
> --- a/configure
> +++ b/configure
> @@ -395,6 +395,7 @@ gprof="no"
>  debug_tcg="no"
>  debug="no"
>  sanitizers="no"
> +tsan="no"
>  fortify_source=""
>  strip_opt="yes"
>  tcg_interpreter="no"
> @@ -1150,6 +1151,10 @@ for opt do
>    ;;
>    --disable-sanitizers) sanitizers="no"
>    ;;
> +  --enable-tsan) tsan="yes"
> +  ;;
> +  --disable-tsan) tsan="no"
> +  ;;
>    --enable-sparse) sparse="yes"
>    ;;
>    --disable-sparse) sparse="no"
> @@ -1750,6 +1755,7 @@ Advanced options (experts only):
>    --with-pkgversion=VERS   use specified string as sub-version of the package
>    --enable-debug           enable common debug build options
>    --enable-sanitizers      enable default sanitizers
> +  --enable-tsan            enable thread sanitizer
>    --disable-strip          disable stripping binaries
>    --disable-werror         disable compilation abort on warning
>    --disable-stack-protector disable compiler-provided stack protection
> @@ -6192,6 +6198,27 @@ if test "$fuzzing" = "yes" ; then
>    fi
>  fi
>  
> +# Thread sanitizer is, for now, much noisier than the other sanitizers;
> +# keep it separate until that is not the case.

I think we also need to stop both being enabled at once. On my clang-9
setup I get:

  make: *** [qapi/qobject-output-visitor.o] Error 1
  clang: error: invalid argument '-fsanitize=address' not allowed with 
'-fsanitize=thread'
  clang: error: invalid argument '-fsanitize=address' not allowed with 
'-fsanitize=thread'
  clang: errorclang: : errorinvalid argument '-fsanitize=address' not allowed 
with '-fsanitize=thread':
  invalid argument '-fsanitize=address' not allowed with '-fsanitize=thread'
  clang: error: invalid argument '-fsanitize=address' not allowed with 
'-fsanitize=thread'

> +have_tsan=no
> +have_tsan_iface_fiber=no
> +if test "$tsan" = "yes" ; then
> +  write_c_skeleton
> +  if compile_prog "$CPU_CFLAGS -Werror -fsanitize=thread" "" ; then
> +      have_tsan=yes
> +  fi
> +  cat > $TMPC << EOF
> +#include <sanitizer/tsan_interface.h>
> +int main(void) {
> +  __tsan_create_fiber(0);
> +  return 0;
> +}
> +EOF
> +  if compile_prog "$CPU_CFLAGS -Werror -fsanitize=thread" "" ; then
> +      have_tsan_iface_fiber=yes
> +  fi
> +fi
> +
>  ##########################################
>  # check for libpmem
>  
> @@ -6293,6 +6320,16 @@ if test "$have_asan" = "yes"; then
>             "Without code annotation, the report may be inferior."
>    fi
>  fi
> +if test "$have_tsan" = "yes" ; then
> +  if test "$have_tsan_iface_fiber" = "yes" ; then
> +    QEMU_CFLAGS="-fsanitize=thread $QEMU_CFLAGS"
> +    QEMU_LDFLAGS="-fsanitize=thread $QEMU_LDFLAGS"
> +  else
> +    error_exit "Cannot enable TSAN due to missing fiber annotation 
> interface."
> +  fi
> +elif test "$tsan" = "yes" ; then
> +  error_exit "Cannot enable TSAN due to missing sanitize thread interface."
> +fi
>  if test "$have_ubsan" = "yes"; then
>    QEMU_CFLAGS="-fsanitize=undefined $QEMU_CFLAGS"
>    QEMU_LDFLAGS="-fsanitize=undefined $QEMU_LDFLAGS"
> @@ -7382,6 +7419,10 @@ if test "$have_asan_iface_fiber" = "yes" ; then
>      echo "CONFIG_ASAN_IFACE_FIBER=y" >> $config_host_mak
>  fi


Are we missing any LDFLAGS? On Ubuntu 18.04 with clang-9 I'm seeing:

    LINK    qemu-ga
  
/usr/lib/llvm-9/lib/clang/9.0.0/lib/linux/libclang_rt.tsan-x86_64.a(tsan_rtl_amd64.S.o):
 warning: common of `__interception::real_setjmp' overridden by definition
  
/usr/lib/llvm-9/lib/clang/9.0.0/lib/linux/libclang_rt.tsan-x86_64.a(tsan_interceptors.cc.o):
 warning: defined here
  
/usr/lib/llvm-9/lib/clang/9.0.0/lib/linux/libclang_rt.tsan-x86_64.a(tsan_rtl_amd64.S.o):
 warning: common of `__interception::real__setjmp' overridden by definition
  
/usr/lib/llvm-9/lib/clang/9.0.0/lib/linux/libclang_rt.tsan-x86_64.a(tsan_interceptors.cc.o):
 warning: defined here
  
/usr/lib/llvm-9/lib/clang/9.0.0/lib/linux/libclang_rt.tsan-x86_64.a(tsan_rtl_amd64.S.o):
 warning: common of `__interception::real_sigsetjmp' overridden by definition
  
/usr/lib/llvm-9/lib/clang/9.0.0/lib/linux/libclang_rt.tsan-x86_64.a(tsan_interceptors.cc.o):
 warning: defined here
  
/usr/lib/llvm-9/lib/clang/9.0.0/lib/linux/libclang_rt.tsan-x86_64.a(tsan_rtl_amd64.S.o):
 warning: common of `__interception::real___sigsetjmp' overridden by definition
  
/usr/lib/llvm-9/lib/clang/9.0.0/lib/linux/libclang_rt.tsan-x86_64.a(tsan_interceptors.cc.o):
 warning: defined here
  
libqemuutil.a(osdep.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:41:
 multiple definition of `__tsan_mutex_linker_init'
  
libqemuutil.a(control.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:41:
 first defined here
  
libqemuutil.a(osdep.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:50:
 multiple definition of `__tsan_mutex_not_static'
  
libqemuutil.a(control.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:50:
 first defined here
  
libqemuutil.a(osdep.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:55:
 multiple definition of `__tsan_mutex_read_lock'
  
libqemuutil.a(control.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:55:
 first defined here
  
libqemuutil.a(osdep.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:45:
 multiple definition of `__tsan_mutex_read_reentrant'
  
libqemuutil.a(control.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:45:
 first defined here
  
libqemuutil.a(osdep.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:64:
 multiple definition of `__tsan_mutex_recursive_lock'
  
libqemuutil.a(control.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:64:
 first defined here
  
libqemuutil.a(osdep.o):/usr/lib/llvm-9/lib/clang/9.0.0/include/sanitizer/tsan_interface.h:68:
 multiple definition of `__tsan_mutex_recursive_unlock'

<snip>

-- 
Alex Bennée