Dpdk often stuck in write log to /var/log/message when start VM? #4
Description
1, The backtrace for vhost that it hit this issue
(gdb) info threads
Id Target Id Frame
7 Thread 0x7faa860ed700 (LWP 2179359) "eal-intr-thread" 0x00007faa86612923 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
6 Thread 0x7faa858ec700 (LWP 2179363) "reactor_4" 0x0000000000485e27 in reactor_post_process_lw_thread ()
5 Thread 0x7faa850eb700 (LWP 2179364) "reactor_5" 0x00000000004862ec in reactor_run ()
4 Thread 0x7faa848ea700 (LWP 2179365) "reactor_6" 0x00000000004b4612 in spdk_get_ticks ()
3 Thread 0x7faa840e9700 (LWP 2179366) "vhost" 0x00007faa868eb98d in accept () at ../sysdeps/unix/syscall-template.S:81
2 Thread 0x7faa7afd7700 (LWP 2179400) "vhost-events" 0x00007faa8660385d in write () at ../sysdeps/unix/syscall-template.S:81
- 1 Thread 0x7faa87b88bc0 (LWP 2179357) "reactor_3" __lll_lock_wait_private () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
(gdb) bt
#0 __lll_lock_wait_private () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
fix dpdk runtime bug based on spdk/dpdk #1 0x00007faa8656659f in _L_lock_1030 () from /lib64/libc.so.6
Release 19.05? #2 0x00007faa86561344 in _IO_vfprintf_internal (s=0x1a8df80, format=0x5a0d18 "VHOST_CONFIG: vhost-user %s: socket created, fd: %d\n", ap=0x7ffc3e263808) at vfprintf.c:1313
Dpdk build getting failed due to issue in rpath #3 0x00000000004bf355 in rte_vlog ()
Dpdk often stuck in write log to /var/log/message when start VM? #4 0x0000000000405f99 in rte_log ()
Mention move to own git hosting #5 0x00000000005091b2 in create_unix_socket ()
Acomment #6 0x000000000050a1ef in rte_vhost_driver_register ()
#7 0x00000000004833ac in vhost_register_unix_socket ()
#8 0x000000000047d708 in vhost_dev_register ()
#9 0x0000000000482d36 in spdk_vhost_blk_construct ()
#10 0x000000000047eec5 in rpc_vhost_create_blk_controller ()
#11 0x0000000000489c74 in jsonrpc_parse_request ()
#12 0x000000000048a708 in spdk_jsonrpc_server_poll ()
#13 0x0000000000486ca9 in rpc_subsystem_poll ()
#14 0x00000000004a4e61 in spdk_thread_poll ()
#15 0x0000000000486310 in reactor_run ()
#16 0x0000000000486818 in spdk_reactors_start ()
#17 0x000000000048440d in spdk_app_start ()
#18 0x000000000040938f in main ()
2, Host is rehat 7.4 and kernal version is 3.10.0-693.el7.x86_64
3, spdk and dpdk version show as below.
Starting SPDK v21.01-pre git sha1 602b134 / DPDK 20.08.0 initialization...
[2021-06-30 12:45:22.064330] [ DPDK EAL parameters: [2021-06-30 12:45:22.064341] vhost [2021-06-30 12:45:22.064348] --no-shconf [2021-06-30 12:45:22.064355] -c 0x78 [2021-06-30 12:45:22.064361] --log-level=lib.eal:6 [2021-06-30 12:45:22.064367] --log-level=lib.cryptodev:5 [2021-06-30 12:45:22.064373] --log-level=user1:6 [2021-06-30 12:45:22.064380] --base-virtaddr=0x200000000000 [2021-06-30 12:45:22.064386] --match-allocations [2021-06-30 12:45:22.064393] --file-prefix=spdk_pid15246 [2021-06-30 12:45:22.064400] ]
- qemu thread run with parameter as below.
-object memory-backend-file,id=mem0,size=4G,mem-path=/dev/hugepages,share=on -numa node,memdev=mem0 -chardev socket,id=spdk_vhost_blkc44a0c1e-df13-11eb-a280-525400a98761,path=/var/tmp/vhost.c44a0c1e-df13-11eb-a280-525400a98761,reconnect=1
-device vhost-user-blk-pci,chardev=spdk_vhost_blkc44a0c1e-df13-11eb-a280-525400a98761,bootindex=1,num-queues=2 -chardev socket,id=spdk_vhost_blk12ac81ce-62ed-4b18-ae41-79ab44b34fbb,path=/var/tmp/vhost.12ac81ce-62ed-4b18-ae41-79ab44b34fbb,reconnect=1
-device vhost-user-blk-pci,chardev=spdk_vhost_blk12ac81ce-62ed-4b18-ae41-79ab44b34fbb,num-queues=2