问题[mqtt](server)

我尝试从我的 MACOS 对 MQTT 进行负载测试，并且能够成功实现超过 12k 的连接，直到我的带宽耗尽。

我尝试在 GCP 机器上进行相同的测试，一旦打开的端口达到 2048 和 2048 与 MQTT 代理的连接，它就会给我连接超时异常。

连接时，我的 ConnectionTimeout 为 100s（等待 conack）和 KeepAlive = 300s（一旦建立连接）

无论负载测试软件是什么，例如 mzbench、jmeter 和 emqtt-bench，都会发生此问题。所以，我认为这个问题与linux服务器有关。

我不希望实现 100 万个开放连接，而是寻找至少 30K 开放连接。

已经尝试更改 ulimit，这些是我的 ulimit 配置

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 63887
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 102400
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 200000
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

proc 上的 cat 也将最大文件打开为 102400

这些也是我的 sysctl 中设置的值

fs.file-max = 200000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_mem = 50576   64768   98152
net.core.netdev_max_backlog = 2500

编辑：添加了机器详细信息和测试模式

机器类型：n2-highcpu-16（16 个 vCPU，16 GB 内存）

lscpu 的结果

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                16
On-line CPU(s) list:   0-15
Thread(s) per core:    2
Core(s) per socket:    8
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) CPU
Stepping:              7
CPU MHz:               2800.200
BogoMIPS:              5600.40
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              33792K
NUMA node0 CPU(s):     0-15
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat avx512_vnni md_clear arch_capabilities

测试模式：每秒打开 200 个连接并以恒定速率等待 conack。

我正在使用 helm 在我的 kubernetes 集群上安装 vernemq

问题是它无法启动，我接受了 EULA

这是日志：

02:31:56.552 [error] CRASH REPORT Process <0.195.0> with 0 neighbours exited with reason: {{{badmatch,{error,{vmq_generic_msg_store,{bad_return,{{vmq_generic_msg_store_app,start,[normal,[]]},{'EXIT',{{badmatch,{error,{{undef,[{eleveldb,validate_options,[open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{data,[{dir,"./data"}]},{data_root,"./data/leveldb"},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{store_dir,"./data..."},...]],...},...]},...}}},...}}}}}}},...},...} in application_master:init/4 line 138
02:31:56.552 [info] Application vmq_server exited with reason: {{{badmatch,{error,{vmq_generic_msg_store,{bad_return,{{vmq_generic_msg_store_app,start,[normal,[]]},{'EXIT',{{badmatch,{error,{{undef,[{eleveldb,validate_options,[open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{data,[{dir,"./data"}]},{data_root,"./data/leveldb"},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{store_dir,"./data..."},...]],...},...]},...}}},...}}}}}}},...},...}
Kernel pid terminated (application_controller) ({application_start_failure,vmq_server,{bad_return,{{vmq_server_app,start,[normal,[]]},{'EXIT',{{{badmatch,{error,{vmq_generic_msg_store,{bad_return,{{vm

{"Kernel pid terminated",application_controller,"{application_start_failure,vmq_server,{bad_return,{{vmq_server_app,start,[normal,[]]},{'EXIT',{{{badmatch,{error,{vmq_generic_msg_store,{bad_return,{{vmq_generic_msg_store_app,start,[normal,[]]},{'EXIT',{{badmatch,{error,{{undef,[{eleveldb,validate_options,[open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{data,[{dir,\"./data\"}]},{data_root,\"./data/leveldb\"},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{store_dir,\"./data/msgstore\"},{sync,false},{tiered_slow_level,0},{total_leveldb_mem_percent,70},{use_bloomfilter,true},{verify_checksums,true},{verify_compaction,true},{write_buffer_size,41777529},{write_buffer_size_max,62914560},{write_buffer_size_min,31457280}]],[]},{vmq_storage_engine_leveldb,init_state,2,[{file,\"/opt/vernemq/apps/vmq_generic_msg_store/src/engines/vmq_storage_engine_leveldb.erl\"},{line,99}]},{vmq_storage_engine_leveldb,open,2,[{file,\"/opt/vernemq/apps/vmq_generic_msg_store/src/engines/vmq_storage_engine_leveldb.erl\"},{line,39}]},{vmq_generic_msg_store,init,1,[{file,\"/opt/vernemq/apps/vmq_generic_msg_store/src/vmq_generic_msg_store.erl\"},{line,181}]},{gen_server,init_it,2,[{file,\"gen_server.erl\"},{line,374}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,342}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,249}]}]},{child,undefined,{vmq_generic_msg_store_bucket,1},{vmq_generic_msg_store,start_link,[1]},permanent,5000,worker,[vmq_generic_msg_store]}}}},[{vmq_generic_msg_store_sup,'-start_link/0-lc$^0/1-0-',2,[{file,\"/opt/vernemq/apps/vmq_generic_msg_store/src/vmq_generic_msg_store_sup.erl\"},{line,40}]},{vmq_generic_msg_store_sup,start_link,0,[{file,\"/opt/vernemq/apps/vmq_generic_msg_store/src/vmq_generic_msg_store_sup.erl\"},{line,42}]},{application_master,start_it_old,4,[{file,\"application_master.erl\"},{line,277}]}]}}}}}}},[{vmq_plugin_mgr,start_plugin,1,[{file,\"/opt/vernemq/apps/vmq_plugin/src/vmq_plugin_mgr.erl\"},{line,524}]},{vmq_plugin_mgr,start_plugins,1,[{file,\"/opt/vernemq/apps/vmq_plugin/src/vmq_plugin_mgr.erl\"},{line,503}]},{vmq_plugin_mgr,check_updated_plugins,2,[{file,\"/opt/vernemq/apps/vmq_plugin/src/vmq_plugin_mgr.erl\"},{line,444}]},{vmq_plugin_mgr,handle_plugin_call,2,[{file,\"/opt/vernemq/apps/vmq_plugin/src/vmq_plugin_mgr.erl\"},{line,246}]},{gen_server,try_handle_call,4,[{file,\"gen_server.erl\"},{line,661}]},{gen_server,handle_msg,6,[{file,\"gen_server.erl\"},{line,690}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,249}]}]},{gen_server,call,[vmq_plugin_mgr,{enable_system_plugin,vmq_generic_msg_store,[internal]},infinity]}}}}}}"}
Crash dump is being written to: /erl_crash.dump...

所以我的问题在哪里，我只是helm install vernemq vernemq/vernemq用来安装它。