[Lilux-info] Gull Luxembourg - presentation bcache

Début du message réexpédié : De: Frédéric Dumas <f.dumas(a)ellis.siteparc.fr> Objet: Bcache on Dell R710 + P3700 - Benchmark Date: 18 novembre 2021 à 17:24:53 UTC+2 À: Brent Frère <Brent.Frere(a)AbilIT.eu> Cc: Michał Rokitiański <michal.rokitianski(a)softq.pl> Here are some benchmark results on bcache running on Dell R710 + PERC H700 + P3700 NVMe. - ! - Setup Backend device: 6 x HDD SATA 3Gbps in RAID6 hardware (PERC H700 aka LSI MegaRAID SAS 9260-8i) HDD sector size = 512 bytes RAID6 stride = stripe element size = 1024KB RAID6 stripe width = 4096KB bcache --data-offset 8192 (sectors) Cache device : 2 x P3700 NVme in RAID1 software (md) Configuration of bcache: Sequential reads through bcache # echo 0 > /sys/block/bcache0/bcache/sequential_cutoff Writeback by bcache # echo writeback > /sys/block/bcache0/bcache/cache_mode Hints: # smartctl -iA -d megaraid,{0,5} /dev/sda gives access to the smart values of the disks running behind the hardware RAID controller. # make-bcache --data-offset 8192 --bucket 2048k --writeback --bdev /dev/sda4 --cache /dev/md3700 attaches the cache device to the backend device and registers the /dev/bcach0 device. #root@softq-pve-710:~# lsblk NAME SIZE FSTYPE TYPE MOUNTPOINT sda 7.3T disk ├─sda1 1007K part ├─sda2 512M vfat part ├─sda3 31.5G LVM2_member part │ └─pve-root 7.8G ext4 lvm / └─sda4 7.2T bcache part └─bcache0 7.2T disk sr0 1024M rom nvme0n1 745.2G disk └─md3700 745.1G bcache raid1 └─bcache0 7.2T disk nvme1n1 745.2G disk └─md3700 745.1G bcache raid1 └─bcache0 7.2T disk - 2 - Summary Bcache delivers a x11 performance boost on 64KB random read/writes. It delivers a sustained x2 performance boost on 64KB chunks sequential reads; a modest result, as these reads are already accelerated by the hardware raid controller's read-ahead policy. Bcache delivers no performance boost at all on fully random 64KB reads, as they are spread over a very large 7.2TB space. Likely, such a test should be completed on a few dozen GB large file, not on an entire volume. Sequential reads without bcache (adaptative read ahead on hardware raid controller) 1.1 ~655MiB/s ~10.5k IOPS Sequential reads with bcache (adaptative read ahead on hardware raid controller) 1.2 ~1303MiB/s ~20.9k IOPS Random reads without bcache (adaptative read ahead on hardware raid controller) 2.1 ~45.5MiB/s ~728 IOPS Random reads with bcache (adaptative read ahead on hardware raid controller) 2.2 ~45.9MiB/s ~733 IOPS Random read/writes without bcache (adaptative read ahead on hardware raid controller) 3.1 ~6.0MiB/s ~93 IOPS Random read/writes with bcache (adaptative read ahead on hardware raid controller) 3.2 ~67.9MiB/s (R) ~1086 IOPS 3.2 ~68.1MiB/s (W) ~1086 IOPS Once the last read/write test was completed, it took several minutes to bcache to write the data back to the RAID6 array. - 3 - Full results Source: https://docs.oracle.com/en-us/iaas/Content/Block/References/samplefiocomman… 1.1 - Throughput Performance Test - sequential reads - WITH ADAPTIVE READ AHEAD HARDWARE CACHE + WITHOUT BCACHE root@softq-pve-710:~# fio --filename=/dev/sda4 --direct=1 --rw=read --bs=64k --ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting --name=throughput-test-job --eta-newline=1 --readonly throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1009110: Thu Nov 18 12:14:25 2021 read: IOPS=10.5k, BW=655MiB/s (687MB/s)(76.7GiB/120012msec) slat (usec): min=5, max=253, avg=20.03, stdev= 5.96 clat (usec): min=106, max=232703, avg=12192.72, stdev=10140.66 lat (usec): min=122, max=232718, avg=12213.20, stdev=10140.72 clat percentiles (usec): | 1.00th=[ 457], 5.00th=[ 1401], 10.00th=[ 2311], 20.00th=[ 3851], | 30.00th=[ 5211], 40.00th=[ 7570], 50.00th=[ 10028], 60.00th=[ 12911], | 70.00th=[ 16057], 80.00th=[ 19530], 90.00th=[ 24249], 95.00th=[ 29230], | 99.00th=[ 41681], 99.50th=[ 48497], 99.90th=[ 84411], 99.95th=[125305], | 99.99th=[204473] bw ( KiB/s): min=287872, max=1654656, per=100.00%, avg=671072.94, stdev=31210.13, samples=956 iops : min= 4498, max=25854, avg=10485.52, stdev=487.66, samples=956 lat (usec) : 250=0.01%, 500=1.32%, 750=0.95%, 1000=1.11% lat (msec) : 2=4.66%, 4=12.84%, 10=28.89%, 20=31.69%, 50=18.12% lat (msec) : 100=0.35%, 250=0.08% cpu : usr=2.95%, sys=10.94%, ctx=1069436, majf=0, minf=2128 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued rwts: total=1257420,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=32 Run status group 0 (all jobs): READ: bw=655MiB/s (687MB/s), 655MiB/s-655MiB/s (687MB/s-687MB/s), io=76.7GiB (82.4GB), run=120012-120012msec Disk stats (read/write): sda: ios=1255705/246, merge=254/64, ticks=15282727/6083, in_queue=15288810, util=99.99% 1.2 - Throughput Performance Test - sequential reads - WITH ADAPTIVE READ AHEAD HARDWARE CACHE + WITH BCACHE root@softq-pve-710:~# fio --filename=/dev/bcache0 --direct=1 --rw=read --bs=64k --ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting --name=throughput-test-job --eta-newline=1 --readonly throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1027176: Thu Nov 18 14:42:06 2021 read: IOPS=20.9k, BW=1303MiB/s (1367MB/s)(153GiB/120038msec) slat (usec): min=7, max=39687, avg=138.05, stdev=199.59 clat (usec): min=54, max=755004, avg=5996.18, stdev=11372.86 lat (usec): min=74, max=755048, avg=6134.93, stdev=11369.60 clat percentiles (usec): | 1.00th=[ 416], 5.00th=[ 2442], 10.00th=[ 3261], 20.00th=[ 3884], | 30.00th=[ 4228], 40.00th=[ 4555], 50.00th=[ 4817], 60.00th=[ 5145], | 70.00th=[ 5538], 80.00th=[ 5997], 90.00th=[ 6915], 95.00th=[ 9896], | 99.00th=[ 34866], 99.50th=[ 44827], 99.90th=[113771], 99.95th=[261096], | 99.99th=[541066] bw ( MiB/s): min= 48, max= 1701, per=100.00%, avg=1307.64, stdev=109.03, samples=956 iops : min= 768, max=27222, avg=20921.69, stdev=1744.45, samples=956 lat (usec) : 100=0.01%, 250=0.02%, 500=1.66%, 750=1.04%, 1000=0.30% lat (msec) : 2=0.98%, 4=19.10%, 10=71.97%, 20=2.18%, 50=2.37% lat (msec) : 100=0.25%, 250=0.06%, 500=0.05%, 750=0.01%, 1000=0.01% cpu : usr=3.24%, sys=36.24%, ctx=3406755, majf=0, minf=34793 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued rwts: total=2503357,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=32 Run status group 0 (all jobs): READ: bw=1303MiB/s (1367MB/s), 1303MiB/s-1303MiB/s (1367MB/s-1367MB/s), io=153GiB (164GB), run=120038-120038msec Disk stats (read/write): bcache0: ios=2502109/0, merge=0/0, ticks=7149936/0, in_queue=7149936, util=99.86%, aggrios=1262209/1106438, aggrmerge=4/54, aggrticks=2461402/1258470, aggrin_queue=3719872, aggrutil=99.02% sda: ios=2059550/295, merge=9/108, ticks=4031424/35305, in_queue=4066729, util=99.02% md3700: ios=464868/2212581, merge=0/0, ticks=891380/2481636, in_queue=3373016, util=95.12%, aggrios=232423/2114558, aggrmerge=11/101552, aggrticks=398099/1256745, aggrin_queue=1654844, aggrutil=95.05% nvme0n1: ios=238263/2115522, merge=12/100588, ticks=410475/1286394, in_queue=1696870, util=95.05% nvme1n1: ios=226583/2113594, merge=10/102516, ticks=385723/1227097, in_queue=1612819, util=94.99% 2.1 - Throughput Performance Test - random reads - WITH ADAPTIVE READ AHEAD HARDWARE CACHE + WITHOUT BCACHE root@softq-pve-710:~# fio --filename=/dev/sda4 --direct=1 --rw=randread --bs=64k --ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting --name=throughput-test-job --eta-newline=1 --readonly throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1010415: Thu Nov 18 12:24:50 2021 read: IOPS=728, BW=45.5MiB/s (47.7MB/s)(5474MiB/120290msec) slat (usec): min=9, max=350, avg=23.57, stdev= 3.17 clat (usec): min=164, max=2280.6k, avg=175684.34, stdev=159256.80 lat (usec): min=187, max=2280.6k, avg=175708.55, stdev=159256.79 clat percentiles (msec): | 1.00th=[ 25], 5.00th=[ 31], 10.00th=[ 38], 20.00th=[ 54], | 30.00th=[ 74], 40.00th=[ 99], 50.00th=[ 127], 60.00th=[ 163], | 70.00th=[ 207], 80.00th=[ 271], 90.00th=[ 380], 95.00th=[ 489], | 99.00th=[ 760], 99.50th=[ 894], 99.90th=[ 1183], 99.95th=[ 1351], | 99.99th=[ 1636] bw ( KiB/s): min=33664, max=58368, per=100.00%, avg=46646.60, stdev=1045.77, samples=960 iops : min= 526, max= 912, avg=728.84, stdev=16.35, samples=960 lat (usec) : 250=0.01% lat (msec) : 20=0.14%, 50=17.84%, 100=22.94%, 250=36.48%, 500=17.98% lat (msec) : 750=3.58%, 1000=0.77%, 2000=0.27%, >=2000=0.01% cpu : usr=0.36%, sys=1.04%, ctx=86189, majf=0, minf=2109 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued rwts: total=87587,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=32 Run status group 0 (all jobs): READ: bw=45.5MiB/s (47.7MB/s), 45.5MiB/s-45.5MiB/s (47.7MB/s-47.7MB/s), io=5474MiB (5740MB), run=120290-120290msec Disk stats (read/write): sda: ios=87540/205, merge=0/62, ticks=15349921/28134, in_queue=15378056, util=99.99% 2.2 - Throughput Performance Test - random reads - WITH ADAPTIVE READ AHEAD HARDWARE CACHE + WITH BCACHE root@softq-pve-710:~# fio --filename=/dev/bcache0 --direct=1 --rw=randread --bs=64k --ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting --name=throughput-test-job --eta-newline=1 --readonly throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1030606: Thu Nov 18 14:55:16 2021 read: IOPS=733, BW=45.9MiB/s (48.1MB/s)(5522MiB/120391msec) slat (usec): min=21, max=6740, avg=58.67, stdev=90.27 clat (usec): min=234, max=2413.0k, avg=174256.84, stdev=157810.22 lat (usec): min=273, max=2413.1k, avg=174316.24, stdev=157809.87 clat percentiles (msec): | 1.00th=[ 23], 5.00th=[ 31], 10.00th=[ 37], 20.00th=[ 54], | 30.00th=[ 73], 40.00th=[ 97], 50.00th=[ 127], 60.00th=[ 163], | 70.00th=[ 207], 80.00th=[ 268], 90.00th=[ 372], 95.00th=[ 481], | 99.00th=[ 760], 99.50th=[ 877], 99.90th=[ 1183], 99.95th=[ 1334], | 99.99th=[ 1687] bw ( KiB/s): min=33328, max=57955, per=100.00%, avg=47075.48, stdev=1075.87, samples=960 iops : min= 520, max= 905, avg=734.54, stdev=16.81, samples=960 lat (usec) : 250=0.01%, 500=0.44%, 750=0.06%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 20=0.18%, 50=17.58%, 100=22.79% lat (msec) : 250=36.41%, 500=18.09%, 750=3.40%, 1000=0.79%, 2000=0.23% lat (msec) : >=2000=0.01% cpu : usr=0.31%, sys=1.98%, ctx=87526, majf=0, minf=2105 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued rwts: total=88357,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=32 Run status group 0 (all jobs): READ: bw=45.9MiB/s (48.1MB/s), 45.9MiB/s-45.9MiB/s (48.1MB/s-48.1MB/s), io=5522MiB (5791MB), run=120391-120391msec Disk stats (read/write): bcache0: ios=88357/0, merge=0/0, ticks=15365196/0, in_queue=15365196, util=100.00%, aggrios=44195/44624, aggrmerge=0/26, aggrticks=7679901/20858, aggrin_queue=7700759, aggrutil=100.00% sda: ios=87899/210, merge=0/53, ticks=15359634/28724, in_queue=15388358, util=100.00% md3700: ios=492/89038, merge=0/0, ticks=168/12992, in_queue=13160, util=98.35%, aggrios=246/89109, aggrmerge=0/668, aggrticks=98/9289, aggrin_queue=9388, aggrutil=98.16% nvme0n1: ios=489/89108, merge=0/670, ticks=196/8879, in_queue=9075, util=98.16% nvme1n1: ios=3/89111, merge=0/667, ticks=1/9700, in_queue=9702, util=98.15% 3.1 - Throughput Performance Test - random read/writes - WITH ADAPTIVE READ AHEAD HARDWARE CACHE + WITHOUT BCACHE root@softq-pve-710:~# fio --filename=/dev/sda4 --direct=1 --rw=randrw --bs=64k --ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting --name=throughput-test-job --eta-newline=1 throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1011093: Thu Nov 18 12:30:15 2021 read: IOPS=93, BW=5998KiB/s (6142kB/s)(707MiB/120737msec) slat (usec): min=10, max=224, avg=25.81, stdev= 2.94 clat (msec): min=11, max=4247, avg=347.85, stdev=434.68 lat (msec): min=11, max=4247, avg=347.87, stdev=434.68 clat percentiles (msec): | 1.00th=[ 26], 5.00th=[ 34], 10.00th=[ 41], 20.00th=[ 55], | 30.00th=[ 80], 40.00th=[ 118], 50.00th=[ 169], 60.00th=[ 243], | 70.00th=[ 363], 80.00th=[ 592], 90.00th=[ 927], 95.00th=[ 1234], | 99.00th=[ 1989], 99.50th=[ 2400], 99.90th=[ 3205], 99.95th=[ 3406], | 99.99th=[ 4178] bw ( KiB/s): min= 1152, max=12928, per=100.00%, avg=6014.06, stdev=506.65, samples=960 iops : min= 18, max= 202, avg=93.97, stdev= 7.92, samples=960 write: IOPS=96, BW=6204KiB/s (6353kB/s)(732MiB/120737msec); 0 zone resets slat (usec): min=13, max=294, avg=31.81, stdev= 4.31 clat (msec): min=114, max=5783, avg=983.56, stdev=411.47 lat (msec): min=114, max=5783, avg=983.60, stdev=411.47 clat percentiles (msec): | 1.00th=[ 376], 5.00th=[ 510], 10.00th=[ 584], 20.00th=[ 676], | 30.00th=[ 760], 40.00th=[ 827], 50.00th=[ 894], 60.00th=[ 969], | 70.00th=[ 1083], 80.00th=[ 1217], 90.00th=[ 1502], 95.00th=[ 1770], | 99.00th=[ 2433], 99.50th=[ 2702], 99.90th=[ 3440], 99.95th=[ 3943], | 99.99th=[ 5269] bw ( KiB/s): min= 1792, max=11520, per=99.87%, avg=6196.98, stdev=410.22, samples=960 iops : min= 28, max= 180, avg=96.82, stdev= 6.41, samples=960 lat (msec) : 20=0.07%, 50=8.43%, 100=9.20%, 250=12.28%, 500=9.98% lat (msec) : 750=16.81%, 1000=20.10%, 2000=21.25%, >=2000=1.89% cpu : usr=0.11%, sys=0.34%, ctx=22938, majf=0, minf=71 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=99.5%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued rwts: total=11315,11704,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=32 Run status group 0 (all jobs): READ: bw=5998KiB/s (6142kB/s), 5998KiB/s-5998KiB/s (6142kB/s-6142kB/s), io=707MiB (742MB), run=120737-120737msec WRITE: bw=6204KiB/s (6353kB/s), 6204KiB/s-6204KiB/s (6353kB/s-6353kB/s), io=732MiB (767MB), run=120737-120737msec Disk stats (read/write): sda: ios=11370/11806, merge=0/45, ticks=3916382/11605103, in_queue=15521484, util=99.99% 3.2 - Throughput Performance Test - random read/writes - WITH ADAPTIVE READ AHEAD HARDWARE CACHE + WITH BCACHE root@softq-pve-710:~# fio --filename=/dev/bcache0 --direct=1 --rw=randrw --bs=64k --ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting --name=throughput-test-job --eta-newline=1 throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1031738: Thu Nov 18 15:04:05 2021 read: IOPS=1086, BW=67.9MiB/s (71.2MB/s)(8161MiB/120229msec) slat (usec): min=15, max=86127, avg=51.75, stdev=568.52 clat (usec): min=188, max=2007.7k, avg=116178.43, stdev=151824.06 lat (usec): min=242, max=2007.7k, avg=116230.79, stdev=151836.54 clat percentiles (usec): | 1.00th=[ 441], 5.00th=[ 652], 10.00th=[ 832], | 20.00th=[ 1385], 30.00th=[ 2999], 40.00th=[ 36439], | 50.00th=[ 61604], 60.00th=[ 96994], 70.00th=[ 141558], | 80.00th=[ 204473], 90.00th=[ 308282], 95.00th=[ 413139], | 99.00th=[ 683672], 99.50th=[ 801113], 99.90th=[1082131], | 99.95th=[1216349], 99.99th=[1535116] bw ( KiB/s): min=30834, max=1256048, per=100.00%, avg=69675.99, stdev=38920.70, samples=957 iops : min= 481, max=19625, avg=1087.75, stdev=608.16, samples=957 write: IOPS=1089, BW=68.1MiB/s (71.4MB/s)(8190MiB/120229msec); 0 zone resets slat (usec): min=14, max=50364, avg=47.67, stdev=339.27 clat (usec): min=11, max=165668, avg=1454.15, stdev=2560.77 lat (usec): min=74, max=165915, avg=1502.47, stdev=2590.82 clat percentiles (usec): | 1.00th=[ 71], 5.00th=[ 73], 10.00th=[ 80], 20.00th=[ 119], | 30.00th=[ 135], 40.00th=[ 139], 50.00th=[ 145], 60.00th=[ 169], | 70.00th=[ 1221], 80.00th=[ 3359], 90.00th=[ 5080], 95.00th=[ 6194], | 99.00th=[ 8586], 99.50th=[ 9503], 99.90th=[14484], 99.95th=[21627], | 99.99th=[26608] bw ( KiB/s): min=23655, max=1247613, per=100.00%, avg=69995.69, stdev=39278.35, samples=957 iops : min= 368, max=19493, avg=1092.74, stdev=613.74, samples=957 lat (usec) : 20=0.01%, 50=0.01%, 100=7.71%, 250=25.16%, 500=1.76% lat (usec) : 750=3.40%, 1000=3.65% lat (msec) : 2=7.17%, 4=9.16%, 10=8.75%, 20=0.24%, 50=5.96% lat (msec) : 100=7.50%, 250=12.11%, 500=5.94%, 750=1.17%, 1000=0.24% lat (msec) : 2000=0.08%, >=2000=0.01% cpu : usr=0.76%, sys=3.07%, ctx=225226, majf=1, minf=10344 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued rwts: total=130582,131041,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=32 Run status group 0 (all jobs): READ: bw=67.9MiB/s (71.2MB/s), 67.9MiB/s-67.9MiB/s (71.2MB/s-71.2MB/s), io=8161MiB (8558MB), run=120229-120229msec WRITE: bw=68.1MiB/s (71.4MB/s), 68.1MiB/s-68.1MiB/s (71.4MB/s-71.4MB/s), io=8190MiB (8588MB), run=120229-120229msec Disk stats (read/write): bcache0: ios=130619/131041, merge=0/0, ticks=15128128/174484, in_queue=15302612, util=100.00%, aggrios=65454/110552, aggrmerge=0/34, aggrticks=7565070/131037, aggrin_queue=7696107, aggrutil=99.57% sda: ios=86501/300, merge=0/68, ticks=15072824/78563, in_queue=15151387, util=99.57% md3700: ios=44408/220805, merge=0/0, ticks=57316/183512, in_queue=240828, util=98.53%, aggrios=22202/221689, aggrmerge=0/548, aggrticks=28755/138402, aggrin_queue=167156, aggrutil=98.41% nvme0n1: ios=22193/221688, merge=0/549, ticks=28763/138527, in_queue=167289, util=98.39% nvme1n1: ios=22212/221690, merge=0/547, ticks=28747/138277, in_queue=167024, util=98.41% -- Frédéric Dumas f.dumas(a)ellis.siteparc.fr