Bonjour Alain,
Pour la réunion du 11 avril du gull de Luxembourg, ce ne m'est pas possible de
présenter un travail abouti sur bcache (mnémotechnique des commandes, optimisation avec le
strip choisi pour le RAID, benchmark réel avec fio). En particulier, je n'avais
jamais été capable d'expliquer certaines mesures de performances obtenues avec
l'utilitaire fio, alors que c'est le coeur du sujet. Il me faut donc consacrer
de nouveau deux ou trois jours pour remonter une machine de test.
Deux options s'offrent à nous:
reporter à plus tard cette présentation;
accepter cette présentation "en l'état", dans un esprit d'entre-aide
et collaboratif, basée sur mes notes de l'époque, avec de possibles trous de mémoire,
c'est à dire un travail partiel sans garantie hélas d'exactitude.
Je te laisse la décision et me réjouis de vous retrouver le 11 avril, quel que soit le
sujet de la réunion.
Bien amicalement,
Frédéric.
--
f.dumas(a)ellis.siteparc.fr
Début du message réexpédié :
De: Frédéric Dumas <f.dumas(a)ellis.siteparc.fr>
Objet: Bcache on Dell R710 + P3700 - Benchmark
Date: 18 novembre 2021 à 17:24:53 UTC+2
À: Brent Frère <Brent.Frere(a)AbilIT.eu>
Cc: Michał Rokitiański <michal.rokitianski(a)softq.pl>
Here are some benchmark results on bcache running on Dell R710 + PERC H700 + P3700 NVMe.
- ! - Setup
Backend device: 6 x HDD SATA 3Gbps in RAID6 hardware (PERC H700 aka LSI MegaRAID SAS
9260-8i)
HDD sector size = 512 bytes
RAID6 stride = stripe element size = 1024KB
RAID6 stripe width = 4096KB
bcache --data-offset 8192 (sectors)
Cache device : 2 x P3700 NVme in RAID1 software (md)
Configuration of bcache:
Sequential reads through bcache # echo 0 >
/sys/block/bcache0/bcache/sequential_cutoff
Writeback by bcache # echo writeback > /sys/block/bcache0/bcache/cache_mode
Hints:
# smartctl -iA -d megaraid,{0,5} /dev/sda
gives access to the smart values of the disks running behind the hardware RAID
controller.
# make-bcache --data-offset 8192 --bucket 2048k --writeback --bdev /dev/sda4 --cache
/dev/md3700
attaches the cache device to the backend device and registers the /dev/bcach0 device.
#root@softq-pve-710:~# lsblk
NAME SIZE FSTYPE TYPE MOUNTPOINT
sda 7.3T disk
├─sda1 1007K part
├─sda2 512M vfat part
├─sda3 31.5G LVM2_member part
│ └─pve-root 7.8G ext4 lvm /
└─sda4 7.2T bcache part
└─bcache0 7.2T disk
sr0 1024M rom
nvme0n1 745.2G disk
└─md3700 745.1G bcache raid1
└─bcache0 7.2T disk
nvme1n1 745.2G disk
└─md3700 745.1G bcache raid1
└─bcache0 7.2T disk
- 2 - Summary
Bcache delivers a x11 performance boost on 64KB random read/writes. It delivers a
sustained x2 performance boost on 64KB chunks sequential reads; a modest result, as these
reads are already accelerated by the hardware raid controller's read-ahead policy.
Bcache delivers no performance boost at all on fully random 64KB reads, as they are spread
over a very large 7.2TB space. Likely, such a test should be completed on a few dozen GB
large file, not on an entire volume.
Sequential reads without bcache (adaptative read ahead on hardware raid controller)
1.1 ~655MiB/s ~10.5k IOPS
Sequential reads with bcache (adaptative read ahead on hardware raid controller)
1.2 ~1303MiB/s ~20.9k IOPS
Random reads without bcache (adaptative read ahead on hardware raid controller)
2.1 ~45.5MiB/s ~728 IOPS
Random reads with bcache (adaptative read ahead on hardware raid controller)
2.2 ~45.9MiB/s ~733 IOPS
Random read/writes without bcache (adaptative read ahead on hardware raid controller)
3.1 ~6.0MiB/s ~93 IOPS
Random read/writes with bcache (adaptative read ahead on hardware raid controller)
3.2 ~67.9MiB/s (R) ~1086 IOPS
3.2 ~68.1MiB/s (W) ~1086 IOPS
Once the last read/write test was completed, it took several minutes to bcache to write
the data back to the RAID6 array.
- 3 - Full results
Source:
https://docs.oracle.com/en-us/iaas/Content/Block/References/samplefiocomman…
1.1 - Throughput Performance Test - sequential reads - WITH ADAPTIVE READ AHEAD HARDWARE
CACHE + WITHOUT BCACHE
root@softq-pve-710:~# fio --filename=/dev/sda4 --direct=1 --rw=read --bs=64k
--ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting
--name=throughput-test-job --eta-newline=1 --readonly
throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1009110: Thu Nov 18 12:14:25 2021
read: IOPS=10.5k, BW=655MiB/s (687MB/s)(76.7GiB/120012msec)
slat (usec): min=5, max=253, avg=20.03, stdev= 5.96
clat (usec): min=106, max=232703, avg=12192.72, stdev=10140.66
lat (usec): min=122, max=232718, avg=12213.20, stdev=10140.72
clat percentiles (usec):
| 1.00th=[ 457], 5.00th=[ 1401], 10.00th=[ 2311], 20.00th=[ 3851],
| 30.00th=[ 5211], 40.00th=[ 7570], 50.00th=[ 10028], 60.00th=[ 12911],
| 70.00th=[ 16057], 80.00th=[ 19530], 90.00th=[ 24249], 95.00th=[ 29230],
| 99.00th=[ 41681], 99.50th=[ 48497], 99.90th=[ 84411], 99.95th=[125305],
| 99.99th=[204473]
bw ( KiB/s): min=287872, max=1654656, per=100.00%, avg=671072.94, stdev=31210.13,
samples=956
iops : min= 4498, max=25854, avg=10485.52, stdev=487.66, samples=956
lat (usec) : 250=0.01%, 500=1.32%, 750=0.95%, 1000=1.11%
lat (msec) : 2=4.66%, 4=12.84%, 10=28.89%, 20=31.69%, 50=18.12%
lat (msec) : 100=0.35%, 250=0.08%
cpu : usr=2.95%, sys=10.94%, ctx=1069436, majf=0, minf=2128
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=1257420,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=655MiB/s (687MB/s), 655MiB/s-655MiB/s (687MB/s-687MB/s), io=76.7GiB (82.4GB),
run=120012-120012msec
Disk stats (read/write):
sda: ios=1255705/246, merge=254/64, ticks=15282727/6083, in_queue=15288810, util=99.99%
1.2 - Throughput Performance Test - sequential reads - WITH ADAPTIVE READ AHEAD HARDWARE
CACHE + WITH BCACHE
root@softq-pve-710:~# fio --filename=/dev/bcache0 --direct=1 --rw=read --bs=64k
--ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting
--name=throughput-test-job --eta-newline=1 --readonly
throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1027176: Thu Nov 18 14:42:06 2021
read: IOPS=20.9k, BW=1303MiB/s (1367MB/s)(153GiB/120038msec)
slat (usec): min=7, max=39687, avg=138.05, stdev=199.59
clat (usec): min=54, max=755004, avg=5996.18, stdev=11372.86
lat (usec): min=74, max=755048, avg=6134.93, stdev=11369.60
clat percentiles (usec):
| 1.00th=[ 416], 5.00th=[ 2442], 10.00th=[ 3261], 20.00th=[ 3884],
| 30.00th=[ 4228], 40.00th=[ 4555], 50.00th=[ 4817], 60.00th=[ 5145],
| 70.00th=[ 5538], 80.00th=[ 5997], 90.00th=[ 6915], 95.00th=[ 9896],
| 99.00th=[ 34866], 99.50th=[ 44827], 99.90th=[113771], 99.95th=[261096],
| 99.99th=[541066]
bw ( MiB/s): min= 48, max= 1701, per=100.00%, avg=1307.64, stdev=109.03,
samples=956
iops : min= 768, max=27222, avg=20921.69, stdev=1744.45, samples=956
lat (usec) : 100=0.01%, 250=0.02%, 500=1.66%, 750=1.04%, 1000=0.30%
lat (msec) : 2=0.98%, 4=19.10%, 10=71.97%, 20=2.18%, 50=2.37%
lat (msec) : 100=0.25%, 250=0.06%, 500=0.05%, 750=0.01%, 1000=0.01%
cpu : usr=3.24%, sys=36.24%, ctx=3406755, majf=0, minf=34793
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=2503357,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=1303MiB/s (1367MB/s), 1303MiB/s-1303MiB/s (1367MB/s-1367MB/s), io=153GiB
(164GB), run=120038-120038msec
Disk stats (read/write):
bcache0: ios=2502109/0, merge=0/0, ticks=7149936/0, in_queue=7149936, util=99.86%,
aggrios=1262209/1106438, aggrmerge=4/54, aggrticks=2461402/1258470, aggrin_queue=3719872,
aggrutil=99.02%
sda: ios=2059550/295, merge=9/108, ticks=4031424/35305, in_queue=4066729, util=99.02%
md3700: ios=464868/2212581, merge=0/0, ticks=891380/2481636, in_queue=3373016,
util=95.12%, aggrios=232423/2114558, aggrmerge=11/101552, aggrticks=398099/1256745,
aggrin_queue=1654844, aggrutil=95.05%
nvme0n1: ios=238263/2115522, merge=12/100588, ticks=410475/1286394, in_queue=1696870,
util=95.05%
nvme1n1: ios=226583/2113594, merge=10/102516, ticks=385723/1227097, in_queue=1612819,
util=94.99%
2.1 - Throughput Performance Test - random reads - WITH ADAPTIVE READ AHEAD HARDWARE
CACHE + WITHOUT BCACHE
root@softq-pve-710:~# fio --filename=/dev/sda4 --direct=1 --rw=randread --bs=64k
--ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting
--name=throughput-test-job --eta-newline=1 --readonly
throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1010415: Thu Nov 18 12:24:50 2021
read: IOPS=728, BW=45.5MiB/s (47.7MB/s)(5474MiB/120290msec)
slat (usec): min=9, max=350, avg=23.57, stdev= 3.17
clat (usec): min=164, max=2280.6k, avg=175684.34, stdev=159256.80
lat (usec): min=187, max=2280.6k, avg=175708.55, stdev=159256.79
clat percentiles (msec):
| 1.00th=[ 25], 5.00th=[ 31], 10.00th=[ 38], 20.00th=[ 54],
| 30.00th=[ 74], 40.00th=[ 99], 50.00th=[ 127], 60.00th=[ 163],
| 70.00th=[ 207], 80.00th=[ 271], 90.00th=[ 380], 95.00th=[ 489],
| 99.00th=[ 760], 99.50th=[ 894], 99.90th=[ 1183], 99.95th=[ 1351],
| 99.99th=[ 1636]
bw ( KiB/s): min=33664, max=58368, per=100.00%, avg=46646.60, stdev=1045.77,
samples=960
iops : min= 526, max= 912, avg=728.84, stdev=16.35, samples=960
lat (usec) : 250=0.01%
lat (msec) : 20=0.14%, 50=17.84%, 100=22.94%, 250=36.48%, 500=17.98%
lat (msec) : 750=3.58%, 1000=0.77%, 2000=0.27%, >=2000=0.01%
cpu : usr=0.36%, sys=1.04%, ctx=86189, majf=0, minf=2109
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=87587,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=45.5MiB/s (47.7MB/s), 45.5MiB/s-45.5MiB/s (47.7MB/s-47.7MB/s), io=5474MiB
(5740MB), run=120290-120290msec
Disk stats (read/write):
sda: ios=87540/205, merge=0/62, ticks=15349921/28134, in_queue=15378056, util=99.99%
2.2 - Throughput Performance Test - random reads - WITH ADAPTIVE READ AHEAD HARDWARE
CACHE + WITH BCACHE
root@softq-pve-710:~# fio --filename=/dev/bcache0 --direct=1 --rw=randread --bs=64k
--ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting
--name=throughput-test-job --eta-newline=1 --readonly
throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1030606: Thu Nov 18 14:55:16 2021
read: IOPS=733, BW=45.9MiB/s (48.1MB/s)(5522MiB/120391msec)
slat (usec): min=21, max=6740, avg=58.67, stdev=90.27
clat (usec): min=234, max=2413.0k, avg=174256.84, stdev=157810.22
lat (usec): min=273, max=2413.1k, avg=174316.24, stdev=157809.87
clat percentiles (msec):
| 1.00th=[ 23], 5.00th=[ 31], 10.00th=[ 37], 20.00th=[ 54],
| 30.00th=[ 73], 40.00th=[ 97], 50.00th=[ 127], 60.00th=[ 163],
| 70.00th=[ 207], 80.00th=[ 268], 90.00th=[ 372], 95.00th=[ 481],
| 99.00th=[ 760], 99.50th=[ 877], 99.90th=[ 1183], 99.95th=[ 1334],
| 99.99th=[ 1687]
bw ( KiB/s): min=33328, max=57955, per=100.00%, avg=47075.48, stdev=1075.87,
samples=960
iops : min= 520, max= 905, avg=734.54, stdev=16.81, samples=960
lat (usec) : 250=0.01%, 500=0.44%, 750=0.06%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 20=0.18%, 50=17.58%, 100=22.79%
lat (msec) : 250=36.41%, 500=18.09%, 750=3.40%, 1000=0.79%, 2000=0.23%
lat (msec) : >=2000=0.01%
cpu : usr=0.31%, sys=1.98%, ctx=87526, majf=0, minf=2105
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=88357,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=45.9MiB/s (48.1MB/s), 45.9MiB/s-45.9MiB/s (48.1MB/s-48.1MB/s), io=5522MiB
(5791MB), run=120391-120391msec
Disk stats (read/write):
bcache0: ios=88357/0, merge=0/0, ticks=15365196/0, in_queue=15365196, util=100.00%,
aggrios=44195/44624, aggrmerge=0/26, aggrticks=7679901/20858, aggrin_queue=7700759,
aggrutil=100.00%
sda: ios=87899/210, merge=0/53, ticks=15359634/28724, in_queue=15388358, util=100.00%
md3700: ios=492/89038, merge=0/0, ticks=168/12992, in_queue=13160, util=98.35%,
aggrios=246/89109, aggrmerge=0/668, aggrticks=98/9289, aggrin_queue=9388, aggrutil=98.16%
nvme0n1: ios=489/89108, merge=0/670, ticks=196/8879, in_queue=9075, util=98.16%
nvme1n1: ios=3/89111, merge=0/667, ticks=1/9700, in_queue=9702, util=98.15%
3.1 - Throughput Performance Test - random read/writes - WITH ADAPTIVE READ AHEAD
HARDWARE CACHE + WITHOUT BCACHE
root@softq-pve-710:~# fio --filename=/dev/sda4 --direct=1 --rw=randrw --bs=64k
--ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting
--name=throughput-test-job --eta-newline=1
throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1011093: Thu Nov 18 12:30:15 2021
read: IOPS=93, BW=5998KiB/s (6142kB/s)(707MiB/120737msec)
slat (usec): min=10, max=224, avg=25.81, stdev= 2.94
clat (msec): min=11, max=4247, avg=347.85, stdev=434.68
lat (msec): min=11, max=4247, avg=347.87, stdev=434.68
clat percentiles (msec):
| 1.00th=[ 26], 5.00th=[ 34], 10.00th=[ 41], 20.00th=[ 55],
| 30.00th=[ 80], 40.00th=[ 118], 50.00th=[ 169], 60.00th=[ 243],
| 70.00th=[ 363], 80.00th=[ 592], 90.00th=[ 927], 95.00th=[ 1234],
| 99.00th=[ 1989], 99.50th=[ 2400], 99.90th=[ 3205], 99.95th=[ 3406],
| 99.99th=[ 4178]
bw ( KiB/s): min= 1152, max=12928, per=100.00%, avg=6014.06, stdev=506.65,
samples=960
iops : min= 18, max= 202, avg=93.97, stdev= 7.92, samples=960
write: IOPS=96, BW=6204KiB/s (6353kB/s)(732MiB/120737msec); 0 zone resets
slat (usec): min=13, max=294, avg=31.81, stdev= 4.31
clat (msec): min=114, max=5783, avg=983.56, stdev=411.47
lat (msec): min=114, max=5783, avg=983.60, stdev=411.47
clat percentiles (msec):
| 1.00th=[ 376], 5.00th=[ 510], 10.00th=[ 584], 20.00th=[ 676],
| 30.00th=[ 760], 40.00th=[ 827], 50.00th=[ 894], 60.00th=[ 969],
| 70.00th=[ 1083], 80.00th=[ 1217], 90.00th=[ 1502], 95.00th=[ 1770],
| 99.00th=[ 2433], 99.50th=[ 2702], 99.90th=[ 3440], 99.95th=[ 3943],
| 99.99th=[ 5269]
bw ( KiB/s): min= 1792, max=11520, per=99.87%, avg=6196.98, stdev=410.22, samples=960
iops : min= 28, max= 180, avg=96.82, stdev= 6.41, samples=960
lat (msec) : 20=0.07%, 50=8.43%, 100=9.20%, 250=12.28%, 500=9.98%
lat (msec) : 750=16.81%, 1000=20.10%, 2000=21.25%, >=2000=1.89%
cpu : usr=0.11%, sys=0.34%, ctx=22938, majf=0, minf=71
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=99.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=11315,11704,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=5998KiB/s (6142kB/s), 5998KiB/s-5998KiB/s (6142kB/s-6142kB/s), io=707MiB
(742MB), run=120737-120737msec
WRITE: bw=6204KiB/s (6353kB/s), 6204KiB/s-6204KiB/s (6353kB/s-6353kB/s), io=732MiB
(767MB), run=120737-120737msec
Disk stats (read/write):
sda: ios=11370/11806, merge=0/45, ticks=3916382/11605103, in_queue=15521484,
util=99.99%
3.2 - Throughput Performance Test - random read/writes - WITH ADAPTIVE READ AHEAD
HARDWARE CACHE + WITH BCACHE
root@softq-pve-710:~# fio --filename=/dev/bcache0 --direct=1 --rw=randrw --bs=64k
--ioengine=libaio --iodepth=32 --runtime=120 --numjobs=4 --time_based --group_reporting
--name=throughput-test-job --eta-newline=1
throughput-test-job: (groupid=0, jobs=4): err= 0: pid=1031738: Thu Nov 18 15:04:05 2021
read: IOPS=1086, BW=67.9MiB/s (71.2MB/s)(8161MiB/120229msec)
slat (usec): min=15, max=86127, avg=51.75, stdev=568.52
clat (usec): min=188, max=2007.7k, avg=116178.43, stdev=151824.06
lat (usec): min=242, max=2007.7k, avg=116230.79, stdev=151836.54
clat percentiles (usec):
| 1.00th=[ 441], 5.00th=[ 652], 10.00th=[ 832],
| 20.00th=[ 1385], 30.00th=[ 2999], 40.00th=[ 36439],
| 50.00th=[ 61604], 60.00th=[ 96994], 70.00th=[ 141558],
| 80.00th=[ 204473], 90.00th=[ 308282], 95.00th=[ 413139],
| 99.00th=[ 683672], 99.50th=[ 801113], 99.90th=[1082131],
| 99.95th=[1216349], 99.99th=[1535116]
bw ( KiB/s): min=30834, max=1256048, per=100.00%, avg=69675.99, stdev=38920.70,
samples=957
iops : min= 481, max=19625, avg=1087.75, stdev=608.16, samples=957
write: IOPS=1089, BW=68.1MiB/s (71.4MB/s)(8190MiB/120229msec); 0 zone resets
slat (usec): min=14, max=50364, avg=47.67, stdev=339.27
clat (usec): min=11, max=165668, avg=1454.15, stdev=2560.77
lat (usec): min=74, max=165915, avg=1502.47, stdev=2590.82
clat percentiles (usec):
| 1.00th=[ 71], 5.00th=[ 73], 10.00th=[ 80], 20.00th=[ 119],
| 30.00th=[ 135], 40.00th=[ 139], 50.00th=[ 145], 60.00th=[ 169],
| 70.00th=[ 1221], 80.00th=[ 3359], 90.00th=[ 5080], 95.00th=[ 6194],
| 99.00th=[ 8586], 99.50th=[ 9503], 99.90th=[14484], 99.95th=[21627],
| 99.99th=[26608]
bw ( KiB/s): min=23655, max=1247613, per=100.00%, avg=69995.69, stdev=39278.35,
samples=957
iops : min= 368, max=19493, avg=1092.74, stdev=613.74, samples=957
lat (usec) : 20=0.01%, 50=0.01%, 100=7.71%, 250=25.16%, 500=1.76%
lat (usec) : 750=3.40%, 1000=3.65%
lat (msec) : 2=7.17%, 4=9.16%, 10=8.75%, 20=0.24%, 50=5.96%
lat (msec) : 100=7.50%, 250=12.11%, 500=5.94%, 750=1.17%, 1000=0.24%
lat (msec) : 2000=0.08%, >=2000=0.01%
cpu : usr=0.76%, sys=3.07%, ctx=225226, majf=1, minf=10344
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=130582,131041,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=67.9MiB/s (71.2MB/s), 67.9MiB/s-67.9MiB/s (71.2MB/s-71.2MB/s), io=8161MiB
(8558MB), run=120229-120229msec
WRITE: bw=68.1MiB/s (71.4MB/s), 68.1MiB/s-68.1MiB/s (71.4MB/s-71.4MB/s), io=8190MiB
(8588MB), run=120229-120229msec
Disk stats (read/write):
bcache0: ios=130619/131041, merge=0/0, ticks=15128128/174484, in_queue=15302612,
util=100.00%, aggrios=65454/110552, aggrmerge=0/34, aggrticks=7565070/131037,
aggrin_queue=7696107, aggrutil=99.57%
sda: ios=86501/300, merge=0/68, ticks=15072824/78563, in_queue=15151387, util=99.57%
md3700: ios=44408/220805, merge=0/0, ticks=57316/183512, in_queue=240828, util=98.53%,
aggrios=22202/221689, aggrmerge=0/548, aggrticks=28755/138402, aggrin_queue=167156,
aggrutil=98.41%
nvme0n1: ios=22193/221688, merge=0/549, ticks=28763/138527, in_queue=167289,
util=98.39%
nvme1n1: ios=22212/221690, merge=0/547, ticks=28747/138277, in_queue=167024,
util=98.41%
--
Frédéric Dumas
f.dumas(a)ellis.siteparc.fr