forked from iovisor/bcc
-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathblkalgn_example.txt
313 lines (290 loc) · 20 KB
/
blkalgn_example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
Demonstrations of blkalgn, the Linux eBPF/bcc version.
1. fio test with different block sizes.
The blkalgn tool records all write commands issued to the nvme0n2 device and
produces two histograms representing the block sizes of the transmitted data,
and the alignment of the commands.
- fio test commands:
fio -bs=4k -iodepth=1 -rw=write -ioengine=sync -size=4k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 40
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 60
fio -bs=16k -iodepth=1 -rw=write -ioengine=sync -size=16k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 80
fio -bs=32k -iodepth=1 -rw=write -ioengine=sync -size=32k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 100
fio -bs=64k -iodepth=1 -rw=write -ioengine=sync -size=64k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 100
fio -bs=128k -iodepth=1 -rw=write -ioengine=sync -size=128k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 80
fio -bs=256k -iodepth=1 -rw=write -ioengine=sync -size=256k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 60
fio -bs=512k -iodepth=1 -rw=write -ioengine=sync -size=512k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 40
- blkalgn capture command:
tools/blkalgn.py --disk nvme0n2 --ops Write
^C
Block size : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 0 | |
1024 -> 2047 : 0 | |
2048 -> 4095 : 0 | |
4096 -> 8191 : 40 |**************** |
8192 -> 16383 : 60 |************************ |
16384 -> 32767 : 80 |******************************** |
32768 -> 65535 : 100 |****************************************|
65536 -> 131071 : 100 |****************************************|
131072 -> 262143 : 80 |******************************** |
262144 -> 524287 : 60 |************************ |
524288 -> 1048575 : 40 |**************** |
Algn size : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 0 | |
1024 -> 2047 : 0 | |
2048 -> 4095 : 0 | |
4096 -> 8191 : 40 |**************** |
8192 -> 16383 : 60 |************************ |
16384 -> 32767 : 80 |******************************** |
32768 -> 65535 : 100 |****************************************|
65536 -> 131071 : 100 |****************************************|
131072 -> 262143 : 80 |******************************** |
262144 -> 524287 : 60 |************************ |
524288 -> 1048575 : 40 |**************** |
2. fio test with different block sizes with an offset:
The blkalgn tool records all write commands issued to the nvme0n2 device and
produces two histograms representing the block sizes of the transmitted data,
and the alignment of the commands.
- fio test commands:
fio -bs=4k -iodepth=1 -rw=write -ioengine=sync -size=4k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 40 -offset=4096
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 60 -offset=4096
fio -bs=16k -iodepth=1 -rw=write -ioengine=sync -size=16k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 80 -offset=4096
fio -bs=32k -iodepth=1 -rw=write -ioengine=sync -size=32k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 100 -offset=4096
fio -bs=64k -iodepth=1 -rw=write -ioengine=sync -size=64k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 100 -offset=4096
fio -bs=128k -iodepth=1 -rw=write -ioengine=sync -size=128k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 80 -offset=4096
fio -bs=256k -iodepth=1 -rw=write -ioengine=sync -size=256k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 60 -offset=4096
fio -bs=512k -iodepth=1 -rw=write -ioengine=sync -size=512k \
-name=sync -direct=1 -filename=/dev/nvme0n2 -loop 40 -offset=4096
- blkalgn capture command:
tools/blkalgn.py --disk nvme0n2 --ops Write
^C
Block size : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 0 | |
1024 -> 2047 : 0 | |
2048 -> 4095 : 0 | |
4096 -> 8191 : 40 |**************** |
8192 -> 16383 : 60 |************************ |
16384 -> 32767 : 80 |******************************** |
32768 -> 65535 : 100 |****************************************|
65536 -> 131071 : 100 |****************************************|
131072 -> 262143 : 80 |******************************** |
262144 -> 524287 : 60 |************************ |
524288 -> 1048575 : 40 |**************** |
Algn size : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 0 | |
1024 -> 2047 : 0 | |
2048 -> 4095 : 0 | |
4096 -> 8191 : 560 |****************************************|
3. PostgreSQL workload using sybench:
The blkalgn tool records and traces all NVMe write commands issued to the
nvme0n2 device and produces two histograms representing the block sizes of the
transmitted data, and the alignment of the commands.
- blkalgn capture command:
tools/blkalgn.py --disk nvme0n2 --ops Write --trace
Tracing NVMe commands... Hit Ctrl-C to end.
DISK OPS LEN LBA PID COMM ALGN
...
nvme0n1 Write 16384 5592372 1402 postgres 16384
nvme0n1 Write 4096 8131612 1406 postgres 4096
nvme0n1 Write 4096 8476298 1407 postgres 4096
nvme0n1 Write 4096 5562083 1404 postgres 4096
nvme0n1 Write 4096 5562081 1404 postgres 4096
nvme0n1 Write 8192 5740024 977 postgres 8192
nvme0n1 Write 8192 8476258 1407 postgres 8192
nvme0n1 Write 16384 2809028 1400 postgres 16384
nvme0n1 Write 8192 9393492 1402 postgres 8192
nvme0n1 Write 16384 1777248 1401 postgres 16384
nvme0n1 Write 8192 11376510 1404 postgres 8192
nvme0n1 Write 4096 11376508 1404 postgres 4096
nvme0n1 Write 8192 1022036 1405 postgres 8192
nvme0n1 Write 4096 5755970 977 postgres 4096
nvme0n1 Write 16384 1777252 1401 postgres 16384
nvme0n1 Write 4096 9393589 1402 postgres 4096
nvme0n1 Write 4096 2809039 1400 postgres 4096
nvme0n2 Write 8192 9637351 1403 postgres 4096
nvme0n1 Write 4096 5822235 1406 postgres 4096
...
^C
Block size : count distribution
0 -> 1 : 560 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 0 | |
1024 -> 2047 : 0 | |
2048 -> 4095 : 0 | |
4096 -> 8191 : 325687 |****************************************|
8192 -> 16383 : 92798 |*********** |
16384 -> 32767 : 14121 |* |
32768 -> 65535 : 745 | |
65536 -> 131071 : 496 | |
131072 -> 262143 : 640 | |
262144 -> 524287 : 6365 | |
524288 -> 1048575 : 497 | |
Algn size : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 0 | |
1024 -> 2047 : 0 | |
2048 -> 4095 : 0 | |
4096 -> 8191 : 392148 |****************************************|
8192 -> 16383 : 36459 |*** |
16384 -> 32767 : 12781 |* |
32768 -> 65535 : 491 | |
65536 -> 131071 : 49 | |
131072 -> 262143 : 33 | |
262144 -> 524287 : 10 | |
4. Testing Workload Write-Amplification-Factor
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=4k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=8k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=16k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=0
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=4k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=4k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=8k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=0
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=0
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=0
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=4k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=16k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=4k
fio -bs=8k -iodepth=1 -rw=write -ioengine=sync -size=8k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=20k -iodepth=1 -rw=write -ioengine=sync -size=20k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=12k -iodepth=1 -rw=write -ioengine=sync -size=12k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=4k
fio -bs=12k -iodepth=1 -rw=write -ioengine=sync -size=12k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=8k
fio -bs=512k -iodepth=1 -rw=write -ioengine=sync -size=512k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=8k
fio -bs=40k -iodepth=1 -rw=write -ioengine=sync -size=40k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=8k
fio -bs=32k -iodepth=1 -rw=write -ioengine=sync -size=32k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=24k
fio -bs=256k -iodepth=1 -rw=write -ioengine=sync -size=256k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=12k
fio -bs=24k -iodepth=1 -rw=write -ioengine=sync -size=24k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=32k
fio -bs=32k -iodepth=1 -rw=write -ioengine=sync -size=32k -name=sync -direct=1 -filename=/dev/nvme0n1 --offset=8
Results:
Workload WAF:
DISK IU WWAF (LBS: 4k)
nvme0n1 4096 1.00
nvme0n1 8192 1.16
nvme0n1 16384 1.52
nvme0n1 32768 1.77
nvme0n1 65536 2.73
nvme0n1 131072 4.81
nvme0n1 262144 8.96
nvme0n1 524288 17.04
nvme0n1 1048576 33.20
nvme0n1 2097152 66.40
Debug output:
{'nvme0n1': {4096: {4096: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 1200128,
'wwaf': '1.00'},
8192: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 1392640,
'wwaf': '1.16'},
16384: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 1818624,
'wwaf': '1.52'},
32768: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 2129920,
'wwaf': '1.77'},
65536: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 3276800,
'wwaf': '2.73'},
131072: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 5767168,
'wwaf': '4.81'},
262144: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 10747904,
'wwaf': '8.96'},
524288: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 20447232,
'wwaf': '17.04'},
1048576: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 39845888,
'wwaf': '33.20'},
2097152: {'iocnt': 38,
'iohost': 1200128,
'ioiu': 79691776,
'wwaf': '66.40'}}}}