[PATCH v2 00/10] makedumpfile: parallel processing
Atsushi Kumagai
ats-kumagai at wm.jp.nec.com
Fri Jun 26 00:07:13 PDT 2015
Hello Zhou,
>Hello Atsushi Kumagai,
>
>I test this patch set in several machines and the following is the benchmark.
Thanks for your report, it looks good as before.
I also did simple test on kernel 3.19 on a 5GB Virtual Machine,
but I can't get such good result as below:
/ # time makedumpfile -c --num-threads 0 /proc/vmcore /mnt/dumpfile
Copying data : [100.0 %] /
The dumpfile is saved to /mnt/dumpfile.
makedumpfile Completed.
real 0m 44.40s
user 0m 43.62s
sys 0m 0.69s
/ #
/ # time makedumpfile -c --num-threads 4 /proc/vmcore /mnt/dumpfile
Copying data : [100.0 %] -
Copying data : [100.0 %] /
The dumpfile is saved to /mnt/dumpfile.
makedumpfile Completed.
real 5m 29.54s
user 6m 8.18s
sys 16m 33.25s
/ #
There is a big performance degradation.
Do you have any ideas why this happens ?
Thanks
Atsushi Kumagai
>These tables show the time that makedumpfile spends. And the unit is second.
>
>"core-data" in the table means the context in the vmcore.
> For example:
> core-data's value is 256. It means that in the vmcore, 256 * 8 bits of each page
> are set to 1.
>
>threads-num in the table means how many threads are used. 0 means original single thread
>implementation.
>
>"-l" in the table means producing lzo format vmcore
>
>"-c" in the table means producing kdump-compressed format vmcore
>
>###################################
>- System: PRIMERGY RX300 S6
>- CPU: Intel(R) Xeon(R) CPU x5660
>- memory: 16GB
>###################################
>************ makedumpfile -d 0 ******************
> core-data 0 256 512 768 1024 1280 1536 1792 2048 2304
>2560 2816 3072 3328 3584 3840
> threads-num
>-l
> 0 11 112 163 168 167 167 167 167 170 169 167
>166 167 168 169 113
> 4 5 111 158 166 167 167 167 166 169 170 166
>166 167 167 158 111
> 8 5 111 158 167 169 170 167 169 169 170 167
>168 168 168 159 111
> 12 6 111 158 168 167 167 169 168 170 169 167
>168 167 167 161 114
>-c
> 0 54 544 643 666 589 517 468 405 408 429 491
>528 592 676 654 527
> 4 60 179 210 216 206 220 209 214 211 222 220
>208 209 210 245 177
> 8 43 113 146 169 179 215 195 211 199 195 216
>197 185 165 158 109
> 12 44 106 142 162 169 171 168 173 174 171 200
>173 177 165 139 107
>
>************ makedumpfile -d 1 ******************
> core-data 0 256 512 768 1024 1280 1536 1792 2048 2304
>2560 2816 3072 3328 3584 3840
> threads-num
>-l
> 0 10 137 163 167 167 167 167 167 171 170 166
>166 166 168 169 138
> 4 5 111 158 166 167 166 167 166 170 171 166
>167 166 166 159 111
> 8 5 114 159 167 167 169 167 167 169 169 169
>167 168 167 160 111
> 12 6 113 159 168 167 168 167 168 169 170 168
>168 167 168 159 112
>-c
> 0 53 522 645 663 591 512 470 401 412 413 479
>532 587 666 648 524
> 4 57 205 244 216 208 217 216 225 221 225 223
>217 213 209 246 174
> 8 41 118 154 176 191 213 219 223 212 210 222
>207 198 176 164 118
> 12 43 111 148 174 189 194 191 182 178 173 204
>196 194 170 150 112
>
>************ makedumpfile -d 7 ******************
> core-data 0 256 512 768 1024 1280 1536 1792 2048 2304
>2560 2816 3072 3328 3584 3840
> threads-num
>-l
> 0 10 144 199 168 167 167 167 167 171 169 167
>166 166 168 169 144
> 4 5 110 158 167 167 167 166 166 170 169 166
>166 166 166 159 110
> 8 5 111 159 167 166 170 167 166 169 169 168
>167 167 167 159 110
> 12 6 111 159 167 167 166 168 167 169 169 168
>167 167 167 160 111
>-c
> 0 52 523 642 659 592 511 490 402 410 422 464
>525 591 666 647 529
> 4 58 177 210 213 208 224 217 228 223 229 227
>216 210 212 215 175
> 8 41 116 152 178 191 213 219 225 216 211 221
>211 196 180 158 116
> 12 41 111 151 176 192 187 192 190 174 175 218
>186 189 172 151 112
>
>************ makedumpfile -d 31 ******************
> core-data 0 256 512 768 1024 1280 1536 1792 2048 2304
>2560 2816 3072 3328 3584 3840
> threads-num
>-l
> 0 0 0 0 0 0 0 0 0 0 0 0
>0 0 0 0 0
> 4 2 2 2 3 3 3 2 3 3 3 3
>2 3 3 3 3
> 8 2 3 3 2 3 3 3 3 3 3 3
>3 3 3 3 3
> 12 2 3 3 3 3 3 3 3 3 3 3
>3 3 3 3 3
>-c
> 0 3 3 3 3 3 3 3 3 3 3 3
>3 3 3 3 3
> 4 4 5 5 4 5 5 5 5 4 5 5
>5 4 4 5 5
> 8 4 4 4 3 4 4 4 4 4 4 4
>4 4 4 4 4
> 12 4 4 4 4 4 4 4 4 4 4 4
>4 4 4 4 4
>
>###################################
>- System: PRIMERGY RX300 S7
>- CPU: Intel(R) Xeon(R) CPU E5-2620
>- memory: 32GB
>###################################
>************ makedumpfile -d 0 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 23 211 246 255 252 252 253 257
> 4 14 168 234 250 253 251 251 252
> 12 17 165 237 250 251 251 253 253
>-c
> 0 117 1210 1620 1699 1443 1294 1235 1051
> 4 161 438 525 543 506 465 484 415
> 12 131 211 245 250 246 248 260 265
>
>************ makedumpfile -d 7 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 24 215 253 256 255 254 257 255
> 4 14 165 235 247 249 250 252 253
> 12 16 169 237 251 255 253 252 250
>-c
> 0 120 1202 1625 1698 1434 1274 1223 1040
> 4 157 438 533 542 507 461 479 400
> 12 132 204 242 248 242 244 257 263
>
>###################################
>- System: PRIMEQUEST 1800E
>- CPU: Intel(R) Xeon(R) CPU E7540
>- memory: 32GB
>###################################
>************ makedumpfile -d 0 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 34 282 245 179 179 179 179 180
> 4 63 143 224 230 220 212 207 204
> 8 65 129 200 225 235 235 225 220
> 12 67 149 186 211 222 229 237 236
>-c
> 0 158 1505 2119 2129 1707 1483 1440 1273
> 4 207 589 672 673 636 564 536 514
> 8 176 327 377 387 367 336 314 291
> 12 191 272 295 306 288 259 257 240
>
>************ makedumpfile -d 7 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 34 270 248 187 188 187 187 187
> 4 63 154 186 188 189 189 190 190
> 8 64 131 220 218 197 186 187 188
> 12 65 159 198 232 229 207 190 188
>-c
> 0 154 1508 2089 2133 1792 1660 1462 1312
> 4 203 594 684 701 627 592 535 503
> 8 172 326 377 393 366 334 313 286
> 12 182 273 295 308 283 258 249 237
>
>************ makedumpfile -d 31 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 2 1 1 2 1 2 2 2
> 4 48 48 48 48 49 48 48 49
> 8 48 49 48 49 48 47 49 48
> 12 49 50 49 49 49 48 50 49
>-c
> 0 10 9 10 10 9 10 9 9
> 4 52 53 52 52 53 52 53 52
> 8 51 51 52 52 52 51 51 52
> 12 53 52 52 53 52 51 52 52
>
>
>
>
>On 06/19/2015 04:56 PM, Zhou Wenjian wrote:
>> This patch set implements parallel processing by means of multiple threads.
>> With this patch set, it is available to use multiple threads to read
>> and compress pages. This parallel process will save time.
>> This feature only supports creating dumpfile in kdump-compressed format from
>> vmcore in kdump-compressed format or elf format. Currently, sadump and
>> xen kdump are not supported.
>>
>> Qiao Nuohan (10):
>> Add readpage_kdump_compressed_parallel
>> Add mappage_elf_parallel
>> Add readpage_elf_parallel
>> Add read_pfn_parallel
>> Add function to initial bitmap for parallel use
>> Add filter_data_buffer_parallel
>> Add write_kdump_pages_parallel to allow parallel process
>> Initial and free data used for parallel process
>> Make makedumpfile available to read and compress pages parallelly
>> Add usage and manual about multiple threads process
>>
>> Makefile | 2 +
>> erase_info.c | 29 ++-
>> erase_info.h | 2 +
>> makedumpfile.8 | 24 ++
>> makedumpfile.c | 1096 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>> makedumpfile.h | 80 ++++
>> print_info.c | 16 +
>> 7 files changed, 1246 insertions(+), 3 deletions(-)
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec
>
>
>--
>Thanks
>Zhou Wenjian
More information about the kexec
mailing list