Programming Language and compiler
Benchmarks

Python VS Chapel benchmarks

Current benchmark data was generated on Mon Feb 06 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz (Model 85)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2447ms 34ms 217.6MB 2300ms 130ms pypy 3.9.16
chapel 4.chpl 2503ms 10ms 60.2MB 2460ms 20ms chpl 1.29.0
chapel 3.chpl 2798ms 9.2ms 64.2MB 2760ms 27ms chpl 1.29.0
python 1.py timeout 0.0ms 40.8MB 4957ms 23ms cpython 3.11.1
python 1.py timeout 0.0ms 40.4MB 4933ms 47ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 162ms 21ms 74.4MB 110ms 23ms pypy 3.9.16
chapel 4.chpl 235ms 3.4ms 28.0MB 210ms 7ms chpl 1.29.0
chapel 3.chpl 244ms 1.0ms 24.0MB 227ms 0ms chpl 1.29.0
python 1.py 674ms 6.1ms 12.0MB 653ms 7ms pyston 3.8.12
python 1.py 989ms 5.6ms 12.5MB 970ms 7ms cpython 3.11.1

coro-prime-sieve

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1606ms 14ms 219.5MB 1493ms 93ms pypy 3.9.16
chapel 1-m.chpl 4744ms 72ms 61.5MB 9257ms 43ms chpl 1.29.0
python 1.py timeout 0.0ms 20.1MB 4977ms 7ms cpython 3.11.1
python 1.py timeout 0.0ms 17.3MB 4970ms 3ms pyston 3.8.12

Input: 1000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 316ms 15ms 87.9MB 260ms 37ms pypy 3.9.16
chapel 1-m.chpl 319ms 15ms 34.1MB 570ms 10ms chpl 1.29.0
python 1.py 816ms 26ms 20.0MB 783ms 0ms cpython 3.11.1
python 1.py 868ms 5.1ms 16.9MB 843ms 3ms pyston 3.8.12

edigits

Input: 250001

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 88ms 3.7ms 7.4MB 73ms 0ms chpl 1.29.0
python 1.py 613ms 2.3ms 78.4MB 587ms 7ms pypy 3.9.16
python 1.py 1403ms 0.4ms 10.0MB 1387ms 0ms cpython 3.11.1
python 1.py 2299ms 1.8ms 9.8MB 2283ms 0ms pyston 3.8.12

Input: 100000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 42ms 3.1ms 22.8MB 27ms 3ms chpl 1.29.0
python 1.py 173ms 0.5ms 73.7MB 143ms 17ms pypy 3.9.16
python 1.py 263ms 3.7ms 9.3MB 250ms 0ms cpython 3.11.1
python 1.py 404ms 1.1ms 8.9MB 393ms 0ms pyston 3.8.12

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 5-m.chpl 135ms 1.6ms 3.8MB 227ms 3ms chpl 1.29.0
python 5-m.py 3396ms 47ms 12.7MB 4827ms 1273ms pyston 3.8.12
python 1.py 3472ms 22ms 85.0MB 3427ms 23ms pypy 3.9.16
python 1.py 3936ms 36ms 7.9MB 3923ms 0ms pyston 3.8.12
python 5-m.py 4001ms 4.4ms 13.6MB 5693ms 1197ms cpython 3.11.1
python 5-m.py 4662ms 42ms 76.1MB 6713ms 1320ms pypy 3.9.16
python 1.py timeout 0.0ms 8.6MB 4980ms 0ms cpython 3.11.1

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 5.chpl 27ms 3.1ms 14.6MB 27ms 0ms chpl 1.29.0
python 1.py 407ms 1.7ms 8.0MB 390ms 3ms pyston 3.8.12
python 1.py 433ms 0.6ms 73.8MB 407ms 13ms pypy 3.9.16
python 5-m.py 436ms 14ms 12.6MB 557ms 160ms pyston 3.8.12
python 5-m.py 485ms 5.8ms 13.6MB 587ms 193ms cpython 3.11.1
python 1.py 594ms 1.0ms 8.6MB 580ms 0ms cpython 3.11.1
python 5-m.py 845ms 3.6ms 76.2MB 923ms 457ms pypy 3.9.16

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 11ms 0.7ms 2.6MB 10ms 0ms chpl 1.29.0
python 1.py 14ms 0.9ms 8.4MB 10ms 0ms cpython 3.11.1
python 1.py 14ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 37ms 0.6ms 51.6MB 8ms 14ms pypy 3.9.16

http-server

Input: 3000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1784ms 8.6ms 33.7MB 1343ms 930ms pyston 3.8.12
python 1.py 2123ms 9.1ms 29.7MB 1690ms 937ms cpython 3.11.1
python 1.py 4893ms 77ms 125.1MB 3767ms 1003ms pypy 3.9.16

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 403ms 1.0ms 27.7MB 300ms 167ms pyston 3.8.12
python 1.py 436ms 5.0ms 24.9MB 333ms 163ms cpython 3.11.1
python 1.py 1158ms 29ms 95.4MB 917ms 207ms pypy 3.9.16

json-serde

Input: sample 5000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 274ms 1.8ms 38.6MB 223ms 33ms pyston 3.8.12
python 1.py 281ms 0.9ms 41.3MB 253ms 13ms cpython 3.11.1
python 1.py 418ms 5.8ms 98.6MB 370ms 33ms pypy 3.9.16

Input: canada 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2498ms 12ms 220.7MB 2313ms 167ms pyston 3.8.12
python 1.py 2501ms 10ms 223.0MB 2357ms 127ms cpython 3.11.1
python 1.py 2634ms 7.8ms 239.5MB 2510ms 107ms pypy 3.9.16

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3-m.chpl 1105ms 20ms 80.4MB 2030ms 40ms chpl 1.29.0
python 3.py timeout 0.0ms 35.3MB 93ms 20ms cpython 3.11.1
python 3.py timeout 0.0ms 207.6MB 297ms 87ms pypy 3.9.16
python 3.py timeout 0.0ms 34.1MB 77ms 23ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3-m.chpl 187ms 6.2ms 68.8MB 303ms 13ms chpl 1.29.0
python 3-m.py 670ms 8.9ms 86.2MB 1007ms 117ms pypy 3.9.16
python 3-m.py 718ms 12ms 15.9MB 1250ms 47ms pyston 3.8.12
python 3-m.py 942ms 1.3ms 17.0MB 1707ms 30ms cpython 3.11.1

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 232ms 1.2ms 70.9MB 200ms 10ms pypy 3.9.16
python 1.py 284ms 6.6ms 71.9MB 240ms 23ms pypy 3.9.16
python 1.py 934ms 3.2ms 9.1MB 923ms 0ms pyston 3.8.12
python 2.py 1077ms 7.0ms 8.7MB 1063ms 0ms pyston 3.8.12
python 1.py 1255ms 16ms 9.3MB 1243ms 0ms cpython 3.11.1
python 2.py 1427ms 8.5ms 9.1MB 1410ms 0ms cpython 3.11.1

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 557ms 5.6ms 71.3MB 520ms 23ms pypy 3.9.16
python 1.py 616ms 1.8ms 72.5MB 580ms 20ms pypy 3.9.16
python 1.py 2744ms 8.5ms 9.1MB 2723ms 0ms pyston 3.8.12
python 2.py 3201ms 18ms 8.7MB 3183ms 3ms pyston 3.8.12
python 1.py 3733ms 77ms 9.3MB 3717ms 0ms cpython 3.11.1
python 2.py 4232ms 26ms 9.1MB 4217ms 0ms cpython 3.11.1

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 161ms 0.9ms 71.3MB 130ms 10ms pypy 3.9.16
python 1.py 187ms 2.1ms 72.3MB 157ms 17ms pypy 3.9.16
python 1.py 454ms 2.7ms 8.8MB 440ms 0ms pyston 3.8.12
python 2.py 501ms 1.5ms 8.2MB 487ms 0ms pyston 3.8.12
python 1.py 597ms 4.0ms 8.7MB 577ms 3ms cpython 3.11.1
python 2.py 709ms 5.6ms 8.4MB 693ms 0ms cpython 3.11.1

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2938ms 261ms 230.3MB 2777ms 143ms pypy 3.9.16
python 1.py timeout 0.0ms 92.9MB 4947ms 33ms cpython 3.11.1
python 1.py timeout 0.0ms 130.2MB 4930ms 50ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 482ms 20ms 149.6MB 410ms 53ms pypy 3.9.16
python 1.py timeout 0.0ms 36.9MB 4967ms 13ms cpython 3.11.1
python 1.py timeout 0.0ms 41.5MB 4963ms 17ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 400ms 2.9ms 3.6MB 387ms 0ms chpl 1.29.0
python 2.py 2170ms 15ms 71.1MB 2137ms 17ms pypy 3.9.16
python 1.py 3646ms 10ms 70.2MB 3617ms 7ms pypy 3.9.16
python 1.py timeout 0.0ms 8.5MB 4987ms 0ms cpython 3.11.1
python 2.py timeout 0.0ms 8.6MB 4980ms 3ms cpython 3.11.1
python 1.py timeout 0.0ms 8.5MB 4990ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 8.0MB 4983ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 59ms 3.3ms 30.2MB 43ms 0ms chpl 1.29.0
python 2.py 267ms 1.3ms 70.9MB 233ms 17ms pypy 3.9.16
python 1.py 416ms 2.4ms 70.4MB 380ms 17ms pypy 3.9.16
python 1.py 1163ms 10ms 8.4MB 1150ms 0ms pyston 3.8.12
python 2.py 1601ms 24ms 7.9MB 1587ms 0ms pyston 3.8.12
python 1.py 3147ms 98ms 8.5MB 3133ms 0ms cpython 3.11.1
python 2.py 3709ms 215ms 8.6MB 3697ms 0ms cpython 3.11.1

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 4962ms 19ms 382.8MB 4797ms 147ms pypy 3.9.16
python 1.py timeout 0.0ms 321.1MB 4897ms 87ms cpython 3.11.1
python 2.py timeout 0.0ms 633.6MB 4557ms 427ms cpython 3.11.1
python 2.py timeout 0.0ms 1000.1MB 4617ms 363ms pypy 3.9.16
python 1.py timeout 0.0ms 320.5MB 4903ms 80ms pyston 3.8.12
python 2.py timeout 0.0ms 633.0MB 4553ms 427ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1141ms 2.1ms 207.1MB 1077ms 50ms pypy 3.9.16
python 2.py 1452ms 29ms 164.4MB 1347ms 90ms pyston 3.8.12
python 2.py 1926ms 10ms 210.5MB 1793ms 117ms pypy 3.9.16
python 2.py 2006ms 13ms 164.8MB 1907ms 80ms cpython 3.11.1
python 1.py 3114ms 48ms 86.1MB 3063ms 33ms pyston 3.8.12
python 1.py 4358ms 46ms 86.8MB 4307ms 33ms cpython 3.11.1

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 484ms 2.7ms 15.6MB 467ms 0ms chpl 1.29.0
python 4.py 2411ms 0.4ms 9.1MB 2377ms 20ms pyston 3.8.12
python 4.py 2413ms 1.4ms 9.4MB 2383ms 17ms cpython 3.11.1
python 4.py 2667ms 37ms 125.2MB 2610ms 40ms pypy 3.9.16

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 126ms 0.8ms 11.0MB 107ms 7ms chpl 1.29.0
python 4.py 578ms 2.1ms 8.8MB 560ms 0ms pyston 3.8.12
python 4.py 581ms 0.7ms 9.2MB 563ms 0ms cpython 3.11.1
python 4.py 682ms 5.4ms 76.9MB 653ms 13ms pypy 3.9.16

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3.chpl 1908ms 24ms 221.6MB 1723ms 170ms chpl 1.29.0
python 1.py 3319ms 11ms 273.9MB 3187ms 110ms pypy 3.9.16
python 1.py timeout 0.0ms 108.4MB 4927ms 53ms cpython 3.11.1
python 1.py timeout 0.0ms 108.6MB 4927ms 53ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3.chpl 207ms 2.8ms 46.9MB 163ms 23ms chpl 1.29.0
python 1.py 465ms 1.4ms 94.8MB 420ms 30ms pypy 3.9.16
python 1.py 776ms 1.9ms 30.4MB 733ms 23ms pyston 3.8.12
python 1.py 832ms 1.4ms 31.8MB 787ms 30ms cpython 3.11.1

secp256k1

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 1487ms 0.9ms 32.3MB 1467ms 7ms chpl 1.29.0
python 1.py 4421ms 15ms 8.4MB 4410ms 0ms pyston 3.8.12
python 1.py 4722ms 2.6ms 8.4MB 4707ms 0ms cpython 3.11.1
python 1.py timeout 0.0ms 77.8MB 4967ms 17ms pypy 3.9.16

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 389ms 3.8ms 32.3MB 370ms 3ms chpl 1.29.0
python 1.py 1117ms 1.0ms 8.4MB 1107ms 0ms pyston 3.8.12
python 1.py 1198ms 8.4ms 8.5MB 1180ms 0ms cpython 3.11.1
python 1.py 1495ms 7.0ms 74.8MB 1467ms 17ms pypy 3.9.16

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 1998ms 78ms 32.3MB 3857ms 7ms chpl 1.29.0
chapel 1.chpl 3774ms 2.9ms 32.3MB 3747ms 7ms chpl 1.29.0
python 8.py timeout 0.0ms 16.5MB 53ms 13ms cpython 3.11.1
python 8.py timeout 0.0ms 100.3MB 660ms 83ms pypy 3.9.16
python 8.py timeout 0.0ms 17.8MB 77ms 13ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 509ms 1.1ms 32.3MB 963ms 3ms chpl 1.29.0
chapel 1.chpl 961ms 1.5ms 32.3MB 943ms 7ms chpl 1.29.0
python 8.py timeout 0.0ms 16.3MB 100ms 17ms cpython 3.11.1
python 8.py timeout 0.0ms 111.3MB 1097ms 220ms pypy 3.9.16
python 8.py timeout 0.0ms 17.2MB 160ms 30ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 146ms 1.8ms 32.3MB 247ms 3ms chpl 1.29.0
chapel 1.chpl 258ms 1.7ms 32.3MB 240ms 7ms chpl 1.29.0
python 8-m.py 3809ms 23ms 104.7MB 6577ms 407ms pypy 3.9.16
python 8.py timeout 0.0ms 15.8MB 277ms 47ms cpython 3.11.1
python 8.py timeout 0.0ms 16.8MB 297ms 63ms pyston 3.8.12