Programming Language and compiler
Benchmarks

Chapel VS Python benchmarks

Current benchmark data was generated on Thu Sep 22 2022, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1574ms 2.7ms 274.1MB 1453ms 100ms pypy 3.8.13
chapel 4.chpl 2299ms 1.2ms 58.2MB 2277ms 13ms chpl 1.27.0
chapel 3.chpl 2433ms 5.5ms 50.3MB 2413ms 13ms chpl 1.27.0
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 120ms 2.4ms 83.3MB 93ms 10ms pypy 3.8.13
chapel 3.chpl 222ms 0.5ms 13.1MB 210ms 3ms chpl 1.27.0
chapel 4.chpl 222ms 7.3ms 26.1MB 210ms 0ms chpl 1.27.0
python 1.py 610ms 3.8ms 11.9MB 583ms 10ms pyston 3.8.12
python 1.py 1514ms 8.6ms 11.8MB 1493ms 3ms cpython 3.10.7

coro-prime-sieve

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1164ms 15ms 196.7MB 1090ms 53ms pypy 3.8.13
chapel 1-m.chpl 4004ms 54ms 169.4MB 7737ms 47ms chpl 1.27.0
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 1000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 263ms 2.4ms 30.2MB 480ms 3ms chpl 1.27.0
python 1.py 276ms 1.4ms 95.1MB 237ms 17ms pypy 3.8.13
python 1.py 966ms 3.5ms 16.8MB 947ms 3ms pyston 3.8.12
python 1.py 978ms 1.6ms 16.9MB 960ms 0ms cpython 3.10.7

edigits

Input: 250001

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 97ms 0.5ms 36.1MB 90ms 0ms chpl 1.27.0
python 1.py 828ms 1.6ms 87.8MB 797ms 20ms pypy 3.8.13
python 1.py 1670ms 1.4ms 9.3MB 1657ms 0ms cpython 3.10.7
python 1.py 2829ms 1.6ms 9.6MB 2820ms 0ms pyston 3.8.12

Input: 100000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 46ms 1.3ms 34.1MB 37ms 3ms chpl 1.27.0
python 1.py 213ms 0.8ms 81.9MB 190ms 10ms pypy 3.8.13
python 1.py 307ms 0.2ms 8.5MB 293ms 0ms cpython 3.10.7
python 1.py 488ms 0.9ms 8.8MB 470ms 0ms pyston 3.8.12

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 5-m.chpl 147ms 2.2ms 32.0MB 250ms 3ms chpl 1.27.0
python 1.py 2607ms 3.7ms 91.1MB 2573ms 13ms pypy 3.8.13
python 5-m.py 3750ms 328ms 12.6MB 5597ms 1000ms pyston 3.8.12
python 5-m.py 3964ms 11ms 85.4MB 4430ms 853ms pypy 3.8.13
python 1.py 4082ms 19ms 7.8MB 4070ms 0ms pyston 3.8.12
python 5-m.py 4416ms 24ms 12.1MB 6350ms 1030ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 5.chpl 34ms 1.5ms 32.0MB 30ms 7ms chpl 1.27.0
python 1.py 346ms 2.7ms 81.7MB 307ms 20ms pypy 3.8.13
python 1.py 424ms 5.3ms 7.9MB 410ms 0ms pyston 3.8.12
python 5-m.py 449ms 34ms 12.6MB 640ms 120ms pyston 3.8.12
python 5-m.py 513ms 1.8ms 12.1MB 660ms 147ms cpython 3.10.7
python 1.py 708ms 1.5ms 7.8MB 700ms 0ms cpython 3.10.7
python 5-m.py 821ms 4.5ms 85.2MB 950ms 420ms pypy 3.8.13

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 13ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 15ms 0.3ms 7.3MB 10ms 0ms cpython 3.10.7
chapel 1.chpl 18ms 3.7ms 32.4MB 10ms 0ms chpl 1.27.0
python 1.py 34ms 2.4ms 54.0MB 14ms 6ms pypy 3.8.13

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3-m.chpl 936ms 1.2ms 100.4MB 1733ms 13ms chpl 1.27.0
python 3-m.py 3577ms 8.2ms 203.7MB 6277ms 330ms pypy 3.8.13
python 3.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 3.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3-m.chpl 157ms 2.7ms 90.4MB 257ms 10ms chpl 1.27.0
python 3-m.py 533ms 8.0ms 96.8MB 753ms 123ms pypy 3.8.13
python 3-m.py 594ms 7.7ms 14.8MB 1040ms 37ms pyston 3.8.12
python 3-m.py 864ms 11ms 15.3MB 1557ms 30ms cpython 3.10.7

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 363ms 2.4ms 32.4MB 350ms 7ms chpl 1.27.0
python 2.py 2065ms 46ms 80.0MB 2033ms 13ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 56ms 1.3ms 32.4MB 50ms 0ms chpl 1.27.0
python 2.py 269ms 5.8ms 79.6MB 237ms 13ms pypy 3.8.13
python 1.py 583ms 7.9ms 79.4MB 560ms 3ms pypy 3.8.13
python 1.py 1184ms 11ms 7.2MB 1167ms 0ms pyston 3.8.12
python 2.py 1613ms 7.5ms 8.1MB 1600ms 0ms pyston 3.8.12
python 1.py 4703ms 56ms 7.4MB 4687ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 593ms 1.5ms 34.1MB 580ms 7ms chpl 1.27.0
python 4.py 2726ms 13ms 119.6MB 2670ms 37ms pypy 3.8.13
python 4.py 2759ms 0.9ms 9.1MB 2723ms 17ms pyston 3.8.12
python 4.py 2767ms 0.5ms 8.6MB 2737ms 17ms cpython 3.10.7

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 154ms 1.9ms 34.1MB 143ms 3ms chpl 1.27.0
python 4.py 653ms 0.8ms 8.8MB 640ms 0ms pyston 3.8.12
python 4.py 660ms 0.3ms 8.3MB 647ms 0ms cpython 3.10.7
python 4.py 684ms 5.3ms 84.5MB 657ms 10ms pypy 3.8.13

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3.chpl 2015ms 13ms 251.7MB 1870ms 133ms chpl 1.27.0
python 1.py 3326ms 0.7ms 265.3MB 3237ms 70ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3.chpl 222ms 1.0ms 56.8MB 203ms 10ms chpl 1.27.0
python 1.py 465ms 2.9ms 103.5MB 423ms 23ms pypy 3.8.13
python 1.py 699ms 3.7ms 30.3MB 653ms 20ms pyston 3.8.12
python 1.py 847ms 1.1ms 31.0MB 817ms 17ms cpython 3.10.7

secp256k1

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 1536ms 10.0ms 32.3MB 1527ms 0ms chpl 1.27.0
python 1.py 4528ms 9.5ms 8.3MB 4513ms 0ms pyston 3.8.12
python 1.py 4900ms 14ms 87.1MB 4867ms 17ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 398ms 0.9ms 32.3MB 383ms 3ms chpl 1.27.0
python 1.py 1144ms 1.3ms 8.4MB 1130ms 0ms pyston 3.8.12
python 1.py 1375ms 6.1ms 84.2MB 1343ms 13ms pypy 3.8.13
python 1.py 1498ms 22ms 7.3MB 1480ms 0ms cpython 3.10.7

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 2299ms 91ms 32.5MB 4450ms 7ms chpl 1.27.0
chapel 1.chpl 4375ms 0.8ms 32.5MB 4363ms 3ms chpl 1.27.0
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 581ms 0.2ms 32.5MB 1107ms 0ms chpl 1.27.0
chapel 1.chpl 1110ms 0.7ms 32.4MB 1100ms 3ms chpl 1.27.0
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 165ms 2.1ms 32.5MB 287ms 0ms chpl 1.27.0
chapel 1.chpl 306ms 19ms 32.4MB 283ms 10ms chpl 1.27.0
python 8-m.py 3292ms 24ms 109.1MB 5633ms 357ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12