Programming Language and compiler
Benchmarks

Chapel VS Python benchmarks

Current benchmark data was generated on Wed Feb 08 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz (Model 85)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1922ms 8.8ms 217.4MB 1770ms 137ms pypy 3.9.16
chapel 4.chpl 2946ms 5.9ms 64.3MB 2910ms 20ms chpl 1.29.0
chapel 3.chpl 3233ms 39ms 64.3MB 3197ms 17ms chpl 1.29.0
python 1.py timeout 0.0ms 40.8MB 4963ms 17ms cpython 3.11.2
python 1.py timeout 0.0ms 40.3MB 4950ms 27ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 148ms 2.4ms 74.3MB 113ms 13ms pypy 3.9.16
chapel 4.chpl 279ms 2.6ms 34.3MB 260ms 7ms chpl 1.29.0
chapel 3.chpl 294ms 2.5ms 32.1MB 270ms 7ms chpl 1.29.0
python 1.py 776ms 2.6ms 11.9MB 753ms 7ms pyston 3.8.12
python 1.py 1144ms 6.2ms 12.5MB 1120ms 3ms cpython 3.11.2

coro-prime-sieve

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1472ms 22ms 227.4MB 1350ms 107ms pypy 3.9.16
chapel 1-m.chpl 4803ms 115ms 83.5MB 9277ms 70ms chpl 1.29.0
python 1.py timeout 0.0ms 20.1MB 4973ms 7ms cpython 3.11.2
python 1.py timeout 0.0ms 17.1MB 4977ms 3ms pyston 3.8.12

Input: 1000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 312ms 5.3ms 87.6MB 267ms 27ms pypy 3.9.16
chapel 1-m.chpl 348ms 12ms 48.1MB 623ms 20ms chpl 1.29.0
python 1.py 913ms 1.0ms 19.9MB 897ms 3ms cpython 3.11.2
python 1.py 989ms 2.8ms 16.9MB 973ms 0ms pyston 3.8.12

edigits

Input: 250001

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 104ms 0.6ms 18.5MB 90ms 0ms chpl 1.29.0
python 1.py 706ms 5.1ms 78.4MB 660ms 27ms pypy 3.9.16
python 1.py 1670ms 7.4ms 10.1MB 1657ms 0ms cpython 3.11.2
python 1.py 2754ms 4.2ms 9.6MB 2730ms 3ms pyston 3.8.12

Input: 100000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 46ms 3.8ms 34.1MB 37ms 0ms chpl 1.29.0
python 1.py 196ms 1.0ms 73.2MB 163ms 13ms pypy 3.9.16
python 1.py 310ms 1.7ms 9.4MB 297ms 3ms cpython 3.11.2
python 1.py 482ms 2.3ms 8.9MB 463ms 3ms pyston 3.8.12

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 5-m.chpl 164ms 5.7ms 32.0MB 277ms 3ms chpl 1.29.0
python 5-m.py 3319ms 2874ms 76.1MB 6847ms 1390ms pypy 3.9.16
python 1.py 3477ms 32ms 84.7MB 3433ms 23ms pypy 3.9.16
python 5-m.py 3960ms 31ms 12.5MB 5733ms 1390ms pyston 3.8.12
python 1.py 4684ms 24ms 7.9MB 4667ms 0ms pyston 3.8.12
python 5-m.py 4713ms 13ms 13.6MB 6807ms 1347ms cpython 3.11.2
python 1.py timeout 0.0ms 8.6MB 4977ms 3ms cpython 3.11.2

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 5.chpl 33ms 3.1ms 22.0MB 30ms 7ms chpl 1.29.0
python 1.py 425ms 2.0ms 73.7MB 383ms 23ms pypy 3.9.16
python 5-m.py 479ms 2.6ms 12.6MB 630ms 173ms pyston 3.8.12
python 1.py 495ms 15ms 7.9MB 480ms 0ms pyston 3.8.12
python 5-m.py 559ms 1.6ms 13.5MB 737ms 167ms cpython 3.11.2
python 1.py 715ms 1.6ms 8.7MB 700ms 0ms cpython 3.11.2
python 5-m.py 932ms 8.4ms 76.0MB 1000ms 530ms pypy 3.9.16

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 15ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 16ms 1.3ms 8.5MB 10ms 0ms cpython 3.11.2
chapel 1.chpl 17ms 1.4ms 22.2MB 7ms 3ms chpl 1.29.0
python 1.py 39ms 0.9ms 52.8MB 12ms 12ms pypy 3.9.16

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3-m.chpl 1180ms 16ms 85.0MB 2157ms 33ms chpl 1.29.0
python 3-m.py 4923ms 24ms 209.9MB 8630ms 467ms pypy 3.9.16
python 3.py timeout 0.0ms 35.1MB 100ms 27ms cpython 3.11.2
python 3.py timeout 0.0ms 34.2MB 80ms 27ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3-m.chpl 201ms 2.7ms 76.7MB 310ms 23ms chpl 1.29.0
python 3-m.py 686ms 12ms 86.1MB 1000ms 143ms pypy 3.9.16
python 3-m.py 737ms 14ms 17.6MB 1283ms 47ms pyston 3.8.12
python 3-m.py 1024ms 9.7ms 16.9MB 1843ms 43ms cpython 3.11.2

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 485ms 2.4ms 22.2MB 467ms 7ms chpl 1.29.0
python 2.py 2414ms 33ms 71.4MB 2383ms 17ms pypy 3.9.16
python 1.py 4354ms 39ms 70.3MB 4317ms 17ms pypy 3.9.16
python 1.py timeout 0.0ms 8.4MB 4983ms 0ms cpython 3.11.2
python 2.py timeout 0.0ms 8.6MB 4987ms 0ms cpython 3.11.2
python 1.py timeout 0.0ms 8.4MB 4983ms 3ms pyston 3.8.12
python 2.py timeout 0.0ms 7.9MB 4973ms 7ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 70ms 1.7ms 32.2MB 47ms 7ms chpl 1.29.0
python 2.py 309ms 3.3ms 71.0MB 277ms 17ms pypy 3.9.16
python 1.py 499ms 4.8ms 70.3MB 460ms 20ms pypy 3.9.16
python 1.py 1397ms 19ms 8.3MB 1380ms 0ms pyston 3.8.12
python 2.py 1891ms 7.9ms 7.9MB 1873ms 3ms pyston 3.8.12
python 1.py 3917ms 146ms 8.4MB 3903ms 0ms cpython 3.11.2
python 2.py 4128ms 39ms 8.6MB 4107ms 0ms cpython 3.11.2

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 582ms 3.7ms 30.1MB 560ms 3ms chpl 1.29.0
python 4.py 2886ms 4.0ms 9.0MB 2840ms 27ms pyston 3.8.12
python 4.py 2892ms 0.9ms 9.5MB 2843ms 33ms cpython 3.11.2
python 4.py 2991ms 35ms 125.0MB 2913ms 57ms pypy 3.9.16

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 151ms 2.2ms 20.8MB 130ms 3ms chpl 1.29.0
python 4.py 705ms 18ms 9.1MB 687ms 0ms cpython 3.11.2
python 4.py 713ms 23ms 8.8MB 700ms 0ms pyston 3.8.12
python 4.py 774ms 10ms 77.2MB 730ms 23ms pypy 3.9.16

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3.chpl 2230ms 32ms 221.7MB 2030ms 187ms chpl 1.29.0
python 1.py 3909ms 20ms 273.0MB 3747ms 143ms pypy 3.9.16
python 1.py timeout 0.0ms 106.7MB 4930ms 57ms cpython 3.11.2
python 1.py timeout 0.0ms 108.8MB 4920ms 60ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3.chpl 241ms 0.5ms 50.8MB 203ms 20ms chpl 1.29.0
python 1.py 537ms 4.5ms 95.5MB 483ms 40ms pypy 3.9.16
python 1.py 932ms 27ms 30.5MB 890ms 23ms pyston 3.8.12
python 1.py 982ms 2.5ms 31.9MB 937ms 30ms cpython 3.11.2

secp256k1

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 1778ms 5.0ms 32.4MB 1753ms 10ms chpl 1.29.0
python 1.py timeout 0.0ms 8.4MB 4987ms 0ms cpython 3.11.2
python 1.py timeout 0.0ms 78.4MB 4950ms 27ms pypy 3.9.16
python 1.py timeout 0.0ms 8.4MB 4987ms 0ms pyston 3.8.12

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 461ms 0.8ms 32.4MB 447ms 3ms chpl 1.29.0
python 1.py 1339ms 1.6ms 8.3MB 1323ms 0ms pyston 3.8.12
python 1.py 1443ms 2.4ms 8.4MB 1433ms 0ms cpython 3.11.2
python 1.py 1749ms 4.6ms 74.8MB 1720ms 13ms pypy 3.9.16

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 2324ms 13ms 32.3MB 4533ms 10ms chpl 1.29.0
chapel 1.chpl 4465ms 18ms 32.4MB 4443ms 7ms chpl 1.29.0
python 8.py timeout 0.0ms 16.5MB 60ms 17ms cpython 3.11.2
python 8.py timeout 0.0ms 98.9MB 643ms 93ms pypy 3.9.16
python 8.py timeout 0.0ms 17.5MB 57ms 20ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 654ms 50ms 32.3MB 1177ms 10ms chpl 1.29.0
chapel 1.chpl 1147ms 4.0ms 32.3MB 1130ms 3ms chpl 1.29.0
python 8.py timeout 0.0ms 14.3MB 133ms 20ms cpython 3.11.2
python 8.py timeout 0.0ms 103.4MB 1023ms 183ms pypy 3.9.16
python 8.py timeout 0.0ms 17.1MB 150ms 33ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 179ms 4.2ms 32.3MB 300ms 10ms chpl 1.29.0
chapel 1.chpl 309ms 5.9ms 32.3MB 287ms 7ms chpl 1.29.0
python 8-m.py 4367ms 34ms 102.1MB 7580ms 460ms pypy 3.9.16
python 8.py timeout 0.0ms 15.9MB 243ms 33ms cpython 3.11.2
python 8.py timeout 0.0ms 17.1MB 333ms 63ms pyston 3.8.12