Programming Language and compiler
Benchmarks

Chapel VS Python benchmarks

Current benchmark data was generated on Sun Dec 01 2024, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][4 cores] AMD EPYC 7763 64-Core Processor (Model 1)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1149ms 16ms 95.7MB 1087ms 47ms pypy 3.10.14
chapel 4.chpl 1950ms 11ms 66.4MB 1937ms 0ms chpl 1.31.0
chapel 3.chpl 2095ms 33ms 66.4MB 2087ms 3ms chpl 1.31.0
python 1.py timeout 0.0ms 40.8MB 4970ms 17ms cpython 3.13.0
python 1.py timeout 0.0ms 40.6MB 4743ms 240ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 94ms 1.9ms 68.0MB 63ms 20ms pypy 3.10.14
chapel 4.chpl 193ms 3.0ms 36.4MB 183ms 7ms chpl 1.31.0
chapel 3.chpl 205ms 18ms 34.3MB 190ms 3ms chpl 1.31.0
python 1.py 491ms 25ms 11.9MB 463ms 7ms pyston 3.8.12
python 1.py 710ms 4.2ms 12.8MB 697ms 0ms cpython 3.13.0

coro-prime-sieve

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1178ms 15ms 102.5MB 1133ms 30ms pypy 3.10.14
chapel 1-m.chpl 2252ms 19ms 555.7MB 4367ms 70ms chpl 1.31.0
python 1.py timeout 0.0ms 21.1MB 4977ms 7ms cpython 3.13.0
python 1.py timeout 0.0ms 17.1MB 4977ms 7ms pyston 3.8.12

Input: 1000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 196ms 3.7ms 508.7MB 307ms 43ms chpl 1.31.0
python 1.py 219ms 6.3ms 84.3MB 183ms 20ms pypy 3.10.14
python 1.py 572ms 3.4ms 20.4MB 543ms 10ms cpython 3.13.0
python 1.py 685ms 13ms 16.8MB 663ms 7ms pyston 3.8.12

edigits

Input: 250001

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 80ms 1.0ms 36.4MB 73ms 0ms chpl 1.31.0
python 1.py 250ms 4.1ms 82.8MB 203ms 30ms pypy 3.10.14
python 1.py 356ms 1.6ms 12.6MB 337ms 3ms cpython 3.13.0
python 1.py 1997ms 6.3ms 9.6MB 1983ms 0ms pyston 3.8.12

Input: 100000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 41ms 1.3ms 34.4MB 30ms 7ms chpl 1.31.0
python 1.py 101ms 1.8ms 62.9MB 70ms 17ms pypy 3.10.14
python 1.py 112ms 8.0ms 11.9MB 97ms 0ms cpython 3.13.0
python 1.py 346ms 1.7ms 8.8MB 330ms 0ms pyston 3.8.12

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 5-m.chpl 130ms 27ms 32.4MB 217ms 0ms chpl 1.31.0
python 1.py 1909ms 10.0ms 88.3MB 1867ms 23ms pypy 3.10.14
python 5-m.py 2472ms 27ms 12.5MB 3610ms 997ms pyston 3.8.12
python 5-m.py 2826ms 10ms 64.9MB 4047ms 780ms pypy 3.10.14
python 1.py 3033ms 47ms 8.1MB 3017ms 0ms pyston 3.8.12
python 5-m.py 3411ms 65ms 13.9MB 4617ms 930ms cpython 3.13.0
python 1.py 4670ms 96ms 8.8MB 4650ms 3ms cpython 3.13.0

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 5.chpl 29ms 0.1ms 32.4MB 27ms 0ms chpl 1.31.0
python 1.py 250ms 0.3ms 63.8MB 223ms 10ms pypy 3.10.14
python 5-m.py 312ms 1.4ms 12.5MB 420ms 123ms pyston 3.8.12
python 1.py 312ms 2.4ms 8.1MB 297ms 0ms pyston 3.8.12
python 5-m.py 408ms 10ms 13.9MB 520ms 120ms cpython 3.13.0
python 1.py 478ms 11ms 8.8MB 463ms 0ms cpython 3.13.0
python 5-m.py 555ms 5.0ms 65.0MB 817ms 313ms pypy 3.10.14

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 12ms 1.2ms 7.6MB 5ms 5ms pyston 3.8.12
python 1.py 15ms 0.3ms 8.5MB 10ms 0ms cpython 3.13.0
chapel 1.chpl 16ms 0.7ms 32.8MB 7ms 3ms chpl 1.31.0
python 1.py 29ms 1.5ms 51.0MB 4ms 12ms pypy 3.10.14

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3-m.chpl 760ms 11ms 91.1MB 1427ms 7ms chpl 1.31.0
python 3-m.py 2277ms 16ms 137.9MB 7727ms 313ms pypy 3.10.14
python 3-m.py 3092ms 33ms 33.7MB 11747ms 113ms pyston 3.8.12
python 3.py timeout 0.0ms 35.4MB 83ms 30ms cpython 3.13.0

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3-m.chpl 122ms 1.7ms 92.9MB 200ms 7ms chpl 1.31.0
python 3-m.py 376ms 0.7ms 77.1MB 960ms 143ms pypy 3.10.14
python 3-m.py 397ms 3.1ms 17.7MB 1317ms 63ms pyston 3.8.12
python 3-m.py 664ms 8.0ms 17.0MB 2327ms 67ms cpython 3.13.0

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 334ms 4.0ms 32.8MB 317ms 7ms chpl 1.31.0
python 2.py 1352ms 14ms 59.8MB 1320ms 17ms pypy 3.10.14
python 1.py 2641ms 30ms 56.6MB 2610ms 13ms pypy 3.10.14
python 1.py timeout 0.0ms 8.8MB 4987ms 0ms cpython 3.13.0
python 2.py timeout 0.0ms 8.8MB 4983ms 0ms cpython 3.13.0
python 1.py timeout 0.0ms 8.3MB 4990ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 8.1MB 4990ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 52ms 0.2ms 32.8MB 40ms 0ms chpl 1.31.0
python 2.py 180ms 2.7ms 57.6MB 153ms 10ms pypy 3.10.14
python 1.py 304ms 5.3ms 56.8MB 277ms 10ms pypy 3.10.14
python 1.py 1078ms 255ms 8.3MB 1063ms 0ms pyston 3.8.12
python 2.py 1319ms 15ms 8.1MB 1303ms 0ms pyston 3.8.12
python 1.py 2912ms 17ms 8.8MB 2900ms 0ms cpython 3.13.0
python 2.py 3132ms 41ms 8.8MB 3123ms 0ms cpython 3.13.0

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 452ms 1.7ms 32.4MB 440ms 3ms chpl 1.31.0
python 4.py 1907ms 12ms 8.9MB 1853ms 37ms pyston 3.8.12
python 4.py 1961ms 9.8ms 9.5MB 1897ms 47ms cpython 3.13.0
python 4.py 2141ms 12ms 89.1MB 2093ms 33ms pypy 3.10.14

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 2.chpl 121ms 1.7ms 34.4MB 117ms 3ms chpl 1.31.0
python 4.py 450ms 3.5ms 8.6MB 437ms 0ms pyston 3.8.12
python 4.py 484ms 30ms 9.1MB 470ms 0ms cpython 3.13.0
python 4.py 545ms 13ms 83.5MB 503ms 27ms pypy 3.10.14

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3.chpl 1395ms 6.8ms 224.1MB 1333ms 53ms chpl 1.31.0
python 1.py 2217ms 5.8ms 243.1MB 2157ms 43ms pypy 3.10.14
python 1.py 4742ms 15ms 226.2MB 4557ms 170ms pyston 3.8.12
python 1.py timeout 0.0ms 179.6MB 4903ms 77ms cpython 3.13.0

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 3.chpl 170ms 2.9ms 53.0MB 143ms 20ms chpl 1.31.0
python 1.py 330ms 5.3ms 85.5MB 283ms 30ms pypy 3.10.14
python 1.py 493ms 3.6ms 30.3MB 450ms 27ms pyston 3.8.12
python 1.py 587ms 4.1ms 29.9MB 547ms 23ms cpython 3.13.0

secp256k1

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 1235ms 79ms 33.1MB 1227ms 3ms chpl 1.31.0
python 1.py 3142ms 2.3ms 8.3MB 3133ms 0ms pyston 3.8.12
python 1.py 3709ms 13ms 74.6MB 3663ms 30ms pypy 3.10.14
python 1.py 3866ms 69ms 8.8MB 3857ms 0ms cpython 3.13.0

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1.chpl 313ms 1.6ms 33.1MB 300ms 7ms chpl 1.31.0
python 1.py 799ms 3.3ms 8.3MB 783ms 0ms pyston 3.8.12
python 1.py 976ms 18ms 8.8MB 960ms 0ms cpython 3.13.0
python 1.py 1043ms 10ms 67.5MB 1003ms 17ms pypy 3.10.14

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 1933ms 21ms 33.0MB 3823ms 0ms chpl 1.31.0
chapel 1.chpl 3709ms 10.0ms 32.8MB 3703ms 0ms chpl 1.31.0
python 8.py timeout 0.0ms 16.7MB 53ms 10ms cpython 3.13.0
python 8.py timeout 0.0ms 100.0MB 833ms 137ms pypy 3.10.14
python 8.py timeout 0.0ms 18.2MB 70ms 13ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 496ms 1.0ms 32.8MB 960ms 0ms chpl 1.31.0
chapel 1.chpl 946ms 5.3ms 32.8MB 937ms 7ms chpl 1.31.0
python 8.py timeout 0.0ms 16.4MB 93ms 30ms cpython 3.13.0
python 8.py timeout 0.0ms 99.8MB 1533ms 517ms pypy 3.10.14
python 8.py timeout 0.0ms 17.5MB 140ms 33ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
chapel 1-m.chpl 140ms 0.3ms 32.9MB 243ms 3ms chpl 1.31.0
chapel 1.chpl 257ms 3.3ms 32.8MB 247ms 0ms chpl 1.31.0
python 8-m.py 1973ms 18ms 95.1MB 6307ms 370ms pypy 3.10.14
python 8.py timeout 0.0ms 16.1MB 277ms 40ms cpython 3.13.0
python 8.py timeout 0.0ms 17.3MB 397ms 113ms pyston 3.8.12