Programming Language and compiler
Benchmarks

OCaml VS Python benchmarks

Current benchmark data was generated on Thu Sep 22 2022, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1574ms 2.7ms 274.1MB 1453ms 100ms pypy 3.8.13
ocaml 1.ml 3969ms 9.3ms 121.0MB 3927ms 23ms ocaml 4.14.0
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 120ms 2.4ms 83.3MB 93ms 10ms pypy 3.8.13
ocaml 1.ml 271ms 1.1ms 17.4MB 250ms 3ms ocaml 4.14.0
python 1.py 610ms 3.8ms 11.9MB 583ms 10ms pyston 3.8.12
python 1.py 1514ms 8.6ms 11.8MB 1493ms 3ms cpython 3.10.7

fannkuch-redux

Input: 11

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 3940ms 31ms 2.5MB 3927ms 0ms ocaml 4.14.0

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 326ms 0.7ms 2.6MB 313ms 0ms ocaml 4.14.0

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 6.ml 423ms 2.2ms 23.9MB 403ms 0ms ocaml 4.14.0
python 1.py 2607ms 3.7ms 91.1MB 2573ms 13ms pypy 3.8.13
python 5-m.py 3750ms 328ms 12.6MB 5597ms 1000ms pyston 3.8.12
python 5-m.py 3964ms 11ms 85.4MB 4430ms 853ms pypy 3.8.13
python 1.py 4082ms 19ms 7.8MB 4070ms 0ms pyston 3.8.12
python 5-m.py 4416ms 24ms 12.1MB 6350ms 1030ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 6.ml 49ms 3.5ms 4.8MB 37ms 0ms ocaml 4.14.0
python 1.py 346ms 2.7ms 81.7MB 307ms 20ms pypy 3.8.13
python 1.py 424ms 5.3ms 7.9MB 410ms 0ms pyston 3.8.12
python 5-m.py 449ms 34ms 12.6MB 640ms 120ms pyston 3.8.12
python 5-m.py 513ms 1.8ms 12.1MB 660ms 147ms cpython 3.10.7
python 1.py 708ms 1.5ms 7.8MB 700ms 0ms cpython 3.10.7
python 5-m.py 821ms 4.5ms 85.2MB 950ms 420ms pypy 3.8.13

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 1.9ms 0.5ms 2.5MB 0ms 0ms ocaml 4.14.0
python 1.py 13ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 15ms 0.3ms 7.3MB 10ms 0ms cpython 3.10.7
python 1.py 34ms 2.4ms 54.0MB 14ms 6ms pypy 3.8.13

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 199ms 0.9ms 80.3MB 170ms 17ms pypy 3.8.13
ocaml 1.ml 238ms 0.5ms 5.9MB 227ms 0ms ocaml 4.14.0
python 1.py 241ms 0.6ms 81.0MB 213ms 13ms pypy 3.8.13
python 1.py 993ms 2.8ms 8.9MB 977ms 0ms pyston 3.8.12
python 2.py 1176ms 4.2ms 8.7MB 1163ms 0ms pyston 3.8.12
python 1.py 1831ms 10ms 8.5MB 1817ms 0ms cpython 3.10.7
python 2.py 2615ms 35ms 8.3MB 2597ms 0ms cpython 3.10.7

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 468ms 4.4ms 80.4MB 443ms 7ms pypy 3.8.13
python 1.py 509ms 2.2ms 81.5MB 477ms 13ms pypy 3.8.13
ocaml 1.ml 707ms 1.5ms 5.8MB 693ms 0ms ocaml 4.14.0
python 1.py 3011ms 54ms 9.0MB 3000ms 0ms pyston 3.8.12
python 2.py 3482ms 6.9ms 8.7MB 3470ms 0ms pyston 3.8.12
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 104ms 0.4ms 5.1MB 90ms 0ms ocaml 4.14.0
python 2.py 145ms 1.3ms 80.1MB 117ms 13ms pypy 3.8.13
python 1.py 164ms 3.1ms 81.1MB 140ms 10ms pypy 3.8.13
python 1.py 481ms 2.8ms 8.7MB 467ms 0ms pyston 3.8.12
python 2.py 546ms 3.1ms 8.1MB 530ms 0ms pyston 3.8.12
python 1.py 862ms 4.4ms 7.8MB 853ms 0ms cpython 3.10.7
python 2.py 1257ms 5.4ms 7.3MB 1240ms 0ms cpython 3.10.7

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 1503ms 5.0ms 62.6MB 1460ms 23ms ocaml 4.14.0
python 1.py 2434ms 16ms 293.6MB 2253ms 163ms pypy 3.8.13
ocaml 2.ml 2552ms 7.6ms 71.4MB 2520ms 20ms ocaml 4.14.0
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 192ms 0.9ms 18.6MB 177ms 0ms ocaml 4.14.0
python 1.py 351ms 5.0ms 145.1MB 290ms 40ms pypy 3.8.13
ocaml 2.ml 430ms 3.0ms 23.6MB 410ms 3ms ocaml 4.14.0
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 470ms 0.2ms 2.6MB 460ms 0ms ocaml 4.14.0
python 2.py 2065ms 46ms 80.0MB 2033ms 13ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 52ms 3.8ms 2.6MB 40ms 0ms ocaml 4.14.0
python 2.py 269ms 5.8ms 79.6MB 237ms 13ms pypy 3.8.13
python 1.py 583ms 7.9ms 79.4MB 560ms 3ms pypy 3.8.13
python 1.py 1184ms 11ms 7.2MB 1167ms 0ms pyston 3.8.12
python 2.py 1613ms 7.5ms 8.1MB 1600ms 0ms pyston 3.8.12
python 1.py 4703ms 56ms 7.4MB 4687ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 1976ms 10ms 555.7MB 1877ms 87ms ocaml 4.14.0
ocaml 2.ml 2265ms 1.2ms 19.9MB 2247ms 0ms ocaml 4.14.0
python 1.py 4008ms 8.7ms 391.4MB 3873ms 117ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 390ms 10ms 142.6MB 353ms 20ms ocaml 4.14.0
ocaml 2.ml 551ms 4.1ms 9.5MB 537ms 0ms ocaml 4.14.0
python 1.py 1030ms 33ms 215.7MB 967ms 40ms pypy 3.8.13
python 1.py 1868ms 12ms 85.9MB 1830ms 17ms pyston 3.8.12
python 1.py 4326ms 5.5ms 85.9MB 4293ms 20ms cpython 3.10.7

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 3326ms 0.7ms 265.3MB 3237ms 70ms pypy 3.8.13
ocaml 2.ml timeout 0.0ms 0.0MB 0ms 0ms ocaml 4.14.0
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 465ms 2.9ms 103.5MB 423ms 23ms pypy 3.8.13
ocaml 2.ml 683ms 0.6ms 50.4MB 657ms 10ms ocaml 4.14.0
python 1.py 699ms 3.7ms 30.3MB 653ms 20ms pyston 3.8.12
python 1.py 847ms 1.1ms 31.0MB 817ms 17ms cpython 3.10.7

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml timeout 0.0ms 0.0MB 0ms 0ms ocaml 4.14.0
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 1645ms 0.1ms 3.4MB 1633ms 0ms ocaml 4.14.0
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 415ms 0.7ms 3.1MB 400ms 0ms ocaml 4.14.0
python 8-m.py 3292ms 24ms 109.1MB 5633ms 357ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12