Programming Language and compiler
Benchmarks

OCaml VS Python benchmarks

Current benchmark data was generated on Thu May 04 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz (Model 85)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1790ms 46ms 217.6MB 1663ms 110ms pypy 3.9.16
ocaml 1.ml timeout 0.0ms 145.4MB 4903ms 73ms ocaml 5.0.0
python 1.py timeout 0.0ms 40.9MB 4960ms 17ms cpython 3.11.3
python 1.py timeout 0.0ms 40.3MB 4917ms 63ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 160ms 14ms 74.6MB 110ms 23ms pypy 3.9.16
ocaml 1.ml 359ms 8.8ms 22.9MB 337ms 7ms ocaml 5.0.0
python 1.py 671ms 2.6ms 12.0MB 633ms 17ms pyston 3.8.12
python 1.py 966ms 14ms 12.5MB 947ms 3ms cpython 3.11.3

fannkuch-redux

Input: 11

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 3637ms 2.2ms 2.7MB 3620ms 0ms ocaml 5.0.0

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 303ms 0.9ms 2.8MB 290ms 0ms ocaml 5.0.0

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 6.ml 387ms 4.7ms 22.7MB 360ms 7ms ocaml 5.0.0
python 5-m.py 3471ms 17ms 12.7MB 4990ms 1227ms pyston 3.8.12
python 1.py 3566ms 105ms 85.0MB 3510ms 37ms pypy 3.9.16
python 1.py 3928ms 17ms 8.0MB 3910ms 0ms pyston 3.8.12
python 5-m.py 4067ms 23ms 13.6MB 5703ms 1260ms cpython 3.11.3
python 5-m.py 4917ms 70ms 76.1MB 5657ms 1113ms pypy 3.9.16
python 1.py timeout 0.0ms 8.6MB 4980ms 3ms cpython 3.11.3

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 6.ml 42ms 0.4ms 4.9MB 30ms 0ms ocaml 5.0.0
python 1.py 409ms 0.7ms 7.9MB 393ms 3ms pyston 3.8.12
python 1.py 427ms 11ms 73.7MB 390ms 20ms pypy 3.9.16
python 5-m.py 428ms 8.5ms 12.6MB 557ms 143ms pyston 3.8.12
python 5-m.py 505ms 14ms 13.6MB 593ms 207ms cpython 3.11.3
python 1.py 604ms 1.0ms 8.6MB 587ms 0ms cpython 3.11.3
python 5-m.py 874ms 16ms 76.3MB 983ms 463ms pypy 3.9.16

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 2.9ms 0.5ms 2.7MB 0ms 0ms ocaml 5.0.0
python 1.py 13ms 0.4ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 15ms 0.9ms 8.4MB 7ms 3ms cpython 3.11.3
python 1.py 40ms 2.2ms 52.0MB 14ms 10ms pypy 3.9.16

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 232ms 1.0ms 71.0MB 203ms 17ms pypy 3.9.16
ocaml 1.ml 250ms 4.9ms 6.7MB 240ms 0ms ocaml 5.0.0
python 1.py 278ms 6.3ms 71.9MB 247ms 13ms pypy 3.9.16
python 1.py 925ms 7.8ms 9.0MB 907ms 0ms pyston 3.8.12
python 2.py 1073ms 2.7ms 8.8MB 1057ms 0ms pyston 3.8.12
python 1.py 1276ms 27ms 9.3MB 1257ms 0ms cpython 3.11.3
python 2.py 1421ms 16ms 9.1MB 1407ms 0ms cpython 3.11.3

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 557ms 2.9ms 71.5MB 527ms 20ms pypy 3.9.16
python 1.py 608ms 1.1ms 72.4MB 587ms 7ms pypy 3.9.16
ocaml 1.ml 727ms 1.6ms 6.7MB 717ms 0ms ocaml 5.0.0
python 1.py 2726ms 8.2ms 9.0MB 2710ms 0ms pyston 3.8.12
python 2.py 3173ms 4.4ms 8.7MB 3160ms 0ms pyston 3.8.12
python 1.py 3748ms 29ms 9.2MB 3733ms 0ms cpython 3.11.3
python 2.py 4262ms 61ms 9.1MB 4247ms 0ms cpython 3.11.3

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 110ms 0.4ms 6.0MB 93ms 0ms ocaml 5.0.0
python 2.py 162ms 0.5ms 71.2MB 130ms 13ms pypy 3.9.16
python 1.py 181ms 1.1ms 72.4MB 140ms 23ms pypy 3.9.16
python 1.py 444ms 1.3ms 8.8MB 430ms 0ms pyston 3.8.12
python 2.py 506ms 6.4ms 8.3MB 490ms 0ms pyston 3.8.12
python 1.py 598ms 1.3ms 8.7MB 583ms 0ms cpython 3.11.3
python 2.py 712ms 6.9ms 8.4MB 697ms 0ms cpython 3.11.3

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 1931ms 14ms 69.1MB 1880ms 30ms ocaml 5.0.0
python 1.py 2892ms 160ms 230.4MB 2713ms 160ms pypy 3.9.16
ocaml 2.ml 3617ms 28ms 74.2MB 3557ms 43ms ocaml 5.0.0
python 1.py timeout 0.0ms 92.9MB 4940ms 40ms cpython 3.11.3
python 1.py timeout 0.0ms 130.3MB 4920ms 53ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 275ms 3.8ms 25.6MB 253ms 3ms ocaml 5.0.0
python 1.py 423ms 11ms 149.9MB 360ms 47ms pypy 3.9.16
ocaml 2.ml 629ms 19ms 30.1MB 610ms 7ms ocaml 5.0.0
python 1.py timeout 0.0ms 37.1MB 4977ms 7ms cpython 3.11.3
python 1.py timeout 0.0ms 41.4MB 4960ms 17ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 546ms 8.2ms 2.9MB 533ms 0ms ocaml 5.0.0
python 2.py 2132ms 23ms 71.3MB 2103ms 10ms pypy 3.9.16
python 1.py 3664ms 19ms 70.4MB 3623ms 23ms pypy 3.9.16
python 1.py timeout 0.0ms 8.4MB 4987ms 0ms cpython 3.11.3
python 2.py timeout 0.0ms 8.6MB 4983ms 0ms cpython 3.11.3
python 1.py timeout 0.0ms 8.3MB 4987ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 7.9MB 4980ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 59ms 0.7ms 2.8MB 50ms 0ms ocaml 5.0.0
python 2.py 279ms 3.4ms 71.2MB 253ms 10ms pypy 3.9.16
python 1.py 425ms 14ms 70.4MB 390ms 17ms pypy 3.9.16
python 1.py 1156ms 2.0ms 8.4MB 1137ms 3ms pyston 3.8.12
python 2.py 1597ms 31ms 7.9MB 1577ms 3ms pyston 3.8.12
python 1.py 3179ms 60ms 8.4MB 3167ms 3ms cpython 3.11.3
python 2.py 3574ms 7.6ms 8.6MB 3557ms 0ms cpython 3.11.3

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 2718ms 31ms 19.1MB 2703ms 0ms ocaml 5.0.0
ocaml 1.ml 3571ms 30ms 550.6MB 3400ms 157ms ocaml 5.0.0
python 1.py timeout 0.0ms 321.0MB 4897ms 80ms cpython 3.11.3
python 2.py timeout 0.0ms 633.5MB 4677ms 307ms cpython 3.11.3
python 1.py timeout 0.0ms 382.8MB 4850ms 133ms pypy 3.9.16
python 2.py timeout 0.0ms 991.9MB 4623ms 360ms pypy 3.9.16
python 1.py timeout 0.0ms 320.4MB 4903ms 80ms pyston 3.8.12
python 2.py timeout 0.0ms 633.3MB 4557ms 423ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 640ms 4.4ms 9.3MB 627ms 0ms ocaml 5.0.0
ocaml 1.ml 763ms 24ms 140.2MB 713ms 30ms ocaml 5.0.0
python 1.py 1331ms 14ms 207.2MB 1257ms 60ms pypy 3.9.16
python 2.py 1563ms 21ms 164.4MB 1463ms 83ms pyston 3.8.12
python 2.py 2088ms 6.3ms 164.9MB 1990ms 83ms cpython 3.11.3
python 2.py 2161ms 4.7ms 210.5MB 2013ms 133ms pypy 3.9.16
python 1.py 3370ms 74ms 86.0MB 3313ms 37ms pyston 3.8.12
python 1.py 4719ms 66ms 86.6MB 4660ms 40ms cpython 3.11.3

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 3388ms 38ms 273.2MB 3253ms 117ms pypy 3.9.16
ocaml 2.ml timeout 0.0ms 181.9MB 4887ms 90ms ocaml 5.0.0
python 1.py timeout 0.0ms 108.2MB 4930ms 57ms cpython 3.11.3
python 1.py timeout 0.0ms 107.6MB 4930ms 53ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 475ms 3.2ms 94.3MB 430ms 27ms pypy 3.9.16
ocaml 2.ml 715ms 2.6ms 52.5MB 673ms 27ms ocaml 5.0.0
python 1.py 774ms 1.5ms 30.6MB 743ms 13ms pyston 3.8.12
python 1.py 821ms 4.6ms 31.9MB 783ms 23ms cpython 3.11.3

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml timeout 0.0ms 4.2MB 4987ms 0ms ocaml 5.0.0
python 8.py timeout 0.0ms 16.3MB 50ms 10ms cpython 3.11.3
python 8.py timeout 0.0ms 99.0MB 657ms 100ms pypy 3.9.16
python 8.py timeout 0.0ms 16.3MB 70ms 20ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 1540ms 14ms 3.5MB 1527ms 0ms ocaml 5.0.0
python 8.py timeout 0.0ms 16.2MB 97ms 17ms cpython 3.11.3
python 8.py timeout 0.0ms 102.4MB 1130ms 193ms pypy 3.9.16
python 8.py timeout 0.0ms 17.6MB 167ms 27ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 394ms 0.4ms 3.3MB 380ms 0ms ocaml 5.0.0
python 8-m.py 4105ms 62ms 103.1MB 7120ms 430ms pypy 3.9.16
python 8.py timeout 0.0ms 15.9MB 230ms 23ms cpython 3.11.3
python 8.py timeout 0.0ms 16.7MB 310ms 70ms pyston 3.8.12