Programming Language and compiler
Benchmarks

OCaml VS Python benchmarks

Current benchmark data was generated on Sun Jan 29 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1514ms 15ms 271.7MB 1403ms 93ms pypy 3.9.16
ocaml 1.ml 4932ms 8.7ms 153.4MB 4857ms 60ms ocaml 5.0.0
python 1.py timeout 0.0ms 40.8MB 4970ms 10ms cpython 3.11.1
python 1.py timeout 0.0ms 40.3MB 4897ms 80ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 123ms 15ms 80.6MB 87ms 13ms pypy 3.9.16
ocaml 1.ml 346ms 2.9ms 22.9MB 330ms 0ms ocaml 5.0.0
python 1.py 604ms 3.0ms 11.9MB 580ms 10ms pyston 3.8.12
python 1.py 976ms 8.7ms 12.4MB 960ms 0ms cpython 3.11.1

fannkuch-redux

Input: 11

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 3965ms 11ms 2.8MB 3953ms 0ms ocaml 5.0.0

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 329ms 0.4ms 2.9MB 320ms 0ms ocaml 5.0.0

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 6.ml 422ms 0.6ms 22.7MB 407ms 0ms ocaml 5.0.0
python 1.py 2504ms 14ms 88.0MB 2467ms 17ms pypy 3.9.16
python 5-m.py 3892ms 25ms 82.5MB 5217ms 1040ms pypy 3.9.16
python 5-m.py 4026ms 85ms 12.8MB 6157ms 910ms pyston 3.8.12
python 1.py 4078ms 6.7ms 8.0MB 4067ms 0ms pyston 3.8.12
python 5-m.py 4121ms 5.2ms 13.6MB 5753ms 987ms cpython 3.11.1
python 1.py timeout 0.0ms 8.6MB 4983ms 3ms cpython 3.11.1

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 6.ml 46ms 0.8ms 4.8MB 40ms 0ms ocaml 5.0.0
python 1.py 319ms 9.7ms 79.2MB 287ms 20ms pypy 3.9.16
python 1.py 422ms 1.4ms 7.9MB 410ms 0ms pyston 3.8.12
python 5-m.py 469ms 10ms 12.7MB 687ms 123ms pyston 3.8.12
python 5-m.py 477ms 7.3ms 13.6MB 623ms 113ms cpython 3.11.1
python 1.py 600ms 2.2ms 8.6MB 587ms 0ms cpython 3.11.1
python 5-m.py 759ms 7.0ms 82.5MB 653ms 390ms pypy 3.9.16

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 2.5ms 0.8ms 2.8MB 0ms 0ms ocaml 5.0.0
python 1.py 13ms 0.5ms 8.5MB 10ms 0ms cpython 3.11.1
python 1.py 14ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 30ms 0.8ms 52.1MB 14ms 6ms pypy 3.9.16

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 194ms 1.0ms 77.2MB 160ms 17ms pypy 3.9.16
python 1.py 234ms 1.2ms 78.5MB 210ms 10ms pypy 3.9.16
ocaml 1.ml 261ms 2.5ms 6.8MB 250ms 0ms ocaml 5.0.0
python 1.py 979ms 6.8ms 9.1MB 967ms 0ms pyston 3.8.12
python 2.py 1153ms 2.6ms 8.7MB 1140ms 0ms pyston 3.8.12
python 1.py 1305ms 7.7ms 9.4MB 1290ms 0ms cpython 3.11.1
python 2.py 1490ms 8.2ms 9.1MB 1477ms 0ms cpython 3.11.1

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 453ms 0.3ms 77.4MB 423ms 13ms pypy 3.9.16
python 1.py 502ms 5.7ms 78.7MB 480ms 7ms pypy 3.9.16
ocaml 1.ml 768ms 0.5ms 6.8MB 760ms 0ms ocaml 5.0.0
python 1.py 2891ms 4.2ms 9.1MB 2877ms 0ms pyston 3.8.12
python 2.py 3429ms 23ms 8.8MB 3413ms 0ms pyston 3.8.12
python 1.py 3863ms 37ms 9.3MB 3853ms 0ms cpython 3.11.1
python 2.py 4468ms 77ms 9.1MB 4453ms 0ms cpython 3.11.1

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 116ms 1.2ms 6.0MB 110ms 0ms ocaml 5.0.0
python 2.py 137ms 0.4ms 77.3MB 113ms 13ms pypy 3.9.16
python 1.py 158ms 3.4ms 78.2MB 127ms 17ms pypy 3.9.16
python 1.py 472ms 0.5ms 8.7MB 460ms 0ms pyston 3.8.12
python 2.py 539ms 3.8ms 8.2MB 527ms 0ms pyston 3.8.12
python 1.py 616ms 2.1ms 8.7MB 607ms 0ms cpython 3.11.1
python 2.py 741ms 11ms 8.4MB 730ms 0ms cpython 3.11.1

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 1554ms 4.5ms 69.1MB 1510ms 23ms ocaml 5.0.0
python 1.py 2251ms 26ms 292.0MB 2047ms 183ms pypy 3.9.16
ocaml 2.ml 3269ms 59ms 74.3MB 3220ms 37ms ocaml 5.0.0
python 1.py timeout 0.0ms 93.1MB 4967ms 13ms cpython 3.11.1
python 1.py timeout 0.0ms 130.3MB 4933ms 50ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 229ms 1.2ms 25.6MB 213ms 7ms ocaml 5.0.0
python 1.py 317ms 1.6ms 138.4MB 273ms 27ms pypy 3.9.16
ocaml 2.ml 579ms 2.5ms 30.1MB 557ms 0ms ocaml 5.0.0
python 1.py 4438ms 27ms 37.1MB 4410ms 10ms cpython 3.11.1
python 1.py timeout 0.0ms 48.9MB 4947ms 33ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 469ms 0.4ms 2.9MB 460ms 0ms ocaml 5.0.0
python 2.py 1901ms 13ms 77.5MB 1870ms 13ms pypy 3.9.16
python 1.py 3515ms 41ms 76.6MB 3483ms 13ms pypy 3.9.16
python 1.py timeout 0.0ms 8.5MB 4987ms 0ms cpython 3.11.1
python 2.py timeout 0.0ms 8.6MB 4980ms 0ms cpython 3.11.1
python 1.py timeout 0.0ms 8.3MB 4987ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 8.0MB 4987ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 50ms 0.3ms 3.5MB 40ms 0ms ocaml 5.0.0
python 2.py 243ms 1.5ms 77.1MB 220ms 7ms pypy 3.9.16
python 1.py 392ms 1.7ms 76.5MB 370ms 10ms pypy 3.9.16
python 1.py 1193ms 2.2ms 8.3MB 1173ms 0ms pyston 3.8.12
python 2.py 1646ms 27ms 7.9MB 1630ms 0ms pyston 3.8.12
python 2.py 3484ms 156ms 8.6MB 3473ms 0ms cpython 3.11.1
python 1.py 3541ms 305ms 8.5MB 3527ms 0ms cpython 3.11.1

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 1997ms 3.1ms 550.5MB 1863ms 113ms ocaml 5.0.0
ocaml 2.ml 2386ms 11ms 19.0MB 2367ms 3ms ocaml 5.0.0
python 1.py 3855ms 9.2ms 388.9MB 3733ms 110ms pypy 3.9.16
python 2.py 4940ms 25ms 633.1MB 4617ms 307ms pyston 3.8.12
python 1.py timeout 0.0ms 321.1MB 4930ms 50ms cpython 3.11.1
python 2.py timeout 0.0ms 633.6MB 4710ms 270ms cpython 3.11.1
python 2.py timeout 0.0ms 1013.9MB 4687ms 290ms pypy 3.9.16
python 1.py timeout 0.0ms 320.3MB 4893ms 87ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 387ms 13ms 140.4MB 350ms 27ms ocaml 5.0.0
ocaml 2.ml 579ms 0.8ms 9.1MB 567ms 0ms ocaml 5.0.0
python 1.py 879ms 18ms 213.2MB 833ms 33ms pypy 3.9.16
python 2.py 1150ms 4.6ms 164.3MB 1083ms 53ms pyston 3.8.12
python 2.py 1427ms 11ms 266.8MB 1317ms 93ms pypy 3.9.16
python 2.py 1657ms 5.2ms 164.9MB 1597ms 43ms cpython 3.11.1
python 1.py 1781ms 26ms 86.1MB 1747ms 17ms pyston 3.8.12
python 1.py 2819ms 23ms 86.8MB 2777ms 20ms cpython 3.11.1

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 3319ms 5.0ms 262.3MB 3227ms 77ms pypy 3.9.16
ocaml 2.ml timeout 0.0ms 181.9MB 4923ms 53ms ocaml 5.0.0
python 1.py timeout 0.0ms 108.7MB 4937ms 47ms cpython 3.11.1
python 1.py timeout 0.0ms 107.7MB 4950ms 33ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 456ms 1.8ms 99.9MB 420ms 20ms pypy 3.9.16
python 1.py 694ms 1.4ms 30.5MB 667ms 13ms pyston 3.8.12
ocaml 2.ml 719ms 5.1ms 52.5MB 677ms 20ms ocaml 5.0.0
python 1.py 830ms 4.3ms 32.0MB 797ms 13ms cpython 3.11.1

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml timeout 0.0ms 4.1MB 4987ms 0ms ocaml 5.0.0
python 8.py timeout 0.0ms 16.3MB 60ms 3ms cpython 3.11.1
python 8.py timeout 0.0ms 104.5MB 650ms 90ms pypy 3.9.16
python 8.py timeout 0.0ms 17.6MB 67ms 13ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 1633ms 0.1ms 3.6MB 1623ms 0ms ocaml 5.0.0
python 8.py timeout 0.0ms 16.1MB 113ms 17ms cpython 3.11.1
python 8.py timeout 0.0ms 115.0MB 1113ms 200ms pypy 3.9.16
python 8.py timeout 0.0ms 17.5MB 167ms 30ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 411ms 0.7ms 3.3MB 400ms 0ms ocaml 5.0.0
python 8-m.py 3099ms 18ms 105.8MB 5327ms 327ms pypy 3.9.16
python 8.py timeout 0.0ms 15.8MB 257ms 23ms cpython 3.11.1
python 8.py timeout 0.0ms 17.1MB 353ms 47ms pyston 3.8.12