Programming Language and compiler
Benchmarks

OCaml VS Python benchmarks

Current benchmark data was generated on Thu Jul 13 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1431ms 7.2ms 271.9MB 1300ms 110ms pypy 3.10.12
ocaml 1.ml 4948ms 10ms 153.4MB 4887ms 43ms ocaml 5.0.0
python 1.py timeout 0.0ms 40.9MB 4970ms 10ms cpython 3.11.4
python 1.py timeout 0.0ms 40.3MB 4903ms 80ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 119ms 16ms 80.1MB 83ms 13ms pypy 3.10.12
ocaml 1.ml 341ms 3.5ms 22.9MB 323ms 3ms ocaml 5.0.0
python 1.py 610ms 3.8ms 12.0MB 587ms 7ms pyston 3.8.12
python 1.py 994ms 18ms 12.5MB 973ms 3ms cpython 3.11.4

fannkuch-redux

Input: 11

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 3984ms 14ms 2.8MB 3973ms 0ms ocaml 5.0.0

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 331ms 1.3ms 2.9MB 320ms 0ms ocaml 5.0.0

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 6.ml 425ms 0.3ms 22.7MB 407ms 3ms ocaml 5.0.0
python 1.py 2471ms 36ms 87.9MB 2433ms 20ms pypy 3.10.12
python 5-m.py 3926ms 65ms 83.2MB 4250ms 843ms pypy 3.10.12
python 5-m.py 3978ms 31ms 12.7MB 6003ms 1020ms pyston 3.8.12
python 5-m.py 4105ms 12ms 13.6MB 5733ms 1033ms cpython 3.11.4
python 1.py 4108ms 2.7ms 7.9MB 4093ms 0ms pyston 3.8.12
python 1.py timeout 0.0ms 8.6MB 4980ms 0ms cpython 3.11.4

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 6.ml 47ms 2.4ms 4.9MB 37ms 0ms ocaml 5.0.0
python 1.py 335ms 15ms 78.9MB 307ms 17ms pypy 3.10.12
python 1.py 428ms 4.2ms 8.0MB 407ms 3ms pyston 3.8.12
python 5-m.py 469ms 12ms 12.7MB 693ms 123ms pyston 3.8.12
python 5-m.py 472ms 1.7ms 13.6MB 607ms 133ms cpython 3.11.4
python 1.py 617ms 1.4ms 8.6MB 603ms 0ms cpython 3.11.4
python 5-m.py 769ms 8.0ms 82.7MB 833ms 440ms pypy 3.10.12

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 2.7ms 0.6ms 3.1MB 0ms 0ms ocaml 5.0.0
python 1.py 13ms 0.6ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 14ms 1.4ms 8.5MB 10ms 0ms cpython 3.11.4
python 1.py 30ms 0.7ms 52.1MB 12ms 2ms pypy 3.10.12

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 196ms 0.4ms 77.2MB 170ms 10ms pypy 3.10.12
python 1.py 234ms 0.3ms 78.0MB 217ms 3ms pypy 3.10.12
ocaml 1.ml 264ms 2.8ms 6.7MB 250ms 0ms ocaml 5.0.0
python 1.py 981ms 0.5ms 9.0MB 967ms 0ms pyston 3.8.12
python 2.py 1165ms 1.0ms 8.7MB 1147ms 3ms pyston 3.8.12
python 1.py 1337ms 3.7ms 9.3MB 1323ms 0ms cpython 3.11.4
python 2.py 1548ms 20ms 9.1MB 1533ms 0ms cpython 3.11.4

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 459ms 4.4ms 77.7MB 433ms 10ms pypy 3.10.12
python 1.py 507ms 11ms 78.6MB 477ms 10ms pypy 3.10.12
ocaml 1.ml 781ms 8.1ms 6.7MB 767ms 0ms ocaml 5.0.0
python 1.py 2937ms 37ms 8.9MB 2920ms 0ms pyston 3.8.12
python 2.py 3476ms 17ms 8.7MB 3467ms 0ms pyston 3.8.12
python 1.py 3949ms 14ms 9.4MB 3937ms 0ms cpython 3.11.4
python 2.py 4639ms 19ms 9.1MB 4620ms 3ms cpython 3.11.4

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 118ms 1.6ms 6.0MB 110ms 0ms ocaml 5.0.0
python 2.py 138ms 1.8ms 77.3MB 113ms 10ms pypy 3.10.12
python 1.py 156ms 2.2ms 78.1MB 127ms 13ms pypy 3.10.12
python 1.py 475ms 2.7ms 8.8MB 460ms 0ms pyston 3.8.12
python 2.py 542ms 1.7ms 8.2MB 530ms 0ms pyston 3.8.12
python 1.py 632ms 4.6ms 8.7MB 613ms 0ms cpython 3.11.4
python 2.py 773ms 8.4ms 8.4MB 757ms 3ms cpython 3.11.4

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 1524ms 6.3ms 69.4MB 1483ms 23ms ocaml 5.0.0
python 1.py 2210ms 33ms 292.6MB 2010ms 183ms pypy 3.10.12
ocaml 2.ml 3227ms 33ms 74.1MB 3187ms 23ms ocaml 5.0.0
python 1.py timeout 0.0ms 93.2MB 4963ms 20ms cpython 3.11.4
python 1.py timeout 0.0ms 130.2MB 4930ms 47ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 229ms 7.2ms 25.6MB 213ms 3ms ocaml 5.0.0
python 1.py 310ms 2.7ms 137.0MB 247ms 53ms pypy 3.10.12
ocaml 2.ml 576ms 3.7ms 30.2MB 557ms 3ms ocaml 5.0.0
python 1.py 4349ms 24ms 37.3MB 4320ms 10ms cpython 3.11.4
python 1.py timeout 0.0ms 48.8MB 4947ms 33ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 470ms 0.5ms 2.9MB 460ms 0ms ocaml 5.0.0
python 2.py 1880ms 4.6ms 77.3MB 1853ms 10ms pypy 3.10.12
python 1.py 3370ms 20ms 76.1MB 3337ms 13ms pypy 3.10.12
python 1.py timeout 0.0ms 8.4MB 4983ms 0ms cpython 3.11.4
python 2.py timeout 0.0ms 8.6MB 4983ms 0ms cpython 3.11.4
python 1.py timeout 0.0ms 8.3MB 4980ms 3ms pyston 3.8.12
python 2.py timeout 0.0ms 7.9MB 4980ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 50ms 0.6ms 2.9MB 40ms 0ms ocaml 5.0.0
python 2.py 242ms 1.9ms 77.0MB 217ms 10ms pypy 3.10.12
python 1.py 381ms 1.2ms 76.4MB 360ms 7ms pypy 3.10.12
python 1.py 1202ms 7.3ms 8.4MB 1190ms 0ms pyston 3.8.12
python 2.py 1603ms 2.5ms 7.9MB 1590ms 0ms pyston 3.8.12
python 1.py 3302ms 19ms 8.4MB 3287ms 0ms cpython 3.11.4
python 2.py 3358ms 33ms 8.6MB 3343ms 0ms cpython 3.11.4

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 1999ms 32ms 550.5MB 1893ms 87ms ocaml 5.0.0
ocaml 2.ml 2424ms 16ms 19.0MB 2410ms 0ms ocaml 5.0.0
python 1.py 3962ms 23ms 388.8MB 3823ms 117ms pypy 3.10.12
python 2.py 4945ms 31ms 633.0MB 4633ms 293ms pyston 3.8.12
python 1.py timeout 0.0ms 321.1MB 4937ms 50ms cpython 3.11.4
python 2.py timeout 0.0ms 633.6MB 4703ms 277ms cpython 3.11.4
python 2.py timeout 0.0ms 1013.9MB 4687ms 287ms pypy 3.10.12
python 1.py timeout 0.0ms 320.5MB 4910ms 73ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 1.ml 339ms 11ms 140.3MB 307ms 17ms ocaml 5.0.0
ocaml 2.ml 609ms 28ms 9.1MB 590ms 0ms ocaml 5.0.0
python 1.py 883ms 15ms 213.0MB 840ms 30ms pypy 3.10.12
python 2.py 1136ms 6.5ms 164.4MB 1070ms 50ms pyston 3.8.12
python 2.py 1412ms 11ms 266.7MB 1277ms 117ms pypy 3.10.12
python 1.py 1677ms 20ms 86.1MB 1640ms 23ms pyston 3.8.12
python 2.py 1697ms 29ms 164.8MB 1647ms 37ms cpython 3.11.4
python 1.py 2679ms 11ms 86.8MB 2640ms 20ms cpython 3.11.4

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 3325ms 5.1ms 258.9MB 3233ms 73ms pypy 3.10.12
ocaml 2.ml timeout 0.0ms 181.9MB 4927ms 47ms ocaml 5.0.0
python 1.py timeout 0.0ms 107.1MB 4937ms 43ms cpython 3.11.4
python 1.py timeout 0.0ms 107.2MB 4937ms 47ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 461ms 0.9ms 101.7MB 420ms 20ms pypy 3.10.12
python 1.py 697ms 1.5ms 30.5MB 660ms 17ms pyston 3.8.12
ocaml 2.ml 705ms 2.8ms 52.5MB 683ms 10ms ocaml 5.0.0
python 1.py 818ms 2.0ms 31.7MB 780ms 20ms cpython 3.11.4

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml timeout 0.0ms 4.1MB 4987ms 0ms ocaml 5.0.0
python 8.py timeout 0.0ms 16.6MB 53ms 10ms cpython 3.11.4
python 8.py timeout 0.0ms 106.2MB 663ms 87ms pypy 3.10.12
python 8.py timeout 0.0ms 18.0MB 80ms 13ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 1639ms 5.6ms 3.7MB 1623ms 0ms ocaml 5.0.0
python 8.py timeout 0.0ms 14.4MB 127ms 13ms cpython 3.11.4
python 8.py timeout 0.0ms 111.5MB 1110ms 217ms pypy 3.10.12
python 8.py timeout 0.0ms 15.6MB 140ms 30ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
ocaml 2.ml 412ms 0.4ms 3.3MB 400ms 0ms ocaml 5.0.0
python 8-m.py 3124ms 25ms 105.3MB 5340ms 337ms pypy 3.10.12
python 8.py timeout 0.0ms 16.2MB 263ms 17ms cpython 3.11.4
python 8.py timeout 0.0ms 16.7MB 320ms 57ms pyston 3.8.12