Programming Language and compiler
Benchmarks

Kotlin VS Python benchmarks

Current benchmark data was generated on Sun Jan 29 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
kotlin 1.kt 637ms 22ms 413.1MB 650ms 83ms kotlin/jvm 17.0.2
python 1.py 1514ms 15ms 271.7MB 1403ms 93ms pypy 3.9.16
kotlin 1.kt timeout 0.0ms 214.4MB 8513ms 1040ms kotlin/native 1.8.0
python 1.py timeout 0.0ms 40.8MB 4970ms 10ms cpython 3.11.1
python 1.py timeout 0.0ms 40.3MB 4897ms 80ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 123ms 15ms 80.6MB 87ms 13ms pypy 3.9.16
kotlin 1.kt 126ms 1.9ms 109.3MB 110ms 27ms kotlin/jvm 17.0.2
kotlin 1-m.kt 498ms 4.9ms 30.5MB 830ms 97ms kotlin/native 1.8.0
python 1.py 604ms 3.0ms 11.9MB 580ms 10ms pyston 3.8.12
python 1.py 976ms 8.7ms 12.4MB 960ms 0ms cpython 3.11.1

coro-prime-sieve

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1017ms 21ms 193.8MB 943ms 50ms pypy 3.9.16
kotlin 1.kt 2838ms 51ms 154.4MB 3633ms 70ms kotlin/jvm 17.0.2
kotlin 2-m.kt 3112ms 65ms 160.9MB 5930ms 70ms kotlin/jvm 17.0.2
kotlin 1.kt timeout 0.0ms 10.8MB 6237ms 273ms kotlin/native 1.8.0
kotlin 2-m.kt timeout 0.0ms 32.3MB 9277ms 130ms kotlin/native 1.8.0
python 1.py timeout 0.0ms 20.1MB 4980ms 7ms cpython 3.11.1
python 1.py timeout 0.0ms 17.1MB 4983ms 3ms pyston 3.8.12

Input: 1000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 259ms 5.5ms 93.1MB 227ms 20ms pypy 3.9.16
kotlin 1-m.kt 706ms 31ms 100.0MB 1227ms 43ms kotlin/jvm 17.0.2
python 1.py 820ms 6.1ms 20.0MB 800ms 3ms cpython 3.11.1
kotlin 2-m.kt 847ms 31ms 23.0MB 1523ms 23ms kotlin/native 1.8.0
python 1.py 928ms 0.8ms 16.9MB 907ms 3ms pyston 3.8.12
kotlin 2-m.kt 983ms 35ms 100.4MB 1770ms 33ms kotlin/jvm 17.0.2
kotlin 1.kt 1022ms 5.5ms 8.8MB 1277ms 57ms kotlin/native 1.8.0

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
kotlin 1.kt 2.4ms 0.6ms 2.7MB 0ms 0ms kotlin/native 1.8.0
python 1.py 13ms 0.5ms 8.5MB 10ms 0ms cpython 3.11.1
python 1.py 14ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 30ms 0.8ms 52.1MB 14ms 6ms pypy 3.9.16
kotlin 1.kt 68ms 1.8ms 40.9MB 50ms 4ms kotlin/jvm 17.0.2

http-server

Input: 3000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1464ms 6.9ms 35.2MB 1123ms 747ms pyston 3.8.12
python 1.py 1797ms 7.3ms 29.8MB 1427ms 797ms cpython 3.11.1
python 1.py 3854ms 37ms 126.1MB 2977ms 713ms pypy 3.9.16
kotlin 1.kt timeout 0.0ms 251.2MB 8897ms 667ms kotlin/jvm 17.0.2
kotlin 2.kt timeout 0.0ms 311.1MB 8893ms 620ms kotlin/jvm 17.0.2

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 342ms 0.9ms 29.6MB 253ms 140ms pyston 3.8.12
python 1.py 371ms 6.0ms 24.8MB 287ms 140ms cpython 3.11.1
python 1.py 957ms 12ms 102.6MB 750ms 160ms pypy 3.9.16
kotlin 2.kt 2540ms 64ms 201.4MB 2793ms 183ms kotlin/jvm 17.0.2
kotlin 1-m.kt 2622ms 5.8ms 190.0MB 4000ms 290ms kotlin/jvm 17.0.2

json-serde

Input: sample 5000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 250ms 1.0ms 38.4MB 210ms 20ms pyston 3.8.12
python 1.py 255ms 0.7ms 41.3MB 223ms 17ms cpython 3.11.1
python 1.py 378ms 1.5ms 104.5MB 337ms 23ms pypy 3.9.16
kotlin 1-m.kt 1142ms 8.4ms 217.5MB 1933ms 87ms kotlin/jvm 17.0.2

Input: canada 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2323ms 7.0ms 220.7MB 2177ms 130ms pyston 3.8.12
python 1.py 2352ms 28ms 223.2MB 2253ms 83ms cpython 3.11.1
python 1.py 2429ms 1.5ms 244.8MB 2330ms 83ms pypy 3.9.16
kotlin 1-m.kt 4104ms 270ms 1188.0MB 5900ms 297ms kotlin/jvm 17.0.2

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 194ms 1.0ms 77.2MB 160ms 17ms pypy 3.9.16
kotlin 2-m.kt 223ms 7.8ms 113.7MB 313ms 33ms kotlin/jvm 17.0.2
python 1.py 234ms 1.2ms 78.5MB 210ms 10ms pypy 3.9.16
kotlin 1.kt 235ms 8.3ms 109.5MB 287ms 27ms kotlin/jvm 17.0.2
kotlin 2-m.kt 447ms 4.4ms 42.5MB 747ms 3ms kotlin/native 1.8.0
python 1.py 979ms 6.8ms 9.1MB 967ms 0ms pyston 3.8.12
python 2.py 1153ms 2.6ms 8.7MB 1140ms 0ms pyston 3.8.12
python 1.py 1305ms 7.7ms 9.4MB 1290ms 0ms cpython 3.11.1
python 2.py 1490ms 8.2ms 9.1MB 1477ms 0ms cpython 3.11.1
kotlin 1.kt timeout 0.0ms 15.0MB 5000ms 3ms kotlin/native 1.8.0

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
kotlin 2.kt 370ms 3.7ms 113.5MB 463ms 37ms kotlin/jvm 17.0.2
python 2.py 453ms 0.3ms 77.4MB 423ms 13ms pypy 3.9.16
kotlin 1.kt 456ms 3.4ms 132.4MB 513ms 17ms kotlin/jvm 17.0.2
python 1.py 502ms 5.7ms 78.7MB 480ms 7ms pypy 3.9.16
kotlin 2-m.kt 1371ms 27ms 145.0MB 2247ms 63ms kotlin/native 1.8.0
python 1.py 2891ms 4.2ms 9.1MB 2877ms 0ms pyston 3.8.12
python 2.py 3429ms 23ms 8.8MB 3413ms 0ms pyston 3.8.12
python 1.py 3863ms 37ms 9.3MB 3853ms 0ms cpython 3.11.1
python 2.py 4468ms 77ms 9.1MB 4453ms 0ms cpython 3.11.1
kotlin 1.kt timeout 0.0ms 15.0MB 4997ms 3ms kotlin/native 1.8.0

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 137ms 0.4ms 77.3MB 113ms 13ms pypy 3.9.16
python 1.py 158ms 3.4ms 78.2MB 127ms 17ms pypy 3.9.16
kotlin 1.kt 171ms 5.5ms 74.0MB 233ms 13ms kotlin/jvm 17.0.2
kotlin 2-m.kt 182ms 4.2ms 78.2MB 253ms 20ms kotlin/jvm 17.0.2
kotlin 2-m.kt 230ms 8.2ms 32.3MB 390ms 7ms kotlin/native 1.8.0
python 1.py 472ms 0.5ms 8.7MB 460ms 0ms pyston 3.8.12
python 2.py 539ms 3.8ms 8.2MB 527ms 0ms pyston 3.8.12
python 1.py 616ms 2.1ms 8.7MB 607ms 0ms cpython 3.11.1
python 2.py 741ms 11ms 8.4MB 730ms 0ms cpython 3.11.1
kotlin 1.kt timeout 0.0ms 17.1MB 4993ms 7ms kotlin/native 1.8.0

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
kotlin 1.kt 462ms 24ms 560.0MB 447ms 70ms kotlin/jvm 17.0.2
python 1.py 2251ms 26ms 292.0MB 2047ms 183ms pypy 3.9.16
kotlin 1-m.kt 2769ms 141ms 140.6MB 4437ms 550ms kotlin/native 1.8.0
python 1.py timeout 0.0ms 93.1MB 4967ms 13ms cpython 3.11.1
python 1.py timeout 0.0ms 130.3MB 4933ms 50ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
kotlin 1.kt 178ms 1.2ms 221.3MB 157ms 50ms kotlin/jvm 17.0.2
python 1.py 317ms 1.6ms 138.4MB 273ms 27ms pypy 3.9.16
kotlin 1-m.kt 520ms 3.0ms 38.4MB 823ms 90ms kotlin/native 1.8.0
python 1.py 4438ms 27ms 37.1MB 4410ms 10ms cpython 3.11.1
python 1.py timeout 0.0ms 48.9MB 4947ms 33ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
kotlin 1n.kt 558ms 17ms 1.5MB 550ms 0ms kotlin/native 1.8.0
kotlin 1.kt 573ms 5.3ms 42.8MB 577ms 17ms kotlin/jvm 17.0.2
python 2.py 1901ms 13ms 77.5MB 1870ms 13ms pypy 3.9.16
python 1.py 3515ms 41ms 76.6MB 3483ms 13ms pypy 3.9.16
python 1.py timeout 0.0ms 8.5MB 4987ms 0ms cpython 3.11.1
python 2.py timeout 0.0ms 8.6MB 4980ms 0ms cpython 3.11.1
python 1.py timeout 0.0ms 8.3MB 4987ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 8.0MB 4987ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
kotlin 1n.kt 59ms 1.8ms 1.5MB 47ms 0ms kotlin/native 1.8.0
kotlin 1.kt 137ms 3.7ms 42.8MB 143ms 7ms kotlin/jvm 17.0.2
python 2.py 243ms 1.5ms 77.1MB 220ms 7ms pypy 3.9.16
python 1.py 392ms 1.7ms 76.5MB 370ms 10ms pypy 3.9.16
python 1.py 1193ms 2.2ms 8.3MB 1173ms 0ms pyston 3.8.12
python 2.py 1646ms 27ms 7.9MB 1630ms 0ms pyston 3.8.12
python 2.py 3484ms 156ms 8.6MB 3473ms 0ms cpython 3.11.1
python 1.py 3541ms 305ms 8.5MB 3527ms 0ms cpython 3.11.1

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 4.py 2575ms 5.0ms 117.3MB 2523ms 33ms pypy 3.9.16
python 4.py 2775ms 1.4ms 9.1MB 2750ms 10ms pyston 3.8.12
python 4.py 2781ms 1.0ms 9.4MB 2740ms 27ms cpython 3.11.1
kotlin 1.kt timeout 0.0ms 209.1MB 5367ms 70ms kotlin/jvm 17.0.2
kotlin 1n.kt timeout 0.0ms 8.9MB 6317ms 127ms kotlin/native 1.8.0
kotlin 2n.kt timeout 0.0ms 10.5MB 5033ms 87ms kotlin/native 1.8.0

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 4.py 635ms 4.8ms 81.7MB 603ms 17ms pypy 3.9.16
python 4.py 656ms 0.8ms 8.7MB 647ms 0ms pyston 3.8.12
python 4.py 660ms 1.1ms 9.1MB 643ms 0ms cpython 3.11.1
kotlin 1.kt 1451ms 36ms 224.1MB 1807ms 43ms kotlin/jvm 17.0.2
kotlin 1n.kt timeout 0.0ms 14.9MB 6290ms 127ms kotlin/native 1.8.0
kotlin 2n.kt timeout 0.0ms 9.1MB 5033ms 93ms kotlin/native 1.8.0

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 3319ms 5.0ms 262.3MB 3227ms 77ms pypy 3.9.16
kotlin 1.kt timeout 0.0ms 161.4MB 5330ms 47ms kotlin/jvm 17.0.2
kotlin 1n.kt timeout 0.0ms 277.3MB 5660ms 510ms kotlin/native 1.8.0
python 1.py timeout 0.0ms 108.7MB 4937ms 47ms cpython 3.11.1
python 1.py timeout 0.0ms 107.7MB 4950ms 33ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 456ms 1.8ms 99.9MB 420ms 20ms pypy 3.9.16
python 1.py 694ms 1.4ms 30.5MB 667ms 13ms pyston 3.8.12
python 1.py 830ms 4.3ms 32.0MB 797ms 13ms cpython 3.11.1
kotlin 1.kt 1080ms 4.0ms 95.7MB 1457ms 30ms kotlin/jvm 17.0.2
kotlin 1n.kt 4371ms 24ms 65.2MB 5643ms 20ms kotlin/native 1.8.0

secp256k1

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 4443ms 0.8ms 8.4MB 4427ms 0ms pyston 3.8.12
kotlin 1.kt 4484ms 24ms 183.5MB 5367ms 77ms kotlin/jvm 17.0.2
python 1.py 4645ms 18ms 8.4MB 4627ms 0ms cpython 3.11.1
python 1.py 4747ms 35ms 84.1MB 4710ms 17ms pypy 3.9.16

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1124ms 2.4ms 8.4MB 1110ms 0ms pyston 3.8.12
python 1.py 1175ms 4.4ms 8.5MB 1163ms 0ms cpython 3.11.1
python 1.py 1315ms 2.7ms 82.2MB 1283ms 17ms pypy 3.9.16
kotlin 1-m.kt 1366ms 12ms 187.1MB 2070ms 50ms kotlin/jvm 17.0.2