Python VS Codon benchmarks

Current benchmark data was generated on Sun Jan 29 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1514ms 15ms 271.7MB 1403ms 93ms pypy 3.9.16
python 1.py timeout 0.0ms 40.8MB 4970ms 10ms cpython 3.11.1
python 1.py timeout 0.0ms 40.3MB 4897ms 80ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 123ms 15ms 80.6MB 87ms 13ms pypy 3.9.16
python 1.py 604ms 3.0ms 11.9MB 580ms 10ms pyston 3.8.12
python 1.py 976ms 8.7ms 12.4MB 960ms 0ms cpython 3.11.1

coro-prime-sieve

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1017ms 21ms 193.8MB 943ms 50ms pypy 3.9.16
python 1.py timeout 0.0ms 20.1MB 4980ms 7ms cpython 3.11.1
python 1.py timeout 0.0ms 17.1MB 4983ms 3ms pyston 3.8.12

Input: 1000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 259ms 5.5ms 93.1MB 227ms 20ms pypy 3.9.16
python 1.py 820ms 6.1ms 20.0MB 800ms 3ms cpython 3.11.1
python 1.py 928ms 0.8ms 16.9MB 907ms 3ms pyston 3.8.12

edigits

Input: 250001

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 691ms 3.5ms 81.8MB 667ms 3ms pypy 3.9.16
python 1.py 1657ms 0.4ms 10.1MB 1647ms 0ms cpython 3.11.1
python 1.py 2786ms 0.9ms 9.8MB 2773ms 0ms pyston 3.8.12

Input: 100000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 175ms 0.5ms 77.9MB 147ms 13ms pypy 3.9.16
python 1.py 303ms 3.7ms 9.4MB 287ms 0ms cpython 3.11.1
python 1.py 480ms 1.1ms 8.9MB 467ms 0ms pyston 3.8.12

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2504ms 14ms 88.0MB 2467ms 17ms pypy 3.9.16
python 5-m.py 3892ms 25ms 82.5MB 5217ms 1040ms pypy 3.9.16
python 5-m.py 4026ms 85ms 12.8MB 6157ms 910ms pyston 3.8.12
python 1.py 4078ms 6.7ms 8.0MB 4067ms 0ms pyston 3.8.12
python 5-m.py 4121ms 5.2ms 13.6MB 5753ms 987ms cpython 3.11.1
python 1.py timeout 0.0ms 8.6MB 4983ms 3ms cpython 3.11.1

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 319ms 9.7ms 79.2MB 287ms 20ms pypy 3.9.16
python 1.py 422ms 1.4ms 7.9MB 410ms 0ms pyston 3.8.12
python 5-m.py 469ms 10ms 12.7MB 687ms 123ms pyston 3.8.12
python 5-m.py 477ms 7.3ms 13.6MB 623ms 113ms cpython 3.11.1
python 1.py 600ms 2.2ms 8.6MB 587ms 0ms cpython 3.11.1
python 5-m.py 759ms 7.0ms 82.5MB 653ms 390ms pypy 3.9.16

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
codon 1.py 4.1ms 0.8ms 5.3MB 0ms 0ms codon 0.15.4
python 1.py 13ms 0.5ms 8.5MB 10ms 0ms cpython 3.11.1
python 1.py 14ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 30ms 0.8ms 52.1MB 14ms 6ms pypy 3.9.16

http-server

Input: 3000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1464ms 6.9ms 35.2MB 1123ms 747ms pyston 3.8.12
python 1.py 1797ms 7.3ms 29.8MB 1427ms 797ms cpython 3.11.1
python 1.py 3854ms 37ms 126.1MB 2977ms 713ms pypy 3.9.16

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 342ms 0.9ms 29.6MB 253ms 140ms pyston 3.8.12
python 1.py 371ms 6.0ms 24.8MB 287ms 140ms cpython 3.11.1
python 1.py 957ms 12ms 102.6MB 750ms 160ms pypy 3.9.16

json-serde

Input: sample 5000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 250ms 1.0ms 38.4MB 210ms 20ms pyston 3.8.12
python 1.py 255ms 0.7ms 41.3MB 223ms 17ms cpython 3.11.1
python 1.py 378ms 1.5ms 104.5MB 337ms 23ms pypy 3.9.16

Input: canada 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2323ms 7.0ms 220.7MB 2177ms 130ms pyston 3.8.12
python 1.py 2352ms 28ms 223.2MB 2253ms 83ms cpython 3.11.1
python 1.py 2429ms 1.5ms 244.8MB 2330ms 83ms pypy 3.9.16

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 3-m.py 3451ms 24ms 204.9MB 6053ms 317ms pypy 3.9.16
python 3-m.py 4892ms 85ms 34.3MB 9350ms 50ms pyston 3.8.12
python 3.py timeout 0.0ms 35.0MB 90ms 10ms cpython 3.11.1

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 3-m.py 518ms 3.6ms 94.6MB 727ms 123ms pypy 3.9.16
python 3-m.py 579ms 12ms 17.7MB 1013ms 30ms pyston 3.8.12
python 3-m.py 812ms 13ms 16.8MB 1477ms 17ms cpython 3.11.1

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 194ms 1.0ms 77.2MB 160ms 17ms pypy 3.9.16
python 1.py 234ms 1.2ms 78.5MB 210ms 10ms pypy 3.9.16
python 1.py 979ms 6.8ms 9.1MB 967ms 0ms pyston 3.8.12
python 2.py 1153ms 2.6ms 8.7MB 1140ms 0ms pyston 3.8.12
python 1.py 1305ms 7.7ms 9.4MB 1290ms 0ms cpython 3.11.1
python 2.py 1490ms 8.2ms 9.1MB 1477ms 0ms cpython 3.11.1

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 453ms 0.3ms 77.4MB 423ms 13ms pypy 3.9.16
python 1.py 502ms 5.7ms 78.7MB 480ms 7ms pypy 3.9.16
python 1.py 2891ms 4.2ms 9.1MB 2877ms 0ms pyston 3.8.12
python 2.py 3429ms 23ms 8.8MB 3413ms 0ms pyston 3.8.12
python 1.py 3863ms 37ms 9.3MB 3853ms 0ms cpython 3.11.1
python 2.py 4468ms 77ms 9.1MB 4453ms 0ms cpython 3.11.1

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 137ms 0.4ms 77.3MB 113ms 13ms pypy 3.9.16
python 1.py 158ms 3.4ms 78.2MB 127ms 17ms pypy 3.9.16
python 1.py 472ms 0.5ms 8.7MB 460ms 0ms pyston 3.8.12
python 2.py 539ms 3.8ms 8.2MB 527ms 0ms pyston 3.8.12
python 1.py 616ms 2.1ms 8.7MB 607ms 0ms cpython 3.11.1
python 2.py 741ms 11ms 8.4MB 730ms 0ms cpython 3.11.1

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2251ms 26ms 292.0MB 2047ms 183ms pypy 3.9.16
python 1.py timeout 0.0ms 93.1MB 4967ms 13ms cpython 3.11.1
python 1.py timeout 0.0ms 130.3MB 4933ms 50ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 317ms 1.6ms 138.4MB 273ms 27ms pypy 3.9.16
python 1.py 4438ms 27ms 37.1MB 4410ms 10ms cpython 3.11.1
python 1.py timeout 0.0ms 48.9MB 4947ms 33ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 1901ms 13ms 77.5MB 1870ms 13ms pypy 3.9.16
codon 1.py 1947ms 23ms 5.3MB 1933ms 0ms codon 0.15.4
python 1.py 3515ms 41ms 76.6MB 3483ms 13ms pypy 3.9.16
python 1.py timeout 0.0ms 8.5MB 4987ms 0ms cpython 3.11.1
python 2.py timeout 0.0ms 8.6MB 4980ms 0ms cpython 3.11.1
python 1.py timeout 0.0ms 8.3MB 4987ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 8.0MB 4987ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
codon 1.py 197ms 0.8ms 5.3MB 187ms 0ms codon 0.15.4
python 2.py 243ms 1.5ms 77.1MB 220ms 7ms pypy 3.9.16
python 1.py 392ms 1.7ms 76.5MB 370ms 10ms pypy 3.9.16
python 1.py 1193ms 2.2ms 8.3MB 1173ms 0ms pyston 3.8.12
python 2.py 1646ms 27ms 7.9MB 1630ms 0ms pyston 3.8.12
python 2.py 3484ms 156ms 8.6MB 3473ms 0ms cpython 3.11.1
python 1.py 3541ms 305ms 8.5MB 3527ms 0ms cpython 3.11.1

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
codon 1.py 873ms 79ms 48.3MB 850ms 3ms codon 0.15.4
codon 2.py 1003ms 24ms 106.7MB 977ms 13ms codon 0.15.4
python 1.py 3855ms 9.2ms 388.9MB 3733ms 110ms pypy 3.9.16
python 2.py 4940ms 25ms 633.1MB 4617ms 307ms pyston 3.8.12
python 1.py timeout 0.0ms 321.1MB 4930ms 50ms cpython 3.11.1
python 2.py timeout 0.0ms 633.6MB 4710ms 270ms cpython 3.11.1
python 2.py timeout 0.0ms 1013.9MB 4687ms 290ms pypy 3.9.16
python 1.py timeout 0.0ms 320.3MB 4893ms 87ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
codon 1.py 150ms 0.2ms 15.1MB 133ms 3ms codon 0.15.4
codon 2.py 185ms 3.8ms 23.9MB 173ms 3ms codon 0.15.4
python 1.py 879ms 18ms 213.2MB 833ms 33ms pypy 3.9.16
python 2.py 1150ms 4.6ms 164.3MB 1083ms 53ms pyston 3.8.12
python 2.py 1427ms 11ms 266.8MB 1317ms 93ms pypy 3.9.16
python 2.py 1657ms 5.2ms 164.9MB 1597ms 43ms cpython 3.11.1
python 1.py 1781ms 26ms 86.1MB 1747ms 17ms pyston 3.8.12
python 1.py 2819ms 23ms 86.8MB 2777ms 20ms cpython 3.11.1

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 4.py 2575ms 5.0ms 117.3MB 2523ms 33ms pypy 3.9.16
python 4.py 2775ms 1.4ms 9.1MB 2750ms 10ms pyston 3.8.12
python 4.py 2781ms 1.0ms 9.4MB 2740ms 27ms cpython 3.11.1

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 4.py 635ms 4.8ms 81.7MB 603ms 17ms pypy 3.9.16
python 4.py 656ms 0.8ms 8.7MB 647ms 0ms pyston 3.8.12
python 4.py 660ms 1.1ms 9.1MB 643ms 0ms cpython 3.11.1

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 3319ms 5.0ms 262.3MB 3227ms 77ms pypy 3.9.16
python 1.py timeout 0.0ms 108.7MB 4937ms 47ms cpython 3.11.1
python 1.py timeout 0.0ms 107.7MB 4950ms 33ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 456ms 1.8ms 99.9MB 420ms 20ms pypy 3.9.16
python 1.py 694ms 1.4ms 30.5MB 667ms 13ms pyston 3.8.12
python 1.py 830ms 4.3ms 32.0MB 797ms 13ms cpython 3.11.1

secp256k1

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 4443ms 0.8ms 8.4MB 4427ms 0ms pyston 3.8.12
python 1.py 4645ms 18ms 8.4MB 4627ms 0ms cpython 3.11.1
python 1.py 4747ms 35ms 84.1MB 4710ms 17ms pypy 3.9.16

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1124ms 2.4ms 8.4MB 1110ms 0ms pyston 3.8.12
python 1.py 1175ms 4.4ms 8.5MB 1163ms 0ms cpython 3.11.1
python 1.py 1315ms 2.7ms 82.2MB 1283ms 17ms pypy 3.9.16

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 8.py timeout 0.0ms 16.3MB 60ms 3ms cpython 3.11.1
python 8.py timeout 0.0ms 104.5MB 650ms 90ms pypy 3.9.16
python 8.py timeout 0.0ms 17.6MB 67ms 13ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 8.py timeout 0.0ms 16.1MB 113ms 17ms cpython 3.11.1
python 8.py timeout 0.0ms 115.0MB 1113ms 200ms pypy 3.9.16
python 8.py timeout 0.0ms 17.5MB 167ms 30ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 8-m.py 3099ms 18ms 105.8MB 5327ms 327ms pypy 3.9.16
python 8.py timeout 0.0ms 15.8MB 257ms 23ms cpython 3.11.1
python 8.py timeout 0.0ms 17.1MB 353ms 47ms pyston 3.8.12