Programming Language and compiler
Benchmarks

Julia VS Python benchmarks

Current benchmark data was generated on Thu Sep 22 2022, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 13ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 15ms 0.3ms 7.3MB 10ms 0ms cpython 3.10.7
python 1.py 34ms 2.4ms 54.0MB 14ms 6ms pypy 3.8.13
julia 1.jl 197ms 0.7ms 187.2MB 124ms 86ms julia 1.8.1
julia 1.jl 205ms 48ms 178.5MB 114ms 86ms julia/aot 1.8.1

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 7.jl 554ms 1.2ms 180.3MB 510ms 120ms julia/aot 1.8.1
julia 7.jl 715ms 1.2ms 245.9MB 680ms 110ms julia 1.8.1
python 2.py 2065ms 46ms 80.0MB 2033ms 13ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 269ms 5.8ms 79.6MB 237ms 13ms pypy 3.8.13
julia 7.jl 285ms 60ms 182.1MB 203ms 127ms julia/aot 1.8.1
julia 7.jl 405ms 0.7ms 245.8MB 343ms 143ms julia 1.8.1
python 1.py 583ms 7.9ms 79.4MB 560ms 3ms pypy 3.8.13
python 1.py 1184ms 11ms 7.2MB 1167ms 0ms pyston 3.8.12
python 2.py 1613ms 7.5ms 8.1MB 1600ms 0ms pyston 3.8.12
python 1.py 4703ms 56ms 7.4MB 4687ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 1.jl 427ms 1.7ms 175.5MB 410ms 97ms julia/aot 1.8.1
julia 1.jl 538ms 1.0ms 220.4MB 510ms 110ms julia 1.8.1
python 1.py 4008ms 8.7ms 391.4MB 3873ms 117ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 1.jl 239ms 30ms 169.1MB 187ms 87ms julia/aot 1.8.1
julia 1.jl 336ms 1.0ms 211.2MB 310ms 107ms julia 1.8.1
python 1.py 1030ms 33ms 215.7MB 967ms 40ms pypy 3.8.13
python 1.py 1868ms 12ms 85.9MB 1830ms 17ms pyston 3.8.12
python 1.py 4326ms 5.5ms 85.9MB 4293ms 20ms cpython 3.10.7

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 4.jl 1155ms 55ms 258.8MB 1073ms 157ms julia 1.8.1
python 1.py 1574ms 2.7ms 274.1MB 1453ms 100ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 120ms 2.4ms 83.3MB 93ms 10ms pypy 3.8.13
julia 4.jl 307ms 1.6ms 240.5MB 250ms 133ms julia 1.8.1
python 1.py 610ms 3.8ms 11.9MB 583ms 10ms pyston 3.8.12
python 1.py 1514ms 8.6ms 11.8MB 1493ms 3ms cpython 3.10.7

fannkuch-redux

Input: 11

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 1825ms 22ms 230.2MB 3183ms 130ms julia 1.8.1

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 555ms 1.4ms 230.5MB 700ms 133ms julia 1.8.1

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 7-m.jl 636ms 1.4ms 299.2MB 890ms 133ms julia 1.8.1
python 1.py 2607ms 3.7ms 91.1MB 2573ms 13ms pypy 3.8.13
python 5-m.py 3750ms 328ms 12.6MB 5597ms 1000ms pyston 3.8.12
python 5-m.py 3964ms 11ms 85.4MB 4430ms 853ms pypy 3.8.13
python 1.py 4082ms 19ms 7.8MB 4070ms 0ms pyston 3.8.12
python 5-m.py 4416ms 24ms 12.1MB 6350ms 1030ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 346ms 2.7ms 81.7MB 307ms 20ms pypy 3.8.13
python 1.py 424ms 5.3ms 7.9MB 410ms 0ms pyston 3.8.12
python 5-m.py 449ms 34ms 12.6MB 640ms 120ms pyston 3.8.12
julia 7.jl 460ms 1.4ms 228.1MB 557ms 113ms julia 1.8.1
python 5-m.py 513ms 1.8ms 12.1MB 660ms 147ms cpython 3.10.7
python 1.py 708ms 1.5ms 7.8MB 700ms 0ms cpython 3.10.7
python 5-m.py 821ms 4.5ms 85.2MB 950ms 420ms pypy 3.8.13

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 8.jl 1162ms 1.6ms 318.3MB 1493ms 167ms julia 1.8.1
python 3-m.py 3577ms 8.2ms 203.7MB 6277ms 330ms pypy 3.8.13
python 3.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 3.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 3-m.py 533ms 8.0ms 96.8MB 753ms 123ms pypy 3.8.13
python 3-m.py 594ms 7.7ms 14.8MB 1040ms 37ms pyston 3.8.12
julia 8.jl 817ms 3.9ms 303.8MB 927ms 123ms julia 1.8.1
python 3-m.py 864ms 11ms 15.3MB 1557ms 30ms cpython 3.10.7

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 3.jl 793ms 3.9ms 196.0MB 767ms 100ms julia 1.8.1
julia 1.jl 1425ms 3.7ms 276.0MB 1337ms 170ms julia 1.8.1
python 4.py 2726ms 13ms 119.6MB 2670ms 37ms pypy 3.8.13
python 4.py 2759ms 0.9ms 9.1MB 2723ms 17ms pyston 3.8.12
python 4.py 2767ms 0.5ms 8.6MB 2737ms 17ms cpython 3.10.7

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 3.jl 349ms 1.4ms 193.8MB 310ms 120ms julia 1.8.1
python 4.py 653ms 0.8ms 8.8MB 640ms 0ms pyston 3.8.12
python 4.py 660ms 0.3ms 8.3MB 647ms 0ms cpython 3.10.7
python 4.py 684ms 5.3ms 84.5MB 657ms 10ms pypy 3.8.13
julia 1.jl 801ms 26ms 277.4MB 707ms 157ms julia 1.8.1

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 1373ms 2.1ms 205.1MB 2407ms 117ms julia 1.8.1
julia 3-m.jl 2069ms 3.9ms 220.3MB 3703ms 130ms julia 1.8.1
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 582ms 1.2ms 206.8MB 873ms 103ms julia 1.8.1
julia 3-m.jl 795ms 4.6ms 219.1MB 1213ms 120ms julia 1.8.1
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 385ms 1.0ms 204.8MB 477ms 107ms julia 1.8.1
julia 3.jl 476ms 2.8ms 220.2MB 580ms 130ms julia 1.8.1
python 8-m.py 3292ms 24ms 109.1MB 5633ms 357ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12