Programming Language and compiler
Benchmarks

Julia VS Python benchmarks

Current benchmark data was generated on Sun Jan 29 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 13ms 0.5ms 8.5MB 10ms 0ms cpython 3.11.1
python 1.py 14ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 30ms 0.8ms 52.1MB 14ms 6ms pypy 3.9.16
julia 1.jl 179ms 0.4ms 163.9MB 114ms 70ms julia/aot 1.8.5
julia 1.jl 194ms 1.8ms 171.2MB 132ms 76ms julia 1.8.5

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 7.jl 549ms 0.1ms 168.6MB 507ms 120ms julia/aot 1.8.5
julia 7.jl 709ms 1.2ms 224.5MB 663ms 120ms julia 1.8.5
python 2.py 1901ms 13ms 77.5MB 1870ms 13ms pypy 3.9.16
python 1.py 3515ms 41ms 76.6MB 3483ms 13ms pypy 3.9.16
python 1.py timeout 0.0ms 8.5MB 4987ms 0ms cpython 3.11.1
python 2.py timeout 0.0ms 8.6MB 4980ms 0ms cpython 3.11.1
python 1.py timeout 0.0ms 8.3MB 4987ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 8.0MB 4987ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 243ms 1.5ms 77.1MB 220ms 7ms pypy 3.9.16
julia 7.jl 246ms 3.3ms 169.8MB 210ms 100ms julia/aot 1.8.5
python 1.py 392ms 1.7ms 76.5MB 370ms 10ms pypy 3.9.16
julia 7.jl 395ms 6.8ms 223.7MB 333ms 133ms julia 1.8.5
python 1.py 1193ms 2.2ms 8.3MB 1173ms 0ms pyston 3.8.12
python 2.py 1646ms 27ms 7.9MB 1630ms 0ms pyston 3.8.12
python 2.py 3484ms 156ms 8.6MB 3473ms 0ms cpython 3.11.1
python 1.py 3541ms 305ms 8.5MB 3527ms 0ms cpython 3.11.1

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 1.jl 423ms 0.6ms 156.3MB 390ms 113ms julia/aot 1.8.5
julia 1.jl 528ms 2.4ms 203.3MB 487ms 113ms julia 1.8.5
python 1.py 3855ms 9.2ms 388.9MB 3733ms 110ms pypy 3.9.16
python 2.py 4940ms 25ms 633.1MB 4617ms 307ms pyston 3.8.12
python 1.py timeout 0.0ms 321.1MB 4930ms 50ms cpython 3.11.1
python 2.py timeout 0.0ms 633.6MB 4710ms 270ms cpython 3.11.1
python 2.py timeout 0.0ms 1013.9MB 4687ms 290ms pypy 3.9.16
python 1.py timeout 0.0ms 320.3MB 4893ms 87ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 1.jl 220ms 2.5ms 150.2MB 177ms 87ms julia/aot 1.8.5
julia 1.jl 326ms 1.7ms 195.1MB 293ms 113ms julia 1.8.5
python 1.py 879ms 18ms 213.2MB 833ms 33ms pypy 3.9.16
python 2.py 1150ms 4.6ms 164.3MB 1083ms 53ms pyston 3.8.12
python 2.py 1427ms 11ms 266.8MB 1317ms 93ms pypy 3.9.16
python 2.py 1657ms 5.2ms 164.9MB 1597ms 43ms cpython 3.11.1
python 1.py 1781ms 26ms 86.1MB 1747ms 17ms pyston 3.8.12
python 1.py 2819ms 23ms 86.8MB 2777ms 20ms cpython 3.11.1

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 4.jl 1112ms 26ms 239.5MB 1023ms 163ms julia 1.8.5
python 1.py 1514ms 15ms 271.7MB 1403ms 93ms pypy 3.9.16
python 1.py timeout 0.0ms 40.8MB 4970ms 10ms cpython 3.11.1
python 1.py timeout 0.0ms 40.3MB 4897ms 80ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 123ms 15ms 80.6MB 87ms 13ms pypy 3.9.16
julia 4.jl 304ms 1.4ms 221.2MB 263ms 120ms julia 1.8.5
python 1.py 604ms 3.0ms 11.9MB 580ms 10ms pyston 3.8.12
python 1.py 976ms 8.7ms 12.4MB 960ms 0ms cpython 3.11.1

fannkuch-redux

Input: 11

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 1775ms 12ms 210.1MB 3120ms 127ms julia 1.8.5

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 527ms 5.9ms 209.8MB 693ms 117ms julia 1.8.5

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 7-m.jl 630ms 2.1ms 291.8MB 860ms 147ms julia 1.8.5
python 1.py 2504ms 14ms 88.0MB 2467ms 17ms pypy 3.9.16
python 5-m.py 3892ms 25ms 82.5MB 5217ms 1040ms pypy 3.9.16
python 5-m.py 4026ms 85ms 12.8MB 6157ms 910ms pyston 3.8.12
python 1.py 4078ms 6.7ms 8.0MB 4067ms 0ms pyston 3.8.12
python 5-m.py 4121ms 5.2ms 13.6MB 5753ms 987ms cpython 3.11.1
python 1.py timeout 0.0ms 8.6MB 4983ms 3ms cpython 3.11.1

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 319ms 9.7ms 79.2MB 287ms 20ms pypy 3.9.16
python 1.py 422ms 1.4ms 7.9MB 410ms 0ms pyston 3.8.12
julia 7-m.jl 451ms 0.5ms 226.0MB 563ms 117ms julia 1.8.5
python 5-m.py 469ms 10ms 12.7MB 687ms 123ms pyston 3.8.12
python 5-m.py 477ms 7.3ms 13.6MB 623ms 113ms cpython 3.11.1
python 1.py 600ms 2.2ms 8.6MB 587ms 0ms cpython 3.11.1
python 5-m.py 759ms 7.0ms 82.5MB 653ms 390ms pypy 3.9.16

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 8.jl 1118ms 9.1ms 273.3MB 1470ms 157ms julia 1.8.5
python 3-m.py 3451ms 24ms 204.9MB 6053ms 317ms pypy 3.9.16
python 3-m.py 4892ms 85ms 34.3MB 9350ms 50ms pyston 3.8.12
python 3.py timeout 0.0ms 35.0MB 90ms 10ms cpython 3.11.1

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 3-m.py 518ms 3.6ms 94.6MB 727ms 123ms pypy 3.9.16
python 3-m.py 579ms 12ms 17.7MB 1013ms 30ms pyston 3.8.12
julia 8.jl 763ms 4.0ms 259.1MB 847ms 143ms julia 1.8.5
python 3-m.py 812ms 13ms 16.8MB 1477ms 17ms cpython 3.11.1

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 3.jl 791ms 4.6ms 176.9MB 760ms 103ms julia 1.8.5
julia 1.jl 1373ms 3.2ms 260.8MB 1303ms 147ms julia 1.8.5
python 4.py 2575ms 5.0ms 117.3MB 2523ms 33ms pypy 3.9.16
python 4.py 2775ms 1.4ms 9.1MB 2750ms 10ms pyston 3.8.12
python 4.py 2781ms 1.0ms 9.4MB 2740ms 27ms cpython 3.11.1

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 3.jl 342ms 0.3ms 176.2MB 307ms 113ms julia 1.8.5
python 4.py 635ms 4.8ms 81.7MB 603ms 17ms pypy 3.9.16
python 4.py 656ms 0.8ms 8.7MB 647ms 0ms pyston 3.8.12
python 4.py 660ms 1.1ms 9.1MB 643ms 0ms cpython 3.11.1
julia 1.jl 780ms 23ms 261.5MB 710ms 133ms julia 1.8.5

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 1370ms 7.3ms 192.9MB 2410ms 103ms julia 1.8.5
julia 3-m.jl 2107ms 0.8ms 211.4MB 3787ms 123ms julia 1.8.5
python 8.py timeout 0.0ms 16.3MB 60ms 3ms cpython 3.11.1
python 8.py timeout 0.0ms 104.5MB 650ms 90ms pypy 3.9.16
python 8.py timeout 0.0ms 17.6MB 67ms 13ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 578ms 0.1ms 192.5MB 853ms 110ms julia 1.8.5
julia 3-m.jl 804ms 1.2ms 210.3MB 1233ms 120ms julia 1.8.5
python 8.py timeout 0.0ms 16.1MB 113ms 17ms cpython 3.11.1
python 8.py timeout 0.0ms 115.0MB 1113ms 200ms pypy 3.9.16
python 8.py timeout 0.0ms 17.5MB 167ms 30ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
julia 2-m.jl 381ms 2.7ms 192.3MB 467ms 113ms julia 1.8.5
julia 3.jl 477ms 5.1ms 211.5MB 577ms 137ms julia 1.8.5
python 8-m.py 3099ms 18ms 105.8MB 5327ms 327ms pypy 3.9.16
python 8.py timeout 0.0ms 15.8MB 257ms 23ms cpython 3.11.1
python 8.py timeout 0.0ms 17.1MB 353ms 47ms pyston 3.8.12