Programming Language and compiler
Benchmarks

Python VS Acton benchmarks

Current benchmark data was generated on Wed Mar 22 2023, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1845ms 31ms 272.0MB 1717ms 110ms pypy 3.9.16
python 1.py timeout 0.0ms 40.9MB 4970ms 10ms cpython 3.11.2
python 1.py timeout 0.0ms 40.3MB 4903ms 73ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 141ms 17ms 80.6MB 107ms 7ms pypy 3.9.16
python 1.py 611ms 1.9ms 11.9MB 583ms 7ms pyston 3.8.12
python 1.py 978ms 6.1ms 12.6MB 963ms 0ms cpython 3.11.2

coro-prime-sieve

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1253ms 5.3ms 193.1MB 1167ms 67ms pypy 3.9.16
python 1.py timeout 0.0ms 20.0MB 4980ms 3ms cpython 3.11.2
python 1.py timeout 0.0ms 17.0MB 4973ms 10ms pyston 3.8.12

Input: 1000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 277ms 8.3ms 92.7MB 243ms 17ms pypy 3.9.16
python 1.py 823ms 17ms 20.1MB 800ms 0ms cpython 3.11.2
python 1.py 932ms 1.9ms 16.9MB 910ms 0ms pyston 3.8.12

edigits

Input: 250001

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
acton 1.act 539ms 20ms 7.4MB 490ms 20ms actonc 0.14.2
python 1.py 699ms 1.8ms 81.7MB 670ms 10ms pypy 3.9.16
python 1.py 1660ms 0.6ms 10.2MB 1643ms 3ms cpython 3.11.2
python 1.py 2787ms 1.4ms 9.7MB 2770ms 3ms pyston 3.8.12

Input: 100000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 180ms 1.3ms 78.0MB 153ms 10ms pypy 3.9.16
acton 1.act 235ms 21ms 5.3MB 195ms 0ms actonc 0.14.2
python 1.py 302ms 0.4ms 9.3MB 287ms 0ms cpython 3.11.2
python 1.py 482ms 2.2ms 9.0MB 463ms 0ms pyston 3.8.12

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2863ms 33ms 88.2MB 2830ms 17ms pypy 3.9.16
python 5-m.py 4003ms 131ms 12.6MB 6103ms 1033ms pyston 3.8.12
python 5-m.py 4052ms 20ms 82.6MB 5697ms 1163ms pypy 3.9.16
python 1.py 4089ms 4.6ms 7.9MB 4077ms 0ms pyston 3.8.12
python 5-m.py 4268ms 10ms 13.6MB 5963ms 990ms cpython 3.11.2
python 1.py timeout 0.0ms 8.6MB 4980ms 0ms cpython 3.11.2

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 360ms 3.2ms 79.1MB 333ms 10ms pypy 3.9.16
python 1.py 423ms 0.8ms 7.9MB 407ms 0ms pyston 3.8.12
python 5-m.py 474ms 16ms 12.7MB 673ms 147ms pyston 3.8.12
python 5-m.py 498ms 1.4ms 13.6MB 633ms 137ms cpython 3.11.2
python 1.py 617ms 5.0ms 8.6MB 597ms 0ms cpython 3.11.2
python 5-m.py 811ms 26ms 82.6MB 880ms 480ms pypy 3.9.16

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
acton 1.act 8.0ms 3.1ms 3.4MB 0ms 0ms actonc 0.14.2
python 1.py 14ms 0.8ms 8.5MB 0ms 0ms cpython 3.11.2
python 1.py 15ms 2.2ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 32ms 0.5ms 52.8MB 8ms 12ms pypy 3.9.16

http-server

Input: 3000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1500ms 13ms 33.6MB 1147ms 763ms pyston 3.8.12
python 1.py 1872ms 11ms 29.6MB 1447ms 873ms cpython 3.11.2
python 1.py 4103ms 42ms 128.3MB 3167ms 783ms pypy 3.9.16

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 352ms 3.3ms 29.8MB 263ms 140ms pyston 3.8.12
python 1.py 390ms 13ms 24.7MB 303ms 137ms cpython 3.11.2
python 1.py 1015ms 14ms 101.5MB 800ms 170ms pypy 3.9.16

json-serde

Input: sample 5000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 255ms 2.1ms 38.4MB 223ms 20ms pyston 3.8.12
python 1.py 260ms 1.2ms 41.1MB 230ms 17ms cpython 3.11.2
python 1.py 387ms 0.5ms 104.4MB 347ms 27ms pypy 3.9.16

Input: canada 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2371ms 12ms 220.6MB 2223ms 130ms pyston 3.8.12
python 1.py 2382ms 8.7ms 223.1MB 2260ms 107ms cpython 3.11.2
python 1.py 2464ms 17ms 244.4MB 2377ms 77ms pypy 3.9.16

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 3-m.py 3683ms 60ms 202.0MB 6537ms 313ms pypy 3.9.16
python 3.py timeout 0.0ms 35.5MB 83ms 20ms cpython 3.11.2
python 3.py timeout 0.0ms 33.8MB 80ms 10ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 3-m.py 544ms 2.5ms 94.6MB 780ms 127ms pypy 3.9.16
python 3-m.py 645ms 11ms 17.8MB 1127ms 40ms pyston 3.8.12
python 3-m.py 909ms 10ms 16.9MB 1647ms 33ms cpython 3.11.2

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 197ms 2.7ms 77.6MB 170ms 10ms pypy 3.9.16
python 1.py 238ms 0.4ms 78.3MB 220ms 7ms pypy 3.9.16
python 1.py 976ms 2.4ms 9.1MB 960ms 0ms pyston 3.8.12
python 2.py 1153ms 3.8ms 8.7MB 1137ms 0ms pyston 3.8.12
python 1.py 1318ms 14ms 9.4MB 1307ms 0ms cpython 3.11.2
python 2.py 1492ms 21ms 9.1MB 1477ms 0ms cpython 3.11.2

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 467ms 2.4ms 77.3MB 433ms 17ms pypy 3.9.16
python 1.py 507ms 1.1ms 78.6MB 480ms 10ms pypy 3.9.16
python 1.py 2915ms 26ms 9.1MB 2900ms 0ms pyston 3.8.12
python 2.py 3580ms 267ms 8.7MB 3567ms 0ms pyston 3.8.12
python 1.py 3919ms 9.3ms 9.4MB 3897ms 0ms cpython 3.11.2
python 2.py 4476ms 51ms 9.1MB 4460ms 0ms cpython 3.11.2

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 141ms 0.2ms 77.2MB 110ms 17ms pypy 3.9.16
python 1.py 160ms 0.5ms 78.4MB 133ms 10ms pypy 3.9.16
python 1.py 474ms 1.5ms 8.7MB 457ms 0ms pyston 3.8.12
python 2.py 536ms 1.5ms 8.3MB 520ms 0ms pyston 3.8.12
python 1.py 633ms 6.4ms 8.7MB 620ms 0ms cpython 3.11.2
python 2.py 748ms 6.3ms 8.4MB 737ms 0ms cpython 3.11.2

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 2821ms 91ms 291.8MB 2653ms 150ms pypy 3.9.16
python 1.py timeout 0.0ms 93.0MB 4953ms 23ms cpython 3.11.2
python 1.py timeout 0.0ms 130.3MB 4937ms 40ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 360ms 2.7ms 138.4MB 307ms 37ms pypy 3.9.16
python 1.py 4964ms 25ms 37.4MB 4927ms 13ms cpython 3.11.2
python 1.py timeout 0.0ms 41.4MB 4970ms 13ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 2360ms 24ms 77.2MB 2330ms 13ms pypy 3.9.16
python 1.py 3530ms 68ms 76.3MB 3503ms 10ms pypy 3.9.16
python 1.py timeout 0.0ms 8.5MB 4983ms 0ms cpython 3.11.2
python 2.py timeout 0.0ms 8.6MB 4987ms 0ms cpython 3.11.2
python 1.py timeout 0.0ms 8.3MB 4983ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 8.0MB 4987ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 2.py 288ms 7.8ms 77.1MB 263ms 10ms pypy 3.9.16
python 1.py 395ms 4.4ms 76.5MB 360ms 17ms pypy 3.9.16
python 1.py 1272ms 110ms 8.4MB 1257ms 3ms pyston 3.8.12
python 2.py 1600ms 1.1ms 7.9MB 1590ms 0ms pyston 3.8.12
python 2.py 3294ms 5.4ms 8.6MB 3280ms 0ms cpython 3.11.2
python 1.py 3542ms 392ms 8.4MB 3530ms 0ms cpython 3.11.2

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 4146ms 26ms 388.8MB 4003ms 123ms pypy 3.9.16
python 1.py timeout 0.0ms 321.1MB 4927ms 60ms cpython 3.11.2
python 2.py timeout 0.0ms 633.6MB 4703ms 277ms cpython 3.11.2
python 2.py timeout 0.0ms 1013.7MB 4707ms 270ms pypy 3.9.16
python 1.py timeout 0.0ms 320.5MB 4897ms 83ms pyston 3.8.12
python 2.py timeout 0.0ms 633.1MB 4643ms 337ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 993ms 16ms 213.2MB 940ms 37ms pypy 3.9.16
python 2.py 1218ms 4.4ms 164.4MB 1137ms 60ms pyston 3.8.12
python 2.py 1564ms 8.5ms 266.7MB 1433ms 113ms pypy 3.9.16
python 2.py 1726ms 13ms 164.9MB 1650ms 60ms cpython 3.11.2
python 1.py 2009ms 31ms 86.0MB 1970ms 23ms pyston 3.8.12
python 1.py 3489ms 5.9ms 86.7MB 3447ms 27ms cpython 3.11.2

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 4.py 2781ms 1.0ms 9.1MB 2743ms 23ms pyston 3.8.12
python 4.py 2784ms 1.0ms 9.4MB 2743ms 23ms cpython 3.11.2
python 4.py 2854ms 4.7ms 117.5MB 2800ms 37ms pypy 3.9.16
acton 1.act 4313ms 37ms 7.9MB 3970ms 700ms actonc 0.14.2

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 4.py 659ms 2.7ms 8.8MB 643ms 0ms pyston 3.8.12
python 4.py 660ms 1.0ms 9.1MB 647ms 0ms cpython 3.11.2
python 4.py 704ms 4.0ms 81.6MB 677ms 10ms pypy 3.9.16
acton 1.act 1394ms 250ms 6.8MB 1220ms 253ms actonc 0.14.2

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 3373ms 4.9ms 256.9MB 3267ms 90ms pypy 3.9.16
python 1.py timeout 0.0ms 107.7MB 4940ms 40ms cpython 3.11.2
python 1.py timeout 0.0ms 107.8MB 4937ms 50ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 467ms 1.9ms 101.6MB 423ms 27ms pypy 3.9.16
python 1.py 701ms 1.2ms 30.5MB 660ms 20ms pyston 3.8.12
python 1.py 835ms 1.6ms 32.1MB 797ms 20ms cpython 3.11.2

secp256k1

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 4456ms 20ms 8.5MB 4443ms 0ms pyston 3.8.12
python 1.py 4694ms 2.8ms 8.5MB 4680ms 0ms cpython 3.11.2
python 1.py 4882ms 6.5ms 84.2MB 4850ms 17ms pypy 3.9.16

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1126ms 1.2ms 8.5MB 1107ms 3ms pyston 3.8.12
python 1.py 1179ms 1.4ms 8.5MB 1167ms 0ms cpython 3.11.2
python 1.py 1368ms 4.0ms 82.0MB 1340ms 10ms pypy 3.9.16

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 8.py timeout 0.0ms 16.3MB 50ms 7ms cpython 3.11.2
python 8.py timeout 0.0ms 104.6MB 687ms 87ms pypy 3.9.16
python 8.py timeout 0.0ms 16.3MB 73ms 13ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 8.py timeout 0.0ms 16.1MB 93ms 27ms cpython 3.11.2
python 8.py timeout 0.0ms 111.5MB 1170ms 180ms pypy 3.9.16
python 8.py timeout 0.0ms 17.6MB 157ms 20ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 8-m.py 3211ms 20ms 105.8MB 5503ms 343ms pypy 3.9.16
python 8.py timeout 0.0ms 14.2MB 263ms 23ms cpython 3.11.2
python 8.py timeout 0.0ms 17.3MB 317ms 57ms pyston 3.8.12