Programming Language and compiler
Benchmarks

Python VS C# benchmarks

Current benchmark data was generated on Thu Sep 22 2022, full log can be found HERE

CONTRIBUTIONS are WELCOME!

[x86_64][2 cores] Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (Model 106)

* -m in a file name stands for multi-threading or multi-processing

* -i in a file name stands for direct intrinsics usage. (Usage of simd intrinsics via libraries is not counted)

* -ffi in a file name stands for non-stdlib FFI usage

* (You may find time < time(user) + time(sys) for some non-parallelized programs, the overhead is from GC or JIT compiler, which are allowed to take advantage of multi-cores as that's more close to real-world scenarios.)

binarytrees

Input: 18

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 1.cs 1354ms 10ms 390.0MB 1237ms 177ms dotnet 6.0.401
python 1.py 1574ms 2.7ms 274.1MB 1453ms 100ms pypy 3.8.13
csharp 1.cs 2186ms 21ms 103.2MB 2480ms 190ms mono 6.12.0
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 120ms 2.4ms 83.3MB 93ms 10ms pypy 3.8.13
csharp 1.cs 151ms 0.2ms 44.4MB 127ms 10ms mono 6.12.0
csharp 1.cs 192ms 5.2ms 95.3MB 140ms 27ms dotnet 6.0.401
python 1.py 610ms 3.8ms 11.9MB 583ms 10ms pyston 3.8.12
python 1.py 1514ms 8.6ms 11.8MB 1493ms 3ms cpython 3.10.7

coro-prime-sieve

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1164ms 15ms 196.7MB 1090ms 53ms pypy 3.8.13
csharp 1-m.cs 4393ms 104ms 106.0MB 8267ms 50ms dotnet 6.0.401
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 1000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 276ms 1.4ms 95.1MB 237ms 17ms pypy 3.8.13
csharp 1-m.cs 430ms 27ms 85.9MB 683ms 27ms dotnet 6.0.401
python 1.py 966ms 3.5ms 16.8MB 947ms 3ms pyston 3.8.12
python 1.py 978ms 1.6ms 16.9MB 960ms 0ms cpython 3.10.7

edigits

Input: 250001

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 828ms 1.6ms 87.8MB 797ms 20ms pypy 3.8.13
python 1.py 1670ms 1.4ms 9.3MB 1657ms 0ms cpython 3.10.7
python 1.py 2829ms 1.6ms 9.6MB 2820ms 0ms pyston 3.8.12
csharp 1.cs timeout 0.0ms 0.0MB 0ms 0ms dotnet 6.0.401

Input: 100000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 213ms 0.8ms 81.9MB 190ms 10ms pypy 3.8.13
python 1.py 307ms 0.2ms 8.5MB 293ms 0ms cpython 3.10.7
python 1.py 488ms 0.9ms 8.8MB 470ms 0ms pyston 3.8.12
csharp 1.cs 1103ms 5.2ms 45.8MB 1063ms 13ms dotnet 6.0.401

fasta

Input: 2500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 5-m.cs 413ms 14ms 88.4MB 620ms 113ms dotnet 6.0.401
csharp 5-m.cs 505ms 24ms 58.8MB 850ms 97ms mono 6.12.0
python 1.py 2607ms 3.7ms 91.1MB 2573ms 13ms pypy 3.8.13
python 5-m.py 3750ms 328ms 12.6MB 5597ms 1000ms pyston 3.8.12
python 5-m.py 3964ms 11ms 85.4MB 4430ms 853ms pypy 3.8.13
python 1.py 4082ms 19ms 7.8MB 4070ms 0ms pyston 3.8.12
python 5-m.py 4416ms 24ms 12.1MB 6350ms 1030ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

Input: 250000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 5.cs 93ms 9.3ms 37.3MB 120ms 17ms mono 6.12.0
csharp 5.cs 112ms 18ms 41.4MB 97ms 27ms dotnet 6.0.401
python 1.py 346ms 2.7ms 81.7MB 307ms 20ms pypy 3.8.13
python 1.py 424ms 5.3ms 7.9MB 410ms 0ms pyston 3.8.12
python 5-m.py 449ms 34ms 12.6MB 640ms 120ms pyston 3.8.12
python 5-m.py 513ms 1.8ms 12.1MB 660ms 147ms cpython 3.10.7
python 1.py 708ms 1.5ms 7.8MB 700ms 0ms cpython 3.10.7
python 5-m.py 821ms 4.5ms 85.2MB 950ms 420ms pypy 3.8.13

helloworld

Input: QwQ

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 13ms 0.7ms 7.3MB 10ms 0ms pyston 3.8.12
python 1.py 15ms 0.3ms 7.3MB 10ms 0ms cpython 3.10.7
csharp 1.cs 19ms 0.3ms 23.8MB 10ms 0ms mono 6.12.0
python 1.py 34ms 2.4ms 54.0MB 14ms 6ms pypy 3.8.13
csharp 1.cs 51ms 4.5ms 29.0MB 22ms 4ms dotnet 6.0.401

http-server

Input: 3000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 2-http2.cs 661ms 25ms 120.4MB 753ms 63ms dotnet 6.0.401
csharp 1-m.cs 859ms 28ms 157.4MB 913ms 380ms dotnet 6.0.401
csharp 2-http3.cs 860ms 98ms 271.3MB 843ms 257ms dotnet 7.0.100-rc.1.22431.12
csharp 2.cs 862ms 33ms 146.0MB 797ms 393ms dotnet 6.0.401
python 1.py 1563ms 4.8ms 29.1MB 1310ms 903ms pyston 3.8.12
python 1.py 2194ms 14ms 25.2MB 1920ms 1007ms cpython 3.10.7
python 1.py 4362ms 46ms 130.3MB 3430ms 830ms pypy 3.8.13

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 358ms 2.2ms 23.3MB 303ms 147ms pyston 3.8.12
python 1.py 439ms 1.0ms 21.8MB 390ms 160ms cpython 3.10.7
csharp 2.cs 440ms 8.9ms 76.8MB 373ms 93ms dotnet 6.0.401
csharp 1.cs 465ms 8.0ms 84.1MB 490ms 90ms dotnet 6.0.401
csharp 2-http3.cs 512ms 1.5ms 182.4MB 417ms 137ms dotnet 7.0.100-rc.1.22431.12
csharp 2-http2.cs 524ms 11ms 78.7MB 520ms 50ms dotnet 6.0.401
python 1.py 1070ms 9.9ms 105.3MB 870ms 170ms pypy 3.8.13

json-serde

Input: sample 5000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 257ms 1.3ms 38.3MB 220ms 17ms pyston 3.8.12
python 1.py 264ms 4.8ms 38.6MB 230ms 17ms cpython 3.10.7
csharp 2.cs 379ms 20ms 66.3MB 313ms 37ms dotnet 6.0.401
python 1.py 422ms 1.4ms 109.1MB 380ms 27ms pypy 3.8.13
csharp 1.cs 536ms 2.0ms 146.8MB 503ms 67ms dotnet 6.0.401

Input: canada 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 2.cs 2027ms 7.5ms 260.8MB 2113ms 97ms dotnet 6.0.401
python 1.py 2345ms 5.4ms 219.7MB 2230ms 103ms cpython 3.10.7
python 1.py 2387ms 2.0ms 220.5MB 2260ms 110ms pyston 3.8.12
python 1.py 2589ms 8.8ms 248.0MB 2497ms 77ms pypy 3.8.13
csharp 1.cs 3386ms 174ms 762.4MB 3480ms 303ms dotnet 6.0.401

knucleotide

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 6.cs 1202ms 73ms 75.3MB 1703ms 40ms dotnet 6.0.401
csharp 6-m.cs 1582ms 59ms 72.3MB 2573ms 23ms mono 6.12.0
python 3-m.py 3577ms 8.2ms 203.7MB 6277ms 330ms pypy 3.8.13
python 3.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 3.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 6.cs 238ms 5.4ms 63.9MB 257ms 17ms dotnet 6.0.401
csharp 6.cs 246ms 11ms 56.4MB 313ms 20ms mono 6.12.0
python 3-m.py 533ms 8.0ms 96.8MB 753ms 123ms pypy 3.8.13
python 3-m.py 594ms 7.7ms 14.8MB 1040ms 37ms pyston 3.8.12
python 3-m.py 864ms 11ms 15.3MB 1557ms 30ms cpython 3.10.7

lru

Input: 1000 1000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 2.cs 186ms 1.6ms 25.5MB 173ms 0ms mono 6.12.0
python 2.py 199ms 0.9ms 80.3MB 170ms 17ms pypy 3.8.13
csharp 2.cs 206ms 1.5ms 30.2MB 177ms 0ms dotnet 6.0.401
python 1.py 241ms 0.6ms 81.0MB 213ms 13ms pypy 3.8.13
python 1.py 993ms 2.8ms 8.9MB 977ms 0ms pyston 3.8.12
python 2.py 1176ms 4.2ms 8.7MB 1163ms 0ms pyston 3.8.12
csharp 1.cs 1658ms 11ms 89.7MB 1613ms 30ms dotnet 6.0.401
python 1.py 1831ms 10ms 8.5MB 1817ms 0ms cpython 3.10.7
python 2.py 2615ms 35ms 8.3MB 2597ms 0ms cpython 3.10.7
csharp 1.cs 3025ms 18ms 49.7MB 2983ms 30ms mono 6.12.0

Input: 1000 3000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 2.cs 458ms 3.7ms 28.2MB 433ms 0ms dotnet 6.0.401
python 2.py 468ms 4.4ms 80.4MB 443ms 7ms pypy 3.8.13
csharp 2.cs 506ms 0.4ms 23.7MB 490ms 0ms mono 6.12.0
python 1.py 509ms 2.2ms 81.5MB 477ms 13ms pypy 3.8.13
python 1.py 3011ms 54ms 9.0MB 3000ms 0ms pyston 3.8.12
python 2.py 3482ms 6.9ms 8.7MB 3470ms 0ms pyston 3.8.12
csharp 1.cs 4679ms 40ms 91.1MB 4627ms 23ms dotnet 6.0.401
csharp 1.cs timeout 0.0ms 0.0MB 0ms 0ms mono 6.12.0
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

Input: 100 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 2.cs 105ms 0.4ms 25.4MB 87ms 3ms mono 6.12.0
csharp 2.cs 130ms 2.2ms 28.0MB 100ms 0ms dotnet 6.0.401
python 2.py 145ms 1.3ms 80.1MB 117ms 13ms pypy 3.8.13
python 1.py 164ms 3.1ms 81.1MB 140ms 10ms pypy 3.8.13
python 1.py 481ms 2.8ms 8.7MB 467ms 0ms pyston 3.8.12
python 2.py 546ms 3.1ms 8.1MB 530ms 0ms pyston 3.8.12
python 1.py 862ms 4.4ms 7.8MB 853ms 0ms cpython 3.10.7
python 2.py 1257ms 5.4ms 7.3MB 1240ms 0ms cpython 3.10.7
csharp 1.cs 3185ms 1.5ms 31.5MB 3167ms 3ms mono 6.12.0
csharp 1.cs 3194ms 6.3ms 77.0MB 3133ms 23ms dotnet 6.0.401

merkletrees

Input: 17

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 1.cs 932ms 13ms 393.5MB 783ms 183ms dotnet 6.0.401
csharp 2.cs 934ms 20ms 379.2MB 803ms 160ms dotnet 6.0.401
csharp 1.cs 1680ms 15ms 144.9MB 2053ms 127ms mono 6.12.0
python 1.py 2434ms 16ms 293.6MB 2253ms 163ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 15

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 2.cs 255ms 6.6ms 85.1MB 200ms 30ms dotnet 6.0.401
csharp 1.cs 293ms 6.0ms 183.8MB 203ms 67ms dotnet 6.0.401
csharp 1.cs 312ms 3.4ms 72.7MB 280ms 43ms mono 6.12.0
python 1.py 351ms 5.0ms 145.1MB 290ms 40ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

nbody

Input: 5000000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 9.cs 405ms 2.4ms 27.3MB 373ms 0ms dotnet 6.0.401
csharp 8.cs 535ms 3.1ms 29.3MB 507ms 0ms dotnet 6.0.401
csharp 8.cs 1127ms 33ms 24.8MB 1113ms 0ms mono 6.12.0
python 2.py 2065ms 46ms 80.0MB 2033ms 13ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12
python 2.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 500000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 9.cs 89ms 5.9ms 27.3MB 60ms 3ms dotnet 6.0.401
csharp 8.cs 103ms 4.1ms 31.3MB 73ms 0ms dotnet 6.0.401
csharp 8.cs 134ms 1.5ms 25.1MB 120ms 0ms mono 6.12.0
python 2.py 269ms 5.8ms 79.6MB 237ms 13ms pypy 3.8.13
python 1.py 583ms 7.9ms 79.4MB 560ms 3ms pypy 3.8.13
python 1.py 1184ms 11ms 7.2MB 1167ms 0ms pyston 3.8.12
python 2.py 1613ms 7.5ms 8.1MB 1600ms 0ms pyston 3.8.12
python 1.py 4703ms 56ms 7.4MB 4687ms 0ms cpython 3.10.7
python 2.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

nsieve

Input: 12

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 1.cs 858ms 20ms 88.0MB 813ms 13ms dotnet 6.0.401
csharp 1.cs 910ms 21ms 84.8MB 870ms 20ms mono 6.12.0
csharp 2.cs 996ms 3.9ms 38.1MB 963ms 3ms dotnet 6.0.401
csharp 2.cs 2432ms 1.1ms 34.3MB 2413ms 7ms mono 6.12.0
python 1.py 4008ms 8.7ms 391.4MB 3873ms 117ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 10

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 1.cs 183ms 0.3ms 40.8MB 163ms 7ms mono 6.12.0
csharp 1.cs 186ms 3.5ms 45.9MB 153ms 7ms dotnet 6.0.401
csharp 2.cs 278ms 1.1ms 29.7MB 247ms 0ms dotnet 6.0.401
csharp 2.cs 605ms 2.9ms 27.9MB 590ms 3ms mono 6.12.0
python 1.py 1030ms 33ms 215.7MB 967ms 40ms pypy 3.8.13
python 1.py 1868ms 12ms 85.9MB 1830ms 17ms pyston 3.8.12
python 1.py 4326ms 5.5ms 85.9MB 4293ms 20ms cpython 3.10.7

pidigits

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 4.py 2726ms 13ms 119.6MB 2670ms 37ms pypy 3.8.13
python 4.py 2759ms 0.9ms 9.1MB 2723ms 17ms pyston 3.8.12
python 4.py 2767ms 0.5ms 8.6MB 2737ms 17ms cpython 3.10.7
csharp 1.cs timeout 0.0ms 0.0MB 0ms 0ms dotnet 6.0.401

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 4.py 653ms 0.8ms 8.8MB 640ms 0ms pyston 3.8.12
python 4.py 660ms 0.3ms 8.3MB 647ms 0ms cpython 3.10.7
python 4.py 684ms 5.3ms 84.5MB 657ms 10ms pypy 3.8.13
csharp 1.cs 1271ms 0.5ms 94.7MB 1260ms 30ms dotnet 6.0.401

regex-redux

Input: 2500000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 5.cs 3093ms 65ms 613.1MB 2987ms 247ms dotnet 7.0.100-rc.1.22431.12
python 1.py 3326ms 0.7ms 265.3MB 3237ms 70ms pypy 3.8.13
csharp 5.cs 4190ms 147ms 533.7MB 4130ms 210ms dotnet 6.0.401
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 1.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 250000_in

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 465ms 2.9ms 103.5MB 423ms 23ms pypy 3.8.13
csharp 5.cs 484ms 11ms 174.1MB 393ms 47ms dotnet 7.0.100-rc.1.22431.12
csharp 5.cs 535ms 11ms 88.6MB 500ms 30ms dotnet 6.0.401
python 1.py 699ms 3.7ms 30.3MB 653ms 20ms pyston 3.8.12
python 1.py 847ms 1.1ms 31.0MB 817ms 17ms cpython 3.10.7

secp256k1

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 1.cs 4358ms 92ms 89.9MB 4307ms 37ms dotnet 6.0.401
python 1.py 4528ms 9.5ms 8.3MB 4513ms 0ms pyston 3.8.12
python 1.py 4900ms 14ms 87.1MB 4867ms 17ms pypy 3.8.13
python 1.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7

Input: 500

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
python 1.py 1144ms 1.3ms 8.4MB 1130ms 0ms pyston 3.8.12
csharp 1.cs 1173ms 4.8ms 92.6MB 1150ms 27ms dotnet 6.0.401
python 1.py 1375ms 6.1ms 84.2MB 1343ms 13ms pypy 3.8.13
python 1.py 1498ms 22ms 7.3MB 1480ms 0ms cpython 3.10.7

spectral-norm

Input: 8000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 3-m.cs 3104ms 414ms 30.4MB 5183ms 13ms dotnet 6.0.401
csharp 3.cs timeout 0.0ms 0.0MB 0ms 0ms mono 6.12.0
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 4000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 3-m.cs 716ms 4.2ms 31.4MB 1293ms 13ms dotnet 6.0.401
csharp 3-m.cs 3178ms 135ms 29.2MB 6037ms 3ms mono 6.12.0
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12

Input: 2000

lang code time stddev peak-mem mem time(user) time(sys) compiler compiler/runtime
csharp 3-m.cs 231ms 1.3ms 30.4MB 353ms 7ms dotnet 6.0.401
csharp 3-m.cs 802ms 1.9ms 29.5MB 1527ms 3ms mono 6.12.0
python 8-m.py 3292ms 24ms 109.1MB 5633ms 357ms pypy 3.8.13
python 8.py timeout 0.0ms 0.0MB 0ms 0ms cpython 3.10.7
python 8.py timeout 0.0ms 0.0MB 0ms 0ms pyston 3.8.12