Aggregated results by problem

In the tables below we provide the aggregated results of the solvers for each problem (SE, EE, DC and DS). The column Solver, #Corr, #Cra, #Inc, #Fail, #Inv, #TO, #OOM, Time, Memory, Score corresponds to: name of the solver, number of correct answers, number of crashes, number of incorrect answers, number of failures, number of invalid answers, number of timeouts, number of out of memory, running time (total), used memory (max) and final score, respectively. In detail, #Cra refers to the number of times the solver crashed due to memory issues, #Fail are all those runs that did not produce an answer (e.g., because of timeouts), #Inc refers to syntactically correct, but wrong, answers, and #Inv is the number of syntactically incorrect answers.

Back to all results

SE
Solver #Corr #Cra #Inc #Fail #Inv #TO #OOM Time Memory Score
coquiaas 1277 0 20 0 0 7 0 10019.608 98557952 1177
mu-toksia 1304 0 0 0 0 0 0 4162.335 69242880 1304
pyglaf 1060 0 244 0 0 0 0 4173.753 50429952 -160
EE
Solver #Corr #Cra #Inc #Fail #Inv #TO #OOM Time Memory Score
coquiaas 956 0 0 2 0 20 0 25241.555 97075200 956
mu-toksia 971 0 0 0 1 6 0 10895.509 80859136 971
pyglaf 665 0 307 0 0 6 0 11216.167 99147776 -870
DC
Solver #Corr #Cra #Inc #Fail #Inv #TO #OOM Time Memory Score
coquiaas 1304 0 0 0 0 0 0 1649.492 3059712 1304
mu-toksia 1304 0 0 0 0 0 0 3986.903 409604096 1304
pyglaf 1083 0 221 0 0 0 0 4055.128 85327872 -22
DS
Solver #Corr #Cra #Inc #Fail #Inv #TO #OOM Time Memory Score
coquiaas 974 0 0 0 0 4 0 7458.918 93229056 974
mu-toksia 978 0 0 0 0 0 0 3092.785 409702400 978
pyglaf 785 0 193 0 0 0 0 3252.190 85516288 -180