Mercurial > hg > graal-compiler
annotate perf/benchmarktool.sh @ 3211:76507b87dd25
global absolute probability analysis:
* added switch probability, changed branch probability from int to double
* absolute probability on each FixedNode computed by new ComputeProbabilityPhase
* loopFrequency estimation on LoopBegin nodes
* added GraalOptions.ProbabilityAnalysis flag: builds probability information and let inlining and escape analysis use it
author | Lukas Stadler <lukas.stadler@jku.at> |
---|---|
date | Tue, 12 Jul 2011 17:00:25 +0200 |
parents | 6ff76a1b8339 |
children |
rev | line source |
---|---|
3185
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
1 #!/usr/bin/python |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
2 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
3 import subprocess |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
4 import os |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
5 import re |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
6 import sys |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
7 import argparse |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
8 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
9 DEFAULT_DACAPO_OPTS = " -XX:MaxPermSize=512m -Xms1g -Xmx2g " |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
10 DEFAULT_SCIMARK_OPTS = " -Xms32m -Xmx100m " |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
11 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
12 def runBash(cmd): |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
13 p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
14 return p.stdout |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
15 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
16 def s2msString(floatString): |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
17 return str(round(float(floatString)*1000)) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
18 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
19 # Raw String Notation (r"") : \ instead of \\ |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
20 graalTime = re.compile(r"Total compilation time\s+:\s+([0-9]+\.[0-9]+) s") |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
21 graalSubTime = re.compile(r"([a-zA-Z0-9_ ]+)\s+:\s+([0-9]+\.[0-9]+) s \([0-9 ][0-9]\.[0-9]{2}%\)") |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
22 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
23 def matchGraalTime(string, csvOutput, csvOutputLine, writeHeaderAt): |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
24 match1 = graalTime.search(string) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
25 match2 = graalSubTime.search(string) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
26 if match1: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
27 if csvOutputLine == writeHeaderAt: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
28 csvOutput[0].append('graal total') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
29 print('Graal total time: '+ s2msString(match1.group(1))) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
30 csvOutput[csvOutputLine].append(s2msString(match1.group(1))) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
31 elif match2: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
32 if csvOutputLine == writeHeaderAt: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
33 csvOutput[0].append(match2.group(1).strip()) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
34 print('Graal specific time: '+match2.group(1)+': '+s2msString(match2.group(2))) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
35 csvOutput[csvOutputLine].append(s2msString(match2.group(2))) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
36 else: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
37 print('# '+string) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
38 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
39 def writeout(outputFile, csvOutput): |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
40 for csvLine in csvOutput : |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
41 outputFile.write(';'.join(csvLine)+';'+os.linesep) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
42 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
43 def main(): |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
44 # Check for environment variables |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
45 if os.getenv('JDK7') is None: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
46 print('Environment variable JDK7 is not defined.') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
47 return 1 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
48 if os.getenv('DACAPO') is None: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
49 print('Environment variable DACAPO is not defined. It must point to a directory with the DaCapo benchmark jar.') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
50 return 1 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
51 if os.getenv('SCIMARK') is None: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
52 print('Environment variable SCIMARK is not defined. It must point to a directory with the SciMark benchmark jar.') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
53 return 1 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
54 if os.getenv('REFERENCE_JDK') is None: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
55 print('Environment variable REFERENCE_JDK is not defined.') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
56 return 1 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
57 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
58 # Option parsing |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
59 parser = argparse.ArgumentParser(description='Automated DaCapo and Scimark bechmarks') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
60 parser.add_argument('-a', '-all', help='run all benchmarks for all compilers', action='store_true') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
61 parser.add_argument('-c', type=str, help='compiler to use', default='graal', choices=['client', 'server', 'graal'], required=False) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
62 parser.add_argument('-n', type=int, help='number of DaCapo benchmarks to run', default=20) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
63 parser.add_argument('-o', type=str, help='graalVM options(quoted!)', default='') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
64 parser.add_argument('-runonly', type=str, help='run specified benchmark only', default='all') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
65 options = parser.parse_args() |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
66 compilerFlags = {'graal' : '-client -graal -G:+Time -XX:-GraalBailoutIsFatal -G:QuietBailout ', |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
67 'client' : '-client', |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
68 'server' : '-server'} |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
69 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
70 if options.a: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
71 compilers = ['graal', 'client', 'server'] |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
72 else: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
73 compilers = [options.c] |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
74 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
75 for compiler in compilers: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
76 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
77 outputFile = open(compiler+'.csv', 'w') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
78 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
79 # DaCapo Benchmarks |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
80 if compiler == 'graal': |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
81 vm = os.environ['JDK7'] |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
82 else : |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
83 vm = os.environ['REFERENCE_JDK'] |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
84 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
85 cmd = vm + '/bin/java ' + compilerFlags[compiler] + ' -d64 ' + DEFAULT_DACAPO_OPTS + options.o + ' -classpath ' + \ |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
86 os.environ['DACAPO'] + '/dacapo-9.12-bach.jar Harness -n ' + str(options.n) + ' ' |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
87 benchmarks = runBash('java -jar ' + os.environ['DACAPO'] + '/dacapo-9.12-bach.jar -l').read().decode().split(' ') |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
88 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
89 benchmarkTime = re.compile(r"===== DaCapo 9\.12 ([a-zA-Z0-9_]+) ((PASSED)|(completed warmup [0-9]+)) in ([0-9]+) msec =====") |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
90 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
91 csvOutput = [['benchmark', 'type', 'time']] |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
92 csvOutputLine = 0 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
93 for benchmark in benchmarks: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
94 if options.runonly != 'all' and benchmark != options.runonly: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
95 continue |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
96 nRuns = 0 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
97 dcOutput = runBash(cmd + benchmark) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
98 while True: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
99 line = dcOutput.readline().decode() |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
100 if not line: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
101 break |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
102 line = line.strip() |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
103 match = benchmarkTime.search(line) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
104 if match: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
105 csvOutputLine = csvOutputLine + 1 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
106 nRuns = nRuns + 1 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
107 csvOutput.append(list()) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
108 csvOutput[csvOutputLine].append(str(nRuns)) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
109 print('Benchmark type: '+match.group(1)) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
110 print('Benchmark time: '+match.group(5)) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
111 csvOutput[csvOutputLine].append(match.group(1)) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
112 csvOutput[csvOutputLine].append(match.group(5)) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
113 else: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
114 matchGraalTime(line, csvOutput, csvOutputLine, options.n) |
3187
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
115 |
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
116 if nRuns < options.n: |
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
117 csvOutputLine = csvOutputLine + (options.n - nRuns) |
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
118 for i in range(options.n - nRuns): |
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
119 csvOutput.append([str(nRuns + i), benchmark, '0']) |
3185
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
120 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
121 writeout(outputFile, csvOutput) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
122 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
123 if options.runonly != 'all' and options.runonly != 'scimark': |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
124 outputFile.close() |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
125 return 0 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
126 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
127 # Scimark Benchmarks |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
128 writeout(outputFile, [['']]) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
129 cmd = vm + '/bin/java ' + compilerFlags[compiler] + ' -d64 ' + DEFAULT_SCIMARK_OPTS + options.o + \ |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
130 ' -Xbootclasspath/a:' + os.environ['SCIMARK'] + '/scimark2lib.jar jnt.scimark2.commandline' |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
131 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
132 benchmarkScore = re.compile(r"([a-zA-Z0-9_\(\),= ]+):\s+([0-9]+\.[0-9]+)$") |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
133 |
3187
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
134 csvOutput = [['run'],[]] |
3185
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
135 scOutput = runBash(cmd) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
136 while True: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
137 line = scOutput.readline().decode() |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
138 if not line: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
139 break |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
140 line = line.strip() |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
141 match = benchmarkScore.search(line) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
142 if match: |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
143 print('Scimark '+match.group(1)+' score: '+match.group(2)) |
3187
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
144 csvOutput[0].append(match.group(1).strip()) |
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
145 csvOutput[1].append(match.group(2)) |
3185
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
146 else: |
3187
6ff76a1b8339
The benchmark tool should now print zero values to the csv file, if a benchmark fails
Josef Haider <josef.haider@khg.jku.at>
parents:
3185
diff
changeset
|
147 matchGraalTime(line,csvOutput, 1, 1) |
3185
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
148 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
149 writeout(outputFile, csvOutput) |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
150 outputFile.close() |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
151 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
152 return 0 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
153 |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
154 #This idiom means the below code only runs when executed from command line |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
155 if __name__ == '__main__': |
4ccb3d5e4da1
Added a small python script for automated benchmarks.
Josef Haider <josef.haider@khg.jku.at>
parents:
diff
changeset
|
156 sys.exit(main()) |