Mercurial > hg > truffle
annotate mx/sanitycheck.py @ 17260:222b60e248ba
Truffle-DSL: fixed regression with @SlowPath on specialize0.
author | Christian Humer <christian.humer@gmail.com> |
---|---|
date | Tue, 30 Sep 2014 00:48:47 +0200 |
parents | fdb912b1eb7b |
children | 03eef43d364f |
rev | line source |
---|---|
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
1 # ---------------------------------------------------------------------------------------------------- |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
2 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
5 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
6 # This code is free software; you can redistribute it and/or modify it |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
7 # under the terms of the GNU General Public License version 2 only, as |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
8 # published by the Free Software Foundation. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
9 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
10 # This code is distributed in the hope that it will be useful, but WITHOUT |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
13 # version 2 for more details (a copy is included in the LICENSE file that |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
14 # accompanied this code). |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
15 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
16 # You should have received a copy of the GNU General Public License version |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
17 # 2 along with this work; if not, write to the Free Software Foundation, |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
19 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
21 # or visit www.oracle.com if you need additional information or have any |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
22 # questions. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
23 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
24 # ---------------------------------------------------------------------------------------------------- |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
25 |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
26 from outputparser import OutputParser, ValuesMatcher |
12769
99769479f9ce
s/commands/mx_graal/ in mx/sanitycheck.py
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
12647
diff
changeset
|
27 import re, mx, mx_graal, os, sys, StringIO, subprocess |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
28 from os.path import isfile, join, exists |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
29 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
30 gc = 'UseSerialGC' |
8501
41fc46da946a
-More fixes and passrate(fop)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
7656
diff
changeset
|
31 |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
32 dacapoSanityWarmup = { |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
33 'avrora': [0, 0, 3, 6, 13], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
34 'batik': [0, 0, 5, 5, 20], |
14877
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
35 'eclipse': [0, 0, 0, 0, 0], |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
36 'fop': [4, 8, 10, 20, 30], |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
37 'h2': [0, 0, 5, 5, 8], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
38 'jython': [0, 0, 5, 10, 13], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
39 'luindex': [0, 0, 5, 10, 10], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
40 'lusearch': [0, 4, 5, 5, 8], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
41 'pmd': [0, 0, 5, 10, 13], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
42 'sunflow': [0, 2, 5, 10, 15], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
43 'tomcat': [0, 0, 5, 10, 15], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
44 'tradebeans': [0, 0, 5, 10, 13], |
14877
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
45 'tradesoap': [0, 0, 5, 10, 15], |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
46 'xalan': [0, 0, 5, 10, 18], |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
47 } |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
48 |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
49 dacapoScalaSanityWarmup = { |
14877
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
50 'actors': [0, 0, 2, 5, 5], |
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
51 'apparat': [0, 0, 2, 5, 5], |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
52 'factorie': [0, 0, 2, 5, 5], |
14877
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
53 'kiama': [0, 4, 3, 13, 15], |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
54 'scalac': [0, 0, 5, 15, 20], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
55 'scaladoc': [0, 0, 5, 15, 15], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
56 'scalap': [0, 0, 5, 15, 20], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
57 'scalariform':[0, 0, 6, 15, 20], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
58 'scalatest': [0, 0, 2, 10, 12], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
59 'scalaxb': [0, 0, 5, 15, 25], |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
60 # (gdub) specs sometimes returns a non-zero value event though there is no apparent failure |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
61 'specs': [0, 0, 0, 0, 0], |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
62 'tmt': [0, 0, 3, 10, 12] |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
63 } |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
64 |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
65 dacapoGateBuildLevels = { |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
66 'avrora': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
67 'batik': ['product', 'fastdebug', 'debug'], |
14877
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
68 # (lewurm): does not work with JDK8 |
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
69 'eclipse': [], |
13967
4cd7c6629841
mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents:
12769
diff
changeset
|
70 'fop': ['fastdebug', 'debug'], |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
71 'h2': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
72 'jython': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
73 'luindex': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
74 'lusearch': ['product'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
75 'pmd': ['product', 'fastdebug', 'debug'], |
13967
4cd7c6629841
mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents:
12769
diff
changeset
|
76 'sunflow': ['fastdebug', 'debug'], |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
77 'tomcat': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
78 'tradebeans': ['product', 'fastdebug', 'debug'], |
14877
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
79 # tradesoap is too unreliable for the gate, often crashing with concurrency problems: |
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
80 # http://sourceforge.net/p/dacapobench/bugs/99/ |
13967
4cd7c6629841
mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents:
12769
diff
changeset
|
81 'tradesoap': [], |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
82 'xalan': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
83 } |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
84 |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
85 dacapoScalaGateBuildLevels = { |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
86 'actors': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
87 'apparat': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
88 'factorie': ['product', 'fastdebug', 'debug'], |
14877
fdb912b1eb7b
gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents:
13967
diff
changeset
|
89 'kiama': ['fastdebug', 'debug'], |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
90 'scalac': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
91 'scaladoc': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
92 'scalap': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
93 'scalariform':['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
94 'scalatest': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
95 'scalaxb': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
96 'specs': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
97 'tmt': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
98 } |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
99 |
11367
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
100 specjvm2008Names = [ |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
101 'startup.helloworld', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
102 'startup.compiler.compiler', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
103 'startup.compiler.sunflow', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
104 'startup.compress', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
105 'startup.crypto.aes', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
106 'startup.crypto.rsa', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
107 'startup.crypto.signverify', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
108 'startup.mpegaudio', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
109 'startup.scimark.fft', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
110 'startup.scimark.lu', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
111 'startup.scimark.monte_carlo', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
112 'startup.scimark.sor', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
113 'startup.scimark.sparse', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
114 'startup.serial', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
115 'startup.sunflow', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
116 'startup.xml.transform', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
117 'startup.xml.validation', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
118 'compiler.compiler', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
119 'compiler.sunflow', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
120 'compress', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
121 'crypto.aes', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
122 'crypto.rsa', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
123 'crypto.signverify', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
124 'derby', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
125 'mpegaudio', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
126 'scimark.fft.large', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
127 'scimark.lu.large', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
128 'scimark.sor.large', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
129 'scimark.sparse.large', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
130 'scimark.fft.small', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
131 'scimark.lu.small', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
132 'scimark.sor.small', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
133 'scimark.sparse.small', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
134 'scimark.monte_carlo', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
135 'serial', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
136 'sunflow', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
137 'xml.transform', |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
138 'xml.validation' |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
139 ] |
39b86b83ddeb
normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents:
10895
diff
changeset
|
140 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
141 def _noneAsEmptyList(a): |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
142 if a is None: |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
143 return [] |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
144 return a |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
145 |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
146 class SanityCheckLevel: |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
147 Fast, Gate, Normal, Extensive, Benchmark = range(5) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
148 |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
149 def getSPECjbb2005(benchArgs=None): |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
150 benchArgs = [] if benchArgs is None else benchArgs |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
151 |
5877
0e54d9bb922d
Add SPECjbb2005 to the benchmarks
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5196
diff
changeset
|
152 specjbb2005 = mx.get_env('SPECJBB2005') |
0e54d9bb922d
Add SPECjbb2005 to the benchmarks
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5196
diff
changeset
|
153 if specjbb2005 is None or not exists(join(specjbb2005, 'jbb.jar')): |
0e54d9bb922d
Add SPECjbb2005 to the benchmarks
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5196
diff
changeset
|
154 mx.abort('Please set the SPECJBB2005 environment variable to a SPECjbb2005 directory') |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
155 |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
156 score = re.compile(r"^Valid run, Score is (?P<score>[0-9]+)$", re.MULTILINE) |
5877
0e54d9bb922d
Add SPECjbb2005 to the benchmarks
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5196
diff
changeset
|
157 error = re.compile(r"VALIDATION ERROR") |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
158 success = re.compile(r"^Valid run, Score is [0-9]+$", re.MULTILINE) |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
159 matcher = ValuesMatcher(score, {'group' : 'SPECjbb2005', 'name' : 'score', 'score' : '<score>'}) |
5877
0e54d9bb922d
Add SPECjbb2005 to the benchmarks
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5196
diff
changeset
|
160 classpath = ['jbb.jar', 'check.jar'] |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
161 return Test("SPECjbb2005", ['spec.jbb.JBBmain', '-propfile', 'SPECjbb.props'] + benchArgs, [success], [error], [matcher], vmOpts=['-Xms3g', '-XX:+' + gc, '-XX:-UseCompressedOops', '-cp', os.pathsep.join(classpath)], defaultCwd=specjbb2005) |
7565 | 162 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
163 def getSPECjbb2013(benchArgs=None): |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
164 |
7565 | 165 specjbb2013 = mx.get_env('SPECJBB2013') |
166 if specjbb2013 is None or not exists(join(specjbb2013, 'specjbb2013.jar')): | |
167 mx.abort('Please set the SPECJBB2013 environment variable to a SPECjbb2013 directory') | |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
168 |
7565 | 169 jops = re.compile(r"^RUN RESULT: hbIR \(max attempted\) = [0-9]+, hbIR \(settled\) = [0-9]+, max-jOPS = (?P<max>[0-9]+), critical-jOPS = (?P<critical>[0-9]+)$", re.MULTILINE) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
170 # error? |
7565 | 171 success = re.compile(r"org.spec.jbb.controller: Run finished", re.MULTILINE) |
172 matcherMax = ValuesMatcher(jops, {'group' : 'SPECjbb2013', 'name' : 'max', 'score' : '<max>'}) | |
173 matcherCritical = ValuesMatcher(jops, {'group' : 'SPECjbb2013', 'name' : 'critical', 'score' : '<critical>'}) | |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
174 return Test("SPECjbb2013", ['-jar', 'specjbb2013.jar', '-m', 'composite'] + |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
175 _noneAsEmptyList(benchArgs), [success], [], [matcherCritical, matcherMax], |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
176 vmOpts=['-Xmx6g', '-Xms6g', '-Xmn3g', '-XX:+UseParallelOldGC', '-XX:-UseAdaptiveSizePolicy', '-XX:-UseBiasedLocking', '-XX:-UseCompressedOops'], defaultCwd=specjbb2013) |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
177 |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
178 def getSPECjvm2008(benchArgs=None): |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
179 |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
180 specjvm2008 = mx.get_env('SPECJVM2008') |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
181 if specjvm2008 is None or not exists(join(specjvm2008, 'SPECjvm2008.jar')): |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
182 mx.abort('Please set the SPECJVM2008 environment variable to a SPECjvm2008 directory') |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
183 |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
184 score = re.compile(r"^(Score on|Noncompliant) (?P<benchmark>[a-zA-Z0-9\._]+)( result)?: (?P<score>[0-9]+((,|\.)[0-9]+)?)( SPECjvm2008 Base)? ops/m$", re.MULTILINE) |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
185 error = re.compile(r"^Errors in benchmark: ", re.MULTILINE) |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
186 # The ' ops/m' at the end of the success string is important : it's how you can tell valid and invalid runs apart |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
187 success = re.compile(r"^(Noncompliant c|C)omposite result: [0-9]+((,|\.)[0-9]+)?( SPECjvm2008 (Base|Peak))? ops/m$", re.MULTILINE) |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
188 matcher = ValuesMatcher(score, {'group' : 'SPECjvm2008', 'name' : '<benchmark>', 'score' : '<score>'}) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
189 |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
190 return Test("SPECjvm2008", ['-jar', 'SPECjvm2008.jar'] + _noneAsEmptyList(benchArgs), [success], [error], [matcher], vmOpts=['-Xms3g', '-XX:+' + gc, '-XX:-UseCompressedOops'], defaultCwd=specjvm2008) |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
191 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
192 def getDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=None): |
4217
f3271682fe5a
Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4215
diff
changeset
|
193 checks = [] |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
194 |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
195 for (bench, ns) in dacapoSanityWarmup.items(): |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
196 if ns[level] > 0: |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
197 if gateBuildLevel is None or gateBuildLevel in dacapoGateBuildLevels[bench]: |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
198 checks.append(getDacapo(bench, ['-n', str(ns[level])] + _noneAsEmptyList(dacapoArgs))) |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
199 |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
200 return checks |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
201 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
202 def getDacapo(name, dacapoArgs=None): |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
203 dacapo = mx.get_env('DACAPO_CP') |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
204 if dacapo is None: |
4244
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
205 l = mx.library('DACAPO', False) |
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
206 if l is not None: |
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
207 dacapo = l.get_path(True) |
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
208 else: |
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
209 mx.abort('DaCapo 9.12 jar file must be specified with DACAPO_CP environment variable or as DACAPO library') |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
210 |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
211 if not isfile(dacapo) or not dacapo.endswith('.jar'): |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
212 mx.abort('Specified DaCapo jar file does not exist or is not a jar file: ' + dacapo) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
213 |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
214 dacapoSuccess = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====$", re.MULTILINE) |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
215 dacapoFail = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) FAILED (warmup|) =====$", re.MULTILINE) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
216 dacapoTime = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====") |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
217 dacapoTime1 = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) completed warmup 1 in (?P<time>[0-9]+) msec =====") |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
218 |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
219 dacapoMatcher = ValuesMatcher(dacapoTime, {'group' : 'DaCapo', 'name' : '<benchmark>', 'score' : '<time>'}) |
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
220 dacapoMatcher1 = ValuesMatcher(dacapoTime1, {'group' : 'DaCapo-1stRun', 'name' : '<benchmark>', 'score' : '<time>'}) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
221 |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
222 return Test("DaCapo-" + name, ['-jar', dacapo, name] + _noneAsEmptyList(dacapoArgs), [dacapoSuccess], [dacapoFail], [dacapoMatcher, dacapoMatcher1], ['-Xms2g', '-XX:+' + gc, '-XX:-UseCompressedOops']) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
223 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
224 def getScalaDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=None): |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
225 checks = [] |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
226 |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
227 for (bench, ns) in dacapoScalaSanityWarmup.items(): |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
228 if ns[level] > 0: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
229 if gateBuildLevel is None or gateBuildLevel in dacapoScalaGateBuildLevels[bench]: |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
230 checks.append(getScalaDacapo(bench, ['-n', str(ns[level])] + _noneAsEmptyList(dacapoArgs))) |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
231 |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
232 return checks |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
233 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
234 def getScalaDacapo(name, dacapoArgs=None): |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
235 dacapo = mx.get_env('DACAPO_SCALA_CP') |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
236 if dacapo is None: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
237 l = mx.library('DACAPO_SCALA', False) |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
238 if l is not None: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
239 dacapo = l.get_path(True) |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
240 else: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
241 mx.abort('Scala DaCapo 0.1.0 jar file must be specified with DACAPO_SCALA_CP environment variable or as DACAPO_SCALA library') |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
242 |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
243 if not isfile(dacapo) or not dacapo.endswith('.jar'): |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
244 mx.abort('Specified Scala DaCapo jar file does not exist or is not a jar file: ' + dacapo) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
245 |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
246 dacapoSuccess = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====$", re.MULTILINE) |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
247 dacapoFail = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) FAILED (warmup|) =====$", re.MULTILINE) |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
248 dacapoTime = re.compile(r"===== DaCapo 0\.1\.0(-SNAPSHOT)? (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====") |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
249 |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
250 dacapoMatcher = ValuesMatcher(dacapoTime, {'group' : "Scala-DaCapo", 'name' : '<benchmark>', 'score' : '<time>'}) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
251 |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
252 return Test("Scala-DaCapo-" + name, ['-jar', dacapo, name] + _noneAsEmptyList(dacapoArgs), [dacapoSuccess], [dacapoFail], [dacapoMatcher], ['-Xms2g', '-XX:+' + gc, '-XX:-UseCompressedOops']) |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
253 |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
254 def getBootstraps(): |
11658
01512eb1ae7b
bootstrap: print number of compiled methods
Bernhard Urban <bernhard.urban@jku.at>
parents:
11533
diff
changeset
|
255 time = re.compile(r"Bootstrapping Graal\.+ in (?P<time>[0-9]+) ms( \(compiled (?P<methods>[0-9]+) methods\))?") |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
256 scoreMatcher = ValuesMatcher(time, {'group' : 'Bootstrap', 'name' : 'BootstrapTime', 'score' : '<time>'}) |
11658
01512eb1ae7b
bootstrap: print number of compiled methods
Bernhard Urban <bernhard.urban@jku.at>
parents:
11533
diff
changeset
|
257 methodMatcher = ValuesMatcher(time, {'group' : 'Bootstrap', 'name' : 'BootstrapMethods', 'score' : '<methods>'}) |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
258 scoreMatcherBig = ValuesMatcher(time, {'group' : 'Bootstrap-bigHeap', 'name' : 'BootstrapTime', 'score' : '<time>'}) |
11658
01512eb1ae7b
bootstrap: print number of compiled methods
Bernhard Urban <bernhard.urban@jku.at>
parents:
11533
diff
changeset
|
259 methodMatcherBig = ValuesMatcher(time, {'group' : 'Bootstrap-bigHeap', 'name' : 'BootstrapMethods', 'score' : '<methods>'}) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
260 |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
261 tests = [] |
11658
01512eb1ae7b
bootstrap: print number of compiled methods
Bernhard Urban <bernhard.urban@jku.at>
parents:
11533
diff
changeset
|
262 tests.append(Test("Bootstrap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher, methodMatcher], ignoredVMs=['client', 'server'], benchmarkCompilationRate=False)) |
01512eb1ae7b
bootstrap: print number of compiled methods
Bernhard Urban <bernhard.urban@jku.at>
parents:
11533
diff
changeset
|
263 tests.append(Test("Bootstrap-bigHeap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcherBig, methodMatcherBig], vmOpts=['-Xms2g'], ignoredVMs=['client', 'server'], benchmarkCompilationRate=False)) |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
264 return tests |
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
265 |
9157
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
266 class CTWMode: |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
267 Full, NoInline, NoComplex = range(3) |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
268 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
269 def getCTW(vm, mode): |
9157
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
270 time = re.compile(r"CompileTheWorld : Done \([0-9]+ classes, [0-9]+ methods, (?P<time>[0-9]+) ms\)") |
9180
4e6df9021a59
Assemble the CTW bench in the same value, remove PEA from nocomplex
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
9173
diff
changeset
|
271 scoreMatcher = ValuesMatcher(time, {'group' : 'CompileTheWorld', 'name' : 'CompileTime', 'score' : '<time>'}) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
272 |
9157
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
273 jre = os.environ.get('JAVA_HOME') |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
274 if exists(join(jre, 'jre')): |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
275 jre = join(jre, 'jre') |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
276 rtjar = join(jre, 'lib', 'rt.jar') |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
277 |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
278 |
9157
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
279 args = ['-XX:+CompileTheWorld', '-Xbootclasspath/p:' + rtjar] |
12647
b038b643a3a4
restricted BootstrapGraal option to GRAALVM builds only
Doug Simon <doug.simon@oracle.com>
parents:
11658
diff
changeset
|
280 if vm == 'graal': |
b038b643a3a4
restricted BootstrapGraal option to GRAALVM builds only
Doug Simon <doug.simon@oracle.com>
parents:
11658
diff
changeset
|
281 args += ['-XX:+BootstrapGraal'] |
9157
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
282 if mode >= CTWMode.NoInline: |
12769
99769479f9ce
s/commands/mx_graal/ in mx/sanitycheck.py
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
12647
diff
changeset
|
283 if not mx_graal.isGraalEnabled(vm): |
9157
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
284 args.append('-XX:-Inline') |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
285 else: |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
286 args.append('-G:-Inline') |
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
287 if mode >= CTWMode.NoComplex: |
12769
99769479f9ce
s/commands/mx_graal/ in mx/sanitycheck.py
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
12647
diff
changeset
|
288 if mx_graal.isGraalEnabled(vm): |
10895
0aba970c89f9
mx/ctw: disable new memory aware scheduling
Bernhard Urban <bernhard.urban@jku.at>
parents:
10878
diff
changeset
|
289 args += ['-G:-OptLoopTransform', '-G:-OptTailDuplication', '-G:-FullUnroll', '-G:-MemoryAwareScheduling', '-G:-NewMemoryAwareScheduling', '-G:-PartialEscapeAnalysis'] |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
290 |
9180
4e6df9021a59
Assemble the CTW bench in the same value, remove PEA from nocomplex
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
9173
diff
changeset
|
291 return Test("CompileTheWorld", args, successREs=[time], scoreMatchers=[scoreMatcher], benchmarkCompilationRate=False) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
292 |
9157
a38d748d4130
Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8506
diff
changeset
|
293 |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
294 class Tee: |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
295 def __init__(self): |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
296 self.output = StringIO.StringIO() |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
297 def eat(self, line): |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
298 self.output.write(line) |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
299 sys.stdout.write(line) |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
300 |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
301 """ |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
302 Encapsulates a single program that is a sanity test and/or a benchmark. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
303 """ |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
304 class Test: |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
305 def __init__(self, name, cmd, successREs=None, failureREs=None, scoreMatchers=None, vmOpts=None, defaultCwd=None, ignoredVMs=None, benchmarkCompilationRate=False): |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
306 |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
307 self.name = name |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
308 self.successREs = _noneAsEmptyList(successREs) |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
309 self.failureREs = _noneAsEmptyList(failureREs) + [re.compile(r"Exception occurred in scope: ")] |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
310 self.scoreMatchers = _noneAsEmptyList(scoreMatchers) |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
311 self.vmOpts = _noneAsEmptyList(vmOpts) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
312 self.cmd = cmd |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
313 self.defaultCwd = defaultCwd |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
314 self.ignoredVMs = _noneAsEmptyList(ignoredVMs) |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
315 self.benchmarkCompilationRate = benchmarkCompilationRate |
7654
641a4c6ac1ce
fixed repetition of compilation metric option on command line when running multiple benchmarks (JIRA issue GRAAL-66)
Doug Simon <doug.simon@oracle.com>
parents:
7571
diff
changeset
|
316 if benchmarkCompilationRate: |
641a4c6ac1ce
fixed repetition of compilation metric option on command line when running multiple benchmarks (JIRA issue GRAAL-66)
Doug Simon <doug.simon@oracle.com>
parents:
7571
diff
changeset
|
317 self.vmOpts = self.vmOpts + ['-XX:+CITime'] |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
318 |
4238
e0d09e05aa9b
Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents:
4230
diff
changeset
|
319 def __str__(self): |
e0d09e05aa9b
Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents:
4230
diff
changeset
|
320 return self.name |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
321 |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
322 def test(self, vm, cwd=None, extraVmOpts=None, vmbuild=None): |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
323 """ |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
324 Run this program as a sanity test. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
325 """ |
13967
4cd7c6629841
mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents:
12769
diff
changeset
|
326 if vm in self.ignoredVMs: |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
327 return True |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
328 if cwd is None: |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
329 cwd = self.defaultCwd |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
330 parser = OutputParser() |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
331 jvmError = re.compile(r"(?P<jvmerror>([A-Z]:|/).*[/\\]hs_err_pid[0-9]+\.log)") |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
332 parser.addMatcher(ValuesMatcher(jvmError, {'jvmError' : '<jvmerror>'})) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
333 |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
334 for successRE in self.successREs: |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
335 parser.addMatcher(ValuesMatcher(successRE, {'passed' : '1'})) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
336 for failureRE in self.failureREs: |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
337 parser.addMatcher(ValuesMatcher(failureRE, {'failed' : '1'})) |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
338 |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
339 tee = Tee() |
12769
99769479f9ce
s/commands/mx_graal/ in mx/sanitycheck.py
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
12647
diff
changeset
|
340 retcode = mx_graal.vm(self.vmOpts + _noneAsEmptyList(extraVmOpts) + self.cmd, vm, nonZeroIsFatal=False, out=tee.eat, err=subprocess.STDOUT, cwd=cwd, vmbuild=vmbuild) |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
341 output = tee.output.getvalue() |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
342 valueMaps = parser.parse(output) |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
343 |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
344 if len(valueMaps) == 0: |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
345 return False |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
346 |
8206
22429580c7a8
sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8147
diff
changeset
|
347 record = {} |
22429580c7a8
sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8147
diff
changeset
|
348 for valueMap in valueMaps: |
22429580c7a8
sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8147
diff
changeset
|
349 for key, value in valueMap.items(): |
22429580c7a8
sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8147
diff
changeset
|
350 if record.has_key(key) and record[key] != value: |
22429580c7a8
sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8147
diff
changeset
|
351 mx.abort('Inconsistant values returned by test machers : ' + str(valueMaps)) |
22429580c7a8
sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
8147
diff
changeset
|
352 record[key] = value |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
353 |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
354 jvmErrorFile = record.get('jvmError') |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
355 if jvmErrorFile: |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
356 mx.log('/!\\JVM Error : dumping error log...') |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
357 with open(jvmErrorFile, 'rb') as fp: |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
358 mx.log(fp.read()) |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
359 os.unlink(jvmErrorFile) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
360 return False |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
361 |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
362 if record.get('failed') == '1': |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
363 return False |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
364 |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
365 return retcode == 0 and record.get('passed') == '1' |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
366 |
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
367 def bench(self, vm, cwd=None, extraVmOpts=None, vmbuild=None): |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
368 """ |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
369 Run this program as a benchmark. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
370 """ |
13967
4cd7c6629841
mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents:
12769
diff
changeset
|
371 if vm in self.ignoredVMs: |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
372 return {} |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
373 if cwd is None: |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
374 cwd = self.defaultCwd |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
375 parser = OutputParser() |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
376 |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
377 for successRE in self.successREs: |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
378 parser.addMatcher(ValuesMatcher(successRE, {'passed' : '1'})) |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
379 for failureRE in self.failureREs: |
7564
c420a487b10f
changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution
Doug Simon <doug.simon@oracle.com>
parents:
7563
diff
changeset
|
380 parser.addMatcher(ValuesMatcher(failureRE, {'failed' : '1'})) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
381 for scoreMatcher in self.scoreMatchers: |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
382 parser.addMatcher(scoreMatcher) |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
383 |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
384 if self.benchmarkCompilationRate: |
7570
2025455e7d80
added collection of compilation speed metric for C1 and C2 to bench command
Doug Simon <doug.simon@oracle.com>
parents:
7569
diff
changeset
|
385 if vm == 'graal': |
2025455e7d80
added collection of compilation speed metric for C1 and C2 to bench command
Doug Simon <doug.simon@oracle.com>
parents:
7569
diff
changeset
|
386 bps = re.compile(r"ParsedBytecodesPerSecond@final: (?P<rate>[0-9]+)") |
2025455e7d80
added collection of compilation speed metric for C1 and C2 to bench command
Doug Simon <doug.simon@oracle.com>
parents:
7569
diff
changeset
|
387 ibps = re.compile(r"InlinedBytecodesPerSecond@final: (?P<rate>[0-9]+)") |
2025455e7d80
added collection of compilation speed metric for C1 and C2 to bench command
Doug Simon <doug.simon@oracle.com>
parents:
7569
diff
changeset
|
388 parser.addMatcher(ValuesMatcher(bps, {'group' : 'ParsedBytecodesPerSecond', 'name' : self.name, 'score' : '<rate>'})) |
2025455e7d80
added collection of compilation speed metric for C1 and C2 to bench command
Doug Simon <doug.simon@oracle.com>
parents:
7569
diff
changeset
|
389 parser.addMatcher(ValuesMatcher(ibps, {'group' : 'InlinedBytecodesPerSecond', 'name' : self.name, 'score' : '<rate>'})) |
2025455e7d80
added collection of compilation speed metric for C1 and C2 to bench command
Doug Simon <doug.simon@oracle.com>
parents:
7569
diff
changeset
|
390 else: |
2025455e7d80
added collection of compilation speed metric for C1 and C2 to bench command
Doug Simon <doug.simon@oracle.com>
parents:
7569
diff
changeset
|
391 ibps = re.compile(r"(?P<compiler>[\w]+) compilation speed: +(?P<rate>[0-9]+) bytes/s {standard") |
2025455e7d80
added collection of compilation speed metric for C1 and C2 to bench command
Doug Simon <doug.simon@oracle.com>
parents:
7569
diff
changeset
|
392 parser.addMatcher(ValuesMatcher(ibps, {'group' : 'InlinedBytecodesPerSecond', 'name' : '<compiler>:' + self.name, 'score' : '<rate>'})) |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
393 |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
394 startDelim = 'START: ' + self.name |
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
395 endDelim = 'END: ' + self.name |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
396 |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
397 outputfile = os.environ.get('BENCH_OUTPUT', None) |
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
398 if outputfile: |
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
399 # Used only to debug output parsing |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
400 with open(outputfile) as fp: |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
401 output = fp.read() |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
402 start = output.find(startDelim) |
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
403 end = output.find(endDelim, start) |
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
404 if start == -1 and end == -1: |
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
405 return {} |
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
406 output = output[start + len(startDelim + os.linesep): end] |
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
407 mx.log(startDelim) |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
408 mx.log(output) |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
409 mx.log(endDelim) |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
410 else: |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
411 tee = Tee() |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
412 mx.log(startDelim) |
12769
99769479f9ce
s/commands/mx_graal/ in mx/sanitycheck.py
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
12647
diff
changeset
|
413 if mx_graal.vm(self.vmOpts + _noneAsEmptyList(extraVmOpts) + self.cmd, vm, nonZeroIsFatal=False, out=tee.eat, err=subprocess.STDOUT, cwd=cwd, vmbuild=vmbuild) != 0: |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
414 mx.abort("Benchmark failed (non-zero retcode)") |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
415 mx.log(endDelim) |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
416 output = tee.output.getvalue() |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
417 |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
418 groups = {} |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
419 passed = False |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
420 for valueMap in parser.parse(output): |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
421 assert (valueMap.has_key('name') and valueMap.has_key('score') and valueMap.has_key('group')) or valueMap.has_key('passed') or valueMap.has_key('failed'), valueMap |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
422 if valueMap.get('failed') == '1': |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
423 mx.abort("Benchmark failed") |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
424 if valueMap.get('passed') == '1': |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
425 passed = True |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
426 groupName = valueMap.get('group') |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
427 if groupName: |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
428 group = groups.setdefault(groupName, {}) |
7563
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
429 name = valueMap.get('name') |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
430 score = valueMap.get('score') |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
431 if name and score: |
3aab15f42934
moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents:
7562
diff
changeset
|
432 group[name] = score |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
433 |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
434 if not passed: |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
435 mx.abort("Benchmark failed (not passed)") |
11514
dc3c8df55905
added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents:
11367
diff
changeset
|
436 |
7567
a8bc60aeacb8
fix bug in parsing of SPECjvm2008 output
Doug Simon <doug.simon@oracle.com>
parents:
7564
diff
changeset
|
437 return groups |