Mercurial > hg > graal-compiler
annotate mx/sanitycheck.py @ 5813:3b8bc07f8d17
Add ability to give an existing to getDebugProperties
Add nameTemplate to NodeInfo, use it in binary graphs
improve binary graph protocol (transmit NodeClass through the constant pool including nameTemplate and edges names, transmit edges as part of nodes and blocks)
author | Gilles Duboscq <duboscq@ssw.jku.at> |
---|---|
date | Tue, 10 Jul 2012 15:02:41 +0200 |
parents | beb13d0326ba |
children | 0e54d9bb922d |
rev | line source |
---|---|
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
1 # ---------------------------------------------------------------------------------------------------- |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
2 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
5 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
6 # This code is free software; you can redistribute it and/or modify it |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
7 # under the terms of the GNU General Public License version 2 only, as |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
8 # published by the Free Software Foundation. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
9 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
10 # This code is distributed in the hope that it will be useful, but WITHOUT |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
13 # version 2 for more details (a copy is included in the LICENSE file that |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
14 # accompanied this code). |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
15 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
16 # You should have received a copy of the GNU General Public License version |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
17 # 2 along with this work; if not, write to the Free Software Foundation, |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
19 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
21 # or visit www.oracle.com if you need additional information or have any |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
22 # questions. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
23 # |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
24 # ---------------------------------------------------------------------------------------------------- |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
25 |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
26 from outputparser import OutputParser, Matcher |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
27 import re |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
28 import mx |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
29 import os |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
30 from os.path import isfile, join, exists |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
31 |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
32 dacapoSanityWarmup = { |
4286
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
33 'avrora': [0, 0, 3, 6, 13], |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
34 'batik': [0, 0, 5, 5, 20], |
4286
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
35 'eclipse': [2, 4, 5, 10, 16], |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
36 'fop': [4, 8, 10, 20, 30], |
4286
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
37 'h2': [0, 0, 5, 5, 8], |
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
38 'jython': [0, 0, 5, 10, 13], |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
39 'luindex': [0, 0, 5, 10, 10], |
4286
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
40 'lusearch': [0, 4, 5, 5, 8], |
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
41 'pmd': [0, 0, 5, 10, 13], |
4482
7903b6c28f9c
Add hsdis command to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4480
diff
changeset
|
42 'sunflow': [0, 2, 5, 10, 15], |
4286
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
43 'tomcat': [0, 0, 5, 10, 15], |
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
44 'tradebeans': [0, 0, 5, 10, 13], |
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
45 'tradesoap': [2, 4, 5, 10, 15], |
76190108a9f2
Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4284
diff
changeset
|
46 'xalan': [0, 0, 5, 10, 18], |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
47 } |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
48 |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
49 dacapoScalaSanityWarmup = { |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
50 'actors': [0, 0, 2, 8, 10], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
51 'apparat': [0, 0, 1, 2, 3], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
52 'factorie': [0, 0, 2, 5, 5], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
53 'kiama': [0, 0, 3, 13, 15], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
54 'scalac': [0, 0, 5, 15, 20], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
55 'scaladoc': [0, 0, 5, 15, 15], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
56 'scalap': [0, 0, 5, 15, 20], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
57 'scalariform':[0, 0, 6, 15, 20], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
58 'scalatest': [0, 0, 2, 10, 12], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
59 'scalaxb': [0, 0, 5, 15, 25], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
60 'specs': [0, 0, 3, 13, 18], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
61 'tmt': [0, 0, 3, 10, 12] |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
62 } |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
63 |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
64 dacapoGateBuildLevels = { |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
65 'avrora': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
66 'batik': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
67 'eclipse': ['product'], |
4284
1339f1817d28
gate : do not run tradesoap in fastdebug mode, run fop in fastdebug mode only
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4282
diff
changeset
|
68 'fop': [ 'fastdebug', 'debug'], |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
69 'h2': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
70 'jython': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
71 'luindex': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
72 'lusearch': ['product'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
73 'pmd': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
74 'sunflow': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
75 'tomcat': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
76 'tradebeans': ['product', 'fastdebug', 'debug'], |
4663
77a997c2bf4e
Added fastdebug version of tradesoap to gate check.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
4482
diff
changeset
|
77 'tradesoap': ['product', 'fastdebug'], |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
78 'xalan': ['product', 'fastdebug', 'debug'], |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
79 } |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
80 |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
81 dacapoScalaGateBuildLevels = { |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
82 'actors': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
83 'apparat': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
84 'factorie': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
85 'kiama': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
86 'scalac': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
87 'scaladoc': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
88 'scalap': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
89 'scalariform':['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
90 'scalatest': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
91 'scalaxb': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
92 'specs': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
93 'tmt': ['product', 'fastdebug', 'debug'], |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
94 } |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
95 |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
96 class SanityCheckLevel: |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
97 Fast, Gate, Normal, Extensive, Benchmark = range(5) |
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
98 |
4334
0893aef10ed4
fix for SPECjvm2008 bench
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4332
diff
changeset
|
99 def getSPECjvm2008(benchArgs = [], skipKitValidation=False, warmupTime=None, iterationTime=None): |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
100 |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
101 specjvm2008 = mx.get_env('SPECJVM2008') |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
102 if specjvm2008 is None or not exists(join(specjvm2008, 'SPECjvm2008.jar')): |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
103 mx.abort('Please set the SPECJVM2008 environment variable to a SPECjvm2008 directory') |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
104 |
4335
0d661161fa24
Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4334
diff
changeset
|
105 score = re.compile(r"^(Score on|Noncompliant) (?P<benchmark>[a-zA-Z0-9\._]+)( result)?: (?P<score>[0-9]+(,|\.)[0-9]+)( SPECjvm2008 Base)? ops/m$") |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
106 error = re.compile(r"^Errors in benchmark: ") |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
107 # The ' ops/m' at the end of the success string is important : it's how you can tell valid and invalid runs apart |
4335
0d661161fa24
Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4334
diff
changeset
|
108 success = re.compile(r"^(Noncompliant c|C)omposite result: [0-9]+(,|\.)[0-9]+( SPECjvm2008 (Base|Peak))? ops/m$") |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
109 matcher = Matcher(score, {'const:group' : "const:SPECjvm2008", 'const:name' : 'benchmark', 'const:score' : 'score'}, startNewLine=True) |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
110 |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
111 opts = [] |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
112 if warmupTime is not None: |
4230
b780ecb920c9
Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4228
diff
changeset
|
113 opts += ['-wt', str(warmupTime)] |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
114 if iterationTime is not None: |
4230
b780ecb920c9
Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4228
diff
changeset
|
115 opts += ['-it', str(iterationTime)] |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
116 if skipKitValidation: |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
117 opts += ['-ikv'] |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
118 |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
119 return Test("SPECjvm2008", ['-jar', 'SPECjvm2008.jar'] + opts + benchArgs, [success], [error], [matcher], vmOpts=['-Xms3g', '-XX:+UseSerialGC'], defaultCwd=specjvm2008) |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
120 |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
121 def getDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=[]): |
4217
f3271682fe5a
Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4215
diff
changeset
|
122 checks = [] |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
123 |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
124 for (bench, ns) in dacapoSanityWarmup.items(): |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
125 if ns[level] > 0: |
4240
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
126 if gateBuildLevel is None or gateBuildLevel in dacapoGateBuildLevels[bench]: |
394424b7df1a
Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents:
4238
diff
changeset
|
127 checks.append(getDacapo(bench, ns[level], dacapoArgs)) |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
128 |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
129 return checks |
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
130 |
4217
f3271682fe5a
Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4215
diff
changeset
|
131 def getDacapo(name, n, dacapoArgs=[]): |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
132 dacapo = mx.get_env('DACAPO_CP') |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
133 if dacapo is None: |
4244
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
134 l = mx.library('DACAPO', False) |
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
135 if l is not None: |
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
136 dacapo = l.get_path(True) |
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
137 else: |
a69889e5a8a5
Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents:
4240
diff
changeset
|
138 mx.abort('DaCapo 9.12 jar file must be specified with DACAPO_CP environment variable or as DACAPO library') |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
139 |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
140 if not isfile(dacapo) or not dacapo.endswith('.jar'): |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
141 mx.abort('Specified DaCapo jar file does not exist or is not a jar file: ' + dacapo) |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
142 |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
143 dacapoSuccess = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====$") |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
144 dacapoFail = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) FAILED (warmup|) =====$") |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
145 dacapoTime = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====") |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
146 dacapoTime1 = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) completed warmup 1 in (?P<time>[0-9]+) msec =====") |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
147 |
5196
beb13d0326ba
make DaCapo benchmark matchers produce multiple result lines
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5195
diff
changeset
|
148 dacapoMatcher = Matcher(dacapoTime, {'const:group' : "const:DaCapo", 'const:name' : 'benchmark', 'const:score' : 'time'}, startNewLine=True) |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
149 dacapoMatcher1 = Matcher(dacapoTime1, {'const:group' : "const:DaCapo-1stRun", 'const:name' : 'benchmark', 'const:score' : 'time'}) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
150 |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
151 return Test("DaCapo-" + name, ['-jar', dacapo, name, '-n', str(n), ] + dacapoArgs, [dacapoSuccess], [dacapoFail], [dacapoMatcher, dacapoMatcher1], ['-Xms2g', '-XX:MaxPermSize=256m', '-XX:+UseSerialGC']) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
152 |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
153 def getScalaDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=[]): |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
154 checks = [] |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
155 |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
156 for (bench, ns) in dacapoScalaSanityWarmup.items(): |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
157 if ns[level] > 0: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
158 if gateBuildLevel is None or gateBuildLevel in dacapoScalaGateBuildLevels[bench]: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
159 checks.append(getScalaDacapo(bench, ns[level], dacapoArgs)) |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
160 |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
161 return checks |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
162 |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
163 def getScalaDacapo(name, n, dacapoArgs=[]): |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
164 dacapo = mx.get_env('DACAPO_SCALA_CP') |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
165 if dacapo is None: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
166 l = mx.library('DACAPO_SCALA', False) |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
167 if l is not None: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
168 dacapo = l.get_path(True) |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
169 else: |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
170 mx.abort('Scala DaCapo 0.1.0 jar file must be specified with DACAPO_SCALA_CP environment variable or as DACAPO_SCALA library') |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
171 |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
172 if not isfile(dacapo) or not dacapo.endswith('.jar'): |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
173 mx.abort('Specified Scala DaCapo jar file does not exist or is not a jar file: ' + dacapo) |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
174 |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
175 dacapoSuccess = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====$") |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
176 dacapoFail = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) FAILED (warmup|) =====$") |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
177 dacapoTime = re.compile(r"===== DaCapo 0\.1\.0(-SNAPSHOT)? (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====") |
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
178 |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
179 dacapoMatcher = Matcher(dacapoTime, {'const:group' : "const:Scala-DaCapo", 'const:name' : 'benchmark', 'const:score' : 'time'}) |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
180 |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
181 return Test("Scala-DaCapo-" + name, ['-jar', dacapo, name, '-n', str(n), ] + dacapoArgs, [dacapoSuccess], [dacapoFail], [dacapoMatcher], ['-Xms2g', '-XX:MaxPermSize=256m', '-XX:+UseSerialGC']) |
4480
7d6490436b57
Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4416
diff
changeset
|
182 |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
183 def getBootstraps(): |
4282
063ea022532c
mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4244
diff
changeset
|
184 time = re.compile(r"Bootstrapping Graal\.+ in (?P<time>[0-9]+) ms") |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
185 scoreMatcher = Matcher(time, {'const:group' : 'const:Bootstrap', 'const:name' : 'const:BootstrapTime', 'const:score' : 'time'}) |
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
186 scoreMatcherBig = Matcher(time, {'const:group' : 'const:Bootstrap-bigHeap', 'const:name' : 'const:BootstrapTime', 'const:score' : 'time'}) |
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
187 |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
188 tests = [] |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
189 tests.append(Test("Bootstrap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher], ingoreVms=['client', 'server'])) |
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
190 tests.append(Test("Bootstrap-bigHeap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcherBig], vmOpts=['-Xms2g'], ingoreVms=['client', 'server'])) |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
191 return tests |
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
192 |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
193 """ |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
194 Encapsulates a single program that is a sanity test and/or a benchmark. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
195 """ |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
196 class Test: |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
197 def __init__(self, name, cmd, successREs=[], failureREs=[], scoreMatchers=[], vmOpts=[], defaultCwd=None, ingoreVms=[]): |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
198 self.name = name |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
199 self.successREs = successREs |
4416
be787de79394
Fail bench/test on "Exception occured in scope: "
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4335
diff
changeset
|
200 self.failureREs = failureREs + [re.compile(r"Exception occured in scope: ")] |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
201 self.scoreMatchers = scoreMatchers |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
202 self.vmOpts = vmOpts |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
203 self.cmd = cmd |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
204 self.defaultCwd = defaultCwd |
5143
f5511facb897
Add posibility to ignore some tests/benchmarks with some vms : use it to disable bootstrap benchmarks for client/server
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5045
diff
changeset
|
205 self.ingoreVms = ingoreVms; |
f5511facb897
Add posibility to ignore some tests/benchmarks with some vms : use it to disable bootstrap benchmarks for client/server
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5045
diff
changeset
|
206 |
4238
e0d09e05aa9b
Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents:
4230
diff
changeset
|
207 |
e0d09e05aa9b
Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents:
4230
diff
changeset
|
208 def __str__(self): |
e0d09e05aa9b
Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents:
4230
diff
changeset
|
209 return self.name |
4150
c78bace5086a
start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff
changeset
|
210 |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
211 def test(self, vm, cwd=None, opts=[], vmbuild=None): |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
212 """ |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
213 Run this program as a sanity test. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
214 """ |
5143
f5511facb897
Add posibility to ignore some tests/benchmarks with some vms : use it to disable bootstrap benchmarks for client/server
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5045
diff
changeset
|
215 if (vm in self.ingoreVms): |
f5511facb897
Add posibility to ignore some tests/benchmarks with some vms : use it to disable bootstrap benchmarks for client/server
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5045
diff
changeset
|
216 return True; |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
217 if cwd is None: |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
218 cwd = self.defaultCwd |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
219 parser = OutputParser(nonZeroIsFatal = False) |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
220 jvmError = re.compile(r"(?P<jvmerror>([A-Z]:|/).*[/\\]hs_err_pid[0-9]+\.log)") |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
221 parser.addMatcher(Matcher(jvmError, {'const:jvmError' : 'jvmerror'})) |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
222 |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
223 for successRE in self.successREs: |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
224 parser.addMatcher(Matcher(successRE, {'const:passed' : 'const:1'})) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
225 for failureRE in self.failureREs: |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
226 parser.addMatcher(Matcher(failureRE, {'const:failed' : 'const:1'})) |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
227 |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
228 result = parser.parse(vm, self.vmOpts + opts + self.cmd, cwd, vmbuild) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
229 |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
230 parsedLines = result['parsed'] |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
231 if len(parsedLines) == 0: |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
232 return False |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
233 |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
234 assert len(parsedLines) == 1, 'Test matchers should not return more than one line' |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
235 |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
236 parsed = parsedLines[0] |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
237 |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
238 if parsed.has_key('jvmError'): |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
239 mx.log('/!\\JVM Error : dumping error log...') |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
240 f = open(parsed['jvmError'], 'rb'); |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
241 for line in iter(f.readline, ''): |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
242 mx.log(line.rstrip()) |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
243 f.close() |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
244 os.unlink(parsed['jvmError']) |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
245 return False |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
246 |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
247 if parsed.has_key('failed') and parsed['failed'] is '1': |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
248 return False |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
249 |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
250 return result['retcode'] is 0 and parsed.has_key('passed') and parsed['passed'] is '1' |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
251 |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
252 def bench(self, vm, cwd=None, opts=[], vmbuild=None): |
4225
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
253 """ |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
254 Run this program as a benchmark. |
339cf8d4904d
Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents:
4219
diff
changeset
|
255 """ |
5143
f5511facb897
Add posibility to ignore some tests/benchmarks with some vms : use it to disable bootstrap benchmarks for client/server
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5045
diff
changeset
|
256 if (vm in self.ingoreVms): |
f5511facb897
Add posibility to ignore some tests/benchmarks with some vms : use it to disable bootstrap benchmarks for client/server
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5045
diff
changeset
|
257 return {}; |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
258 if cwd is None: |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
259 cwd = self.defaultCwd |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
260 parser = OutputParser(nonZeroIsFatal = False) |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
261 |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
262 for successRE in self.successREs: |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
263 parser.addMatcher(Matcher(successRE, {'const:passed' : 'const:1'})) |
4228
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
264 for failureRE in self.failureREs: |
e872562f95f8
add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4225
diff
changeset
|
265 parser.addMatcher(Matcher(failureRE, {'const:failed' : 'const:1'})) |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
266 for scoreMatcher in self.scoreMatchers: |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
267 parser.addMatcher(scoreMatcher) |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
268 |
4219
47f7d91d34cf
Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4217
diff
changeset
|
269 result = parser.parse(vm, self.vmOpts + opts + self.cmd, cwd, vmbuild) |
4217
f3271682fe5a
Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4215
diff
changeset
|
270 if result['retcode'] is not 0: |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
271 mx.abort("Benchmark failed (non-zero retcode)") |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
272 |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
273 parsed = result['parsed'] |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
274 |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
275 ret = {} |
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
276 |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
277 passed = False; |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
278 |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
279 for line in parsed: |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
280 assert (line.has_key('name') and line.has_key('score') and line.has_key('group')) or line.has_key('passed') or line.has_key('failed') |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
281 if line.has_key('failed') and line['failed'] is '1': |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
282 mx.abort("Benchmark failed") |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
283 if line.has_key('passed') and line['passed'] is '1': |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
284 passed = True |
5195
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
285 if line.has_key('name') and line.has_key('score') and line.has_key('group'): |
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
286 if not ret.has_key(line['group']): |
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
287 ret[line['group']] = {}; |
8125c1f6472b
Add DaCapo 1st run as a benchmark group
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
5148
diff
changeset
|
288 ret[line['group']][line['name']] = line['score'] |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
289 |
4332
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
290 if not passed: |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
291 mx.abort("Benchmark failed (not passed)") |
9dbcdd6f1464
If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4331
diff
changeset
|
292 |
4215
a2caa019ba3a
Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
4150
diff
changeset
|
293 return ret |