Mercurial > hg > graal-compiler

annotate mx/sanitycheck.py @ 4582:b24386206122

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal). Made HotSpot build jobs use the number of CPUs on the host machine.
author Doug Simon <doug.simon@oracle.com>
date Mon, 13 Feb 2012 23:13:37 +0100
parents 7903b6c28f9c
children 77a997c2bf4e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
1 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
2 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
5 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
6 # This code is free software; you can redistribute it and/or modify it
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
7 # under the terms of the GNU General Public License version 2 only, as
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
8 # published by the Free Software Foundation.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
9 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
10 # This code is distributed in the hope that it will be useful, but WITHOUT
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
13 # version 2 for more details (a copy is included in the LICENSE file that
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
14 # accompanied this code).
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
15 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
16 # You should have received a copy of the GNU General Public License version
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
17 # 2 along with this work; if not, write to the Free Software Foundation,
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
19 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
21 # or visit www.oracle.com if you need additional information or have any
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
22 # questions.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
23 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
24 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
25
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
26 from outputparser import OutputParser, Matcher
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
27 import re
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
28 import mx
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
29 import os
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
30 from os.path import isfile, join, exists
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
31
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
32 dacapoSanityWarmup = {
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
33 'avrora': [0, 0, 3, 6, 13],
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
34 'batik': [0, 0, 5, 5, 20],
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
35 'eclipse': [2, 4, 5, 10, 16],
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
36 'fop': [4, 8, 10, 20, 30],
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
37 'h2': [0, 0, 5, 5, 8],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
38 'jython': [0, 0, 5, 10, 13],
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
39 'luindex': [0, 0, 5, 10, 10],
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
40 'lusearch': [0, 4, 5, 5, 8],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
41 'pmd': [0, 0, 5, 10, 13],
4482
7903b6c28f9c Add hsdis command to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4480
diff changeset
42 'sunflow': [0, 2, 5, 10, 15],
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
43 'tomcat': [0, 0, 5, 10, 15],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
44 'tradebeans': [0, 0, 5, 10, 13],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
45 'tradesoap': [2, 4, 5, 10, 15],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
46 'xalan': [0, 0, 5, 10, 18],
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
47 }
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
48
4480
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
49 dacapoScalaSanityWarmup = {
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
50 'actors': [0, 0, 2, 8, 10],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
51 'apparat': [0, 0, 1, 2, 3],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
52 'factorie': [0, 0, 2, 5, 5],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
53 'kiama': [0, 0, 3, 13, 15],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
54 'scalac': [0, 0, 5, 15, 20],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
55 'scaladoc': [0, 0, 5, 15, 15],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
56 'scalap': [0, 0, 5, 15, 20],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
57 'scalariform':[0, 0, 6, 15, 20],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
58 'scalatest': [0, 0, 2, 10, 12],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
59 'scalaxb': [0, 0, 5, 15, 25],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
60 'specs': [0, 0, 3, 13, 18],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
61 'tmt': [0, 0, 3, 10, 12]
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
62 }
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
63
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
64 dacapoGateBuildLevels = {
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
65 'avrora': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
66 'batik': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
67 'eclipse': ['product'],
4284
1339f1817d28 gate : do not run tradesoap in fastdebug mode, run fop in fastdebug mode only
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4282
diff changeset
68 'fop': [ 'fastdebug', 'debug'],
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
69 'h2': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
70 'jython': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
71 'luindex': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
72 'lusearch': ['product'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
73 'pmd': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
74 'sunflow': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
75 'tomcat': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
76 'tradebeans': ['product', 'fastdebug', 'debug'],
4284
1339f1817d28 gate : do not run tradesoap in fastdebug mode, run fop in fastdebug mode only
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4282
diff changeset
77 'tradesoap': ['product'],
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
78 'xalan': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
79 }
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
80
4480
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
81 dacapoScalaGateBuildLevels = {
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
82 'actors': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
83 'apparat': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
84 'factorie': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
85 'kiama': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
86 'scalac': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
87 'scaladoc': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
88 'scalap': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
89 'scalariform':['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
90 'scalatest': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
91 'scalaxb': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
92 'specs': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
93 'tmt': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
94 }
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
95
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
96 class SanityCheckLevel:
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
97 Fast, Gate, Normal, Extensive, Benchmark = range(5)
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
98
4334
0893aef10ed4 fix for SPECjvm2008 bench
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4332
diff changeset
99 def getSPECjvm2008(benchArgs = [], skipKitValidation=False, warmupTime=None, iterationTime=None):
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
100
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
101 specjvm2008 = mx.get_env('SPECJVM2008')
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
102 if specjvm2008 is None or not exists(join(specjvm2008, 'SPECjvm2008.jar')):
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
103 mx.abort('Please set the SPECJVM2008 environment variable to a SPECjvm2008 directory')
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
104
4335
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4334
diff changeset
105 score = re.compile(r"^(Score on|Noncompliant) (?P<benchmark>[a-zA-Z0-9\._]+)( result)?: (?P<score>[0-9]+(,|\.)[0-9]+)( SPECjvm2008 Base)? ops/m$")
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
106 error = re.compile(r"^Errors in benchmark: ")
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
107 # The ' ops/m' at the end of the success string is important : it's how you can tell valid and invalid runs apart
4335
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4334
diff changeset
108 success = re.compile(r"^(Noncompliant c|C)omposite result: [0-9]+(,|\.)[0-9]+( SPECjvm2008 (Base|Peak))? ops/m$")
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
109 matcher = Matcher(score, {'const:name' : 'benchmark', 'const:score' : 'score'}, startNewLine=True)
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
110
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
111 opts = []
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
112 if warmupTime is not None:
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
113 opts += ['-wt', str(warmupTime)]
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
114 if iterationTime is not None:
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
115 opts += ['-it', str(iterationTime)]
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
116 if skipKitValidation:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
117 opts += ['-ikv']
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
118
4329
4aacce9c9cb9 Add posibility to give arguments for a SPECjvm2008 run
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4286
diff changeset
119 return Test("SPECjvm2008", "SPECjvm2008", ['-jar', 'SPECjvm2008.jar'] + opts + benchArgs, [success], [error], [matcher], vmOpts=['-Xms3g'], defaultCwd=specjvm2008)
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
120
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
121 def getDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=[]):
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
122 checks = []
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
123
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
124 for (bench, ns) in dacapoSanityWarmup.items():
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
125 if ns[level] > 0:
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
126 if gateBuildLevel is None or gateBuildLevel in dacapoGateBuildLevels[bench]:
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
127 checks.append(getDacapo(bench, ns[level], dacapoArgs))
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
128
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
129 return checks
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
130
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
131 def getDacapo(name, n, dacapoArgs=[]):
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
132 dacapo = mx.get_env('DACAPO_CP')
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
133 if dacapo is None:
4244
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
134 l = mx.library('DACAPO', False)
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
135 if l is not None:
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
136 dacapo = l.get_path(True)
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
137 else:
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
138 mx.abort('DaCapo 9.12 jar file must be specified with DACAPO_CP environment variable or as DACAPO library')
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
139
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
140 if not isfile(dacapo) or not dacapo.endswith('.jar'):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
141 mx.abort('Specified DaCapo jar file does not exist or is not a jar file: ' + dacapo)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
142
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
143 dacapoSuccess = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====$")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
144 dacapoFail = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) FAILED (warmup|) =====$")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
145 dacapoTime = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====")
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
146
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
147 dacapoMatcher = Matcher(dacapoTime, {'const:name' : 'benchmark', 'const:score' : 'time'})
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
148
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
149 return Test("DaCapo-" + name, "DaCapo", ['-jar', dacapo, name, '-n', str(n), ] + dacapoArgs, [dacapoSuccess], [dacapoFail], [dacapoMatcher], ['-Xms2g', '-XX:MaxPermSize=256m'])
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
150
4480
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
151 def getScalaDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=[]):
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
152 checks = []
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
153
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
154 for (bench, ns) in dacapoScalaSanityWarmup.items():
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
155 if ns[level] > 0:
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
156 if gateBuildLevel is None or gateBuildLevel in dacapoScalaGateBuildLevels[bench]:
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
157 checks.append(getScalaDacapo(bench, ns[level], dacapoArgs))
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
158
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
159 return checks
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
160
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
161 def getScalaDacapo(name, n, dacapoArgs=[]):
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
162 dacapo = mx.get_env('DACAPO_SCALA_CP')
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
163 if dacapo is None:
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
164 l = mx.library('DACAPO_SCALA', False)
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
165 if l is not None:
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
166 dacapo = l.get_path(True)
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
167 else:
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
168 mx.abort('Scala DaCapo 0.1.0 jar file must be specified with DACAPO_SCALA_CP environment variable or as DACAPO_SCALA library')
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
169
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
170 if not isfile(dacapo) or not dacapo.endswith('.jar'):
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
171 mx.abort('Specified Scala DaCapo jar file does not exist or is not a jar file: ' + dacapo)
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
172
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
173 dacapoSuccess = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====$")
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
174 dacapoFail = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) FAILED (warmup|) =====$")
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
175 dacapoTime = re.compile(r"===== DaCapo 0\.1\.0(-SNAPSHOT)? (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====")
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
176
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
177 dacapoMatcher = Matcher(dacapoTime, {'const:name' : 'benchmark', 'const:score' : 'time'})
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
178
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
179 return Test("Scala-DaCapo-" + name, "Scala-DaCapo", ['-jar', dacapo, name, '-n', str(n), ] + dacapoArgs, [dacapoSuccess], [dacapoFail], [dacapoMatcher], ['-Xms2g', '-XX:MaxPermSize=256m'])
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4416
diff changeset
180
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
181 def getBootstraps():
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4244
diff changeset
182 time = re.compile(r"Bootstrapping Graal\.+ in (?P<time>[0-9]+) ms")
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
183 scoreMatcher = Matcher(time, {'const:name' : 'const:BootstrapTime', 'const:score' : 'time'})
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
184 tests = []
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
185 tests.append(Test("Bootstrap", "Bootstrap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher]))
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
186 tests.append(Test("Bootstrap-bigHeap", "Bootstrap-bigHeap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher], vmOpts=['-Xms2g']))
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
187 return tests
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
188
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
189 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
190 Encapsulates a single program that is a sanity test and/or a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
191 """
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
192 class Test:
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
193 def __init__(self, name, group, cmd, successREs=[], failureREs=[], scoreMatchers=[], vmOpts=[], defaultCwd=None):
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
194 self.name = name
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
195 self.group = group
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
196 self.successREs = successREs
4416
be787de79394 Fail bench/test on "Exception occured in scope: "
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4335
diff changeset
197 self.failureREs = failureREs + [re.compile(r"Exception occured in scope: ")]
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
198 self.scoreMatchers = scoreMatchers
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
199 self.vmOpts = vmOpts
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
200 self.cmd = cmd
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
201 self.defaultCwd = defaultCwd
4238
e0d09e05aa9b Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
202
e0d09e05aa9b Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
203 def __str__(self):
e0d09e05aa9b Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
204 return self.name
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
205
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
206 def test(self, vm, cwd=None, opts=[], vmbuild=None):
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
207 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
208 Run this program as a sanity test.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
209 """
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
210 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
211 cwd = self.defaultCwd
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
212 parser = OutputParser(nonZeroIsFatal = False)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
213 jvmError = re.compile(r"(?P<jvmerror>([A-Z]:|/).*[/\\]hs_err_pid[0-9]+\.log)")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
214 parser.addMatcher(Matcher(jvmError, {'const:jvmError' : 'jvmerror'}))
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
215
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
216 for successRE in self.successREs:
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
217 parser.addMatcher(Matcher(successRE, {'const:passed' : 'const:1'}))
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
218 for failureRE in self.failureREs:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
219 parser.addMatcher(Matcher(failureRE, {'const:failed' : 'const:1'}))
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
220
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
221 result = parser.parse(vm, self.vmOpts + opts + self.cmd, cwd, vmbuild)
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
222
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
223 parsedLines = result['parsed']
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
224 if len(parsedLines) == 0:
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
225 return False
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
226
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
227 assert len(parsedLines) == 1, 'Test matchers should not return more than one line'
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
228
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
229 parsed = parsedLines[0]
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
230
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
231 if parsed.has_key('jvmError'):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
232 mx.log('/!\\JVM Error : dumping error log...')
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
233 f = open(parsed['jvmError'], 'rb');
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
234 for line in iter(f.readline, ''):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
235 mx.log(line.rstrip())
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
236 f.close()
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
237 os.unlink(parsed['jvmError'])
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
238 return False
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
239
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
240 if parsed.has_key('failed') and parsed['failed'] is '1':
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
241 return False
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
242
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
243 return result['retcode'] is 0 and parsed.has_key('passed') and parsed['passed'] is '1'
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
244
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
245 def bench(self, vm, cwd=None, opts=[], vmbuild=None):
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
246 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
247 Run this program as a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
248 """
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
249 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
250 cwd = self.defaultCwd
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
251 parser = OutputParser(nonZeroIsFatal = False)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
252
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
253 for successRE in self.successREs:
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
254 parser.addMatcher(Matcher(successRE, {'const:passed' : 'const:1'}))
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
255 for failureRE in self.failureREs:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
256 parser.addMatcher(Matcher(failureRE, {'const:failed' : 'const:1'}))
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
257 for scoreMatcher in self.scoreMatchers:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
258 parser.addMatcher(scoreMatcher)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
259
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
260 result = parser.parse(vm, self.vmOpts + opts + self.cmd, cwd, vmbuild)
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
261 if result['retcode'] is not 0:
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
262 mx.abort("Benchmark failed (non-zero retcode)")
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
263
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
264 parsed = result['parsed']
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
265
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
266 ret = {}
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
267
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
268 passed = False;
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
269
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
270 for line in parsed:
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
271 assert (line.has_key('name') and line.has_key('score')) or line.has_key('passed')
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
272 if line.has_key('failed') and line['failed'] is '1':
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
273 mx.abort("Benchmark failed")
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
274 if line.has_key('passed') and line['passed'] is '1':
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
275 passed = True
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
276 ret[line['name']] = line['score']
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
277
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
278 if not passed:
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
279 mx.abort("Benchmark failed (not passed)")
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
280
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
281 return ret