Mercurial > hg > truffle

annotate mx/sanitycheck.py @ 4477:27c5466afd55

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Changes to debug framework. Intercept not only RuntimeException but Throwable. Support for named IGV groups through new DebugDumpScope. Printer bugfix. Open scope with method context in VMToCompilerImpl (was in GraalCompiler).
author Andreas Woess <andreas.woess@jku.at>
date Tue, 07 Feb 2012 12:35:22 +0100
parents be787de79394
children 7d6490436b57
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
1 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
2 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
5 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
6 # This code is free software; you can redistribute it and/or modify it
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
7 # under the terms of the GNU General Public License version 2 only, as
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
8 # published by the Free Software Foundation.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
9 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
10 # This code is distributed in the hope that it will be useful, but WITHOUT
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
13 # version 2 for more details (a copy is included in the LICENSE file that
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
14 # accompanied this code).
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
15 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
16 # You should have received a copy of the GNU General Public License version
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
17 # 2 along with this work; if not, write to the Free Software Foundation,
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
19 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
21 # or visit www.oracle.com if you need additional information or have any
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
22 # questions.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
23 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
24 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
25
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
26 from outputparser import OutputParser, Matcher
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
27 import re
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
28 import mx
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
29 import os
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
30 from os.path import isfile, join, exists
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
31
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
32 dacapoSanityWarmup = {
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
33 'avrora': [0, 0, 3, 6, 13],
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
34 'batik': [0, 0, 5, 5, 20],
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
35 'eclipse': [2, 4, 5, 10, 16],
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
36 'fop': [4, 8, 10, 20, 30],
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
37 'h2': [0, 0, 5, 5, 8],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
38 'jython': [0, 0, 5, 10, 13],
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
39 'luindex': [0, 0, 5, 10, 10],
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
40 'lusearch': [0, 4, 5, 5, 8],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
41 'pmd': [0, 0, 5, 10, 13],
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
42 'sunflow': [0, 0, 5, 10, 15],
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
43 'tomcat': [0, 0, 5, 10, 15],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
44 'tradebeans': [0, 0, 5, 10, 13],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
45 'tradesoap': [2, 4, 5, 10, 15],
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
46 'xalan': [0, 0, 5, 10, 18],
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
47 }
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
48
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
49 dacapoGateBuildLevels = {
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
50 'avrora': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
51 'batik': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
52 'eclipse': ['product'],
4284
1339f1817d28 gate : do not run tradesoap in fastdebug mode, run fop in fastdebug mode only
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4282
diff changeset
53 'fop': [ 'fastdebug', 'debug'],
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
54 'h2': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
55 'jython': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
56 'luindex': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
57 'lusearch': ['product'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
58 'pmd': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
59 'sunflow': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
60 'tomcat': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
61 'tradebeans': ['product', 'fastdebug', 'debug'],
4284
1339f1817d28 gate : do not run tradesoap in fastdebug mode, run fop in fastdebug mode only
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4282
diff changeset
62 'tradesoap': ['product'],
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
63 'xalan': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
64 }
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
65
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
66 class SanityCheckLevel:
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
67 Fast, Gate, Normal, Extensive, Benchmark = range(5)
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
68
4334
0893aef10ed4 fix for SPECjvm2008 bench
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4332
diff changeset
69 def getSPECjvm2008(benchArgs = [], skipKitValidation=False, warmupTime=None, iterationTime=None):
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
70
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
71 specjvm2008 = mx.get_env('SPECJVM2008')
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
72 if specjvm2008 is None or not exists(join(specjvm2008, 'SPECjvm2008.jar')):
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
73 mx.abort('Please set the SPECJVM2008 environment variable to a SPECjvm2008 directory')
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
74
4335
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4334
diff changeset
75 score = re.compile(r"^(Score on|Noncompliant) (?P<benchmark>[a-zA-Z0-9\._]+)( result)?: (?P<score>[0-9]+(,|\.)[0-9]+)( SPECjvm2008 Base)? ops/m$")
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
76 error = re.compile(r"^Errors in benchmark: ")
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
77 # The ' ops/m' at the end of the success string is important : it's how you can tell valid and invalid runs apart
4335
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4334
diff changeset
78 success = re.compile(r"^(Noncompliant c|C)omposite result: [0-9]+(,|\.)[0-9]+( SPECjvm2008 (Base|Peak))? ops/m$")
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
79 matcher = Matcher(score, {'const:name' : 'benchmark', 'const:score' : 'score'}, startNewLine=True)
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
80
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
81 opts = []
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
82 if warmupTime is not None:
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
83 opts += ['-wt', str(warmupTime)]
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
84 if iterationTime is not None:
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
85 opts += ['-it', str(iterationTime)]
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
86 if skipKitValidation:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
87 opts += ['-ikv']
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
88
4329
4aacce9c9cb9 Add posibility to give arguments for a SPECjvm2008 run
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4286
diff changeset
89 return Test("SPECjvm2008", "SPECjvm2008", ['-jar', 'SPECjvm2008.jar'] + opts + benchArgs, [success], [error], [matcher], vmOpts=['-Xms3g'], defaultCwd=specjvm2008)
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
90
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
91 def getDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=[]):
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
92 checks = []
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
93
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
94 for (bench, ns) in dacapoSanityWarmup.items():
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
95 if ns[level] > 0:
4240
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
96 if gateBuildLevel is None or gateBuildLevel in dacapoGateBuildLevels[bench]:
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4238
diff changeset
97 checks.append(getDacapo(bench, ns[level], dacapoArgs))
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
98
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
99 return checks
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
100
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
101 def getDacapo(name, n, dacapoArgs=[]):
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
102 dacapo = mx.get_env('DACAPO_CP')
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
103 if dacapo is None:
4244
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
104 l = mx.library('DACAPO', False)
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
105 if l is not None:
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
106 dacapo = l.get_path(True)
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
107 else:
a69889e5a8a5 Made the DaCapo library be downloaded only on demand.
Doug Simon <doug.simon@oracle.com>
parents: 4240
diff changeset
108 mx.abort('DaCapo 9.12 jar file must be specified with DACAPO_CP environment variable or as DACAPO library')
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
109
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
110 if not isfile(dacapo) or not dacapo.endswith('.jar'):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
111 mx.abort('Specified DaCapo jar file does not exist or is not a jar file: ' + dacapo)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
112
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
113 dacapoSuccess = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====$")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
114 dacapoFail = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) FAILED (warmup|) =====$")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
115 dacapoTime = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====")
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
116
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
117 dacapoMatcher = Matcher(dacapoTime, {'const:name' : 'benchmark', 'const:score' : 'time'})
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
118
4286
76190108a9f2 Tweak some benchmark settings
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4284
diff changeset
119 return Test("DaCapo-" + name, "DaCapo", ['-jar', dacapo, name, '-n', str(n), ] + dacapoArgs, [dacapoSuccess], [dacapoFail], [dacapoMatcher], ['-Xms2g', '-XX:MaxPermSize=256m'])
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
120
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
121 def getBootstraps():
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4244
diff changeset
122 time = re.compile(r"Bootstrapping Graal\.+ in (?P<time>[0-9]+) ms")
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
123 scoreMatcher = Matcher(time, {'const:name' : 'const:BootstrapTime', 'const:score' : 'time'})
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
124 tests = []
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
125 tests.append(Test("Bootstrap", "Bootstrap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher]))
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
126 tests.append(Test("Bootstrap-bigHeap", "Bootstrap-bigHeap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher], vmOpts=['-Xms2g']))
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
127 return tests
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
128
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
129 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
130 Encapsulates a single program that is a sanity test and/or a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
131 """
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
132 class Test:
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
133 def __init__(self, name, group, cmd, successREs=[], failureREs=[], scoreMatchers=[], vmOpts=[], defaultCwd=None):
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
134 self.name = name
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
135 self.group = group
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
136 self.successREs = successREs
4416
be787de79394 Fail bench/test on "Exception occured in scope: "
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4335
diff changeset
137 self.failureREs = failureREs + [re.compile(r"Exception occured in scope: ")]
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
138 self.scoreMatchers = scoreMatchers
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
139 self.vmOpts = vmOpts
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
140 self.cmd = cmd
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
141 self.defaultCwd = defaultCwd
4238
e0d09e05aa9b Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
142
e0d09e05aa9b Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
143 def __str__(self):
e0d09e05aa9b Made the gate time each DaCapo test individually.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
144 return self.name
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
145
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
146 def test(self, vm, cwd=None, opts=[], vmbuild=None):
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
147 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
148 Run this program as a sanity test.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
149 """
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
150 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
151 cwd = self.defaultCwd
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
152 parser = OutputParser(nonZeroIsFatal = False)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
153 jvmError = re.compile(r"(?P<jvmerror>([A-Z]:|/).*[/\\]hs_err_pid[0-9]+\.log)")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
154 parser.addMatcher(Matcher(jvmError, {'const:jvmError' : 'jvmerror'}))
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
155
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
156 for successRE in self.successREs:
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
157 parser.addMatcher(Matcher(successRE, {'const:passed' : 'const:1'}))
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
158 for failureRE in self.failureREs:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
159 parser.addMatcher(Matcher(failureRE, {'const:failed' : 'const:1'}))
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
160
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
161 result = parser.parse(vm, self.vmOpts + opts + self.cmd, cwd, vmbuild)
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
162
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
163 parsedLines = result['parsed']
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
164 if len(parsedLines) == 0:
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
165 return False
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
166
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
167 assert len(parsedLines) == 1, 'Test matchers should not return more than one line'
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
168
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
169 parsed = parsedLines[0]
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
170
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
171 if parsed.has_key('jvmError'):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
172 mx.log('/!\\JVM Error : dumping error log...')
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
173 f = open(parsed['jvmError'], 'rb');
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
174 for line in iter(f.readline, ''):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
175 mx.log(line.rstrip())
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
176 f.close()
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
177 os.unlink(parsed['jvmError'])
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
178 return False
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
179
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
180 if parsed.has_key('failed') and parsed['failed'] is '1':
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
181 return False
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
182
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
183 return result['retcode'] is 0 and parsed.has_key('passed') and parsed['passed'] is '1'
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
184
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
185 def bench(self, vm, cwd=None, opts=[], vmbuild=None):
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
186 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
187 Run this program as a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
188 """
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
189 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
190 cwd = self.defaultCwd
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
191 parser = OutputParser(nonZeroIsFatal = False)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
192
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
193 for successRE in self.successREs:
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
194 parser.addMatcher(Matcher(successRE, {'const:passed' : 'const:1'}))
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
195 for failureRE in self.failureREs:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
196 parser.addMatcher(Matcher(failureRE, {'const:failed' : 'const:1'}))
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
197 for scoreMatcher in self.scoreMatchers:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
198 parser.addMatcher(scoreMatcher)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
199
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
200 result = parser.parse(vm, self.vmOpts + opts + self.cmd, cwd, vmbuild)
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
201 if result['retcode'] is not 0:
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
202 mx.abort("Benchmark failed (non-zero retcode)")
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
203
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
204 parsed = result['parsed']
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
205
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
206 ret = {}
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
207
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
208 passed = False;
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
209
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
210 for line in parsed:
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
211 assert (line.has_key('name') and line.has_key('score')) or line.has_key('passed')
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
212 if line.has_key('failed') and line['failed'] is '1':
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
213 mx.abort("Benchmark failed")
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
214 if line.has_key('passed') and line['passed'] is '1':
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
215 passed = True
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
216 ret[line['name']] = line['score']
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
217
4332
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
218 if not passed:
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
219 mx.abort("Benchmark failed (not passed)")
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4331
diff changeset
220
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
221 return ret