Mercurial > hg > truffle

annotate mx/sanitycheck.py @ 4230:b780ecb920c9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
author Gilles Duboscq <gilles.m.duboscq@gmail.com>
date Thu, 05 Jan 2012 20:15:45 +0100
parents e872562f95f8
children e0d09e05aa9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
1 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
2 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
5 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
6 # This code is free software; you can redistribute it and/or modify it
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
7 # under the terms of the GNU General Public License version 2 only, as
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
8 # published by the Free Software Foundation.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
9 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
10 # This code is distributed in the hope that it will be useful, but WITHOUT
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
13 # version 2 for more details (a copy is included in the LICENSE file that
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
14 # accompanied this code).
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
15 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
16 # You should have received a copy of the GNU General Public License version
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
17 # 2 along with this work; if not, write to the Free Software Foundation,
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
19 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
21 # or visit www.oracle.com if you need additional information or have any
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
22 # questions.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
23 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
24 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
25
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
26 from outputparser import OutputParser, Matcher
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
27 import re
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
28 import mx
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
29 import os
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
30 import commands
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
31 from os.path import isfile, join, exists
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
32
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
33 dacapoSanityWarmup = {
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
34 'avrora': [0, 0, 3, 6, 10],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
35 'batik': [0, 0, 5, 5, 20],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
36 'eclipse': [2, 4, 5, 10, 13],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
37 'fop': [4, 8, 10, 20, 30],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
38 'h2': [0, 0, 5, 5, 5],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
39 'jython': [0, 0, 5, 10, 10],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
40 'luindex': [0, 0, 5, 10, 10],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
41 'lusearch': [0, 4, 5, 5, 5],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
42 'pmd': [0, 0, 5, 10, 10],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
43 'sunflow': [0, 0, 5, 10, 15],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
44 'tomcat': [0, 0, 5, 10, 10],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
45 'tradebeans': [0, 0, 5, 10, 10],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
46 'tradesoap': [2, 4, 5, 10, 10],
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
47 'xalan': [0, 0, 5, 10, 15],
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
48 }
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
49
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
50 class SanityCheckLevel:
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
51 Fast, Gate, Normal, Extensive, Benchmark = range(5)
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
52
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
53 def getSPECjvm2008(skipKitValidation=False, warmupTime=None, iterationTime=None):
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
54
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
55 specjvm2008 = mx.get_env('SPECJVM2008')
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
56 if specjvm2008 is None or not exists(join(specjvm2008, 'SPECjvm2008.jar')):
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
57 mx.abort('Please set the SPECJVM2008 environment variable to a SPECjvm2008 directory')
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
58
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
59 score = re.compile(r"^(Score on|Noncompliant) (?P<benchmark>[a-zA-Z0-9\.\-]+)( result)?: (?P<score>[0-9]+(,|\.)[0-9]+)( SPECjvm2008 Base)? ops/m$")
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
60 error = re.compile(r"^Errors in benchmark: ")
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
61 # The ' ops/m' at the end of the success string is important : it's how you can tell valid and invalid runs apart
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
62 success = re.compile(r"^(Noncompliant c|C)omposite result: [0-9]+,[0-9]+( SPECjvm2008 (Base|Peak))? ops/m$")
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
63 matcher = Matcher(score, {'const:name' : 'benchmark', 'const:score' : 'score'}, startNewLine=True)
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
64
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
65 opts = []
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
66 if warmupTime is not None:
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
67 opts += ['-wt', str(warmupTime)]
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
68 if iterationTime is not None:
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
69 opts += ['-it', str(iterationTime)]
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
70 if skipKitValidation:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
71 opts += ['-ikv']
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
72
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
73 return Test("SPECjvm2008", "SPECjvm2008", ['-jar', 'SPECjvm2008.jar'] + opts, [success], [error], [matcher], vmOpts=['-Xms2g'], defaultCwd=specjvm2008)
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
74
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
75 def getDacapos(level=SanityCheckLevel.Normal, dacapoArgs=[]):
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
76 checks = []
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
77
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
78 for (bench, ns) in dacapoSanityWarmup.items():
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
79 if ns[level] > 0:
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
80 checks.append(getDacapo(bench, ns[level], dacapoArgs))
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
81
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
82 return checks
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
83
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
84 def getDacapo(name, n, dacapoArgs=[]):
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
85 dacapo = mx.get_env('DACAPO_CP')
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
86 if dacapo is None:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
87 dacapo = commands._graal_home + r'/lib/dacapo-9.12-bach.jar'
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
88
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
89 if not isfile(dacapo) or not dacapo.endswith('.jar'):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
90 mx.abort('Specified DaCapo jar file does not exist or is not a jar file: ' + dacapo)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
91
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
92 dacapoSuccess = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====$")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
93 dacapoFail = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) FAILED (warmup|) =====$")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
94 dacapoTime = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====")
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
95
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
96 dacapoMatcher = Matcher(dacapoTime, {'const:name' : 'benchmark', 'const:score' : 'time'})
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
97
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
98 return Test("DaCapo-" + name, "DaCapo", ['-jar', dacapo, name, '-n', str(n), ] + dacapoArgs, [dacapoSuccess], [dacapoFail], [dacapoMatcher], ['-Xms1g', '-Xmx2g', '-XX:MaxPermSize=256m'])
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
99
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
100 def getBootstraps():
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
101 time = re.compile(r"Bootstrapping Graal............... in (?P<time>[0-9]+) ms")
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
102 scoreMatcher = Matcher(time, {'const:name' : 'const:BootstrapTime', 'const:score' : 'time'})
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
103 tests = []
4230
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
104 tests.append(Test("Bootstrap", "Bootstrap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher]))
b780ecb920c9 Add SPECjvm2008 to benchmarks, use dacapos defined in sanitycheck for the gate
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4228
diff changeset
105 tests.append(Test("Bootstrap-bigHeap", "Bootstrap-bigHeap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher], vmOpts=['-Xms2g']))
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
106 return tests
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
107
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
108 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
109 Encapsulates a single program that is a sanity test and/or a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
110 """
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
111 class Test:
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
112 def __init__(self, name, group, cmd, successREs=[], failureREs=[], scoreMatchers=[], vmOpts=[], defaultCwd=None):
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
113 self.name = name
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
114 self.group = group
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
115 self.successREs = successREs
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
116 self.failureREs = failureREs
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
117 self.scoreMatchers = scoreMatchers
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
118 self.vmOpts = vmOpts
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
119 self.cmd = cmd
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
120 self.defaultCwd = defaultCwd
4150
c78bace5086a start work on integrating old hooks into mx, work on sanity checks
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
121
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
122 def test(self, vm, cwd=None, opts=[], vmbuild=None):
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
123 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
124 Run this program as a sanity test.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
125 """
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
126 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
127 cwd = self.defaultCwd
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
128 parser = OutputParser(nonZeroIsFatal = False)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
129 jvmError = re.compile(r"(?P<jvmerror>([A-Z]:|/).*[/\\]hs_err_pid[0-9]+\.log)")
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
130 parser.addMatcher(Matcher(jvmError, {'const:jvmError' : 'jvmerror'}))
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
131
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
132 for successRE in self.successREs:
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
133 parser.addMatcher(Matcher(successRE, {'const:passed' : 'const:1'}))
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
134 for failureRE in self.failureREs:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
135 parser.addMatcher(Matcher(failureRE, {'const:failed' : 'const:1'}))
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
136
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
137 result = parser.parse(vm, self.vmOpts + opts + self.cmd, cwd, vmbuild)
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
138
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
139 parsedLines = result['parsed']
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
140 assert len(parsedLines) == 1, 'Test matchers should not return more than one line'
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
141
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
142 parsed = parsedLines[0]
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
143
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
144 if parsed.has_key('jvmError'):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
145 mx.log('/!\\JVM Error : dumping error log...')
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
146 f = open(parsed['jvmError'], 'rb');
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
147 for line in iter(f.readline, ''):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
148 mx.log(line.rstrip())
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
149 f.close()
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
150 os.unlink(parsed['jvmError'])
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
151 return False
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
152
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
153 if parsed.has_key('failed') and parsed['failed'] is 1:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
154 return False
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
155
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
156 return result['retcode'] is 0 and parsed.has_key('passed') and parsed['passed'] is '1'
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
157
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
158 def bench(self, vm, cwd=None, opts=[], vmbuild=None):
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
159 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
160 Run this program as a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
161 """
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
162 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
163 cwd = self.defaultCwd
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
164 parser = OutputParser(nonZeroIsFatal = False)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
165
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
166 for failureRE in self.failureREs:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
167 parser.addMatcher(Matcher(failureRE, {'const:failed' : 'const:1'}))
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
168 for scoreMatcher in self.scoreMatchers:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
169 parser.addMatcher(scoreMatcher)
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
170
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4217
diff changeset
171 result = parser.parse(vm, self.vmOpts + opts + self.cmd, cwd, vmbuild)
4217
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
172 if result['retcode'] is not 0:
f3271682fe5a Various small fix around benchmarks in gate scripts
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
173 return {}
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
174
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
175 parsed = result['parsed']
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
176
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
177 ret = {}
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
178
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
179 for line in parsed:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
180 assert line.has_key('name') and line.has_key('score')
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
181 if line.has_key('failed') and parsed['failed'] is 1:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
182 return {}
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
183 ret[line['name']] = line['score']
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
184
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4150
diff changeset
185 return ret