truffle: mx/sanitycheck.py annotate

annotate mx/sanitycheck.py @ 18891:9afee75cee46

mx: add microbench command to run JMH benchmarks

author	Roland Schatz <roland.schatz@oracle.com>
date	Wed, 21 Jan 2015 14:49:13 +0100
parents	d42e11af980d
children	6df10903c955

rev	line source
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	1 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	2 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	5 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	6 # This code is free software; you can redistribute it and/or modify it
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	7 # under the terms of the GNU General Public License version 2 only, as
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	8 # published by the Free Software Foundation.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	9 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	10 # This code is distributed in the hope that it will be useful, but WITHOUT
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	13 # version 2 for more details (a copy is included in the LICENSE file that
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	14 # accompanied this code).
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	15 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	16 # You should have received a copy of the GNU General Public License version
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	17 # 2 along with this work; if not, write to the Free Software Foundation,
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	19 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	21 # or visit www.oracle.com if you need additional information or have any
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	22 # questions.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	23 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	24 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	25
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	26 from outputparser import OutputParser, ValuesMatcher
12769 99769479f9ce s/commands/mx_graal/ in mx/sanitycheck.py Gilles Duboscq <duboscq@ssw.jku.at> parents: 12647 diff changeset	27 import re, mx, mx_graal, os, sys, StringIO, subprocess
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	28 from os.path import isfile, join, exists
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	29
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	30 gc = 'UseSerialGC'
8501 41fc46da946a -More fixes and passrate(fop) Christos Kotselidis <christos.kotselidis@oracle.com> parents: 7656 diff changeset	31
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	32 dacapoSanityWarmup = {
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	33 'avrora': [0, 0, 3, 6, 13],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	34 'batik': [0, 0, 5, 5, 20],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	35 'eclipse': [0, 0, 0, 0, 0],
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	36 'fop': [4, 8, 10, 20, 30],
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	37 'h2': [0, 0, 5, 5, 8],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	38 'jython': [0, 0, 5, 10, 13],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	39 'luindex': [0, 0, 5, 10, 10],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	40 'lusearch': [0, 4, 5, 5, 8],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	41 'pmd': [0, 0, 5, 10, 13],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	42 'sunflow': [0, 2, 5, 10, 15],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	43 'tomcat': [0, 0, 5, 10, 15],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	44 'tradebeans': [0, 0, 5, 10, 13],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	45 'tradesoap': [0, 0, 5, 10, 15],
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	46 'xalan': [0, 0, 5, 10, 18],
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	47 }
c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	48
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	49 dacapoScalaSanityWarmup = {
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	50 'actors': [0, 0, 2, 5, 5],
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	51 'apparat': [0, 0, 2, 5, 5],
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	52 'factorie': [0, 0, 2, 5, 5],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	53 'kiama': [0, 4, 3, 13, 15],
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	54 'scalac': [0, 0, 5, 15, 20],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	55 'scaladoc': [0, 0, 5, 15, 15],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	56 'scalap': [0, 0, 5, 15, 20],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	57 'scalariform':[0, 0, 6, 15, 20],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	58 'scalatest': [0, 0, 2, 10, 12],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	59 'scalaxb': [0, 0, 5, 15, 25],
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	60 # (gdub) specs sometimes returns a non-zero value event though there is no apparent failure
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	61 'specs': [0, 0, 0, 0, 0],
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	62 'tmt': [0, 0, 3, 10, 12]
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	63 }
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	64
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	65 dacapoGateBuildLevels = {
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	66 'avrora': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	67 'batik': ['product', 'fastdebug', 'debug'],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	68 # (lewurm): does not work with JDK8
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	69 'eclipse': [],
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	70 'fop': ['fastdebug', 'debug'],
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	71 'h2': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	72 'jython': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	73 'luindex': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	74 'lusearch': ['product'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	75 'pmd': ['product', 'fastdebug', 'debug'],
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	76 'sunflow': ['fastdebug', 'debug'],
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	77 'tomcat': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	78 'tradebeans': ['product', 'fastdebug', 'debug'],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	79 # tradesoap is too unreliable for the gate, often crashing with concurrency problems:
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	80 # http://sourceforge.net/p/dacapobench/bugs/99/
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	81 'tradesoap': [],
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	82 'xalan': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	83 }
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	84
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	85 dacapoScalaGateBuildLevels = {
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	86 'actors': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	87 'apparat': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	88 'factorie': ['product', 'fastdebug', 'debug'],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	89 'kiama': ['fastdebug', 'debug'],
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	90 'scalac': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	91 'scaladoc': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	92 'scalap': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	93 'scalariform':['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	94 'scalatest': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	95 'scalaxb': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	96 'specs': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	97 'tmt': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	98 }
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	99
11367 39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	100 specjvm2008Names = [
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	101 'startup.helloworld',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	102 'startup.compiler.compiler',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	103 'startup.compiler.sunflow',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	104 'startup.compress',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	105 'startup.crypto.aes',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	106 'startup.crypto.rsa',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	107 'startup.crypto.signverify',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	108 'startup.mpegaudio',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	109 'startup.scimark.fft',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	110 'startup.scimark.lu',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	111 'startup.scimark.monte_carlo',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	112 'startup.scimark.sor',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	113 'startup.scimark.sparse',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	114 'startup.serial',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	115 'startup.sunflow',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	116 'startup.xml.transform',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	117 'startup.xml.validation',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	118 'compiler.compiler',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	119 'compiler.sunflow',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	120 'compress',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	121 'crypto.aes',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	122 'crypto.rsa',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	123 'crypto.signverify',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	124 'derby',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	125 'mpegaudio',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	126 'scimark.fft.large',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	127 'scimark.lu.large',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	128 'scimark.sor.large',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	129 'scimark.sparse.large',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	130 'scimark.fft.small',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	131 'scimark.lu.small',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	132 'scimark.sor.small',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	133 'scimark.sparse.small',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	134 'scimark.monte_carlo',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	135 'serial',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	136 'sunflow',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	137 'xml.transform',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	138 'xml.validation'
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	139 ]
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	140
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	141 def _noneAsEmptyList(a):
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	142 if a is None:
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	143 return []
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	144 return a
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	145
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	146 class SanityCheckLevel:
4219 47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	147 Fast, Gate, Normal, Extensive, Benchmark = range(5)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	148
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	149 def getSPECjbb2005(benchArgs=None):
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	150 benchArgs = [] if benchArgs is None else benchArgs
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	151
5877 0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	152 specjbb2005 = mx.get_env('SPECJBB2005')
0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	153 if specjbb2005 is None or not exists(join(specjbb2005, 'jbb.jar')):
0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	154 mx.abort('Please set the SPECJBB2005 environment variable to a SPECjbb2005 directory')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	155
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	156 score = re.compile(r"^Valid run, Score is (?P<score>[0-9]+)$", re.MULTILINE)
5877 0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	157 error = re.compile(r"VALIDATION ERROR")
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	158 success = re.compile(r"^Valid run, Score is [0-9]+$", re.MULTILINE)
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	159 matcher = ValuesMatcher(score, {'group' : 'SPECjbb2005', 'name' : 'score', 'score' : '<score>'})
5877 0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	160 classpath = ['jbb.jar', 'check.jar']
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	161 return Test("SPECjbb2005", ['spec.jbb.JBBmain', '-propfile', 'SPECjbb.props'] + benchArgs, [success], [error], [matcher], vmOpts=['-Xms3g', '-XX:+' + gc, '-XX:-UseCompressedOops', '-cp', os.pathsep.join(classpath)], defaultCwd=specjbb2005)
7565 9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	162
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	163 def getSPECjbb2013(benchArgs=None):
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	164
7565 9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	165 specjbb2013 = mx.get_env('SPECJBB2013')
9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	166 if specjbb2013 is None or not exists(join(specjbb2013, 'specjbb2013.jar')):
9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	167 mx.abort('Please set the SPECJBB2013 environment variable to a SPECjbb2013 directory')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	168
7565 9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	169 jops = re.compile(r"^RUN RESULT: hbIR $max attempted$ = [0-9]+, hbIR $settled$ = [0-9]+, max-jOPS = (?P<max>[0-9]+), critical-jOPS = (?P<critical>[0-9]+)$", re.MULTILINE)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	170 # error?
7565 9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	171 success = re.compile(r"org.spec.jbb.controller: Run finished", re.MULTILINE)
9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	172 matcherMax = ValuesMatcher(jops, {'group' : 'SPECjbb2013', 'name' : 'max', 'score' : '<max>'})
9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	173 matcherCritical = ValuesMatcher(jops, {'group' : 'SPECjbb2013', 'name' : 'critical', 'score' : '<critical>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	174 return Test("SPECjbb2013", ['-jar', 'specjbb2013.jar', '-m', 'composite'] +
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	175 _noneAsEmptyList(benchArgs), [success], [], [matcherCritical, matcherMax],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	176 vmOpts=['-Xmx6g', '-Xms6g', '-Xmn3g', '-XX:+UseParallelOldGC', '-XX:-UseAdaptiveSizePolicy', '-XX:-UseBiasedLocking', '-XX:-UseCompressedOops'], defaultCwd=specjbb2013)
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	177
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	178 def getSPECjvm2008(benchArgs=None):
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	179
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	180 specjvm2008 = mx.get_env('SPECJVM2008')
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	181 if specjvm2008 is None or not exists(join(specjvm2008, 'SPECjvm2008.jar')):
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	182 mx.abort('Please set the SPECJVM2008 environment variable to a SPECjvm2008 directory')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	183
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	184 score = re.compile(r"^(Score on\|Noncompliant) (?P<benchmark>[a-zA-Z0-9\._]+)( result)?: (?P<score>[0-9]+((,\|\.)[0-9]+)?)( SPECjvm2008 Base)? ops/m$", re.MULTILINE)
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	185 error = re.compile(r"^Errors in benchmark: ", re.MULTILINE)
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	186 # The ' ops/m' at the end of the success string is important : it's how you can tell valid and invalid runs apart
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	187 success = re.compile(r"^(Noncompliant c\|C)omposite result: [0-9]+((,\|\.)[0-9]+)?( SPECjvm2008 (Base\|Peak))? ops/m$", re.MULTILINE)
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	188 matcher = ValuesMatcher(score, {'group' : 'SPECjvm2008', 'name' : '<benchmark>', 'score' : '<score>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	189
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	190 return Test("SPECjvm2008", ['-jar', 'SPECjvm2008.jar'] + _noneAsEmptyList(benchArgs), [success], [error], [matcher], vmOpts=['-Xms3g', '-XX:+' + gc, '-XX:-UseCompressedOops'], defaultCwd=specjvm2008)
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	191
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	192 def getDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=None):
4217 f3271682fe5a Various small fix around benchmarks in gate scripts Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4215 diff changeset	193 checks = []
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	194
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	195 for (bench, ns) in dacapoSanityWarmup.items():
c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	196 if ns[level] > 0:
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	197 if gateBuildLevel is None or gateBuildLevel in dacapoGateBuildLevels[bench]:
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	198 checks.append(getDacapo(bench, ['-n', str(ns[level])] + _noneAsEmptyList(dacapoArgs)))
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	199
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	200 return checks
c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	201
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	202 def getDacapo(name, dacapoArgs=None):
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	203 dacapo = mx.get_env('DACAPO_CP')
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	204 if dacapo is None:
4244 a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	205 l = mx.library('DACAPO', False)
a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	206 if l is not None:
a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	207 dacapo = l.get_path(True)
a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	208 else:
a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	209 mx.abort('DaCapo 9.12 jar file must be specified with DACAPO_CP environment variable or as DACAPO library')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	210
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	211 if not isfile(dacapo) or not dacapo.endswith('.jar'):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	212 mx.abort('Specified DaCapo jar file does not exist or is not a jar file: ' + dacapo)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	213
17362 03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	214 dacapoSuccess = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====", re.MULTILINE)
03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	215 dacapoFail = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) FAILED (warmup\|) =====", re.MULTILINE)
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	216 dacapoTime = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====")
5195 8125c1f6472b Add DaCapo 1st run as a benchmark group Gilles Duboscq <duboscq@ssw.jku.at> parents: 5148 diff changeset	217 dacapoTime1 = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) completed warmup 1 in (?P<time>[0-9]+) msec =====")
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	218
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	219 dacapoMatcher = ValuesMatcher(dacapoTime, {'group' : 'DaCapo', 'name' : '<benchmark>', 'score' : '<time>'})
c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	220 dacapoMatcher1 = ValuesMatcher(dacapoTime1, {'group' : 'DaCapo-1stRun', 'name' : '<benchmark>', 'score' : '<time>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	221
17362 03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	222 return Test("DaCapo-" + name, ['-jar', mx._cygpathU2W(dacapo), name] + _noneAsEmptyList(dacapoArgs), [dacapoSuccess], [dacapoFail], [dacapoMatcher, dacapoMatcher1], ['-Xms2g', '-XX:+' + gc, '-XX:-UseCompressedOops'])
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	223
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	224 def getScalaDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=None):
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	225 checks = []
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	226
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	227 for (bench, ns) in dacapoScalaSanityWarmup.items():
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	228 if ns[level] > 0:
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	229 if gateBuildLevel is None or gateBuildLevel in dacapoScalaGateBuildLevels[bench]:
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	230 checks.append(getScalaDacapo(bench, ['-n', str(ns[level])] + _noneAsEmptyList(dacapoArgs)))
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	231
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	232 return checks
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	233
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	234 def getScalaDacapo(name, dacapoArgs=None):
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	235 dacapo = mx.get_env('DACAPO_SCALA_CP')
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	236 if dacapo is None:
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	237 l = mx.library('DACAPO_SCALA', False)
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	238 if l is not None:
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	239 dacapo = l.get_path(True)
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	240 else:
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	241 mx.abort('Scala DaCapo 0.1.0 jar file must be specified with DACAPO_SCALA_CP environment variable or as DACAPO_SCALA library')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	242
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	243 if not isfile(dacapo) or not dacapo.endswith('.jar'):
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	244 mx.abort('Specified Scala DaCapo jar file does not exist or is not a jar file: ' + dacapo)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	245
17362 03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	246 dacapoSuccess = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====", re.MULTILINE)
03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	247 dacapoFail = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) FAILED (warmup\|) =====", re.MULTILINE)
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	248 dacapoTime = re.compile(r"===== DaCapo 0\.1\.0(-SNAPSHOT)? (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====")
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	249
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	250 dacapoMatcher = ValuesMatcher(dacapoTime, {'group' : "Scala-DaCapo", 'name' : '<benchmark>', 'score' : '<time>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	251
17362 03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	252 return Test("Scala-DaCapo-" + name, ['-jar', mx._cygpathU2W(dacapo), name] + _noneAsEmptyList(dacapoArgs), [dacapoSuccess], [dacapoFail], [dacapoMatcher], ['-Xms2g', '-XX:+' + gc, '-XX:-UseCompressedOops'])
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	253
4219 47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	254 def getBootstraps():
11658 01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	255 time = re.compile(r"Bootstrapping Graal\.+ in (?P<time>[0-9]+) ms( $compiled (?P<methods>[0-9]+) methods$)?")
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	256 scoreMatcher = ValuesMatcher(time, {'group' : 'Bootstrap', 'name' : 'BootstrapTime', 'score' : '<time>'})
11658 01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	257 methodMatcher = ValuesMatcher(time, {'group' : 'Bootstrap', 'name' : 'BootstrapMethods', 'score' : '<methods>'})
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	258 scoreMatcherBig = ValuesMatcher(time, {'group' : 'Bootstrap-bigHeap', 'name' : 'BootstrapTime', 'score' : '<time>'})
11658 01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	259 methodMatcherBig = ValuesMatcher(time, {'group' : 'Bootstrap-bigHeap', 'name' : 'BootstrapMethods', 'score' : '<methods>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	260
4219 47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	261 tests = []
11658 01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	262 tests.append(Test("Bootstrap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher, methodMatcher], ignoredVMs=['client', 'server'], benchmarkCompilationRate=False))
01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	263 tests.append(Test("Bootstrap-bigHeap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcherBig, methodMatcherBig], vmOpts=['-Xms2g'], ignoredVMs=['client', 'server'], benchmarkCompilationRate=False))
4219 47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	264 return tests
47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	265
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	266 class CTWMode:
17378 d42e11af980d mx: simplified ctw command Doug Simon <doug.simon@oracle.com> parents: 17362 diff changeset	267 Full, NoInline = range(2)
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	268
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	269 def getCTW(vm, mode):
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	270 time = re.compile(r"CompileTheWorld : Done $[0-9]+ classes, [0-9]+ methods, (?P<time>[0-9]+) ms$")
9180 4e6df9021a59 Assemble the CTW bench in the same value, remove PEA from nocomplex Gilles Duboscq <duboscq@ssw.jku.at> parents: 9173 diff changeset	271 scoreMatcher = ValuesMatcher(time, {'group' : 'CompileTheWorld', 'name' : 'CompileTime', 'score' : '<time>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	272
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	273 jre = os.environ.get('JAVA_HOME')
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	274 if exists(join(jre, 'jre')):
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	275 jre = join(jre, 'jre')
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	276 rtjar = join(jre, 'lib', 'rt.jar')
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	277
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	278
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	279 args = ['-XX:+CompileTheWorld', '-Xbootclasspath/p:' + rtjar]
12647 b038b643a3a4 restricted BootstrapGraal option to GRAALVM builds only Doug Simon <doug.simon@oracle.com> parents: 11658 diff changeset	280 if vm == 'graal':
b038b643a3a4 restricted BootstrapGraal option to GRAALVM builds only Doug Simon <doug.simon@oracle.com> parents: 11658 diff changeset	281 args += ['-XX:+BootstrapGraal']
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	282 if mode >= CTWMode.NoInline:
12769 99769479f9ce s/commands/mx_graal/ in mx/sanitycheck.py Gilles Duboscq <duboscq@ssw.jku.at> parents: 12647 diff changeset	283 if not mx_graal.isGraalEnabled(vm):
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	284 args.append('-XX:-Inline')
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	285 else:
17378 d42e11af980d mx: simplified ctw command Doug Simon <doug.simon@oracle.com> parents: 17362 diff changeset	286 args.append('-G:CompileTheWordConfig=-Inline')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	287
9180 4e6df9021a59 Assemble the CTW bench in the same value, remove PEA from nocomplex Gilles Duboscq <duboscq@ssw.jku.at> parents: 9173 diff changeset	288 return Test("CompileTheWorld", args, successREs=[time], scoreMatchers=[scoreMatcher], benchmarkCompilationRate=False)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	289
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	290
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	291 class Tee:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	292 def __init__(self):
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	293 self.output = StringIO.StringIO()
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	294 def eat(self, line):
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	295 self.output.write(line)
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	296 sys.stdout.write(line)
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	297
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	298 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	299 Encapsulates a single program that is a sanity test and/or a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	300 """
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	301 class Test:
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	302 def __init__(self, name, cmd, successREs=None, failureREs=None, scoreMatchers=None, vmOpts=None, defaultCwd=None, ignoredVMs=None, benchmarkCompilationRate=False):
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	303
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	304 self.name = name
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	305 self.successREs = _noneAsEmptyList(successREs)
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	306 self.failureREs = _noneAsEmptyList(failureREs) + [re.compile(r"Exception occurred in scope: ")]
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	307 self.scoreMatchers = _noneAsEmptyList(scoreMatchers)
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	308 self.vmOpts = _noneAsEmptyList(vmOpts)
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	309 self.cmd = cmd
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	310 self.defaultCwd = defaultCwd
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	311 self.ignoredVMs = _noneAsEmptyList(ignoredVMs)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	312 self.benchmarkCompilationRate = benchmarkCompilationRate
7654 641a4c6ac1ce fixed repetition of compilation metric option on command line when running multiple benchmarks (JIRA issue GRAAL-66) Doug Simon <doug.simon@oracle.com> parents: 7571 diff changeset	313 if benchmarkCompilationRate:
641a4c6ac1ce fixed repetition of compilation metric option on command line when running multiple benchmarks (JIRA issue GRAAL-66) Doug Simon <doug.simon@oracle.com> parents: 7571 diff changeset	314 self.vmOpts = self.vmOpts + ['-XX:+CITime']
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	315
4238 e0d09e05aa9b Made the gate time each DaCapo test individually. Doug Simon <doug.simon@oracle.com> parents: 4230 diff changeset	316 def __str__(self):
e0d09e05aa9b Made the gate time each DaCapo test individually. Doug Simon <doug.simon@oracle.com> parents: 4230 diff changeset	317 return self.name
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	318
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	319 def test(self, vm, cwd=None, extraVmOpts=None, vmbuild=None):
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	320 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	321 Run this program as a sanity test.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	322 """
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	323 if vm in self.ignoredVMs:
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	324 return True
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	325 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	326 cwd = self.defaultCwd
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	327 parser = OutputParser()
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	328 jvmError = re.compile(r"(?P<jvmerror>([A-Z]:\|/).*[/\\]hs_err_pid[0-9]+\.log)")
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	329 parser.addMatcher(ValuesMatcher(jvmError, {'jvmError' : '<jvmerror>'}))
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	330
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	331 for successRE in self.successREs:
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	332 parser.addMatcher(ValuesMatcher(successRE, {'passed' : '1'}))
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	333 for failureRE in self.failureREs:
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	334 parser.addMatcher(ValuesMatcher(failureRE, {'failed' : '1'}))
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	335
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	336 tee = Tee()
12769 99769479f9ce s/commands/mx_graal/ in mx/sanitycheck.py Gilles Duboscq <duboscq@ssw.jku.at> parents: 12647 diff changeset	337 retcode = mx_graal.vm(self.vmOpts + _noneAsEmptyList(extraVmOpts) + self.cmd, vm, nonZeroIsFatal=False, out=tee.eat, err=subprocess.STDOUT, cwd=cwd, vmbuild=vmbuild)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	338 output = tee.output.getvalue()
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	339 valueMaps = parser.parse(output)
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	340
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	341 if len(valueMaps) == 0:
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	342 return False
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	343
8206 22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	344 record = {}
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	345 for valueMap in valueMaps:
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	346 for key, value in valueMap.items():
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	347 if record.has_key(key) and record[key] != value:
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	348 mx.abort('Inconsistant values returned by test machers : ' + str(valueMaps))
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	349 record[key] = value
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	350
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	351 jvmErrorFile = record.get('jvmError')
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	352 if jvmErrorFile:
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	353 mx.log('/!\\JVM Error : dumping error log...')
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	354 with open(jvmErrorFile, 'rb') as fp:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	355 mx.log(fp.read())
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	356 os.unlink(jvmErrorFile)
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	357 return False
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	358
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	359 if record.get('failed') == '1':
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	360 return False
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	361
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	362 return retcode == 0 and record.get('passed') == '1'
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	363
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	364 def bench(self, vm, cwd=None, extraVmOpts=None, vmbuild=None):
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	365 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	366 Run this program as a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	367 """
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	368 if vm in self.ignoredVMs:
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	369 return {}
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	370 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	371 cwd = self.defaultCwd
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	372 parser = OutputParser()
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	373
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	374 for successRE in self.successREs:
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	375 parser.addMatcher(ValuesMatcher(successRE, {'passed' : '1'}))
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	376 for failureRE in self.failureREs:
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	377 parser.addMatcher(ValuesMatcher(failureRE, {'failed' : '1'}))
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	378 for scoreMatcher in self.scoreMatchers:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	379 parser.addMatcher(scoreMatcher)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	380
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	381 if self.benchmarkCompilationRate:
7570 2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	382 if vm == 'graal':
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	383 bps = re.compile(r"ParsedBytecodesPerSecond@final: (?P<rate>[0-9]+)")
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	384 ibps = re.compile(r"InlinedBytecodesPerSecond@final: (?P<rate>[0-9]+)")
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	385 parser.addMatcher(ValuesMatcher(bps, {'group' : 'ParsedBytecodesPerSecond', 'name' : self.name, 'score' : '<rate>'}))
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	386 parser.addMatcher(ValuesMatcher(ibps, {'group' : 'InlinedBytecodesPerSecond', 'name' : self.name, 'score' : '<rate>'}))
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	387 else:
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	388 ibps = re.compile(r"(?P<compiler>[\w]+) compilation speed: +(?P<rate>[0-9]+) bytes/s {standard")
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	389 parser.addMatcher(ValuesMatcher(ibps, {'group' : 'InlinedBytecodesPerSecond', 'name' : '<compiler>:' + self.name, 'score' : '<rate>'}))
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	390
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	391 startDelim = 'START: ' + self.name
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	392 endDelim = 'END: ' + self.name
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	393
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	394 outputfile = os.environ.get('BENCH_OUTPUT', None)
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	395 if outputfile:
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	396 # Used only to debug output parsing
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	397 with open(outputfile) as fp:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	398 output = fp.read()
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	399 start = output.find(startDelim)
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	400 end = output.find(endDelim, start)
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	401 if start == -1 and end == -1:
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	402 return {}
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	403 output = output[start + len(startDelim + os.linesep): end]
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	404 mx.log(startDelim)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	405 mx.log(output)
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	406 mx.log(endDelim)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	407 else:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	408 tee = Tee()
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	409 mx.log(startDelim)
12769 99769479f9ce s/commands/mx_graal/ in mx/sanitycheck.py Gilles Duboscq <duboscq@ssw.jku.at> parents: 12647 diff changeset	410 if mx_graal.vm(self.vmOpts + _noneAsEmptyList(extraVmOpts) + self.cmd, vm, nonZeroIsFatal=False, out=tee.eat, err=subprocess.STDOUT, cwd=cwd, vmbuild=vmbuild) != 0:
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	411 mx.abort("Benchmark failed (non-zero retcode)")
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	412 mx.log(endDelim)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	413 output = tee.output.getvalue()
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	414
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	415 groups = {}
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	416 passed = False
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	417 for valueMap in parser.parse(output):
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	418 assert (valueMap.has_key('name') and valueMap.has_key('score') and valueMap.has_key('group')) or valueMap.has_key('passed') or valueMap.has_key('failed'), valueMap
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	419 if valueMap.get('failed') == '1':
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	420 mx.abort("Benchmark failed")
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	421 if valueMap.get('passed') == '1':
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	422 passed = True
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	423 groupName = valueMap.get('group')
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	424 if groupName:
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	425 group = groups.setdefault(groupName, {})
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	426 name = valueMap.get('name')
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	427 score = valueMap.get('score')
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	428 if name and score:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	429 group[name] = score
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	430
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	431 if not passed:
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	432 mx.abort("Benchmark failed (not passed)")
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	433
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	434 return groups

Mercurial > hg > truffle

annotate mx/sanitycheck.py @ 18891:9afee75cee46