truffle: mx/sanitycheck.py annotate

annotate mx/sanitycheck.py @ 18843:5e80dd2f1783

Extend emitMul, emitAdd and emitSub in ArithmeticLIRGenerator for explicitly requesting to set the flags register. (In AMD64 flags are set implicitly, but on SPARC it is not)

author	Stefan Anzinger <stefan.anzinger@oracle.com>
date	Mon, 12 Jan 2015 19:34:31 +0100
parents	d42e11af980d
children	6df10903c955

rev	line source
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	1 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	2 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	5 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	6 # This code is free software; you can redistribute it and/or modify it
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	7 # under the terms of the GNU General Public License version 2 only, as
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	8 # published by the Free Software Foundation.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	9 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	10 # This code is distributed in the hope that it will be useful, but WITHOUT
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	13 # version 2 for more details (a copy is included in the LICENSE file that
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	14 # accompanied this code).
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	15 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	16 # You should have received a copy of the GNU General Public License version
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	17 # 2 along with this work; if not, write to the Free Software Foundation,
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	19 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	21 # or visit www.oracle.com if you need additional information or have any
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	22 # questions.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	23 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	24 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	25
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	26 from outputparser import OutputParser, ValuesMatcher
12769 99769479f9ce s/commands/mx_graal/ in mx/sanitycheck.py Gilles Duboscq <duboscq@ssw.jku.at> parents: 12647 diff changeset	27 import re, mx, mx_graal, os, sys, StringIO, subprocess
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	28 from os.path import isfile, join, exists
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	29
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	30 gc = 'UseSerialGC'
8501 41fc46da946a -More fixes and passrate(fop) Christos Kotselidis <christos.kotselidis@oracle.com> parents: 7656 diff changeset	31
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	32 dacapoSanityWarmup = {
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	33 'avrora': [0, 0, 3, 6, 13],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	34 'batik': [0, 0, 5, 5, 20],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	35 'eclipse': [0, 0, 0, 0, 0],
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	36 'fop': [4, 8, 10, 20, 30],
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	37 'h2': [0, 0, 5, 5, 8],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	38 'jython': [0, 0, 5, 10, 13],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	39 'luindex': [0, 0, 5, 10, 10],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	40 'lusearch': [0, 4, 5, 5, 8],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	41 'pmd': [0, 0, 5, 10, 13],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	42 'sunflow': [0, 2, 5, 10, 15],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	43 'tomcat': [0, 0, 5, 10, 15],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	44 'tradebeans': [0, 0, 5, 10, 13],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	45 'tradesoap': [0, 0, 5, 10, 15],
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	46 'xalan': [0, 0, 5, 10, 18],
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	47 }
c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	48
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	49 dacapoScalaSanityWarmup = {
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	50 'actors': [0, 0, 2, 5, 5],
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	51 'apparat': [0, 0, 2, 5, 5],
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	52 'factorie': [0, 0, 2, 5, 5],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	53 'kiama': [0, 4, 3, 13, 15],
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	54 'scalac': [0, 0, 5, 15, 20],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	55 'scaladoc': [0, 0, 5, 15, 15],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	56 'scalap': [0, 0, 5, 15, 20],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	57 'scalariform':[0, 0, 6, 15, 20],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	58 'scalatest': [0, 0, 2, 10, 12],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	59 'scalaxb': [0, 0, 5, 15, 25],
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	60 # (gdub) specs sometimes returns a non-zero value event though there is no apparent failure
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	61 'specs': [0, 0, 0, 0, 0],
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	62 'tmt': [0, 0, 3, 10, 12]
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	63 }
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	64
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	65 dacapoGateBuildLevels = {
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	66 'avrora': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	67 'batik': ['product', 'fastdebug', 'debug'],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	68 # (lewurm): does not work with JDK8
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	69 'eclipse': [],
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	70 'fop': ['fastdebug', 'debug'],
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	71 'h2': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	72 'jython': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	73 'luindex': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	74 'lusearch': ['product'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	75 'pmd': ['product', 'fastdebug', 'debug'],
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	76 'sunflow': ['fastdebug', 'debug'],
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	77 'tomcat': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	78 'tradebeans': ['product', 'fastdebug', 'debug'],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	79 # tradesoap is too unreliable for the gate, often crashing with concurrency problems:
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	80 # http://sourceforge.net/p/dacapobench/bugs/99/
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	81 'tradesoap': [],
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	82 'xalan': ['product', 'fastdebug', 'debug'],
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	83 }
394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	84
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	85 dacapoScalaGateBuildLevels = {
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	86 'actors': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	87 'apparat': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	88 'factorie': ['product', 'fastdebug', 'debug'],
14877 fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug) Bernhard Urban <bernhard.urban@jku.at> parents: 13967 diff changeset	89 'kiama': ['fastdebug', 'debug'],
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	90 'scalac': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	91 'scaladoc': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	92 'scalap': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	93 'scalariform':['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	94 'scalatest': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	95 'scalaxb': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	96 'specs': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	97 'tmt': ['product', 'fastdebug', 'debug'],
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	98 }
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	99
11367 39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	100 specjvm2008Names = [
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	101 'startup.helloworld',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	102 'startup.compiler.compiler',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	103 'startup.compiler.sunflow',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	104 'startup.compress',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	105 'startup.crypto.aes',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	106 'startup.crypto.rsa',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	107 'startup.crypto.signverify',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	108 'startup.mpegaudio',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	109 'startup.scimark.fft',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	110 'startup.scimark.lu',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	111 'startup.scimark.monte_carlo',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	112 'startup.scimark.sor',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	113 'startup.scimark.sparse',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	114 'startup.serial',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	115 'startup.sunflow',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	116 'startup.xml.transform',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	117 'startup.xml.validation',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	118 'compiler.compiler',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	119 'compiler.sunflow',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	120 'compress',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	121 'crypto.aes',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	122 'crypto.rsa',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	123 'crypto.signverify',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	124 'derby',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	125 'mpegaudio',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	126 'scimark.fft.large',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	127 'scimark.lu.large',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	128 'scimark.sor.large',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	129 'scimark.sparse.large',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	130 'scimark.fft.small',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	131 'scimark.lu.small',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	132 'scimark.sor.small',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	133 'scimark.sparse.small',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	134 'scimark.monte_carlo',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	135 'serial',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	136 'sunflow',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	137 'xml.transform',
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	138 'xml.validation'
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	139 ]
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands Doug Simon <doug.simon@oracle.com> parents: 10895 diff changeset	140
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	141 def _noneAsEmptyList(a):
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	142 if a is None:
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	143 return []
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	144 return a
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	145
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	146 class SanityCheckLevel:
4219 47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	147 Fast, Gate, Normal, Extensive, Benchmark = range(5)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	148
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	149 def getSPECjbb2005(benchArgs=None):
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	150 benchArgs = [] if benchArgs is None else benchArgs
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	151
5877 0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	152 specjbb2005 = mx.get_env('SPECJBB2005')
0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	153 if specjbb2005 is None or not exists(join(specjbb2005, 'jbb.jar')):
0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	154 mx.abort('Please set the SPECJBB2005 environment variable to a SPECjbb2005 directory')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	155
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	156 score = re.compile(r"^Valid run, Score is (?P<score>[0-9]+)$", re.MULTILINE)
5877 0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	157 error = re.compile(r"VALIDATION ERROR")
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	158 success = re.compile(r"^Valid run, Score is [0-9]+$", re.MULTILINE)
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	159 matcher = ValuesMatcher(score, {'group' : 'SPECjbb2005', 'name' : 'score', 'score' : '<score>'})
5877 0e54d9bb922d Add SPECjbb2005 to the benchmarks Gilles Duboscq <duboscq@ssw.jku.at> parents: 5196 diff changeset	160 classpath = ['jbb.jar', 'check.jar']
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	161 return Test("SPECjbb2005", ['spec.jbb.JBBmain', '-propfile', 'SPECjbb.props'] + benchArgs, [success], [error], [matcher], vmOpts=['-Xms3g', '-XX:+' + gc, '-XX:-UseCompressedOops', '-cp', os.pathsep.join(classpath)], defaultCwd=specjbb2005)
7565 9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	162
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	163 def getSPECjbb2013(benchArgs=None):
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	164
7565 9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	165 specjbb2013 = mx.get_env('SPECJBB2013')
9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	166 if specjbb2013 is None or not exists(join(specjbb2013, 'specjbb2013.jar')):
9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	167 mx.abort('Please set the SPECJBB2013 environment variable to a SPECjbb2013 directory')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	168
7565 9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	169 jops = re.compile(r"^RUN RESULT: hbIR $max attempted$ = [0-9]+, hbIR $settled$ = [0-9]+, max-jOPS = (?P<max>[0-9]+), critical-jOPS = (?P<critical>[0-9]+)$", re.MULTILINE)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	170 # error?
7565 9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	171 success = re.compile(r"org.spec.jbb.controller: Run finished", re.MULTILINE)
9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	172 matcherMax = ValuesMatcher(jops, {'group' : 'SPECjbb2013', 'name' : 'max', 'score' : '<max>'})
9a521597686b Add SPECjbb2013 Gilles Duboscq <duboscq@ssw.jku.at> parents: 7564 diff changeset	173 matcherCritical = ValuesMatcher(jops, {'group' : 'SPECjbb2013', 'name' : 'critical', 'score' : '<critical>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	174 return Test("SPECjbb2013", ['-jar', 'specjbb2013.jar', '-m', 'composite'] +
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	175 _noneAsEmptyList(benchArgs), [success], [], [matcherCritical, matcherMax],
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	176 vmOpts=['-Xmx6g', '-Xms6g', '-Xmn3g', '-XX:+UseParallelOldGC', '-XX:-UseAdaptiveSizePolicy', '-XX:-UseBiasedLocking', '-XX:-UseCompressedOops'], defaultCwd=specjbb2013)
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	177
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	178 def getSPECjvm2008(benchArgs=None):
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	179
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	180 specjvm2008 = mx.get_env('SPECJVM2008')
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	181 if specjvm2008 is None or not exists(join(specjvm2008, 'SPECjvm2008.jar')):
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	182 mx.abort('Please set the SPECJVM2008 environment variable to a SPECjvm2008 directory')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	183
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	184 score = re.compile(r"^(Score on\|Noncompliant) (?P<benchmark>[a-zA-Z0-9\._]+)( result)?: (?P<score>[0-9]+((,\|\.)[0-9]+)?)( SPECjvm2008 Base)? ops/m$", re.MULTILINE)
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	185 error = re.compile(r"^Errors in benchmark: ", re.MULTILINE)
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	186 # The ' ops/m' at the end of the success string is important : it's how you can tell valid and invalid runs apart
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	187 success = re.compile(r"^(Noncompliant c\|C)omposite result: [0-9]+((,\|\.)[0-9]+)?( SPECjvm2008 (Base\|Peak))? ops/m$", re.MULTILINE)
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	188 matcher = ValuesMatcher(score, {'group' : 'SPECjvm2008', 'name' : '<benchmark>', 'score' : '<score>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	189
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	190 return Test("SPECjvm2008", ['-jar', 'SPECjvm2008.jar'] + _noneAsEmptyList(benchArgs), [success], [error], [matcher], vmOpts=['-Xms3g', '-XX:+' + gc, '-XX:-UseCompressedOops'], defaultCwd=specjvm2008)
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	191
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	192 def getDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=None):
4217 f3271682fe5a Various small fix around benchmarks in gate scripts Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4215 diff changeset	193 checks = []
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	194
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	195 for (bench, ns) in dacapoSanityWarmup.items():
c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	196 if ns[level] > 0:
4240 394424b7df1a Disabled certain DaCapo tests for fastdebug testing in the gate. Doug Simon <doug.simon@oracle.com> parents: 4238 diff changeset	197 if gateBuildLevel is None or gateBuildLevel in dacapoGateBuildLevels[bench]:
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	198 checks.append(getDacapo(bench, ['-n', str(ns[level])] + _noneAsEmptyList(dacapoArgs)))
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	199
4150 c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	200 return checks
c78bace5086a start work on integrating old hooks into mx, work on sanity checks Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: diff changeset	201
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	202 def getDacapo(name, dacapoArgs=None):
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	203 dacapo = mx.get_env('DACAPO_CP')
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	204 if dacapo is None:
4244 a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	205 l = mx.library('DACAPO', False)
a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	206 if l is not None:
a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	207 dacapo = l.get_path(True)
a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	208 else:
a69889e5a8a5 Made the DaCapo library be downloaded only on demand. Doug Simon <doug.simon@oracle.com> parents: 4240 diff changeset	209 mx.abort('DaCapo 9.12 jar file must be specified with DACAPO_CP environment variable or as DACAPO library')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	210
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	211 if not isfile(dacapo) or not dacapo.endswith('.jar'):
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	212 mx.abort('Specified DaCapo jar file does not exist or is not a jar file: ' + dacapo)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	213
17362 03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	214 dacapoSuccess = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====", re.MULTILINE)
03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	215 dacapoFail = re.compile(r"^===== DaCapo 9\.12 ([a-zA-Z0-9_]+) FAILED (warmup\|) =====", re.MULTILINE)
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	216 dacapoTime = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====")
5195 8125c1f6472b Add DaCapo 1st run as a benchmark group Gilles Duboscq <duboscq@ssw.jku.at> parents: 5148 diff changeset	217 dacapoTime1 = re.compile(r"===== DaCapo 9\.12 (?P<benchmark>[a-zA-Z0-9_]+) completed warmup 1 in (?P<time>[0-9]+) msec =====")
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	218
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	219 dacapoMatcher = ValuesMatcher(dacapoTime, {'group' : 'DaCapo', 'name' : '<benchmark>', 'score' : '<time>'})
c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	220 dacapoMatcher1 = ValuesMatcher(dacapoTime1, {'group' : 'DaCapo-1stRun', 'name' : '<benchmark>', 'score' : '<time>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	221
17362 03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	222 return Test("DaCapo-" + name, ['-jar', mx._cygpathU2W(dacapo), name] + _noneAsEmptyList(dacapoArgs), [dacapoSuccess], [dacapoFail], [dacapoMatcher, dacapoMatcher1], ['-Xms2g', '-XX:+' + gc, '-XX:-UseCompressedOops'])
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	223
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	224 def getScalaDacapos(level=SanityCheckLevel.Normal, gateBuildLevel=None, dacapoArgs=None):
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	225 checks = []
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	226
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	227 for (bench, ns) in dacapoScalaSanityWarmup.items():
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	228 if ns[level] > 0:
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	229 if gateBuildLevel is None or gateBuildLevel in dacapoScalaGateBuildLevels[bench]:
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	230 checks.append(getScalaDacapo(bench, ['-n', str(ns[level])] + _noneAsEmptyList(dacapoArgs)))
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	231
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	232 return checks
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	233
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	234 def getScalaDacapo(name, dacapoArgs=None):
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	235 dacapo = mx.get_env('DACAPO_SCALA_CP')
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	236 if dacapo is None:
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	237 l = mx.library('DACAPO_SCALA', False)
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	238 if l is not None:
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	239 dacapo = l.get_path(True)
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	240 else:
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	241 mx.abort('Scala DaCapo 0.1.0 jar file must be specified with DACAPO_SCALA_CP environment variable or as DACAPO_SCALA library')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	242
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	243 if not isfile(dacapo) or not dacapo.endswith('.jar'):
7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	244 mx.abort('Specified Scala DaCapo jar file does not exist or is not a jar file: ' + dacapo)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	245
17362 03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	246 dacapoSuccess = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) PASSED in ([0-9]+) msec =====", re.MULTILINE)
03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	247 dacapoFail = re.compile(r"^===== DaCapo 0\.1\.0(-SNAPSHOT)? ([a-zA-Z0-9_]+) FAILED (warmup\|) =====", re.MULTILINE)
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	248 dacapoTime = re.compile(r"===== DaCapo 0\.1\.0(-SNAPSHOT)? (?P<benchmark>[a-zA-Z0-9_]+) PASSED in (?P<time>[0-9]+) msec =====")
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	249
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	250 dacapoMatcher = ValuesMatcher(dacapoTime, {'group' : "Scala-DaCapo", 'name' : '<benchmark>', 'score' : '<time>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	251
17362 03eef43d364f mx: fix {,scala}dacapo paths and parsing issues on windows Bernhard Urban <bernhard.urban@jku.at> parents: 14877 diff changeset	252 return Test("Scala-DaCapo-" + name, ['-jar', mx._cygpathU2W(dacapo), name] + _noneAsEmptyList(dacapoArgs), [dacapoSuccess], [dacapoFail], [dacapoMatcher], ['-Xms2g', '-XX:+' + gc, '-XX:-UseCompressedOops'])
4480 7d6490436b57 Add Scala DaCapo benchs Gilles Duboscq <duboscq@ssw.jku.at> parents: 4416 diff changeset	253
4219 47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	254 def getBootstraps():
11658 01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	255 time = re.compile(r"Bootstrapping Graal\.+ in (?P<time>[0-9]+) ms( $compiled (?P<methods>[0-9]+) methods$)?")
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	256 scoreMatcher = ValuesMatcher(time, {'group' : 'Bootstrap', 'name' : 'BootstrapTime', 'score' : '<time>'})
11658 01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	257 methodMatcher = ValuesMatcher(time, {'group' : 'Bootstrap', 'name' : 'BootstrapMethods', 'score' : '<methods>'})
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	258 scoreMatcherBig = ValuesMatcher(time, {'group' : 'Bootstrap-bigHeap', 'name' : 'BootstrapTime', 'score' : '<time>'})
11658 01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	259 methodMatcherBig = ValuesMatcher(time, {'group' : 'Bootstrap-bigHeap', 'name' : 'BootstrapMethods', 'score' : '<methods>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	260
4219 47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	261 tests = []
11658 01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	262 tests.append(Test("Bootstrap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcher, methodMatcher], ignoredVMs=['client', 'server'], benchmarkCompilationRate=False))
01512eb1ae7b bootstrap: print number of compiled methods Bernhard Urban <bernhard.urban@jku.at> parents: 11533 diff changeset	263 tests.append(Test("Bootstrap-bigHeap", ['-version'], successREs=[time], scoreMatchers=[scoreMatcherBig, methodMatcherBig], vmOpts=['-Xms2g'], ignoredVMs=['client', 'server'], benchmarkCompilationRate=False))
4219 47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	264 return tests
47f7d91d34cf Fix javac build issues with classpath Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4217 diff changeset	265
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	266 class CTWMode:
17378 d42e11af980d mx: simplified ctw command Doug Simon <doug.simon@oracle.com> parents: 17362 diff changeset	267 Full, NoInline = range(2)
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	268
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	269 def getCTW(vm, mode):
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	270 time = re.compile(r"CompileTheWorld : Done $[0-9]+ classes, [0-9]+ methods, (?P<time>[0-9]+) ms$")
9180 4e6df9021a59 Assemble the CTW bench in the same value, remove PEA from nocomplex Gilles Duboscq <duboscq@ssw.jku.at> parents: 9173 diff changeset	271 scoreMatcher = ValuesMatcher(time, {'group' : 'CompileTheWorld', 'name' : 'CompileTime', 'score' : '<time>'})
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	272
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	273 jre = os.environ.get('JAVA_HOME')
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	274 if exists(join(jre, 'jre')):
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	275 jre = join(jre, 'jre')
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	276 rtjar = join(jre, 'lib', 'rt.jar')
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	277
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	278
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	279 args = ['-XX:+CompileTheWorld', '-Xbootclasspath/p:' + rtjar]
12647 b038b643a3a4 restricted BootstrapGraal option to GRAALVM builds only Doug Simon <doug.simon@oracle.com> parents: 11658 diff changeset	280 if vm == 'graal':
b038b643a3a4 restricted BootstrapGraal option to GRAALVM builds only Doug Simon <doug.simon@oracle.com> parents: 11658 diff changeset	281 args += ['-XX:+BootstrapGraal']
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	282 if mode >= CTWMode.NoInline:
12769 99769479f9ce s/commands/mx_graal/ in mx/sanitycheck.py Gilles Duboscq <duboscq@ssw.jku.at> parents: 12647 diff changeset	283 if not mx_graal.isGraalEnabled(vm):
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	284 args.append('-XX:-Inline')
a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	285 else:
17378 d42e11af980d mx: simplified ctw command Doug Simon <doug.simon@oracle.com> parents: 17362 diff changeset	286 args.append('-G:CompileTheWordConfig=-Inline')
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	287
9180 4e6df9021a59 Assemble the CTW bench in the same value, remove PEA from nocomplex Gilles Duboscq <duboscq@ssw.jku.at> parents: 9173 diff changeset	288 return Test("CompileTheWorld", args, successREs=[time], scoreMatchers=[scoreMatcher], benchmarkCompilationRate=False)
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	289
9157 a38d748d4130 Add support for the 'optimized' build in mx Gilles Duboscq <duboscq@ssw.jku.at> parents: 8506 diff changeset	290
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	291 class Tee:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	292 def __init__(self):
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	293 self.output = StringIO.StringIO()
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	294 def eat(self, line):
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	295 self.output.write(line)
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	296 sys.stdout.write(line)
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	297
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	298 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	299 Encapsulates a single program that is a sanity test and/or a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	300 """
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	301 class Test:
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	302 def __init__(self, name, cmd, successREs=None, failureREs=None, scoreMatchers=None, vmOpts=None, defaultCwd=None, ignoredVMs=None, benchmarkCompilationRate=False):
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	303
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	304 self.name = name
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	305 self.successREs = _noneAsEmptyList(successREs)
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	306 self.failureREs = _noneAsEmptyList(failureREs) + [re.compile(r"Exception occurred in scope: ")]
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	307 self.scoreMatchers = _noneAsEmptyList(scoreMatchers)
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	308 self.vmOpts = _noneAsEmptyList(vmOpts)
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	309 self.cmd = cmd
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	310 self.defaultCwd = defaultCwd
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	311 self.ignoredVMs = _noneAsEmptyList(ignoredVMs)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	312 self.benchmarkCompilationRate = benchmarkCompilationRate
7654 641a4c6ac1ce fixed repetition of compilation metric option on command line when running multiple benchmarks (JIRA issue GRAAL-66) Doug Simon <doug.simon@oracle.com> parents: 7571 diff changeset	313 if benchmarkCompilationRate:
641a4c6ac1ce fixed repetition of compilation metric option on command line when running multiple benchmarks (JIRA issue GRAAL-66) Doug Simon <doug.simon@oracle.com> parents: 7571 diff changeset	314 self.vmOpts = self.vmOpts + ['-XX:+CITime']
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	315
4238 e0d09e05aa9b Made the gate time each DaCapo test individually. Doug Simon <doug.simon@oracle.com> parents: 4230 diff changeset	316 def __str__(self):
e0d09e05aa9b Made the gate time each DaCapo test individually. Doug Simon <doug.simon@oracle.com> parents: 4230 diff changeset	317 return self.name
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	318
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	319 def test(self, vm, cwd=None, extraVmOpts=None, vmbuild=None):
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	320 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	321 Run this program as a sanity test.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	322 """
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	323 if vm in self.ignoredVMs:
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	324 return True
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	325 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	326 cwd = self.defaultCwd
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	327 parser = OutputParser()
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	328 jvmError = re.compile(r"(?P<jvmerror>([A-Z]:\|/).*[/\\]hs_err_pid[0-9]+\.log)")
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	329 parser.addMatcher(ValuesMatcher(jvmError, {'jvmError' : '<jvmerror>'}))
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	330
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	331 for successRE in self.successREs:
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	332 parser.addMatcher(ValuesMatcher(successRE, {'passed' : '1'}))
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	333 for failureRE in self.failureREs:
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	334 parser.addMatcher(ValuesMatcher(failureRE, {'failed' : '1'}))
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	335
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	336 tee = Tee()
12769 99769479f9ce s/commands/mx_graal/ in mx/sanitycheck.py Gilles Duboscq <duboscq@ssw.jku.at> parents: 12647 diff changeset	337 retcode = mx_graal.vm(self.vmOpts + _noneAsEmptyList(extraVmOpts) + self.cmd, vm, nonZeroIsFatal=False, out=tee.eat, err=subprocess.STDOUT, cwd=cwd, vmbuild=vmbuild)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	338 output = tee.output.getvalue()
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	339 valueMaps = parser.parse(output)
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	340
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	341 if len(valueMaps) == 0:
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	342 return False
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	343
8206 22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	344 record = {}
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	345 for valueMap in valueMaps:
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	346 for key, value in valueMap.items():
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	347 if record.has_key(key) and record[key] != value:
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	348 mx.abort('Inconsistant values returned by test machers : ' + str(valueMaps))
22429580c7a8 sanitychecks.py: Collapse valueMaps in test for the case where multiple matcher return matches Gilles Duboscq <duboscq@ssw.jku.at> parents: 8147 diff changeset	349 record[key] = value
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	350
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	351 jvmErrorFile = record.get('jvmError')
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	352 if jvmErrorFile:
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	353 mx.log('/!\\JVM Error : dumping error log...')
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	354 with open(jvmErrorFile, 'rb') as fp:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	355 mx.log(fp.read())
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	356 os.unlink(jvmErrorFile)
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	357 return False
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	358
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	359 if record.get('failed') == '1':
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	360 return False
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	361
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	362 return retcode == 0 and record.get('passed') == '1'
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	363
dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	364 def bench(self, vm, cwd=None, extraVmOpts=None, vmbuild=None):
4225 339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	365 """
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	366 Run this program as a benchmark.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout. Doug Simon <doug.simon@oracle.com> parents: 4219 diff changeset	367 """
13967 4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings Bernhard Urban <bernhard.urban@jku.at> parents: 12769 diff changeset	368 if vm in self.ignoredVMs:
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	369 return {}
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	370 if cwd is None:
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	371 cwd = self.defaultCwd
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	372 parser = OutputParser()
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	373
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	374 for successRE in self.successREs:
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	375 parser.addMatcher(ValuesMatcher(successRE, {'passed' : '1'}))
4228 e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4225 diff changeset	376 for failureRE in self.failureREs:
7564 c420a487b10f changed convention for specifying constants versus named groups in a template for a value to extract from the output of a benchmark execution Doug Simon <doug.simon@oracle.com> parents: 7563 diff changeset	377 parser.addMatcher(ValuesMatcher(failureRE, {'failed' : '1'}))
4215 a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	378 for scoreMatcher in self.scoreMatchers:
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments. Gilles Duboscq <gilles.m.duboscq@gmail.com> parents: 4150 diff changeset	379 parser.addMatcher(scoreMatcher)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	380
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	381 if self.benchmarkCompilationRate:
7570 2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	382 if vm == 'graal':
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	383 bps = re.compile(r"ParsedBytecodesPerSecond@final: (?P<rate>[0-9]+)")
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	384 ibps = re.compile(r"InlinedBytecodesPerSecond@final: (?P<rate>[0-9]+)")
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	385 parser.addMatcher(ValuesMatcher(bps, {'group' : 'ParsedBytecodesPerSecond', 'name' : self.name, 'score' : '<rate>'}))
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	386 parser.addMatcher(ValuesMatcher(ibps, {'group' : 'InlinedBytecodesPerSecond', 'name' : self.name, 'score' : '<rate>'}))
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	387 else:
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	388 ibps = re.compile(r"(?P<compiler>[\w]+) compilation speed: +(?P<rate>[0-9]+) bytes/s {standard")
2025455e7d80 added collection of compilation speed metric for C1 and C2 to bench command Doug Simon <doug.simon@oracle.com> parents: 7569 diff changeset	389 parser.addMatcher(ValuesMatcher(ibps, {'group' : 'InlinedBytecodesPerSecond', 'name' : '<compiler>:' + self.name, 'score' : '<rate>'}))
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	390
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	391 startDelim = 'START: ' + self.name
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	392 endDelim = 'END: ' + self.name
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	393
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	394 outputfile = os.environ.get('BENCH_OUTPUT', None)
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	395 if outputfile:
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	396 # Used only to debug output parsing
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	397 with open(outputfile) as fp:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	398 output = fp.read()
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	399 start = output.find(startDelim)
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	400 end = output.find(endDelim, start)
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	401 if start == -1 and end == -1:
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	402 return {}
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	403 output = output[start + len(startDelim + os.linesep): end]
a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	404 mx.log(startDelim)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	405 mx.log(output)
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	406 mx.log(endDelim)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	407 else:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	408 tee = Tee()
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	409 mx.log(startDelim)
12769 99769479f9ce s/commands/mx_graal/ in mx/sanitycheck.py Gilles Duboscq <duboscq@ssw.jku.at> parents: 12647 diff changeset	410 if mx_graal.vm(self.vmOpts + _noneAsEmptyList(extraVmOpts) + self.cmd, vm, nonZeroIsFatal=False, out=tee.eat, err=subprocess.STDOUT, cwd=cwd, vmbuild=vmbuild) != 0:
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	411 mx.abort("Benchmark failed (non-zero retcode)")
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	412 mx.log(endDelim)
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	413 output = tee.output.getvalue()
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	414
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	415 groups = {}
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	416 passed = False
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	417 for valueMap in parser.parse(output):
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	418 assert (valueMap.has_key('name') and valueMap.has_key('score') and valueMap.has_key('group')) or valueMap.has_key('passed') or valueMap.has_key('failed'), valueMap
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	419 if valueMap.get('failed') == '1':
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	420 mx.abort("Benchmark failed")
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	421 if valueMap.get('passed') == '1':
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	422 passed = True
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	423 groupName = valueMap.get('group')
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	424 if groupName:
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	425 group = groups.setdefault(groupName, {})
7563 3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	426 name = valueMap.get('name')
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	427 score = valueMap.get('score')
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	428 if name and score:
3aab15f42934 moved execution of a benchmark out of OutputParser Doug Simon <doug.simon@oracle.com> parents: 7562 diff changeset	429 group[name] = score
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	430
4332 9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	431 if not passed:
9dbcdd6f1464 If a benchmark fails it should fail hard, not silently Gilles Duboscq <duboscq@ssw.jku.at> parents: 4331 diff changeset	432 mx.abort("Benchmark failed (not passed)")
11514 dc3c8df55905 added support for pylint and fixed errors/warnings it found Doug Simon <doug.simon@oracle.com> parents: 11367 diff changeset	433
7567 a8bc60aeacb8 fix bug in parsing of SPECjvm2008 output Doug Simon <doug.simon@oracle.com> parents: 7564 diff changeset	434 return groups

Mercurial > hg > truffle

annotate mx/sanitycheck.py @ 18843:5e80dd2f1783