annotate mx/outputparser.py @ 4225:339cf8d4904d

Made mx.run work properly when stderr is redirected to stdout. Made outputparser redirect stderr to stdout. Added copyright headers to outputparser.py and sanitycheck.py. Reverted class path construction in mx.build() to use includeSelf=True to fix regression when running 'mx build -c'.
author Doug Simon <doug.simon@oracle.com>
date Thu, 05 Jan 2012 11:31:46 +0100
parents 47f7d91d34cf
children 063ea022532c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
1 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
2 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
5 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
6 # This code is free software; you can redistribute it and/or modify it
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
7 # under the terms of the GNU General Public License version 2 only, as
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
8 # published by the Free Software Foundation.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
9 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
10 # This code is distributed in the hope that it will be useful, but WITHOUT
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
13 # version 2 for more details (a copy is included in the LICENSE file that
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
14 # accompanied this code).
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
15 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
16 # You should have received a copy of the GNU General Public License version
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
17 # 2 along with this work; if not, write to the Free Software Foundation,
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
19 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
21 # or visit www.oracle.com if you need additional information or have any
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
22 # questions.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
23 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
24 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
25
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
26 import mx
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
27 import commands
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
28 import subprocess
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
29
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
30 class OutputParser:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
31
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
32 def __init__(self, nonZeroIsFatal=True):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
33 self.matchers = []
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
34 self.nonZeroIsFatal = nonZeroIsFatal
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
35
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
36 def addMatcher(self, matcher):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
37 self.matchers.append(matcher)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
38
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
39 def parse(self, vm, cmd, cwd=None, vmbuild=None):
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
40 ret = [{}]
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
41
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
42 def parseLine(line):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
43 anyMatch = False
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
44 for matcher in self.matchers:
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4151
diff changeset
45 parsed = matcher.parse(line.strip())
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
46 if parsed:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
47 anyMatch = True
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
48 if matcher.startNewLine and len(ret[0]) > 0:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
49 ret.append({})
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
50 ret[len(ret)-1].update(parsed)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
51 if anyMatch :
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4151
diff changeset
52 mx.log('>' + line.rstrip())
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
53 else :
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4151
diff changeset
54 mx.log( line.rstrip())
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
55
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
56 retcode = commands.vm(cmd, vm, nonZeroIsFatal=self.nonZeroIsFatal, out=parseLine, err=subprocess.STDOUT, cwd=cwd, vmbuild=vmbuild)
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4151
diff changeset
57 return {'parsed' : ret, 'retcode' : retcode}
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
58
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
59 class Matcher:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
60
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
61 def __init__(self, regex, valuesToParse, startNewLine=False):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
62 assert isinstance(valuesToParse, dict)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
63 self.regex = regex
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
64 self.valuesToParse = valuesToParse
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
65 self.startNewLine = startNewLine
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
66
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
67 def parse(self, line):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
68 match = self.regex.search(line)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
69 if not match:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
70 return False
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
71 ret = {}
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
72 for key, value in self.valuesToParse.items():
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
73 ret[self.parsestr(match, key)] = self.parsestr(match, value)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
74 return ret
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
75
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
76 def parsestr(self, match, key):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
77 if isinstance(key, tuple):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
78 if len(key) != 2:
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
79 raise Exception('Tuple arguments must have a length of 2')
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
80 tup1, tup2 = key
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
81 # check if key is a function
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
82 if hasattr(tup1, '__call__'):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
83 return tup1(self.parsestr(match, tup2))
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
84 elif hasattr(tup2, '__call__'):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
85 return tup2(self.parsestr(match, tup1))
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
86 else:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
87 raise Exception('Tuple must contain a function pointer')
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
88 elif key.startswith('const:'):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
89 return key.split(':')[1]
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
90 else:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
91 return match.group(key)