annotate mx/outputparser.py @ 4282:063ea022532c

mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
author Gilles Duboscq <gilles.m.duboscq@gmail.com>
date Fri, 13 Jan 2012 16:11:18 +0100
parents 339cf8d4904d
children 3aab15f42934
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
1 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
2 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
3 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
4 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
5 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
6 # This code is free software; you can redistribute it and/or modify it
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
7 # under the terms of the GNU General Public License version 2 only, as
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
8 # published by the Free Software Foundation.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
9 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
10 # This code is distributed in the hope that it will be useful, but WITHOUT
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
11 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
12 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
13 # version 2 for more details (a copy is included in the LICENSE file that
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
14 # accompanied this code).
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
15 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
16 # You should have received a copy of the GNU General Public License version
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
17 # 2 along with this work; if not, write to the Free Software Foundation,
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
18 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
19 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
20 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
21 # or visit www.oracle.com if you need additional information or have any
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
22 # questions.
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
23 #
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
24 # ----------------------------------------------------------------------------------------------------
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
25
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
26 import mx
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
27 import commands
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
28 import subprocess
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
29
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
30 class OutputParser:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
31
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
32 def __init__(self, nonZeroIsFatal=True):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
33 self.matchers = []
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
34 self.nonZeroIsFatal = nonZeroIsFatal
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
35
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
36 def addMatcher(self, matcher):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
37 self.matchers.append(matcher)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
38
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
39 def parse(self, vm, cmd, cwd=None, vmbuild=None):
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
40 ret = []
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
41
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
42 def parseLine(line):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
43 anyMatch = False
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
44 for matcher in self.matchers:
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4151
diff changeset
45 parsed = matcher.parse(line.strip())
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
46 if parsed:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
47 anyMatch = True
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4225
diff changeset
48 if len(ret) is 0 or (matcher.startNewLine and len(ret[len(ret)-1]) > 0):
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
49 ret.append({})
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
50 ret[len(ret)-1].update(parsed)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
51 if anyMatch :
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4151
diff changeset
52 mx.log('>' + line.rstrip())
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
53 else :
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4151
diff changeset
54 mx.log( line.rstrip())
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
55
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
56 retcode = commands.vm(cmd, vm, nonZeroIsFatal=self.nonZeroIsFatal, out=parseLine, err=subprocess.STDOUT, cwd=cwd, vmbuild=vmbuild)
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4151
diff changeset
57 return {'parsed' : ret, 'retcode' : retcode}
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
58
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
59 class Matcher:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
60
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
61 def __init__(self, regex, valuesToParse, startNewLine=False):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
62 assert isinstance(valuesToParse, dict)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
63 self.regex = regex
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
64 self.valuesToParse = valuesToParse
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
65 self.startNewLine = startNewLine
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
66
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
67 def parse(self, line):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
68 match = self.regex.search(line)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
69 if not match:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
70 return False
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
71 ret = {}
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
72 for key, value in self.valuesToParse.items():
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
73 ret[self.parsestr(match, key)] = self.parsestr(match, value)
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
74 return ret
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
75
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
76 def parsestr(self, match, key):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
77 if isinstance(key, tuple):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
78 if len(key) != 2:
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
79 raise Exception('Tuple arguments must have a length of 2')
4151
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
80 tup1, tup2 = key
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
81 # check if key is a function
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
82 if hasattr(tup1, '__call__'):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
83 return tup1(self.parsestr(match, tup2))
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
84 elif hasattr(tup2, '__call__'):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
85 return tup2(self.parsestr(match, tup1))
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
86 else:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
87 raise Exception('Tuple must contain a function pointer')
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
88 elif key.startswith('const:'):
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
89 return key.split(':')[1]
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
90 else:
cb22fcb2e2fc missing file
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents:
diff changeset
91 return match.group(key)