Mercurial > hg > truffle

annotate mx/mx_graal.py @ 15461:f73fc9309f12

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
gate: use parallelized Java building in the gate
author Doug Simon <doug.simon@oracle.com>
date Thu, 01 May 2014 23:55:22 +0200
parents 5947bb02474f
children f2132fab8a6f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
1 #
4225
339cf8d4904d Made mx.run work properly when stderr is redirected to stdout.
Doug Simon <doug.simon@oracle.com>
parents: 4219
diff changeset
2 # commands.py - the GraalVM specific commands
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
3 #
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
4 # ----------------------------------------------------------------------------------------------------
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
5 #
4187
ac5c2bdfcca2 Disabled copyright check in the gate until the CheckCopyright tools supports the HotSpot copyright notice format.
Doug Simon <doug.simon@oracle.com>
parents: 4185
diff changeset
6 # Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
7 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
8 #
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
9 # This code is free software; you can redistribute it and/or modify it
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
10 # under the terms of the GNU General Public License version 2 only, as
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
11 # published by the Free Software Foundation.
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
12 #
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
13 # This code is distributed in the hope that it will be useful, but WITHOUT
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
14 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
15 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
16 # version 2 for more details (a copy is included in the LICENSE file that
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
17 # accompanied this code).
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
18 #
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
19 # You should have received a copy of the GNU General Public License version
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
20 # 2 along with this work; if not, write to the Free Software Foundation,
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
21 # Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
22 #
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
23 # Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
24 # or visit www.oracle.com if you need additional information or have any
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
25 # questions.
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
26 #
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
27 # ----------------------------------------------------------------------------------------------------
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
28
15402
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
29 import os, sys, shutil, zipfile, tarfile, tempfile, re, time, datetime, platform, subprocess, multiprocessing, StringIO, socket
8406
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
30 from os.path import join, exists, dirname, basename, getmtime
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
31 from argparse import ArgumentParser, RawDescriptionHelpFormatter, REMAINDER
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
32 from outputparser import OutputParser, ValuesMatcher
4144
34527a24bcf8 Adopted mxtool changes.
Doug Simon <doug.simon@oracle.com>
parents: 3736
diff changeset
33 import mx
13472
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
34 import xml.dom.minidom
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4187
diff changeset
35 import sanitycheck
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
36 import itertools
9116
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
37 import json, textwrap
15329
3b56c9bbf60c mx: fix imports
Bernhard Urban <bernhard.urban@jku.at>
parents: 15328
diff changeset
38 import fnmatch
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
39
10574
0cad5096735e commands.py: Make sure _jdk returns an absolute path. Use _jdk and _jdksDir where necessary
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10570
diff changeset
40 # This works because when mx loads this file, it makes sure __file__ gets an absolute path
4144
34527a24bcf8 Adopted mxtool changes.
Doug Simon <doug.simon@oracle.com>
parents: 3736
diff changeset
41 _graal_home = dirname(dirname(__file__))
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
42
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
43 """ Used to distinguish an exported GraalVM (see 'mx export'). """
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
44 _vmSourcesAvailable = exists(join(_graal_home, 'make')) and exists(join(_graal_home, 'src'))
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
45
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
46 """ The VMs that can be built and run along with an optional description. Only VMs with a
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
47 description are listed in the dialogue for setting the default VM (see _get_vm()). """
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
48 _vmChoices = {
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
49 'graal' : 'All compilation is performed with Graal. This includes bootstrapping Graal itself unless -XX:-BootstrapGraal is used.',
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
50 'server' : 'Normal compilation is performed with the tiered system (i.e., client + server), Truffle compilation is performed with Graal. Use this for optimal Truffle performance.',
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
51 'client' : None, # normal compilation with client compiler, explicit compilation (e.g., by Truffle) with Graal
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
52 'server-nograal' : None, # all compilation with tiered system (i.e., client + server), Graal omitted
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
53 'client-nograal' : None, # all compilation with client compiler, Graal omitted
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
54 'original' : None, # default VM copied from bootstrap JDK
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
55 }
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
56
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
57 """ The VM that will be run by the 'vm' command and built by default by the 'build' command.
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
58 This can be set via the global '--vm' option or the DEFAULT_VM environment variable.
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
59 It can also be temporarily set by using of a VM context manager object in a 'with' statement. """
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
60 _vm = None
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
61
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
62 """ The VM builds that will be run by the 'vm' command - default is first in list """
9157
a38d748d4130 Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9130
diff changeset
63 _vmbuildChoices = ['product', 'fastdebug', 'debug', 'optimized']
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
64
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
65 """ The VM build that will be run by the 'vm' command.
11371
74414b248381 mx: added --vmbuild global option, removed --product, --fastdebug, --debug options as well as suffix of build command
Doug Simon <doug.simon@oracle.com>
parents: 11368
diff changeset
66 This can be set via the global '--vmbuild' option.
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
67 It can also be temporarily set by using of a VM context manager object in a 'with' statement. """
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
68 _vmbuild = _vmbuildChoices[0]
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
69
5024
7ccdae96e98a Allow JaCoCon to append to the execution file
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5016
diff changeset
70 _jacoco = 'off'
4684
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
71
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
72 """ The current working directory to switch to before running the VM. """
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
73 _vm_cwd = None
10454
590b0c26877f mx: add --workdir argument
Bernhard Urban <bernhard.urban@jku.at>
parents: 10417
diff changeset
74
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
75 """ The base directory in which the JDKs cloned from $JAVA_HOME exist. """
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
76 _installed_jdks = None
10570
2f80624df8a2 Add a --vmdir argument to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10568
diff changeset
77
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
78 """ Prefix for running the VM. """
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
79 _vm_prefix = None
5875
000fb0550afe Add an option to launch the vm from a debugger in mx's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5871
diff changeset
80
5194
a6eceb5efb0e added --ecl option to mx for saving VM execution(s) as Eclipse launch configurations
Doug Simon <doug.simon@oracle.com>
parents: 5189
diff changeset
81 _make_eclipse_launch = False
a6eceb5efb0e added --ecl option to mx for saving VM execution(s) as Eclipse launch configurations
Doug Simon <doug.simon@oracle.com>
parents: 5189
diff changeset
82
14762
c6f4b780fa88 mx: increase minimum required jdk version
Bernhard Urban <bernhard.urban@jku.at>
parents: 14751
diff changeset
83 # @CallerSensitive introduced in 7u25
c6f4b780fa88 mx: increase minimum required jdk version
Bernhard Urban <bernhard.urban@jku.at>
parents: 14751
diff changeset
84 _minVersion = mx.VersionSpec('1.7.0_25')
7916
9bff64f43299 Better java version numbers support in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 7788
diff changeset
85
13919
9d70445ea369 mx: set correct permissions for graal.jar
Bernhard Urban <bernhard.urban@jku.at>
parents: 13846
diff changeset
86 JDK_UNIX_PERMISSIONS = 0755
9d70445ea369 mx: set correct permissions for graal.jar
Bernhard Urban <bernhard.urban@jku.at>
parents: 13846
diff changeset
87
14602
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
88 def isVMSupported(vm):
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
89 if 'client' in vm and len(platform.mac_ver()[0]) != 0:
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
90 # Client VM not supported: java launcher on Mac OS X translates '-client' to '-server'
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
91 return False
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
92 return True
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
93
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
94 def _get_vm():
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
95 """
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
96 Gets the configured VM, presenting a dialogue if there is no currently configured VM.
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
97 """
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
98 global _vm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
99 if _vm:
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
100 return _vm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
101 vm = mx.get_env('DEFAULT_VM')
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
102 if vm is None:
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
103 if not sys.stdout.isatty():
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
104 mx.abort('Need to specify VM with --vm option or DEFAULT_VM environment variable')
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
105 envPath = join(_graal_home, 'mx', 'env')
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
106 mx.log('Please select the VM to be executed from the following: ')
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
107 items = [k for k in _vmChoices.keys() if _vmChoices[k] is not None]
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
108 descriptions = [_vmChoices[k] for k in _vmChoices.keys() if _vmChoices[k] is not None]
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
109 vm = mx.select_items(items, descriptions, allowMultiple=False)
11511
3110bea9a6b0 mx: add helper method for yes/no question
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11493
diff changeset
110 if mx.ask_yes_no('Persist this choice by adding "DEFAULT_VM=' + vm + '" to ' + envPath, 'y'):
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
111 with open(envPath, 'a') as fp:
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
112 print >> fp, 'DEFAULT_VM=' + vm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
113 _vm = vm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
114 return vm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
115
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
116 """
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
117 A context manager that can be used with the 'with' statement to set the VM
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
118 used by all VM executions within the scope of the 'with' statement. For example:
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
119
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
120 with VM('server'):
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
121 dacapo(['pmd'])
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
122 """
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
123 class VM:
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
124 def __init__(self, vm=None, build=None):
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
125 assert vm is None or vm in _vmChoices.keys()
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
126 assert build is None or build in _vmbuildChoices
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
127 self.vm = vm if vm else _vm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
128 self.build = build if build else _vmbuild
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
129 self.previousVm = _vm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
130 self.previousBuild = _vmbuild
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
131
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
132 def __enter__(self):
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
133 global _vm, _vmbuild
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
134 _vm = self.vm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
135 _vmbuild = self.build
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
136
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
137 def __exit__(self, exc_type, exc_value, traceback):
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
138 global _vm, _vmbuild
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
139 _vm = self.previousVm
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
140 _vmbuild = self.previousBuild
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
141
6705
0cb1ac637dd0 Added recursive chmod for cases where JAVA_HOME is not owned by graal workspace owner.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6702
diff changeset
142 def _chmodDir(chmodFlags, dirname, fnames):
0cb1ac637dd0 Added recursive chmod for cases where JAVA_HOME is not owned by graal workspace owner.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6702
diff changeset
143 os.chmod(dirname, chmodFlags)
7509
442668d41bc2 fixed warnings
Doug Simon <doug.simon@oracle.com>
parents: 7345
diff changeset
144 for name in fnames:
442668d41bc2 fixed warnings
Doug Simon <doug.simon@oracle.com>
parents: 7345
diff changeset
145 os.chmod(os.path.join(dirname, name), chmodFlags)
6705
0cb1ac637dd0 Added recursive chmod for cases where JAVA_HOME is not owned by graal workspace owner.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6702
diff changeset
146
0cb1ac637dd0 Added recursive chmod for cases where JAVA_HOME is not owned by graal workspace owner.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6702
diff changeset
147 def chmodRecursive(dirname, chmodFlags):
0cb1ac637dd0 Added recursive chmod for cases where JAVA_HOME is not owned by graal workspace owner.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6702
diff changeset
148 os.path.walk(dirname, _chmodDir, chmodFlags)
0cb1ac637dd0 Added recursive chmod for cases where JAVA_HOME is not owned by graal workspace owner.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6702
diff changeset
149
4144
34527a24bcf8 Adopted mxtool changes.
Doug Simon <doug.simon@oracle.com>
parents: 3736
diff changeset
150 def clean(args):
6291
633136426f26 consolidated jtt command into unittest command and removed need for explicit 'testHarness' property in 'projects' file
Doug Simon <doug.simon@oracle.com>
parents: 6278
diff changeset
151 """clean the GraalVM source tree"""
4247
3f6c6e61614e Changed 'mx build' such that all Java sources for a project are compiled together instead of by source directory.
Doug Simon <doug.simon@oracle.com>
parents: 4244
diff changeset
152 opts = mx.clean(args, parser=ArgumentParser(prog='mx clean'))
4226
e03ff10d4bfc Made the gate clean the Java projects only.
Doug Simon <doug.simon@oracle.com>
parents: 4225
diff changeset
153 if opts.native:
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
154 def rmIfExists(name):
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
155 if os.path.isdir(name):
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
156 shutil.rmtree(name)
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
157 elif os.path.isfile(name):
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
158 os.unlink(name)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
159
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
160 rmIfExists(join(_graal_home, 'build'))
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
161 rmIfExists(join(_graal_home, 'build-nograal'))
9119
0c9c4ae236b2 refactoring to ensure the path to the JDKs dir is consistent between the _jdk() and clean() command
Doug Simon <doug.simon@oracle.com>
parents: 9118
diff changeset
162 rmIfExists(_jdksDir())
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
163 rmIfExists(mx.distribution('GRAAL').path)
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
164
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
165 def export(args):
15420
5947bb02474f mx export: change the way of marking a repo as dirty
Bernhard Urban <bernhard.urban@jku.at>
parents: 15402
diff changeset
166 """create archives of builds split by vmbuild and vm"""
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
167
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
168 parser = ArgumentParser(prog='mx export')
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
169 args = parser.parse_args(args)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
170
15402
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
171 # collect data about export
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
172 infos = dict()
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
173 infos['timestamp'] = time.time()
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
174
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
175 hgcfg = mx.HgConfig()
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
176 hgcfg.check()
15420
5947bb02474f mx export: change the way of marking a repo as dirty
Bernhard Urban <bernhard.urban@jku.at>
parents: 15402
diff changeset
177 infos['revision'] = hgcfg.tip('.') + ('+' if hgcfg.isDirty('.') else '')
15402
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
178 # TODO: infos['repository']
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
179
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
180 infos['jdkversion'] = str(mx.java().version)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
181
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
182 infos['architecture'] = _arch()
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
183 infos['platform'] = mx.get_os()
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
184
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
185 if mx.get_os != 'windows':
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
186 pass
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
187 # infos['ccompiler']
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
188 # infos['linker']
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
189
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
190 infos['hostname'] = socket.gethostname()
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
191
15402
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
192 def _writeJson(suffix, properties):
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
193 d = infos.copy()
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
194 for k, v in properties.iteritems():
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
195 assert not d.has_key(k)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
196 d[k] = v
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
197
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
198 jsonFileName = 'export-' + suffix + '.json'
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
199 with open(jsonFileName, 'w') as f:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
200 print >> f, json.dumps(d)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
201 return jsonFileName
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
202
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
203
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
204 def _genFileName(archivtype, middle):
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
205 idPrefix = infos['revision'] + '_'
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
206 idSuffix = '.tar.gz'
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
207 return join(_graal_home, "graalvm_" + archivtype + "_" + idPrefix + middle + idSuffix)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
208
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
209 def _genFileArchPlatformName(archivtype, middle):
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
210 return _genFileName(archivtype, infos['platform'] + '_' + infos['architecture'] + '_' + middle)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
211
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
212
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
213 # archive different build types of hotspot
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
214 for vmBuild in _vmbuildChoices:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
215 jdkpath = join(_jdksDir(), vmBuild)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
216 if not exists(jdkpath):
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
217 mx.logv("skipping " + vmBuild)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
218 continue
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
219
15402
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
220 tarName = _genFileArchPlatformName('basejdk', vmBuild)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
221 mx.logv("creating basejdk " + tarName)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
222 vmSet = set()
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
223 with tarfile.open(tarName, 'w:gz') as tar:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
224 for root, _, files in os.walk(jdkpath):
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
225 if basename(root) in _vmChoices.keys():
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
226 # TODO: add some assert to check path assumption
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
227 vmSet.add(root)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
228 continue
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
229
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
230 for f in files:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
231 name = join(root, f)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
232 # print name
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
233 tar.add(name, name)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
234
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
235 n = _writeJson("basejdk-" + vmBuild, {'vmbuild' : vmBuild})
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
236 tar.add(n, n)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
237
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
238 # create a separate archive for each VM
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
239 for vm in vmSet:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
240 bVm = basename(vm)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
241 vmTarName = _genFileArchPlatformName('vm', vmBuild + '_' + bVm)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
242 mx.logv("creating vm " + vmTarName)
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
243
15402
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
244 debugFiles = set()
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
245 with tarfile.open(vmTarName, 'w:gz') as tar:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
246 for root, _, files in os.walk(vm):
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
247 for f in files:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
248 # TODO: mac, windows, solaris?
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
249 if any(map(f.endswith, [".debuginfo"])):
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
250 debugFiles.add(f)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
251 else:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
252 name = join(root, f)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
253 # print name
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
254 tar.add(name, name)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
255
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
256 n = _writeJson("vm-" + vmBuild + "-" + bVm, {'vmbuild' : vmBuild, 'vm' : bVm})
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
257 tar.add(n, n)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
258
15402
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
259 if len(debugFiles) > 0:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
260 debugTarName = _genFileArchPlatformName('debugfilesvm', vmBuild + '_' + bVm)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
261 mx.logv("creating debugfilesvm " + debugTarName)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
262 with tarfile.open(debugTarName, 'w:gz') as tar:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
263 for f in debugFiles:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
264 name = join(root, f)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
265 # print name
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
266 tar.add(name, name)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
267
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
268 n = _writeJson("debugfilesvm-" + vmBuild + "-" + bVm, {'vmbuild' : vmBuild, 'vm' : bVm})
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
269 tar.add(n, n)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
270
15402
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
271 # graal directory
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
272 graalDirTarName = _genFileName('classfiles', 'javac')
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
273 mx.logv("creating graal " + graalDirTarName)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
274 with tarfile.open(graalDirTarName, 'w:gz') as tar:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
275 for root, _, files in os.walk("graal"):
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
276 for f in [f for f in files if not f.endswith('.java')]:
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
277 name = join(root, f)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
278 # print name
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
279 tar.add(name, name)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
280
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
281 n = _writeJson("graal", {'javacompiler' : 'javac'})
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
282 tar.add(n, n)
5dcf0ae606f3 mx: new export command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15401
diff changeset
283
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
284
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
285 def _run_benchmark(args, availableBenchmarks, runBenchmark):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
286
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
287 vmOpts, benchmarksAndOptions = _extract_VM_args(args, useDoubleDash=availableBenchmarks is None)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
288
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
289 if availableBenchmarks is None:
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
290 harnessArgs = benchmarksAndOptions
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
291 return runBenchmark(None, harnessArgs, vmOpts)
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4187
diff changeset
292
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
293 if len(benchmarksAndOptions) == 0:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
294 mx.abort('at least one benchmark name or "all" must be specified')
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
295 benchmarks = list(itertools.takewhile(lambda x: not x.startswith('-'), benchmarksAndOptions))
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
296 harnessArgs = benchmarksAndOptions[len(benchmarks):]
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
297
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
298 if 'all' in benchmarks:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
299 benchmarks = availableBenchmarks
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
300 else:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
301 for bm in benchmarks:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
302 if bm not in availableBenchmarks:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
303 mx.abort('unknown benchmark: ' + bm + '\nselect one of: ' + str(availableBenchmarks))
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
304
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4187
diff changeset
305 failed = []
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
306 for bm in benchmarks:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
307 if not runBenchmark(bm, harnessArgs, vmOpts):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
308 failed.append(bm)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
309
4157
b26279781d95 Simplified 'dacapo' command and made it more suitable for gate usage.
Doug Simon <doug.simon@oracle.com>
parents: 4156
diff changeset
310 if len(failed) != 0:
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
311 mx.abort('Benchmark failures: ' + str(failed))
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
312
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
313 def dacapo(args):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
314 """run one or more DaCapo benchmarks"""
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
315
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
316 def launcher(bm, harnessArgs, extraVmOpts):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
317 return sanitycheck.getDacapo(bm, harnessArgs).test(_get_vm(), extraVmOpts=extraVmOpts)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
318
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
319 _run_benchmark(args, sanitycheck.dacapoSanityWarmup.keys(), launcher)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
320
4480
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4438
diff changeset
321 def scaladacapo(args):
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
322 """run one or more Scala DaCapo benchmarks"""
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
323
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
324 def launcher(bm, harnessArgs, extraVmOpts):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
325 return sanitycheck.getScalaDacapo(bm, harnessArgs).test(_get_vm(), extraVmOpts=extraVmOpts)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
326
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
327 _run_benchmark(args, sanitycheck.dacapoScalaSanityWarmup.keys(), launcher)
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
328
6495
75f130f2b30f moved AMD64 specific HotSpot code in com.oracle.graal.hotspot.amd64 project
Doug Simon <doug.simon@oracle.com>
parents: 6340
diff changeset
329 def _arch():
6512
edea9ba7ac7b Windows-specific fix for mx.
Christian Haeubl <haeubl@ssw.jku.at>
parents: 6506
diff changeset
330 machine = platform.uname()[4]
6558
0a6d8857306d arch is i86pc on solaris x64 boxes.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6512
diff changeset
331 if machine in ['amd64', 'AMD64', 'x86_64', 'i86pc']:
6495
75f130f2b30f moved AMD64 specific HotSpot code in com.oracle.graal.hotspot.amd64 project
Doug Simon <doug.simon@oracle.com>
parents: 6340
diff changeset
332 return 'amd64'
11784
a2958b7bf83f add "sun4u" as a valid identifier for the SPARC architecture
Doug Simon <doug.simon@oracle.com>
parents: 11777
diff changeset
333 if machine in ['sun4v', 'sun4u']:
12540
0aa37fd2f33e mx: fix detection for sparc. ignore unparsable lines in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12497
diff changeset
334 return 'sparcv9'
6566
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
335 if machine == 'i386' and mx.get_os() == 'darwin':
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
336 try:
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
337 # Support for Snow Leopard and earlier version of MacOSX
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
338 if subprocess.check_output(['sysctl', '-n', 'hw.cpu64bit_capable']).strip() == '1':
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
339 return 'amd64'
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
340 except OSError:
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
341 # sysctl is not available
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
342 pass
ef48be91f5fd fixed detection of architecture on Snow Leopard and earlier versions of MacOSX
Doug Simon <doug.simon@oracle.com>
parents: 6558
diff changeset
343 mx.abort('unknown or unsupported architecture: os=' + mx.get_os() + ', machine=' + machine)
6495
75f130f2b30f moved AMD64 specific HotSpot code in com.oracle.graal.hotspot.amd64 project
Doug Simon <doug.simon@oracle.com>
parents: 6340
diff changeset
344
4607
ee87cfe2f8f9 Made GraalVM build work on Mac OS X.
Doug Simon <doug.simon@oracle.com>
parents: 4602
diff changeset
345 def _vmLibDirInJdk(jdk):
ee87cfe2f8f9 Made GraalVM build work on Mac OS X.
Doug Simon <doug.simon@oracle.com>
parents: 4602
diff changeset
346 """
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
347 Get the directory within a JDK where the server and client
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
348 subdirectories are located.
4607
ee87cfe2f8f9 Made GraalVM build work on Mac OS X.
Doug Simon <doug.simon@oracle.com>
parents: 4602
diff changeset
349 """
ee87cfe2f8f9 Made GraalVM build work on Mac OS X.
Doug Simon <doug.simon@oracle.com>
parents: 4602
diff changeset
350 if platform.system() == 'Darwin':
ee87cfe2f8f9 Made GraalVM build work on Mac OS X.
Doug Simon <doug.simon@oracle.com>
parents: 4602
diff changeset
351 return join(jdk, 'jre', 'lib')
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
352 if platform.system() == 'Windows':
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
353 return join(jdk, 'jre', 'bin')
6495
75f130f2b30f moved AMD64 specific HotSpot code in com.oracle.graal.hotspot.amd64 project
Doug Simon <doug.simon@oracle.com>
parents: 6340
diff changeset
354 return join(jdk, 'jre', 'lib', _arch())
4607
ee87cfe2f8f9 Made GraalVM build work on Mac OS X.
Doug Simon <doug.simon@oracle.com>
parents: 4602
diff changeset
355
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
356 def _vmCfgInJdk(jdk):
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
357 """
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
358 Get the jvm.cfg file.
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
359 """
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
360 if platform.system() == 'Windows':
6495
75f130f2b30f moved AMD64 specific HotSpot code in com.oracle.graal.hotspot.amd64 project
Doug Simon <doug.simon@oracle.com>
parents: 6340
diff changeset
361 return join(jdk, 'jre', 'lib', _arch(), 'jvm.cfg')
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
362 return join(_vmLibDirInJdk(jdk), 'jvm.cfg')
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
363
9119
0c9c4ae236b2 refactoring to ensure the path to the JDKs dir is consistent between the _jdk() and clean() command
Doug Simon <doug.simon@oracle.com>
parents: 9118
diff changeset
364 def _jdksDir():
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
365 return os.path.abspath(join(_installed_jdks if _installed_jdks else _graal_home, 'jdk' + str(mx.java().version)))
9119
0c9c4ae236b2 refactoring to ensure the path to the JDKs dir is consistent between the _jdk() and clean() command
Doug Simon <doug.simon@oracle.com>
parents: 9118
diff changeset
366
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
367 def _handle_missing_VM(bld, vm):
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
368 mx.log('The ' + bld + ' ' + vm + ' VM has not been created')
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
369 if sys.stdout.isatty():
11511
3110bea9a6b0 mx: add helper method for yes/no question
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11493
diff changeset
370 if mx.ask_yes_no('Build it now', 'y'):
11371
74414b248381 mx: added --vmbuild global option, removed --product, --fastdebug, --debug options as well as suffix of build command
Doug Simon <doug.simon@oracle.com>
parents: 11368
diff changeset
371 with VM(vm, bld):
74414b248381 mx: added --vmbuild global option, removed --product, --fastdebug, --debug options as well as suffix of build command
Doug Simon <doug.simon@oracle.com>
parents: 11368
diff changeset
372 build([])
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
373 return
11493
94779c895aad fixed documentation of --installed-jdks mx option
Doug Simon <doug.simon@oracle.com>
parents: 11424
diff changeset
374 mx.abort('You need to run "mx --vm ' + vm + ' --vmbuild ' + bld + ' build" to build the selected VM')
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
375
10574
0cad5096735e commands.py: Make sure _jdk returns an absolute path. Use _jdk and _jdksDir where necessary
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10570
diff changeset
376 def _jdk(build='product', vmToCheck=None, create=False, installGraalJar=True):
4155
394404b2d9bd Removed strict requirement for GRAAL environment variable. It only needs to be set now if the graal directory is not in the directory hierarchy of GraalVM JDK.
Doug Simon <doug.simon@oracle.com>
parents: 4153
diff changeset
377 """
394404b2d9bd Removed strict requirement for GRAAL environment variable. It only needs to be set now if the graal directory is not in the directory hierarchy of GraalVM JDK.
Doug Simon <doug.simon@oracle.com>
parents: 4153
diff changeset
378 Get the JDK into which Graal is installed, creating it first if necessary.
394404b2d9bd Removed strict requirement for GRAAL environment variable. It only needs to be set now if the graal directory is not in the directory hierarchy of GraalVM JDK.
Doug Simon <doug.simon@oracle.com>
parents: 4153
diff changeset
379 """
9119
0c9c4ae236b2 refactoring to ensure the path to the JDKs dir is consistent between the _jdk() and clean() command
Doug Simon <doug.simon@oracle.com>
parents: 9118
diff changeset
380 jdk = join(_jdksDir(), build)
4582
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
381 if create:
10574
0cad5096735e commands.py: Make sure _jdk returns an absolute path. Use _jdk and _jdksDir where necessary
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10570
diff changeset
382 srcJdk = mx.java().jdk
4582
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
383 if not exists(jdk):
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
384 mx.log('Creating ' + jdk + ' from ' + srcJdk)
13525
7cbd4f2194aa mx: omit weird logic to copy bootstrap jdk, just do cp -R
Bernhard Urban <bernhard.urban@jku.at>
parents: 13524
diff changeset
385 shutil.copytree(srcJdk, jdk)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
386
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
387 # Make a copy of the default VM so that this JDK can be
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
388 # reliably used as the bootstrap for a HotSpot build.
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
389 jvmCfg = _vmCfgInJdk(jdk)
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
390 if not exists(jvmCfg):
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
391 mx.abort(jvmCfg + ' does not exist')
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
392
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
393 defaultVM = None
8647
5f8299106dd9 A server0 "build" needs to create a jdk copy if it does not exist yet
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8646
diff changeset
394 jvmCfgLines = []
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
395 with open(jvmCfg) as f:
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
396 for line in f:
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
397 if line.startswith('-') and defaultVM is None:
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
398 parts = line.split()
12540
0aa37fd2f33e mx: fix detection for sparc. ignore unparsable lines in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12497
diff changeset
399 if len(parts) == 2:
0aa37fd2f33e mx: fix detection for sparc. ignore unparsable lines in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12497
diff changeset
400 assert parts[1] == 'KNOWN', parts[1]
0aa37fd2f33e mx: fix detection for sparc. ignore unparsable lines in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12497
diff changeset
401 defaultVM = parts[0][1:]
0aa37fd2f33e mx: fix detection for sparc. ignore unparsable lines in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12497
diff changeset
402 jvmCfgLines += ['# default VM is a copy of the unmodified ' + defaultVM + ' VM\n']
0aa37fd2f33e mx: fix detection for sparc. ignore unparsable lines in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12497
diff changeset
403 jvmCfgLines += ['-original KNOWN\n']
0aa37fd2f33e mx: fix detection for sparc. ignore unparsable lines in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12497
diff changeset
404 else:
0aa37fd2f33e mx: fix detection for sparc. ignore unparsable lines in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12497
diff changeset
405 # skip lines which we cannot parse (e.g. '-hotspot ALIASED_TO -client')
12583
110c3faa57e9 mx: print warning if encounter a non-parsable line in jvm.cfg
Bernhard Urban <bernhard.urban@jku.at>
parents: 12540
diff changeset
406 mx.log("WARNING: skipping not parsable line \"" + line + "\"")
8647
5f8299106dd9 A server0 "build" needs to create a jdk copy if it does not exist yet
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8646
diff changeset
407 else:
5f8299106dd9 A server0 "build" needs to create a jdk copy if it does not exist yet
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8646
diff changeset
408 jvmCfgLines += [line]
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
409
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
410 assert defaultVM is not None, 'Could not find default VM in ' + jvmCfg
6687
a9a50f21e000 Added change to copied directory's protection when source JDK directories are read-only.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6566
diff changeset
411 if mx.get_os() != 'windows':
13919
9d70445ea369 mx: set correct permissions for graal.jar
Bernhard Urban <bernhard.urban@jku.at>
parents: 13846
diff changeset
412 chmodRecursive(jdk, JDK_UNIX_PERMISSIONS)
9117
cbfcb1054619 renamed 'boot' VM to 'original' VM
Doug Simon <doug.simon@oracle.com>
parents: 9116
diff changeset
413 shutil.move(join(_vmLibDirInJdk(jdk), defaultVM), join(_vmLibDirInJdk(jdk), 'original'))
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
414
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
415
8261
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
416 with open(jvmCfg, 'w') as fp:
8647
5f8299106dd9 A server0 "build" needs to create a jdk copy if it does not exist yet
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8646
diff changeset
417 for line in jvmCfgLines:
5f8299106dd9 A server0 "build" needs to create a jdk copy if it does not exist yet
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8646
diff changeset
418 fp.write(line)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
419
13523
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
420 # patch 'release' file (append graalvm revision)
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
421 releaseFile = join(jdk, 'release')
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
422 if exists(releaseFile):
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
423 releaseFileLines = []
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
424 with open(releaseFile) as f:
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
425 for line in f:
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
426 releaseFileLines.append(line)
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
427
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
428 with open(releaseFile, 'w') as fp:
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
429 for line in releaseFileLines:
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
430 if line.startswith("SOURCE="):
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
431 try:
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
432 sourceLine = line[0:-2] # remove last char
13523
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
433 hgcfg = mx.HgConfig()
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
434 hgcfg.check()
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
435 revision = hgcfg.tip('.')[:12] # take first 12 chars
13523
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
436 fp.write(sourceLine + ' graal:' + revision + '\"\n')
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
437 except:
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
438 fp.write(line)
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
439 else:
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
440 fp.write(line)
137142009b0b mx: append graal revision to release file of jdk
Bernhard Urban <bernhard.urban@jku.at>
parents: 13522
diff changeset
441
5596
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
442 # Install a copy of the disassembler library
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
443 try:
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
444 hsdis([], copyToDir=_vmLibDirInJdk(jdk))
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
445 except SystemExit:
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
446 pass
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
447 else:
4582
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
448 if not exists(jdk):
11797
65dbed1fdf46 be verbose when the JDK specified by --installed-jdks is missing
Doug Simon <doug.simon@oracle.com>
parents: 11784
diff changeset
449 if _installed_jdks:
65dbed1fdf46 be verbose when the JDK specified by --installed-jdks is missing
Doug Simon <doug.simon@oracle.com>
parents: 11784
diff changeset
450 mx.log("The selected JDK directory does not (yet) exist: " + jdk)
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
451 _handle_missing_VM(build, vmToCheck if vmToCheck else 'graal')
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
452
10574
0cad5096735e commands.py: Make sure _jdk returns an absolute path. Use _jdk and _jdksDir where necessary
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10570
diff changeset
453 if installGraalJar:
0cad5096735e commands.py: Make sure _jdk returns an absolute path. Use _jdk and _jdksDir where necessary
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10570
diff changeset
454 _installGraalJarInJdks(mx.distribution('GRAAL'))
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
455
8261
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
456 if vmToCheck is not None:
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
457 jvmCfg = _vmCfgInJdk(jdk)
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
458 found = False
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
459 with open(jvmCfg) as f:
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
460 for line in f:
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
461 if line.strip() == '-' + vmToCheck + ' KNOWN':
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
462 found = True
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
463 break
2237260c6fdb the contents of jvm.cfg are reset when copying the JDK so that the non-default VMs must be built before being run
Doug Simon <doug.simon@oracle.com>
parents: 8223
diff changeset
464 if not found:
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
465 _handle_missing_VM(build, vmToCheck)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
466
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
467 return jdk
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
468
11936
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
469 def _updateInstalledGraalOptionsFile(jdk):
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
470 graalOptions = join(_graal_home, 'graal.options')
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
471 jreLibDir = join(jdk, 'jre', 'lib')
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
472 if exists(graalOptions):
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
473 shutil.copy(graalOptions, join(jreLibDir, 'graal.options'))
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
474 else:
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
475 toDelete = join(jreLibDir, 'graal.options')
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
476 if exists(toDelete):
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
477 os.unlink(toDelete)
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
478
8134
6e3ebc6fd5a4 graal.jar is installed in all JDKs every time it is updated (GRAAL-136)
Doug Simon <doug.simon@oracle.com>
parents: 8132
diff changeset
479 def _installGraalJarInJdks(graalDist):
6e3ebc6fd5a4 graal.jar is installed in all JDKs every time it is updated (GRAAL-136)
Doug Simon <doug.simon@oracle.com>
parents: 8132
diff changeset
480 graalJar = graalDist.path
10574
0cad5096735e commands.py: Make sure _jdk returns an absolute path. Use _jdk and _jdksDir where necessary
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10570
diff changeset
481 jdks = _jdksDir()
14930
5c7718ae8c86 install graal.jar into local Maven repo if MAVEN_INSTALL_GRAAL_JAR environment variable is "true"
Doug Simon <doug.simon@oracle.com>
parents: 14918
diff changeset
482
8134
6e3ebc6fd5a4 graal.jar is installed in all JDKs every time it is updated (GRAAL-136)
Doug Simon <doug.simon@oracle.com>
parents: 8132
diff changeset
483 if exists(jdks):
6e3ebc6fd5a4 graal.jar is installed in all JDKs every time it is updated (GRAAL-136)
Doug Simon <doug.simon@oracle.com>
parents: 8132
diff changeset
484 for e in os.listdir(jdks):
6e3ebc6fd5a4 graal.jar is installed in all JDKs every time it is updated (GRAAL-136)
Doug Simon <doug.simon@oracle.com>
parents: 8132
diff changeset
485 jreLibDir = join(jdks, e, 'jre', 'lib')
6e3ebc6fd5a4 graal.jar is installed in all JDKs every time it is updated (GRAAL-136)
Doug Simon <doug.simon@oracle.com>
parents: 8132
diff changeset
486 if exists(jreLibDir):
14938
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
487 def install(srcJar, dstDir):
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
488 # do a copy and then a move to get atomic updating (on Unix)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
489 name = os.path.basename(srcJar)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
490 fd, tmp = tempfile.mkstemp(suffix='', prefix=name, dir=dstDir)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
491 shutil.copyfile(srcJar, tmp)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
492 os.close(fd)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
493 dstJar = join(dstDir, name)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
494 shutil.move(tmp, dstJar)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
495 os.chmod(dstJar, JDK_UNIX_PERMISSIONS)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
496
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
497 install(graalJar, jreLibDir)
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
498 if graalDist.sourcesPath:
9d32e87ea98b GRAAL distribution now includes sources (graal.src.zip)
Doug Simon <doug.simon@oracle.com>
parents: 14930
diff changeset
499 install(graalDist.sourcesPath, join(jdks, e))
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
500
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
501 # run a command in the windows SDK Debug Shell
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
502 def _runInDebugShell(cmd, workingDir, logFile=None, findInOutput=None, respondTo=None):
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
503 if respondTo is None:
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
504 respondTo = {}
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
505 newLine = os.linesep
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
506 startToken = 'RUNINDEBUGSHELL_STARTSEQUENCE'
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
507 endToken = 'RUNINDEBUGSHELL_ENDSEQUENCE'
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
508
4252
67e88b7624d5 Removed need for GRAAL environment variable on Windows.
Doug Simon <doug.simon@oracle.com>
parents: 4247
diff changeset
509 winSDK = mx.get_env('WIN_SDK', 'C:\\Program Files\\Microsoft SDKs\\Windows\\v7.1\\')
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
510
6306
2a819543cc45 Warnings for windows build if there is no SDK
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6294
diff changeset
511 if not exists(winSDK):
2a819543cc45 Warnings for windows build if there is no SDK
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6294
diff changeset
512 mx.abort("Could not find Windows SDK : '" + winSDK + "' does not exist")
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
513
6306
2a819543cc45 Warnings for windows build if there is no SDK
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6294
diff changeset
514 if not exists(join(winSDK, 'Bin', 'SetEnv.cmd')):
2a819543cc45 Warnings for windows build if there is no SDK
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6294
diff changeset
515 mx.abort("Invalid Windows SDK path (" + winSDK + ") : could not find Bin/SetEnv.cmd (you can use the WIN_SDK environment variable to specify an other path)")
4252
67e88b7624d5 Removed need for GRAAL environment variable on Windows.
Doug Simon <doug.simon@oracle.com>
parents: 4247
diff changeset
516
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
517 p = subprocess.Popen('cmd.exe /E:ON /V:ON /K ""' + winSDK + '/Bin/SetEnv.cmd" & echo ' + startToken + '"', \
4252
67e88b7624d5 Removed need for GRAAL environment variable on Windows.
Doug Simon <doug.simon@oracle.com>
parents: 4247
diff changeset
518 shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
4269
ffd5ce8fc736 Moved IDE project configuration into mx.py.
Doug Simon <doug.simon@oracle.com>
parents: 4257
diff changeset
519 stdout = p.stdout
ffd5ce8fc736 Moved IDE project configuration into mx.py.
Doug Simon <doug.simon@oracle.com>
parents: 4257
diff changeset
520 stdin = p.stdin
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
521 if logFile:
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
522 log = open(logFile, 'w')
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
523 ret = False
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
524 while True:
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
525
7291
a748e4d44694 Truffle API to specify type-specalized Node classes; annotation processor for automatic code generation of the type-specialized Node classes during the build process
Christian Humer <christian.humer@gmail.com>
parents: 7290
diff changeset
526 # encoding may be None on windows plattforms
a748e4d44694 Truffle API to specify type-specalized Node classes; annotation processor for automatic code generation of the type-specialized Node classes during the build process
Christian Humer <christian.humer@gmail.com>
parents: 7290
diff changeset
527 if sys.stdout.encoding is None:
a748e4d44694 Truffle API to specify type-specalized Node classes; annotation processor for automatic code generation of the type-specialized Node classes during the build process
Christian Humer <christian.humer@gmail.com>
parents: 7290
diff changeset
528 encoding = 'utf-8'
a748e4d44694 Truffle API to specify type-specalized Node classes; annotation processor for automatic code generation of the type-specialized Node classes during the build process
Christian Humer <christian.humer@gmail.com>
parents: 7290
diff changeset
529 else:
a748e4d44694 Truffle API to specify type-specalized Node classes; annotation processor for automatic code generation of the type-specialized Node classes during the build process
Christian Humer <christian.humer@gmail.com>
parents: 7290
diff changeset
530 encoding = sys.stdout.encoding
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
531
7291
a748e4d44694 Truffle API to specify type-specalized Node classes; annotation processor for automatic code generation of the type-specialized Node classes during the build process
Christian Humer <christian.humer@gmail.com>
parents: 7290
diff changeset
532 line = stdout.readline().decode(encoding)
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
533 if logFile:
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
534 log.write(line.encode('utf-8'))
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
535 line = line.strip()
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
536 mx.log(line)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
537 if line == startToken:
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
538 stdin.write('cd /D ' + workingDir + ' & ' + cmd + ' & echo ' + endToken + newLine)
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
539 for regex in respondTo.keys():
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
540 match = regex.search(line)
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
541 if match:
4269
ffd5ce8fc736 Moved IDE project configuration into mx.py.
Doug Simon <doug.simon@oracle.com>
parents: 4257
diff changeset
542 stdin.write(respondTo[regex] + newLine)
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
543 if findInOutput:
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
544 match = findInOutput.search(line)
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
545 if match:
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
546 ret = True
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
547 if line == endToken:
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
548 if not findInOutput:
5868
a10e72af4dc5 Make windows build work even if Visual Studio is not installed (only the Windows SDK is required)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5782
diff changeset
549 stdin.write('echo ERRXXX%errorlevel%' + newLine)
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
550 else:
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
551 break
5868
a10e72af4dc5 Make windows build work even if Visual Studio is not installed (only the Windows SDK is required)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5782
diff changeset
552 if line.startswith('ERRXXX'):
a10e72af4dc5 Make windows build work even if Visual Studio is not installed (only the Windows SDK is required)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5782
diff changeset
553 if line == 'ERRXXX0':
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
554 ret = True
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
555 break
4269
ffd5ce8fc736 Moved IDE project configuration into mx.py.
Doug Simon <doug.simon@oracle.com>
parents: 4257
diff changeset
556 stdin.write('exit' + newLine)
4177
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
557 if logFile:
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
558 log.close()
c843578c269d Make building work on Windows.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4163
diff changeset
559 return ret
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
560
11605
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
561 def jdkhome(vm=None):
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
562 """return the JDK directory selected for the 'vm' command"""
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
563 build = _vmbuild if _vmSourcesAvailable else 'product'
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
564 return _jdk(build, installGraalJar=False)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
565
11605
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
566 def print_jdkhome(args, vm=None):
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
567 """print the JDK directory selected for the 'vm' command"""
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
568 print jdkhome(vm)
5103
276e14614531 Added jdkhome command to mx.
Doug Simon <doug.simon@oracle.com>
parents: 5094
diff changeset
569
9116
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
570 def buildvars(args):
11377
0fbc1e418c88 fixed capitalization and tense in documentation of some mx commands
Doug Simon <doug.simon@oracle.com>
parents: 11373
diff changeset
571 """describe the variables that can be set by the -D option to the 'mx build' commmand"""
9116
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
572
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
573 buildVars = {
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
574 'ALT_BOOTDIR' : 'The location of the bootstrap JDK installation (default: ' + mx.java().jdk + ')',
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
575 'ALT_OUTPUTDIR' : 'Build directory',
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
576 'HOTSPOT_BUILD_JOBS' : 'Number of CPUs used by make (default: ' + str(multiprocessing.cpu_count()) + ')',
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
577 'INSTALL' : 'Install the built VM into the JDK? (default: y)',
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
578 'ZIP_DEBUGINFO_FILES' : 'Install zipped debug symbols file? (default: 0)',
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
579 }
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
580
9116
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
581 mx.log('HotSpot build variables that can be set by the -D option to "mx build":')
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
582 mx.log('')
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
583 for n in sorted(buildVars.iterkeys()):
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
584 mx.log(n)
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
585 mx.log(textwrap.fill(buildVars[n], initial_indent=' ', subsequent_indent=' ', width=200))
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
586
9120
9f361c0f912b added note about mx/env to the output of 'mx buildvars'
Doug Simon <doug.simon@oracle.com>
parents: 9119
diff changeset
587 mx.log('')
9f361c0f912b added note about mx/env to the output of 'mx buildvars'
Doug Simon <doug.simon@oracle.com>
parents: 9119
diff changeset
588 mx.log('Note that these variables can be given persistent values in the file ' + join(_graal_home, 'mx', 'env') + ' (see \'mx about\').')
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
589
4577
bc8b58c11768 Added debug build of HotSpot Client to the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4575
diff changeset
590 def build(args, vm=None):
4575
ccb5369481a2 Clarified mx help text on how to build/run [Graal|Client|Server].
Doug Simon <doug.simon@oracle.com>
parents: 4572
diff changeset
591 """build the VM binary
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
592
11392
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
593 The global '--vm' and '--vmbuild' options select which VM type and build target to build."""
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
594
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
595 # Override to fail quickly if extra arguments are given
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
596 # at the end of the command line. This allows for a more
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
597 # helpful error message.
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
598 class AP(ArgumentParser):
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
599 def __init__(self):
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
600 ArgumentParser.__init__(self, prog='mx build')
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
601 def parse_args(self, args):
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
602 result = ArgumentParser.parse_args(self, args)
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
603 if len(result.remainder) != 0:
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
604 firstBuildTarget = result.remainder[0]
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
605 mx.abort('To specify the ' + firstBuildTarget + ' VM build target, you need to use the global "--vmbuild" option. For example:\n' +
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
606 ' mx --vmbuild ' + firstBuildTarget + ' build')
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
607 return result
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
608
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
609 # Call mx.build to compile the Java sources
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
610 parser = AP()
10083
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
611 parser.add_argument('--export-dir', help='directory to which graal.jar and graal.options will be copied', metavar='<path>')
9116
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
612 parser.add_argument('-D', action='append', help='set a HotSpot build variable (run \'mx buildvars\' to list variables)', metavar='name=value')
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
613 opts2 = mx.build(['--source', '1.7'] + args, parser=parser)
11392
66251d9f62ab fail faster with better error message if deprecated and unsupported mx build command line is given
Doug Simon <doug.simon@oracle.com>
parents: 11377
diff changeset
614 assert len(opts2.remainder) == 0
4145
9aee7df31417 Removed 'tests' command.
Doug Simon <doug.simon@oracle.com>
parents: 4144
diff changeset
615
10083
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
616 if opts2.export_dir is not None:
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
617 if not exists(opts2.export_dir):
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
618 os.makedirs(opts2.export_dir)
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
619 else:
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
620 assert os.path.isdir(opts2.export_dir), '{} is not a directory'.format(opts2.export_dir)
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
621
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
622 shutil.copy(mx.distribution('GRAAL').path, opts2.export_dir)
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
623 graalOptions = join(_graal_home, 'graal.options')
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
624 if exists(graalOptions):
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
625 shutil.copy(graalOptions, opts2.export_dir)
8dc4cdde75fb remove build-graal.xml and have make directly call mx to generate graal.jar
Doug Simon <doug.simon@oracle.com>
parents: 10067
diff changeset
626
4559
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
627 if not _vmSourcesAvailable or not opts2.native:
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
628 return
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
629
11371
74414b248381 mx: added --vmbuild global option, removed --product, --fastdebug, --debug options as well as suffix of build command
Doug Simon <doug.simon@oracle.com>
parents: 11368
diff changeset
630 builds = [_vmbuild]
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
631
4577
bc8b58c11768 Added debug build of HotSpot Client to the gate.
Doug Simon <doug.simon@oracle.com>
parents: 4575
diff changeset
632 if vm is None:
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
633 vm = _get_vm()
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
634
9117
cbfcb1054619 renamed 'boot' VM to 'original' VM
Doug Simon <doug.simon@oracle.com>
parents: 9116
diff changeset
635 if vm == 'original':
cbfcb1054619 renamed 'boot' VM to 'original' VM
Doug Simon <doug.simon@oracle.com>
parents: 9116
diff changeset
636 pass
cbfcb1054619 renamed 'boot' VM to 'original' VM
Doug Simon <doug.simon@oracle.com>
parents: 9116
diff changeset
637 elif vm.startswith('server'):
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
638 buildSuffix = ''
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
639 elif vm.startswith('client'):
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
640 buildSuffix = '1'
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
641 else:
4559
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
642 assert vm == 'graal', vm
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
643 buildSuffix = 'graal'
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
644
11530
be9e54fbb699 Use a better warning message for building with --installed-jdks set
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11526
diff changeset
645 if _installed_jdks and _installed_jdks != _graal_home:
be9e54fbb699 Use a better warning message for building with --installed-jdks set
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11526
diff changeset
646 if not mx.ask_yes_no("Warning: building while --installed-jdks is set (" + _installed_jdks + ") is not recommanded - are you sure you want to continue", 'n'):
11512
38acec26d535 If _installed_jdk is set, ask confirmation before building
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11511
diff changeset
647 mx.abort(1)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
648
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
649 for build in builds:
5094
d3f547b08304 added mechanism for disabling automatic native builds from the IDE (set IDE_BUILD_TARGET="" in mx/env)
Doug Simon <doug.simon@oracle.com>
parents: 5090
diff changeset
650 if build == 'ide-build-target':
11302
fcb4cf14a3c3 absence of IDE_BUILD_TARGET environment variable prevents HotSpot builds when editing C++ sources in Eclipse
Doug Simon <doug.simon@oracle.com>
parents: 11296
diff changeset
651 build = os.environ.get('IDE_BUILD_TARGET', None)
fcb4cf14a3c3 absence of IDE_BUILD_TARGET environment variable prevents HotSpot builds when editing C++ sources in Eclipse
Doug Simon <doug.simon@oracle.com>
parents: 11296
diff changeset
652 if build is None or len(build) == 0:
5094
d3f547b08304 added mechanism for disabling automatic native builds from the IDE (set IDE_BUILD_TARGET="" in mx/env)
Doug Simon <doug.simon@oracle.com>
parents: 5090
diff changeset
653 continue
d3f547b08304 added mechanism for disabling automatic native builds from the IDE (set IDE_BUILD_TARGET="" in mx/env)
Doug Simon <doug.simon@oracle.com>
parents: 5090
diff changeset
654
9098
d766fd8eede0 VM copy made during initial copy of the JDK is now named 'boot' instead of 'server0' to better reflect that it is the default VM (which may not be 'server') from the boot JDK
Doug Simon <doug.simon@oracle.com>
parents: 9097
diff changeset
655 jdk = _jdk(build, create=True)
8647
5f8299106dd9 A server0 "build" needs to create a jdk copy if it does not exist yet
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8646
diff changeset
656
9117
cbfcb1054619 renamed 'boot' VM to 'original' VM
Doug Simon <doug.simon@oracle.com>
parents: 9116
diff changeset
657 if vm == 'original':
9098
d766fd8eede0 VM copy made during initial copy of the JDK is now named 'boot' instead of 'server0' to better reflect that it is the default VM (which may not be 'server') from the boot JDK
Doug Simon <doug.simon@oracle.com>
parents: 9097
diff changeset
658 if build != 'product':
9117
cbfcb1054619 renamed 'boot' VM to 'original' VM
Doug Simon <doug.simon@oracle.com>
parents: 9116
diff changeset
659 mx.log('only product build of original VM exists')
9098
d766fd8eede0 VM copy made during initial copy of the JDK is now named 'boot' instead of 'server0' to better reflect that it is the default VM (which may not be 'server') from the boot JDK
Doug Simon <doug.simon@oracle.com>
parents: 9097
diff changeset
660 continue
14605
5953ac9e0d93 fixed pylint errors
Doug Simon <doug.simon@oracle.com>
parents: 14602
diff changeset
661
14602
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
662 if not isVMSupported(vm):
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
663 mx.log('The ' + vm + ' VM is not supported on this platform - skipping')
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
664 continue
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
665
4607
ee87cfe2f8f9 Made GraalVM build work on Mac OS X.
Doug Simon <doug.simon@oracle.com>
parents: 4602
diff changeset
666 vmDir = join(_vmLibDirInJdk(jdk), vm)
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
667 if not exists(vmDir):
6687
a9a50f21e000 Added change to copied directory's protection when source JDK directories are read-only.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6566
diff changeset
668 if mx.get_os() != 'windows':
13919
9d70445ea369 mx: set correct permissions for graal.jar
Bernhard Urban <bernhard.urban@jku.at>
parents: 13846
diff changeset
669 chmodRecursive(jdk, JDK_UNIX_PERMISSIONS)
14780
aaecb0ca0c7d fixed hard coded reference to JDK7
Doug Simon <doug.simon@oracle.com>
parents: 14777
diff changeset
670 mx.log('Creating VM directory in JDK: ' + vmDir)
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
671 os.makedirs(vmDir)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
672
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
673 def filterXusage(line):
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
674 if not 'Xusage.txt' in line:
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
675 sys.stderr.write(line + os.linesep)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
676
5239
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
677 # Check if a build really needs to be done
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
678 timestampFile = join(vmDir, '.build-timestamp')
9098
d766fd8eede0 VM copy made during initial copy of the JDK is now named 'boot' instead of 'server0' to better reflect that it is the default VM (which may not be 'server') from the boot JDK
Doug Simon <doug.simon@oracle.com>
parents: 9097
diff changeset
679 if opts2.force or not exists(timestampFile):
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
680 mustBuild = True
5239
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
681 else:
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
682 mustBuild = False
5239
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
683 timestamp = os.path.getmtime(timestampFile)
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
684 sources = []
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
685 for d in ['src', 'make']:
5244
55ff4ba8d7b1 ignore src/share/tools directory in native build out-of-date check
Doug Simon <doug.simon@oracle.com>
parents: 5239
diff changeset
686 for root, dirnames, files in os.walk(join(_graal_home, d)):
5239
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
687 # ignore <graal>/src/share/tools
5244
55ff4ba8d7b1 ignore src/share/tools directory in native build out-of-date check
Doug Simon <doug.simon@oracle.com>
parents: 5239
diff changeset
688 if root == join(_graal_home, 'src', 'share'):
55ff4ba8d7b1 ignore src/share/tools directory in native build out-of-date check
Doug Simon <doug.simon@oracle.com>
parents: 5239
diff changeset
689 dirnames.remove('tools')
55ff4ba8d7b1 ignore src/share/tools directory in native build out-of-date check
Doug Simon <doug.simon@oracle.com>
parents: 5239
diff changeset
690 sources += [join(root, name) for name in files]
5239
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
691 for f in sources:
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
692 if len(f) != 0 and os.path.getmtime(f) > timestamp:
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
693 mustBuild = True
5244
55ff4ba8d7b1 ignore src/share/tools directory in native build out-of-date check
Doug Simon <doug.simon@oracle.com>
parents: 5239
diff changeset
694 break
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
695
5239
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
696 if not mustBuild:
10568
070b4a3c56f3 disabled "noisy" log statements unless -v option (i.e. verbose) is specified to mx
Doug Simon <doug.simon@oracle.com>
parents: 10527
diff changeset
697 mx.logv('[all files in src and make directories are older than ' + timestampFile[len(_graal_home) + 1:] + ' - skipping native build]')
5239
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
698 continue
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
699
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
700 if platform.system() == 'Windows':
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
701 compilelogfile = _graal_home + '/graalCompile.log'
4252
67e88b7624d5 Removed need for GRAAL environment variable on Windows.
Doug Simon <doug.simon@oracle.com>
parents: 4247
diff changeset
702 mksHome = mx.get_env('MKS_HOME', 'C:\\cygwin\\bin')
67e88b7624d5 Removed need for GRAAL environment variable on Windows.
Doug Simon <doug.simon@oracle.com>
parents: 4247
diff changeset
703
4572
76841bdd5f3e (Windows) Add Graal build configurations (with GRAAL defined) and fix build command.
Andreas Woess <andreas.woess@jku.at>
parents: 4559
diff changeset
704 variant = {'client': 'compiler1', 'server': 'compiler2'}.get(vm, vm)
76841bdd5f3e (Windows) Add Graal build configurations (with GRAAL defined) and fix build command.
Andreas Woess <andreas.woess@jku.at>
parents: 4559
diff changeset
705 project_config = variant + '_' + build
76841bdd5f3e (Windows) Add Graal build configurations (with GRAAL defined) and fix build command.
Andreas Woess <andreas.woess@jku.at>
parents: 4559
diff changeset
706 _runInDebugShell('msbuild ' + _graal_home + r'\build\vs-amd64\jvm.vcproj /p:Configuration=' + project_config + ' /target:clean', _graal_home)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
707 winCompileCmd = r'set HotSpotMksHome=' + mksHome + r'& set OUT_DIR=' + jdk + r'& set JAVA_HOME=' + jdk + r'& set path=%JAVA_HOME%\bin;%path%;%HotSpotMksHome%& cd /D "' + _graal_home + r'\make\windows"& call create.bat ' + _graal_home
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
708 print winCompileCmd
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
709 winCompileSuccess = re.compile(r"^Writing \.vcxproj file:")
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
710 if not _runInDebugShell(winCompileCmd, _graal_home, compilelogfile, winCompileSuccess):
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
711 mx.log('Error executing create command')
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
712 return
4572
76841bdd5f3e (Windows) Add Graal build configurations (with GRAAL defined) and fix build command.
Andreas Woess <andreas.woess@jku.at>
parents: 4559
diff changeset
713 winBuildCmd = 'msbuild ' + _graal_home + r'\build\vs-amd64\jvm.vcxproj /p:Configuration=' + project_config + ' /p:Platform=x64'
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
714 if not _runInDebugShell(winBuildCmd, _graal_home, compilelogfile):
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
715 mx.log('Error building project')
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
716 return
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
717 else:
4582
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
718 cpus = multiprocessing.cpu_count()
6718
cce59a7ee92c Avoids stripping libjvm.so when building debug on Solaris (.debuginfo files don't work well with Oracle Solaris Studio 12.3's solaris).
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6705
diff changeset
719 runCmd = [mx.gmake_cmd()]
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
720 runCmd.append(build + buildSuffix)
8884
51a8368d0231 Adapt to changes in hotspot makefiles : pass our JAVA_HOME as ALT_BOOTDIR (so that things are built with a 'safe' jdk) and our jdk copy as JAVA_HOME (so that the jvm gets installed there)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8647
diff changeset
721 env = os.environ.copy()
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
722
9116
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
723 if opts2.D:
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
724 for nv in opts2.D:
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
725 name, value = nv.split('=', 1)
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
726 env[name.strip()] = value
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
727
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
728 env.setdefault('ARCH_DATA_MODEL', '64')
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
729 env.setdefault('LANG', 'C')
4582
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
730 env.setdefault('HOTSPOT_BUILD_JOBS', str(cpus))
9116
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
731 env.setdefault('ALT_BOOTDIR', mx.java().jdk)
13846
1e72cd05b77e hs build: add graal tag to hotspot build version
Bernhard Urban <bernhard.urban@jku.at>
parents: 13761
diff changeset
732
9161
8d4174140745 Make the native build less verbose unless mx -v is used
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9159
diff changeset
733 if not mx._opts.verbose:
8d4174140745 Make the native build less verbose unless mx -v is used
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9159
diff changeset
734 runCmd.append('MAKE_VERBOSE=')
8884
51a8368d0231 Adapt to changes in hotspot makefiles : pass our JAVA_HOME as ALT_BOOTDIR (so that things are built with a 'safe' jdk) and our jdk copy as JAVA_HOME (so that the jvm gets installed there)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8647
diff changeset
735 env['JAVA_HOME'] = jdk
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
736 if vm.endswith('nograal'):
9166
0f3fe35513a0 renamed build variable used to make a Graal/non-Graal build from OMIT_GRAAL to INCLUDE_GRAAL and ensured it is only used internally
Doug Simon <doug.simon@oracle.com>
parents: 9163
diff changeset
737 env['INCLUDE_GRAAL'] = 'false'
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
738 env.setdefault('ALT_OUTPUTDIR', join(_graal_home, 'build-nograal', mx.get_os()))
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
739 else:
15000
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
740 # extract latest release tag for graal
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
741 try:
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
742 tags = [x.split(' ')[0] for x in subprocess.check_output(['hg', '-R', _graal_home, 'tags']).split('\n') if x.startswith("graal-")]
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
743 except:
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
744 # not a mercurial repository or hg commands are not available.
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
745 tags = None
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
746
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
747 if tags:
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
748 # extract the most recent tag
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
749 tag = sorted(tags, key=lambda e: [int(x) for x in e[len("graal-"):].split('.')], reverse=True)[0]
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
750 env.setdefault('USER_RELEASE_SUFFIX', tag)
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
751 env.setdefault('GRAAL_VERSION', tag[len("graal-"):])
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
752 else:
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
753 version = 'unknown-{}-{}'.format(platform.node(), time.strftime('%Y-%m-%d_%H-%M-%S_%Z'))
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
754 env.setdefault('USER_RELEASE_SUFFIX', 'graal-' + version)
cc844a4512f1 added graal.version system property that is (only) defined by Graal enabled VMs
Doug Simon <doug.simon@oracle.com>
parents: 14965
diff changeset
755 env.setdefault('GRAAL_VERSION', version)
9166
0f3fe35513a0 renamed build variable used to make a Graal/non-Graal build from OMIT_GRAAL to INCLUDE_GRAAL and ensured it is only used internally
Doug Simon <doug.simon@oracle.com>
parents: 9163
diff changeset
756 env['INCLUDE_GRAAL'] = 'true'
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
757 env.setdefault('INSTALL', 'y')
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
758 if mx.get_os() == 'solaris':
6702
602ee4d1db61 added setup of ENFORCE_COMPILER_REV and ENFORCE_CC_COMPILER_REV when using sunstudio compilers on Solaris to avoid build complaint from HotSpot makefiles.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6701
diff changeset
759 # If using sparcWorks, setup flags to avoid make complaining about CC version
602ee4d1db61 added setup of ENFORCE_COMPILER_REV and ENFORCE_CC_COMPILER_REV when using sunstudio compilers on Solaris to avoid build complaint from HotSpot makefiles.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6701
diff changeset
760 cCompilerVersion = subprocess.Popen('CC -V', stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).stderr.readlines()[0]
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
761 if cCompilerVersion.startswith('CC: Sun C++'):
6702
602ee4d1db61 added setup of ENFORCE_COMPILER_REV and ENFORCE_CC_COMPILER_REV when using sunstudio compilers on Solaris to avoid build complaint from HotSpot makefiles.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6701
diff changeset
762 compilerRev = cCompilerVersion.split(' ')[3]
6718
cce59a7ee92c Avoids stripping libjvm.so when building debug on Solaris (.debuginfo files don't work well with Oracle Solaris Studio 12.3's solaris).
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6705
diff changeset
763 env.setdefault('ENFORCE_COMPILER_REV', compilerRev)
cce59a7ee92c Avoids stripping libjvm.so when building debug on Solaris (.debuginfo files don't work well with Oracle Solaris Studio 12.3's solaris).
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6705
diff changeset
764 env.setdefault('ENFORCE_CC_COMPILER_REV', compilerRev)
cce59a7ee92c Avoids stripping libjvm.so when building debug on Solaris (.debuginfo files don't work well with Oracle Solaris Studio 12.3's solaris).
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6705
diff changeset
765 if build == 'jvmg':
cce59a7ee92c Avoids stripping libjvm.so when building debug on Solaris (.debuginfo files don't work well with Oracle Solaris Studio 12.3's solaris).
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6705
diff changeset
766 # I want ALL the symbols when I'm debugging on Solaris
cce59a7ee92c Avoids stripping libjvm.so when building debug on Solaris (.debuginfo files don't work well with Oracle Solaris Studio 12.3's solaris).
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6705
diff changeset
767 # Some Makefile variable are overloaded by environment variable so we need to explicitely
cce59a7ee92c Avoids stripping libjvm.so when building debug on Solaris (.debuginfo files don't work well with Oracle Solaris Studio 12.3's solaris).
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6705
diff changeset
768 # pass them down in the command line. This one is an example of that.
cce59a7ee92c Avoids stripping libjvm.so when building debug on Solaris (.debuginfo files don't work well with Oracle Solaris Studio 12.3's solaris).
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6705
diff changeset
769 runCmd.append('STRIP_POLICY=no_strip')
6276
03c4c25ae726 disabled zipping of lib*.debuginfo files to simplify debugging with gdb
Doug Simon <doug.simon@oracle.com>
parents: 5881
diff changeset
770 # This removes the need to unzip the *.diz files before debugging in gdb
03c4c25ae726 disabled zipping of lib*.debuginfo files to simplify debugging with gdb
Doug Simon <doug.simon@oracle.com>
parents: 5881
diff changeset
771 env.setdefault('ZIP_DEBUGINFO_FILES', '0')
03c4c25ae726 disabled zipping of lib*.debuginfo files to simplify debugging with gdb
Doug Simon <doug.simon@oracle.com>
parents: 5881
diff changeset
772
4582
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
773 # Clear these 2 variables as having them set can cause very confusing build problems
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
774 env.pop('LD_LIBRARY_PATH', None)
b24386206122 Made all vm builds go into subdirectories, even product builds to simplify building the various types of VMs (server, client and graal).
Doug Simon <doug.simon@oracle.com>
parents: 4577
diff changeset
775 env.pop('CLASSPATH', None)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
776
8884
51a8368d0231 Adapt to changes in hotspot makefiles : pass our JAVA_HOME as ALT_BOOTDIR (so that things are built with a 'safe' jdk) and our jdk copy as JAVA_HOME (so that the jvm gets installed there)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8647
diff changeset
777 mx.run(runCmd, cwd=join(_graal_home, 'make'), err=filterXusage, env=env)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
778
5680
87fc13b59258 Split _vmLibDirInJdk into _vmLibDirInJdk and _vmCfgInJdk to account for Windows jdk layout
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5649
diff changeset
779 jvmCfg = _vmCfgInJdk(jdk)
4559
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
780 if not exists(jvmCfg):
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
781 mx.abort(jvmCfg + ' does not exist')
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
782
8901
8ddaac81cb21 Be more careful while updating jvm.cfg (fixes a bug where the -server configuration would overwrite the -server0 one)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8900
diff changeset
783 prefix = '-' + vm + ' '
8ddaac81cb21 Be more careful while updating jvm.cfg (fixes a bug where the -server configuration would overwrite the -server0 one)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8900
diff changeset
784 vmKnown = prefix + 'KNOWN\n'
8889
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
785 lines = []
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
786 found = False
4559
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
787 with open(jvmCfg) as f:
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
788 for line in f:
8889
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
789 if line.strip() == vmKnown.strip():
4559
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
790 found = True
8889
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
791 lines.append(line)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
792
4559
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
793 if not found:
8901
8ddaac81cb21 Be more careful while updating jvm.cfg (fixes a bug where the -server configuration would overwrite the -server0 one)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8900
diff changeset
794 mx.log('Appending "' + prefix + 'KNOWN" to ' + jvmCfg)
6688
806821a1c0a8 Make sure files in copy of JDK has write-protection unset before modifying it.
Laurent Daynes <Laurent.Daynes@oracle.com>
parents: 6687
diff changeset
795 if mx.get_os() != 'windows':
13919
9d70445ea369 mx: set correct permissions for graal.jar
Bernhard Urban <bernhard.urban@jku.at>
parents: 13846
diff changeset
796 os.chmod(jvmCfg, JDK_UNIX_PERMISSIONS)
8889
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
797 with open(jvmCfg, 'w') as f:
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
798 for line in lines:
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
799 if line.startswith(prefix):
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
800 line = vmKnown
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
801 found = True
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
802 f.write(line)
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
803 if not found:
c669e8e621a1 Update the -client line in jvm.cfg when it already exists (ie IGNORE)
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8887
diff changeset
804 f.write(vmKnown)
5194
a6eceb5efb0e added --ecl option to mx for saving VM execution(s) as Eclipse launch configurations
Doug Simon <doug.simon@oracle.com>
parents: 5189
diff changeset
805
5239
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
806 if exists(timestampFile):
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
807 os.utime(timestampFile, None)
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
808 else:
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
809 file(timestampFile, 'a')
b9db4fee6eb2 skip a native build if all files in src and make are older than the timestamp of the previous build
Doug Simon <doug.simon@oracle.com>
parents: 5234
diff changeset
810
5257
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
811 def vmg(args):
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
812 """run the debug build of VM selected by the '--vm' option"""
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
813 return vm(args, vmbuild='debug')
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
814
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
815 def vmfg(args):
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
816 """run the fastdebug build of VM selected by the '--vm' option"""
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
817 return vm(args, vmbuild='fastdebug')
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
818
13951
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
819 def _parseVmArgs(args, vm=None, cwd=None, vmbuild=None):
4575
ccb5369481a2 Clarified mx help text on how to build/run [Graal|Client|Server].
Doug Simon <doug.simon@oracle.com>
parents: 4572
diff changeset
820 """run the VM selected by the '--vm' option"""
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4216
diff changeset
821
4417
648a7873cea2 Made it possible to build and run the client and server VM with a new '--vm' option to 'mx build' and a global option of the same name.
Doug Simon <doug.simon@oracle.com>
parents: 4407
diff changeset
822 if vm is None:
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
823 vm = _get_vm()
14164
10e07aaeb54f fixed pylint warning
Doug Simon <doug.simon@oracle.com>
parents: 14161
diff changeset
824
14602
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
825 if not isVMSupported(vm):
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
826 mx.abort('The ' + vm + ' is not supported on this platform')
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
827
10454
590b0c26877f mx: add --workdir argument
Bernhard Urban <bernhard.urban@jku.at>
parents: 10417
diff changeset
828 if cwd is None:
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
829 cwd = _vm_cwd
11424
ece40b452e65 mx: Fix commands.vm when an explicit cwd is passed and no --vmcwd is given
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11415
diff changeset
830 elif _vm_cwd is not None and _vm_cwd != cwd:
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
831 mx.abort("conflicting working directories: do not set --vmcwd for this command")
10454
590b0c26877f mx: add --workdir argument
Bernhard Urban <bernhard.urban@jku.at>
parents: 10417
diff changeset
832
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4216
diff changeset
833 build = vmbuild if vmbuild is not None else _vmbuild if _vmSourcesAvailable else 'product'
11286
485a5f029b32 mx: graal.jar is not silently updated anymore in the vm function
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11237
diff changeset
834 jdk = _jdk(build, vmToCheck=vm, installGraalJar=False)
11936
4cdf7623fbe6 ensure graal.options in the JDK is deployed/updated/deleted each time the VM is run
Doug Simon <doug.simon@oracle.com>
parents: 11867
diff changeset
835 _updateInstalledGraalOptionsFile(jdk)
5189
6fbf12b8e572 automated download of hsdis lib if PrintAssembly option is used; added support for hsdis on the Mac
Doug Simon <doug.simon@oracle.com>
parents: 5153
diff changeset
836 mx.expand_project_in_args(args)
5194
a6eceb5efb0e added --ecl option to mx for saving VM execution(s) as Eclipse launch configurations
Doug Simon <doug.simon@oracle.com>
parents: 5189
diff changeset
837 if _make_eclipse_launch:
a6eceb5efb0e added --ecl option to mx for saving VM execution(s) as Eclipse launch configurations
Doug Simon <doug.simon@oracle.com>
parents: 5189
diff changeset
838 mx.make_eclipse_launch(args, 'graal-' + build, name=None, deps=mx.project('com.oracle.graal.hotspot').all_deps([], True))
5024
7ccdae96e98a Allow JaCoCon to append to the execution file
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5016
diff changeset
839 if _jacoco == 'on' or _jacoco == 'append':
4684
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
840 jacocoagent = mx.library("JACOCOAGENT", True)
5366
67e63e8dcbd2 build JaCoCo exclude list by scanning for snippets
Doug Simon <doug.simon@oracle.com>
parents: 5295
diff changeset
841 # Exclude all compiler tests and snippets
5881
472a648769e4 better JaCoCo : only include com.oracle.graal.* and com.oracle.max.*, exclude com.oracle.graal.jtt.*
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5878
diff changeset
842 excludes = ['com.oracle.graal.compiler.tests.*', 'com.oracle.graal.jtt.*']
5366
67e63e8dcbd2 build JaCoCo exclude list by scanning for snippets
Doug Simon <doug.simon@oracle.com>
parents: 5295
diff changeset
843 for p in mx.projects():
8607
c7672a325faf search for classes containing annotations returns source file as well as class name
Doug Simon <doug.simon@oracle.com>
parents: 8554
diff changeset
844 excludes += _find_classes_with_annotations(p, None, ['@Snippet', '@ClassSubstitution', '@Test'], includeInnerClasses=True).keys()
c7672a325faf search for classes containing annotations returns source file as well as class name
Doug Simon <doug.simon@oracle.com>
parents: 8554
diff changeset
845 excludes += p.find_classes_with_matching_source_line(None, lambda line: 'JaCoCo Exclude' in line, includeInnerClasses=True).keys()
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
846
9343
cdc21fd3d389 removed obsolete JaCoCo inclusion filter
Doug Simon <doug.simon@oracle.com>
parents: 9291
diff changeset
847 includes = ['com.oracle.graal.*']
4684
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
848 agentOptions = {
5024
7ccdae96e98a Allow JaCoCon to append to the execution file
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5016
diff changeset
849 'append' : 'true' if _jacoco == 'append' else 'false',
4684
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
850 'bootclasspath' : 'true',
5881
472a648769e4 better JaCoCo : only include com.oracle.graal.* and com.oracle.max.*, exclude com.oracle.graal.jtt.*
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5878
diff changeset
851 'includes' : ':'.join(includes),
5878
191e08da4de4 Apply jacoco coverage to all gate tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5877
diff changeset
852 'excludes' : ':'.join(excludes),
191e08da4de4 Apply jacoco coverage to all gate tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5877
diff changeset
853 'destfile' : 'jacoco.exec'
4684
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
854 }
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
855 args = ['-javaagent:' + jacocoagent.get_path(True) + '=' + ','.join([k + '=' + v for k, v in agentOptions.items()])] + args
5596
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
856 exe = join(jdk, 'bin', mx.exe_suffix('java'))
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
857 pfx = _vm_prefix.split() if _vm_prefix is not None else []
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
858
9667
1ef7b26e9177 mx/commands.py: warn about arguments after '-version' when launching the vm
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9639
diff changeset
859 if '-version' in args:
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
860 ignoredArgs = args[args.index('-version') + 1:]
9667
1ef7b26e9177 mx/commands.py: warn about arguments after '-version' when launching the vm
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9639
diff changeset
861 if len(ignoredArgs) > 0:
1ef7b26e9177 mx/commands.py: warn about arguments after '-version' when launching the vm
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9639
diff changeset
862 mx.log("Warning: The following options will be ignored by the vm because they come after the '-version' argument: " + ' '.join(ignoredArgs))
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
863
13952
b076b5c13c3f mx: factor out JavaConfig.processArgs and use it in mx vm. remove default -J arguments.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 13951
diff changeset
864 args = mx.java().processArgs(args)
13951
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
865 return (pfx, exe, vm, args, cwd)
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
866
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
867 def vm(args, vm=None, nonZeroIsFatal=True, out=None, err=None, cwd=None, timeout=None, vmbuild=None):
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
868 (pfx_, exe_, vm_, args_, cwd) = _parseVmArgs(args, vm, cwd, vmbuild)
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
869 return mx.run(pfx_ + [exe_, '-' + vm_] + args_, nonZeroIsFatal=nonZeroIsFatal, out=out, err=err, cwd=cwd, timeout=timeout)
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
870
5762
b30cced39597 generalized functionality for finding classes based on searching for patterns in source code and moved it from commands.py to mx.py
Doug Simon <doug.simon@oracle.com>
parents: 5729
diff changeset
871 def _find_classes_with_annotations(p, pkgRoot, annotations, includeInnerClasses=False):
5366
67e63e8dcbd2 build JaCoCo exclude list by scanning for snippets
Doug Simon <doug.simon@oracle.com>
parents: 5295
diff changeset
872 """
5482
9f4783c0269e folded -G:+CheckcastCounters functionality into checkcast snippets
Doug Simon <doug.simon@oracle.com>
parents: 5438
diff changeset
873 Scan the sources of project 'p' for Java source files containing a line starting with 'annotation'
5762
b30cced39597 generalized functionality for finding classes based on searching for patterns in source code and moved it from commands.py to mx.py
Doug Simon <doug.simon@oracle.com>
parents: 5729
diff changeset
874 (ignoring preceding whitespace) and return the fully qualified class name for each Java
b30cced39597 generalized functionality for finding classes based on searching for patterns in source code and moved it from commands.py to mx.py
Doug Simon <doug.simon@oracle.com>
parents: 5729
diff changeset
875 source file matched in a list.
5366
67e63e8dcbd2 build JaCoCo exclude list by scanning for snippets
Doug Simon <doug.simon@oracle.com>
parents: 5295
diff changeset
876 """
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
877
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
878 matches = lambda line: len([a for a in annotations if line == a or line.startswith(a + '(')]) != 0
5762
b30cced39597 generalized functionality for finding classes based on searching for patterns in source code and moved it from commands.py to mx.py
Doug Simon <doug.simon@oracle.com>
parents: 5729
diff changeset
879 return p.find_classes_with_matching_source_line(pkgRoot, matches, includeInnerClasses)
5366
67e63e8dcbd2 build JaCoCo exclude list by scanning for snippets
Doug Simon <doug.simon@oracle.com>
parents: 5295
diff changeset
880
14744
bf51a92a1bdd Add new optional method parameter
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14720
diff changeset
881 def _extract_VM_args(args, allowClasspath=False, useDoubleDash=False, defaultAllVMArgs=True):
11359
51b0b1104114 changed unittest command to remove use of '@' character. The format is now: mx unittest [VM options] [test filters...]
Doug Simon <doug.simon@oracle.com>
parents: 11302
diff changeset
882 """
51b0b1104114 changed unittest command to remove use of '@' character. The format is now: mx unittest [VM options] [test filters...]
Doug Simon <doug.simon@oracle.com>
parents: 11302
diff changeset
883 Partitions a command line into a leading sequence of HotSpot VM options and the rest.
51b0b1104114 changed unittest command to remove use of '@' character. The format is now: mx unittest [VM options] [test filters...]
Doug Simon <doug.simon@oracle.com>
parents: 11302
diff changeset
884 """
51b0b1104114 changed unittest command to remove use of '@' character. The format is now: mx unittest [VM options] [test filters...]
Doug Simon <doug.simon@oracle.com>
parents: 11302
diff changeset
885 for i in range(0, len(args)):
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
886 if useDoubleDash:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
887 if args[i] == '--':
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
888 vmArgs = args[:i]
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
889 remainder = args[i + 1:]
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
890 return vmArgs, remainder
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
891 else:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
892 if not args[i].startswith('-'):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
893 if i != 0 and (args[i - 1] == '-cp' or args[i - 1] == '-classpath'):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
894 if not allowClasspath:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
895 mx.abort('Cannot supply explicit class path option')
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
896 else:
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
897 continue
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
898 vmArgs = args[:i]
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
899 remainder = args[i:]
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
900 return vmArgs, remainder
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
901
14744
bf51a92a1bdd Add new optional method parameter
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14720
diff changeset
902 if defaultAllVMArgs:
bf51a92a1bdd Add new optional method parameter
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14720
diff changeset
903 return args, []
bf51a92a1bdd Add new optional method parameter
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14720
diff changeset
904 else:
bf51a92a1bdd Add new optional method parameter
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14720
diff changeset
905 return [], args
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
906
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
907 def _run_tests(args, harness, annotations, testfile, whitelist):
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
908
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
909
11359
51b0b1104114 changed unittest command to remove use of '@' character. The format is now: mx unittest [VM options] [test filters...]
Doug Simon <doug.simon@oracle.com>
parents: 11302
diff changeset
910 vmArgs, tests = _extract_VM_args(args)
51b0b1104114 changed unittest command to remove use of '@' character. The format is now: mx unittest [VM options] [test filters...]
Doug Simon <doug.simon@oracle.com>
parents: 11302
diff changeset
911 for t in tests:
51b0b1104114 changed unittest command to remove use of '@' character. The format is now: mx unittest [VM options] [test filters...]
Doug Simon <doug.simon@oracle.com>
parents: 11302
diff changeset
912 if t.startswith('-'):
11403
e45b540b0ef2 clearer error message
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
913 mx.abort('VM option ' + t + ' must precede ' + tests[0])
4159
e253ca26b2d5 Added 'unittest' command to run the Graal unit tests.
Doug Simon <doug.simon@oracle.com>
parents: 4157
diff changeset
914
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
915 candidates = {}
11777
f3e5cbd1efae move pylint to mxtool
Mick Jordan <mick.jordan@oracle.com>
parents: 11758
diff changeset
916 for p in mx.projects_opt_limit_to_suites():
9639
fde5ccbab3a3 made unittest command take Java compliance of a project into account
Doug Simon <doug.simon@oracle.com>
parents: 9598
diff changeset
917 if mx.java().javaCompliance < p.javaCompliance:
fde5ccbab3a3 made unittest command take Java compliance of a project into account
Doug Simon <doug.simon@oracle.com>
parents: 9598
diff changeset
918 continue
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
919 for c in _find_classes_with_annotations(p, None, annotations).keys():
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
920 candidates[c] = p
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
921
10606
a6c0ae38e05e more robust command line checking for unittest (at cost of removing support for negative filters)
Doug Simon <doug.simon@oracle.com>
parents: 10574
diff changeset
922 classes = []
10608
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
923 if len(tests) == 0:
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
924 classes = candidates.keys()
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
925 projectscp = mx.classpath([pcp.name for pcp in mx.projects_opt_limit_to_suites() if pcp.javaCompliance <= mx.java().javaCompliance])
10608
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
926 else:
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
927 projs = set()
10608
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
928 for t in tests:
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
929 found = False
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
930 for c, p in candidates.iteritems():
10608
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
931 if t in c:
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
932 found = True
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
933 classes.append(c)
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
934 projs.add(p.name)
10608
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
935 if not found:
eeb80dcd60d8 fixed regression in unittest command when no tests are specified
Doug Simon <doug.simon@oracle.com>
parents: 10606
diff changeset
936 mx.log('warning: no tests matched by substring "' + t)
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
937 projectscp = mx.classpath(projs)
8343
a3c30d467f96 unittest: just do a single JVM call
Bernhard Urban <bernhard.urban@jku.at>
parents: 8340
diff changeset
938
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
939 if whitelist:
15328
d89ed48ae349 mx: use generators
Bernhard Urban <bernhard.urban@jku.at>
parents: 15327
diff changeset
940 classes = [c for c in classes if any((glob.match(c) for glob in whitelist))]
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
941
8343
a3c30d467f96 unittest: just do a single JVM call
Bernhard Urban <bernhard.urban@jku.at>
parents: 8340
diff changeset
942 if len(classes) != 0:
8406
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
943 f_testfile = open(testfile, 'w')
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
944 for c in classes:
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
945 f_testfile.write(c + '\n')
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
946 f_testfile.close()
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
947 harness(projectscp, vmArgs)
5649
14505f3e6b4c modified harness for running JUnit and JTT tests such that only projects declaring a 'testHarness' attribute in mx/projects are scanned for tests
Doug Simon <doug.simon@oracle.com>
parents: 5596
diff changeset
948
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
949 def _unittest(args, annotations, prefixcp="", whitelist=None):
8406
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
950 mxdir = dirname(__file__)
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
951 name = 'JUnitWrapper'
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
952 javaSource = join(mxdir, name + '.java')
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
953 javaClass = join(mxdir, name + '.class')
9010
08a16c26907f MX_TESTFILE environment variable can be used to specify (and preserve) the test list file used by 'mx unittest'.
Doug Simon <doug.simon@oracle.com>
parents: 8901
diff changeset
954 testfile = os.environ.get('MX_TESTFILE', None)
08a16c26907f MX_TESTFILE environment variable can be used to specify (and preserve) the test list file used by 'mx unittest'.
Doug Simon <doug.simon@oracle.com>
parents: 8901
diff changeset
955 if testfile is None:
08a16c26907f MX_TESTFILE environment variable can be used to specify (and preserve) the test list file used by 'mx unittest'.
Doug Simon <doug.simon@oracle.com>
parents: 8901
diff changeset
956 (_, testfile) = tempfile.mkstemp(".testclasses", "graal")
9236
ec607ddaa99b windows-specific mx fix for executing unit tests selectively
Christian Haeubl <haeubl@ssw.jku.at>
parents: 9177
diff changeset
957 os.close(_)
8406
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
958
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
959 def harness(projectscp, vmArgs):
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
960 if not exists(javaClass) or getmtime(javaClass) < getmtime(javaSource):
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
961 subprocess.check_call([mx.java().javac, '-cp', projectscp, '-d', mxdir, javaSource])
12647
b038b643a3a4 restricted BootstrapGraal option to GRAALVM builds only
Doug Simon <doug.simon@oracle.com>
parents: 12605
diff changeset
962 if _get_vm() != 'graal':
8440
2d0160c35f8f Fix unittest on server0.
Roland Schatz <roland.schatz@oracle.com>
parents: 8433
diff changeset
963 prefixArgs = ['-esa', '-ea']
2d0160c35f8f Fix unittest on server0.
Roland Schatz <roland.schatz@oracle.com>
parents: 8433
diff changeset
964 else:
2d0160c35f8f Fix unittest on server0.
Roland Schatz <roland.schatz@oracle.com>
parents: 8433
diff changeset
965 prefixArgs = ['-XX:-BootstrapGraal', '-esa', '-ea']
11227
bd0e589a9a65 avoid use of JUnitWrapper when exactly one test is being executed
Doug Simon <doug.simon@oracle.com>
parents: 11179
diff changeset
966 with open(testfile) as fp:
bd0e589a9a65 avoid use of JUnitWrapper when exactly one test is being executed
Doug Simon <doug.simon@oracle.com>
parents: 11179
diff changeset
967 testclasses = [l.rstrip() for l in fp.readlines()]
bd0e589a9a65 avoid use of JUnitWrapper when exactly one test is being executed
Doug Simon <doug.simon@oracle.com>
parents: 11179
diff changeset
968 if len(testclasses) == 1:
bd0e589a9a65 avoid use of JUnitWrapper when exactly one test is being executed
Doug Simon <doug.simon@oracle.com>
parents: 11179
diff changeset
969 # Execute Junit directly when one test is being run. This simplifies
bd0e589a9a65 avoid use of JUnitWrapper when exactly one test is being executed
Doug Simon <doug.simon@oracle.com>
parents: 11179
diff changeset
970 # replaying the VM execution in a native debugger (e.g., gdb).
13320
093353894575 Test the Truffle JAR after building.
Chris Seaton <chris.seaton@oracle.com>
parents: 13319
diff changeset
971 vm(prefixArgs + vmArgs + ['-cp', prefixcp + projectscp, 'org.junit.runner.JUnitCore'] + testclasses)
11227
bd0e589a9a65 avoid use of JUnitWrapper when exactly one test is being executed
Doug Simon <doug.simon@oracle.com>
parents: 11179
diff changeset
972 else:
13320
093353894575 Test the Truffle JAR after building.
Chris Seaton <chris.seaton@oracle.com>
parents: 13319
diff changeset
973 vm(prefixArgs + vmArgs + ['-cp', prefixcp + projectscp + os.pathsep + mxdir, name] + [testfile])
8406
2bfb9644dcc2 unittest: call wrapper to avoid long command lines
Bernhard Urban <bernhard.urban@jku.at>
parents: 8343
diff changeset
974
8433
87346100d7a5 temporary file is removed when unittests fail
Doug Simon <doug.simon@oracle.com>
parents: 8406
diff changeset
975 try:
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
976 _run_tests(args, harness, annotations, testfile, whitelist)
8433
87346100d7a5 temporary file is removed when unittests fail
Doug Simon <doug.simon@oracle.com>
parents: 8406
diff changeset
977 finally:
9010
08a16c26907f MX_TESTFILE environment variable can be used to specify (and preserve) the test list file used by 'mx unittest'.
Doug Simon <doug.simon@oracle.com>
parents: 8901
diff changeset
978 if os.environ.get('MX_TESTFILE') is None:
08a16c26907f MX_TESTFILE environment variable can be used to specify (and preserve) the test list file used by 'mx unittest'.
Doug Simon <doug.simon@oracle.com>
parents: 8901
diff changeset
979 os.remove(testfile)
8340
d9d883aeb96f unittest: seperate target `longunittest' and `shortunittest'
Bernhard Urban <bernhard.urban@jku.at>
parents: 8317
diff changeset
980
9598
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
981 _unittestHelpSuffix = """
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
982 Unittest options:
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
983
15323
6b3bb5a9a889 mx: allow to specify a path for the whitelist in the unittest command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15322
diff changeset
984 --whitelist run only testcases which are included
6b3bb5a9a889 mx: allow to specify a path for the whitelist in the unittest command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15322
diff changeset
985 in the given whitelist
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
986
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
987 To avoid conflicts with VM options '--' can be used as delimiter.
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
988
5649
14505f3e6b4c modified harness for running JUnit and JTT tests such that only projects declaring a 'testHarness' attribute in mx/projects are scanned for tests
Doug Simon <doug.simon@oracle.com>
parents: 5596
diff changeset
989 If filters are supplied, only tests whose fully qualified name
10606
a6c0ae38e05e more robust command line checking for unittest (at cost of removing support for negative filters)
Doug Simon <doug.simon@oracle.com>
parents: 10574
diff changeset
990 includes a filter as a substring are run.
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
991
9598
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
992 For example, this command line:
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
993
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
994 mx unittest -G:Dump= -G:MethodFilter=BC_aload.* -G:+PrintCFG BC_aload
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
995
9598
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
996 will run all JUnit test classes that contain 'BC_aload' in their
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
997 fully qualified name and will pass these options to the VM:
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
998
9598
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
999 -G:Dump= -G:MethodFilter=BC_aload.* -G:+PrintCFG
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1000
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1001 To get around command line length limitations on some OSes, the
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1002 JUnit class names to be executed are written to a file that a
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1003 custom JUnit wrapper reads and passes onto JUnit proper. The
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1004 MX_TESTFILE environment variable can be set to specify a
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1005 file which will not be deleted once the unittests are done
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1006 (unlike the temporary file otherwise used).
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1007
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1008 As with all other commands, using the global '-v' before 'unittest'
11359
51b0b1104114 changed unittest command to remove use of '@' character. The format is now: mx unittest [VM options] [test filters...]
Doug Simon <doug.simon@oracle.com>
parents: 11302
diff changeset
1009 command will cause mx to show the complete command line
9598
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1010 it uses to run the VM.
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1011 """
9598
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1012
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1013 def unittest(args):
0fc67e300471 better help message for the unittest commands
Doug Simon <doug.simon@oracle.com>
parents: 9343
diff changeset
1014 """run the JUnit tests (all testcases){0}"""
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1015
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1016 parser = ArgumentParser(prog='mx unittest',
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1017 description='run the JUnit tests',
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1018 add_help=False,
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1019 formatter_class=RawDescriptionHelpFormatter,
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1020 epilog=_unittestHelpSuffix,
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1021 )
15323
6b3bb5a9a889 mx: allow to specify a path for the whitelist in the unittest command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15322
diff changeset
1022 parser.add_argument('--whitelist', help='run testcases specified in whitelist only', metavar='<path>')
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1023
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1024 ut_args = []
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1025 delimiter = False
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1026 # check for delimiter
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1027 while len(args) > 0:
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1028 arg = args.pop(0)
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1029 if arg == '--':
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1030 delimiter = True
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1031 break
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1032 ut_args.append(arg)
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1033
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1034 if delimiter:
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1035 # all arguments before '--' must be recognized
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1036 parsed_args = parser.parse_args(ut_args)
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1037 else:
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1038 # parse all know arguments
15197
06bfcf5f941d mx: improve unittest options parsing.
Josef Eisl <josef.eisl@jku.at>
parents: 15178
diff changeset
1039 parsed_args, args = parser.parse_known_args(ut_args)
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1040
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
1041 whitelist = None
15323
6b3bb5a9a889 mx: allow to specify a path for the whitelist in the unittest command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15322
diff changeset
1042 if parsed_args.whitelist:
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
1043 try:
15323
6b3bb5a9a889 mx: allow to specify a path for the whitelist in the unittest command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15322
diff changeset
1044 with open(join(_graal_home, parsed_args.whitelist)) as fp:
15327
25633b639fd7 mx: support simple filename globbing in whitelist
Bernhard Urban <bernhard.urban@jku.at>
parents: 15326
diff changeset
1045 whitelist = [re.compile(fnmatch.translate(l.rstrip())) for l in fp.readlines() if not l.startswith('#')]
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
1046 except IOError:
15323
6b3bb5a9a889 mx: allow to specify a path for the whitelist in the unittest command
Bernhard Urban <bernhard.urban@jku.at>
parents: 15322
diff changeset
1047 mx.log('warning: could not read whitelist: ' + parsed_args.whitelist)
15114
2082889fc8f6 mx: add unittest option --baseline-whitelist.
Josef Eisl <josef.eisl@jku.at>
parents: 15113
diff changeset
1048
15324
5b5f47104c0d mx: add whitelist for shortunittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15323
diff changeset
1049 _unittest(args, ['@Test', '@Parameters'], whitelist=whitelist)
8340
d9d883aeb96f unittest: seperate target `longunittest' and `shortunittest'
Bernhard Urban <bernhard.urban@jku.at>
parents: 8317
diff changeset
1050
d9d883aeb96f unittest: seperate target `longunittest' and `shortunittest'
Bernhard Urban <bernhard.urban@jku.at>
parents: 8317
diff changeset
1051 def shortunittest(args):
15324
5b5f47104c0d mx: add whitelist for shortunittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15323
diff changeset
1052 """alias for 'unittest --whitelist test/whitelist_shortunittest.txt'{0}"""
8340
d9d883aeb96f unittest: seperate target `longunittest' and `shortunittest'
Bernhard Urban <bernhard.urban@jku.at>
parents: 8317
diff changeset
1053
15324
5b5f47104c0d mx: add whitelist for shortunittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15323
diff changeset
1054 args = ['--whitelist', 'test/whitelist_shortunittest.txt'] + args
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
1055 unittest(args)
8340
d9d883aeb96f unittest: seperate target `longunittest' and `shortunittest'
Bernhard Urban <bernhard.urban@jku.at>
parents: 8317
diff changeset
1056
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1057 def buildvms(args):
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1058 """build one or more VMs in various configurations"""
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1059
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
1060 vmsDefault = ','.join(_vmChoices.keys())
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
1061 vmbuildsDefault = ','.join(_vmbuildChoices)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1062
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1063 parser = ArgumentParser(prog='mx buildvms')
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
1064 parser.add_argument('--vms', help='a comma separated list of VMs to build (default: ' + vmsDefault + ')', metavar='<args>', default=vmsDefault)
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
1065 parser.add_argument('--builds', help='a comma separated list of build types (default: ' + vmbuildsDefault + ')', metavar='<args>', default=vmbuildsDefault)
9096
d4b868ed9cfb buildvms command now runs "java -version" for each VM built unless -n option is specified
Doug Simon <doug.simon@oracle.com>
parents: 9095
diff changeset
1066 parser.add_argument('-n', '--no-check', action='store_true', help='omit running "java -version" after each build')
9097
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1067 parser.add_argument('-c', '--console', action='store_true', help='send build output to console instead of log file')
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1068
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1069 args = parser.parse_args(args)
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1070 vms = args.vms.split(',')
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1071 builds = args.builds.split(',')
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1072
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1073 allStart = time.time()
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1074 for v in vms:
14602
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
1075 if not isVMSupported(v):
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
1076 mx.log('The ' + v + ' VM is not supported on this platform - skipping')
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
1077 continue
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
1078
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1079 for vmbuild in builds:
9117
cbfcb1054619 renamed 'boot' VM to 'original' VM
Doug Simon <doug.simon@oracle.com>
parents: 9116
diff changeset
1080 if v == 'original' and vmbuild != 'product':
9095
d538dce8f403 support for non-GRAAL VMs co-existing with GRAAL VMs
Doug Simon <doug.simon@oracle.com>
parents: 9010
diff changeset
1081 continue
9097
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1082 if not args.console:
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1083 logFile = join(v + '-' + vmbuild + '.log')
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1084 log = open(join(_graal_home, logFile), 'wb')
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1085 start = time.time()
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1086 mx.log('BEGIN: ' + v + '-' + vmbuild + '\t(see: ' + logFile + ')')
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1087 # Run as subprocess so that output can be directed to a file
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1088 subprocess.check_call([sys.executable, '-u', join('mxtool', 'mx.py'), '--vm', v, '--vmbuild',
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1089 vmbuild, 'build'], cwd=_graal_home, stdout=log, stderr=subprocess.STDOUT)
9097
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1090 duration = datetime.timedelta(seconds=time.time() - start)
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1091 mx.log('END: ' + v + '-' + vmbuild + '\t[' + str(duration) + ']')
a0ab945fb95f build output during buildvms command is sent to console (instead of log files) if -c option is given
Doug Simon <doug.simon@oracle.com>
parents: 9096
diff changeset
1092 else:
11371
74414b248381 mx: added --vmbuild global option, removed --product, --fastdebug, --debug options as well as suffix of build command
Doug Simon <doug.simon@oracle.com>
parents: 11368
diff changeset
1093 with VM(v, vmbuild):
74414b248381 mx: added --vmbuild global option, removed --product, --fastdebug, --debug options as well as suffix of build command
Doug Simon <doug.simon@oracle.com>
parents: 11368
diff changeset
1094 build([])
9096
d4b868ed9cfb buildvms command now runs "java -version" for each VM built unless -n option is specified
Doug Simon <doug.simon@oracle.com>
parents: 9095
diff changeset
1095 if not args.no_check:
d4b868ed9cfb buildvms command now runs "java -version" for each VM built unless -n option is specified
Doug Simon <doug.simon@oracle.com>
parents: 9095
diff changeset
1096 vmargs = ['-version']
d4b868ed9cfb buildvms command now runs "java -version" for each VM built unless -n option is specified
Doug Simon <doug.simon@oracle.com>
parents: 9095
diff changeset
1097 if v == 'graal':
d4b868ed9cfb buildvms command now runs "java -version" for each VM built unless -n option is specified
Doug Simon <doug.simon@oracle.com>
parents: 9095
diff changeset
1098 vmargs.insert(0, '-XX:-BootstrapGraal')
d4b868ed9cfb buildvms command now runs "java -version" for each VM built unless -n option is specified
Doug Simon <doug.simon@oracle.com>
parents: 9095
diff changeset
1099 vm(vmargs, vm=v, vmbuild=vmbuild)
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1100 allDuration = datetime.timedelta(seconds=time.time() - allStart)
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1101 mx.log('TOTAL TIME: ' + '[' + str(allDuration) + ']')
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1102
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1103 class Task:
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1104 def __init__(self, title):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1105 self.start = time.time()
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1106 self.title = title
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1107 self.end = None
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1108 self.duration = None
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1109 mx.log(time.strftime('gate: %d %b %Y %H:%M:%S: BEGIN: ') + title)
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1110 def stop(self):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1111 self.end = time.time()
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1112 self.duration = datetime.timedelta(seconds=self.end - self.start)
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1113 mx.log(time.strftime('gate: %d %b %Y %H:%M:%S: END: ') + self.title + ' [' + str(self.duration) + ']')
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1114 return self
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1115 def abort(self, codeOrMessage):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1116 self.end = time.time()
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1117 self.duration = datetime.timedelta(seconds=self.end - self.start)
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1118 mx.log(time.strftime('gate: %d %b %Y %H:%M:%S: ABORT: ') + self.title + ' [' + str(self.duration) + ']')
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1119 mx.abort(codeOrMessage)
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1120 return self
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1121
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1122 def _basic_gate_body(args, tasks):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1123 t = Task('BuildHotSpotGraal: fastdebug,product')
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1124 buildvms(['--vms', 'graal,server', '--builds', 'fastdebug,product'])
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1125 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1126
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1127 with VM('graal', 'fastdebug'):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1128 t = Task('BootstrapWithSystemAssertions:fastdebug')
14877
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents: 14872
diff changeset
1129 vm(['-esa', '-XX:-TieredCompilation', '-version'])
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1130 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1131
14123
dc41eab09fe8 gate: add non-tiered bootstrap
Bernhard Urban <bernhard.urban@jku.at>
parents: 14116
diff changeset
1132 with VM('graal', 'fastdebug'):
14877
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents: 14872
diff changeset
1133 t = Task('BootstrapWithSystemAssertionsNoCoop:fastdebug')
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents: 14872
diff changeset
1134 vm(['-esa', '-XX:-TieredCompilation', '-XX:-UseCompressedOops', '-version'])
14123
dc41eab09fe8 gate: add non-tiered bootstrap
Bernhard Urban <bernhard.urban@jku.at>
parents: 14116
diff changeset
1135 tasks.append(t.stop())
dc41eab09fe8 gate: add non-tiered bootstrap
Bernhard Urban <bernhard.urban@jku.at>
parents: 14116
diff changeset
1136
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1137 with VM('graal', 'product'):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1138 t = Task('BootstrapWithGCVerification:product')
12042
fbe1ee508936 added ability to suppress duplicate lines on an output stream and used it to filter the GC verification log messages in the gate
Doug Simon <doug.simon@oracle.com>
parents: 11936
diff changeset
1139 out = mx.DuplicateSuppressingStream(['VerifyAfterGC:', 'VerifyBeforeGC:']).write
14877
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents: 14872
diff changeset
1140 vm(['-XX:-TieredCompilation', '-XX:+UnlockDiagnosticVMOptions', '-XX:+VerifyBeforeGC', '-XX:+VerifyAfterGC', '-version'], out=out)
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1141 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1142
12412
3c11430f62d8 Re-enable G1 verification in gate
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 12359
diff changeset
1143 with VM('graal', 'product'):
3c11430f62d8 Re-enable G1 verification in gate
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 12359
diff changeset
1144 t = Task('BootstrapWithG1GCVerification:product')
3c11430f62d8 Re-enable G1 verification in gate
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 12359
diff changeset
1145 out = mx.DuplicateSuppressingStream(['VerifyAfterGC:', 'VerifyBeforeGC:']).write
14877
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents: 14872
diff changeset
1146 vm(['-XX:-TieredCompilation', '-XX:+UnlockDiagnosticVMOptions', '-XX:-UseSerialGC', '-XX:+UseG1GC', '-XX:+VerifyBeforeGC', '-XX:+VerifyAfterGC', '-version'], out=out)
12412
3c11430f62d8 Re-enable G1 verification in gate
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 12359
diff changeset
1147 tasks.append(t.stop())
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1148
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1149 with VM('graal', 'product'):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1150 t = Task('BootstrapWithRegisterPressure:product')
14877
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents: 14872
diff changeset
1151 vm(['-XX:-TieredCompilation', '-G:RegisterPressure=rbx,r11,r10,r14,xmm3,xmm11,xmm14', '-esa', '-version'])
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1152 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1153
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1154 with VM('graal', 'product'):
13378
16d99e9d77ad Options: rename flag (AOTCompilation -> ImmutableCode)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13347
diff changeset
1155 t = Task('BootstrapWithImmutableCode:product')
14877
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents: 14872
diff changeset
1156 vm(['-XX:-TieredCompilation', '-G:+ImmutableCode', '-G:+VerifyPhases', '-esa', '-version'])
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1157 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1158
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1159 with VM('server', 'product'): # hosted mode
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1160 t = Task('UnitTests:hosted-product')
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1161 unittest([])
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1162 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1163
15325
25ce2866f105 gate: add baseline compiler unittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15324
diff changeset
1164 with VM('server', 'product'): # hosted mode
25ce2866f105 gate: add baseline compiler unittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15324
diff changeset
1165 t = Task('UnitTests-BaselineCompiler:hosted-product')
25ce2866f105 gate: add baseline compiler unittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15324
diff changeset
1166 unittest(['--whitelist', 'test/whitelist_baseline.txt', '-G:+UseBaselineCompiler'])
25ce2866f105 gate: add baseline compiler unittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15324
diff changeset
1167 tasks.append(t.stop())
25ce2866f105 gate: add baseline compiler unittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15324
diff changeset
1168
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1169 for vmbuild in ['fastdebug', 'product']:
14877
fdb912b1eb7b gate: use no tiered to test bootstrap, add scaladacapo-kiama (fastdebug)
Bernhard Urban <bernhard.urban@jku.at>
parents: 14872
diff changeset
1170 for test in sanitycheck.getDacapos(level=sanitycheck.SanityCheckLevel.Gate, gateBuildLevel=vmbuild) + sanitycheck.getScalaDacapos(level=sanitycheck.SanityCheckLevel.Gate, gateBuildLevel=vmbuild):
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1171 t = Task(str(test) + ':' + vmbuild)
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1172 if not test.test('graal'):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1173 t.abort(test.name + ' Failed')
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1174 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1175
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1176 if args.jacocout is not None:
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1177 jacocoreport([args.jacocout])
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1178
11758
4ac92e735a16 pylint fixes (retry)
Mick Jordan <mick.jordan@oracle.com>
parents: 11757
diff changeset
1179 global _jacoco
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1180 _jacoco = 'off'
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1181
12497
98caa3872d83 gate: clean and build IGV (not GV)
Bernhard Urban <bernhard.urban@jku.at>
parents: 12412
diff changeset
1182 t = Task('CleanAndBuildIdealGraphVisualizer')
98caa3872d83 gate: clean and build IGV (not GV)
Bernhard Urban <bernhard.urban@jku.at>
parents: 12412
diff changeset
1183 mx.run(['ant', '-f', join(_graal_home, 'src', 'share', 'tools', 'IdealGraphVisualizer', 'build.xml'), '-q', 'clean', 'build'])
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1184 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1185
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1186 # Prevent Graal modifications from breaking the standard builds
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1187 if args.buildNonGraal:
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1188 t = Task('BuildHotSpotVarieties')
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1189 buildvms(['--vms', 'client,server', '--builds', 'fastdebug,product'])
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1190 buildvms(['--vms', 'server-nograal', '--builds', 'product'])
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1191 buildvms(['--vms', 'server-nograal', '--builds', 'optimized'])
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1192 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1193
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1194 for vmbuild in ['product', 'fastdebug']:
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1195 for theVm in ['client', 'server']:
14606
f50dece27798 fixed regression in gate command
Doug Simon <doug.simon@oracle.com>
parents: 14605
diff changeset
1196 if not isVMSupported(theVm):
f50dece27798 fixed regression in gate command
Doug Simon <doug.simon@oracle.com>
parents: 14605
diff changeset
1197 mx.log('The' + theVm + ' VM is not supported on this platform')
14602
591f4a575ebf issue warning/error where ever relevant if a non-supported VM (e.g., client VM on Mac) is used in mx
Doug Simon <doug.simon@oracle.com>
parents: 14595
diff changeset
1198 continue
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1199 with VM(theVm, vmbuild):
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1200 t = Task('DaCapo_pmd:' + theVm + ':' + vmbuild)
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1201 dacapo(['pmd'])
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1202 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1203
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1204 t = Task('UnitTests:' + theVm + ':' + vmbuild)
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1205 unittest(['-XX:CompileCommand=exclude,*::run*', 'graal.api'])
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1206 tasks.append(t.stop())
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1207
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1208
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1209 def gate(args, gate_body=_basic_gate_body):
4159
e253ca26b2d5 Added 'unittest' command to run the Graal unit tests.
Doug Simon <doug.simon@oracle.com>
parents: 4157
diff changeset
1210 """run the tests used to validate a push
e253ca26b2d5 Added 'unittest' command to run the Graal unit tests.
Doug Simon <doug.simon@oracle.com>
parents: 4157
diff changeset
1211
4438
0312460af9fc Made gate do a full clean and added timing report for all gate tasks to end of gate log.
Doug Simon <doug.simon@oracle.com>
parents: 4417
diff changeset
1212 If this command exits with a 0 exit code, then the source code is in
4159
e253ca26b2d5 Added 'unittest' command to run the Graal unit tests.
Doug Simon <doug.simon@oracle.com>
parents: 4157
diff changeset
1213 a state that would be accepted for integration into the main repository."""
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1214
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1215 parser = ArgumentParser(prog='mx gate')
5232
c005ca943790 added -j option to gate command to disable cleaning of Java class files
Doug Simon <doug.simon@oracle.com>
parents: 5197
diff changeset
1216 parser.add_argument('-j', '--omit-java-clean', action='store_false', dest='cleanJava', help='omit cleaning Java native code')
5750
30876d0bb92d In gate command, allways build natives, use '-n' to avoid cleaning them only
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5729
diff changeset
1217 parser.add_argument('-n', '--omit-native-clean', action='store_false', dest='cleanNative', help='omit cleaning and building native code')
4608
05a33fabcfe6 Added -g option to 'mx gate' to omit the server and client builds (i.e. -g == graal only).
Doug Simon <doug.simon@oracle.com>
parents: 4607
diff changeset
1218 parser.add_argument('-g', '--only-build-graalvm', action='store_false', dest='buildNonGraal', help='only build the Graal VM')
5033
d7ecce178ad2 Add jacoco report and a -XX:+DeoptimizeALot -XX:+VerifyOops bootstrap run to the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5032
diff changeset
1219 parser.add_argument('--jacocout', help='specify the output directory for jacoco report')
4601
ade18666f2be Added --omit-native-build to gate command so that native cleaning and building can be omitted for changesets that made no changes to the C/C++ code.
Doug Simon <doug.simon@oracle.com>
parents: 4584
diff changeset
1220
ade18666f2be Added --omit-native-build to gate command so that native cleaning and building can be omitted for changesets that made no changes to the C/C++ code.
Doug Simon <doug.simon@oracle.com>
parents: 4584
diff changeset
1221 args = parser.parse_args(args)
ade18666f2be Added --omit-native-build to gate command so that native cleaning and building can be omitted for changesets that made no changes to the C/C++ code.
Doug Simon <doug.simon@oracle.com>
parents: 4584
diff changeset
1222
5431
0bd1ba69db2a removed double declaration of _vmbuild global variable in gate command
Doug Simon <doug.simon@oracle.com>
parents: 5430
diff changeset
1223 global _jacoco
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1224
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1225 tasks = []
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
1226 total = Task('Gate')
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
1227 try:
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1228
11526
db297343d44e added pylint to gate
Doug Simon <doug.simon@oracle.com>
parents: 11525
diff changeset
1229 t = Task('Pylint')
11777
f3e5cbd1efae move pylint to mxtool
Mick Jordan <mick.jordan@oracle.com>
parents: 11758
diff changeset
1230 mx.pylint([])
11526
db297343d44e added pylint to gate
Doug Simon <doug.simon@oracle.com>
parents: 11525
diff changeset
1231 tasks.append(t.stop())
db297343d44e added pylint to gate
Doug Simon <doug.simon@oracle.com>
parents: 11525
diff changeset
1232
13928
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1233 def _clean(name='Clean'):
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1234 t = Task(name)
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1235 cleanArgs = []
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1236 if not args.cleanNative:
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1237 cleanArgs.append('--no-native')
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1238 if not args.cleanJava:
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1239 cleanArgs.append('--no-java')
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1240 clean(cleanArgs)
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1241 tasks.append(t.stop())
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1242 _clean()
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1243
9824
6fa4b4933892 added check to gate that generated IDE configurations don't break the build
Doug Simon <doug.simon@oracle.com>
parents: 9803
diff changeset
1244 t = Task('IDEConfigCheck')
6fa4b4933892 added check to gate that generated IDE configurations don't break the build
Doug Simon <doug.simon@oracle.com>
parents: 9803
diff changeset
1245 mx.ideclean([])
6fa4b4933892 added check to gate that generated IDE configurations don't break the build
Doug Simon <doug.simon@oracle.com>
parents: 9803
diff changeset
1246 mx.ideinit([])
6fa4b4933892 added check to gate that generated IDE configurations don't break the build
Doug Simon <doug.simon@oracle.com>
parents: 9803
diff changeset
1247 tasks.append(t.stop())
6fa4b4933892 added check to gate that generated IDE configurations don't break the build
Doug Simon <doug.simon@oracle.com>
parents: 9803
diff changeset
1248
13928
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1249 eclipse_exe = mx.get_env('ECLIPSE_EXE')
7533
29b0768b7ba6 added eclipseformat to the gate
Doug Simon <doug.simon@oracle.com>
parents: 7509
diff changeset
1250 if eclipse_exe is not None:
29b0768b7ba6 added eclipseformat to the gate
Doug Simon <doug.simon@oracle.com>
parents: 7509
diff changeset
1251 t = Task('CodeFormatCheck')
29b0768b7ba6 added eclipseformat to the gate
Doug Simon <doug.simon@oracle.com>
parents: 7509
diff changeset
1252 if mx.eclipseformat(['-e', eclipse_exe]) != 0:
29b0768b7ba6 added eclipseformat to the gate
Doug Simon <doug.simon@oracle.com>
parents: 7509
diff changeset
1253 t.abort('Formatter modified files - run "mx eclipseformat", check in changes and repush')
29b0768b7ba6 added eclipseformat to the gate
Doug Simon <doug.simon@oracle.com>
parents: 7509
diff changeset
1254 tasks.append(t.stop())
8211
74896b25297a Test canonicalization and checkstyle early in gate process
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8183
diff changeset
1255
74896b25297a Test canonicalization and checkstyle early in gate process
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8183
diff changeset
1256 t = Task('Canonicalization Check')
74896b25297a Test canonicalization and checkstyle early in gate process
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8183
diff changeset
1257 mx.log(time.strftime('%d %b %Y %H:%M:%S - Ensuring mx/projects files are canonicalized...'))
74896b25297a Test canonicalization and checkstyle early in gate process
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8183
diff changeset
1258 if mx.canonicalizeprojects([]) != 0:
74896b25297a Test canonicalization and checkstyle early in gate process
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8183
diff changeset
1259 t.abort('Rerun "mx canonicalizeprojects" and check-in the modified mx/projects files.')
74896b25297a Test canonicalization and checkstyle early in gate process
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8183
diff changeset
1260 tasks.append(t.stop())
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1261
13928
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1262 if mx.get_env('JDT'):
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1263 t = Task('BuildJavaWithEcj')
15461
f73fc9309f12 gate: use parallelized Java building in the gate
Doug Simon <doug.simon@oracle.com>
parents: 15420
diff changeset
1264 build(['-p', '--no-native', '--jdt-warning-as-error'])
13928
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1265 tasks.append(t.stop())
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1266
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1267 _clean('CleanAfterEcjBuild')
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1268
c4e5a685c6a1 gate: compile java with ECJ if available
Bernhard Urban <bernhard.urban@jku.at>
parents: 13919
diff changeset
1269 t = Task('BuildJavaWithJavac')
15461
f73fc9309f12 gate: use parallelized Java building in the gate
Doug Simon <doug.simon@oracle.com>
parents: 15420
diff changeset
1270 build(['-p', '--no-native', '--force-javac'])
4438
0312460af9fc Made gate do a full clean and added timing report for all gate tasks to end of gate log.
Doug Simon <doug.simon@oracle.com>
parents: 4417
diff changeset
1271 tasks.append(t.stop())
9172
bc5c5336008b Add gate test to check build-graal.xml is up to date.
Roland Schatz <roland.schatz@oracle.com>
parents: 9169
diff changeset
1272
14773
78343531acc7 added checkheaders to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14772
diff changeset
1273 t = Task('Checkheaders')
78343531acc7 added checkheaders to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14772
diff changeset
1274 if checkheaders([]) != 0:
78343531acc7 added checkheaders to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14772
diff changeset
1275 t.abort('Checkheaders warnings were found')
78343531acc7 added checkheaders to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14772
diff changeset
1276 tasks.append(t.stop())
78343531acc7 added checkheaders to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14772
diff changeset
1277
14751
015f84f0b375 added findbugs to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14744
diff changeset
1278 t = Task('FindBugs')
015f84f0b375 added findbugs to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14744
diff changeset
1279 if findbugs([]) != 0:
015f84f0b375 added findbugs to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14744
diff changeset
1280 t.abort('FindBugs warnings were found')
015f84f0b375 added findbugs to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14744
diff changeset
1281 tasks.append(t.stop())
015f84f0b375 added findbugs to the gate
Doug Simon <doug.simon@oracle.com>
parents: 14744
diff changeset
1282
5878
191e08da4de4 Apply jacoco coverage to all gate tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5877
diff changeset
1283 if exists('jacoco.exec'):
191e08da4de4 Apply jacoco coverage to all gate tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5877
diff changeset
1284 os.unlink('jacoco.exec')
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1285
5878
191e08da4de4 Apply jacoco coverage to all gate tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5877
diff changeset
1286 if args.jacocout is not None:
191e08da4de4 Apply jacoco coverage to all gate tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5877
diff changeset
1287 _jacoco = 'append'
191e08da4de4 Apply jacoco coverage to all gate tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5877
diff changeset
1288 else:
191e08da4de4 Apply jacoco coverage to all gate tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5877
diff changeset
1289 _jacoco = 'off'
11757
ec058ce90a3d pylint fixes
Mick Jordan <mick.jordan@oracle.com>
parents: 11756
diff changeset
1290
11756
78e6109ee411 mx refactoring for split-repo suites; should have no effect on existing repos
Mick Jordan <mick.jordan@oracle.com>
parents: 11605
diff changeset
1291 gate_body(args, tasks)
7155
c356cab093bc added execution of the Graal API unittest with non-GraalVM HotSpot builds to the gate
Doug Simon <doug.simon@oracle.com>
parents: 7154
diff changeset
1292
4247
3f6c6e61614e Changed 'mx build' such that all Java sources for a project are compiled together instead of by source directory.
Doug Simon <doug.simon@oracle.com>
parents: 4244
diff changeset
1293 except KeyboardInterrupt:
3f6c6e61614e Changed 'mx build' such that all Java sources for a project are compiled together instead of by source directory.
Doug Simon <doug.simon@oracle.com>
parents: 4244
diff changeset
1294 total.abort(1)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1295
4584
7e5d8d1c74a1 Modified JDK replication process such that a copy of the default VM is made. This prevents issues with the replicated JDK being used to bootstrap a HotSpot build.
Doug Simon <doug.simon@oracle.com>
parents: 4582
diff changeset
1296 except BaseException as e:
4247
3f6c6e61614e Changed 'mx build' such that all Java sources for a project are compiled together instead of by source directory.
Doug Simon <doug.simon@oracle.com>
parents: 4244
diff changeset
1297 import traceback
3f6c6e61614e Changed 'mx build' such that all Java sources for a project are compiled together instead of by source directory.
Doug Simon <doug.simon@oracle.com>
parents: 4244
diff changeset
1298 traceback.print_exc()
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
1299 total.abort(str(e))
4159
e253ca26b2d5 Added 'unittest' command to run the Graal unit tests.
Doug Simon <doug.simon@oracle.com>
parents: 4157
diff changeset
1300
4234
057620486c90 Improved time stamps of gate tasks to show duration of each task explicitly.
Doug Simon <doug.simon@oracle.com>
parents: 4230
diff changeset
1301 total.stop()
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1302
4438
0312460af9fc Made gate do a full clean and added timing report for all gate tasks to end of gate log.
Doug Simon <doug.simon@oracle.com>
parents: 4417
diff changeset
1303 mx.log('Gate task times:')
0312460af9fc Made gate do a full clean and added timing report for all gate tasks to end of gate log.
Doug Simon <doug.simon@oracle.com>
parents: 4417
diff changeset
1304 for t in tasks:
0312460af9fc Made gate do a full clean and added timing report for all gate tasks to end of gate log.
Doug Simon <doug.simon@oracle.com>
parents: 4417
diff changeset
1305 mx.log(' ' + str(t.duration) + '\t' + t.title)
0312460af9fc Made gate do a full clean and added timing report for all gate tasks to end of gate log.
Doug Simon <doug.simon@oracle.com>
parents: 4417
diff changeset
1306 mx.log(' =======')
0312460af9fc Made gate do a full clean and added timing report for all gate tasks to end of gate log.
Doug Simon <doug.simon@oracle.com>
parents: 4417
diff changeset
1307 mx.log(' ' + str(total.duration))
11757
ec058ce90a3d pylint fixes
Mick Jordan <mick.jordan@oracle.com>
parents: 11756
diff changeset
1308
6340
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1309 def deoptalot(args):
11377
0fbc1e418c88 fixed capitalization and tense in documentation of some mx commands
Doug Simon <doug.simon@oracle.com>
parents: 11373
diff changeset
1310 """bootstrap a fastdebug Graal VM with DeoptimizeALot and VerifyOops on
6340
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1311
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1312 If the first argument is a number, the process will be repeated
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1313 this number of times. All other arguments are passed to the VM."""
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1314 count = 1
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1315 if len(args) > 0 and args[0].isdigit():
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1316 count = int(args[0])
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1317 del args[0]
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1318
7509
442668d41bc2 fixed warnings
Doug Simon <doug.simon@oracle.com>
parents: 7345
diff changeset
1319 for _ in range(count):
6340
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1320 if not vm(['-XX:+DeoptimizeALot', '-XX:+VerifyOops'] + args + ['-version'], vmbuild='fastdebug') == 0:
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1321 mx.abort("Failed")
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1322
6340
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1323 def longtests(args):
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1324
6340
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1325 deoptalot(['15', '-Xmx48m'])
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1326
6340
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
1327 dacapo(['100', 'eclipse', '-esa'])
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4187
diff changeset
1328
5089
206e68bb66c6 Made the IdealGraphVisualizer runnable from mx with new 'igv' command.
Doug Simon <doug.simon@oracle.com>
parents: 5064
diff changeset
1329 def igv(args):
206e68bb66c6 Made the IdealGraphVisualizer runnable from mx with new 'igv' command.
Doug Simon <doug.simon@oracle.com>
parents: 5064
diff changeset
1330 """run the Ideal Graph Visualizer"""
206e68bb66c6 Made the IdealGraphVisualizer runnable from mx with new 'igv' command.
Doug Simon <doug.simon@oracle.com>
parents: 5064
diff changeset
1331 with open(join(_graal_home, '.ideal_graph_visualizer.log'), 'w') as fp:
14116
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1332 # When the http_proxy environment variable is set, convert it to the proxy settings that ant needs
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1333 env = os.environ
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1334 proxy = os.environ.get('http_proxy')
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1335 if not (proxy is None) and len(proxy) > 0:
14590
e754ac1331ef fix string contains call
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 14164
diff changeset
1336 if '://' in proxy:
14116
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1337 # Remove the http:// prefix (or any other protocol prefix)
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1338 proxy = proxy.split('://', 1)[1]
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1339 # Separate proxy server name and port number
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1340 proxyName, proxyPort = proxy.split(':', 1)
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1341 proxyEnv = '-DproxyHost="' + proxyName + '" -DproxyPort=' + proxyPort
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1342 env['ANT_OPTS'] = proxyEnv
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1343
10568
070b4a3c56f3 disabled "noisy" log statements unless -v option (i.e. verbose) is specified to mx
Doug Simon <doug.simon@oracle.com>
parents: 10527
diff changeset
1344 mx.logv('[Ideal Graph Visualizer log is in ' + fp.name + ']')
13472
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1345 nbplatform = join(_graal_home, 'src', 'share', 'tools', 'IdealGraphVisualizer', 'nbplatform')
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1346
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1347 # Remove NetBeans platform if it is earlier than the current supported version
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1348 if exists(nbplatform):
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1349 dom = xml.dom.minidom.parse(join(nbplatform, 'platform', 'update_tracking', 'org-netbeans-core.xml'))
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1350 currentVersion = mx.VersionSpec(dom.getElementsByTagName('module_version')[0].getAttribute('specification_version'))
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1351 supportedVersion = mx.VersionSpec('3.43.1')
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1352 if currentVersion < supportedVersion:
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1353 mx.log('Replacing NetBeans platform version ' + str(currentVersion) + ' with version ' + str(supportedVersion))
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1354 shutil.rmtree(nbplatform)
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1355 elif supportedVersion < currentVersion:
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1356 mx.log('Supported NetBeans version in igv command should be updated to ' + str(currentVersion))
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1357
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1358 if not exists(nbplatform):
72e2ec923b7b added support to replace NetBeans platform used by IGV if it is out of date (GRAAL-420)
Doug Simon <doug.simon@oracle.com>
parents: 13471
diff changeset
1359 mx.logv('[This execution may take a while as the NetBeans platform needs to be downloaded]')
14116
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1360 mx.run(['ant', '-f', join(_graal_home, 'src', 'share', 'tools', 'IdealGraphVisualizer', 'build.xml'), '-l', fp.name, 'run'], env=env)
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1361
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1362 def c1visualizer(args):
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1363 """run the Cl Compiler Visualizer"""
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1364 libpath = join(_graal_home, 'lib')
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1365 if mx.get_os() == 'windows':
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1366 executable = join(libpath, 'c1visualizer', 'bin', 'c1visualizer.exe')
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1367 else:
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1368 executable = join(libpath, 'c1visualizer', 'bin', 'c1visualizer')
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1369
15275
0c9406f9338f Update c1visualizer to version compatible with Java 8
Christian Wimmer <christian.wimmer@oracle.com>
parents: 15253
diff changeset
1370 archive = join(libpath, 'c1visualizer_2014-04-22.zip')
15310
dc2dd913c47e mx: ensure newest version of c1visualizer is being used
Doug Simon <doug.simon@oracle.com>
parents: 15275
diff changeset
1371 if not exists(executable) or not exists(archive):
14116
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1372 if not exists(archive):
15275
0c9406f9338f Update c1visualizer to version compatible with Java 8
Christian Wimmer <christian.wimmer@oracle.com>
parents: 15253
diff changeset
1373 mx.download(archive, ['https://java.net/downloads/c1visualizer/c1visualizer_2014-04-22.zip'])
14116
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1374 zf = zipfile.ZipFile(archive, 'r')
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1375 zf.extractall(libpath)
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1376
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1377 if not exists(executable):
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1378 mx.abort('C1Visualizer binary does not exist: ' + executable)
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1379
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1380 if mx.get_os() != 'windows':
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1381 # Make sure that execution is allowed. The zip file does not always specfiy that correctly
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1382 os.chmod(executable, 0777)
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1383
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
1384 mx.run([executable])
5089
206e68bb66c6 Made the IdealGraphVisualizer runnable from mx with new 'igv' command.
Doug Simon <doug.simon@oracle.com>
parents: 5064
diff changeset
1385
4215
a2caa019ba3a Fix mx : commands' scripts mx_init hook should be called before parsing command line arguments.
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4187
diff changeset
1386 def bench(args):
4324
b0aa4a52b89c Added support for project specs in classpath.
Doug Simon <doug.simon@oracle.com>
parents: 4294
diff changeset
1387 """run benchmarks and parse their output for results
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1388
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1389 Results are JSON formated : {group : {benchmark : score}}."""
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1390 resultFile = None
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1391 if '-resultfile' in args:
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1392 index = args.index('-resultfile')
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1393 if index + 1 < len(args):
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1394 resultFile = args[index + 1]
4293
d4906ea4255b Handle SIGTERM : killing mx will properly kill subprocesses
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4288
diff changeset
1395 del args[index]
d4906ea4255b Handle SIGTERM : killing mx will properly kill subprocesses
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4288
diff changeset
1396 del args[index]
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1397 else:
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1398 mx.abort('-resultfile must be followed by a file name')
11296
4e943a311d9c mx presents a command line dialogue to select the default VM if it is not configured (GRAAL-416)
Doug Simon <doug.simon@oracle.com>
parents: 11287
diff changeset
1399 vm = _get_vm()
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1400 if len(args) is 0:
7563
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1401 args = ['all']
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1402
8459
3393f870e6a4 Allow vm arguments in mx bench
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8440
diff changeset
1403 vmArgs = [arg for arg in args if arg.startswith('-')]
7563
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1404
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1405 def benchmarks_in_group(group):
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1406 prefix = group + ':'
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1407 return [a[len(prefix):] for a in args if a.startswith(prefix)]
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1408
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4216
diff changeset
1409 results = {}
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1410 benchmarks = []
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1411 # DaCapo
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1412 if 'dacapo' in args or 'all' in args:
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1413 benchmarks += sanitycheck.getDacapos(level=sanitycheck.SanityCheckLevel.Benchmark)
4335
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4330
diff changeset
1414 else:
7563
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1415 dacapos = benchmarks_in_group('dacapo')
4335
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4330
diff changeset
1416 for dacapo in dacapos:
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4330
diff changeset
1417 if dacapo not in sanitycheck.dacapoSanityWarmup.keys():
7563
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1418 mx.abort('Unknown DaCapo : ' + dacapo)
8891
ca82d06ec93a mx: bench command should not run dacapos with -n 0 when using a specification such as scaladacapo:specs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8890
diff changeset
1419 iterations = sanitycheck.dacapoSanityWarmup[dacapo][sanitycheck.SanityCheckLevel.Benchmark]
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1420 if iterations > 0:
8891
ca82d06ec93a mx: bench command should not run dacapos with -n 0 when using a specification such as scaladacapo:specs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8890
diff changeset
1421 benchmarks += [sanitycheck.getDacapo(dacapo, iterations)]
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1422
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1423 if 'scaladacapo' in args or 'all' in args:
4480
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4438
diff changeset
1424 benchmarks += sanitycheck.getScalaDacapos(level=sanitycheck.SanityCheckLevel.Benchmark)
7d6490436b57 Add Scala DaCapo benchs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4438
diff changeset
1425 else:
7563
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1426 scaladacapos = benchmarks_in_group('scaladacapo')
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1427 for scaladacapo in scaladacapos:
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1428 if scaladacapo not in sanitycheck.dacapoScalaSanityWarmup.keys():
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1429 mx.abort('Unknown Scala DaCapo : ' + scaladacapo)
8891
ca82d06ec93a mx: bench command should not run dacapos with -n 0 when using a specification such as scaladacapo:specs
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8890
diff changeset
1430 iterations = sanitycheck.dacapoScalaSanityWarmup[scaladacapo][sanitycheck.SanityCheckLevel.Benchmark]
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1431 if iterations > 0:
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1432 benchmarks += [sanitycheck.getScalaDacapo(scaladacapo, ['-n', str(iterations)])]
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1433
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1434 # Bootstrap
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1435 if 'bootstrap' in args or 'all' in args:
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1436 benchmarks += sanitycheck.getBootstraps()
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1437 # SPECjvm2008
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1438 if 'specjvm2008' in args or 'all' in args:
11373
6675167d52b6 fixed specjvm2008 regression in bench command
Doug Simon <doug.simon@oracle.com>
parents: 11372
diff changeset
1439 benchmarks += [sanitycheck.getSPECjvm2008(['-ikv', '-wt', '120', '-it', '120'])]
4335
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4330
diff changeset
1440 else:
7563
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1441 specjvms = benchmarks_in_group('specjvm2008')
4335
0d661161fa24 Add possibility to run individual DaCapo & SPECjvm2008 benchmarks, small fix in success and score regular expressions for SPECjvm2008
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4330
diff changeset
1442 for specjvm in specjvms:
11373
6675167d52b6 fixed specjvm2008 regression in bench command
Doug Simon <doug.simon@oracle.com>
parents: 11372
diff changeset
1443 benchmarks += [sanitycheck.getSPECjvm2008(['-ikv', '-wt', '120', '-it', '120', specjvm])]
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1444
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1445 if 'specjbb2005' in args or 'all' in args:
5877
0e54d9bb922d Add SPECjbb2005 to the benchmarks
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5875
diff changeset
1446 benchmarks += [sanitycheck.getSPECjbb2005()]
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1447
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1448 if 'specjbb2013' in args: # or 'all' in args //currently not in default set
7565
9a521597686b Add SPECjbb2013
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 7563
diff changeset
1449 benchmarks += [sanitycheck.getSPECjbb2013()]
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1450
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1451 if 'ctw-full' in args:
9157
a38d748d4130 Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9130
diff changeset
1452 benchmarks.append(sanitycheck.getCTW(vm, sanitycheck.CTWMode.Full))
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1453 if 'ctw-noinline' in args:
9157
a38d748d4130 Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9130
diff changeset
1454 benchmarks.append(sanitycheck.getCTW(vm, sanitycheck.CTWMode.NoInline))
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1455 if 'ctw-nocomplex' in args:
9157
a38d748d4130 Add support for the 'optimized' build in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9130
diff changeset
1456 benchmarks.append(sanitycheck.getCTW(vm, sanitycheck.CTWMode.NoComplex))
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1457
4219
47f7d91d34cf Fix javac build issues with classpath
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4216
diff changeset
1458 for test in benchmarks:
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1459 for (groupName, res) in test.bench(vm, extraVmOpts=vmArgs).items():
7563
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1460 group = results.setdefault(groupName, {})
3aab15f42934 moved execution of a benchmark out of OutputParser
Doug Simon <doug.simon@oracle.com>
parents: 7552
diff changeset
1461 group.update(res)
4282
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1462 mx.log(json.dumps(results))
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1463 if resultFile:
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1464 with open(resultFile, 'w') as f:
063ea022532c mx commands : improve bench command, fix in the outparser so that no empty 'line' is retruned if there was no match, fix bootstrap's regex
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4276
diff changeset
1465 f.write(json.dumps(results))
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1466
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1467 def _get_jmh_path():
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1468 path = mx.get_env('JMH_BENCHMARKS', None)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1469 if not path:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1470 probe = join(dirname(_graal_home), 'java-benchmarks')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1471 if exists(probe):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1472 path = probe
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1473
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1474 if not path:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1475 mx.abort("Please set the JMH_BENCHMARKS environment variable to point to the java-benchmarks workspace")
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1476 if not exists(path):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1477 mx.abort("The directory denoted by the JMH_BENCHMARKS environment variable does not exist: " + path)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1478 return path
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1479
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1480 def makejmhdeps(args):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1481 """creates and installs Maven dependencies required by the JMH benchmarks
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1482
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1483 The dependencies are specified by files named pom.mxdeps in the
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1484 JMH directory tree. Each such file contains a list of dependencies
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1485 defined in JSON format. For example:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1486
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1487 '[{"artifactId" : "compiler.test", "groupId" : "com.oracle.graal", "deps" : ["com.oracle.graal.compiler.test"]}]'
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1488
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1489 will result in a dependency being installed in the local Maven repository
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1490 that can be referenced in a pom.xml file as follows:
15178
0c53453c4d5e mx: improved jmh command to (a) look for JMH benchmarks in ../java-benchmarks, (b) omit building benchmarks if they are up to date and (c) offer better message with 'mx help jmh'
Doug Simon <doug.simon@oracle.com>
parents: 15114
diff changeset
1491
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1492 <dependency>
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1493 <groupId>com.oracle.graal</groupId>
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1494 <artifactId>compiler.test</artifactId>
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1495 <version>1.0-SNAPSHOT</version>
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1496 </dependency>"""
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1497
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1498 parser = ArgumentParser(prog='mx makejmhdeps')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1499 parser.add_argument('-s', '--settings', help='alternative path for Maven user settings file', metavar='<path>')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1500 parser.add_argument('-p', '--permissive', action='store_true', help='issue note instead of error if a Maven dependency cannot be built due to missing projects/libraries')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1501 args = parser.parse_args(args)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1502
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1503 def makejmhdep(artifactId, groupId, deps):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1504 graalSuite = mx.suite("graal")
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1505 path = artifactId + '.jar'
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1506 if args.permissive:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1507 for name in deps:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1508 if not mx.project(name, fatalIfMissing=False):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1509 if not mx.library(name, fatalIfMissing=False):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1510 mx.log('Skipping ' + groupId + '.' + artifactId + '.jar as ' + name + ' cannot be resolved')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1511 return
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1512 d = mx.Distribution(graalSuite, name=artifactId, path=path, sourcesPath=path, deps=deps, excludedLibs=[])
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1513 d.make_archive()
15253
9ff9f6643986 mx: unconditionally ensure mx injected dependencies are up to date in buildjmh command
Doug Simon <doug.simon@oracle.com>
parents: 15250
diff changeset
1514 cmd = ['mvn', 'install:install-file', '-DgroupId=' + groupId, '-DartifactId=' + artifactId,
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1515 '-Dversion=1.0-SNAPSHOT', '-Dpackaging=jar', '-Dfile=' + d.path]
15253
9ff9f6643986 mx: unconditionally ensure mx injected dependencies are up to date in buildjmh command
Doug Simon <doug.simon@oracle.com>
parents: 15250
diff changeset
1516 if not mx._opts.verbose:
9ff9f6643986 mx: unconditionally ensure mx injected dependencies are up to date in buildjmh command
Doug Simon <doug.simon@oracle.com>
parents: 15250
diff changeset
1517 cmd.append('-q')
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1518 if args.settings:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1519 cmd = cmd + ['-s', args.settings]
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1520 mx.run(cmd)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1521 os.unlink(d.path)
15178
0c53453c4d5e mx: improved jmh command to (a) look for JMH benchmarks in ../java-benchmarks, (b) omit building benchmarks if they are up to date and (c) offer better message with 'mx help jmh'
Doug Simon <doug.simon@oracle.com>
parents: 15114
diff changeset
1522
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1523 jmhPath = _get_jmh_path()
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1524 for root, _, filenames in os.walk(jmhPath):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1525 for f in [join(root, n) for n in filenames if n == 'pom.mxdeps']:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1526 mx.logv('[processing ' + f + ']')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1527 try:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1528 with open(f) as fp:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1529 for d in json.load(fp):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1530 artifactId = d['artifactId']
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1531 groupId = d['groupId']
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1532 deps = d['deps']
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1533 makejmhdep(artifactId, groupId, deps)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1534 except ValueError as e:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1535 mx.abort('Error parsing {}:\n{}'.format(f, e))
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1536
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1537 def buildjmh(args):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1538 """build the JMH benchmarks"""
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1539
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1540 parser = ArgumentParser(prog='mx buildjmh')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1541 parser.add_argument('-s', '--settings', help='alternative path for Maven user settings file', metavar='<path>')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1542 parser.add_argument('-c', action='store_true', dest='clean', help='clean before building')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1543 args = parser.parse_args(args)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1544
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1545 jmhPath = _get_jmh_path()
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1546 mx.log('JMH benchmarks: ' + jmhPath)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1547
15253
9ff9f6643986 mx: unconditionally ensure mx injected dependencies are up to date in buildjmh command
Doug Simon <doug.simon@oracle.com>
parents: 15250
diff changeset
1548 # Ensure the mx injected dependencies are up to date
9ff9f6643986 mx: unconditionally ensure mx injected dependencies are up to date in buildjmh command
Doug Simon <doug.simon@oracle.com>
parents: 15250
diff changeset
1549 makejmhdeps(['-p'] + (['-s', args.settings] if args.settings else []))
9ff9f6643986 mx: unconditionally ensure mx injected dependencies are up to date in buildjmh command
Doug Simon <doug.simon@oracle.com>
parents: 15250
diff changeset
1550
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1551 timestamp = mx.TimeStampFile(join(_graal_home, 'mx', 'jmh', jmhPath.replace(os.sep, '_') + '.timestamp'))
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1552 mustBuild = args.clean
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1553 if not mustBuild:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1554 try:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1555 hgfiles = [join(jmhPath, f) for f in subprocess.check_output(['hg', '-R', jmhPath, 'locate']).split('\n')]
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1556 mustBuild = timestamp.isOlderThan(hgfiles)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1557 except:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1558 # not a Mercurial repository or hg commands are not available.
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1559 mustBuild = True
15178
0c53453c4d5e mx: improved jmh command to (a) look for JMH benchmarks in ../java-benchmarks, (b) omit building benchmarks if they are up to date and (c) offer better message with 'mx help jmh'
Doug Simon <doug.simon@oracle.com>
parents: 15114
diff changeset
1560
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1561 if mustBuild:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1562 buildOutput = []
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1563 def _redirect(x):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1564 if mx._opts.verbose:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1565 mx.log(x[:-1])
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1566 else:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1567 buildOutput.append(x)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1568 env = os.environ.copy()
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1569 env['JAVA_HOME'] = _jdk(vmToCheck='server')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1570 env['MAVEN_OPTS'] = '-server'
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1571 mx.log("Building benchmarks...")
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1572 cmd = ['mvn']
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1573 if args.settings:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1574 cmd = cmd + ['-s', args.settings]
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1575 if args.clean:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1576 cmd.append('clean')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1577 cmd.append('package')
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1578 retcode = mx.run(cmd, cwd=jmhPath, out=_redirect, env=env, nonZeroIsFatal=False)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1579 if retcode != 0:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1580 mx.log(''.join(buildOutput))
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1581 mx.abort(retcode)
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1582 timestamp.touch()
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1583 else:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1584 mx.logv('[all Mercurial controlled files in ' + jmhPath + ' are older than ' + timestamp.path + ' - skipping build]')
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1585
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1586 def jmh(args):
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1587 """run the JMH benchmarks
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1588
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1589 This command respects the standard --vm and --vmbuild options
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1590 for choosing which VM to run the benchmarks with."""
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1591 if '-h' in args:
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1592 mx.help_(['jmh'])
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1593 mx.abort(1)
15021
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1594
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1595 vmArgs, benchmarksAndJsons = _extract_VM_args(args)
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1596
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1597 benchmarks = [b for b in benchmarksAndJsons if not b.startswith('{')]
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1598 jmhArgJsons = [b for b in benchmarksAndJsons if b.startswith('{')]
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1599
15178
0c53453c4d5e mx: improved jmh command to (a) look for JMH benchmarks in ../java-benchmarks, (b) omit building benchmarks if they are up to date and (c) offer better message with 'mx help jmh'
Doug Simon <doug.simon@oracle.com>
parents: 15114
diff changeset
1600 jmhArgs = {'-rff' : join(_graal_home, 'mx', 'jmh', 'jmh.out'), '-v' : 'EXTRA' if mx._opts.verbose else 'NORMAL'}
15021
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1601
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1602 # e.g. '{"-wi" : 20}'
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1603 for j in jmhArgJsons:
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1604 try:
15051
2df054b37edc add support for removing default JMH args (e.g., '{"-wi" : null, "-i" : null}')
Doug Simon <doug.simon@oracle.com>
parents: 15048
diff changeset
1605 for n, v in json.loads(j).iteritems():
2df054b37edc add support for removing default JMH args (e.g., '{"-wi" : null, "-i" : null}')
Doug Simon <doug.simon@oracle.com>
parents: 15048
diff changeset
1606 if v is None:
2df054b37edc add support for removing default JMH args (e.g., '{"-wi" : null, "-i" : null}')
Doug Simon <doug.simon@oracle.com>
parents: 15048
diff changeset
1607 del jmhArgs[n]
2df054b37edc add support for removing default JMH args (e.g., '{"-wi" : null, "-i" : null}')
Doug Simon <doug.simon@oracle.com>
parents: 15048
diff changeset
1608 else:
2df054b37edc add support for removing default JMH args (e.g., '{"-wi" : null, "-i" : null}')
Doug Simon <doug.simon@oracle.com>
parents: 15048
diff changeset
1609 jmhArgs[n] = v
15021
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1610 except ValueError as e:
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1611 mx.abort('error parsing JSON input: {}\n{}'.format(j, e))
14965
c64cd1dd4bd1 moved Maven installation of Graal dependency from build to jmh command
Doug Simon <doug.simon@oracle.com>
parents: 14938
diff changeset
1612
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1613 jmhPath = _get_jmh_path()
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1614 mx.log('Using benchmarks in ' + jmhPath)
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1615
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1616 matchedSuites = set()
13949
f694daada5bf mx jmh: display the number of benchmarks that will run
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 13948
diff changeset
1617 numBench = [0]
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1618 for micros in os.listdir(jmhPath):
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1619 absoluteMicro = os.path.join(jmhPath, micros)
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1620 if not os.path.isdir(absoluteMicro):
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1621 continue
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1622 if not micros.startswith("micros-"):
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1623 mx.logv('JMH: ignored ' + absoluteMicro + " because it doesn't start with 'micros-'")
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1624 continue
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1625
13948
69928d77bc0a mx jmh: avoid mx crash if JMH_BENCHMARKS is not defined at all and skip suites that do not contain the correct jar
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 13935
diff changeset
1626 microJar = os.path.join(absoluteMicro, "target", "microbenchmarks.jar")
69928d77bc0a mx jmh: avoid mx crash if JMH_BENCHMARKS is not defined at all and skip suites that do not contain the correct jar
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 13935
diff changeset
1627 if not exists(microJar):
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1628 mx.abort('Missing ' + microJar + ' - please run "mx buildjmh"')
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1629 if benchmarks:
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1630 def _addBenchmark(x):
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1631 if x.startswith("Benchmark:"):
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1632 return
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1633 match = False
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1634 for b in benchmarks:
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1635 match = match or (b in x)
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1636
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1637 if match:
13949
f694daada5bf mx jmh: display the number of benchmarks that will run
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 13948
diff changeset
1638 numBench[0] += 1
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1639 matchedSuites.add(micros)
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1640
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
1641 mx.run_java(['-jar', microJar, "-l"], cwd=jmhPath, out=_addBenchmark, addDefaultArgs=False)
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1642 else:
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1643 matchedSuites.add(micros)
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1644
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1645 mx.logv("matchedSuites: " + str(matchedSuites))
13949
f694daada5bf mx jmh: display the number of benchmarks that will run
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 13948
diff changeset
1646 plural = 's' if not benchmarks or numBench[0] > 1 else ''
f694daada5bf mx jmh: display the number of benchmarks that will run
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 13948
diff changeset
1647 number = str(numBench[0]) if benchmarks else "all"
f694daada5bf mx jmh: display the number of benchmarks that will run
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 13948
diff changeset
1648 mx.log("Running " + number + " benchmark" + plural + '...')
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1649
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1650 regex = []
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1651 if benchmarks:
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1652 regex.append(r".*(" + "|".join(benchmarks) + ").*")
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1653
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1654 for suite in matchedSuites:
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1655 absoluteMicro = os.path.join(jmhPath, suite)
13951
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
1656 (pfx, exe, vm, forkedVmArgs, _) = _parseVmArgs(vmArgs)
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
1657 if pfx:
392b6ac8da36 Allow using run_java without the arguments from -J -Ja -Jp. Factor out the argument processing of mx_graal.vm and use it to pass tested-vm args down through the jmh harness
Bernhard Urban <bernhard.urban@jku.at>
parents: 13949
diff changeset
1658 mx.warn("JMH ignores prefix: \"" + pfx + "\"")
15021
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1659 javaArgs = ['-jar', os.path.join(absoluteMicro, "target", "microbenchmarks.jar"),
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1660 '--jvm', exe,
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1661 '--jvmArgs', ' '.join(["-" + vm] + forkedVmArgs)]
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1662 for k, v in jmhArgs.iteritems():
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1663 javaArgs.append(k)
15051
2df054b37edc add support for removing default JMH args (e.g., '{"-wi" : null, "-i" : null}')
Doug Simon <doug.simon@oracle.com>
parents: 15048
diff changeset
1664 if len(str(v)):
2df054b37edc add support for removing default JMH args (e.g., '{"-wi" : null, "-i" : null}')
Doug Simon <doug.simon@oracle.com>
parents: 15048
diff changeset
1665 javaArgs.append(str(v))
15021
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
1666 mx.run_java(javaArgs + regex, addDefaultArgs=False, cwd=jmhPath)
13935
ca0e1af320f6 mx: add support for jmh benchmark suites
Bernhard Urban <bernhard.urban@jku.at>
parents: 13928
diff changeset
1667
4228
e872562f95f8 add a simple specjvm2008 command, fail a benchmark if one of the failureRE matches
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4227
diff changeset
1668 def specjvm2008(args):
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1669 """run one or more SPECjvm2008 benchmarks"""
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1670
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1671 def launcher(bm, harnessArgs, extraVmOpts):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1672 return sanitycheck.getSPECjvm2008(harnessArgs + [bm]).bench(_get_vm(), extraVmOpts=extraVmOpts)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1673
11372
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1674 availableBenchmarks = set(sanitycheck.specjvm2008Names)
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1675 for name in sanitycheck.specjvm2008Names:
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1676 parts = name.rsplit('.', 1)
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1677 if len(parts) > 1:
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1678 assert len(parts) == 2
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1679 group = parts[0]
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1680 availableBenchmarks.add(group)
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1681
76c45fb64191 added support to the specjvm2008 command for specifying a benchmark group
Doug Simon <doug.simon@oracle.com>
parents: 11371
diff changeset
1682 _run_benchmark(args, sorted(availableBenchmarks), launcher)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1683
8147
9786ac8fff61 Use different vm args for SPECjbb2013
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 8135
diff changeset
1684 def specjbb2013(args):
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1685 """runs the composite SPECjbb2013 benchmark"""
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1686
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1687 def launcher(bm, harnessArgs, extraVmOpts):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1688 assert bm is None
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1689 return sanitycheck.getSPECjbb2013(harnessArgs).bench(_get_vm(), extraVmOpts=extraVmOpts)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1690
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1691 _run_benchmark(args, None, launcher)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1692
8506
c3657d00e343 -Merge with tip
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8406
diff changeset
1693 def specjbb2005(args):
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1694 """runs the composite SPECjbb2005 benchmark"""
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1695
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1696 def launcher(bm, harnessArgs, extraVmOpts):
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1697 assert bm is None
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1698 return sanitycheck.getSPECjbb2005(harnessArgs).bench(_get_vm(), extraVmOpts=extraVmOpts)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1699
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
1700 _run_benchmark(args, None, launcher)
8506
c3657d00e343 -Merge with tip
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8406
diff changeset
1701
5596
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
1702 def hsdis(args, copyToDir=None):
6291
633136426f26 consolidated jtt command into unittest command and removed need for explicit 'testHarness' property in 'projects' file
Doug Simon <doug.simon@oracle.com>
parents: 6278
diff changeset
1703 """download the hsdis library
4482
7903b6c28f9c Add hsdis command to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4480
diff changeset
1704
7903b6c28f9c Add hsdis command to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4480
diff changeset
1705 This is needed to support HotSpot's assembly dumping features.
5596
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
1706 By default it downloads the Intel syntax version, use the 'att' argument to install AT&T syntax."""
4482
7903b6c28f9c Add hsdis command to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4480
diff changeset
1707 flavor = 'intel'
7903b6c28f9c Add hsdis command to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4480
diff changeset
1708 if 'att' in args:
7903b6c28f9c Add hsdis command to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4480
diff changeset
1709 flavor = 'att'
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
1710 lib = mx.add_lib_suffix('hsdis-' + _arch())
5596
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
1711 path = join(_graal_home, 'lib', lib)
5189
6fbf12b8e572 automated download of hsdis lib if PrintAssembly option is used; added support for hsdis on the Mac
Doug Simon <doug.simon@oracle.com>
parents: 5153
diff changeset
1712 if not exists(path):
6fbf12b8e572 automated download of hsdis lib if PrintAssembly option is used; added support for hsdis on the Mac
Doug Simon <doug.simon@oracle.com>
parents: 5153
diff changeset
1713 mx.download(path, ['http://lafo.ssw.uni-linz.ac.at/hsdis/' + flavor + "/" + lib])
5596
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
1714 if copyToDir is not None and exists(copyToDir):
e4b1bc5e29e0 hsdis library is downloaded into newly created Graal JDK
Doug Simon <doug.simon@oracle.com>
parents: 5517
diff changeset
1715 shutil.copy(path, copyToDir)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1716
5245
71ac4221b1b0 added hcfdis command
Doug Simon <doug.simon@oracle.com>
parents: 5244
diff changeset
1717 def hcfdis(args):
6291
633136426f26 consolidated jtt command into unittest command and removed need for explicit 'testHarness' property in 'projects' file
Doug Simon <doug.simon@oracle.com>
parents: 6278
diff changeset
1718 """disassemble HexCodeFiles embedded in text files
5245
71ac4221b1b0 added hcfdis command
Doug Simon <doug.simon@oracle.com>
parents: 5244
diff changeset
1719
71ac4221b1b0 added hcfdis command
Doug Simon <doug.simon@oracle.com>
parents: 5244
diff changeset
1720 Run a tool over the input files to convert all embedded HexCodeFiles
71ac4221b1b0 added hcfdis command
Doug Simon <doug.simon@oracle.com>
parents: 5244
diff changeset
1721 to a disassembled format."""
6958
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1722
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1723 parser = ArgumentParser(prog='mx hcfdis')
6958
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1724 parser.add_argument('-m', '--map', help='address to symbol map applied to disassembler output')
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1725 parser.add_argument('files', nargs=REMAINDER, metavar='files...')
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1726
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1727 args = parser.parse_args(args)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1728
6278
4f9574b2893e removed dependence on private java.lang.String internals in HexCodeFile
Doug Simon <doug.simon@oracle.com>
parents: 6276
diff changeset
1729 path = join(_graal_home, 'lib', 'hcfdis-1.jar')
5245
71ac4221b1b0 added hcfdis command
Doug Simon <doug.simon@oracle.com>
parents: 5244
diff changeset
1730 if not exists(path):
6278
4f9574b2893e removed dependence on private java.lang.String internals in HexCodeFile
Doug Simon <doug.simon@oracle.com>
parents: 6276
diff changeset
1731 mx.download(path, ['http://lafo.ssw.uni-linz.ac.at/hcfdis-1.jar'])
6958
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1732 mx.run_java(['-jar', path] + args.files)
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1733
6958
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1734 if args.map is not None:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1735 addressRE = re.compile(r'0[xX]([A-Fa-f0-9]+)')
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1736 with open(args.map) as fp:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1737 lines = fp.read().splitlines()
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1738 symbols = dict()
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1739 for l in lines:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1740 addressAndSymbol = l.split(' ', 1)
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1741 if len(addressAndSymbol) == 2:
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1742 address, symbol = addressAndSymbol
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
1743 if address.startswith('0x'):
6958
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1744 address = long(address, 16)
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1745 symbols[address] = symbol
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1746 for f in args.files:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1747 with open(f) as fp:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1748 lines = fp.read().splitlines()
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1749 updated = False
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1750 for i in range(0, len(lines)):
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1751 l = lines[i]
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1752 for m in addressRE.finditer(l):
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1753 sval = m.group(0)
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1754 val = long(sval, 16)
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1755 sym = symbols.get(val)
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1756 if sym:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1757 l = l.replace(sval, sym)
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1758 updated = True
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1759 lines[i] = l
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1760 if updated:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1761 mx.log('updating ' + f)
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1762 with open('new_' + f, "w") as fp:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1763 for l in lines:
abbe4faaf0c1 added support for dumping an address to symbol map (with -G:+PrintAddressMap) to translate addresses in disassembled output (via new '-m' option to the hcfdis command)
Doug Simon <doug.simon@oracle.com>
parents: 6718
diff changeset
1764 print >> fp, l
5245
71ac4221b1b0 added hcfdis command
Doug Simon <doug.simon@oracle.com>
parents: 5244
diff changeset
1765
4684
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1766 def jacocoreport(args):
6291
633136426f26 consolidated jtt command into unittest command and removed need for explicit 'testHarness' property in 'projects' file
Doug Simon <doug.simon@oracle.com>
parents: 6278
diff changeset
1767 """create a JaCoCo coverage report
4684
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1768
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1769 Creates the report from the 'jacoco.exec' file in the current directory.
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1770 Default output directory is 'coverage', but an alternative can be provided as an argument."""
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1771 jacocoreport = mx.library("JACOCOREPORT", True)
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1772 out = 'coverage'
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1773 if len(args) == 1:
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1774 out = args[0]
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1775 elif len(args) > 1:
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1776 mx.abort('jacocoreport takes only one argument : an output directory')
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
1777 mx.run_java(['-jar', jacocoreport.get_path(True), '-in', 'jacoco.exec', '-g', join(_graal_home, 'graal'), out])
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1778
11867
952ecf32788a mx sl command.
Chris Seaton <chris.seaton@oracle.com>
parents: 11797
diff changeset
1779 def sl(args):
12693
c6b833f7935e mx sl: change vm args syntax
Andreas Woess <andreas.woess@jku.at>
parents: 12647
diff changeset
1780 """run an SL program"""
c6b833f7935e mx sl: change vm args syntax
Andreas Woess <andreas.woess@jku.at>
parents: 12647
diff changeset
1781 vmArgs, slArgs = _extract_VM_args(args)
13761
7c418666c6c9 Refactoring and cleanup of Simple Language (more to come soon)
Christian Wimmer <christian.wimmer@oracle.com>
parents: 13695
diff changeset
1782 vm(vmArgs + ['-cp', mx.classpath("com.oracle.truffle.sl"), "com.oracle.truffle.sl.SLMain"] + slArgs)
11867
952ecf32788a mx sl command.
Chris Seaton <chris.seaton@oracle.com>
parents: 11797
diff changeset
1783
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
1784 def trufflejar(args=None):
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
1785 """make truffle.jar"""
13347
e3b0608d6ab8 fixed pylint warnings
Doug Simon <doug.simon@oracle.com>
parents: 13332
diff changeset
1786
13320
093353894575 Test the Truffle JAR after building.
Chris Seaton <chris.seaton@oracle.com>
parents: 13319
diff changeset
1787 # Test with the built classes
15322
404d29e8cee6 mx: remove longunittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15310
diff changeset
1788 _unittest(["com.oracle.truffle.api.test", "com.oracle.truffle.api.dsl.test"], ['@Test', '@Parameters'])
13347
e3b0608d6ab8 fixed pylint warnings
Doug Simon <doug.simon@oracle.com>
parents: 13332
diff changeset
1789
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
1790 # We use the DSL processor as the starting point for the classpath - this
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
1791 # therefore includes the DSL processor, the DSL and the API.
13319
30e57b49fdb1 Include annotation processor meta-info in Truffle JAR.
Chris Seaton <chris.seaton@oracle.com>
parents: 13303
diff changeset
1792 packagejar(mx.classpath("com.oracle.truffle.dsl.processor").split(os.pathsep), "truffle.jar", None, "com.oracle.truffle.dsl.processor.TruffleProcessor")
13347
e3b0608d6ab8 fixed pylint warnings
Doug Simon <doug.simon@oracle.com>
parents: 13332
diff changeset
1793
13320
093353894575 Test the Truffle JAR after building.
Chris Seaton <chris.seaton@oracle.com>
parents: 13319
diff changeset
1794 # Test with the JAR
15322
404d29e8cee6 mx: remove longunittest
Bernhard Urban <bernhard.urban@jku.at>
parents: 15310
diff changeset
1795 _unittest(["com.oracle.truffle.api.test", "com.oracle.truffle.api.dsl.test"], ['@Test', '@Parameters'], "truffle.jar:")
13320
093353894575 Test the Truffle JAR after building.
Chris Seaton <chris.seaton@oracle.com>
parents: 13319
diff changeset
1796
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
1797
9803
8e33b4ebfef1 add isGraalEnabled(vm) function in commands.py
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9667
diff changeset
1798 def isGraalEnabled(vm):
8e33b4ebfef1 add isGraalEnabled(vm) function in commands.py
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9667
diff changeset
1799 return vm != 'original' and not vm.endswith('nograal')
8e33b4ebfef1 add isGraalEnabled(vm) function in commands.py
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 9667
diff changeset
1800
5700
12a34d1bcaa2 added site command to generate a javadoc-based website
Doug Simon <doug.simon@oracle.com>
parents: 5688
diff changeset
1801 def site(args):
6291
633136426f26 consolidated jtt command into unittest command and removed need for explicit 'testHarness' property in 'projects' file
Doug Simon <doug.simon@oracle.com>
parents: 6278
diff changeset
1802 """create a website containing javadoc and the project dependency graph"""
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1803
5771
613a3ddb9a71 moved core site functionality into mx.py
Doug Simon <doug.simon@oracle.com>
parents: 5764
diff changeset
1804 return mx.site(['--name', 'Graal',
5782
488864d5069a added support for passing -tag option to javadoc to suppress warnings about unrecognized tags in javadoc comments
Doug Simon <doug.simon@oracle.com>
parents: 5771
diff changeset
1805 '--jd', '@-tag', '--jd', '@test:X',
488864d5069a added support for passing -tag option to javadoc to suppress warnings about unrecognized tags in javadoc comments
Doug Simon <doug.simon@oracle.com>
parents: 5771
diff changeset
1806 '--jd', '@-tag', '--jd', '@run:X',
488864d5069a added support for passing -tag option to javadoc to suppress warnings about unrecognized tags in javadoc comments
Doug Simon <doug.simon@oracle.com>
parents: 5771
diff changeset
1807 '--jd', '@-tag', '--jd', '@bug:X',
488864d5069a added support for passing -tag option to javadoc to suppress warnings about unrecognized tags in javadoc comments
Doug Simon <doug.simon@oracle.com>
parents: 5771
diff changeset
1808 '--jd', '@-tag', '--jd', '@summary:X',
488864d5069a added support for passing -tag option to javadoc to suppress warnings about unrecognized tags in javadoc comments
Doug Simon <doug.simon@oracle.com>
parents: 5771
diff changeset
1809 '--jd', '@-tag', '--jd', '@vmoption:X',
5771
613a3ddb9a71 moved core site functionality into mx.py
Doug Simon <doug.simon@oracle.com>
parents: 5764
diff changeset
1810 '--overview', join(_graal_home, 'graal', 'overview.html'),
613a3ddb9a71 moved core site functionality into mx.py
Doug Simon <doug.simon@oracle.com>
parents: 5764
diff changeset
1811 '--title', 'Graal OpenJDK Project Documentation',
6506
305b9166b455 added projects table to the top level javadoc page generated by the site command
Doug Simon <doug.simon@oracle.com>
parents: 6495
diff changeset
1812 '--dot-output-base', 'projects'] + args)
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
1813
13507
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1814 def generateZshCompletion(args):
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1815 """generate zsh completion for mx"""
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1816 try:
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1817 from genzshcomp import CompletionGenerator
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1818 except ImportError:
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1819 mx.abort("install genzshcomp (pip install genzshcomp)")
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1820
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1821 # need to fake module for the custom mx arg parser, otherwise a check in genzshcomp fails
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1822 originalModule = mx._argParser.__module__
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1823 mx._argParser.__module__ = "argparse"
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1824 generator = CompletionGenerator("mx", mx._argParser)
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1825 mx._argParser.__module__ = originalModule
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1826
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1827 # strip last line and define local variable "ret"
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1828 complt = "\n".join(generator.get().split('\n')[0:-1]).replace('context state line', 'context state line ret=1')
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1829
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1830 # add array of possible subcommands (as they are not part of the argument parser)
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1831 complt += '\n ": :->command" \\\n'
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1832 complt += ' "*::args:->args" && ret=0\n'
13507
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1833 complt += '\n'
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1834 complt += 'case $state in\n'
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1835 complt += '\t(command)\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1836 complt += '\t\tlocal -a main_commands\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1837 complt += '\t\tmain_commands=(\n'
13507
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1838 for cmd in sorted(mx._commands.iterkeys()):
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1839 c, _ = mx._commands[cmd][:2]
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1840 doc = c.__doc__
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1841 complt += '\t\t\t"{0}'.format(cmd)
13507
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1842 if doc:
13509
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1843 complt += ':{0}'.format(_fixQuotes(doc.split('\n', 1)[0]))
13507
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1844 complt += '"\n'
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1845 complt += '\t\t)\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1846 complt += '\t\t_describe -t main_commands command main_commands && ret=0\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1847 complt += '\t\t;;\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1848
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1849 complt += '\t(args)\n'
13509
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1850 # TODO: improve matcher: if mx args are given, this doesn't work
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1851 complt += '\t\tcase $line[1] in\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1852 complt += '\t\t\t(vm)\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1853 complt += '\t\t\t\tnoglob \\\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1854 complt += '\t\t\t\t\t_arguments -s -S \\\n'
13509
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1855 complt += _appendOptions("graal", r"G\:")
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1856 # TODO: fix -XX:{-,+}Use* flags
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1857 complt += _appendOptions("hotspot", r"XX\:")
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1858 complt += '\t\t\t\t\t"-version" && ret=0 \n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1859 complt += '\t\t\t\t;;\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1860 complt += '\t\tesac\n'
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1861 complt += '\t\t;;\n'
13507
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1862 complt += 'esac\n'
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1863 complt += '\n'
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1864 complt += 'return $ret'
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1865 print complt
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1866
13509
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1867 def _fixQuotes(arg):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1868 return arg.replace('\"', '').replace('\'', '').replace('`', '').replace('{', '\\{').replace('}', '\\}').replace('[', '\\[').replace(']', '\\]')
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1869
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1870 def _appendOptions(optionType, optionPrefix):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1871 def isBoolean(vmap, field):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1872 return vmap[field] == "Boolean" or vmap[field] == "bool"
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1873
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1874 def hasDescription(vmap):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1875 return vmap['optDefault'] or vmap['optDoc']
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1876
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1877 complt = ""
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1878 for vmap in _parseVMOptions(optionType):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1879 complt += '\t\t\t\t\t-"'
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1880 complt += optionPrefix
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1881 if isBoolean(vmap, 'optType'):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1882 complt += '"{-,+}"'
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1883 complt += vmap['optName']
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1884 if not isBoolean(vmap, 'optType'):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1885 complt += '='
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1886 if hasDescription(vmap):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1887 complt += "["
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1888 if vmap['optDefault']:
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1889 complt += r"(default\: " + vmap['optDefault'] + ")"
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1890 if vmap['optDoc']:
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1891 complt += _fixQuotes(vmap['optDoc'])
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1892 if hasDescription(vmap):
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1893 complt += "]"
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1894 complt += '" \\\n'
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1895 return complt
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1896
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1897 def _parseVMOptions(optionType):
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1898 parser = OutputParser()
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1899 # TODO: the optDoc part can wrapped accross multiple lines, currently only the first line will be captured
13509
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1900 # TODO: fix matching for float literals
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1901 jvmOptions = re.compile(
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1902 r"^[ \t]*"
13509
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1903 r"(?P<optType>(Boolean|Integer|Float|Double|String|bool|intx|uintx|ccstr|double)) "
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1904 r"(?P<optName>[a-zA-Z0-9]+)"
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1905 r"[ \t]+=[ \t]*"
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1906 r"(?P<optDefault>([\-0-9]+(\.[0-9]+(\.[0-9]+\.[0-9]+))?|false|true|null|Name|sun\.boot\.class\.path))?"
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1907 r"[ \t]*"
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1908 r"(?P<optDoc>.+)?",
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1909 re.MULTILINE)
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1910 parser.addMatcher(ValuesMatcher(jvmOptions, {
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1911 'optType' : '<optType>',
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1912 'optName' : '<optName>',
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1913 'optDefault' : '<optDefault>',
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1914 'optDoc' : '<optDoc>',
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1915 }))
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1916
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1917 # gather graal options
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1918 output = StringIO.StringIO()
13509
a8831418ff04 mx: add completion support for hotspot options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13508
diff changeset
1919 vm(['-XX:-BootstrapGraal', '-G:+PrintFlags' if optionType == "graal" else '-XX:+PrintFlagsWithComments'],
13695
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
1920 vm="graal",
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
1921 vmbuild="optimized",
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
1922 nonZeroIsFatal=False,
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
1923 out=output.write,
b688e7da4c69 refined class path used by unittest to only include dependencies of the tests that will be run
Doug Simon <doug.simon@oracle.com>
parents: 13526
diff changeset
1924 err=subprocess.STDOUT)
13508
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1925
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1926 valueMap = parser.parse(output.getvalue())
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1927 return valueMap
8a5b39d0bfb5 mx: add completion support for graal options (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13507
diff changeset
1928
14546
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1929 def findbugs(args):
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1930 '''run FindBugs against non-test Java projects'''
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1931 findBugsHome = mx.get_env('FINDBUGS_HOME', None)
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1932 if findBugsHome:
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1933 findbugsJar = join(findBugsHome, 'lib', 'findbugs.jar')
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1934 else:
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1935 findbugsLib = join(_graal_home, 'lib', 'findbugs-3.0.0')
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1936 if not exists(findbugsLib):
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1937 tmp = tempfile.mkdtemp(prefix='findbugs-download-tmp', dir=_graal_home)
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1938 try:
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1939 findbugsDist = join(tmp, 'findbugs.zip')
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1940 mx.download(findbugsDist, ['http://sourceforge.net/projects/findbugs/files/findbugs/3.0.0/findbugs-3.0.0-dev-20131204-e3cbbd5.zip'])
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1941 with zipfile.ZipFile(findbugsDist) as zf:
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1942 candidates = [e for e in zf.namelist() if e.endswith('/lib/findbugs.jar')]
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1943 assert len(candidates) == 1, candidates
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1944 libDirInZip = os.path.dirname(candidates[0])
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1945 zf.extractall(tmp)
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1946 shutil.copytree(join(tmp, libDirInZip), findbugsLib)
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1947 finally:
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1948 shutil.rmtree(tmp)
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1949 findbugsJar = join(findbugsLib, 'findbugs.jar')
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1950 assert exists(findbugsJar)
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1951 nonTestProjects = [p for p in mx.projects() if not p.name.endswith('.test') and not p.name.endswith('.jtt')]
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1952 outputDirs = [p.output_dir() for p in nonTestProjects]
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1953 findbugsResults = join(_graal_home, 'findbugs.results')
14769
141d570b397c fixed pylint errors
Doug Simon <doug.simon@oracle.com>
parents: 14767
diff changeset
1954
14777
10c6700908ef mx: removed use of exclusion filter files from findbugs command
Doug Simon <doug.simon@oracle.com>
parents: 14773
diff changeset
1955 cmd = ['-jar', findbugsJar, '-textui', '-low', '-maxRank', '15']
14767
ded08e344e4a findbugs will also use any findbugsExcludeFilter.xml it finds in a suite's root
Doug Simon <doug.simon@oracle.com>
parents: 14762
diff changeset
1956 if sys.stdout.isatty():
ded08e344e4a findbugs will also use any findbugsExcludeFilter.xml it finds in a suite's root
Doug Simon <doug.simon@oracle.com>
parents: 14762
diff changeset
1957 cmd.append('-progress')
14872
e6ce6d9f477f made findbugs quiet if running headless
Doug Simon <doug.simon@oracle.com>
parents: 14780
diff changeset
1958 cmd = cmd + ['-auxclasspath', mx.classpath([p.name for p in nonTestProjects]), '-output', findbugsResults, '-exitcode'] + args + outputDirs
14767
ded08e344e4a findbugs will also use any findbugsExcludeFilter.xml it finds in a suite's root
Doug Simon <doug.simon@oracle.com>
parents: 14762
diff changeset
1959 exitcode = mx.run_java(cmd, nonZeroIsFatal=False)
14546
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1960 if exitcode != 0:
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1961 with open(findbugsResults) as fp:
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1962 mx.log(fp.read())
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1963 os.unlink(findbugsResults)
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
1964 return exitcode
13507
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
1965
14772
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1966 def checkheaders(args):
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1967 """check Java source headers against any required pattern"""
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1968 failures = {}
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1969 for p in mx.projects():
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1970 if p.native:
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1971 continue
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1972
14918
31a9c79399c8 disabled current Checkstyle configurations (until Checkstyle supports Java8)
Doug Simon <doug.simon@oracle.com>
parents: 14905
diff changeset
1973 csConfig = join(mx.project(p.checkstyleProj).dir, '.checkstyle_checks.xml.disabled')
14772
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1974 dom = xml.dom.minidom.parse(csConfig)
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1975 for module in dom.getElementsByTagName('module'):
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1976 if module.getAttribute('name') == 'RegexpHeader':
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1977 for prop in module.getElementsByTagName('property'):
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1978 if prop.getAttribute('name') == 'header':
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1979 value = prop.getAttribute('value')
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1980 matcher = re.compile(value, re.MULTILINE)
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1981 for sourceDir in p.source_dirs():
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1982 for root, _, files in os.walk(sourceDir):
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1983 for name in files:
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1984 if name.endswith('.java') and name != 'package-info.java':
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1985 f = join(root, name)
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1986 with open(f) as fp:
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1987 content = fp.read()
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1988 if not matcher.match(content):
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1989 failures[f] = csConfig
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1990 for n, v in failures.iteritems():
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1991 mx.log('{}: header does not match RegexpHeader defined in {}'.format(n, v))
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1992 return len(failures)
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
1993
11605
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
1994 def mx_init(suite):
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
1995 commands = {
11371
74414b248381 mx: added --vmbuild global option, removed --product, --fastdebug, --debug options as well as suffix of build command
Doug Simon <doug.simon@oracle.com>
parents: 11368
diff changeset
1996 'build': [build, ''],
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
1997 'buildjmh': [buildjmh, '[-options]'],
9116
e2dae636732a added 'mx buildvars' command to list (some of) the variables that can be passed to 'mx build' with the -D option
Doug Simon <doug.simon@oracle.com>
parents: 9098
diff changeset
1998 'buildvars': [buildvars, ''],
4601
ade18666f2be Added --omit-native-build to gate command so that native cleaning and building can be omitted for changesets that made no changes to the C/C++ code.
Doug Simon <doug.simon@oracle.com>
parents: 4584
diff changeset
1999 'buildvms': [buildvms, '[-options]'],
14116
be7ebdf41bea mx: new command to start c1visualizer; support for IGV download when using a proxy server
Christian Wimmer <christian.wimmer@oracle.com>
parents: 14048
diff changeset
2000 'c1visualizer' : [c1visualizer, ''],
14772
c929a4a3b6c5 added checkheaders command for checking Java source headers without relying on Checkstyle
Doug Simon <doug.simon@oracle.com>
parents: 14769
diff changeset
2001 'checkheaders': [checkheaders, ''],
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
2002 'clean': [clean, ''],
14546
942c4daa9db9 added findbugs command to mx
Doug Simon <doug.simon@oracle.com>
parents: 14164
diff changeset
2003 'findbugs': [findbugs, ''],
13507
d3ec6003856d mx: experimental autocompletion generation for zsh (GRAAL-297)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13486
diff changeset
2004 'generateZshCompletion' : [generateZshCompletion, ''],
4482
7903b6c28f9c Add hsdis command to mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4480
diff changeset
2005 'hsdis': [hsdis, '[att]'],
5245
71ac4221b1b0 added hcfdis command
Doug Simon <doug.simon@oracle.com>
parents: 5244
diff changeset
2006 'hcfdis': [hcfdis, ''],
5089
206e68bb66c6 Made the IdealGraphVisualizer runnable from mx with new 'igv' command.
Doug Simon <doug.simon@oracle.com>
parents: 5064
diff changeset
2007 'igv' : [igv, ''],
11605
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
2008 'jdkhome': [print_jdkhome, ''],
15021
9dcd5407a603 added support for overriding/complementing JMH arguments in jmh command with a JSON string
Doug Simon <doug.simon@oracle.com>
parents: 15014
diff changeset
2009 'jmh': [jmh, '[VM options] [filters|JMH-args-as-json...]'],
11367
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
2010 'dacapo': [dacapo, '[VM options] benchmarks...|"all" [DaCapo options]'],
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
2011 'scaladacapo': [scaladacapo, '[VM options] benchmarks...|"all" [Scala DaCapo options]'],
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
2012 'specjvm2008': [specjvm2008, '[VM options] benchmarks...|"all" [SPECjvm2008 options]'],
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
2013 'specjbb2013': [specjbb2013, '[VM options] [-- [SPECjbb2013 options]]'],
39b86b83ddeb normalized the command line interface for the dacapo, scaladacapo, specjvm2008, specjbb2005 and specjbb2013 commands
Doug Simon <doug.simon@oracle.com>
parents: 11359
diff changeset
2014 'specjbb2005': [specjbb2005, '[VM options] [-- [SPECjbb2005 options]]'],
4601
ade18666f2be Added --omit-native-build to gate command so that native cleaning and building can be omitted for changesets that made no changes to the C/C++ code.
Doug Simon <doug.simon@oracle.com>
parents: 4584
diff changeset
2015 'gate' : [gate, '[-options]'],
4559
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
2016 'bench' : [bench, '[-resultfile file] [all(default)|dacapo|specjvm2008|bootstrap]'],
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
2017 'unittest' : [unittest, '[unittest options] [--] [VM options] [filters...]', _unittestHelpSuffix],
15250
4c68a0eb69ca mx: refactored JMH benchmark building and dependency creation out of 'jmh' into 'buildjmh' and 'makejmhdeps' respectively
Doug Simon <doug.simon@oracle.com>
parents: 15229
diff changeset
2018 'makejmhdeps' : [makejmhdeps, ''],
15113
4da162518b39 mx: add unittest options.
Josef Eisl <josef.eisl@jku.at>
parents: 15057
diff changeset
2019 'shortunittest' : [shortunittest, '[unittest options] [--] [VM options] [filters...]', _unittestHelpSuffix],
4684
e136f2d0d002 Activate jtt for the gate
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4660
diff changeset
2020 'jacocoreport' : [jacocoreport, '[output directory]'],
5700
12a34d1bcaa2 added site command to generate a javadoc-based website
Doug Simon <doug.simon@oracle.com>
parents: 5688
diff changeset
2021 'site' : [site, '[-options]'],
5257
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
2022 'vm': [vm, '[-options] class [args...]'],
a549662f6c6f added 'vmg' and 'vmfg' commands to mx as convenient aliases for running the debug and fastdebug builds of the selected VM
Doug Simon <doug.simon@oracle.com>
parents: 5245
diff changeset
2023 'vmg': [vmg, '[-options] class [args...]'],
6340
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
2024 'vmfg': [vmfg, '[-options] class [args...]'],
74560fdffd51 Add a draft longtests command to mc's commands
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6319
diff changeset
2025 'deoptalot' : [deoptalot, '[n]'],
11867
952ecf32788a mx sl command.
Chris Seaton <chris.seaton@oracle.com>
parents: 11797
diff changeset
2026 'longtests' : [longtests, ''],
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2027 'sl' : [sl, '[SL args|@VM options]'],
14048
d2c84a0bf37a Remove Ruby implementation.
Chris Seaton <chris.seaton@oracle.com>
parents: 14028
diff changeset
2028 'trufflejar' : [trufflejar, '']
3718
42e655a6a6f3 Added mx configuration.
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
2029 }
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
2030
5064
8a88c903e381 Updated mx commands.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 5047
diff changeset
2031 mx.add_argument('--jacoco', help='instruments com.oracle.* classes using JaCoCo', default='off', choices=['off', 'on', 'append'])
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2032 mx.add_argument('--vmcwd', dest='vm_cwd', help='current directory will be changed to <path> before the VM is executed', default=None, metavar='<path>')
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2033 mx.add_argument('--installed-jdks', help='the base directory in which the JDKs cloned from $JAVA_HOME exist. ' +
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2034 'The VM selected by --vm and --vmbuild options is under this directory (i.e., ' +
11493
94779c895aad fixed documentation of --installed-jdks mx option
Doug Simon <doug.simon@oracle.com>
parents: 11424
diff changeset
2035 join('<path>', '<jdk-version>', '<vmbuild>', 'jre', 'lib', '<vm>', mx.add_lib_prefix(mx.add_lib_suffix('jvm'))) + ')', default=None, metavar='<path>')
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
2036
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
2037 if _vmSourcesAvailable:
11371
74414b248381 mx: added --vmbuild global option, removed --product, --fastdebug, --debug options as well as suffix of build command
Doug Simon <doug.simon@oracle.com>
parents: 11368
diff changeset
2038 mx.add_argument('--vm', action='store', dest='vm', choices=_vmChoices.keys(), help='the VM type to build/run')
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
2039 mx.add_argument('--vmbuild', action='store', dest='vmbuild', choices=_vmbuildChoices, help='the VM build to build/run (default: ' + _vmbuildChoices[0] + ')')
5194
a6eceb5efb0e added --ecl option to mx for saving VM execution(s) as Eclipse launch configurations
Doug Simon <doug.simon@oracle.com>
parents: 5189
diff changeset
2040 mx.add_argument('--ecl', action='store_true', dest='make_eclipse_launch', help='create launch configuration for running VM execution(s) in Eclipse')
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2041 mx.add_argument('--vmprefix', action='store', dest='vm_prefix', help='prefix for running the VM (e.g. "/usr/bin/gdb --args")', metavar='<prefix>')
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2042 mx.add_argument('--gdb', action='store_const', const='/usr/bin/gdb --args', dest='vm_prefix', help='alias for --vmprefix "/usr/bin/gdb --args"')
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
2043
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
2044 commands.update({
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
2045 'export': [export, '[-options] [zipfile]'],
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
2046 })
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
2047
11605
3676540f71cf Allow mxtool suites to be in separate repositories (CR-1367)
Mick Jordan <mick.jordan@oracle.com>
parents: 11534
diff changeset
2048 mx.update_commands(suite, commands)
4144
34527a24bcf8 Adopted mxtool changes.
Doug Simon <doug.simon@oracle.com>
parents: 3736
diff changeset
2049
11514
dc3c8df55905 added support for pylint and fixed errors/warnings it found
Doug Simon <doug.simon@oracle.com>
parents: 11512
diff changeset
2050 def mx_post_parse_cmd_line(opts): #
7916
9bff64f43299 Better java version numbers support in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 7788
diff changeset
2051 # TODO _minVersion check could probably be part of a Suite in mx?
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
2052 if mx.java().version < _minVersion:
7916
9bff64f43299 Better java version numbers support in mx
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 7788
diff changeset
2053 mx.abort('Requires Java version ' + str(_minVersion) + ' or greater, got version ' + str(mx.java().version))
5729
dab877fe7c31 prevented error during mx site from corrupting an existing site
Doug Simon <doug.simon@oracle.com>
parents: 5708
diff changeset
2054
13967
4cd7c6629841 mx_graal: fix pylint 1.1.0 warnings
Bernhard Urban <bernhard.urban@jku.at>
parents: 13952
diff changeset
2055 if _vmSourcesAvailable:
4559
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
2056 if hasattr(opts, 'vm') and opts.vm is not None:
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
2057 global _vm
723df37192d6 Make it possible again to build a real client libjvm, drop the UseGraal flag.
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4540
diff changeset
2058 _vm = opts.vm
4216
a13d61d3910e fix for when no vm build is explicitely selected
Gilles Duboscq <gilles.m.duboscq@gmail.com>
parents: 4215
diff changeset
2059 if hasattr(opts, 'vmbuild') and opts.vmbuild is not None:
4178
d1b26c17910a Add the Dacapo benchmarks to the "lib" folder instead of using the environment variable. Fixed an issue in the downloader.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 4177
diff changeset
2060 global _vmbuild
4156
843c8d6720da Added 'export' command for creating a GraalVM zip file distribution without the Mercurial data or VM sources.
Doug Simon <doug.simon@oracle.com>
parents: 4155
diff changeset
2061 _vmbuild = opts.vmbuild
5194
a6eceb5efb0e added --ecl option to mx for saving VM execution(s) as Eclipse launch configurations
Doug Simon <doug.simon@oracle.com>
parents: 5189
diff changeset
2062 global _make_eclipse_launch
a6eceb5efb0e added --ecl option to mx for saving VM execution(s) as Eclipse launch configurations
Doug Simon <doug.simon@oracle.com>
parents: 5189
diff changeset
2063 _make_eclipse_launch = getattr(opts, 'make_eclipse_launch', False)
5024
7ccdae96e98a Allow JaCoCon to append to the execution file
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5016
diff changeset
2064 global _jacoco
7ccdae96e98a Allow JaCoCon to append to the execution file
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5016
diff changeset
2065 _jacoco = opts.jacoco
11401
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2066 global _vm_cwd
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2067 _vm_cwd = opts.vm_cwd
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2068 global _installed_jdks
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2069 _installed_jdks = opts.installed_jdks
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2070 global _vm_prefix
2d228d304247 minor improvements/cleanups to mx (CR-1313)
Doug Simon <doug.simon@oracle.com>
parents: 11395
diff changeset
2071 _vm_prefix = opts.vm_prefix
8134
6e3ebc6fd5a4 graal.jar is installed in all JDKs every time it is updated (GRAAL-136)
Doug Simon <doug.simon@oracle.com>
parents: 8132
diff changeset
2072
6e3ebc6fd5a4 graal.jar is installed in all JDKs every time it is updated (GRAAL-136)
Doug Simon <doug.simon@oracle.com>
parents: 8132
diff changeset
2073 mx.distribution('GRAAL').add_update_listener(_installGraalJarInJdks)
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2074
13332
ecea358f97be mx trufflejar: do not strip debug attributes
Andreas Woess <andreas.woess@jku.at>
parents: 13320
diff changeset
2075 def packagejar(classpath, outputFile, mainClass=None, annotationProcessor=None, stripDebug=False):
13347
e3b0608d6ab8 fixed pylint warnings
Doug Simon <doug.simon@oracle.com>
parents: 13332
diff changeset
2076 prefix = '' if mx.get_os() != 'windows' else '\\??\\' # long file name hack
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2077 print "creating", outputFile
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2078 filecount, totalsize = 0, 0
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2079 with zipfile.ZipFile(outputFile, 'w', zipfile.ZIP_DEFLATED) as zf:
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2080 manifest = "Manifest-Version: 1.0\n"
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2081 if mainClass != None:
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2082 manifest += "Main-Class: %s\n\n" % (mainClass)
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2083 zf.writestr("META-INF/MANIFEST.MF", manifest)
13319
30e57b49fdb1 Include annotation processor meta-info in Truffle JAR.
Chris Seaton <chris.seaton@oracle.com>
parents: 13303
diff changeset
2084 if annotationProcessor != None:
30e57b49fdb1 Include annotation processor meta-info in Truffle JAR.
Chris Seaton <chris.seaton@oracle.com>
parents: 13303
diff changeset
2085 zf.writestr("META-INF/services/javax.annotation.processing.Processor", annotationProcessor)
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2086 for cp in classpath:
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2087 print "+", cp
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2088 if cp.endswith(".jar"):
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2089 with zipfile.ZipFile(cp, 'r') as jar:
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2090 for arcname in jar.namelist():
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2091 if arcname.endswith('/') or arcname == 'META-INF/MANIFEST.MF' or arcname.endswith('.java') or arcname.lower().startswith("license") or arcname in [".project", ".classpath"]:
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2092 continue
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2093 zf.writestr(arcname, jar.read(arcname))
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2094 else:
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2095 for root, _, files in os.walk(cp):
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2096 for f in files:
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2097 fullname = os.path.join(root, f)
13347
e3b0608d6ab8 fixed pylint warnings
Doug Simon <doug.simon@oracle.com>
parents: 13332
diff changeset
2098 arcname = fullname[len(cp) + 1:].replace('\\', '/')
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2099 if f.endswith(".class"):
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2100 zf.write(prefix + fullname, arcname)
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2101
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2102 for zi in zf.infolist():
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2103 filecount += 1
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2104 totalsize += zi.file_size
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2105 print "%d files (total size: %.2f kB, jar size: %.2f kB)" % (filecount, totalsize / 1e3, os.path.getsize(outputFile) / 1e3)
13332
ecea358f97be mx trufflejar: do not strip debug attributes
Andreas Woess <andreas.woess@jku.at>
parents: 13320
diff changeset
2106 mx.run([mx.exe_suffix(join(mx.java().jdk, 'bin', 'pack200')), '-r'] + (['-G'] if stripDebug else []) + [outputFile])
13303
d8692e751c65 Added trufflejar command, and abstracted packagejar.
Chris Seaton <chris.seaton@oracle.com>
parents: 13267
diff changeset
2107 print "repacked jar size: %.2f kB" % (os.path.getsize(outputFile) / 1e3)