# HG changeset patch # User Doug Simon # Date 1387233220 -3600 # Node ID 0e5c4f9fa9a5a434868b94047287818ce305dabb # Parent cd22c6bb4a3576a83bcb9b933c26b55e2edf2b5f enabled non-hosted CompileTheWorld execution with complete bootstrapping and the ability to override compilation options separately for CTW compilations diff -r cd22c6bb4a35 -r 0e5c4f9fa9a5 graal/com.oracle.graal.hotspot.test/src/com/oracle/graal/hotspot/test/CompileTheWorldTest.java --- a/graal/com.oracle.graal.hotspot.test/src/com/oracle/graal/hotspot/test/CompileTheWorldTest.java Mon Dec 16 12:10:10 2013 -0800 +++ b/graal/com.oracle.graal.hotspot.test/src/com/oracle/graal/hotspot/test/CompileTheWorldTest.java Mon Dec 16 23:33:40 2013 +0100 @@ -39,7 +39,7 @@ boolean originalSetting = ExitVMOnException.getValue(); // Compile a couple classes in rt.jar String file = System.getProperty("java.home") + "/lib/rt.jar"; - new CompileTheWorld(file, 1, 5, false).compile(); + new CompileTheWorld(file, null, 1, 5, false).compile(); ExitVMOnException.setValue(originalSetting); } diff -r cd22c6bb4a35 -r 0e5c4f9fa9a5 graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/CompilationTask.java --- a/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/CompilationTask.java Mon Dec 16 12:10:10 2013 -0800 +++ b/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/CompilationTask.java Mon Dec 16 23:33:40 2013 +0100 @@ -47,7 +47,7 @@ import com.oracle.graal.phases.*; import com.oracle.graal.phases.tiers.*; -public final class CompilationTask implements Runnable { +public class CompilationTask implements Runnable { public static final ThreadLocal withinEnqueue = new ThreadLocal() { @@ -77,7 +77,7 @@ return new CompilationTask(backend, plan, optimisticOpts, profilingInfo, method, entryBCI, id); } - private CompilationTask(HotSpotBackend backend, PhasePlan plan, OptimisticOptimizations optimisticOpts, ProfilingInfo profilingInfo, HotSpotResolvedJavaMethod method, int entryBCI, int id) { + protected CompilationTask(HotSpotBackend backend, PhasePlan plan, OptimisticOptimizations optimisticOpts, ProfilingInfo profilingInfo, HotSpotResolvedJavaMethod method, int entryBCI, int id) { assert id >= 0; this.backend = backend; this.plan = plan; @@ -120,6 +120,10 @@ public static final DebugTimer CodeInstallationTime = Debug.timer("CodeInstallation"); + protected Suites getSuites(HotSpotProviders providers) { + return providers.getSuites().getDefaultSuites(); + } + public void runCompilation() { /* * no code must be outside this try/finally because it could happen otherwise that @@ -164,7 +168,7 @@ } InlinedBytecodes.add(method.getCodeSize()); CallingConvention cc = getCallingConvention(providers.getCodeCache(), Type.JavaCallee, graph.method(), false); - Suites suites = providers.getSuites().getDefaultSuites(); + Suites suites = getSuites(providers); result = compileGraph(graph, cc, method, providers, backend, backend.getTarget(), graphCache, plan, optimisticOpts, profilingInfo, method.getSpeculationLog(), suites, true, new CompilationResult(), CompilationResultBuilderFactory.Default); diff -r cd22c6bb4a35 -r 0e5c4f9fa9a5 graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/CompileTheWorld.java --- a/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/CompileTheWorld.java Mon Dec 16 12:10:10 2013 -0800 +++ b/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/CompileTheWorld.java Mon Dec 16 23:33:40 2013 +0100 @@ -22,7 +22,9 @@ */ package com.oracle.graal.hotspot; +import static com.oracle.graal.hotspot.CompileTheWorld.Options.*; import static com.oracle.graal.hotspot.HotSpotGraalRuntime.*; +import static com.oracle.graal.nodes.StructuredGraph.*; import static com.oracle.graal.phases.GraalOptions.*; import java.io.*; @@ -34,10 +36,14 @@ import com.oracle.graal.api.meta.*; import com.oracle.graal.bytecode.*; import com.oracle.graal.debug.*; +import com.oracle.graal.graph.*; +import com.oracle.graal.hotspot.HotSpotOptions.OptionConsumer; import com.oracle.graal.hotspot.bridge.*; import com.oracle.graal.hotspot.meta.*; -import com.oracle.graal.nodes.*; +import com.oracle.graal.options.*; +import com.oracle.graal.options.OptionValue.OverrideScope; import com.oracle.graal.phases.*; +import com.oracle.graal.phases.tiers.*; import com.oracle.graal.replacements.*; /** @@ -45,6 +51,83 @@ */ public final class CompileTheWorld { + static class Options { + // @formatter:off + @Option(help = "Compile all methods in all classes on given class path") + public static final OptionValue CompileTheWorldClasspath = new OptionValue<>(null); + @Option(help = "First class to consider when using -XX:+CompileTheWorld") + public static final OptionValue CompileTheWorldStartAt = new OptionValue<>(1); + @Option(help = "Last class to consider when using -XX:+CompileTheWorld") + public static final OptionValue CompileTheWorldStopAt = new OptionValue<>(Integer.MAX_VALUE); + @Option(help = "Option value overrides to use during compile the world. For example, " + + "to disable inlining and partial escape analysis specify '-PartialEscapeAnalysis -Inline'. " + + "The format for each option is the same as on the command line just without the '-G:' prefix.") + public static final OptionValue CompileTheWorldConfig = new OptionValue<>(null); + // @formatter:on + } + + /** + * A mechanism for overriding Graal options that effect compilation. A {@link Config} object + * should be used in a try-with-resources statement to ensure overriding of options is scoped + * properly. For example: + * + * 
+     *     Config config = ...;
+     *     try (AutoCloseable s = config == null ? null : config.apply()) {
+     *         // perform a Graal compilation
+     *     }
+     *
+ */ + @SuppressWarnings("serial") + static class Config extends HashMap, Object> implements AutoCloseable, OptionConsumer { + OverrideScope scope; + + /** + * Creates a {@link Config} object by parsing a set of space separated override options. + * + * @param options a space separated set of option value settings with each option setting in + * a format compatible with + * {@link HotSpotOptions#parseOption(String, OptionConsumer)} + */ + Config(String options) { + for (String option : options.split("\\s+")) { + if (!HotSpotOptions.parseOption(option, this)) { + throw new GraalInternalError("Invalid option specified: %s", option); + } + } + } + + /** + * Applies the overrides represented by this object. The overrides are in effect until + * {@link #close()} is called on this object. + */ + Config apply() { + assert scope == null; + scope = OptionValue.override(this); + return this; + } + + public void close() { + assert scope != null; + scope.close(); + + scope = null; + + } + + public void set(OptionDescriptor desc, Object value) { + put(desc.getOptionValue(), value); + } + } + + static Config parseConfig(String input) { + if (input == null) { + return null; + } else { + return new Config(input); + } + } + /** * This is our magic token to trigger reading files from the boot class path. */ @@ -54,13 +137,13 @@ private final HotSpotGraalRuntime runtime = runtime(); private final VMToCompilerImpl vmToCompiler = (VMToCompilerImpl) runtime.getVMToCompiler(); - /** List of Zip/Jar files to compile (see {@link GraalOptions#CompileTheWorld}. */ + /** List of Zip/Jar files to compile (see {@link #CompileTheWorldClasspath}. */ private final String files; - /** Class index to start compilation at (see {@link GraalOptions#CompileTheWorldStartAt}. */ + /** Class index to start compilation at (see {@link #CompileTheWorldStartAt}. */ private final int startAt; - /** Class index to stop compilation at (see {@link GraalOptions#CompileTheWorldStopAt}. */ + /** Class index to stop compilation at (see {@link #CompileTheWorldStopAt}. */ private final int stopAt; // Counters @@ -69,28 +152,30 @@ private long compileTime = 0; private boolean verbose; + private final Config config; /** - * Create a compile-the-world instance with default values from - * {@link GraalOptions#CompileTheWorld}, {@link GraalOptions#CompileTheWorldStartAt} and - * {@link GraalOptions#CompileTheWorldStopAt}. + * Creates a compile-the-world instance with default values from + * {@link Options#CompileTheWorldClasspath}, {@link Options#CompileTheWorldStartAt} and + * {@link Options#CompileTheWorldStopAt}. */ public CompileTheWorld() { - this(CompileTheWorld.getValue(), CompileTheWorldStartAt.getValue(), CompileTheWorldStopAt.getValue(), true); + this(CompileTheWorldClasspath.getValue(), parseConfig(CompileTheWorldConfig.getValue()), CompileTheWorldStartAt.getValue(), CompileTheWorldStopAt.getValue(), true); } /** - * Create a compile-the-world instance. + * Creates a compile-the-world instance. * * @param files {@link File#pathSeparator} separated list of Zip/Jar files to compile * @param startAt index of the class file to start compilation at * @param stopAt index of the class file to stop compilation at */ - public CompileTheWorld(String files, int startAt, int stopAt, boolean verbose) { + public CompileTheWorld(String files, Config config, int startAt, int stopAt, boolean verbose) { this.files = files; this.startAt = startAt; this.stopAt = stopAt; this.verbose = verbose; + this.config = config; // We don't want the VM to exit when a method fails to compile... ExitVMOnException.setValue(false); @@ -101,12 +186,10 @@ } /** - * Compile all methods in all classes in the Zip/Jar files in - * {@link GraalOptions#CompileTheWorld}. If the GraalOptions.CompileTheWorld contains the magic - * token {@link CompileTheWorld#SUN_BOOT_CLASS_PATH} passed up from HotSpot we take the files - * from the boot class path. - * - * @throws Throwable + * Compiles all methods in all classes in the Zip/Jar archive files in + * {@link #CompileTheWorldClasspath}. If {@link #CompileTheWorldClasspath} contains the magic + * token {@link #SUN_BOOT_CLASS_PATH} passed up from HotSpot we take the files from the boot + * class path. */ public void compile() throws Throwable { if (SUN_BOOT_CLASS_PATH.equals(files)) { @@ -145,7 +228,7 @@ } /** - * Compile all methods in all classes in the Zip/Jar files passed. + * Compiles all methods in all classes in the Zip/Jar files passed. * * @param fileList {@link File#pathSeparator} separated list of Zip/Jar files to compile * @throws Throwable @@ -186,7 +269,7 @@ String className = je.getName().substring(0, je.getName().length() - ".class".length()); classFileCounter++; - try { + try (AutoCloseable s = config == null ? null : config.apply()) { // Load and initialize class Class javaClass = Class.forName(className.replace('/', '.'), true, loader); @@ -233,25 +316,49 @@ } /** - * Helper method to schedule a method for compilation and gather some statistics. + * A compilation task that creates a fresh compilation suite for its compilation. This is + * required so that a CTW compilation can be {@linkplain Config configured} differently from a + * VM triggered compilation. + */ + static class CTWCompilationTask extends CompilationTask { + + CTWCompilationTask(HotSpotBackend backend, PhasePlan plan, OptimisticOptimizations optimisticOpts, ProfilingInfo profilingInfo, HotSpotResolvedJavaMethod method, int entryBCI, int id) { + super(backend, plan, optimisticOpts, profilingInfo, method, entryBCI, id); + } + + @Override + protected Suites getSuites(HotSpotProviders providers) { + return providers.getSuites().createSuites(); + } + } + + /** + * Compiles a method and gathers some statistics. */ private void compileMethod(HotSpotResolvedJavaMethod method) { try { long start = System.currentTimeMillis(); - vmToCompiler.compileMethod(method, StructuredGraph.INVOCATION_ENTRY_BCI, true); + + final ProfilingInfo profilingInfo = method.getCompilationProfilingInfo(false); + final OptimisticOptimizations optimisticOpts = new OptimisticOptimizations(profilingInfo); + int id = vmToCompiler.allocateCompileTaskId(); + HotSpotBackend backend = runtime.getHostBackend(); + PhasePlan phasePlan = vmToCompiler.createPhasePlan(backend.getProviders(), optimisticOpts, false); + CompilationTask task = new CTWCompilationTask(backend, phasePlan, optimisticOpts, profilingInfo, method, INVOCATION_ENTRY_BCI, id); + task.runCompilation(); + compileTime += (System.currentTimeMillis() - start); compiledMethodsCounter++; method.reprofile(); // makes the method also not-entrant } catch (Throwable t) { // Catch everything and print a message - println("CompileTheWorld (%d) : Error compiling method: %s", classFileCounter, MetaUtil.format("%H.%n(%p):%r", method)); + println("CompileTheWorldClasspath (%d) : Error compiling method: %s", classFileCounter, MetaUtil.format("%H.%n(%p):%r", method)); t.printStackTrace(TTY.cachedOut); } } /** - * Helper method for CompileTheWorld to determine if a method should be compiled (Cf. - * CompilationPolicy::can_be_compiled). + * Determines if a method should be compiled (Cf. CompilationPolicy::can_be_compiled). * * @return true if it can be compiled, false otherwise */ diff -r cd22c6bb4a35 -r 0e5c4f9fa9a5 graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/HotSpotGraalRuntime.java --- a/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/HotSpotGraalRuntime.java Mon Dec 16 12:10:10 2013 -0800 +++ b/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/HotSpotGraalRuntime.java Mon Dec 16 23:33:40 2013 +0100 @@ -23,7 +23,9 @@ package com.oracle.graal.hotspot; import static com.oracle.graal.graph.UnsafeAccess.*; +import static com.oracle.graal.hotspot.CompileTheWorld.*; import static com.oracle.graal.hotspot.HotSpotGraalRuntime.Options.*; +import static com.oracle.graal.phases.GraalOptions.*; import java.lang.reflect.*; import java.util.*; @@ -232,23 +234,23 @@ initMirror(typeVoid); // Set some global options: - if (config.compileTheWorld) { - GraalOptions.CompileTheWorld.setValue(CompileTheWorld.SUN_BOOT_CLASS_PATH); + if (config.compileTheWorld && CompileTheWorld.Options.CompileTheWorldClasspath.getValue() == null) { + CompileTheWorld.Options.CompileTheWorldClasspath.setValue(SUN_BOOT_CLASS_PATH); } if (config.compileTheWorldStartAt != 1) { - GraalOptions.CompileTheWorldStartAt.setValue(config.compileTheWorldStartAt); + CompileTheWorld.Options.CompileTheWorldStartAt.setValue(config.compileTheWorldStartAt); } if (config.compileTheWorldStopAt != Integer.MAX_VALUE) { - GraalOptions.CompileTheWorldStopAt.setValue(config.compileTheWorldStopAt); + CompileTheWorld.Options.CompileTheWorldStopAt.setValue(config.compileTheWorldStopAt); } // Only set HotSpotPrintCompilation and HotSpotPrintInlining if they still have their // default value (false). - if (GraalOptions.HotSpotPrintCompilation.getValue() == false) { - GraalOptions.HotSpotPrintCompilation.setValue(config.printCompilation); + if (HotSpotPrintCompilation.getValue() == false) { + HotSpotPrintCompilation.setValue(config.printCompilation); } - if (GraalOptions.HotSpotPrintInlining.getValue() == false) { - GraalOptions.HotSpotPrintInlining.setValue(config.printInlining); + if (HotSpotPrintInlining.getValue() == false) { + HotSpotPrintInlining.setValue(config.printInlining); } if (Boolean.valueOf(System.getProperty("graal.printconfig"))) { diff -r cd22c6bb4a35 -r 0e5c4f9fa9a5 graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/HotSpotOptions.java --- a/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/HotSpotOptions.java Mon Dec 16 12:10:10 2013 -0800 +++ b/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/HotSpotOptions.java Mon Dec 16 23:33:40 2013 +0100 @@ -103,6 +103,22 @@ // Called from VM code public static boolean setOption(String option) { + return parseOption(option, null); + } + + interface OptionConsumer { + void set(OptionDescriptor desc, Object value); + } + + /** + * Parses a given option value specification. + * + * @param option the specification of an option and its value + * @param setter the object to notify of the parsed option and value. If null, the + * {@link OptionValue#setValue(Object)} method of the specified option is called + * instead. + */ + public static boolean parseOption(String option, OptionConsumer setter) { if (option.length() == 0) { return false; } @@ -175,9 +191,13 @@ } if (value != null) { - OptionValue optionValue = desc.getOptionValue(); - optionValue.setValue(value); - // Logger.info("Set option " + desc.getName() + " to " + value); + if (setter != null) { + setter.set(desc, value); + } else { + OptionValue optionValue = desc.getOptionValue(); + optionValue.setValue(value); + // Logger.info("Set option " + desc.getName() + " to " + value); + } } else { Logger.info("Wrong value \"" + valueString + "\" for option " + optionName); return false; diff -r cd22c6bb4a35 -r 0e5c4f9fa9a5 graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/bridge/VMToCompilerImpl.java --- a/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/bridge/VMToCompilerImpl.java Mon Dec 16 12:10:10 2013 -0800 +++ b/graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/bridge/VMToCompilerImpl.java Mon Dec 16 23:33:40 2013 +0100 @@ -97,6 +97,10 @@ this.runtime = runtime; } + public int allocateCompileTaskId() { + return compileTaskIds.incrementAndGet(); + } + public void startCompiler(boolean bootstrapEnabled) throws Throwable { FastNodeClassRegistry.initialize(); @@ -556,7 +560,7 @@ final ProfilingInfo profilingInfo = method.getCompilationProfilingInfo(osrCompilation); final OptimisticOptimizations optimisticOpts = new OptimisticOptimizations(profilingInfo); - int id = compileTaskIds.incrementAndGet(); + int id = allocateCompileTaskId(); HotSpotBackend backend = runtime.getHostBackend(); CompilationTask task = CompilationTask.create(backend, createPhasePlan(backend.getProviders(), optimisticOpts, osrCompilation), optimisticOpts, profilingInfo, method, entryBCI, id); diff -r cd22c6bb4a35 -r 0e5c4f9fa9a5 graal/com.oracle.graal.phases/src/com/oracle/graal/phases/GraalOptions.java --- a/graal/com.oracle.graal.phases/src/com/oracle/graal/phases/GraalOptions.java Mon Dec 16 12:10:10 2013 -0800 +++ b/graal/com.oracle.graal.phases/src/com/oracle/graal/phases/GraalOptions.java Mon Dec 16 23:33:40 2013 +0100 @@ -87,14 +87,6 @@ @Option(help = "") public static final OptionValue DeoptsToDisableOptimisticOptimization = new OptionValue<>(40); - // compilation queue - @Option(help = "Compile all methods in all classes on given class path") - public static final OptionValue CompileTheWorld = new OptionValue<>(null); - @Option(help = "First class to consider when using CompileTheWorld") - public static final OptionValue CompileTheWorldStartAt = new OptionValue<>(1); - @Option(help = "Last class to consider when using CompileTheWorld") - public static final OptionValue CompileTheWorldStopAt = new OptionValue<>(Integer.MAX_VALUE); - // graph caching @Option(help = "") public static final OptionValue CacheGraphs = new OptionValue<>(true); diff -r cd22c6bb4a35 -r 0e5c4f9fa9a5 src/share/vm/graal/graalCompiler.cpp --- a/src/share/vm/graal/graalCompiler.cpp Mon Dec 16 12:10:10 2013 -0800 +++ b/src/share/vm/graal/graalCompiler.cpp Mon Dec 16 23:33:40 2013 +0100 @@ -115,6 +115,11 @@ // Avoid -Xcomp and -Xbatch problems by turning on interpreter and background compilation for bootstrapping. FlagSetting a(UseInterpreter, true); FlagSetting b(BackgroundCompilation, true); +#ifndef PRODUCT + // Turn off CompileTheWorld during bootstrap + // so that a complete bootstrap occurs + FlagSetting c(CompileTheWorld, false); +#endif VMToCompiler::bootstrap(); }