# HG changeset patch # User Christian Humer # Date 1383833409 -3600 # Node ID 85bab2227295dc92446134dc616ec3357871062b # Parent 0e998f0daddf8596d7c3ea3ea819686df6933764 Truffle: refactored inlining to a new class. diff -r 0e998f0daddf -r 85bab2227295 graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/CompilationPolicy.java --- a/graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/CompilationPolicy.java Thu Nov 07 11:17:23 2013 +0100 +++ b/graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/CompilationPolicy.java Thu Nov 07 15:10:09 2013 +0100 @@ -60,7 +60,7 @@ return loopAndInvokeCounter; } - public void compilationInvalidated() { + public void reportCompilationInvalidated() { int invalidationReprofileCount = TruffleInvalidationReprofileCount.getValue(); invokeCounter = invalidationReprofileCount; if (TruffleFunctionInlining.getValue()) { @@ -68,12 +68,12 @@ } } - public void countInterpreterCall() { + public void reportInterpreterCall() { invokeCounter--; loopAndInvokeCounter--; } - public void inlined(int minInvokesAfterInlining) { + public void reportCallInlined(int minInvokesAfterInlining) { invokeCounter = minInvokesAfterInlining; int inliningReprofileCount = TruffleInliningReprofileCount.getValue(); loopAndInvokeCounter = inliningReprofileCount; @@ -84,7 +84,7 @@ loopAndInvokeCounter = Math.max(0, loopAndInvokeCounter - count); } - public void nodeReplaced() { + public void notifyNodeReplaced() { // delay compilation until tree is deemed stable enough int replaceBackoff = TruffleReplaceReprofileCount.getValue(); if (loopAndInvokeCounter < replaceBackoff) { @@ -92,7 +92,7 @@ } } - public boolean compileOrInline() { + public boolean shouldCompileOrInline() { if (invokeCounter <= 0 && loopAndInvokeCounter <= 0) { if (TruffleUseTimeForCompilationDecision.getValue()) { long timestamp = System.nanoTime(); diff -r 0e998f0daddf -r 85bab2227295 graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/OptimizedCallTarget.java --- a/graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/OptimizedCallTarget.java Thu Nov 07 11:17:23 2013 +0100 +++ b/graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/OptimizedCallTarget.java Thu Nov 07 15:10:09 2013 +0100 @@ -47,6 +47,7 @@ super(rootNode, descriptor); this.compiler = compiler; this.compilationPolicy = new CompilationPolicy(compilationThreshold, invokeCounter, rootNode.toString()); + this.inlining = new TruffleInliningImpl(); this.rootNode.setCallTarget(this); if (TruffleCallTargetProfiling.getValue()) { @@ -58,6 +59,7 @@ private Future installedCodeTask; private final TruffleCompiler compiler; private final CompilationPolicy compilationPolicy; + private final TruffleInlining inlining; private boolean disableCompilation; private int callCount; @@ -76,6 +78,10 @@ return callHelper(caller, args); } + CompilationPolicy getCompilationPolicy() { + return compilationPolicy; + } + private Object callHelper(PackedFrame caller, Arguments args) { if (installedCode != null && installedCode.isValid()) { TruffleRuntime runtime = Truffle.getRuntime(); @@ -110,7 +116,7 @@ CompilerAsserts.neverPartOfCompilation(); installedCode = null; invalidationCount++; - compilationPolicy.compilationInvalidated(); + compilationPolicy.reportCompilationInvalidated(); if (TraceTruffleCompilation.getValue()) { OUT.printf("[truffle] invalidated %-48s |Inv# %d |Replace# %d\n", rootNode, invalidationCount, nodeReplaceCount); } @@ -120,14 +126,14 @@ if (task != null) { task.cancel(true); this.installedCodeTask = null; - compilationPolicy.compilationInvalidated(); + compilationPolicy.reportCompilationInvalidated(); } } private Object interpreterCall(PackedFrame caller, Arguments args) { CompilerAsserts.neverPartOfCompilation(); - compilationPolicy.countInterpreterCall(); - if (disableCompilation || !compilationPolicy.compileOrInline()) { + compilationPolicy.reportInterpreterCall(); + if (disableCompilation || !compilationPolicy.shouldCompileOrInline()) { return executeHelper(caller, args); } else { return compileOrInline(caller, args); @@ -148,7 +154,7 @@ } if (TruffleFunctionInlining.getValue() && inline()) { - compilationPolicy.inlined(MIN_INVOKES_AFTER_INLINING); + compilationPolicy.reportCallInlined(MIN_INVOKES_AFTER_INLINING); return call(caller, args); } else { compile(); @@ -181,7 +187,7 @@ public boolean inline() { CompilerAsserts.neverPartOfCompilation(); - return new InliningHelper(this).inline(); + return inlining.performInlining(this); } public void compile() { @@ -215,185 +221,7 @@ public void nodeReplaced() { nodeReplaceCount++; invalidate(); - compilationPolicy.nodeReplaced(); - } - - private static class InliningHelper { - - private final OptimizedCallTarget target; - - public InliningHelper(OptimizedCallTarget target) { - this.target = target; - } - - public boolean inline() { - final InliningPolicy policy = new InliningPolicy(target); - if (!policy.continueInlining()) { - if (TraceTruffleInliningDetails.getValue()) { - List inlinableCallSites = getInlinableCallSites(target); - if (!inlinableCallSites.isEmpty()) { - OUT.printf("[truffle] inlining hit caller size limit (%3d >= %3d).%3d remaining call sites in %s:\n", policy.callerNodeCount, TruffleInliningMaxCallerSize.getValue(), - inlinableCallSites.size(), target.getRootNode()); - policy.sortByRelevance(inlinableCallSites); - printCallSiteInfo(policy, inlinableCallSites, ""); - } - } - return false; - } - - List inlinableCallSites = getInlinableCallSites(target); - if (inlinableCallSites.isEmpty()) { - return false; - } - - policy.sortByRelevance(inlinableCallSites); - - boolean inlined = false; - for (InlinableCallSiteInfo inlinableCallSite : inlinableCallSites) { - if (!policy.isWorthInlining(inlinableCallSite)) { - break; - } - if (inlinableCallSite.getCallSite().inline(target)) { - if (TraceTruffleInlining.getValue()) { - printCallSiteInfo(policy, inlinableCallSite, "inlined"); - } - inlined = true; - break; - } - } - - if (inlined) { - for (InlinableCallSiteInfo callSite : inlinableCallSites) { - callSite.getCallSite().resetCallCount(); - } - } else { - if (TraceTruffleInliningDetails.getValue()) { - OUT.printf("[truffle] inlining stopped.%3d remaining call sites in %s:\n", inlinableCallSites.size(), target.getRootNode()); - printCallSiteInfo(policy, inlinableCallSites, ""); - } - } - - return inlined; - } - - private static void printCallSiteInfo(InliningPolicy policy, List inlinableCallSites, String msg) { - for (InlinableCallSiteInfo candidate : inlinableCallSites) { - printCallSiteInfo(policy, candidate, msg); - } - } - - private static void printCallSiteInfo(InliningPolicy policy, InlinableCallSiteInfo callSite, String msg) { - String calls = String.format("%4s/%4s", callSite.getCallCount(), policy.callerInvocationCount); - String nodes = String.format("%3s/%3s", callSite.getInlineNodeCount(), policy.callerNodeCount); - OUT.printf("[truffle] %-9s %-50s |Nodes %6s |Calls %6s %7.3f |%s\n", msg, callSite.getCallSite(), nodes, calls, policy.metric(callSite), callSite.getCallSite().getCallTarget()); - } - - private static final class InliningPolicy { - - private final int callerNodeCount; - private final int callerInvocationCount; - - public InliningPolicy(OptimizedCallTarget caller) { - this.callerNodeCount = NodeUtil.countNodes(caller.getRootNode()); - this.callerInvocationCount = caller.compilationPolicy.getOriginalInvokeCounter(); - } - - public boolean continueInlining() { - return callerNodeCount < TruffleInliningMaxCallerSize.getValue(); - } - - public boolean isWorthInlining(InlinableCallSiteInfo callSite) { - return callSite.getInlineNodeCount() <= TruffleInliningMaxCalleeSize.getValue() && callSite.getInlineNodeCount() + callerNodeCount <= TruffleInliningMaxCallerSize.getValue() && - callSite.getCallCount() > 0 && callSite.getRecursiveDepth() < TruffleInliningMaxRecursiveDepth.getValue() && - (frequency(callSite) >= TruffleInliningMinFrequency.getValue() || callSite.getInlineNodeCount() <= TruffleInliningTrivialSize.getValue()); - } - - public double metric(InlinableCallSiteInfo callSite) { - double cost = callSite.getInlineNodeCount(); - double metric = frequency(callSite) / cost; - return metric; - } - - private double frequency(InlinableCallSiteInfo callSite) { - return (double) callSite.getCallCount() / (double) callerInvocationCount; - } - - public void sortByRelevance(List inlinableCallSites) { - Collections.sort(inlinableCallSites, new Comparator() { - - @Override - public int compare(InlinableCallSiteInfo cs1, InlinableCallSiteInfo cs2) { - int result = (isWorthInlining(cs2) ? 1 : 0) - (isWorthInlining(cs1) ? 1 : 0); - if (result == 0) { - return Double.compare(metric(cs2), metric(cs1)); - } - return result; - } - }); - } - } - - private static final class InlinableCallSiteInfo { - - private final InlinableCallSite callSite; - private final int callCount; - private final int nodeCount; - private final int recursiveDepth; - - public InlinableCallSiteInfo(InlinableCallSite callSite) { - this.callSite = callSite; - this.callCount = callSite.getCallCount(); - this.nodeCount = NodeUtil.countNodes(callSite.getInlineTree()); - this.recursiveDepth = calculateRecursiveDepth(); - } - - public int getRecursiveDepth() { - return recursiveDepth; - } - - private int calculateRecursiveDepth() { - int depth = 0; - Node parent = ((Node) callSite).getParent(); - while (!(parent instanceof RootNode)) { - assert parent != null; - if (parent instanceof InlinedCallSite && ((InlinedCallSite) parent).getCallTarget() == callSite.getCallTarget()) { - depth++; - } - parent = parent.getParent(); - } - if (((RootNode) parent).getCallTarget() == callSite.getCallTarget()) { - depth++; - } - return depth; - } - - public InlinableCallSite getCallSite() { - return callSite; - } - - public int getCallCount() { - return callCount; - } - - public int getInlineNodeCount() { - return nodeCount; - } - } - - private static List getInlinableCallSites(final DefaultCallTarget target) { - final ArrayList inlinableCallSites = new ArrayList<>(); - target.getRootNode().accept(new NodeVisitor() { - - @Override - public boolean visit(Node node) { - if (node instanceof InlinableCallSite) { - inlinableCallSites.add(new InlinableCallSiteInfo((InlinableCallSite) node)); - } - return true; - } - }); - return inlinableCallSites; - } + compilationPolicy.notifyNodeReplaced(); } private static void resetProfiling() { @@ -425,7 +253,7 @@ continue; } - int notInlinedCallSiteCount = InliningHelper.getInlinableCallSites(callTarget).size(); + int notInlinedCallSiteCount = TruffleInliningImpl.getInlinableCallSites(callTarget).size(); int nodeCount = NodeUtil.countNodes(callTarget.rootNode); int inlinedCallSiteCount = NodeUtil.countNodes(callTarget.rootNode, InlinedCallSite.class); String comment = callTarget.installedCode == null ? " int" : ""; diff -r 0e998f0daddf -r 85bab2227295 graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/TruffleInlining.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/TruffleInlining.java Thu Nov 07 15:10:09 2013 +0100 @@ -0,0 +1,28 @@ +/* + * Copyright (c) 2013, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ +package com.oracle.graal.truffle; + +public interface TruffleInlining { + + boolean performInlining(OptimizedCallTarget callTarget); +} diff -r 0e998f0daddf -r 85bab2227295 graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/TruffleInliningImpl.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/graal/com.oracle.graal.truffle/src/com/oracle/graal/truffle/TruffleInliningImpl.java Thu Nov 07 15:10:09 2013 +0100 @@ -0,0 +1,207 @@ +/* + * Copyright (c) 2013, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ +package com.oracle.graal.truffle; + +import static com.oracle.graal.truffle.TruffleCompilerOptions.*; + +import java.io.*; +import java.util.*; + +import com.oracle.graal.debug.*; +import com.oracle.truffle.api.impl.*; +import com.oracle.truffle.api.nodes.*; + +class TruffleInliningImpl implements TruffleInlining { + + private static final PrintStream OUT = TTY.out().out(); + + @Override + public boolean performInlining(OptimizedCallTarget target) { + final InliningPolicy policy = new InliningPolicy(target); + if (!policy.continueInlining()) { + if (TraceTruffleInliningDetails.getValue()) { + List inlinableCallSites = getInlinableCallSites(target); + if (!inlinableCallSites.isEmpty()) { + OUT.printf("[truffle] inlining hit caller size limit (%3d >= %3d).%3d remaining call sites in %s:\n", policy.callerNodeCount, TruffleInliningMaxCallerSize.getValue(), + inlinableCallSites.size(), target.getRootNode()); + policy.sortByRelevance(inlinableCallSites); + printCallSiteInfo(policy, inlinableCallSites, ""); + } + } + return false; + } + + List inlinableCallSites = getInlinableCallSites(target); + if (inlinableCallSites.isEmpty()) { + return false; + } + + policy.sortByRelevance(inlinableCallSites); + + boolean inlined = false; + for (InlinableCallSiteInfo inlinableCallSite : inlinableCallSites) { + if (!policy.isWorthInlining(inlinableCallSite)) { + break; + } + if (inlinableCallSite.getCallSite().inline(target)) { + if (TraceTruffleInlining.getValue()) { + printCallSiteInfo(policy, inlinableCallSite, "inlined"); + } + inlined = true; + break; + } + } + + if (inlined) { + for (InlinableCallSiteInfo callSite : inlinableCallSites) { + callSite.getCallSite().resetCallCount(); + } + } else { + if (TraceTruffleInliningDetails.getValue()) { + OUT.printf("[truffle] inlining stopped.%3d remaining call sites in %s:\n", inlinableCallSites.size(), target.getRootNode()); + printCallSiteInfo(policy, inlinableCallSites, ""); + } + } + + return inlined; + } + + private static void printCallSiteInfo(InliningPolicy policy, List inlinableCallSites, String msg) { + for (InlinableCallSiteInfo candidate : inlinableCallSites) { + printCallSiteInfo(policy, candidate, msg); + } + } + + private static void printCallSiteInfo(InliningPolicy policy, InlinableCallSiteInfo callSite, String msg) { + String calls = String.format("%4s/%4s", callSite.getCallCount(), policy.callerInvocationCount); + String nodes = String.format("%3s/%3s", callSite.getInlineNodeCount(), policy.callerNodeCount); + OUT.printf("[truffle] %-9s %-50s |Nodes %6s |Calls %6s %7.3f |%s\n", msg, callSite.getCallSite(), nodes, calls, policy.metric(callSite), callSite.getCallSite().getCallTarget()); + } + + private static final class InliningPolicy { + + private final int callerNodeCount; + private final int callerInvocationCount; + + public InliningPolicy(OptimizedCallTarget caller) { + this.callerNodeCount = NodeUtil.countNodes(caller.getRootNode()); + this.callerInvocationCount = caller.getCompilationPolicy().getOriginalInvokeCounter(); + } + + public boolean continueInlining() { + return callerNodeCount < TruffleInliningMaxCallerSize.getValue(); + } + + public boolean isWorthInlining(InlinableCallSiteInfo callSite) { + return callSite.getInlineNodeCount() <= TruffleInliningMaxCalleeSize.getValue() && callSite.getInlineNodeCount() + callerNodeCount <= TruffleInliningMaxCallerSize.getValue() && + callSite.getCallCount() > 0 && callSite.getRecursiveDepth() < TruffleInliningMaxRecursiveDepth.getValue() && + (frequency(callSite) >= TruffleInliningMinFrequency.getValue() || callSite.getInlineNodeCount() <= TruffleInliningTrivialSize.getValue()); + } + + public double metric(InlinableCallSiteInfo callSite) { + double cost = callSite.getInlineNodeCount(); + double metric = frequency(callSite) / cost; + return metric; + } + + private double frequency(InlinableCallSiteInfo callSite) { + return (double) callSite.getCallCount() / (double) callerInvocationCount; + } + + public void sortByRelevance(List inlinableCallSites) { + Collections.sort(inlinableCallSites, new Comparator() { + + @Override + public int compare(InlinableCallSiteInfo cs1, InlinableCallSiteInfo cs2) { + int result = (isWorthInlining(cs2) ? 1 : 0) - (isWorthInlining(cs1) ? 1 : 0); + if (result == 0) { + return Double.compare(metric(cs2), metric(cs1)); + } + return result; + } + }); + } + } + + private static final class InlinableCallSiteInfo { + + private final InlinableCallSite callSite; + private final int callCount; + private final int nodeCount; + private final int recursiveDepth; + + public InlinableCallSiteInfo(InlinableCallSite callSite) { + this.callSite = callSite; + this.callCount = callSite.getCallCount(); + this.nodeCount = NodeUtil.countNodes(callSite.getInlineTree()); + this.recursiveDepth = calculateRecursiveDepth(); + } + + public int getRecursiveDepth() { + return recursiveDepth; + } + + private int calculateRecursiveDepth() { + int depth = 0; + Node parent = ((Node) callSite).getParent(); + while (!(parent instanceof RootNode)) { + assert parent != null; + if (parent instanceof InlinedCallSite && ((InlinedCallSite) parent).getCallTarget() == callSite.getCallTarget()) { + depth++; + } + parent = parent.getParent(); + } + if (((RootNode) parent).getCallTarget() == callSite.getCallTarget()) { + depth++; + } + return depth; + } + + public InlinableCallSite getCallSite() { + return callSite; + } + + public int getCallCount() { + return callCount; + } + + public int getInlineNodeCount() { + return nodeCount; + } + } + + static List getInlinableCallSites(final DefaultCallTarget target) { + final ArrayList inlinableCallSites = new ArrayList<>(); + target.getRootNode().accept(new NodeVisitor() { + + @Override + public boolean visit(Node node) { + if (node instanceof InlinableCallSite) { + inlinableCallSites.add(new InlinableCallSiteInfo((InlinableCallSite) node)); + } + return true; + } + }); + return inlinableCallSites; + } +}