Mercurial > hg > graal-compiler

annotate graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/replacements/ArrayCopyNode.java @ 15259:d90e5c22ba55

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Move GraalOptions to graal.compiler.common.
author Josef Eisl <josef.eisl@jku.at>
date Tue, 22 Apr 2014 11:37:15 +0200
parents 39076a984c33
children 61363577a184
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
1 /*
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
2 * Copyright (c) 2013, Oracle and/or its affiliates. All rights reserved.
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
4 *
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
5 * This code is free software; you can redistribute it and/or modify it
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
6 * under the terms of the GNU General Public License version 2 only, as
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
7 * published by the Free Software Foundation.
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
8 *
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
9 * This code is distributed in the hope that it will be useful, but WITHOUT
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
12 * version 2 for more details (a copy is included in the LICENSE file that
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
13 * accompanied this code).
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
14 *
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
15 * You should have received a copy of the GNU General Public License version
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
16 * 2 along with this work; if not, write to the Free Software Foundation,
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
18 *
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
20 * or visit www.oracle.com if you need additional information or have any
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
21 * questions.
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
22 */
8415
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 7902
diff changeset
23 package com.oracle.graal.hotspot.replacements;
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
24
11491
a03452edfc4d made enabling/disabling of intrinsifications extensible (GRAAL-111)
Doug Simon <doug.simon@oracle.com>
parents: 11419
diff changeset
25 import static com.oracle.graal.compiler.GraalCompiler.*;
13933
ebd2dfc2b780 use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13615
diff changeset
26 import static com.oracle.graal.hotspot.replacements.HotSpotReplacementsUtil.*;
9864
063a712fe8d8 converted remaining options in GraalOptions to new system (GRAAL-27)
Doug Simon <doug.simon@oracle.com>
parents: 9265
diff changeset
27
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
28 import com.oracle.graal.api.meta.*;
15259
d90e5c22ba55 Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents: 13977
diff changeset
29 import com.oracle.graal.compiler.common.*;
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
30 import com.oracle.graal.debug.*;
13197
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
31 import com.oracle.graal.debug.Debug.Scope;
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
32 import com.oracle.graal.loop.phases.*;
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
33 import com.oracle.graal.nodes.*;
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
34 import com.oracle.graal.nodes.spi.*;
11265
ef6915cf1e59 Add illegal stamp
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10930
diff changeset
35 import com.oracle.graal.nodes.type.*;
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
36 import com.oracle.graal.nodes.virtual.*;
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
37 import com.oracle.graal.phases.common.*;
9052
989db61f2007 Make LoopFullUnrollPhase reentrant.
Roland Schatz <roland.schatz@oracle.com>
parents: 9049
diff changeset
38 import com.oracle.graal.phases.tiers.*;
8415
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 7902
diff changeset
39 import com.oracle.graal.replacements.nodes.*;
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
40
11633
a0566c8dcabf removed IterableNodeType from some classes for which typed node iterators were never used or were only used in tests
Doug Simon <doug.simon@oracle.com>
parents: 11631
diff changeset
41 public class ArrayCopyNode extends MacroNode implements Virtualizable, Lowerable {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
42
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
43 public ArrayCopyNode(Invoke invoke) {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
44 super(invoke);
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
45 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
46
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
47 private ValueNode getSource() {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
48 return arguments.get(0);
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
49 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
50
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
51 private ValueNode getSourcePosition() {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
52 return arguments.get(1);
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
53 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
54
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
55 private ValueNode getDestination() {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
56 return arguments.get(2);
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
57 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
58
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
59 private ValueNode getDestinationPosition() {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
60 return arguments.get(3);
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
61 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
62
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
63 private ValueNode getLength() {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
64 return arguments.get(4);
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
65 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
66
13977
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
67 static boolean isHeapWordAligned(Constant value, Kind kind) {
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
68 return (arrayBaseOffset(kind) + (long) value.asInt() * arrayIndexScale(kind)) % heapWordSize() == 0;
13933
ebd2dfc2b780 use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13615
diff changeset
69 }
ebd2dfc2b780 use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13615
diff changeset
70
11716
bffe5758c209 Snippets processing in ArrayCopyNode and ObjectCloneNode need proper scoping
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11633
diff changeset
71 private StructuredGraph selectSnippet(LoweringTool tool, final Replacements replacements) {
11265
ef6915cf1e59 Add illegal stamp
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10930
diff changeset
72 ResolvedJavaType srcType = ObjectStamp.typeOrNull(getSource().stamp());
ef6915cf1e59 Add illegal stamp
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10930
diff changeset
73 ResolvedJavaType destType = ObjectStamp.typeOrNull(getDestination().stamp());
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
74
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
75 if (srcType == null || !srcType.isArray() || destType == null || !destType.isArray()) {
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
76 return null;
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
77 }
13977
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
78 if (!destType.getComponentType().isAssignableFrom(srcType.getComponentType())) {
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
79 return null;
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
80 }
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
81 if (!isExact()) {
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
82 return null;
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
83 }
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
84 Kind componentKind = srcType.getComponentType().getKind();
13977
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
85 final ResolvedJavaMethod snippetMethod = tool.getMetaAccess().lookupJavaMethod(ArrayCopySnippets.getSnippetForKind(componentKind, shouldUnroll(), isExact()));
13197
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
86 try (Scope s = Debug.scope("ArrayCopySnippet", snippetMethod)) {
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
87 return replacements.getSnippet(snippetMethod);
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
88 } catch (Throwable e) {
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
89 throw Debug.handle(e);
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
90 }
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
91 }
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
92
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
93 private static void unrollFixedLengthLoop(StructuredGraph snippetGraph, int length, LoweringTool tool) {
13614
0774f3303c2e rename LocalNode to ParameterNode
Lukas Stadler <lukas.stadler@jku.at>
parents: 13197
diff changeset
94 ParameterNode lengthParam = snippetGraph.getParameter(4);
0774f3303c2e rename LocalNode to ParameterNode
Lukas Stadler <lukas.stadler@jku.at>
parents: 13197
diff changeset
95 if (lengthParam != null) {
0774f3303c2e rename LocalNode to ParameterNode
Lukas Stadler <lukas.stadler@jku.at>
parents: 13197
diff changeset
96 snippetGraph.replaceFloating(lengthParam, ConstantNode.forInt(length, snippetGraph));
7540
a77f22f2759d remove errorneous assert, fix ArrayCopyNode's "replace local" code
Lukas Stadler <lukas.stadler@jku.at>
parents: 7539
diff changeset
97 }
a77f22f2759d remove errorneous assert, fix ArrayCopyNode's "replace local" code
Lukas Stadler <lukas.stadler@jku.at>
parents: 7539
diff changeset
98 // the canonicalization before loop unrolling is needed to propagate the length into
a77f22f2759d remove errorneous assert, fix ArrayCopyNode's "replace local" code
Lukas Stadler <lukas.stadler@jku.at>
parents: 7539
diff changeset
99 // additions, etc.
13137
438ed35bed29 Change PhaseContext so that it does not subclass Providers, to avoid leaking low-level provider objects such as the CodeCacheProvdier to high-level optimization phases
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12617
diff changeset
100 PhaseContext context = new PhaseContext(tool.getMetaAccess(), tool.getConstantReflection(), tool.getLowerer(), tool.getReplacements(), tool.assumptions());
9916
491cd7d69539 CanonicalizerPhase: remove it from context, add it to tiers instead and configure/pass it there (GRAAL-309)
Bernhard Urban <bernhard.urban@jku.at>
parents: 9908
diff changeset
101 new CanonicalizerPhase(true).apply(snippetGraph, context);
11572
a63b63822183 Pass canonicalizer into loop transformations.
Roland Schatz <roland.schatz@oracle.com>
parents: 11556
diff changeset
102 new LoopFullUnrollPhase(new CanonicalizerPhase(true)).apply(snippetGraph, context);
9916
491cd7d69539 CanonicalizerPhase: remove it from context, add it to tiers instead and configure/pass it there (GRAAL-309)
Bernhard Urban <bernhard.urban@jku.at>
parents: 9908
diff changeset
103 new CanonicalizerPhase(true).apply(snippetGraph, context);
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
104 }
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
105
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
106 @Override
11754
6e734982f89f fixed concurrency issue in lowering of MacroNode replacement graphs
Doug Simon <doug.simon@oracle.com>
parents: 11716
diff changeset
107 protected StructuredGraph getLoweredSnippetGraph(final LoweringTool tool) {
11491
a03452edfc4d made enabling/disabling of intrinsifications extensible (GRAAL-111)
Doug Simon <doug.simon@oracle.com>
parents: 11419
diff changeset
108 if (!shouldIntrinsify(getTargetMethod())) {
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
109 return null;
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
110 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
111
11716
bffe5758c209 Snippets processing in ArrayCopyNode and ObjectCloneNode need proper scoping
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 11633
diff changeset
112 final Replacements replacements = tool.getReplacements();
8627
75db7afee829 implemented lazy installation of replacements (GRAAL-137)
Doug Simon <doug.simon@oracle.com>
parents: 8559
diff changeset
113 StructuredGraph snippetGraph = selectSnippet(tool, replacements);
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
114 if (snippetGraph == null) {
11959
23ccaa863eda made CodeCacheProvider independent of MetaAccessProvider (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 11754
diff changeset
115 final ResolvedJavaMethod snippetMethod = tool.getMetaAccess().lookupJavaMethod(ArrayCopySnippets.genericArraycopySnippet);
13197
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
116 snippetGraph = null;
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
117 try (Scope s = Debug.scope("ArrayCopySnippet", snippetMethod)) {
13604
bd21ee1a874c fix recursion problem around macro nodes (e.g. array copy)
Bernhard Urban <bernhard.urban@jku.at>
parents: 13197
diff changeset
118 snippetGraph = replacements.getSnippet(snippetMethod, getTargetMethod()).copy();
13197
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
119 } catch (Throwable e) {
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
120 throw Debug.handle(e);
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
121 }
8550
53cc1dd7ee18 pull replaceSnippetInvokes into MacroNode
Lukas Stadler <lukas.stadler@jku.at>
parents: 8545
diff changeset
122 replaceSnippetInvokes(snippetGraph);
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
123 } else {
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
124 assert snippetGraph != null : "ArrayCopySnippets should be installed";
11754
6e734982f89f fixed concurrency issue in lowering of MacroNode replacement graphs
Doug Simon <doug.simon@oracle.com>
parents: 11716
diff changeset
125 snippetGraph = snippetGraph.copy();
13977
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
126 if (shouldUnroll()) {
11754
6e734982f89f fixed concurrency issue in lowering of MacroNode replacement graphs
Doug Simon <doug.simon@oracle.com>
parents: 11716
diff changeset
127 final StructuredGraph copy = snippetGraph;
13197
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
128 try (Scope s = Debug.scope("ArrayCopySnippetSpecialization", snippetGraph.method())) {
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
129 unrollFixedLengthLoop(copy, getLength().asConstant().asInt(), tool);
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
130 } catch (Throwable e) {
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
131 throw Debug.handle(e);
8569b9e047cd change debug scopes implementation to prevent extra frames related to mechanism being added to call stack
Doug Simon <doug.simon@oracle.com>
parents: 13137
diff changeset
132 }
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7827
diff changeset
133 }
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
134 }
11754
6e734982f89f fixed concurrency issue in lowering of MacroNode replacement graphs
Doug Simon <doug.simon@oracle.com>
parents: 11716
diff changeset
135 return lowerReplacement(snippetGraph, tool);
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
136 }
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
137
13977
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
138 private boolean shouldUnroll() {
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
139 return getLength().isConstant() && getLength().asConstant().asInt() <= GraalOptions.MaximumEscapeAnalysisArrayLength.getValue();
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
140 }
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
141
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
142 /*
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
143 * Returns true if this copy doesn't require store checks. Trivially true for primitive arrays.
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
144 */
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
145 private boolean isExact() {
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
146 ResolvedJavaType srcType = ObjectStamp.typeOrNull(getSource().stamp());
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
147 if (srcType.getComponentType().getKind().isPrimitive() || getSource() == getDestination()) {
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
148 return true;
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
149 }
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
150
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
151 ResolvedJavaType destType = ObjectStamp.typeOrNull(getDestination().stamp());
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
152 if (ObjectStamp.isExactType(getDestination().stamp())) {
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
153 if (destType != null && destType.isAssignableFrom(srcType)) {
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
154 return true;
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
155 }
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
156 }
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
157 return false;
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
158 }
39076a984c33 lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13933
diff changeset
159
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
160 private static boolean checkBounds(int position, int length, VirtualObjectNode virtualObject) {
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
161 return position >= 0 && position + length <= virtualObject.entryCount();
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
162 }
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
163
9958
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
164 private static boolean checkEntryTypes(int srcPos, int length, State srcState, ResolvedJavaType destComponentType, VirtualizerTool tool) {
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
165 if (destComponentType.getKind() == Kind.Object) {
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
166 for (int i = 0; i < length; i++) {
9958
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
167 ValueNode entry = srcState.getEntry(srcPos + i);
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
168 State state = tool.getObjectState(entry);
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
169 ResolvedJavaType type;
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
170 if (state != null) {
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
171 if (state.getState() == EscapeState.Virtual) {
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
172 type = state.getVirtualObject().type();
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
173 } else {
11265
ef6915cf1e59 Add illegal stamp
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10930
diff changeset
174 type = ObjectStamp.typeOrNull(state.getMaterializedValue());
9958
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
175 }
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
176 } else {
11265
ef6915cf1e59 Add illegal stamp
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 10930
diff changeset
177 type = ObjectStamp.typeOrNull(entry);
9958
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
178 }
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
179 if (type == null || !destComponentType.isAssignableFrom(type)) {
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
180 return false;
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
181 }
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
182 }
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
183 }
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
184 return true;
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
185 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
186
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
187 @Override
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
188 public void virtualize(VirtualizerTool tool) {
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
189 if (getSourcePosition().isConstant() && getDestinationPosition().isConstant() && getLength().isConstant()) {
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
190 int srcPos = getSourcePosition().asConstant().asInt();
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
191 int destPos = getDestinationPosition().asConstant().asInt();
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
192 int length = getLength().asConstant().asInt();
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
193 State srcState = tool.getObjectState(getSource());
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
194 State destState = tool.getObjectState(getDestination());
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
195
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
196 if (srcState != null && srcState.getState() == EscapeState.Virtual && destState != null && destState.getState() == EscapeState.Virtual) {
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
197 VirtualObjectNode srcVirtual = srcState.getVirtualObject();
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
198 VirtualObjectNode destVirtual = destState.getVirtualObject();
9958
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
199 if (!(srcVirtual instanceof VirtualArrayNode) || !(destVirtual instanceof VirtualArrayNode)) {
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
200 return;
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
201 }
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
202 if (((VirtualArrayNode) srcVirtual).componentType().getKind() != Kind.Object || ((VirtualArrayNode) destVirtual).componentType().getKind() != Kind.Object) {
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
203 return;
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
204 }
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
205 if (length < 0 || !checkBounds(srcPos, length, srcVirtual) || !checkBounds(destPos, length, destVirtual)) {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
206 return;
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
207 }
9958
8efb5a58a799 more checks for ArrayCopyNode virtualization
Lukas Stadler <lukas.stadler@jku.at>
parents: 9917
diff changeset
208 if (!checkEntryTypes(srcPos, length, srcState, destVirtual.type().getComponentType(), tool)) {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
209 return;
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
210 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
211 for (int i = 0; i < length; i++) {
12617
bca33c3135de PEA: support for unsafe stores of mismatching sizes, cleanup, documentation
Lukas Stadler <lukas.stadler@jku.at>
parents: 12503
diff changeset
212 tool.setVirtualEntry(destState, destPos + i, srcState.getEntry(srcPos + i), false);
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
213 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
214 tool.delete();
7538
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
215 if (Debug.isLogEnabled()) {
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
216 Debug.log("virtualized arraycopyf(%s, %d, %s, %d, %d)", getSource(), srcPos, getDestination(), destPos, length);
50793b11b74d ArrayCopyNode: cleanup, unrolling for fixed length arraycopy
Lukas Stadler <lukas.stadler@jku.at>
parents: 7511
diff changeset
217 }
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
218 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
219 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
220 }
13933
ebd2dfc2b780 use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 13615
diff changeset
221
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
diff changeset
222 }