Mercurial > hg > truffle
annotate graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/replacements/arraycopy/ArrayCopySnippets.java @ 21554:b1530a6cce8c
renamed com.oracle.graal.[debug|options|hotspotvmconfig]* modules to com.oracle.jvmci.[debug|options|hotspotvmconfig]* modules (JBS:GRAAL-53)
author | Doug Simon <doug.simon@oracle.com> |
---|---|
date | Tue, 26 May 2015 23:21:15 +0200 |
parents | 5324104ac4f3 |
children | 48c1ebd24120 |
rev | line source |
---|---|
3733
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
1 /* |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
2 * Copyright (c) 2011, 2015, Oracle and/or its affiliates. All rights reserved. |
3733
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
4 * |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
5 * This code is free software; you can redistribute it and/or modify it |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
6 * under the terms of the GNU General Public License version 2 only, as |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
7 * published by the Free Software Foundation. |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
8 * |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
9 * This code is distributed in the hope that it will be useful, but WITHOUT |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
12 * version 2 for more details (a copy is included in the LICENSE file that |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
13 * accompanied this code). |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
14 * |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
15 * You should have received a copy of the GNU General Public License version |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
16 * 2 along with this work; if not, write to the Free Software Foundation, |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
18 * |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
20 * or visit www.oracle.com if you need additional information or have any |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
21 * questions. |
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
22 */ |
18596
5afa718edb8b
arraycopy: move related classes to dedicated package
Bernhard Urban <bernhard.urban@jku.at>
parents:
18594
diff
changeset
|
23 package com.oracle.graal.hotspot.replacements.arraycopy; |
7530
5e3d1a68664e
applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents:
7510
diff
changeset
|
24 |
15259
d90e5c22ba55
Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents:
15193
diff
changeset
|
25 import static com.oracle.graal.compiler.common.GraalOptions.*; |
9513
659bb6cf930c
rename: HotSpotSnippetUtils -> HotSpotReplacementsUtil
Doug Simon <doug.simon@oracle.com>
parents:
9318
diff
changeset
|
26 import static com.oracle.graal.hotspot.replacements.HotSpotReplacementsUtil.*; |
10637
ba1fbbfac0cd
remove null check semantics from LoadHubNode (GRAAL-248)
Doug Simon <doug.simon@oracle.com>
parents:
10438
diff
changeset
|
27 import static com.oracle.graal.nodes.GuardingPiNode.*; |
10787
388fbd0dd4a4
moved BranchNodeProbabilities to graal.nodes project
Doug Simon <doug.simon@oracle.com>
parents:
10691
diff
changeset
|
28 import static com.oracle.graal.nodes.extended.BranchProbabilityNode.*; |
6362
6d232ee6a62d
factored shared snippet functionality into HotSpotSnippetUtils
Doug Simon <doug.simon@oracle.com>
parents:
6328
diff
changeset
|
29 |
7510
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
30 import java.lang.reflect.*; |
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
31 import java.util.*; |
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
32 |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
33 import com.oracle.graal.api.code.*; |
5507
dc71b06d09f8
Moving classes from cri.ri to api.meta.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
5504
diff
changeset
|
34 import com.oracle.graal.api.meta.*; |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
35 import com.oracle.graal.api.replacements.*; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
36 import com.oracle.graal.graph.*; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
37 import com.oracle.graal.hotspot.meta.*; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
38 import com.oracle.graal.hotspot.nodes.type.*; |
18401
e7ab82e7cc37
Move metaspace pointer handling to hotspot specific WordTypeRewriter.
Roland Schatz <roland.schatz@oracle.com>
parents:
18377
diff
changeset
|
39 import com.oracle.graal.hotspot.word.*; |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
40 import com.oracle.graal.loop.phases.*; |
5060
4ed4295ce15f
Update import statements.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
5059
diff
changeset
|
41 import com.oracle.graal.nodes.*; |
4ed4295ce15f
Update import statements.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
5059
diff
changeset
|
42 import com.oracle.graal.nodes.extended.*; |
7116
e83ee37dae40
Factor out the precondition checking code in ArrayCopySnippets
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
6390
diff
changeset
|
43 import com.oracle.graal.nodes.java.*; |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
44 import com.oracle.graal.nodes.spi.*; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
45 import com.oracle.graal.nodes.type.*; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
46 import com.oracle.graal.phases.common.*; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
47 import com.oracle.graal.phases.tiers.*; |
8415
2361bf148c06
rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents:
8413
diff
changeset
|
48 import com.oracle.graal.replacements.*; |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
49 import com.oracle.graal.replacements.Snippet.ConstantParameter; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
50 import com.oracle.graal.replacements.SnippetTemplate.Arguments; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
51 import com.oracle.graal.replacements.SnippetTemplate.SnippetInfo; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
52 import com.oracle.graal.replacements.nodes.*; |
7902
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
53 import com.oracle.graal.word.*; |
21543
93c50cefb9e8
moved GraalInternalError to com.oracle.jvmci.common and renamed it to JVMCIError (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents:
21528
diff
changeset
|
54 import com.oracle.jvmci.common.*; |
21554
b1530a6cce8c
renamed com.oracle.graal.[debug|options|hotspotvmconfig]* modules to com.oracle.jvmci.[debug|options|hotspotvmconfig]* modules (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents:
21551
diff
changeset
|
55 import com.oracle.jvmci.debug.*; |
b1530a6cce8c
renamed com.oracle.graal.[debug|options|hotspotvmconfig]* modules to com.oracle.jvmci.[debug|options|hotspotvmconfig]* modules (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents:
21551
diff
changeset
|
56 import com.oracle.jvmci.debug.Debug.Scope; |
21551
5324104ac4f3
moved com.oracle.graal.hotspot.jvmci classes to com.oracle.jvmci.hotspot module (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents:
21543
diff
changeset
|
57 import com.oracle.jvmci.hotspot.*; |
3733
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
58 |
8413
d9347ee39fbe
rename: SnippetsInterface -> Snippets
Doug Simon <doug.simon@oracle.com>
parents:
7902
diff
changeset
|
59 public class ArrayCopySnippets implements Snippets { |
7510
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
60 |
18401
e7ab82e7cc37
Move metaspace pointer handling to hotspot specific WordTypeRewriter.
Roland Schatz <roland.schatz@oracle.com>
parents:
18377
diff
changeset
|
61 private static int checkArrayType(KlassPointer hub) { |
7902
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
62 int layoutHelper = readLayoutHelper(hub); |
12546
d084c3db9c61
ArrayCopySnippets: use slow path probability for deopt branches
Bernhard Urban <bernhard.urban@jku.at>
parents:
10787
diff
changeset
|
63 if (probability(SLOW_PATH_PROBABILITY, layoutHelper >= 0)) { |
7537
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
64 DeoptimizeNode.deopt(DeoptimizationAction.None, DeoptimizationReason.RuntimeConstraint); |
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
65 } |
7902
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
66 return layoutHelper; |
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
67 } |
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
68 |
12552
0db1d96dd8f0
ArrayCopySnippets: small cleanup
Bernhard Urban <bernhard.urban@jku.at>
parents:
12551
diff
changeset
|
69 private static void checkLimits(Object src, int srcPos, Object dest, int destPos, int length) { |
12546
d084c3db9c61
ArrayCopySnippets: use slow path probability for deopt branches
Bernhard Urban <bernhard.urban@jku.at>
parents:
10787
diff
changeset
|
70 if (probability(SLOW_PATH_PROBABILITY, srcPos < 0)) { |
7537
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
71 checkAIOOBECounter.inc(); |
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
72 DeoptimizeNode.deopt(DeoptimizationAction.None, DeoptimizationReason.RuntimeConstraint); |
3733
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
73 } |
12546
d084c3db9c61
ArrayCopySnippets: use slow path probability for deopt branches
Bernhard Urban <bernhard.urban@jku.at>
parents:
10787
diff
changeset
|
74 if (probability(SLOW_PATH_PROBABILITY, destPos < 0)) { |
7537
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
75 checkAIOOBECounter.inc(); |
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
76 DeoptimizeNode.deopt(DeoptimizationAction.None, DeoptimizationReason.RuntimeConstraint); |
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
77 } |
12546
d084c3db9c61
ArrayCopySnippets: use slow path probability for deopt branches
Bernhard Urban <bernhard.urban@jku.at>
parents:
10787
diff
changeset
|
78 if (probability(SLOW_PATH_PROBABILITY, length < 0)) { |
7510
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
79 checkAIOOBECounter.inc(); |
7537
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
80 DeoptimizeNode.deopt(DeoptimizationAction.None, DeoptimizationReason.RuntimeConstraint); |
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
81 } |
12546
d084c3db9c61
ArrayCopySnippets: use slow path probability for deopt branches
Bernhard Urban <bernhard.urban@jku.at>
parents:
10787
diff
changeset
|
82 if (probability(SLOW_PATH_PROBABILITY, srcPos + length > ArrayLengthNode.arrayLength(src))) { |
7537
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
83 checkAIOOBECounter.inc(); |
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
84 DeoptimizeNode.deopt(DeoptimizationAction.None, DeoptimizationReason.RuntimeConstraint); |
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
85 } |
12546
d084c3db9c61
ArrayCopySnippets: use slow path probability for deopt branches
Bernhard Urban <bernhard.urban@jku.at>
parents:
10787
diff
changeset
|
86 if (probability(SLOW_PATH_PROBABILITY, destPos + length > ArrayLengthNode.arrayLength(dest))) { |
7537
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
87 checkAIOOBECounter.inc(); |
263a666388b1
arraycopy snippets: deopt on NPE and AIOOBE, add probabilities
Lukas Stadler <lukas.stadler@jku.at>
parents:
7510
diff
changeset
|
88 DeoptimizeNode.deopt(DeoptimizationAction.None, DeoptimizationReason.RuntimeConstraint); |
3733
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
89 } |
7510
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
90 checkSuccessCounter.inc(); |
7116
e83ee37dae40
Factor out the precondition checking code in ArrayCopySnippets
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
6390
diff
changeset
|
91 } |
e83ee37dae40
Factor out the precondition checking code in ArrayCopySnippets
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
6390
diff
changeset
|
92 |
e83ee37dae40
Factor out the precondition checking code in ArrayCopySnippets
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
6390
diff
changeset
|
93 @Snippet |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
94 public static void arraycopyZeroLengthIntrinsic(Object src, int srcPos, Object dest, int destPos, int length) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
95 Object nonNullSrc = guardingNonNull(src); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
96 Object nonNullDest = guardingNonNull(dest); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
97 KlassPointer srcHub = loadHub(nonNullSrc); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
98 KlassPointer destHub = loadHub(nonNullDest); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
99 checkArrayType(srcHub); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
100 checkArrayType(destHub); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
101 checkLimits(nonNullSrc, srcPos, nonNullDest, destPos, length); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
102 zeroLengthStaticCounter.inc(); |
4412
ccbdd90bf356
Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4249
diff
changeset
|
103 } |
ccbdd90bf356
Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4249
diff
changeset
|
104 |
ccbdd90bf356
Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents:
4249
diff
changeset
|
105 @Snippet |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
106 public static void arraycopyExactIntrinsic(Object src, int srcPos, Object dest, int destPos, int length, @ConstantParameter Kind elementKind, @ConstantParameter SnippetCounter counter) { |
18593
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
107 Object nonNullSrc = guardingNonNull(src); |
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
108 Object nonNullDest = guardingNonNull(dest); |
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
109 checkLimits(nonNullSrc, srcPos, nonNullDest, destPos, length); |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
110 counter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
111 ArrayCopyCallNode.arraycopy(nonNullSrc, srcPos, nonNullDest, destPos, length, elementKind); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
112 if (length == 0) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
113 zeroLengthDynamicCounter.inc(); |
18600
f6ca61099649
arraycopy: don't use checkcast stub if the arrays overlap
Bernhard Urban <bernhard.urban@jku.at>
parents:
18596
diff
changeset
|
114 } else { |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
115 nonZeroLengthDynamicCounter.inc(); |
18600
f6ca61099649
arraycopy: don't use checkcast stub if the arrays overlap
Bernhard Urban <bernhard.urban@jku.at>
parents:
18596
diff
changeset
|
116 } |
18593
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
117 } |
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
118 |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
119 /** |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
120 * This intrinsic is useful for the case where we know something statically about one of the |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
121 * inputs but not the other. |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
122 */ |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
123 @Snippet |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
124 public static void arraycopyPredictedExactIntrinsic(Object src, int srcPos, Object dest, int destPos, int length, @ConstantParameter Kind elementKind, @ConstantParameter SnippetCounter counter) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
125 Object nonNullSrc = guardingNonNull(src); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
126 Object nonNullDest = guardingNonNull(dest); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
127 KlassPointer srcHub = loadHub(nonNullSrc); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
128 KlassPointer destHub = loadHub(nonNullDest); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
129 if (probability(SLOW_PATH_PROBABILITY, srcHub != destHub)) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
130 DeoptimizeNode.deopt(DeoptimizationAction.None, DeoptimizationReason.RuntimeConstraint); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
131 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
132 checkLimits(nonNullSrc, srcPos, nonNullDest, destPos, length); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
133 counter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
134 ArrayCopyCallNode.arraycopy(nonNullSrc, srcPos, nonNullDest, destPos, length, elementKind); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
135 if (length == 0) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
136 zeroLengthDynamicCounter.inc(); |
21109
05e3ec9d5aa2
Make sure an after state is available when lowering checkcast arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
18611
diff
changeset
|
137 } else { |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
138 nonZeroLengthDynamicCounter.inc(); |
18593
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
139 } |
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
140 } |
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
141 |
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
142 @Snippet |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
143 public static void arraycopyPredictedObjectWork(Object nonNullSrc, int srcPos, Object nonNullDest, int destPos, int length, KlassPointer objectArrayKlass, @ConstantParameter SnippetCounter counter) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
144 if (length > 0) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
145 KlassPointer srcHub = loadHub(nonNullSrc); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
146 KlassPointer destHub = loadHub(nonNullDest); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
147 if (probability(FAST_PATH_PROBABILITY, srcHub == destHub || destHub == objectArrayKlass)) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
148 counter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
149 predictedObjectArrayCopyFastPathCounter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
150 ArrayCopyCallNode.arraycopyObjectKillsAny(nonNullSrc, srcPos, nonNullDest, destPos, length); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
151 } else { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
152 predictedObjectArrayCopySlowPathCounter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
153 System.arraycopy(nonNullSrc, srcPos, nonNullDest, destPos, length); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
154 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
155 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
156 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
157 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
158 /** |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
159 * This is the basic template for the full arraycopy checks, including a check that the |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
160 * underlying type is really an array type. |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
161 */ |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
162 @Snippet |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
163 public static void arraycopySlowPathIntrinsic(Object src, int srcPos, Object dest, int destPos, int length, @ConstantParameter Kind elementKind, @ConstantParameter SnippetInfo slowPath) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
164 Object nonNullSrc = guardingNonNull(src); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
165 Object nonNullDest = guardingNonNull(dest); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
166 KlassPointer srcHub = loadHub(nonNullSrc); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
167 KlassPointer destHub = loadHub(nonNullDest); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
168 checkArrayType(srcHub); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
169 checkArrayType(destHub); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
170 checkLimits(nonNullSrc, srcPos, nonNullDest, destPos, length); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
171 if (length == 0) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
172 zeroLengthDynamicCounter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
173 } else { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
174 nonZeroLengthDynamicCounter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
175 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
176 ArrayCopySlowPathNode.arraycopy(nonNullSrc, srcPos, nonNullDest, destPos, length, elementKind, slowPath); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
177 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
178 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
179 @Snippet |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
180 public static void checkcastArraycopyWork(Object nonNullSrc, int srcPos, Object nonNullDest, int destPos, int length) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
181 if (length > 0) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
182 KlassPointer destKlass = loadHub(nonNullDest); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
183 KlassPointer srcKlass = loadHub(nonNullSrc); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
184 if (probability(SLOW_PATH_PROBABILITY, srcKlass == destKlass)) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
185 // no storecheck required. |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
186 objectCheckcastSameTypeCounter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
187 ArrayCopyCallNode.arraycopyObjectKillsAny(nonNullSrc, srcPos, nonNullDest, destPos, length); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
188 } else { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
189 KlassPointer destElemKlass = destKlass.readKlassPointer(arrayClassElementOffset(), OBJ_ARRAY_KLASS_ELEMENT_KLASS_LOCATION); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
190 Word superCheckOffset = Word.signed(destElemKlass.readInt(superCheckOffsetOffset(), KLASS_SUPER_CHECK_OFFSET_LOCATION)); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
191 objectCheckcastCounter.inc(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
192 int copiedElements = CheckcastArrayCopyCallNode.checkcastArraycopy(nonNullSrc, srcPos, nonNullDest, destPos, length, superCheckOffset, destElemKlass, false); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
193 if (copiedElements != 0) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
194 /* |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
195 * the checkcast stub doesn't throw the ArrayStoreException, but returns the |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
196 * number of copied elements (xor'd with -1). |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
197 */ |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
198 copiedElements ^= -1; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
199 System.arraycopy(nonNullSrc, srcPos + copiedElements, nonNullDest, destPos + copiedElements, length - copiedElements); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
200 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
201 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
202 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
203 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
204 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
205 @Snippet |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
206 public static void arraycopyGeneric(Object src, int srcPos, Object dest, int destPos, int length) { |
10691
77b83e903703
make use of guardingNonNull node intrinsic
Doug Simon <doug.simon@oracle.com>
parents:
10637
diff
changeset
|
207 Object nonNullSrc = guardingNonNull(src); |
77b83e903703
make use of guardingNonNull node intrinsic
Doug Simon <doug.simon@oracle.com>
parents:
10637
diff
changeset
|
208 Object nonNullDest = guardingNonNull(dest); |
18401
e7ab82e7cc37
Move metaspace pointer handling to hotspot specific WordTypeRewriter.
Roland Schatz <roland.schatz@oracle.com>
parents:
18377
diff
changeset
|
209 KlassPointer srcHub = loadHub(nonNullSrc); |
e7ab82e7cc37
Move metaspace pointer handling to hotspot specific WordTypeRewriter.
Roland Schatz <roland.schatz@oracle.com>
parents:
18377
diff
changeset
|
210 KlassPointer destHub = loadHub(nonNullDest); |
e7ab82e7cc37
Move metaspace pointer handling to hotspot specific WordTypeRewriter.
Roland Schatz <roland.schatz@oracle.com>
parents:
18377
diff
changeset
|
211 if (probability(FAST_PATH_PROBABILITY, srcHub.equal(destHub)) && probability(FAST_PATH_PROBABILITY, nonNullSrc != nonNullDest)) { |
10436
8c8285e345cc
Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents:
9864
diff
changeset
|
212 int layoutHelper = checkArrayType(srcHub); |
8c8285e345cc
Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents:
9864
diff
changeset
|
213 final boolean isObjectArray = ((layoutHelper & layoutHelperElementTypePrimitiveInPlace()) == 0); |
10637
ba1fbbfac0cd
remove null check semantics from LoadHubNode (GRAAL-248)
Doug Simon <doug.simon@oracle.com>
parents:
10438
diff
changeset
|
214 checkLimits(nonNullSrc, srcPos, nonNullDest, destPos, length); |
9316
5e1465ec46d6
Change the way branch probabilities are injected. Update all snippets.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
9015
diff
changeset
|
215 if (probability(FAST_PATH_PROBABILITY, isObjectArray)) { |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8634
diff
changeset
|
216 genericObjectExactCallCounter.inc(); |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
217 ArrayCopyCallNode.disjointArraycopy(nonNullSrc, srcPos, nonNullDest, destPos, length, Kind.Object); |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8634
diff
changeset
|
218 } else { |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8634
diff
changeset
|
219 genericPrimitiveCallCounter.inc(); |
10637
ba1fbbfac0cd
remove null check semantics from LoadHubNode (GRAAL-248)
Doug Simon <doug.simon@oracle.com>
parents:
10438
diff
changeset
|
220 UnsafeArrayCopyNode.arraycopyPrimitive(nonNullSrc, srcPos, nonNullDest, destPos, length, layoutHelper); |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8634
diff
changeset
|
221 } |
7902
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
222 } else { |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
223 SystemArraycopyCounter.inc(); |
10637
ba1fbbfac0cd
remove null check semantics from LoadHubNode (GRAAL-248)
Doug Simon <doug.simon@oracle.com>
parents:
10438
diff
changeset
|
224 System.arraycopy(nonNullSrc, srcPos, nonNullDest, destPos, length); |
7902
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
225 } |
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
226 } |
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
227 |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
228 @Fold |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
229 private static LocationIdentity getArrayLocation(Kind kind) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
230 return NamedLocationIdentity.getArrayLocation(kind); |
13933
ebd2dfc2b780
use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
12552
diff
changeset
|
231 } |
ebd2dfc2b780
use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
12552
diff
changeset
|
232 |
ebd2dfc2b780
use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
12552
diff
changeset
|
233 @Snippet |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
234 public static void arraycopyUnrolledWork(Object nonNullSrc, int srcPos, Object nonNullDest, int destPos, @ConstantParameter int length, @ConstantParameter Kind elementKind) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
235 final int scale = arrayIndexScale(elementKind); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
236 int arrayBaseOffset = arrayBaseOffset(elementKind); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
237 LocationIdentity arrayLocation = getArrayLocation(elementKind); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
238 if (nonNullSrc == nonNullDest && srcPos < destPos) { // bad aliased case |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
239 long start = (long) (length - 1) * scale; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
240 long i = start; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
241 ExplodeLoopNode.explodeLoop(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
242 for (int iteration = 0; iteration < length; iteration++) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
243 if (i >= 0) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
244 Object a = UnsafeLoadNode.load(nonNullSrc, arrayBaseOffset + i + (long) srcPos * scale, elementKind, arrayLocation); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
245 DirectObjectStoreNode.storeObject(nonNullDest, arrayBaseOffset, i + (long) destPos * scale, a, arrayLocation, elementKind); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
246 i -= scale; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
247 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
248 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
249 } else { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
250 long end = (long) length * scale; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
251 long i = 0; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
252 ExplodeLoopNode.explodeLoop(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
253 for (int iteration = 0; iteration < length; iteration++) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
254 if (i < end) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
255 Object a = UnsafeLoadNode.load(nonNullSrc, arrayBaseOffset + i + (long) srcPos * scale, elementKind, arrayLocation); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
256 DirectObjectStoreNode.storeObject(nonNullDest, arrayBaseOffset, i + (long) destPos * scale, a, arrayLocation, elementKind); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
257 i += scale; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
258 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
259 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
260 } |
13933
ebd2dfc2b780
use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
12552
diff
changeset
|
261 } |
ebd2dfc2b780
use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
12552
diff
changeset
|
262 |
9864
063a712fe8d8
converted remaining options in GraalOptions to new system (GRAAL-27)
Doug Simon <doug.simon@oracle.com>
parents:
9793
diff
changeset
|
263 private static final SnippetCounter.Group checkCounters = SnippetCounters.getValue() ? new SnippetCounter.Group("System.arraycopy checkInputs") : null; |
7510
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
264 private static final SnippetCounter checkSuccessCounter = new SnippetCounter(checkCounters, "checkSuccess", "checkSuccess"); |
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
265 private static final SnippetCounter checkAIOOBECounter = new SnippetCounter(checkCounters, "checkAIOOBE", "checkAIOOBE"); |
225002aba5a5
added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents:
7262
diff
changeset
|
266 |
9864
063a712fe8d8
converted remaining options in GraalOptions to new system (GRAAL-27)
Doug Simon <doug.simon@oracle.com>
parents:
9793
diff
changeset
|
267 private static final SnippetCounter.Group counters = SnippetCounters.getValue() ? new SnippetCounter.Group("System.arraycopy") : null; |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
268 |
18593
58ea6b98cb09
ArrayCopy: use hotspot stub for checkcastArraycopy.
Bernhard Urban <bernhard.urban@jku.at>
parents:
18591
diff
changeset
|
269 private static final SnippetCounter objectCheckcastCounter = new SnippetCounter(counters, "Object[]", "arraycopy for non-exact Object[] arrays"); |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
270 private static final SnippetCounter objectCheckcastSameTypeCounter = new SnippetCounter(counters, "Object[]", "arraycopy call for src.klass == dest.klass Object[] arrays"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
271 private static final SnippetCounter predictedObjectArrayCopySlowPathCounter = new SnippetCounter(counters, "Object[]", "used System.arraycopy slow path for predicted Object[] arrays"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
272 private static final SnippetCounter predictedObjectArrayCopyFastPathCounter = new SnippetCounter(counters, "Object[]", "used oop_arraycopy for predicted Object[] arrays"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
273 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
274 private static final EnumMap<Kind, SnippetCounter> arraycopyCallCounters = new EnumMap<>(Kind.class); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
275 private static final EnumMap<Kind, SnippetCounter> arraycopyCounters = new EnumMap<>(Kind.class); |
13933
ebd2dfc2b780
use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
12552
diff
changeset
|
276 |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
277 static void createArraycopyCounter(Kind kind) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
278 arraycopyCallCounters.put(kind, new SnippetCounter(counters, kind + "[]", "arraycopy call for " + kind + "[] arrays")); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
279 arraycopyCounters.put(kind, new SnippetCounter(counters, kind + "[]", "inline arraycopy for " + kind + "[] arrays")); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
280 } |
13977
39076a984c33
lower arraycopy calls later and support unchecked object arraycopy
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
13933
diff
changeset
|
281 |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
282 static { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
283 createArraycopyCounter(Kind.Byte); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
284 createArraycopyCounter(Kind.Boolean); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
285 createArraycopyCounter(Kind.Char); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
286 createArraycopyCounter(Kind.Short); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
287 createArraycopyCounter(Kind.Int); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
288 createArraycopyCounter(Kind.Long); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
289 createArraycopyCounter(Kind.Float); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
290 createArraycopyCounter(Kind.Double); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
291 createArraycopyCounter(Kind.Object); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
292 } |
13933
ebd2dfc2b780
use hotspot stubs for primitive arraycopy calls
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
12552
diff
changeset
|
293 |
7902
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
294 private static final SnippetCounter genericPrimitiveCallCounter = new SnippetCounter(counters, "genericPrimitive", "generic arraycopy snippet for primitive arrays"); |
14fedab0419e
improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents:
7868
diff
changeset
|
295 private static final SnippetCounter genericObjectExactCallCounter = new SnippetCounter(counters, "genericObjectExact", "generic arraycopy snippet for special object arrays"); |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
296 private static final SnippetCounter SystemArraycopyCounter = new SnippetCounter(counters, "genericObject", "call to System.arraycopy"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
297 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
298 private static final SnippetCounter.Group lengthCounters = SnippetCounters.getValue() ? new SnippetCounter.Group("System.arraycopy 0-length checks") : null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
299 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
300 private static final SnippetCounter zeroLengthStaticCounter = new SnippetCounter(lengthCounters, "0-lengthcopy static", "arraycopy where the length is statically 0"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
301 private static final SnippetCounter zeroLengthDynamicCounter = new SnippetCounter(lengthCounters, "0-lengthcopy dynamically", "arraycopy where the length is dynamically 0"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
302 private static final SnippetCounter nonZeroLengthDynamicCounter = new SnippetCounter(lengthCounters, "non-0-lengthcopy dynamically", "arraycopy where the length is dynamically not zero"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
303 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
304 public static class Templates extends SnippetTemplate.AbstractTemplates { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
305 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
306 public Templates(HotSpotProviders providers, TargetDescription target) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
307 super(providers, providers.getSnippetReflection(), target); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
308 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
309 |
21543
93c50cefb9e8
moved GraalInternalError to com.oracle.jvmci.common and renamed it to JVMCIError (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents:
21528
diff
changeset
|
310 private ResolvedJavaMethod originalArraycopy() throws JVMCIError { |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
311 if (originalArraycopy == null) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
312 Method method; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
313 try { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
314 method = System.class.getDeclaredMethod("arraycopy", Object.class, int.class, Object.class, int.class, int.class); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
315 } catch (NoSuchMethodException | SecurityException e) { |
21543
93c50cefb9e8
moved GraalInternalError to com.oracle.jvmci.common and renamed it to JVMCIError (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents:
21528
diff
changeset
|
316 throw new JVMCIError(e); |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
317 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
318 originalArraycopy = providers.getMetaAccess().lookupJavaMethod(method); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
319 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
320 return originalArraycopy; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
321 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
322 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
323 private ResolvedJavaMethod originalArraycopy; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
324 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
325 private final SnippetInfo checkcastArraycopyWorkSnippet = snippet("checkcastArraycopyWork"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
326 private final SnippetInfo arraycopyGenericSnippet = snippet("arraycopyGeneric"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
327 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
328 private final SnippetInfo arraycopySlowPathIntrinsicSnippet = snippet("arraycopySlowPathIntrinsic"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
329 private final SnippetInfo arraycopyExactIntrinsicSnippet = snippet("arraycopyExactIntrinsic"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
330 private final SnippetInfo arraycopyZeroLengthIntrinsicSnippet = snippet("arraycopyZeroLengthIntrinsic"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
331 private final SnippetInfo arraycopyPredictedExactIntrinsicSnippet = snippet("arraycopyPredictedExactIntrinsic"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
332 private final SnippetInfo arraycopyPredictedObjectWorkSnippet = snippet("arraycopyPredictedObjectWork"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
333 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
334 private final SnippetInfo arraycopyUnrolledWorkSnippet = snippet("arraycopyUnrolledWork"); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
335 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
336 protected SnippetInfo snippet(String methodName) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
337 SnippetInfo info = snippet(ArrayCopySnippets.class, methodName, LocationIdentity.any()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
338 info.setOriginalMethod(originalArraycopy()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
339 return info; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
340 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
341 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
342 public static Kind selectComponentKind(BasicArrayCopyNode arraycopy) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
343 return selectComponentKind(arraycopy, true); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
344 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
345 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
346 public static Kind selectComponentKind(BasicArrayCopyNode arraycopy, boolean exact) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
347 ResolvedJavaType srcType = StampTool.typeOrNull(arraycopy.getSource().stamp()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
348 ResolvedJavaType destType = StampTool.typeOrNull(arraycopy.getDestination().stamp()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
349 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
350 if (srcType == null || !srcType.isArray() || destType == null || !destType.isArray()) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
351 if (!exact) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
352 Kind component = getComponentKind(srcType); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
353 if (component != null) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
354 return component; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
355 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
356 return getComponentKind(destType); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
357 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
358 return null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
359 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
360 if (exact) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
361 if (!destType.getComponentType().isAssignableFrom(srcType.getComponentType())) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
362 return null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
363 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
364 if (!arraycopy.isExact()) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
365 return null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
366 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
367 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
368 return srcType.getComponentType().getKind(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
369 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
370 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
371 private static Kind getComponentKind(ResolvedJavaType type) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
372 if (type != null && type.isArray()) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
373 return type.getComponentType().getKind(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
374 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
375 return null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
376 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
377 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
378 private static boolean shouldUnroll(ValueNode length) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
379 return length.isConstant() && length.asJavaConstant().asInt() <= 8 && length.asJavaConstant().asInt() != 0; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
380 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
381 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
382 private static void unrollFixedLengthLoop(StructuredGraph snippetGraph, int length, LoweringTool tool) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
383 ParameterNode lengthParam = snippetGraph.getParameter(4); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
384 if (lengthParam != null) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
385 snippetGraph.replaceFloating(lengthParam, ConstantNode.forInt(length, snippetGraph)); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
386 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
387 // the canonicalization before loop unrolling is needed to propagate the length into |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
388 // additions, etc. |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
389 PhaseContext context = new PhaseContext(tool.getMetaAccess(), tool.getConstantReflection(), tool.getLowerer(), tool.getReplacements(), tool.getStampProvider()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
390 new CanonicalizerPhase().apply(snippetGraph, context); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
391 new LoopFullUnrollPhase(new CanonicalizerPhase()).apply(snippetGraph, context); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
392 new CanonicalizerPhase().apply(snippetGraph, context); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
393 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
394 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
395 void unrollSnippet(final LoweringTool tool, StructuredGraph snippetGraph, ArrayCopyNode arraycopy) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
396 if (shouldUnroll(arraycopy.getLength())) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
397 final StructuredGraph copy = snippetGraph; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
398 try (Scope s = Debug.scope("ArrayCopySnippetSpecialization", snippetGraph.method())) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
399 unrollFixedLengthLoop(copy, arraycopy.getLength().asJavaConstant().asInt(), tool); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
400 } catch (Throwable e) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
401 throw Debug.handle(e); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
402 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
403 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
404 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
405 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
406 public void lower(ArrayCopyNode arraycopy, LoweringTool tool) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
407 Kind componentKind = selectComponentKind(arraycopy); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
408 SnippetInfo snippetInfo = null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
409 SnippetInfo slowPathSnippetInfo = null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
410 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
411 if (arraycopy.getLength().isConstant() && arraycopy.getLength().asJavaConstant().asLong() == 0) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
412 snippetInfo = arraycopyZeroLengthIntrinsicSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
413 } else if (arraycopy.isExact()) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
414 snippetInfo = arraycopyExactIntrinsicSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
415 if (shouldUnroll(arraycopy.getLength())) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
416 snippetInfo = arraycopySlowPathIntrinsicSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
417 slowPathSnippetInfo = arraycopyUnrolledWorkSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
418 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
419 } else { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
420 if (componentKind == Kind.Object) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
421 ResolvedJavaType srcType = StampTool.typeOrNull(arraycopy.getSource().stamp()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
422 ResolvedJavaType destType = StampTool.typeOrNull(arraycopy.getDestination().stamp()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
423 ResolvedJavaType srcComponentType = srcType == null ? null : srcType.getComponentType(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
424 ResolvedJavaType destComponentType = destType == null ? null : destType.getComponentType(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
425 if (srcComponentType != null && destComponentType != null && !srcComponentType.isPrimitive() && !destComponentType.isPrimitive()) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
426 snippetInfo = arraycopySlowPathIntrinsicSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
427 slowPathSnippetInfo = checkcastArraycopyWorkSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
428 /* |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
429 * Because this snippet has to use Sysytem.arraycopy as a slow path, we must |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
430 * pretend to kill any() so clear the componentKind. |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
431 */ |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
432 componentKind = null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
433 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
434 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
435 if (componentKind == null && snippetInfo == null) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
436 Kind predictedKind = selectComponentKind(arraycopy, false); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
437 if (predictedKind != null) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
438 /* |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
439 * At least one array is of a known type requiring no store checks, so |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
440 * assume the other is of the same type. Generally this is working around |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
441 * deficiencies in our propation of type information. |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
442 */ |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
443 componentKind = predictedKind; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
444 if (predictedKind == Kind.Object) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
445 snippetInfo = arraycopySlowPathIntrinsicSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
446 slowPathSnippetInfo = arraycopyPredictedObjectWorkSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
447 componentKind = null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
448 } else { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
449 snippetInfo = arraycopyPredictedExactIntrinsicSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
450 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
451 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
452 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
453 if (snippetInfo == null) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
454 snippetInfo = arraycopyGenericSnippet; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
455 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
456 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
457 Arguments args = new Arguments(snippetInfo, arraycopy.graph().getGuardsStage(), tool.getLoweringStage()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
458 args.add("src", arraycopy.getSource()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
459 args.add("srcPos", arraycopy.getSourcePosition()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
460 args.add("dest", arraycopy.getDestination()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
461 args.add("destPos", arraycopy.getDestinationPosition()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
462 args.add("length", arraycopy.getLength()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
463 if (snippetInfo == arraycopySlowPathIntrinsicSnippet) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
464 args.addConst("elementKind", componentKind != null ? componentKind : Kind.Illegal); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
465 args.addConst("slowPath", slowPathSnippetInfo); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
466 } else if (snippetInfo == arraycopyExactIntrinsicSnippet || snippetInfo == arraycopyPredictedExactIntrinsicSnippet) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
467 assert componentKind != null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
468 args.addConst("elementKind", componentKind); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
469 args.addConst("counter", arraycopyCallCounters.get(componentKind)); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
470 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
471 instantiate(args, arraycopy); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
472 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
473 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
474 public void lower(ArrayCopySlowPathNode arraycopy, LoweringTool tool) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
475 StructuredGraph graph = arraycopy.graph(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
476 if (!graph.getGuardsStage().areFrameStatesAtDeopts()) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
477 // Can't be lowered yet |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
478 return; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
479 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
480 SnippetInfo snippetInfo = arraycopy.getSnippet(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
481 Arguments args = new Arguments(snippetInfo, graph.getGuardsStage(), tool.getLoweringStage()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
482 args.add("nonNullSrc", arraycopy.getSource()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
483 args.add("srcPos", arraycopy.getSourcePosition()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
484 args.add("nonNullDest", arraycopy.getDestination()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
485 args.add("destPos", arraycopy.getDestinationPosition()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
486 if (snippetInfo == arraycopyUnrolledWorkSnippet) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
487 args.addConst("length", arraycopy.getLength().asJavaConstant().asInt()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
488 args.addConst("elementKind", arraycopy.getElementKind()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
489 } else { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
490 args.add("length", arraycopy.getLength()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
491 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
492 if (snippetInfo == arraycopyPredictedObjectWorkSnippet) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
493 HotSpotResolvedObjectType arrayKlass = (HotSpotResolvedObjectType) tool.getMetaAccess().lookupJavaType(Object[].class); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
494 ValueNode objectArrayKlass = ConstantNode.forConstant(KlassPointerStamp.klassNonNull(), arrayKlass.klass(), tool.getMetaAccess(), arraycopy.graph()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
495 args.add("objectArrayKlass", objectArrayKlass); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
496 args.addConst("counter", arraycopyCallCounters.get(Kind.Object)); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
497 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
498 instantiate(args, arraycopy); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
499 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
500 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
501 /** |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
502 * Instantiate the snippet template and fix up the FrameState of any Invokes of |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
503 * System.arraycopy and propagate the captured bci in the ArrayCopySlowPathNode. |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
504 * |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
505 * @param args |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
506 * @param arraycopy |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
507 */ |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
508 private void instantiate(Arguments args, BasicArrayCopyNode arraycopy) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
509 StructuredGraph graph = arraycopy.graph(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
510 SnippetTemplate template = template(args); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
511 Map<Node, Node> replacements = template.instantiate(providers.getMetaAccess(), arraycopy, SnippetTemplate.DEFAULT_REPLACER, args); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
512 for (Node originalNode : replacements.keySet()) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
513 if (originalNode instanceof Invoke) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
514 Invoke invoke = (Invoke) replacements.get(originalNode); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
515 assert invoke.asNode().graph() == graph; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
516 CallTargetNode call = invoke.callTarget(); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
517 |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
518 if (!call.targetMethod().equals(originalArraycopy)) { |
21543
93c50cefb9e8
moved GraalInternalError to com.oracle.jvmci.common and renamed it to JVMCIError (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents:
21528
diff
changeset
|
519 throw new JVMCIError("unexpected invoke %s in snippet", call.targetMethod()); |
21463
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
520 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
521 // Here we need to fix the bci of the invoke |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
522 InvokeNode newInvoke = graph.add(new InvokeNode(invoke.callTarget(), arraycopy.getBci())); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
523 if (arraycopy.stateDuring() != null) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
524 newInvoke.setStateDuring(arraycopy.stateDuring()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
525 } else { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
526 assert arraycopy.stateAfter() != null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
527 newInvoke.setStateAfter(arraycopy.stateAfter()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
528 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
529 graph.replaceFixedWithFixed((InvokeNode) invoke.asNode(), newInvoke); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
530 } else if (originalNode instanceof ArrayCopySlowPathNode) { |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
531 ArrayCopySlowPathNode slowPath = (ArrayCopySlowPathNode) replacements.get(originalNode); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
532 assert arraycopy.stateAfter() != null; |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
533 slowPath.setStateAfter(arraycopy.stateAfter()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
534 slowPath.setBci(arraycopy.getBci()); |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
535 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
536 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
537 } |
b1072d72fa2e
Lower checkcast arraycopy idiom after frame state assignment and convert arraycopy lowering into snippets
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
21275
diff
changeset
|
538 } |
3733
e233f5660da4
Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff
changeset
|
539 } |