Mercurial > hg > graal-jvmci-8

annotate graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/replacements/UnsafeArrayCopySnippets.java @ 11816:8db5e8c4f542

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
author Christian Wimmer <christian.wimmer@oracle.com>
date Thu, 26 Sep 2013 15:53:21 -0700
parents 03c781923573
children 88fb9cba8751
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
1 /*
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
2 * Copyright (c) 2013, Oracle and/or its affiliates. All rights reserved.
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
4 *
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
5 * This code is free software; you can redistribute it and/or modify it
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
6 * under the terms of the GNU General Public License version 2 only, as
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
7 * published by the Free Software Foundation.
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
8 *
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
9 * This code is distributed in the hope that it will be useful, but WITHOUT
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
12 * version 2 for more details (a copy is included in the LICENSE file that
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
13 * accompanied this code).
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
14 *
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
15 * You should have received a copy of the GNU General Public License version
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
16 * 2 along with this work; if not, write to the Free Software Foundation,
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
18 *
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
20 * or visit www.oracle.com if you need additional information or have any
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
21 * questions.
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
22 */
8415
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 8413
diff changeset
23 package com.oracle.graal.hotspot.replacements;
7530
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7510
diff changeset
24
9793
b4f12c603be5 added support for the runtime to specify for each foreign call whether it is re-executable and what memory locations it kills
Doug Simon <doug.simon@oracle.com>
parents: 9513
diff changeset
25 import static com.oracle.graal.api.meta.LocationIdentity.*;
9513
659bb6cf930c rename: HotSpotSnippetUtils -> HotSpotReplacementsUtil
Doug Simon <doug.simon@oracle.com>
parents: 9318
diff changeset
26 import static com.oracle.graal.hotspot.replacements.HotSpotReplacementsUtil.*;
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
27 import static com.oracle.graal.hotspot.HotSpotGraalRuntime.*;
10787
388fbd0dd4a4 moved BranchNodeProbabilities to graal.nodes project
Doug Simon <doug.simon@oracle.com>
parents: 10436
diff changeset
28 import static com.oracle.graal.nodes.extended.BranchProbabilityNode.*;
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
29 import static com.oracle.graal.replacements.SnippetTemplate.*;
6362
6d232ee6a62d factored shared snippet functionality into HotSpotSnippetUtils
Doug Simon <doug.simon@oracle.com>
parents: 6328
diff changeset
30
7262
5f21ab202edc removed @Snippet annotation from methods called by the array copy snippets
Doug Simon <doug.simon@oracle.com>
parents: 7116
diff changeset
31 import com.oracle.graal.api.code.*;
5507
dc71b06d09f8 Moving classes from cri.ri to api.meta.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 5504
diff changeset
32 import com.oracle.graal.api.meta.*;
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
33 import com.oracle.graal.asm.*;
5060
4ed4295ce15f Update import statements.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 5059
diff changeset
34 import com.oracle.graal.nodes.extended.*;
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
35 import com.oracle.graal.nodes.spi.*;
8415
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 8413
diff changeset
36 import com.oracle.graal.replacements.*;
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
37 import com.oracle.graal.replacements.SnippetTemplate.AbstractTemplates;
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
38 import com.oracle.graal.replacements.SnippetTemplate.Arguments;
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
39 import com.oracle.graal.replacements.SnippetTemplate.SnippetInfo;
8415
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 8413
diff changeset
40 import com.oracle.graal.replacements.nodes.*;
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
41 import com.oracle.graal.word.*;
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
42
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
43 public class UnsafeArrayCopySnippets implements Snippets {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
44
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
45 private static final boolean supportsUnalignedMemoryAccess = graalRuntime().getTarget().arch.supportsUnalignedMemoryAccess();
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
46
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
47 private static final Kind VECTOR_KIND = Kind.Long;
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
48 private static final long VECTOR_SIZE = arrayIndexScale(Kind.Long);
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
49
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8963
diff changeset
50 private static void vectorizedCopy(Object src, int srcPos, Object dest, int destPos, int length, Kind baseKind) {
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
51 int arrayBaseOffset = arrayBaseOffset(baseKind);
5879
77196bba5575 Change one more GraalCodeCacheProvider to MetaAccessProvider
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5770
diff changeset
52 int elementSize = arrayIndexScale(baseKind);
6313
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
53 long byteLength = (long) length * elementSize;
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
54 long srcOffset = (long) srcPos * elementSize;
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
55 long destOffset = (long) destPos * elementSize;
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
56
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
57 long preLoopBytes;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
58 long mainLoopBytes;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
59 long postLoopBytes;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
60
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
61 // We can easily vectorize the loop if both offsets have the same alignment.
10917
92fc1db20ad9 Fix arraycopy bug when array size is smaller than sizeof(long).
Roland Schatz <roland.schatz@oracle.com>
parents: 10910
diff changeset
62 if (byteLength >= VECTOR_SIZE && (srcOffset % VECTOR_SIZE) == (destOffset % VECTOR_SIZE)) {
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
63 preLoopBytes = NumUtil.roundUp(arrayBaseOffset + srcOffset, VECTOR_SIZE) - (arrayBaseOffset + srcOffset);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
64 postLoopBytes = (byteLength - preLoopBytes) % VECTOR_SIZE;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
65 mainLoopBytes = byteLength - preLoopBytes - postLoopBytes;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
66 } else {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
67 // Does the architecture support unaligned memory accesses?
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
68 if (supportsUnalignedMemoryAccess) {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
69 preLoopBytes = byteLength % VECTOR_SIZE;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
70 mainLoopBytes = byteLength - preLoopBytes;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
71 postLoopBytes = 0;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
72 } else {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
73 // No. Let's do element-wise copying.
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
74 preLoopBytes = byteLength;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
75 mainLoopBytes = 0;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
76 postLoopBytes = 0;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
77 }
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
78 }
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
79
9318
dc04c7e8f714 Add javadoc to BranchProbabilityNode.probability node intrinsic. Avoid combined conditions in connection with the probability.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 9316
diff changeset
80 if (probability(NOT_FREQUENT_PROBABILITY, src == dest) && probability(NOT_FREQUENT_PROBABILITY, srcPos < destPos)) {
9316
5e1465ec46d6 Change the way branch probabilities are injected. Update all snippets.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 9015
diff changeset
81 // bad aliased case
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
82 srcOffset += byteLength;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
83 destOffset += byteLength;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
84
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
85 // Post-loop
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
86 for (long i = 0; i < postLoopBytes; i += elementSize) {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
87 srcOffset -= elementSize;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
88 destOffset -= elementSize;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
89 Object a = UnsafeLoadNode.load(src, arrayBaseOffset, srcOffset, baseKind);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
90 UnsafeStoreNode.store(dest, arrayBaseOffset, destOffset, a, baseKind);
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
91 }
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
92 // Main-loop
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
93 for (long i = 0; i < mainLoopBytes; i += VECTOR_SIZE) {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
94 srcOffset -= VECTOR_SIZE;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
95 destOffset -= VECTOR_SIZE;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
96 Long a = UnsafeLoadNode.load(src, arrayBaseOffset, srcOffset, VECTOR_KIND);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
97 UnsafeStoreNode.store(dest, arrayBaseOffset, destOffset, a.longValue(), VECTOR_KIND);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
98 }
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
99 // Pre-loop
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
100 for (long i = 0; i < preLoopBytes; i += elementSize) {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
101 srcOffset -= elementSize;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
102 destOffset -= elementSize;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
103 Object a = UnsafeLoadNode.load(src, arrayBaseOffset, srcOffset, baseKind);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
104 UnsafeStoreNode.store(dest, arrayBaseOffset, destOffset, a, baseKind);
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
105 }
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
106 } else {
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
107 // Pre-loop
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
108 for (long i = 0; i < preLoopBytes; i += elementSize) {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
109 Object a = UnsafeLoadNode.load(src, arrayBaseOffset, srcOffset, baseKind);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
110 UnsafeStoreNode.store(dest, arrayBaseOffset, destOffset, a, baseKind);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
111 srcOffset += elementSize;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
112 destOffset += elementSize;
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
113 }
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
114 // Main-loop
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
115 for (long i = 0; i < mainLoopBytes; i += VECTOR_SIZE) {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
116 Long a = UnsafeLoadNode.load(src, arrayBaseOffset, srcOffset, VECTOR_KIND);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
117 UnsafeStoreNode.store(dest, arrayBaseOffset, destOffset, a.longValue(), VECTOR_KIND);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
118 srcOffset += VECTOR_SIZE;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
119 destOffset += VECTOR_SIZE;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
120 }
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
121 // Post-loop
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
122 for (long i = 0; i < postLoopBytes; i += elementSize) {
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
123 Object a = UnsafeLoadNode.load(src, arrayBaseOffset, srcOffset, baseKind);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
124 UnsafeStoreNode.store(dest, arrayBaseOffset, destOffset, a, baseKind);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
125 srcOffset += elementSize;
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
126 destOffset += elementSize;
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
127 }
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
128 }
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
129 }
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
130
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
131 @Snippet
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
132 public static void arraycopyByte(byte[] src, int srcPos, byte[] dest, int destPos, int length) {
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
133 vectorizedCopy(src, srcPos, dest, destPos, length, Kind.Byte);
7116
e83ee37dae40 Factor out the precondition checking code in ArrayCopySnippets
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6390
diff changeset
134 }
e83ee37dae40 Factor out the precondition checking code in ArrayCopySnippets
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6390
diff changeset
135
e83ee37dae40 Factor out the precondition checking code in ArrayCopySnippets
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 6390
diff changeset
136 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
137 public static void arraycopyBoolean(boolean[] src, int srcPos, boolean[] dest, int destPos, int length) {
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
138 vectorizedCopy(src, srcPos, dest, destPos, length, Kind.Byte);
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
139 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
140
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
141 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
142 public static void arraycopyChar(char[] src, int srcPos, char[] dest, int destPos, int length) {
5879
77196bba5575 Change one more GraalCodeCacheProvider to MetaAccessProvider
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5770
diff changeset
143 vectorizedCopy(src, srcPos, dest, destPos, length, Kind.Char);
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
144 }
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
145
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
146 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
147 public static void arraycopyShort(short[] src, int srcPos, short[] dest, int destPos, int length) {
5879
77196bba5575 Change one more GraalCodeCacheProvider to MetaAccessProvider
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5770
diff changeset
148 vectorizedCopy(src, srcPos, dest, destPos, length, Kind.Short);
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
149 }
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
150
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
151 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
152 public static void arraycopyInt(int[] src, int srcPos, int[] dest, int destPos, int length) {
5879
77196bba5575 Change one more GraalCodeCacheProvider to MetaAccessProvider
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5770
diff changeset
153 vectorizedCopy(src, srcPos, dest, destPos, length, Kind.Int);
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
154 }
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
155
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
156 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
157 public static void arraycopyFloat(float[] src, int srcPos, float[] dest, int destPos, int length) {
5879
77196bba5575 Change one more GraalCodeCacheProvider to MetaAccessProvider
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5770
diff changeset
158 vectorizedCopy(src, srcPos, dest, destPos, length, Kind.Float);
4412
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
159 }
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
160
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
161 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
162 public static void arraycopyLong(long[] src, int srcPos, long[] dest, int destPos, int length) {
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
163 Kind baseKind = Kind.Long;
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
164 int arrayBaseOffset = arrayBaseOffset(baseKind);
6313
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
165 long byteLength = (long) length * arrayIndexScale(baseKind);
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
166 long srcOffset = (long) srcPos * arrayIndexScale(baseKind);
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
167 long destOffset = (long) destPos * arrayIndexScale(baseKind);
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
168 if (src == dest && srcPos < destPos) { // bad aliased case
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
169 for (long i = byteLength - VECTOR_SIZE; i >= 0; i -= VECTOR_SIZE) {
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
170 Long a = UnsafeLoadNode.load(src, arrayBaseOffset, i + srcOffset, VECTOR_KIND);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
171 UnsafeStoreNode.store(dest, arrayBaseOffset, i + destOffset, a.longValue(), VECTOR_KIND);
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
172 }
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
173 } else {
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
174 for (long i = 0; i < byteLength; i += VECTOR_SIZE) {
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
175 Long a = UnsafeLoadNode.load(src, arrayBaseOffset, i + srcOffset, VECTOR_KIND);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
176 UnsafeStoreNode.store(dest, arrayBaseOffset, i + destOffset, a.longValue(), VECTOR_KIND);
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
177 }
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
178 }
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
179 }
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
180
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
181 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
182 public static void arraycopyDouble(double[] src, int srcPos, double[] dest, int destPos, int length) {
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
183 Kind baseKind = Kind.Double;
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
184 int arrayBaseOffset = arrayBaseOffset(baseKind);
6313
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
185 long byteLength = (long) length * arrayIndexScale(baseKind);
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
186 long srcOffset = (long) srcPos * arrayIndexScale(baseKind);
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
187 long destOffset = (long) destPos * arrayIndexScale(baseKind);
4412
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
188 if (src == dest && srcPos < destPos) { // bad aliased case
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
189 for (long i = byteLength - VECTOR_SIZE; i >= 0; i -= VECTOR_SIZE) {
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
190 Long a = UnsafeLoadNode.load(src, arrayBaseOffset, i + srcOffset, VECTOR_KIND);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
191 UnsafeStoreNode.store(dest, arrayBaseOffset, i + destOffset, a.longValue(), VECTOR_KIND);
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
192 }
4412
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
193 } else {
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
194 for (long i = 0; i < byteLength; i += VECTOR_SIZE) {
11816
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
195 /*
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
196 * TODO atomicity problem on 32-bit architectures: The JVM spec requires double
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
197 * values to be copied atomically, but not long values. For example, on Intel 32-bit
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
198 * this code is not atomic as long as the vector kind remains Kind.Long.
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
199 */
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
200 Long a = UnsafeLoadNode.load(src, arrayBaseOffset, i + srcOffset, VECTOR_KIND);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
201 UnsafeStoreNode.store(dest, arrayBaseOffset, i + destOffset, a.longValue(), VECTOR_KIND);
5644
02f76c2ee0f0 Add aliased case array copy jtt tests
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5576
diff changeset
202 }
4412
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
203 }
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
204 }
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
205
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
206 /**
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
207 * Does NOT perform store checks.
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
208 */
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8634
diff changeset
209 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
210 public static void arraycopyObject(Object[] src, int srcPos, Object[] dest, int destPos, int length) {
5540
a891c53a295b Renaming RiKind => Kind.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 5534
diff changeset
211 final int scale = arrayIndexScale(Kind.Object);
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
212 int arrayBaseOffset = arrayBaseOffset(Kind.Object);
4412
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
213 if (src == dest && srcPos < destPos) { // bad aliased case
6313
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
214 long start = (long) (length - 1) * scale;
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
215 for (long i = start; i >= 0; i -= scale) {
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
216 Object a = UnsafeLoadNode.load(src, arrayBaseOffset, i + (long) srcPos * scale, Kind.Object);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
217 DirectObjectStoreNode.storeObject(dest, arrayBaseOffset, i + (long) destPos * scale, a);
5740
98325620b7e2 Array copies must be element atomic
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5714
diff changeset
218 }
4412
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
219 } else {
6313
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
220 long end = (long) length * scale;
7ac010ae8c97 Add inferStamp for a number of ConvertNode's operations
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 5879
diff changeset
221 for (long i = 0; i < end; i += scale) {
10910
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
222 Object a = UnsafeLoadNode.load(src, arrayBaseOffset, i + (long) srcPos * scale, Kind.Object);
b61c13ad27d0 use aligned memory accesses for vectorized array copy
twisti
parents: 10787
diff changeset
223 DirectObjectStoreNode.storeObject(dest, arrayBaseOffset, i + (long) destPos * scale, a);
8568
9323ff2fbd11 Reverse ArrayCopy WB intrisification
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8540
diff changeset
224 }
9323ff2fbd11 Reverse ArrayCopy WB intrisification
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8540
diff changeset
225 }
4412
ccbdd90bf356 Add new arraycopy intrinsics : Object (when no storecheck required), Float and Double
Gilles Duboscq <duboscq@ssw.jku.at>
parents: 4249
diff changeset
226 }
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
227
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
228 @Snippet
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
229 public static void arraycopyPrimitive(Object src, int srcPos, Object dest, int destPos, int length, int layoutHelper) {
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
230 int log2ElementSize = (layoutHelper >> layoutHelperLog2ElementSizeShift()) & layoutHelperLog2ElementSizeMask();
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
231 int headerSize = (layoutHelper >> layoutHelperHeaderSizeShift()) & layoutHelperHeaderSizeMask();
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
232
11816
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
233 Unsigned srcOffset = Word.unsigned(srcPos).shiftLeft(log2ElementSize).add(headerSize);
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
234 Unsigned destOffset = Word.unsigned(destPos).shiftLeft(log2ElementSize).add(headerSize);
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
235 Unsigned destStart = destOffset;
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
236 Unsigned sizeInBytes = Word.unsigned(length).shiftLeft(log2ElementSize);
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
237 Unsigned destEnd = destOffset.add(Word.unsigned(length).shiftLeft(log2ElementSize));
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
238
11816
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
239 Unsigned nonVectorBytes = sizeInBytes.unsignedRemainder(Word.unsigned(VECTOR_SIZE));
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
240 Unsigned destNonVectorEnd = destStart.add(nonVectorBytes);
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
241
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
242 while (destOffset.belowThan(destNonVectorEnd)) {
11816
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
243 ObjectAccess.writeByte(dest, destOffset, ObjectAccess.readByte(src, srcOffset, ANY_LOCATION), ANY_LOCATION);
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
244 destOffset = destOffset.add(1);
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
245 srcOffset = srcOffset.add(1);
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
246 }
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
247 while (destOffset.belowThan(destEnd)) {
11816
8db5e8c4f542 Provide object read/write methods similar to Pointer.readXxxx/writeXxx that do not require casts to Word first, and use them in relevant places
Christian Wimmer <christian.wimmer@oracle.com>
parents: 11732
diff changeset
248 ObjectAccess.writeWord(dest, destOffset, ObjectAccess.readWord(src, srcOffset, ANY_LOCATION), ANY_LOCATION);
7902
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
249 destOffset = destOffset.add(wordSize());
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
250 srcOffset = srcOffset.add(wordSize());
14fedab0419e improved arraycopy and Object.clone snippets
Lukas Stadler <lukas.stadler@jku.at>
parents: 7868
diff changeset
251 }
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
252 }
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
253
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
254 public static class Templates extends AbstractTemplates {
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
255
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
256 private final SnippetInfo[] arraycopySnippets;
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
257 private final SnippetInfo genericPrimitiveSnippet;
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
258
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
259 public Templates(MetaAccessProvider runtime, Replacements replacements, TargetDescription target) {
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
260 super(runtime, replacements, target);
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
261
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
262 arraycopySnippets = new SnippetInfo[Kind.values().length];
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
263 arraycopySnippets[Kind.Boolean.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyBoolean");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
264 arraycopySnippets[Kind.Byte.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyByte");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
265 arraycopySnippets[Kind.Short.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyShort");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
266 arraycopySnippets[Kind.Char.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyChar");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
267 arraycopySnippets[Kind.Int.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyInt");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
268 arraycopySnippets[Kind.Long.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyLong");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
269 arraycopySnippets[Kind.Float.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyFloat");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
270 arraycopySnippets[Kind.Double.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyDouble");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
271 arraycopySnippets[Kind.Object.ordinal()] = snippet(UnsafeArrayCopySnippets.class, "arraycopyObject");
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
272
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
273 genericPrimitiveSnippet = snippet(UnsafeArrayCopySnippets.class, "arraycopyPrimitive");
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
274 }
7510
225002aba5a5 added new macro node facility, removed ArrayCopyIntrinsificationPhase
Lukas Stadler <lukas.stadler@jku.at>
parents: 7262
diff changeset
275
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
276 public void lower(UnsafeArrayCopyNode node) {
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
277 Kind elementKind = node.getElementKind();
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
278 SnippetInfo snippet;
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
279 if (elementKind == null) {
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
280 // primitive array of unknown kind
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
281 snippet = genericPrimitiveSnippet;
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
282 } else {
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
283 snippet = arraycopySnippets[elementKind.ordinal()];
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
284 assert snippet != null : "arraycopy snippet for " + elementKind.name() + " not found";
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
285 }
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
286
11732
03c781923573 made snippet and substitution graphs lower themselves before being inlined
Doug Simon <doug.simon@oracle.com>
parents: 10917
diff changeset
287 Arguments args = new Arguments(snippet, node.graph().getGuardsStage());
10436
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
288 node.addSnippetArguments(args);
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
289
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
290 SnippetTemplate template = template(args);
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
291 template.instantiate(runtime, node, DEFAULT_REPLACER, args);
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
292 }
8c8285e345cc Later lowering of arraycopy.
Roland Schatz <roland.schatz@oracle.com>
parents: 9864
diff changeset
293 }
3733
e233f5660da4 Added Java files from Maxine project.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
diff changeset
294 }