Mercurial > hg > truffle

annotate graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/stubs/NewInstanceStub.java @ 9232:bb2447c64055

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
strings used in compiled stubs are lowered to malloc'ed C strings so that there are no embedded oops (for the strings) in the resulting installed code
author Doug Simon <doug.simon@oracle.com>
date Mon, 22 Apr 2013 17:59:14 +0200
parents 92d2bedb5dfc
children d4684b468e93
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
1 /*
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
2 * Copyright (c) 2012, Oracle and/or its affiliates. All rights reserved.
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
4 *
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
5 * This code is free software; you can redistribute it and/or modify it
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
6 * under the terms of the GNU General Public License version 2 only, as
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
7 * published by the Free Software Foundation.
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
8 *
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
9 * This code is distributed in the hope that it will be useful, but WITHOUT
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
12 * version 2 for more details (a copy is included in the LICENSE file that
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
13 * accompanied this code).
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
14 *
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
15 * You should have received a copy of the GNU General Public License version
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
16 * 2 along with this work; if not, write to the Free Software Foundation,
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
18 *
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
20 * or visit www.oracle.com if you need additional information or have any
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
21 * questions.
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
22 */
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
23 package com.oracle.graal.hotspot.stubs;
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
24
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
25 import static com.oracle.graal.hotspot.nodes.DirectCompareAndSwapNode.*;
8415
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 7868
diff changeset
26 import static com.oracle.graal.hotspot.replacements.HotSpotSnippetUtils.*;
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 7868
diff changeset
27 import static com.oracle.graal.hotspot.replacements.NewObjectSnippets.*;
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
28
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
29 import com.oracle.graal.api.code.*;
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
30 import com.oracle.graal.api.meta.*;
8627
75db7afee829 implemented lazy installation of replacements (GRAAL-137)
Doug Simon <doug.simon@oracle.com>
parents: 8508
diff changeset
31 import com.oracle.graal.hotspot.*;
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
32 import com.oracle.graal.hotspot.meta.*;
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
33 import com.oracle.graal.hotspot.nodes.*;
8415
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 7868
diff changeset
34 import com.oracle.graal.hotspot.replacements.*;
8637
ce5750014c3d moved Replacements and MacroSubstitution from the graal.api.replacements project to graal.nodes project and reversed the dependency between these two projects (the latter now/again depends on the former)
Doug Simon <doug.simon@oracle.com>
parents: 8627
diff changeset
35 import com.oracle.graal.nodes.spi.*;
8415
2361bf148c06 rename packages: *snippets* -> *replacements*
Doug Simon <doug.simon@oracle.com>
parents: 7868
diff changeset
36 import com.oracle.graal.replacements.*;
8627
75db7afee829 implemented lazy installation of replacements (GRAAL-137)
Doug Simon <doug.simon@oracle.com>
parents: 8508
diff changeset
37 import com.oracle.graal.replacements.Snippet.ConstantParameter;
75db7afee829 implemented lazy installation of replacements (GRAAL-137)
Doug Simon <doug.simon@oracle.com>
parents: 8508
diff changeset
38 import com.oracle.graal.replacements.Snippet.Fold;
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
39 import com.oracle.graal.replacements.SnippetTemplate.Arguments;
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
40 import com.oracle.graal.replacements.SnippetTemplate.SnippetInfo;
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
41 import com.oracle.graal.word.*;
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
42
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
43 /**
7530
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
44 * Stub implementing the fast path for TLAB refill during instance class allocation. This stub is
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
45 * called from the {@linkplain NewObjectSnippets inline} allocation code when TLAB allocation fails.
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
46 * If this stub fails to refill the TLAB or allocate the object, it calls out to the HotSpot C++
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
47 * runtime for to complete the allocation.
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
48 */
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
49 public class NewInstanceStub extends Stub {
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
50
8627
75db7afee829 implemented lazy installation of replacements (GRAAL-137)
Doug Simon <doug.simon@oracle.com>
parents: 8508
diff changeset
51 public NewInstanceStub(final HotSpotRuntime runtime, Replacements replacements, TargetDescription target, HotSpotRuntimeCallTarget linkage) {
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
52 super(runtime, replacements, target, linkage, "newInstance");
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
53 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
54
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
55 @Override
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
56 protected Arguments makeArguments(SnippetInfo stub) {
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
57 HotSpotResolvedObjectType intArrayType = (HotSpotResolvedObjectType) runtime.lookupJavaType(int[].class);
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
58
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
59 Arguments args = new Arguments(stub);
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
60 args.add("hub", null);
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
61 args.addConst("intArrayHub", intArrayType.klass());
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
62 args.addConst("log", Boolean.getBoolean("graal.logNewInstanceStub"));
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
63 return args;
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
64 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
65
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
66 /**
7530
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
67 * Re-attempts allocation after an initial TLAB allocation failed or was skipped (e.g., due to
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
68 * -XX:-UseTLAB).
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
69 *
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
70 * @param hub the hub of the object to be allocated
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
71 * @param intArrayHub the hub for {@code int[].class}
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
72 */
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
73 @Snippet
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8893
diff changeset
74 private static Object newInstance(Word hub, @ConstantParameter Word intArrayHub, @ConstantParameter boolean log) {
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
75 int sizeInBytes = hub.readInt(klassInstanceSizeOffset(), FINAL_LOCATION);
7238
7f55fd5f897d introduced graal.newArrayStub.forceSlowPath and graal.newInstanceStub.forceSlowPath system properties
Doug Simon <doug.simon@oracle.com>
parents: 7237
diff changeset
76 if (!forceSlowPath() && inlineContiguousAllocationSupported()) {
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
77 if (hub.readInt(klassStateOffset(), CLASS_STATE_LOCATION) == klassStateFullyInitialized()) {
7253
c66968130037 replicate c1 tlab refill behavior more closely, fixes problem with huge tlab sizes
Lukas Stadler <lukas.stadler@jku.at>
parents: 7238
diff changeset
78 Word memory = refillAllocate(intArrayHub, sizeInBytes, log);
7701
47467b2c3fc5 Use equal() and notEqual() instead of == and != to compare words
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7556
diff changeset
79 if (memory.notEqual(0)) {
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
80 Word prototypeMarkWord = hub.readWord(prototypeMarkWordOffset(), PROTOTYPE_MARK_WORD_LOCATION);
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
81 initializeObjectHeader(memory, prototypeMarkWord, hub);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
82 for (int offset = 2 * wordSize(); offset < sizeInBytes; offset += wordSize()) {
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
83 memory.writeWord(offset, Word.zero(), ANY_LOCATION);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
84 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
85 return verifyOop(memory.toObject());
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
86 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
87 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
88 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
89 return verifyOop(NewInstanceSlowStubCall.call(hub));
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
90 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
91
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
92 /**
7253
c66968130037 replicate c1 tlab refill behavior more closely, fixes problem with huge tlab sizes
Lukas Stadler <lukas.stadler@jku.at>
parents: 7238
diff changeset
93 * Attempts to refill the current thread's TLAB and retries the allocation.
7530
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
94 *
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
95 * @param intArrayHub the hub for {@code int[].class}
7253
c66968130037 replicate c1 tlab refill behavior more closely, fixes problem with huge tlab sizes
Lukas Stadler <lukas.stadler@jku.at>
parents: 7238
diff changeset
96 * @param sizeInBytes the size of the allocation
7237
509d72a1d81b made the effect of graal.logNewArrayStub independent from graal.logNewInstanceStub
Doug Simon <doug.simon@oracle.com>
parents: 7147
diff changeset
97 * @param log specifies if logging is enabled
7530
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
98 * @return the newly allocated, uninitialized chunk of memory, or {@link Word#zero()} if the
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
99 * operation was unsuccessful
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
100 */
7253
c66968130037 replicate c1 tlab refill behavior more closely, fixes problem with huge tlab sizes
Lukas Stadler <lukas.stadler@jku.at>
parents: 7238
diff changeset
101 static Word refillAllocate(Word intArrayHub, int sizeInBytes, boolean log) {
8502
ff4fa8b0516e -Create push patch
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8501
diff changeset
102 if (!useTLAB()) {
ff4fa8b0516e -Create push patch
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8501
diff changeset
103 return edenAllocate(Word.unsigned(sizeInBytes), log);
ff4fa8b0516e -Create push patch
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8501
diff changeset
104 }
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
105 Word intArrayMarkWord = Word.unsigned(tlabIntArrayMarkWord());
7253
c66968130037 replicate c1 tlab refill behavior more closely, fixes problem with huge tlab sizes
Lukas Stadler <lukas.stadler@jku.at>
parents: 7238
diff changeset
106 int alignmentReserveInBytes = tlabAlignmentReserveInHeapWords() * wordSize();
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
107
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
108 Word thread = thread();
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
109 Word top = readTlabTop(thread);
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
110 Word end = readTlabEnd(thread);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
111
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
112 // calculate amount of free space
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
113 Word tlabFreeSpaceInBytes = end.subtract(top);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
114
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
115 log(log, "refillTLAB: thread=%p\n", thread);
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
116 log(log, "refillTLAB: top=%p\n", top);
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
117 log(log, "refillTLAB: end=%p\n", end);
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
118 log(log, "refillTLAB: tlabFreeSpaceInBytes=%d\n", tlabFreeSpaceInBytes);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
119
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
120 Word tlabFreeSpaceInWords = tlabFreeSpaceInBytes.unsignedShiftRight(log2WordSize());
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
121
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
122 // Retain TLAB and allocate object in shared space if
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
123 // the amount free in the TLAB is too large to discard.
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
124 Word refillWasteLimit = thread.readWord(tlabRefillWasteLimitOffset(), TLAB_REFILL_WASTE_LIMIT_LOCATION);
7237
509d72a1d81b made the effect of graal.logNewArrayStub independent from graal.logNewInstanceStub
Doug Simon <doug.simon@oracle.com>
parents: 7147
diff changeset
125 if (tlabFreeSpaceInWords.belowOrEqual(refillWasteLimit)) {
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
126 if (tlabStats()) {
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
127 // increment number of refills
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
128 thread.writeInt(tlabNumberOfRefillsOffset(), thread.readInt(tlabNumberOfRefillsOffset(), TLAB_NOF_REFILLS_LOCATION) + 1, TLAB_NOF_REFILLS_LOCATION);
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
129 log(log, "thread: %p -- number_of_refills %d\n", thread, thread.readInt(tlabNumberOfRefillsOffset(), TLAB_NOF_REFILLS_LOCATION));
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
130 // accumulate wastage
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
131 Word wastage = thread.readWord(tlabFastRefillWasteOffset(), TLAB_FAST_REFILL_WASTE_LOCATION).add(tlabFreeSpaceInWords);
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
132 log(log, "thread: %p -- accumulated wastage %d\n", thread, wastage);
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
133 thread.writeWord(tlabFastRefillWasteOffset(), wastage, TLAB_FAST_REFILL_WASTE_LOCATION);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
134 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
135
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
136 // if TLAB is currently allocated (top or end != null) then
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
137 // fill [top, end + alignment_reserve) with array object
7701
47467b2c3fc5 Use equal() and notEqual() instead of == and != to compare words
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7556
diff changeset
138 if (top.notEqual(0)) {
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
139 int headerSize = arrayBaseOffset(Kind.Int);
7530
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
140 // just like the HotSpot assembler stubs, assumes that tlabFreeSpaceInInts fits in
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
141 // an int
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
142 int tlabFreeSpaceInInts = (int) tlabFreeSpaceInBytes.rawValue() >>> 2;
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
143 int length = ((alignmentReserveInBytes - headerSize) >>> 2) + tlabFreeSpaceInInts;
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
144 NewObjectSnippets.formatArray(intArrayHub, -1, length, headerSize, top, intArrayMarkWord, false);
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
145
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
146 Word allocated = thread.readWord(threadAllocatedBytesOffset(), TLAB_THREAD_ALLOCATED_BYTES_LOCATION);
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
147 allocated = allocated.add(top.subtract(readTlabStart(thread)));
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
148 thread.writeWord(threadAllocatedBytesOffset(), allocated, TLAB_THREAD_ALLOCATED_BYTES_LOCATION);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
149 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
150
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
151 // refill the TLAB with an eden allocation
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
152 Word tlabRefillSizeInWords = thread.readWord(threadTlabSizeOffset(), TLAB_SIZE_LOCATION);
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
153 Word tlabRefillSizeInBytes = tlabRefillSizeInWords.multiply(wordSize());
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
154 // allocate new TLAB, address returned in top
7237
509d72a1d81b made the effect of graal.logNewArrayStub independent from graal.logNewInstanceStub
Doug Simon <doug.simon@oracle.com>
parents: 7147
diff changeset
155 top = edenAllocate(tlabRefillSizeInBytes, log);
7701
47467b2c3fc5 Use equal() and notEqual() instead of == and != to compare words
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7556
diff changeset
156 if (top.notEqual(0)) {
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
157 end = top.add(tlabRefillSizeInBytes.subtract(alignmentReserveInBytes));
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
158 initializeTlab(thread, top, end);
7253
c66968130037 replicate c1 tlab refill behavior more closely, fixes problem with huge tlab sizes
Lukas Stadler <lukas.stadler@jku.at>
parents: 7238
diff changeset
159
c66968130037 replicate c1 tlab refill behavior more closely, fixes problem with huge tlab sizes
Lukas Stadler <lukas.stadler@jku.at>
parents: 7238
diff changeset
160 return allocate(sizeInBytes);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
161 } else {
7253
c66968130037 replicate c1 tlab refill behavior more closely, fixes problem with huge tlab sizes
Lukas Stadler <lukas.stadler@jku.at>
parents: 7238
diff changeset
162 return Word.zero();
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
163 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
164 } else {
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
165 // Retain TLAB
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
166 Word newRefillWasteLimit = refillWasteLimit.add(tlabRefillWasteIncrement());
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
167 thread.writeWord(tlabRefillWasteLimitOffset(), newRefillWasteLimit, TLAB_REFILL_WASTE_LIMIT_LOCATION);
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
168 log(log, "refillTLAB: retaining TLAB - newRefillWasteLimit=%p\n", newRefillWasteLimit);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
169
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
170 if (tlabStats()) {
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
171 thread.writeInt(tlabSlowAllocationsOffset(), thread.readInt(tlabSlowAllocationsOffset(), TLAB_SLOW_ALLOCATIONS_LOCATION) + 1, TLAB_SLOW_ALLOCATIONS_LOCATION);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
172 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
173
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
174 return edenAllocate(Word.unsigned(sizeInBytes), log);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
175 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
176 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
177
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
178 /**
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
179 * Attempts to allocate a chunk of memory from Eden space.
7530
5e3d1a68664e applied mx eclipseformat to all Java files
Doug Simon <doug.simon@oracle.com>
parents: 7356
diff changeset
180 *
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
181 * @param sizeInBytes the size of the chunk to allocate
7237
509d72a1d81b made the effect of graal.logNewArrayStub independent from graal.logNewInstanceStub
Doug Simon <doug.simon@oracle.com>
parents: 7147
diff changeset
182 * @param log specifies if logging is enabled
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
183 * @return the allocated chunk or {@link Word#zero()} if allocation fails
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
184 */
7237
509d72a1d81b made the effect of graal.logNewArrayStub independent from graal.logNewInstanceStub
Doug Simon <doug.simon@oracle.com>
parents: 7147
diff changeset
185 static Word edenAllocate(Word sizeInBytes, boolean log) {
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
186 Word heapTopAddress = Word.unsigned(heapTopAddress());
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
187 Word heapEndAddress = Word.unsigned(heapEndAddress());
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
188
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
189 while (true) {
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
190 Word heapTop = heapTopAddress.readWord(0, HEAP_TOP_LOCATION);
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
191 Word newHeapTop = heapTop.add(sizeInBytes);
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
192 if (newHeapTop.belowOrEqual(heapTop)) {
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
193 return Word.zero();
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
194 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
195
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
196 Word heapEnd = heapEndAddress.readWord(0, HEAP_END_LOCATION);
7300
2912b72d840a More complete and reusable Word type
Christian Wimmer <christian.wimmer@oracle.com>
parents: 7275
diff changeset
197 if (newHeapTop.aboveThan(heapEnd)) {
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
198 return Word.zero();
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
199 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
200
7868
7a5bbcc36bb2 add location identity to Pointer read and write operations
Lukas Stadler <lukas.stadler@jku.at>
parents: 7701
diff changeset
201 if (compareAndSwap(heapTopAddress, 0, heapTop, newHeapTop, HEAP_TOP_LOCATION).equal(heapTop)) {
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
202 return heapTop;
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
203 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
204 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
205 }
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
206
7238
7f55fd5f897d introduced graal.newArrayStub.forceSlowPath and graal.newInstanceStub.forceSlowPath system properties
Doug Simon <doug.simon@oracle.com>
parents: 7237
diff changeset
207 @Fold
7f55fd5f897d introduced graal.newArrayStub.forceSlowPath and graal.newInstanceStub.forceSlowPath system properties
Doug Simon <doug.simon@oracle.com>
parents: 7237
diff changeset
208 private static boolean forceSlowPath() {
7f55fd5f897d introduced graal.newArrayStub.forceSlowPath and graal.newInstanceStub.forceSlowPath system properties
Doug Simon <doug.simon@oracle.com>
parents: 7237
diff changeset
209 return Boolean.getBoolean("graal.newInstanceStub.forceSlowPath");
7f55fd5f897d introduced graal.newArrayStub.forceSlowPath and graal.newInstanceStub.forceSlowPath system properties
Doug Simon <doug.simon@oracle.com>
parents: 7237
diff changeset
210 }
7143
445193cc2a7d added support for writing stubs in Java and wrote the TLAB fast refill stub
Doug Simon <doug.simon@oracle.com>
parents:
diff changeset
211 }