Mercurial > hg > truffle

annotate graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/replacements/WriteBarrierSnippets.java @ 18661:aa4f2e3629ca

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added card table PIC support
author adlertz
date Thu, 11 Dec 2014 19:35:02 +0100
parents d138867d61c4
children 14e703edb2ab
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
1 /*
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
2 * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved.
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
4 *
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
5 * This code is free software; you can redistribute it and/or modify it
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
6 * under the terms of the GNU General Public License version 2 only, as
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
7 * published by the Free Software Foundation.
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
8 *
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
9 * This code is distributed in the hope that it will be useful, but WITHOUT
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
12 * version 2 for more details (a copy is included in the LICENSE file that
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
13 * accompanied this code).
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
14 *
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
15 * You should have received a copy of the GNU General Public License version
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
16 * 2 along with this work; if not, write to the Free Software Foundation,
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
18 *
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
20 * or visit www.oracle.com if you need additional information or have any
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
21 * questions.
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
22 */
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
23 package com.oracle.graal.hotspot.replacements;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
24
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
25 import static com.oracle.graal.api.code.MemoryBarriers.*;
15259
d90e5c22ba55 Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents: 15018
diff changeset
26 import static com.oracle.graal.compiler.common.GraalOptions.*;
9513
659bb6cf930c rename: HotSpotSnippetUtils -> HotSpotReplacementsUtil
Doug Simon <doug.simon@oracle.com>
parents: 9015
diff changeset
27 import static com.oracle.graal.hotspot.replacements.HotSpotReplacementsUtil.*;
10787
388fbd0dd4a4 moved BranchNodeProbabilities to graal.nodes project
Doug Simon <doug.simon@oracle.com>
parents: 10701
diff changeset
28 import static com.oracle.graal.nodes.extended.BranchProbabilityNode.*;
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
29 import static com.oracle.graal.replacements.SnippetTemplate.*;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
30
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
31 import com.oracle.graal.api.code.*;
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
32 import com.oracle.graal.api.meta.*;
15259
d90e5c22ba55 Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents: 15018
diff changeset
33 import com.oracle.graal.compiler.common.*;
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
34 import com.oracle.graal.graph.Node.ConstantNodeParameter;
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
35 import com.oracle.graal.graph.Node.NodeIntrinsic;
14991
64dcb92ee75a Truffle: Change signature for Truffle calls from (PackedFrame, Arguments) to (Object[]).
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 14945
diff changeset
36 import com.oracle.graal.hotspot.HotSpotVMConfig.CompressEncoding;
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
37 import com.oracle.graal.hotspot.meta.*;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
38 import com.oracle.graal.hotspot.nodes.*;
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
39 import com.oracle.graal.hotspot.nodes.type.*;
8905
c502db57e687 Rename ArrayWriteBarrier to SerialWriteBarrier and move it to graal.nodes
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8904
diff changeset
40 import com.oracle.graal.nodes.*;
10828
b05e1ff3aac0 Rename WriteBarrierType to BarrierType
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10817
diff changeset
41 import com.oracle.graal.nodes.HeapAccess.BarrierType;
10517
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
42 import com.oracle.graal.nodes.extended.*;
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
43 import com.oracle.graal.nodes.spi.*;
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
44 import com.oracle.graal.replacements.*;
8903
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
45 import com.oracle.graal.replacements.Snippet.ConstantParameter;
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
46 import com.oracle.graal.replacements.SnippetTemplate.AbstractTemplates;
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
47 import com.oracle.graal.replacements.SnippetTemplate.Arguments;
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
48 import com.oracle.graal.replacements.SnippetTemplate.SnippetInfo;
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
49 import com.oracle.graal.replacements.nodes.*;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
50 import com.oracle.graal.word.*;
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
51
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
52 public class WriteBarrierSnippets implements Snippets {
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
53
10481
29e9a5d18c70 Clean up.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 9513
diff changeset
54 private static final SnippetCounter.Group countersWriteBarriers = SnippetCounters.getValue() ? new SnippetCounter.Group("WriteBarriers") : null;
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
55 private static final SnippetCounter serialWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "serialWriteBarrier", "Number of Serial Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
56 private static final SnippetCounter g1AttemptedPreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1AttemptedPreWriteBarrier", "Number of G1 attempted Pre Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
57 private static final SnippetCounter g1EffectivePreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectivePreWriteBarrier", "Number of G1 effective Pre Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
58 private static final SnippetCounter g1ExecutedPreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1ExecutedPreWriteBarrier", "Number of G1 executed Pre Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
59 private static final SnippetCounter g1AttemptedPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1AttemptedPostWriteBarrier", "Number of attempted G1 Post Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
60 private static final SnippetCounter g1EffectiveAfterXORPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectiveAfterXORPostWriteBarrier",
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
61 "Number of effective G1 Post Write Barriers (after passing the XOR test)");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
62 private static final SnippetCounter g1EffectiveAfterNullPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectiveAfterNullPostWriteBarrier",
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
63 "Number of effective G1 Post Write Barriers (after passing the NULL test)");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
64 private static final SnippetCounter g1ExecutedPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1ExecutedPostWriteBarrier", "Number of executed G1 Post Write Barriers");
10481
29e9a5d18c70 Clean up.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 9513
diff changeset
65
18307
85f5227a7a37 add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 18259
diff changeset
66 public static final LocationIdentity GC_CARD_LOCATION = NamedLocationIdentity.mutable("GC-Card");
85f5227a7a37 add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 18259
diff changeset
67 public static final LocationIdentity GC_LOG_LOCATION = NamedLocationIdentity.mutable("GC-Log");
85f5227a7a37 add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 18259
diff changeset
68 public static final LocationIdentity GC_INDEX_LOCATION = NamedLocationIdentity.mutable("GC-Index");
11769
427cbe2bd615 SerialWriteBarrier: use GC_CARD_LOCATION instead of ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11732
diff changeset
69
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
70 @Snippet
12746
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
71 public static void serialWriteBarrier(Object object, Object location, @ConstantParameter boolean usePrecise) {
10944
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
72 Object fixedObject = FixedValueAnchorNode.getObject(object);
8903
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
73 Pointer oop;
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
74 if (usePrecise) {
15004
9ecd3da04309 use temporary proxy for locations in snippets
Lukas Stadler <lukas.stadler@oracle.com>
parents: 14991
diff changeset
75 oop = Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location));
8903
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
76 } else {
10944
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
77 oop = Word.fromObject(fixedObject);
8903
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
78 }
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
79 serialWriteBarrierCounter.inc();
18661
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
80 int cardTableShift = (isImmutableCode() && generatePIC()) ? CardTableShiftNode.cardTableShift() : cardTableShift();
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
81 long cardTableAddress = (isImmutableCode() && generatePIC()) ? CardTableAddressNode.cardTableAddress() : cardTableStart();
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
82 Word base = (Word) oop.unsignedShiftRight(cardTableShift);
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
83 long startAddress = cardTableAddress;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
84 int displacement = 0;
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
85 if (((int) startAddress) == startAddress) {
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
86 displacement = (int) startAddress;
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
87 } else {
18661
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
88 base = base.add(Word.unsigned(cardTableAddress));
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
89 }
11769
427cbe2bd615 SerialWriteBarrier: use GC_CARD_LOCATION instead of ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11732
diff changeset
90 base.writeByte(displacement, (byte) 0, GC_CARD_LOCATION);
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
91 }
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
92
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
93 @Snippet
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
94 public static void serialArrayRangeWriteBarrier(Object object, int startIndex, int length) {
10942
56a86b60c9d3 Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10941
diff changeset
95 if (length == 0) {
56a86b60c9d3 Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10941
diff changeset
96 return;
56a86b60c9d3 Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10941
diff changeset
97 }
8989
40e7444d62ee Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8985
diff changeset
98 Object dest = FixedValueAnchorNode.getObject(object);
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
99 int cardShift = cardTableShift();
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
100 long cardStart = cardTableStart();
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
101 final int scale = arrayIndexScale(Kind.Object);
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
102 int header = arrayBaseOffset(Kind.Object);
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
103 long dstAddr = GetObjectAddressNode.get(dest);
8989
40e7444d62ee Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8985
diff changeset
104 long start = (dstAddr + header + (long) startIndex * scale) >>> cardShift;
40e7444d62ee Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8985
diff changeset
105 long end = (dstAddr + header + ((long) startIndex + length - 1) * scale) >>> cardShift;
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
106 long count = end - start + 1;
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
107 while (count-- > 0) {
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
108 DirectStoreNode.store((start + cardStart) + count, false, Kind.Boolean);
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
109 }
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
110 }
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
111
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
112 @Snippet
10623
13c6440ac750 Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10622
diff changeset
113 public static void g1PreWriteBarrier(Object object, Object expectedObject, Object location, @ConstantParameter boolean doLoad, @ConstantParameter boolean nullCheck,
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
114 @ConstantParameter Register threadRegister, @ConstantParameter boolean trace) {
10623
13c6440ac750 Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10622
diff changeset
115 if (nullCheck && object == null) {
13c6440ac750 Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10622
diff changeset
116 DeoptimizeNode.deopt(DeoptimizationAction.InvalidateReprofile, DeoptimizationReason.NullCheckException);
13c6440ac750 Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10622
diff changeset
117 }
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
118 Word thread = registerAsWord(threadRegister);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
119 Object fixedObject = FixedValueAnchorNode.getObject(object);
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
120 verifyOop(fixedObject);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
121 Object fixedExpectedObject = FixedValueAnchorNode.getObject(expectedObject);
15004
9ecd3da04309 use temporary proxy for locations in snippets
Lukas Stadler <lukas.stadler@oracle.com>
parents: 14991
diff changeset
122 Word field = (Word) Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location));
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
123 Word previousOop = (Word) Word.fromObject(fixedExpectedObject);
10520
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
124 byte markingValue = thread.readByte(g1SATBQueueMarkingOffset());
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
125 Word bufferAddress = thread.readWord(g1SATBQueueBufferOffset());
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
126 Word indexAddress = thread.add(g1SATBQueueIndexOffset());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
127 Word indexValue = indexAddress.readWord(0);
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
128 int gcCycle = 0;
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
129 if (trace) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
130 gcCycle = (int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
131 log(trace, "[%d] G1-Pre Thread %p Object %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedObject).rawValue());
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
132 log(trace, "[%d] G1-Pre Thread %p Expected Object %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedExpectedObject).rawValue());
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
133 log(trace, "[%d] G1-Pre Thread %p Field %p\n", gcCycle, thread.rawValue(), field.rawValue());
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
134 log(trace, "[%d] G1-Pre Thread %p Marking %d\n", gcCycle, thread.rawValue(), markingValue);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
135 log(trace, "[%d] G1-Pre Thread %p DoLoad %d\n", gcCycle, thread.rawValue(), doLoad ? 1L : 0L);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
136 }
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
137 g1AttemptedPreWriteBarrierCounter.inc();
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
138 // If the concurrent marker is enabled, the barrier is issued.
11819
1cb614d6d25b Readjust branch probabilities of G1 Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11817
diff changeset
139 if (probability(NOT_FREQUENT_PROBABILITY, markingValue != (byte) 0)) {
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
140 // If the previous value has to be loaded (before the write), the load is issued.
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
141 // The load is always issued except the cases of CAS and referent field.
10622
844be189150c Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10620
diff changeset
142 if (probability(LIKELY_PROBABILITY, doLoad)) {
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
143 previousOop = (Word) Word.fromObject(field.readObject(0, BarrierType.NONE));
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
144 if (trace) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
145 log(trace, "[%d] G1-Pre Thread %p Previous Object %p\n ", gcCycle, thread.rawValue(), previousOop.rawValue());
10631
2e82291febf4 Oop verification after printing oop
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10630
diff changeset
146 verifyOop(previousOop.toObject());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
147 }
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
148 }
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
149 g1EffectivePreWriteBarrierCounter.inc();
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
150 // If the previous value is null the barrier should not be issued.
10622
844be189150c Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10620
diff changeset
151 if (probability(FREQUENT_PROBABILITY, previousOop.notEqual(0))) {
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
152 g1ExecutedPreWriteBarrierCounter.inc();
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
153 // If the thread-local SATB buffer is full issue a native call which will
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
154 // initialize a new one and add the entry.
10622
844be189150c Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10620
diff changeset
155 if (probability(FREQUENT_PROBABILITY, indexValue.notEqual(0))) {
10520
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
156 Word nextIndex = indexValue.subtract(wordSize());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
157 Word logAddress = bufferAddress.add(nextIndex);
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
158 // Log the object to be marked as well as update the SATB's buffer next index.
11774
f6eb4866d558 G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11769
diff changeset
159 logAddress.writeWord(0, previousOop, GC_LOG_LOCATION);
f6eb4866d558 G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11769
diff changeset
160 indexAddress.writeWord(0, nextIndex, GC_INDEX_LOCATION);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
161 } else {
10517
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
162 g1PreBarrierStub(G1WBPRECALL, previousOop.toObject());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
163 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
164 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
165 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
166 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
167
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
168 @Snippet
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
169 public static void g1PostWriteBarrier(Object object, Object value, Object location, @ConstantParameter boolean usePrecise, @ConstantParameter Register threadRegister,
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
170 @ConstantParameter boolean trace) {
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
171 Word thread = registerAsWord(threadRegister);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
172 Object fixedObject = FixedValueAnchorNode.getObject(object);
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
173 Object fixedValue = FixedValueAnchorNode.getObject(value);
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
174 verifyOop(fixedObject);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
175 verifyOop(fixedValue);
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
176 validateObject(fixedObject, fixedValue);
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
177 Word oop;
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
178 if (usePrecise) {
15004
9ecd3da04309 use temporary proxy for locations in snippets
Lukas Stadler <lukas.stadler@oracle.com>
parents: 14991
diff changeset
179 oop = (Word) Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location));
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
180 } else {
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
181 oop = (Word) Word.fromObject(fixedObject);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
182 }
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
183 int gcCycle = 0;
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
184 if (trace) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
185 gcCycle = (int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0);
10630
ca3865947682 Debug log cleanup
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10623
diff changeset
186 log(trace, "[%d] G1-Post Thread: %p Object: %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedObject).rawValue());
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
187 log(trace, "[%d] G1-Post Thread: %p Field: %p\n", gcCycle, thread.rawValue(), oop.rawValue());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
188 }
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
189 Word writtenValue = (Word) Word.fromObject(fixedValue);
10520
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
190 Word bufferAddress = thread.readWord(g1CardQueueBufferOffset());
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
191 Word indexAddress = thread.add(g1CardQueueIndexOffset());
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
192 Word indexValue = thread.readWord(g1CardQueueIndexOffset());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
193 // The result of the xor reveals whether the installed pointer crosses heap regions.
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
194 // In case it does the write barrier has to be issued.
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
195 Word xorResult = (oop.xor(writtenValue)).unsignedShiftRight(logOfHeapRegionGrainBytes());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
196
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
197 // Calculate the address of the card to be enqueued to the
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
198 // thread local card queue.
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
199 Word cardBase = oop.unsignedShiftRight(cardTableShift());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
200 long startAddress = cardTableStart();
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
201 int displacement = 0;
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
202 if (((int) startAddress) == startAddress) {
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
203 displacement = (int) startAddress;
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
204 } else {
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
205 cardBase = cardBase.add(Word.unsigned(cardTableStart()));
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
206 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
207 Word cardAddress = cardBase.add(displacement);
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
208
10955
769ad79b8e0c Add G1 Barrier Counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10954
diff changeset
209 g1AttemptedPostWriteBarrierCounter.inc();
11819
1cb614d6d25b Readjust branch probabilities of G1 Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11817
diff changeset
210 if (probability(FREQUENT_PROBABILITY, xorResult.notEqual(0))) {
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
211 g1EffectiveAfterXORPostWriteBarrierCounter.inc();
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
212
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
213 // If the written value is not null continue with the barrier addition.
10622
844be189150c Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10620
diff changeset
214 if (probability(FREQUENT_PROBABILITY, writtenValue.notEqual(0))) {
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
215 byte cardByte = cardAddress.readByte(0);
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
216 g1EffectiveAfterNullPostWriteBarrierCounter.inc();
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
217
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
218 // If the card is already dirty, (hence already enqueued) skip the insertion.
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
219 if (probability(NOT_FREQUENT_PROBABILITY, cardByte != g1YoungCardValue())) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
220 MembarNode.memoryBarrier(STORE_LOAD);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
221 byte cardByteReload = cardAddress.readByte(0);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
222 if (probability(NOT_FREQUENT_PROBABILITY, cardByteReload != dirtyCardValue())) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
223 log(trace, "[%d] G1-Post Thread: %p Card: %p \n", gcCycle, thread.rawValue(), Word.unsigned(cardByte).rawValue());
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
224 cardAddress.writeByte(0, (byte) 0, GC_CARD_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
225 g1ExecutedPostWriteBarrierCounter.inc();
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
226
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
227 // If the thread local card queue is full, issue a native call which will
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
228 // initialize a new one and add the card entry.
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
229 if (probability(FREQUENT_PROBABILITY, indexValue.notEqual(0))) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
230 Word nextIndex = indexValue.subtract(wordSize());
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
231 Word logAddress = bufferAddress.add(nextIndex);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
232 // Log the object to be scanned as well as update
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
233 // the card queue's next index.
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
234 logAddress.writeWord(0, cardAddress, GC_LOG_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
235 indexAddress.writeWord(0, nextIndex, GC_INDEX_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
236 } else {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
237 g1PostBarrierStub(G1WBPOSTCALL, cardAddress);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
238 }
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
239 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
240 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
241 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
242 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
243 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
244
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
245 @Snippet
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
246 public static void g1ArrayRangePreWriteBarrier(Object object, int startIndex, int length, @ConstantParameter Register threadRegister) {
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
247 Word thread = registerAsWord(threadRegister);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
248 byte markingValue = thread.readByte(g1SATBQueueMarkingOffset());
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
249 // If the concurrent marker is not enabled or the vector length is zero, return.
10951
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
250 if (markingValue == (byte) 0 || length == 0) {
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
251 return;
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
252 }
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
253 Object dest = FixedValueAnchorNode.getObject(object);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
254 Word bufferAddress = thread.readWord(g1SATBQueueBufferOffset());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
255 Word indexAddress = thread.add(g1SATBQueueIndexOffset());
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
256 long dstAddr = GetObjectAddressNode.get(dest);
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
257 long indexValue = indexAddress.readWord(0).rawValue();
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
258 final int scale = arrayIndexScale(Kind.Object);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
259 int header = arrayBaseOffset(Kind.Object);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
260
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
261 for (int i = startIndex; i < length; i++) {
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
262 long address = dstAddr + header + (i * scale);
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
263 Pointer oop = Word.fromObject(Word.unsigned(address).readObject(0, BarrierType.NONE));
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
264 verifyOop(oop.toObject());
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
265 if (oop.notEqual(0)) {
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
266 if (indexValue != 0) {
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
267 indexValue = indexValue - wordSize();
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
268 Word logAddress = bufferAddress.add(Word.unsigned(indexValue));
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
269 // Log the object to be marked as well as update the SATB's buffer next index.
11774
f6eb4866d558 G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11769
diff changeset
270 logAddress.writeWord(0, oop, GC_LOG_LOCATION);
f6eb4866d558 G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11769
diff changeset
271 indexAddress.writeWord(0, Word.unsigned(indexValue), GC_INDEX_LOCATION);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
272 } else {
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
273 g1PreBarrierStub(G1WBPRECALL, oop.toObject());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
274 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
275 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
276 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
277 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
278
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
279 @Snippet
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
280 public static void g1ArrayRangePostWriteBarrier(Object object, int startIndex, int length, @ConstantParameter Register threadRegister) {
10951
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
281 if (length == 0) {
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
282 return;
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
283 }
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
284 Object dest = FixedValueAnchorNode.getObject(object);
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
285 Word thread = registerAsWord(threadRegister);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
286 Word bufferAddress = thread.readWord(g1CardQueueBufferOffset());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
287 Word indexAddress = thread.add(g1CardQueueIndexOffset());
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
288 long indexValue = thread.readWord(g1CardQueueIndexOffset()).rawValue();
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
289
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
290 int cardShift = cardTableShift();
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
291 long cardStart = cardTableStart();
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
292 final int scale = arrayIndexScale(Kind.Object);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
293 int header = arrayBaseOffset(Kind.Object);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
294 long dstAddr = GetObjectAddressNode.get(dest);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
295 long start = (dstAddr + header + (long) startIndex * scale) >>> cardShift;
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
296 long end = (dstAddr + header + ((long) startIndex + length - 1) * scale) >>> cardShift;
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
297 long count = end - start + 1;
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
298
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
299 while (count-- > 0) {
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
300 Word cardAddress = Word.unsigned((start + cardStart) + count);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
301 byte cardByte = cardAddress.readByte(0);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
302 // If the card is already dirty, (hence already enqueued) skip the insertion.
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
303 if (probability(NOT_FREQUENT_PROBABILITY, cardByte != g1YoungCardValue())) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
304 MembarNode.memoryBarrier(STORE_LOAD);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
305 byte cardByteReload = cardAddress.readByte(0);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
306 if (probability(NOT_FREQUENT_PROBABILITY, cardByteReload != dirtyCardValue())) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
307 cardAddress.writeByte(0, (byte) 0, GC_CARD_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
308 // If the thread local card queue is full, issue a native call which will
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
309 // initialize a new one and add the card entry.
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
310 if (indexValue != 0) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
311 indexValue = indexValue - wordSize();
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
312 Word logAddress = bufferAddress.add(Word.unsigned(indexValue));
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
313 // Log the object to be scanned as well as update
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
314 // the card queue's next index.
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
315 logAddress.writeWord(0, cardAddress, GC_LOG_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
316 indexAddress.writeWord(0, Word.unsigned(indexValue), GC_INDEX_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
317 } else {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
318 g1PostBarrierStub(G1WBPOSTCALL, cardAddress);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
319 }
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
320 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
321 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
322 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
323 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
324
10517
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
325 public static final ForeignCallDescriptor G1WBPRECALL = new ForeignCallDescriptor("write_barrier_pre", void.class, Object.class);
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
326
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
327 @NodeIntrinsic(ForeignCallNode.class)
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
328 private static native void g1PreBarrierStub(@ConstantNodeParameter ForeignCallDescriptor descriptor, Object object);
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
329
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
330 public static final ForeignCallDescriptor G1WBPOSTCALL = new ForeignCallDescriptor("write_barrier_post", void.class, Word.class);
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
331
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
332 @NodeIntrinsic(ForeignCallNode.class)
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
333 public static native void g1PostBarrierStub(@ConstantNodeParameter ForeignCallDescriptor descriptor, Word card);
10517
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
334
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
335 public static class Templates extends AbstractTemplates {
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
336
10944
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
337 private final SnippetInfo serialWriteBarrier = snippet(WriteBarrierSnippets.class, "serialWriteBarrier");
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
338 private final SnippetInfo serialArrayRangeWriteBarrier = snippet(WriteBarrierSnippets.class, "serialArrayRangeWriteBarrier");
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
339 private final SnippetInfo g1PreWriteBarrier = snippet(WriteBarrierSnippets.class, "g1PreWriteBarrier");
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
340 private final SnippetInfo g1ReferentReadBarrier = snippet(WriteBarrierSnippets.class, "g1PreWriteBarrier");
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
341 private final SnippetInfo g1PostWriteBarrier = snippet(WriteBarrierSnippets.class, "g1PostWriteBarrier");
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
342 private final SnippetInfo g1ArrayRangePreWriteBarrier = snippet(WriteBarrierSnippets.class, "g1ArrayRangePreWriteBarrier");
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
343 private final SnippetInfo g1ArrayRangePostWriteBarrier = snippet(WriteBarrierSnippets.class, "g1ArrayRangePostWriteBarrier");
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
344
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
345 private final CompressEncoding oopEncoding;
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
346
15018
db4254246f9a Remove Constant.forObject and Constant.asObject to improve compiler/VM separation
Christian Wimmer <christian.wimmer@oracle.com>
parents: 15004
diff changeset
347 public Templates(HotSpotProviders providers, TargetDescription target, CompressEncoding oopEncoding) {
db4254246f9a Remove Constant.forObject and Constant.asObject to improve compiler/VM separation
Christian Wimmer <christian.wimmer@oracle.com>
parents: 15004
diff changeset
348 super(providers, providers.getSnippetReflection(), target);
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
349 this.oopEncoding = oopEncoding;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
350 }
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
351
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
352 public void lower(SerialWriteBarrier writeBarrier, LoweringTool tool) {
12746
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
353 if (writeBarrier.alwaysNull()) {
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
354 writeBarrier.graph().removeFixed(writeBarrier);
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
355 return;
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
356 }
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
357 Arguments args = new Arguments(serialWriteBarrier, writeBarrier.graph().getGuardsStage(), tool.getLoweringStage());
10944
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
358 args.add("object", writeBarrier.getObject());
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
359 args.add("location", writeBarrier.getLocation());
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
360 args.addConst("usePrecise", writeBarrier.usePrecise());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
361 template(args).instantiate(providers.getMetaAccess(), writeBarrier, DEFAULT_REPLACER, args);
8571
0bacc5564d18 Revert WB refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8531
diff changeset
362 }
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
363
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
364 public void lower(SerialArrayRangeWriteBarrier arrayRangeWriteBarrier, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
365 Arguments args = new Arguments(serialArrayRangeWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage());
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
366 args.add("object", arrayRangeWriteBarrier.getObject());
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
367 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex());
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
368 args.add("length", arrayRangeWriteBarrier.getLength());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
369 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args);
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
370 }
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
371
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
372 public void lower(G1PreWriteBarrier writeBarrierPre, HotSpotRegistersProvider registers, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
373 Arguments args = new Arguments(g1PreWriteBarrier, writeBarrierPre.graph().getGuardsStage(), tool.getLoweringStage());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
374 args.add("object", writeBarrierPre.getObject());
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
375
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
376 ValueNode expected = writeBarrierPre.getExpectedObject();
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
377 if (expected != null && expected.stamp() instanceof NarrowOopStamp) {
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
378 assert oopEncoding != null;
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
379 expected = CompressionNode.uncompress(expected, oopEncoding);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
380 }
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
381 args.add("expectedObject", expected);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
382
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
383 args.add("location", writeBarrierPre.getLocation());
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
384 args.addConst("doLoad", writeBarrierPre.doLoad());
10669
f11cd00d4115 Make G1PreWriteBarrier a DeoptimizingNode.
Roland Schatz <roland.schatz@oracle.com>
parents: 10631
diff changeset
385 args.addConst("nullCheck", writeBarrierPre.getNullCheck());
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
386 args.addConst("threadRegister", registers.getThreadRegister());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
387 args.addConst("trace", traceBarrier());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
388 template(args).instantiate(providers.getMetaAccess(), writeBarrierPre, DEFAULT_REPLACER, args);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
389 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
390
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
391 public void lower(G1ReferentFieldReadBarrier readBarrier, HotSpotRegistersProvider registers, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
392 Arguments args = new Arguments(g1ReferentReadBarrier, readBarrier.graph().getGuardsStage(), tool.getLoweringStage());
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
393 args.add("object", readBarrier.getObject());
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
394
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
395 ValueNode expected = readBarrier.getExpectedObject();
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
396 if (expected != null && expected.stamp() instanceof NarrowOopStamp) {
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
397 assert oopEncoding != null;
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
398 expected = CompressionNode.uncompress(expected, oopEncoding);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
399 }
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
400
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
401 args.add("expectedObject", expected);
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
402 args.add("location", readBarrier.getLocation());
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
403 args.addConst("doLoad", readBarrier.doLoad());
10941
b9718191858c G1ReferentFieldReadBarrier is not a deoptimizing node
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10829
diff changeset
404 args.addConst("nullCheck", false);
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
405 args.addConst("threadRegister", registers.getThreadRegister());
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
406 args.addConst("trace", traceBarrier());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
407 template(args).instantiate(providers.getMetaAccess(), readBarrier, DEFAULT_REPLACER, args);
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
408 }
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
409
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
410 public void lower(G1PostWriteBarrier writeBarrierPost, HotSpotRegistersProvider registers, LoweringTool tool) {
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
411 StructuredGraph graph = writeBarrierPost.graph();
12746
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
412 if (writeBarrierPost.alwaysNull()) {
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
413 graph.removeFixed(writeBarrierPost);
12746
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
414 return;
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
415 }
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
416 Arguments args = new Arguments(g1PostWriteBarrier, graph.getGuardsStage(), tool.getLoweringStage());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
417 args.add("object", writeBarrierPost.getObject());
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
418
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
419 ValueNode value = writeBarrierPost.getValue();
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
420 if (value.stamp() instanceof NarrowOopStamp) {
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
421 assert oopEncoding != null;
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
422 value = CompressionNode.uncompress(value, oopEncoding);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
423 }
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
424 args.add("value", value);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
425
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
426 args.add("location", writeBarrierPost.getLocation());
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
427 args.addConst("usePrecise", writeBarrierPost.usePrecise());
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
428 args.addConst("threadRegister", registers.getThreadRegister());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
429 args.addConst("trace", traceBarrier());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
430 template(args).instantiate(providers.getMetaAccess(), writeBarrierPost, DEFAULT_REPLACER, args);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
431 }
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
432
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
433 public void lower(G1ArrayRangePreWriteBarrier arrayRangeWriteBarrier, HotSpotRegistersProvider registers, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
434 Arguments args = new Arguments(g1ArrayRangePreWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage());
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
435 args.add("object", arrayRangeWriteBarrier.getObject());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
436 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
437 args.add("length", arrayRangeWriteBarrier.getLength());
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
438 args.addConst("threadRegister", registers.getThreadRegister());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
439 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
440 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
441
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
442 public void lower(G1ArrayRangePostWriteBarrier arrayRangeWriteBarrier, HotSpotRegistersProvider registers, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
443 Arguments args = new Arguments(g1ArrayRangePostWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage());
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
444 args.add("object", arrayRangeWriteBarrier.getObject());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
445 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
446 args.add("length", arrayRangeWriteBarrier.getLength());
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
447 args.addConst("threadRegister", registers.getThreadRegister());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
448 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
449 }
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
450 }
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
451
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
452 /**
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
453 * Log method of debugging purposes.
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
454 */
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
455 public static void log(boolean enabled, String format, long value) {
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
456 if (enabled) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
457 Log.printf(format, value);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
458 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
459 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
460
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
461 public static void log(boolean enabled, String format, long value1, long value2) {
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
462 if (enabled) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
463 Log.printf(format, value1, value2);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
464 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
465 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
466
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
467 public static void log(boolean enabled, String format, long value1, long value2, long value3) {
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
468 if (enabled) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
469 Log.printf(format, value1, value2, value3);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
470 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
471 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
472
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
473 public static boolean traceBarrier() {
10701
f1cd428fca9f Remove caching of gcStartCycles
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10669
diff changeset
474 return GraalOptions.GCDebugStartCycle.getValue() > 0 && ((int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0) > GraalOptions.GCDebugStartCycle.getValue());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
475 }
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
476
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
477 /**
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
478 * Validation helper method which performs sanity checks on write operations. The addresses of
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
479 * both the object and the value being written are checked in order to determine if they reside
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
480 * in a valid heap region. If an object is stale, an invalid access is performed in order to
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
481 * prematurely crash the VM and debug the stack trace of the faulty method.
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
482 */
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
483 public static void validateObject(Object parent, Object child) {
11221
8d4bd13c6983 Field renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10955
diff changeset
484 if (verifyOops() && child != null && !validateOop(VALIDATE_OBJECT, parent, child)) {
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
485 log(true, "Verification ERROR, Parent: %p Child: %p\n", Word.fromObject(parent).rawValue(), Word.fromObject(child).rawValue());
18649
d138867d61c4 Remove getValueKind() from LocationNode.
Roland Schatz <roland.schatz@oracle.com>
parents: 18307
diff changeset
486 DirectObjectStoreNode.storeObject(null, 0, 0, null, LocationIdentity.ANY_LOCATION, Kind.Object);
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
487 }
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
488 }
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
489
11221
8d4bd13c6983 Field renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10955
diff changeset
490 public static final ForeignCallDescriptor VALIDATE_OBJECT = new ForeignCallDescriptor("validate_object", boolean.class, Word.class, Word.class);
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
491
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
492 @NodeIntrinsic(ForeignCallNode.class)
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
493 private static native boolean validateOop(@ConstantNodeParameter ForeignCallDescriptor descriptor, Object parent, Object object);
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
494
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
495 }