Mercurial > hg > graal-compiler

annotate graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/replacements/WriteBarrierSnippets.java @ 21551:5324104ac4f3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
moved com.oracle.graal.hotspot.jvmci classes to com.oracle.jvmci.hotspot module (JBS:GRAAL-53)
author Doug Simon <doug.simon@oracle.com>
date Tue, 26 May 2015 17:13:37 +0200
parents 93c50cefb9e8
children b1530a6cce8c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
1 /*
20178
7ba8fb19d56d Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 19852
diff changeset
2 * Copyright (c) 2012, 2015, Oracle and/or its affiliates. All rights reserved.
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
4 *
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
5 * This code is free software; you can redistribute it and/or modify it
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
6 * under the terms of the GNU General Public License version 2 only, as
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
7 * published by the Free Software Foundation.
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
8 *
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
9 * This code is distributed in the hope that it will be useful, but WITHOUT
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
12 * version 2 for more details (a copy is included in the LICENSE file that
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
13 * accompanied this code).
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
14 *
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
15 * You should have received a copy of the GNU General Public License version
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
16 * 2 along with this work; if not, write to the Free Software Foundation,
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
18 *
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
20 * or visit www.oracle.com if you need additional information or have any
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
21 * questions.
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
22 */
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
23 package com.oracle.graal.hotspot.replacements;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
24
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
25 import static com.oracle.graal.api.code.MemoryBarriers.*;
15259
d90e5c22ba55 Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents: 15018
diff changeset
26 import static com.oracle.graal.compiler.common.GraalOptions.*;
9513
659bb6cf930c rename: HotSpotSnippetUtils -> HotSpotReplacementsUtil
Doug Simon <doug.simon@oracle.com>
parents: 9015
diff changeset
27 import static com.oracle.graal.hotspot.replacements.HotSpotReplacementsUtil.*;
10787
388fbd0dd4a4 moved BranchNodeProbabilities to graal.nodes project
Doug Simon <doug.simon@oracle.com>
parents: 10701
diff changeset
28 import static com.oracle.graal.nodes.extended.BranchProbabilityNode.*;
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
29 import static com.oracle.graal.replacements.SnippetTemplate.*;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
30
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
31 import com.oracle.graal.api.code.*;
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
32 import com.oracle.graal.api.meta.*;
15259
d90e5c22ba55 Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents: 15018
diff changeset
33 import com.oracle.graal.compiler.common.*;
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
34 import com.oracle.graal.graph.Node.ConstantNodeParameter;
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
35 import com.oracle.graal.graph.Node.NodeIntrinsic;
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
36 import com.oracle.graal.hotspot.meta.*;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
37 import com.oracle.graal.hotspot.nodes.*;
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
38 import com.oracle.graal.hotspot.nodes.type.*;
8905
c502db57e687 Rename ArrayWriteBarrier to SerialWriteBarrier and move it to graal.nodes
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8904
diff changeset
39 import com.oracle.graal.nodes.*;
10517
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
40 import com.oracle.graal.nodes.extended.*;
21543
93c50cefb9e8 moved GraalInternalError to com.oracle.jvmci.common and renamed it to JVMCIError (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents: 21526
diff changeset
41 import com.oracle.graal.nodes.memory.HeapAccess.BarrierType;
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
42 import com.oracle.graal.nodes.spi.*;
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
43 import com.oracle.graal.replacements.*;
8903
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
44 import com.oracle.graal.replacements.Snippet.ConstantParameter;
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
45 import com.oracle.graal.replacements.SnippetTemplate.AbstractTemplates;
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
46 import com.oracle.graal.replacements.SnippetTemplate.Arguments;
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
47 import com.oracle.graal.replacements.SnippetTemplate.SnippetInfo;
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
48 import com.oracle.graal.replacements.nodes.*;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
49 import com.oracle.graal.word.*;
21551
5324104ac4f3 moved com.oracle.graal.hotspot.jvmci classes to com.oracle.jvmci.hotspot module (JBS:GRAAL-53)
Doug Simon <doug.simon@oracle.com>
parents: 21543
diff changeset
50 import com.oracle.jvmci.hotspot.HotSpotVMConfig.*;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
51
8508
ef97193256d0 -Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8507
diff changeset
52 public class WriteBarrierSnippets implements Snippets {
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
53
10481
29e9a5d18c70 Clean up.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 9513
diff changeset
54 private static final SnippetCounter.Group countersWriteBarriers = SnippetCounters.getValue() ? new SnippetCounter.Group("WriteBarriers") : null;
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
55 private static final SnippetCounter serialWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "serialWriteBarrier", "Number of Serial Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
56 private static final SnippetCounter g1AttemptedPreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1AttemptedPreWriteBarrier", "Number of G1 attempted Pre Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
57 private static final SnippetCounter g1EffectivePreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectivePreWriteBarrier", "Number of G1 effective Pre Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
58 private static final SnippetCounter g1ExecutedPreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1ExecutedPreWriteBarrier", "Number of G1 executed Pre Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
59 private static final SnippetCounter g1AttemptedPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1AttemptedPostWriteBarrier", "Number of attempted G1 Post Write Barriers");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
60 private static final SnippetCounter g1EffectiveAfterXORPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectiveAfterXORPostWriteBarrier",
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
61 "Number of effective G1 Post Write Barriers (after passing the XOR test)");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
62 private static final SnippetCounter g1EffectiveAfterNullPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectiveAfterNullPostWriteBarrier",
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
63 "Number of effective G1 Post Write Barriers (after passing the NULL test)");
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
64 private static final SnippetCounter g1ExecutedPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1ExecutedPostWriteBarrier", "Number of executed G1 Post Write Barriers");
10481
29e9a5d18c70 Clean up.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 9513
diff changeset
65
18307
85f5227a7a37 add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 18259
diff changeset
66 public static final LocationIdentity GC_CARD_LOCATION = NamedLocationIdentity.mutable("GC-Card");
85f5227a7a37 add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 18259
diff changeset
67 public static final LocationIdentity GC_LOG_LOCATION = NamedLocationIdentity.mutable("GC-Log");
85f5227a7a37 add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 18259
diff changeset
68 public static final LocationIdentity GC_INDEX_LOCATION = NamedLocationIdentity.mutable("GC-Index");
11769
427cbe2bd615 SerialWriteBarrier: use GC_CARD_LOCATION instead of ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11732
diff changeset
69
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
70 @Snippet
12746
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
71 public static void serialWriteBarrier(Object object, Object location, @ConstantParameter boolean usePrecise) {
10944
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
72 Object fixedObject = FixedValueAnchorNode.getObject(object);
8903
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
73 Pointer oop;
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
74 if (usePrecise) {
15004
9ecd3da04309 use temporary proxy for locations in snippets
Lukas Stadler <lukas.stadler@oracle.com>
parents: 14991
diff changeset
75 oop = Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location));
8903
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
76 } else {
10944
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
77 oop = Word.fromObject(fixedObject);
8903
876a449f17ab Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8895
diff changeset
78 }
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
79 serialWriteBarrierCounter.inc();
18661
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
80 int cardTableShift = (isImmutableCode() && generatePIC()) ? CardTableShiftNode.cardTableShift() : cardTableShift();
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
81 long cardTableAddress = (isImmutableCode() && generatePIC()) ? CardTableAddressNode.cardTableAddress() : cardTableStart();
19775
14e703edb2ab use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents: 18661
diff changeset
82 Word base = Word.fromWordBase(oop.unsignedShiftRight(cardTableShift));
18661
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
83 long startAddress = cardTableAddress;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
84 int displacement = 0;
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
85 if (((int) startAddress) == startAddress) {
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
86 displacement = (int) startAddress;
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
87 } else {
18661
aa4f2e3629ca Added card table PIC support
adlertz
parents: 18649
diff changeset
88 base = base.add(Word.unsigned(cardTableAddress));
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
89 }
11769
427cbe2bd615 SerialWriteBarrier: use GC_CARD_LOCATION instead of ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11732
diff changeset
90 base.writeByte(displacement, (byte) 0, GC_CARD_LOCATION);
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
91 }
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
92
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
93 @Snippet
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
94 public static void serialArrayRangeWriteBarrier(Object object, int startIndex, int length) {
10942
56a86b60c9d3 Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10941
diff changeset
95 if (length == 0) {
56a86b60c9d3 Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10941
diff changeset
96 return;
56a86b60c9d3 Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10941
diff changeset
97 }
8989
40e7444d62ee Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8985
diff changeset
98 Object dest = FixedValueAnchorNode.getObject(object);
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
99 int cardShift = cardTableShift();
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
100 long cardStart = cardTableStart();
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
101 final int scale = arrayIndexScale(Kind.Object);
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
102 int header = arrayBaseOffset(Kind.Object);
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
103 long dstAddr = GetObjectAddressNode.get(dest);
8989
40e7444d62ee Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8985
diff changeset
104 long start = (dstAddr + header + (long) startIndex * scale) >>> cardShift;
40e7444d62ee Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8985
diff changeset
105 long end = (dstAddr + header + ((long) startIndex + length - 1) * scale) >>> cardShift;
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
106 long count = end - start + 1;
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
107 while (count-- > 0) {
19775
14e703edb2ab use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents: 18661
diff changeset
108 DirectStoreNode.storeBoolean((start + cardStart) + count, false, Kind.Boolean);
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
109 }
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
110 }
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
111
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
112 @Snippet
10623
13c6440ac750 Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10622
diff changeset
113 public static void g1PreWriteBarrier(Object object, Object expectedObject, Object location, @ConstantParameter boolean doLoad, @ConstantParameter boolean nullCheck,
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
114 @ConstantParameter Register threadRegister, @ConstantParameter boolean trace) {
21081
2e9250b29d14 Use trapping null check in g1PreWriteBarrier snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 20178
diff changeset
115 if (nullCheck) {
2e9250b29d14 Use trapping null check in g1PreWriteBarrier snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 20178
diff changeset
116 NullCheckNode.nullCheck(object);
10623
13c6440ac750 Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10622
diff changeset
117 }
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
118 Word thread = registerAsWord(threadRegister);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
119 Object fixedObject = FixedValueAnchorNode.getObject(object);
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
120 verifyOop(fixedObject);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
121 Object fixedExpectedObject = FixedValueAnchorNode.getObject(expectedObject);
19775
14e703edb2ab use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents: 18661
diff changeset
122 Word field = Word.fromWordBase(Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location)));
14e703edb2ab use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents: 18661
diff changeset
123 Word previousOop = Word.fromWordBase(Word.fromObject(fixedExpectedObject));
10520
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
124 byte markingValue = thread.readByte(g1SATBQueueMarkingOffset());
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
125 Word bufferAddress = thread.readWord(g1SATBQueueBufferOffset());
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
126 Word indexAddress = thread.add(g1SATBQueueIndexOffset());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
127 Word indexValue = indexAddress.readWord(0);
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
128 int gcCycle = 0;
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
129 if (trace) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
130 gcCycle = (int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
131 log(trace, "[%d] G1-Pre Thread %p Object %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedObject).rawValue());
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
132 log(trace, "[%d] G1-Pre Thread %p Expected Object %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedExpectedObject).rawValue());
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
133 log(trace, "[%d] G1-Pre Thread %p Field %p\n", gcCycle, thread.rawValue(), field.rawValue());
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
134 log(trace, "[%d] G1-Pre Thread %p Marking %d\n", gcCycle, thread.rawValue(), markingValue);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
135 log(trace, "[%d] G1-Pre Thread %p DoLoad %d\n", gcCycle, thread.rawValue(), doLoad ? 1L : 0L);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
136 }
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
137 g1AttemptedPreWriteBarrierCounter.inc();
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
138 // If the concurrent marker is enabled, the barrier is issued.
11819
1cb614d6d25b Readjust branch probabilities of G1 Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11817
diff changeset
139 if (probability(NOT_FREQUENT_PROBABILITY, markingValue != (byte) 0)) {
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
140 // If the previous value has to be loaded (before the write), the load is issued.
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
141 // The load is always issued except the cases of CAS and referent field.
10622
844be189150c Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10620
diff changeset
142 if (probability(LIKELY_PROBABILITY, doLoad)) {
19775
14e703edb2ab use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents: 18661
diff changeset
143 previousOop = Word.fromWordBase(Word.fromObject(field.readObject(0, BarrierType.NONE)));
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
144 if (trace) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
145 log(trace, "[%d] G1-Pre Thread %p Previous Object %p\n ", gcCycle, thread.rawValue(), previousOop.rawValue());
10631
2e82291febf4 Oop verification after printing oop
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10630
diff changeset
146 verifyOop(previousOop.toObject());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
147 }
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
148 }
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
149 g1EffectivePreWriteBarrierCounter.inc();
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
150 // If the previous value is null the barrier should not be issued.
10622
844be189150c Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10620
diff changeset
151 if (probability(FREQUENT_PROBABILITY, previousOop.notEqual(0))) {
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
152 g1ExecutedPreWriteBarrierCounter.inc();
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
153 // If the thread-local SATB buffer is full issue a native call which will
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
154 // initialize a new one and add the entry.
10622
844be189150c Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10620
diff changeset
155 if (probability(FREQUENT_PROBABILITY, indexValue.notEqual(0))) {
10520
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
156 Word nextIndex = indexValue.subtract(wordSize());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
157 Word logAddress = bufferAddress.add(nextIndex);
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
158 // Log the object to be marked as well as update the SATB's buffer next index.
11774
f6eb4866d558 G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11769
diff changeset
159 logAddress.writeWord(0, previousOop, GC_LOG_LOCATION);
f6eb4866d558 G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11769
diff changeset
160 indexAddress.writeWord(0, nextIndex, GC_INDEX_LOCATION);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
161 } else {
10517
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
162 g1PreBarrierStub(G1WBPRECALL, previousOop.toObject());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
163 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
164 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
165 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
166 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
167
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
168 @Snippet
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
169 public static void g1PostWriteBarrier(Object object, Object value, Object location, @ConstantParameter boolean usePrecise, @ConstantParameter Register threadRegister,
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
170 @ConstantParameter boolean trace) {
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
171 Word thread = registerAsWord(threadRegister);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
172 Object fixedObject = FixedValueAnchorNode.getObject(object);
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
173 Object fixedValue = FixedValueAnchorNode.getObject(value);
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
174 verifyOop(fixedObject);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
175 verifyOop(fixedValue);
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
176 validateObject(fixedObject, fixedValue);
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
177 Word oop;
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
178 if (usePrecise) {
19775
14e703edb2ab use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents: 18661
diff changeset
179 oop = Word.fromWordBase(Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location)));
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
180 } else {
19775
14e703edb2ab use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents: 18661
diff changeset
181 oop = Word.fromWordBase(Word.fromObject(fixedObject));
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
182 }
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
183 int gcCycle = 0;
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
184 if (trace) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
185 gcCycle = (int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0);
10630
ca3865947682 Debug log cleanup
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10623
diff changeset
186 log(trace, "[%d] G1-Post Thread: %p Object: %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedObject).rawValue());
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
187 log(trace, "[%d] G1-Post Thread: %p Field: %p\n", gcCycle, thread.rawValue(), oop.rawValue());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
188 }
19775
14e703edb2ab use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents: 18661
diff changeset
189 Word writtenValue = Word.fromWordBase(Word.fromObject(fixedValue));
10520
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
190 Word bufferAddress = thread.readWord(g1CardQueueBufferOffset());
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
191 Word indexAddress = thread.add(g1CardQueueIndexOffset());
590f8f159309 Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10519
diff changeset
192 Word indexValue = thread.readWord(g1CardQueueIndexOffset());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
193 // The result of the xor reveals whether the installed pointer crosses heap regions.
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
194 // In case it does the write barrier has to be issued.
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
195 Word xorResult = (oop.xor(writtenValue)).unsignedShiftRight(logOfHeapRegionGrainBytes());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
196
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
197 // Calculate the address of the card to be enqueued to the
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
198 // thread local card queue.
10950
d2187f88cdd2 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10945
diff changeset
199 Word cardBase = oop.unsignedShiftRight(cardTableShift());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
200 long startAddress = cardTableStart();
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
201 int displacement = 0;
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
202 if (((int) startAddress) == startAddress) {
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
203 displacement = (int) startAddress;
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
204 } else {
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
205 cardBase = cardBase.add(Word.unsigned(cardTableStart()));
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
206 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
207 Word cardAddress = cardBase.add(displacement);
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
208
10955
769ad79b8e0c Add G1 Barrier Counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10954
diff changeset
209 g1AttemptedPostWriteBarrierCounter.inc();
11819
1cb614d6d25b Readjust branch probabilities of G1 Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11817
diff changeset
210 if (probability(FREQUENT_PROBABILITY, xorResult.notEqual(0))) {
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
211 g1EffectiveAfterXORPostWriteBarrierCounter.inc();
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
212
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
213 // If the written value is not null continue with the barrier addition.
10622
844be189150c Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10620
diff changeset
214 if (probability(FREQUENT_PROBABILITY, writtenValue.notEqual(0))) {
21134
a5faa9aafc80 Make the G1PostWriteBarrier snippet kill only private locations.
Roland Schatz <roland.schatz@oracle.com>
parents: 21106
diff changeset
215 byte cardByte = cardAddress.readByte(0, GC_CARD_LOCATION);
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
216 g1EffectiveAfterNullPostWriteBarrierCounter.inc();
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
217
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
218 // If the card is already dirty, (hence already enqueued) skip the insertion.
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
219 if (probability(NOT_FREQUENT_PROBABILITY, cardByte != g1YoungCardValue())) {
21134
a5faa9aafc80 Make the G1PostWriteBarrier snippet kill only private locations.
Roland Schatz <roland.schatz@oracle.com>
parents: 21106
diff changeset
220 MembarNode.memoryBarrier(STORE_LOAD, GC_CARD_LOCATION);
a5faa9aafc80 Make the G1PostWriteBarrier snippet kill only private locations.
Roland Schatz <roland.schatz@oracle.com>
parents: 21106
diff changeset
221 byte cardByteReload = cardAddress.readByte(0, GC_CARD_LOCATION);
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
222 if (probability(NOT_FREQUENT_PROBABILITY, cardByteReload != dirtyCardValue())) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
223 log(trace, "[%d] G1-Post Thread: %p Card: %p \n", gcCycle, thread.rawValue(), Word.unsigned(cardByte).rawValue());
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
224 cardAddress.writeByte(0, (byte) 0, GC_CARD_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
225 g1ExecutedPostWriteBarrierCounter.inc();
11817
4bae51f0c888 Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11776
diff changeset
226
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
227 // If the thread local card queue is full, issue a native call which will
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
228 // initialize a new one and add the card entry.
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
229 if (probability(FREQUENT_PROBABILITY, indexValue.notEqual(0))) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
230 Word nextIndex = indexValue.subtract(wordSize());
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
231 Word logAddress = bufferAddress.add(nextIndex);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
232 // Log the object to be scanned as well as update
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
233 // the card queue's next index.
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
234 logAddress.writeWord(0, cardAddress, GC_LOG_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
235 indexAddress.writeWord(0, nextIndex, GC_INDEX_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
236 } else {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
237 g1PostBarrierStub(G1WBPOSTCALL, cardAddress);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
238 }
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
239 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
240 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
241 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
242 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
243 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
244
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
245 @Snippet
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
246 public static void g1ArrayRangePreWriteBarrier(Object object, int startIndex, int length, @ConstantParameter Register threadRegister) {
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
247 Word thread = registerAsWord(threadRegister);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
248 byte markingValue = thread.readByte(g1SATBQueueMarkingOffset());
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
249 // If the concurrent marker is not enabled or the vector length is zero, return.
10951
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
250 if (markingValue == (byte) 0 || length == 0) {
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
251 return;
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
252 }
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
253 Object dest = FixedValueAnchorNode.getObject(object);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
254 Word bufferAddress = thread.readWord(g1SATBQueueBufferOffset());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
255 Word indexAddress = thread.add(g1SATBQueueIndexOffset());
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
256 long dstAddr = GetObjectAddressNode.get(dest);
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
257 long indexValue = indexAddress.readWord(0).rawValue();
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
258 final int scale = arrayIndexScale(Kind.Object);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
259 int header = arrayBaseOffset(Kind.Object);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
260
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
261 for (int i = startIndex; i < length; i++) {
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
262 long address = dstAddr + header + (i * scale);
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
263 Pointer oop = Word.fromObject(Word.unsigned(address).readObject(0, BarrierType.NONE));
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
264 verifyOop(oop.toObject());
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
265 if (oop.notEqual(0)) {
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
266 if (indexValue != 0) {
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
267 indexValue = indexValue - wordSize();
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
268 Word logAddress = bufferAddress.add(Word.unsigned(indexValue));
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
269 // Log the object to be marked as well as update the SATB's buffer next index.
11774
f6eb4866d558 G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11769
diff changeset
270 logAddress.writeWord(0, oop, GC_LOG_LOCATION);
f6eb4866d558 G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents: 11769
diff changeset
271 indexAddress.writeWord(0, Word.unsigned(indexValue), GC_INDEX_LOCATION);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
272 } else {
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
273 g1PreBarrierStub(G1WBPRECALL, oop.toObject());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
274 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
275 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
276 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
277 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
278
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
279 @Snippet
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
280 public static void g1ArrayRangePostWriteBarrier(Object object, int startIndex, int length, @ConstantParameter Register threadRegister) {
10951
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
281 if (length == 0) {
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
282 return;
e2ac8bf20433 Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10950
diff changeset
283 }
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
284 Object dest = FixedValueAnchorNode.getObject(object);
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
285 Word thread = registerAsWord(threadRegister);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
286 Word bufferAddress = thread.readWord(g1CardQueueBufferOffset());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
287 Word indexAddress = thread.add(g1CardQueueIndexOffset());
10952
b43bc053ce8f Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10951
diff changeset
288 long indexValue = thread.readWord(g1CardQueueIndexOffset()).rawValue();
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
289
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
290 int cardShift = cardTableShift();
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
291 long cardStart = cardTableStart();
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
292 final int scale = arrayIndexScale(Kind.Object);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
293 int header = arrayBaseOffset(Kind.Object);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
294 long dstAddr = GetObjectAddressNode.get(dest);
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
295 long start = (dstAddr + header + (long) startIndex * scale) >>> cardShift;
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
296 long end = (dstAddr + header + ((long) startIndex + length - 1) * scale) >>> cardShift;
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
297 long count = end - start + 1;
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
298
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
299 while (count-- > 0) {
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
300 Word cardAddress = Word.unsigned((start + cardStart) + count);
21134
a5faa9aafc80 Make the G1PostWriteBarrier snippet kill only private locations.
Roland Schatz <roland.schatz@oracle.com>
parents: 21106
diff changeset
301 byte cardByte = cardAddress.readByte(0, GC_CARD_LOCATION);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
302 // If the card is already dirty, (hence already enqueued) skip the insertion.
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
303 if (probability(NOT_FREQUENT_PROBABILITY, cardByte != g1YoungCardValue())) {
21134
a5faa9aafc80 Make the G1PostWriteBarrier snippet kill only private locations.
Roland Schatz <roland.schatz@oracle.com>
parents: 21106
diff changeset
304 MembarNode.memoryBarrier(STORE_LOAD, GC_CARD_LOCATION);
a5faa9aafc80 Make the G1PostWriteBarrier snippet kill only private locations.
Roland Schatz <roland.schatz@oracle.com>
parents: 21106
diff changeset
305 byte cardByteReload = cardAddress.readByte(0, GC_CARD_LOCATION);
16018
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
306 if (probability(NOT_FREQUENT_PROBABILITY, cardByteReload != dirtyCardValue())) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
307 cardAddress.writeByte(0, (byte) 0, GC_CARD_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
308 // If the thread local card queue is full, issue a native call which will
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
309 // initialize a new one and add the card entry.
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
310 if (indexValue != 0) {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
311 indexValue = indexValue - wordSize();
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
312 Word logAddress = bufferAddress.add(Word.unsigned(indexValue));
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
313 // Log the object to be scanned as well as update
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
314 // the card queue's next index.
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
315 logAddress.writeWord(0, cardAddress, GC_LOG_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
316 indexAddress.writeWord(0, Word.unsigned(indexValue), GC_INDEX_LOCATION);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
317 } else {
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
318 g1PostBarrierStub(G1WBPOSTCALL, cardAddress);
6a62ccbd1658 add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents: 15259
diff changeset
319 }
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
320 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
321 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
322 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
323 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
324
10517
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
325 public static final ForeignCallDescriptor G1WBPRECALL = new ForeignCallDescriptor("write_barrier_pre", void.class, Object.class);
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
326
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
327 @NodeIntrinsic(ForeignCallNode.class)
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
328 private static native void g1PreBarrierStub(@ConstantNodeParameter ForeignCallDescriptor descriptor, Object object);
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
329
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
330 public static final ForeignCallDescriptor G1WBPOSTCALL = new ForeignCallDescriptor("write_barrier_post", void.class, Word.class);
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
331
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
332 @NodeIntrinsic(ForeignCallNode.class)
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
333 public static native void g1PostBarrierStub(@ConstantNodeParameter ForeignCallDescriptor descriptor, Word card);
10517
c2c9afee00b5 Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10505
diff changeset
334
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
335 public static class Templates extends AbstractTemplates {
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
336
20178
7ba8fb19d56d Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 19852
diff changeset
337 private final SnippetInfo serialWriteBarrier = snippet(WriteBarrierSnippets.class, "serialWriteBarrier", GC_CARD_LOCATION);
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
338 private final SnippetInfo serialArrayRangeWriteBarrier = snippet(WriteBarrierSnippets.class, "serialArrayRangeWriteBarrier");
20178
7ba8fb19d56d Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 19852
diff changeset
339 private final SnippetInfo g1PreWriteBarrier = snippet(WriteBarrierSnippets.class, "g1PreWriteBarrier", GC_INDEX_LOCATION, GC_LOG_LOCATION);
7ba8fb19d56d Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 19852
diff changeset
340 private final SnippetInfo g1ReferentReadBarrier = snippet(WriteBarrierSnippets.class, "g1PreWriteBarrier", GC_INDEX_LOCATION, GC_LOG_LOCATION);
7ba8fb19d56d Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 19852
diff changeset
341 private final SnippetInfo g1PostWriteBarrier = snippet(WriteBarrierSnippets.class, "g1PostWriteBarrier", GC_CARD_LOCATION, GC_INDEX_LOCATION, GC_LOG_LOCATION);
7ba8fb19d56d Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 19852
diff changeset
342 private final SnippetInfo g1ArrayRangePreWriteBarrier = snippet(WriteBarrierSnippets.class, "g1ArrayRangePreWriteBarrier", GC_INDEX_LOCATION, GC_LOG_LOCATION);
7ba8fb19d56d Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents: 19852
diff changeset
343 private final SnippetInfo g1ArrayRangePostWriteBarrier = snippet(WriteBarrierSnippets.class, "g1ArrayRangePostWriteBarrier", GC_CARD_LOCATION, GC_INDEX_LOCATION, GC_LOG_LOCATION);
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
344
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
345 private final CompressEncoding oopEncoding;
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
346
15018
db4254246f9a Remove Constant.forObject and Constant.asObject to improve compiler/VM separation
Christian Wimmer <christian.wimmer@oracle.com>
parents: 15004
diff changeset
347 public Templates(HotSpotProviders providers, TargetDescription target, CompressEncoding oopEncoding) {
db4254246f9a Remove Constant.forObject and Constant.asObject to improve compiler/VM separation
Christian Wimmer <christian.wimmer@oracle.com>
parents: 15004
diff changeset
348 super(providers, providers.getSnippetReflection(), target);
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
349 this.oopEncoding = oopEncoding;
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
350 }
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
351
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
352 public void lower(SerialWriteBarrier writeBarrier, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
353 Arguments args = new Arguments(serialWriteBarrier, writeBarrier.graph().getGuardsStage(), tool.getLoweringStage());
10944
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
354 args.add("object", writeBarrier.getObject());
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
355 args.add("location", writeBarrier.getLocation());
3a9f3ec8efc6 Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10942
diff changeset
356 args.addConst("usePrecise", writeBarrier.usePrecise());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
357 template(args).instantiate(providers.getMetaAccess(), writeBarrier, DEFAULT_REPLACER, args);
8571
0bacc5564d18 Revert WB refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8531
diff changeset
358 }
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
359
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
360 public void lower(SerialArrayRangeWriteBarrier arrayRangeWriteBarrier, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
361 Arguments args = new Arguments(serialArrayRangeWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage());
9015
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
362 args.add("object", arrayRangeWriteBarrier.getObject());
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
363 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex());
92d2bedb5dfc Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents: 8989
diff changeset
364 args.add("length", arrayRangeWriteBarrier.getLength());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
365 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args);
8963
63eae4723b18 Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 8905
diff changeset
366 }
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
367
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
368 public void lower(G1PreWriteBarrier writeBarrierPre, HotSpotRegistersProvider registers, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
369 Arguments args = new Arguments(g1PreWriteBarrier, writeBarrierPre.graph().getGuardsStage(), tool.getLoweringStage());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
370 args.add("object", writeBarrierPre.getObject());
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
371
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
372 ValueNode expected = writeBarrierPre.getExpectedObject();
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
373 if (expected != null && expected.stamp() instanceof NarrowOopStamp) {
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
374 assert oopEncoding != null;
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
375 expected = CompressionNode.uncompress(expected, oopEncoding);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
376 }
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
377 args.add("expectedObject", expected);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
378
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
379 args.add("location", writeBarrierPre.getLocation());
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
380 args.addConst("doLoad", writeBarrierPre.doLoad());
10669
f11cd00d4115 Make G1PreWriteBarrier a DeoptimizingNode.
Roland Schatz <roland.schatz@oracle.com>
parents: 10631
diff changeset
381 args.addConst("nullCheck", writeBarrierPre.getNullCheck());
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
382 args.addConst("threadRegister", registers.getThreadRegister());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
383 args.addConst("trace", traceBarrier());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
384 template(args).instantiate(providers.getMetaAccess(), writeBarrierPre, DEFAULT_REPLACER, args);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
385 }
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
386
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
387 public void lower(G1ReferentFieldReadBarrier readBarrier, HotSpotRegistersProvider registers, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
388 Arguments args = new Arguments(g1ReferentReadBarrier, readBarrier.graph().getGuardsStage(), tool.getLoweringStage());
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
389 args.add("object", readBarrier.getObject());
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
390
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
391 ValueNode expected = readBarrier.getExpectedObject();
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
392 if (expected != null && expected.stamp() instanceof NarrowOopStamp) {
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
393 assert oopEncoding != null;
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
394 expected = CompressionNode.uncompress(expected, oopEncoding);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
395 }
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
396
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
397 args.add("expectedObject", expected);
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
398 args.add("location", readBarrier.getLocation());
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
399 args.addConst("doLoad", readBarrier.doLoad());
10941
b9718191858c G1ReferentFieldReadBarrier is not a deoptimizing node
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10829
diff changeset
400 args.addConst("nullCheck", false);
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
401 args.addConst("threadRegister", registers.getThreadRegister());
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
402 args.addConst("trace", traceBarrier());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
403 template(args).instantiate(providers.getMetaAccess(), readBarrier, DEFAULT_REPLACER, args);
10829
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
404 }
761002c55728 Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10828
diff changeset
405
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
406 public void lower(G1PostWriteBarrier writeBarrierPost, HotSpotRegistersProvider registers, LoweringTool tool) {
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
407 StructuredGraph graph = writeBarrierPost.graph();
12746
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
408 if (writeBarrierPost.alwaysNull()) {
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
409 graph.removeFixed(writeBarrierPost);
12746
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
410 return;
a7eee87c7897 remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents: 12660
diff changeset
411 }
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
412 Arguments args = new Arguments(g1PostWriteBarrier, graph.getGuardsStage(), tool.getLoweringStage());
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
413 args.add("object", writeBarrierPost.getObject());
14945
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
414
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
415 ValueNode value = writeBarrierPost.getValue();
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
416 if (value.stamp() instanceof NarrowOopStamp) {
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
417 assert oopEncoding != null;
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
418 value = CompressionNode.uncompress(value, oopEncoding);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
419 }
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
420 args.add("value", value);
b14cb2d9253d Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents: 13505
diff changeset
421
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
422 args.add("location", writeBarrierPost.getLocation());
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
423 args.addConst("usePrecise", writeBarrierPost.usePrecise());
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
424 args.addConst("threadRegister", registers.getThreadRegister());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
425 args.addConst("trace", traceBarrier());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
426 template(args).instantiate(providers.getMetaAccess(), writeBarrierPost, DEFAULT_REPLACER, args);
10494
6f331db530f6 Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 9513
diff changeset
427 }
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
428
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
429 public void lower(G1ArrayRangePreWriteBarrier arrayRangeWriteBarrier, HotSpotRegistersProvider registers, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
430 Arguments args = new Arguments(g1ArrayRangePreWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage());
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
431 args.add("object", arrayRangeWriteBarrier.getObject());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
432 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
433 args.add("length", arrayRangeWriteBarrier.getLength());
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
434 args.addConst("threadRegister", registers.getThreadRegister());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
435 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
436 }
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
437
13505
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
438 public void lower(G1ArrayRangePostWriteBarrier arrayRangeWriteBarrier, HotSpotRegistersProvider registers, LoweringTool tool) {
75a67ebd50e8 Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents: 12791
diff changeset
439 Arguments args = new Arguments(g1ArrayRangePostWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage());
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
440 args.add("object", arrayRangeWriteBarrier.getObject());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
441 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex());
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
442 args.add("length", arrayRangeWriteBarrier.getLength());
12791
1cc0a907e1dd pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents: 12746
diff changeset
443 args.addConst("threadRegister", registers.getThreadRegister());
12054
bba234a1670e grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents: 12052
diff changeset
444 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args);
10552
ca8efaf18f27 Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10547
diff changeset
445 }
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
446 }
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
447
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
448 /**
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
449 * Log method of debugging purposes.
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
450 */
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
451 public static void log(boolean enabled, String format, long value) {
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
452 if (enabled) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
453 Log.printf(format, value);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
454 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
455 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
456
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
457 public static void log(boolean enabled, String format, long value1, long value2) {
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
458 if (enabled) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
459 Log.printf(format, value1, value2);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
460 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
461 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
462
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
463 public static void log(boolean enabled, String format, long value1, long value2, long value3) {
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
464 if (enabled) {
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
465 Log.printf(format, value1, value2, value3);
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
466 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
467 }
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
468
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
469 public static boolean traceBarrier() {
10701
f1cd428fca9f Remove caching of gcStartCycles
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10669
diff changeset
470 return GraalOptions.GCDebugStartCycle.getValue() > 0 && ((int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0) > GraalOptions.GCDebugStartCycle.getValue());
10620
260e75dc4b6e Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10552
diff changeset
471 }
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
472
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
473 /**
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
474 * Validation helper method which performs sanity checks on write operations. The addresses of
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
475 * both the object and the value being written are checked in order to determine if they reside
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
476 * in a valid heap region. If an object is stale, an invalid access is performed in order to
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
477 * prematurely crash the VM and debug the stack trace of the faulty method.
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
478 */
11234
73eb4d8a4015 Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 11221
diff changeset
479 public static void validateObject(Object parent, Object child) {
11221
8d4bd13c6983 Field renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10955
diff changeset
480 if (verifyOops() && child != null && !validateOop(VALIDATE_OBJECT, parent, child)) {
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
481 log(true, "Verification ERROR, Parent: %p Child: %p\n", Word.fromObject(parent).rawValue(), Word.fromObject(child).rawValue());
19852
3d0116ec99c5 Create utilities LocationIdentity#isAny, LocationIdentity#isSingle, LocationIdentity#any, LocationIdentity#overlaps.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents: 19775
diff changeset
482 DirectObjectStoreNode.storeObject(null, 0, 0, null, LocationIdentity.any(), Kind.Object);
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
483 }
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
484 }
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
485
11221
8d4bd13c6983 Field renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10955
diff changeset
486 public static final ForeignCallDescriptor VALIDATE_OBJECT = new ForeignCallDescriptor("validate_object", boolean.class, Word.class, Word.class);
10953
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
487
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
488 @NodeIntrinsic(ForeignCallNode.class)
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
489 private static native boolean validateOop(@ConstantNodeParameter ForeignCallDescriptor descriptor, Object parent, Object object);
97e282186b5b Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents: 10952
diff changeset
490
8488
225c984588ee -Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff changeset
491 }