Mercurial > hg > truffle
annotate graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/replacements/WriteBarrierSnippets.java @ 18589:c640f804dbad
Intrinsify Reflection.getClassAccessFlags
author | Tom Rodriguez <tom.rodriguez@oracle.com> |
---|---|
date | Mon, 01 Dec 2014 11:31:53 -0800 |
parents | 85f5227a7a37 |
children | d138867d61c4 |
rev | line source |
---|---|
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
1 /* |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
2 * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
4 * |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
5 * This code is free software; you can redistribute it and/or modify it |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
6 * under the terms of the GNU General Public License version 2 only, as |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
7 * published by the Free Software Foundation. |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
8 * |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
9 * This code is distributed in the hope that it will be useful, but WITHOUT |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
12 * version 2 for more details (a copy is included in the LICENSE file that |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
13 * accompanied this code). |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
14 * |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
15 * You should have received a copy of the GNU General Public License version |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
16 * 2 along with this work; if not, write to the Free Software Foundation, |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
18 * |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
20 * or visit www.oracle.com if you need additional information or have any |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
21 * questions. |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
22 */ |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
23 package com.oracle.graal.hotspot.replacements; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
24 |
16018
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
25 import static com.oracle.graal.api.code.MemoryBarriers.*; |
15259
d90e5c22ba55
Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents:
15018
diff
changeset
|
26 import static com.oracle.graal.compiler.common.GraalOptions.*; |
9513
659bb6cf930c
rename: HotSpotSnippetUtils -> HotSpotReplacementsUtil
Doug Simon <doug.simon@oracle.com>
parents:
9015
diff
changeset
|
27 import static com.oracle.graal.hotspot.replacements.HotSpotReplacementsUtil.*; |
10787
388fbd0dd4a4
moved BranchNodeProbabilities to graal.nodes project
Doug Simon <doug.simon@oracle.com>
parents:
10701
diff
changeset
|
28 import static com.oracle.graal.nodes.extended.BranchProbabilityNode.*; |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
29 import static com.oracle.graal.replacements.SnippetTemplate.*; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
30 |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
31 import com.oracle.graal.api.code.*; |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
32 import com.oracle.graal.api.meta.*; |
15259
d90e5c22ba55
Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents:
15018
diff
changeset
|
33 import com.oracle.graal.compiler.common.*; |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
34 import com.oracle.graal.graph.Node.ConstantNodeParameter; |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
35 import com.oracle.graal.graph.Node.NodeIntrinsic; |
14991
64dcb92ee75a
Truffle: Change signature for Truffle calls from (PackedFrame, Arguments) to (Object[]).
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
14945
diff
changeset
|
36 import com.oracle.graal.hotspot.HotSpotVMConfig.CompressEncoding; |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
37 import com.oracle.graal.hotspot.meta.*; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
38 import com.oracle.graal.hotspot.nodes.*; |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
39 import com.oracle.graal.hotspot.nodes.type.*; |
8905
c502db57e687
Rename ArrayWriteBarrier to SerialWriteBarrier and move it to graal.nodes
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8904
diff
changeset
|
40 import com.oracle.graal.nodes.*; |
10828
b05e1ff3aac0
Rename WriteBarrierType to BarrierType
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10817
diff
changeset
|
41 import com.oracle.graal.nodes.HeapAccess.BarrierType; |
10517
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
42 import com.oracle.graal.nodes.extended.*; |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
43 import com.oracle.graal.nodes.spi.*; |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
44 import com.oracle.graal.replacements.*; |
8903
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
45 import com.oracle.graal.replacements.Snippet.ConstantParameter; |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
46 import com.oracle.graal.replacements.SnippetTemplate.AbstractTemplates; |
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
47 import com.oracle.graal.replacements.SnippetTemplate.Arguments; |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
48 import com.oracle.graal.replacements.SnippetTemplate.SnippetInfo; |
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
49 import com.oracle.graal.replacements.nodes.*; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
50 import com.oracle.graal.word.*; |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
51 |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
52 public class WriteBarrierSnippets implements Snippets { |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
53 |
10481
29e9a5d18c70
Clean up.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
9513
diff
changeset
|
54 private static final SnippetCounter.Group countersWriteBarriers = SnippetCounters.getValue() ? new SnippetCounter.Group("WriteBarriers") : null; |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
55 private static final SnippetCounter serialWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "serialWriteBarrier", "Number of Serial Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
56 private static final SnippetCounter g1AttemptedPreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1AttemptedPreWriteBarrier", "Number of G1 attempted Pre Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
57 private static final SnippetCounter g1EffectivePreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectivePreWriteBarrier", "Number of G1 effective Pre Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
58 private static final SnippetCounter g1ExecutedPreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1ExecutedPreWriteBarrier", "Number of G1 executed Pre Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
59 private static final SnippetCounter g1AttemptedPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1AttemptedPostWriteBarrier", "Number of attempted G1 Post Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
60 private static final SnippetCounter g1EffectiveAfterXORPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectiveAfterXORPostWriteBarrier", |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
61 "Number of effective G1 Post Write Barriers (after passing the XOR test)"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
62 private static final SnippetCounter g1EffectiveAfterNullPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectiveAfterNullPostWriteBarrier", |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
63 "Number of effective G1 Post Write Barriers (after passing the NULL test)"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
64 private static final SnippetCounter g1ExecutedPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1ExecutedPostWriteBarrier", "Number of executed G1 Post Write Barriers"); |
10481
29e9a5d18c70
Clean up.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
9513
diff
changeset
|
65 |
18307
85f5227a7a37
add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
18259
diff
changeset
|
66 public static final LocationIdentity GC_CARD_LOCATION = NamedLocationIdentity.mutable("GC-Card"); |
85f5227a7a37
add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
18259
diff
changeset
|
67 public static final LocationIdentity GC_LOG_LOCATION = NamedLocationIdentity.mutable("GC-Log"); |
85f5227a7a37
add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
18259
diff
changeset
|
68 public static final LocationIdentity GC_INDEX_LOCATION = NamedLocationIdentity.mutable("GC-Index"); |
11769
427cbe2bd615
SerialWriteBarrier: use GC_CARD_LOCATION instead of ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11732
diff
changeset
|
69 |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
70 @Snippet |
12746
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
71 public static void serialWriteBarrier(Object object, Object location, @ConstantParameter boolean usePrecise) { |
10944
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
72 Object fixedObject = FixedValueAnchorNode.getObject(object); |
8903
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
73 Pointer oop; |
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
74 if (usePrecise) { |
15004
9ecd3da04309
use temporary proxy for locations in snippets
Lukas Stadler <lukas.stadler@oracle.com>
parents:
14991
diff
changeset
|
75 oop = Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location)); |
8903
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
76 } else { |
10944
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
77 oop = Word.fromObject(fixedObject); |
8903
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
78 } |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
79 serialWriteBarrierCounter.inc(); |
8518
4a79f7c0d1a4
-Fix assertion errors
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8517
diff
changeset
|
80 Word base = (Word) oop.unsignedShiftRight(cardTableShift()); |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
81 long startAddress = cardTableStart(); |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
82 int displacement = 0; |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
83 if (((int) startAddress) == startAddress) { |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
84 displacement = (int) startAddress; |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
85 } else { |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
86 base = base.add(Word.unsigned(cardTableStart())); |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
87 } |
11769
427cbe2bd615
SerialWriteBarrier: use GC_CARD_LOCATION instead of ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11732
diff
changeset
|
88 base.writeByte(displacement, (byte) 0, GC_CARD_LOCATION); |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
89 } |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
90 |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
91 @Snippet |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
92 public static void serialArrayRangeWriteBarrier(Object object, int startIndex, int length) { |
10942
56a86b60c9d3
Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10941
diff
changeset
|
93 if (length == 0) { |
56a86b60c9d3
Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10941
diff
changeset
|
94 return; |
56a86b60c9d3
Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10941
diff
changeset
|
95 } |
8989
40e7444d62ee
Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8985
diff
changeset
|
96 Object dest = FixedValueAnchorNode.getObject(object); |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
97 int cardShift = cardTableShift(); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
98 long cardStart = cardTableStart(); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
99 final int scale = arrayIndexScale(Kind.Object); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
100 int header = arrayBaseOffset(Kind.Object); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
101 long dstAddr = GetObjectAddressNode.get(dest); |
8989
40e7444d62ee
Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8985
diff
changeset
|
102 long start = (dstAddr + header + (long) startIndex * scale) >>> cardShift; |
40e7444d62ee
Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8985
diff
changeset
|
103 long end = (dstAddr + header + ((long) startIndex + length - 1) * scale) >>> cardShift; |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
104 long count = end - start + 1; |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
105 while (count-- > 0) { |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
106 DirectStoreNode.store((start + cardStart) + count, false, Kind.Boolean); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
107 } |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
108 } |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
109 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
110 @Snippet |
10623
13c6440ac750
Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10622
diff
changeset
|
111 public static void g1PreWriteBarrier(Object object, Object expectedObject, Object location, @ConstantParameter boolean doLoad, @ConstantParameter boolean nullCheck, |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
112 @ConstantParameter Register threadRegister, @ConstantParameter boolean trace) { |
10623
13c6440ac750
Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10622
diff
changeset
|
113 if (nullCheck && object == null) { |
13c6440ac750
Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10622
diff
changeset
|
114 DeoptimizeNode.deopt(DeoptimizationAction.InvalidateReprofile, DeoptimizationReason.NullCheckException); |
13c6440ac750
Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10622
diff
changeset
|
115 } |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
116 Word thread = registerAsWord(threadRegister); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
117 Object fixedObject = FixedValueAnchorNode.getObject(object); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
118 verifyOop(fixedObject); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
119 Object fixedExpectedObject = FixedValueAnchorNode.getObject(expectedObject); |
15004
9ecd3da04309
use temporary proxy for locations in snippets
Lukas Stadler <lukas.stadler@oracle.com>
parents:
14991
diff
changeset
|
120 Word field = (Word) Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location)); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
121 Word previousOop = (Word) Word.fromObject(fixedExpectedObject); |
10520
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
122 byte markingValue = thread.readByte(g1SATBQueueMarkingOffset()); |
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
123 Word bufferAddress = thread.readWord(g1SATBQueueBufferOffset()); |
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
124 Word indexAddress = thread.add(g1SATBQueueIndexOffset()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
125 Word indexValue = indexAddress.readWord(0); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
126 int gcCycle = 0; |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
127 if (trace) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
128 gcCycle = (int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
129 log(trace, "[%d] G1-Pre Thread %p Object %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedObject).rawValue()); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
130 log(trace, "[%d] G1-Pre Thread %p Expected Object %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedExpectedObject).rawValue()); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
131 log(trace, "[%d] G1-Pre Thread %p Field %p\n", gcCycle, thread.rawValue(), field.rawValue()); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
132 log(trace, "[%d] G1-Pre Thread %p Marking %d\n", gcCycle, thread.rawValue(), markingValue); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
133 log(trace, "[%d] G1-Pre Thread %p DoLoad %d\n", gcCycle, thread.rawValue(), doLoad ? 1L : 0L); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
134 } |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
135 g1AttemptedPreWriteBarrierCounter.inc(); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
136 // If the concurrent marker is enabled, the barrier is issued. |
11819
1cb614d6d25b
Readjust branch probabilities of G1 Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11817
diff
changeset
|
137 if (probability(NOT_FREQUENT_PROBABILITY, markingValue != (byte) 0)) { |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
138 // If the previous value has to be loaded (before the write), the load is issued. |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
139 // The load is always issued except the cases of CAS and referent field. |
10622
844be189150c
Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10620
diff
changeset
|
140 if (probability(LIKELY_PROBABILITY, doLoad)) { |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
141 previousOop = (Word) Word.fromObject(field.readObject(0, BarrierType.NONE)); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
142 if (trace) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
143 log(trace, "[%d] G1-Pre Thread %p Previous Object %p\n ", gcCycle, thread.rawValue(), previousOop.rawValue()); |
10631
2e82291febf4
Oop verification after printing oop
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10630
diff
changeset
|
144 verifyOop(previousOop.toObject()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
145 } |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
146 } |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
147 g1EffectivePreWriteBarrierCounter.inc(); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
148 // If the previous value is null the barrier should not be issued. |
10622
844be189150c
Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10620
diff
changeset
|
149 if (probability(FREQUENT_PROBABILITY, previousOop.notEqual(0))) { |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
150 g1ExecutedPreWriteBarrierCounter.inc(); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
151 // If the thread-local SATB buffer is full issue a native call which will |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
152 // initialize a new one and add the entry. |
10622
844be189150c
Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10620
diff
changeset
|
153 if (probability(FREQUENT_PROBABILITY, indexValue.notEqual(0))) { |
10520
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
154 Word nextIndex = indexValue.subtract(wordSize()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
155 Word logAddress = bufferAddress.add(nextIndex); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
156 // Log the object to be marked as well as update the SATB's buffer next index. |
11774
f6eb4866d558
G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11769
diff
changeset
|
157 logAddress.writeWord(0, previousOop, GC_LOG_LOCATION); |
f6eb4866d558
G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11769
diff
changeset
|
158 indexAddress.writeWord(0, nextIndex, GC_INDEX_LOCATION); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
159 } else { |
10517
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
160 g1PreBarrierStub(G1WBPRECALL, previousOop.toObject()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
161 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
162 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
163 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
164 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
165 |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
166 @Snippet |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
167 public static void g1PostWriteBarrier(Object object, Object value, Object location, @ConstantParameter boolean usePrecise, @ConstantParameter Register threadRegister, |
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
168 @ConstantParameter boolean trace) { |
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
169 Word thread = registerAsWord(threadRegister); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
170 Object fixedObject = FixedValueAnchorNode.getObject(object); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
171 Object fixedValue = FixedValueAnchorNode.getObject(value); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
172 verifyOop(fixedObject); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
173 verifyOop(fixedValue); |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
174 validateObject(fixedObject, fixedValue); |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
175 Word oop; |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
176 if (usePrecise) { |
15004
9ecd3da04309
use temporary proxy for locations in snippets
Lukas Stadler <lukas.stadler@oracle.com>
parents:
14991
diff
changeset
|
177 oop = (Word) Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location)); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
178 } else { |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
179 oop = (Word) Word.fromObject(fixedObject); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
180 } |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
181 int gcCycle = 0; |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
182 if (trace) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
183 gcCycle = (int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0); |
10630
ca3865947682
Debug log cleanup
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10623
diff
changeset
|
184 log(trace, "[%d] G1-Post Thread: %p Object: %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedObject).rawValue()); |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
185 log(trace, "[%d] G1-Post Thread: %p Field: %p\n", gcCycle, thread.rawValue(), oop.rawValue()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
186 } |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
187 Word writtenValue = (Word) Word.fromObject(fixedValue); |
10520
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
188 Word bufferAddress = thread.readWord(g1CardQueueBufferOffset()); |
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
189 Word indexAddress = thread.add(g1CardQueueIndexOffset()); |
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
190 Word indexValue = thread.readWord(g1CardQueueIndexOffset()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
191 // The result of the xor reveals whether the installed pointer crosses heap regions. |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
192 // In case it does the write barrier has to be issued. |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
193 Word xorResult = (oop.xor(writtenValue)).unsignedShiftRight(logOfHeapRegionGrainBytes()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
194 |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
195 // Calculate the address of the card to be enqueued to the |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
196 // thread local card queue. |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
197 Word cardBase = oop.unsignedShiftRight(cardTableShift()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
198 long startAddress = cardTableStart(); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
199 int displacement = 0; |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
200 if (((int) startAddress) == startAddress) { |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
201 displacement = (int) startAddress; |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
202 } else { |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
203 cardBase = cardBase.add(Word.unsigned(cardTableStart())); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
204 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
205 Word cardAddress = cardBase.add(displacement); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
206 |
10955
769ad79b8e0c
Add G1 Barrier Counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10954
diff
changeset
|
207 g1AttemptedPostWriteBarrierCounter.inc(); |
11819
1cb614d6d25b
Readjust branch probabilities of G1 Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11817
diff
changeset
|
208 if (probability(FREQUENT_PROBABILITY, xorResult.notEqual(0))) { |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
209 g1EffectiveAfterXORPostWriteBarrierCounter.inc(); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
210 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
211 // If the written value is not null continue with the barrier addition. |
10622
844be189150c
Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10620
diff
changeset
|
212 if (probability(FREQUENT_PROBABILITY, writtenValue.notEqual(0))) { |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
213 byte cardByte = cardAddress.readByte(0); |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
214 g1EffectiveAfterNullPostWriteBarrierCounter.inc(); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
215 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
216 // If the card is already dirty, (hence already enqueued) skip the insertion. |
16018
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
217 if (probability(NOT_FREQUENT_PROBABILITY, cardByte != g1YoungCardValue())) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
218 MembarNode.memoryBarrier(STORE_LOAD); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
219 byte cardByteReload = cardAddress.readByte(0); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
220 if (probability(NOT_FREQUENT_PROBABILITY, cardByteReload != dirtyCardValue())) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
221 log(trace, "[%d] G1-Post Thread: %p Card: %p \n", gcCycle, thread.rawValue(), Word.unsigned(cardByte).rawValue()); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
222 cardAddress.writeByte(0, (byte) 0, GC_CARD_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
223 g1ExecutedPostWriteBarrierCounter.inc(); |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
224 |
16018
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
225 // If the thread local card queue is full, issue a native call which will |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
226 // initialize a new one and add the card entry. |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
227 if (probability(FREQUENT_PROBABILITY, indexValue.notEqual(0))) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
228 Word nextIndex = indexValue.subtract(wordSize()); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
229 Word logAddress = bufferAddress.add(nextIndex); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
230 // Log the object to be scanned as well as update |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
231 // the card queue's next index. |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
232 logAddress.writeWord(0, cardAddress, GC_LOG_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
233 indexAddress.writeWord(0, nextIndex, GC_INDEX_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
234 } else { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
235 g1PostBarrierStub(G1WBPOSTCALL, cardAddress); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
236 } |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
237 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
238 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
239 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
240 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
241 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
242 |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
243 @Snippet |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
244 public static void g1ArrayRangePreWriteBarrier(Object object, int startIndex, int length, @ConstantParameter Register threadRegister) { |
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
245 Word thread = registerAsWord(threadRegister); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
246 byte markingValue = thread.readByte(g1SATBQueueMarkingOffset()); |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
247 // If the concurrent marker is not enabled or the vector length is zero, return. |
10951
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
248 if (markingValue == (byte) 0 || length == 0) { |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
249 return; |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
250 } |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
251 Object dest = FixedValueAnchorNode.getObject(object); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
252 Word bufferAddress = thread.readWord(g1SATBQueueBufferOffset()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
253 Word indexAddress = thread.add(g1SATBQueueIndexOffset()); |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
254 long dstAddr = GetObjectAddressNode.get(dest); |
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
255 long indexValue = indexAddress.readWord(0).rawValue(); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
256 final int scale = arrayIndexScale(Kind.Object); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
257 int header = arrayBaseOffset(Kind.Object); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
258 |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
259 for (int i = startIndex; i < length; i++) { |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
260 long address = dstAddr + header + (i * scale); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
261 Pointer oop = Word.fromObject(Word.unsigned(address).readObject(0, BarrierType.NONE)); |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
262 verifyOop(oop.toObject()); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
263 if (oop.notEqual(0)) { |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
264 if (indexValue != 0) { |
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
265 indexValue = indexValue - wordSize(); |
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
266 Word logAddress = bufferAddress.add(Word.unsigned(indexValue)); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
267 // Log the object to be marked as well as update the SATB's buffer next index. |
11774
f6eb4866d558
G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11769
diff
changeset
|
268 logAddress.writeWord(0, oop, GC_LOG_LOCATION); |
f6eb4866d558
G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11769
diff
changeset
|
269 indexAddress.writeWord(0, Word.unsigned(indexValue), GC_INDEX_LOCATION); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
270 } else { |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
271 g1PreBarrierStub(G1WBPRECALL, oop.toObject()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
272 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
273 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
274 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
275 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
276 |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
277 @Snippet |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
278 public static void g1ArrayRangePostWriteBarrier(Object object, int startIndex, int length, @ConstantParameter Register threadRegister) { |
10951
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
279 if (length == 0) { |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
280 return; |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
281 } |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
282 Object dest = FixedValueAnchorNode.getObject(object); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
283 Word thread = registerAsWord(threadRegister); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
284 Word bufferAddress = thread.readWord(g1CardQueueBufferOffset()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
285 Word indexAddress = thread.add(g1CardQueueIndexOffset()); |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
286 long indexValue = thread.readWord(g1CardQueueIndexOffset()).rawValue(); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
287 |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
288 int cardShift = cardTableShift(); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
289 long cardStart = cardTableStart(); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
290 final int scale = arrayIndexScale(Kind.Object); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
291 int header = arrayBaseOffset(Kind.Object); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
292 long dstAddr = GetObjectAddressNode.get(dest); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
293 long start = (dstAddr + header + (long) startIndex * scale) >>> cardShift; |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
294 long end = (dstAddr + header + ((long) startIndex + length - 1) * scale) >>> cardShift; |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
295 long count = end - start + 1; |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
296 |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
297 while (count-- > 0) { |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
298 Word cardAddress = Word.unsigned((start + cardStart) + count); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
299 byte cardByte = cardAddress.readByte(0); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
300 // If the card is already dirty, (hence already enqueued) skip the insertion. |
16018
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
301 if (probability(NOT_FREQUENT_PROBABILITY, cardByte != g1YoungCardValue())) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
302 MembarNode.memoryBarrier(STORE_LOAD); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
303 byte cardByteReload = cardAddress.readByte(0); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
304 if (probability(NOT_FREQUENT_PROBABILITY, cardByteReload != dirtyCardValue())) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
305 cardAddress.writeByte(0, (byte) 0, GC_CARD_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
306 // If the thread local card queue is full, issue a native call which will |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
307 // initialize a new one and add the card entry. |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
308 if (indexValue != 0) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
309 indexValue = indexValue - wordSize(); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
310 Word logAddress = bufferAddress.add(Word.unsigned(indexValue)); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
311 // Log the object to be scanned as well as update |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
312 // the card queue's next index. |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
313 logAddress.writeWord(0, cardAddress, GC_LOG_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
314 indexAddress.writeWord(0, Word.unsigned(indexValue), GC_INDEX_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
315 } else { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
316 g1PostBarrierStub(G1WBPOSTCALL, cardAddress); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
317 } |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
318 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
319 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
320 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
321 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
322 |
10517
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
323 public static final ForeignCallDescriptor G1WBPRECALL = new ForeignCallDescriptor("write_barrier_pre", void.class, Object.class); |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
324 |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
325 @NodeIntrinsic(ForeignCallNode.class) |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
326 private static native void g1PreBarrierStub(@ConstantNodeParameter ForeignCallDescriptor descriptor, Object object); |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
327 |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
328 public static final ForeignCallDescriptor G1WBPOSTCALL = new ForeignCallDescriptor("write_barrier_post", void.class, Word.class); |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
329 |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
330 @NodeIntrinsic(ForeignCallNode.class) |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
331 public static native void g1PostBarrierStub(@ConstantNodeParameter ForeignCallDescriptor descriptor, Word card); |
10517
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
332 |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
333 public static class Templates extends AbstractTemplates { |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
334 |
10944
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
335 private final SnippetInfo serialWriteBarrier = snippet(WriteBarrierSnippets.class, "serialWriteBarrier"); |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
336 private final SnippetInfo serialArrayRangeWriteBarrier = snippet(WriteBarrierSnippets.class, "serialArrayRangeWriteBarrier"); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
337 private final SnippetInfo g1PreWriteBarrier = snippet(WriteBarrierSnippets.class, "g1PreWriteBarrier"); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
338 private final SnippetInfo g1ReferentReadBarrier = snippet(WriteBarrierSnippets.class, "g1PreWriteBarrier"); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
339 private final SnippetInfo g1PostWriteBarrier = snippet(WriteBarrierSnippets.class, "g1PostWriteBarrier"); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
340 private final SnippetInfo g1ArrayRangePreWriteBarrier = snippet(WriteBarrierSnippets.class, "g1ArrayRangePreWriteBarrier"); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
341 private final SnippetInfo g1ArrayRangePostWriteBarrier = snippet(WriteBarrierSnippets.class, "g1ArrayRangePostWriteBarrier"); |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
342 |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
343 private final CompressEncoding oopEncoding; |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
344 |
15018
db4254246f9a
Remove Constant.forObject and Constant.asObject to improve compiler/VM separation
Christian Wimmer <christian.wimmer@oracle.com>
parents:
15004
diff
changeset
|
345 public Templates(HotSpotProviders providers, TargetDescription target, CompressEncoding oopEncoding) { |
db4254246f9a
Remove Constant.forObject and Constant.asObject to improve compiler/VM separation
Christian Wimmer <christian.wimmer@oracle.com>
parents:
15004
diff
changeset
|
346 super(providers, providers.getSnippetReflection(), target); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
347 this.oopEncoding = oopEncoding; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
348 } |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
349 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
350 public void lower(SerialWriteBarrier writeBarrier, LoweringTool tool) { |
12746
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
351 if (writeBarrier.alwaysNull()) { |
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
352 writeBarrier.graph().removeFixed(writeBarrier); |
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
353 return; |
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
354 } |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
355 Arguments args = new Arguments(serialWriteBarrier, writeBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
10944
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
356 args.add("object", writeBarrier.getObject()); |
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
357 args.add("location", writeBarrier.getLocation()); |
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
358 args.addConst("usePrecise", writeBarrier.usePrecise()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
359 template(args).instantiate(providers.getMetaAccess(), writeBarrier, DEFAULT_REPLACER, args); |
8571
0bacc5564d18
Revert WB refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8531
diff
changeset
|
360 } |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
361 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
362 public void lower(SerialArrayRangeWriteBarrier arrayRangeWriteBarrier, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
363 Arguments args = new Arguments(serialArrayRangeWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
364 args.add("object", arrayRangeWriteBarrier.getObject()); |
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
365 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex()); |
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
366 args.add("length", arrayRangeWriteBarrier.getLength()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
367 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args); |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
368 } |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
369 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
370 public void lower(G1PreWriteBarrier writeBarrierPre, HotSpotRegistersProvider registers, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
371 Arguments args = new Arguments(g1PreWriteBarrier, writeBarrierPre.graph().getGuardsStage(), tool.getLoweringStage()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
372 args.add("object", writeBarrierPre.getObject()); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
373 |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
374 ValueNode expected = writeBarrierPre.getExpectedObject(); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
375 if (expected != null && expected.stamp() instanceof NarrowOopStamp) { |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
376 assert oopEncoding != null; |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
377 expected = CompressionNode.uncompress(expected, oopEncoding); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
378 } |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
379 args.add("expectedObject", expected); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
380 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
381 args.add("location", writeBarrierPre.getLocation()); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
382 args.addConst("doLoad", writeBarrierPre.doLoad()); |
10669
f11cd00d4115
Make G1PreWriteBarrier a DeoptimizingNode.
Roland Schatz <roland.schatz@oracle.com>
parents:
10631
diff
changeset
|
383 args.addConst("nullCheck", writeBarrierPre.getNullCheck()); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
384 args.addConst("threadRegister", registers.getThreadRegister()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
385 args.addConst("trace", traceBarrier()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
386 template(args).instantiate(providers.getMetaAccess(), writeBarrierPre, DEFAULT_REPLACER, args); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
387 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
388 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
389 public void lower(G1ReferentFieldReadBarrier readBarrier, HotSpotRegistersProvider registers, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
390 Arguments args = new Arguments(g1ReferentReadBarrier, readBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
391 args.add("object", readBarrier.getObject()); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
392 |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
393 ValueNode expected = readBarrier.getExpectedObject(); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
394 if (expected != null && expected.stamp() instanceof NarrowOopStamp) { |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
395 assert oopEncoding != null; |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
396 expected = CompressionNode.uncompress(expected, oopEncoding); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
397 } |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
398 |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
399 args.add("expectedObject", expected); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
400 args.add("location", readBarrier.getLocation()); |
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
401 args.addConst("doLoad", readBarrier.doLoad()); |
10941
b9718191858c
G1ReferentFieldReadBarrier is not a deoptimizing node
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10829
diff
changeset
|
402 args.addConst("nullCheck", false); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
403 args.addConst("threadRegister", registers.getThreadRegister()); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
404 args.addConst("trace", traceBarrier()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
405 template(args).instantiate(providers.getMetaAccess(), readBarrier, DEFAULT_REPLACER, args); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
406 } |
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
407 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
408 public void lower(G1PostWriteBarrier writeBarrierPost, HotSpotRegistersProvider registers, LoweringTool tool) { |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
409 StructuredGraph graph = writeBarrierPost.graph(); |
12746
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
410 if (writeBarrierPost.alwaysNull()) { |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
411 graph.removeFixed(writeBarrierPost); |
12746
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
412 return; |
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
413 } |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
414 Arguments args = new Arguments(g1PostWriteBarrier, graph.getGuardsStage(), tool.getLoweringStage()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
415 args.add("object", writeBarrierPost.getObject()); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
416 |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
417 ValueNode value = writeBarrierPost.getValue(); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
418 if (value.stamp() instanceof NarrowOopStamp) { |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
419 assert oopEncoding != null; |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
420 value = CompressionNode.uncompress(value, oopEncoding); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
421 } |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
422 args.add("value", value); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
423 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
424 args.add("location", writeBarrierPost.getLocation()); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
425 args.addConst("usePrecise", writeBarrierPost.usePrecise()); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
426 args.addConst("threadRegister", registers.getThreadRegister()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
427 args.addConst("trace", traceBarrier()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
428 template(args).instantiate(providers.getMetaAccess(), writeBarrierPost, DEFAULT_REPLACER, args); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
429 } |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
430 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
431 public void lower(G1ArrayRangePreWriteBarrier arrayRangeWriteBarrier, HotSpotRegistersProvider registers, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
432 Arguments args = new Arguments(g1ArrayRangePreWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
433 args.add("object", arrayRangeWriteBarrier.getObject()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
434 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
435 args.add("length", arrayRangeWriteBarrier.getLength()); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
436 args.addConst("threadRegister", registers.getThreadRegister()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
437 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
438 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
439 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
440 public void lower(G1ArrayRangePostWriteBarrier arrayRangeWriteBarrier, HotSpotRegistersProvider registers, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
441 Arguments args = new Arguments(g1ArrayRangePostWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
442 args.add("object", arrayRangeWriteBarrier.getObject()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
443 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
444 args.add("length", arrayRangeWriteBarrier.getLength()); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
445 args.addConst("threadRegister", registers.getThreadRegister()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
446 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
447 } |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
448 } |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
449 |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
450 /** |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
451 * Log method of debugging purposes. |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
452 */ |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
453 public static void log(boolean enabled, String format, long value) { |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
454 if (enabled) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
455 Log.printf(format, value); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
456 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
457 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
458 |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
459 public static void log(boolean enabled, String format, long value1, long value2) { |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
460 if (enabled) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
461 Log.printf(format, value1, value2); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
462 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
463 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
464 |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
465 public static void log(boolean enabled, String format, long value1, long value2, long value3) { |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
466 if (enabled) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
467 Log.printf(format, value1, value2, value3); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
468 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
469 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
470 |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
471 public static boolean traceBarrier() { |
10701
f1cd428fca9f
Remove caching of gcStartCycles
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10669
diff
changeset
|
472 return GraalOptions.GCDebugStartCycle.getValue() > 0 && ((int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0) > GraalOptions.GCDebugStartCycle.getValue()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
473 } |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
474 |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
475 /** |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
476 * Validation helper method which performs sanity checks on write operations. The addresses of |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
477 * both the object and the value being written are checked in order to determine if they reside |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
478 * in a valid heap region. If an object is stale, an invalid access is performed in order to |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
479 * prematurely crash the VM and debug the stack trace of the faulty method. |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
480 */ |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
481 public static void validateObject(Object parent, Object child) { |
11221
8d4bd13c6983
Field renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10955
diff
changeset
|
482 if (verifyOops() && child != null && !validateOop(VALIDATE_OBJECT, parent, child)) { |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
483 log(true, "Verification ERROR, Parent: %p Child: %p\n", Word.fromObject(parent).rawValue(), Word.fromObject(child).rawValue()); |
12584
bc0ed539018f
UnsafeArrayCopySnippets: use DirectObjectStoreNode again for kind object. add location identity to DirectObjectStoreNode.
Bernhard Urban <bernhard.urban@jku.at>
parents:
12054
diff
changeset
|
484 DirectObjectStoreNode.storeObject(null, 0, 0, null, LocationIdentity.ANY_LOCATION); |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
485 } |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
486 } |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
487 |
11221
8d4bd13c6983
Field renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10955
diff
changeset
|
488 public static final ForeignCallDescriptor VALIDATE_OBJECT = new ForeignCallDescriptor("validate_object", boolean.class, Word.class, Word.class); |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
489 |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
490 @NodeIntrinsic(ForeignCallNode.class) |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
491 private static native boolean validateOop(@ConstantNodeParameter ForeignCallDescriptor descriptor, Object parent, Object object); |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
492 |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
493 } |