Mercurial > hg > graal-compiler
annotate graal/com.oracle.graal.hotspot/src/com/oracle/graal/hotspot/replacements/WriteBarrierSnippets.java @ 21106:ec894427332d
Don't insert serial barriers for null stores
author | Tom Rodriguez <tom.rodriguez@oracle.com> |
---|---|
date | Thu, 23 Apr 2015 12:55:54 -0700 |
parents | 2e9250b29d14 |
children | a5faa9aafc80 |
rev | line source |
---|---|
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
1 /* |
20178
7ba8fb19d56d
Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
19852
diff
changeset
|
2 * Copyright (c) 2012, 2015, Oracle and/or its affiliates. All rights reserved. |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
4 * |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
5 * This code is free software; you can redistribute it and/or modify it |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
6 * under the terms of the GNU General Public License version 2 only, as |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
7 * published by the Free Software Foundation. |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
8 * |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
9 * This code is distributed in the hope that it will be useful, but WITHOUT |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
12 * version 2 for more details (a copy is included in the LICENSE file that |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
13 * accompanied this code). |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
14 * |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
15 * You should have received a copy of the GNU General Public License version |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
16 * 2 along with this work; if not, write to the Free Software Foundation, |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
18 * |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
20 * or visit www.oracle.com if you need additional information or have any |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
21 * questions. |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
22 */ |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
23 package com.oracle.graal.hotspot.replacements; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
24 |
16018
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
25 import static com.oracle.graal.api.code.MemoryBarriers.*; |
15259
d90e5c22ba55
Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents:
15018
diff
changeset
|
26 import static com.oracle.graal.compiler.common.GraalOptions.*; |
9513
659bb6cf930c
rename: HotSpotSnippetUtils -> HotSpotReplacementsUtil
Doug Simon <doug.simon@oracle.com>
parents:
9015
diff
changeset
|
27 import static com.oracle.graal.hotspot.replacements.HotSpotReplacementsUtil.*; |
10787
388fbd0dd4a4
moved BranchNodeProbabilities to graal.nodes project
Doug Simon <doug.simon@oracle.com>
parents:
10701
diff
changeset
|
28 import static com.oracle.graal.nodes.extended.BranchProbabilityNode.*; |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
29 import static com.oracle.graal.replacements.SnippetTemplate.*; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
30 |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
31 import com.oracle.graal.api.code.*; |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
32 import com.oracle.graal.api.meta.*; |
15259
d90e5c22ba55
Move GraalOptions to graal.compiler.common.
Josef Eisl <josef.eisl@jku.at>
parents:
15018
diff
changeset
|
33 import com.oracle.graal.compiler.common.*; |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
34 import com.oracle.graal.graph.Node.ConstantNodeParameter; |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
35 import com.oracle.graal.graph.Node.NodeIntrinsic; |
14991
64dcb92ee75a
Truffle: Change signature for Truffle calls from (PackedFrame, Arguments) to (Object[]).
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
14945
diff
changeset
|
36 import com.oracle.graal.hotspot.HotSpotVMConfig.CompressEncoding; |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
37 import com.oracle.graal.hotspot.meta.*; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
38 import com.oracle.graal.hotspot.nodes.*; |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
39 import com.oracle.graal.hotspot.nodes.type.*; |
8905
c502db57e687
Rename ArrayWriteBarrier to SerialWriteBarrier and move it to graal.nodes
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8904
diff
changeset
|
40 import com.oracle.graal.nodes.*; |
10828
b05e1ff3aac0
Rename WriteBarrierType to BarrierType
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10817
diff
changeset
|
41 import com.oracle.graal.nodes.HeapAccess.BarrierType; |
10517
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
42 import com.oracle.graal.nodes.extended.*; |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
43 import com.oracle.graal.nodes.spi.*; |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
44 import com.oracle.graal.replacements.*; |
8903
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
45 import com.oracle.graal.replacements.Snippet.ConstantParameter; |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
46 import com.oracle.graal.replacements.SnippetTemplate.AbstractTemplates; |
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
47 import com.oracle.graal.replacements.SnippetTemplate.Arguments; |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
48 import com.oracle.graal.replacements.SnippetTemplate.SnippetInfo; |
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
49 import com.oracle.graal.replacements.nodes.*; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
50 import com.oracle.graal.word.*; |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
51 |
8508
ef97193256d0
-Merge with trunk
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8507
diff
changeset
|
52 public class WriteBarrierSnippets implements Snippets { |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
53 |
10481
29e9a5d18c70
Clean up.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
9513
diff
changeset
|
54 private static final SnippetCounter.Group countersWriteBarriers = SnippetCounters.getValue() ? new SnippetCounter.Group("WriteBarriers") : null; |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
55 private static final SnippetCounter serialWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "serialWriteBarrier", "Number of Serial Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
56 private static final SnippetCounter g1AttemptedPreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1AttemptedPreWriteBarrier", "Number of G1 attempted Pre Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
57 private static final SnippetCounter g1EffectivePreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectivePreWriteBarrier", "Number of G1 effective Pre Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
58 private static final SnippetCounter g1ExecutedPreWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1ExecutedPreWriteBarrier", "Number of G1 executed Pre Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
59 private static final SnippetCounter g1AttemptedPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1AttemptedPostWriteBarrier", "Number of attempted G1 Post Write Barriers"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
60 private static final SnippetCounter g1EffectiveAfterXORPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectiveAfterXORPostWriteBarrier", |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
61 "Number of effective G1 Post Write Barriers (after passing the XOR test)"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
62 private static final SnippetCounter g1EffectiveAfterNullPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1EffectiveAfterNullPostWriteBarrier", |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
63 "Number of effective G1 Post Write Barriers (after passing the NULL test)"); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
64 private static final SnippetCounter g1ExecutedPostWriteBarrierCounter = new SnippetCounter(countersWriteBarriers, "g1ExecutedPostWriteBarrier", "Number of executed G1 Post Write Barriers"); |
10481
29e9a5d18c70
Clean up.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
9513
diff
changeset
|
65 |
18307
85f5227a7a37
add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
18259
diff
changeset
|
66 public static final LocationIdentity GC_CARD_LOCATION = NamedLocationIdentity.mutable("GC-Card"); |
85f5227a7a37
add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
18259
diff
changeset
|
67 public static final LocationIdentity GC_LOG_LOCATION = NamedLocationIdentity.mutable("GC-Log"); |
85f5227a7a37
add mutable/immutable factory methods to NamedLocationIdentity
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
18259
diff
changeset
|
68 public static final LocationIdentity GC_INDEX_LOCATION = NamedLocationIdentity.mutable("GC-Index"); |
11769
427cbe2bd615
SerialWriteBarrier: use GC_CARD_LOCATION instead of ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11732
diff
changeset
|
69 |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
70 @Snippet |
12746
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
71 public static void serialWriteBarrier(Object object, Object location, @ConstantParameter boolean usePrecise) { |
10944
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
72 Object fixedObject = FixedValueAnchorNode.getObject(object); |
8903
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
73 Pointer oop; |
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
74 if (usePrecise) { |
15004
9ecd3da04309
use temporary proxy for locations in snippets
Lukas Stadler <lukas.stadler@oracle.com>
parents:
14991
diff
changeset
|
75 oop = Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location)); |
8903
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
76 } else { |
10944
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
77 oop = Word.fromObject(fixedObject); |
8903
876a449f17ab
Merge FieldWriteBarrier into ArrayWriteBarrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8895
diff
changeset
|
78 } |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
79 serialWriteBarrierCounter.inc(); |
18661 | 80 int cardTableShift = (isImmutableCode() && generatePIC()) ? CardTableShiftNode.cardTableShift() : cardTableShift(); |
81 long cardTableAddress = (isImmutableCode() && generatePIC()) ? CardTableAddressNode.cardTableAddress() : cardTableStart(); | |
19775
14e703edb2ab
use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents:
18661
diff
changeset
|
82 Word base = Word.fromWordBase(oop.unsignedShiftRight(cardTableShift)); |
18661 | 83 long startAddress = cardTableAddress; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
84 int displacement = 0; |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
85 if (((int) startAddress) == startAddress) { |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
86 displacement = (int) startAddress; |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
87 } else { |
18661 | 88 base = base.add(Word.unsigned(cardTableAddress)); |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
89 } |
11769
427cbe2bd615
SerialWriteBarrier: use GC_CARD_LOCATION instead of ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11732
diff
changeset
|
90 base.writeByte(displacement, (byte) 0, GC_CARD_LOCATION); |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
91 } |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
92 |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
93 @Snippet |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
94 public static void serialArrayRangeWriteBarrier(Object object, int startIndex, int length) { |
10942
56a86b60c9d3
Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10941
diff
changeset
|
95 if (length == 0) { |
56a86b60c9d3
Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10941
diff
changeset
|
96 return; |
56a86b60c9d3
Early exit of SerialArrayRange barrier if vector length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10941
diff
changeset
|
97 } |
8989
40e7444d62ee
Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8985
diff
changeset
|
98 Object dest = FixedValueAnchorNode.getObject(object); |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
99 int cardShift = cardTableShift(); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
100 long cardStart = cardTableStart(); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
101 final int scale = arrayIndexScale(Kind.Object); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
102 int header = arrayBaseOffset(Kind.Object); |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
103 long dstAddr = GetObjectAddressNode.get(dest); |
8989
40e7444d62ee
Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8985
diff
changeset
|
104 long start = (dstAddr + header + (long) startIndex * scale) >>> cardShift; |
40e7444d62ee
Variable renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8985
diff
changeset
|
105 long end = (dstAddr + header + ((long) startIndex + length - 1) * scale) >>> cardShift; |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
106 long count = end - start + 1; |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
107 while (count-- > 0) { |
19775
14e703edb2ab
use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents:
18661
diff
changeset
|
108 DirectStoreNode.storeBoolean((start + cardStart) + count, false, Kind.Boolean); |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
109 } |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
110 } |
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
111 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
112 @Snippet |
10623
13c6440ac750
Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10622
diff
changeset
|
113 public static void g1PreWriteBarrier(Object object, Object expectedObject, Object location, @ConstantParameter boolean doLoad, @ConstantParameter boolean nullCheck, |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
114 @ConstantParameter Register threadRegister, @ConstantParameter boolean trace) { |
21081
2e9250b29d14
Use trapping null check in g1PreWriteBarrier snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
20178
diff
changeset
|
115 if (nullCheck) { |
2e9250b29d14
Use trapping null check in g1PreWriteBarrier snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
20178
diff
changeset
|
116 NullCheckNode.nullCheck(object); |
10623
13c6440ac750
Add nullcheck at G1 Pre Barrier
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10622
diff
changeset
|
117 } |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
118 Word thread = registerAsWord(threadRegister); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
119 Object fixedObject = FixedValueAnchorNode.getObject(object); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
120 verifyOop(fixedObject); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
121 Object fixedExpectedObject = FixedValueAnchorNode.getObject(expectedObject); |
19775
14e703edb2ab
use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents:
18661
diff
changeset
|
122 Word field = Word.fromWordBase(Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location))); |
14e703edb2ab
use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents:
18661
diff
changeset
|
123 Word previousOop = Word.fromWordBase(Word.fromObject(fixedExpectedObject)); |
10520
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
124 byte markingValue = thread.readByte(g1SATBQueueMarkingOffset()); |
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
125 Word bufferAddress = thread.readWord(g1SATBQueueBufferOffset()); |
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
126 Word indexAddress = thread.add(g1SATBQueueIndexOffset()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
127 Word indexValue = indexAddress.readWord(0); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
128 int gcCycle = 0; |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
129 if (trace) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
130 gcCycle = (int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
131 log(trace, "[%d] G1-Pre Thread %p Object %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedObject).rawValue()); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
132 log(trace, "[%d] G1-Pre Thread %p Expected Object %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedExpectedObject).rawValue()); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
133 log(trace, "[%d] G1-Pre Thread %p Field %p\n", gcCycle, thread.rawValue(), field.rawValue()); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
134 log(trace, "[%d] G1-Pre Thread %p Marking %d\n", gcCycle, thread.rawValue(), markingValue); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
135 log(trace, "[%d] G1-Pre Thread %p DoLoad %d\n", gcCycle, thread.rawValue(), doLoad ? 1L : 0L); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
136 } |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
137 g1AttemptedPreWriteBarrierCounter.inc(); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
138 // If the concurrent marker is enabled, the barrier is issued. |
11819
1cb614d6d25b
Readjust branch probabilities of G1 Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11817
diff
changeset
|
139 if (probability(NOT_FREQUENT_PROBABILITY, markingValue != (byte) 0)) { |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
140 // If the previous value has to be loaded (before the write), the load is issued. |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
141 // The load is always issued except the cases of CAS and referent field. |
10622
844be189150c
Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10620
diff
changeset
|
142 if (probability(LIKELY_PROBABILITY, doLoad)) { |
19775
14e703edb2ab
use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents:
18661
diff
changeset
|
143 previousOop = Word.fromWordBase(Word.fromObject(field.readObject(0, BarrierType.NONE))); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
144 if (trace) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
145 log(trace, "[%d] G1-Pre Thread %p Previous Object %p\n ", gcCycle, thread.rawValue(), previousOop.rawValue()); |
10631
2e82291febf4
Oop verification after printing oop
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10630
diff
changeset
|
146 verifyOop(previousOop.toObject()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
147 } |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
148 } |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
149 g1EffectivePreWriteBarrierCounter.inc(); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
150 // If the previous value is null the barrier should not be issued. |
10622
844be189150c
Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10620
diff
changeset
|
151 if (probability(FREQUENT_PROBABILITY, previousOop.notEqual(0))) { |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
152 g1ExecutedPreWriteBarrierCounter.inc(); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
153 // If the thread-local SATB buffer is full issue a native call which will |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
154 // initialize a new one and add the entry. |
10622
844be189150c
Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10620
diff
changeset
|
155 if (probability(FREQUENT_PROBABILITY, indexValue.notEqual(0))) { |
10520
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
156 Word nextIndex = indexValue.subtract(wordSize()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
157 Word logAddress = bufferAddress.add(nextIndex); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
158 // Log the object to be marked as well as update the SATB's buffer next index. |
11774
f6eb4866d558
G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11769
diff
changeset
|
159 logAddress.writeWord(0, previousOop, GC_LOG_LOCATION); |
f6eb4866d558
G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11769
diff
changeset
|
160 indexAddress.writeWord(0, nextIndex, GC_INDEX_LOCATION); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
161 } else { |
10517
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
162 g1PreBarrierStub(G1WBPRECALL, previousOop.toObject()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
163 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
164 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
165 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
166 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
167 |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
168 @Snippet |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
169 public static void g1PostWriteBarrier(Object object, Object value, Object location, @ConstantParameter boolean usePrecise, @ConstantParameter Register threadRegister, |
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
170 @ConstantParameter boolean trace) { |
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
171 Word thread = registerAsWord(threadRegister); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
172 Object fixedObject = FixedValueAnchorNode.getObject(object); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
173 Object fixedValue = FixedValueAnchorNode.getObject(value); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
174 verifyOop(fixedObject); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
175 verifyOop(fixedValue); |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
176 validateObject(fixedObject, fixedValue); |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
177 Word oop; |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
178 if (usePrecise) { |
19775
14e703edb2ab
use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents:
18661
diff
changeset
|
179 oop = Word.fromWordBase(Word.fromArray(fixedObject, SnippetLocationProxyNode.location(location))); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
180 } else { |
19775
14e703edb2ab
use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents:
18661
diff
changeset
|
181 oop = Word.fromWordBase(Word.fromObject(fixedObject)); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
182 } |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
183 int gcCycle = 0; |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
184 if (trace) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
185 gcCycle = (int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0); |
10630
ca3865947682
Debug log cleanup
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10623
diff
changeset
|
186 log(trace, "[%d] G1-Post Thread: %p Object: %p\n", gcCycle, thread.rawValue(), Word.fromObject(fixedObject).rawValue()); |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
187 log(trace, "[%d] G1-Post Thread: %p Field: %p\n", gcCycle, thread.rawValue(), oop.rawValue()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
188 } |
19775
14e703edb2ab
use GraphBuilderPlugins for method substitutions, intrinsics and snippets (GRAAL-982)
Doug Simon <doug.simon@oracle.com>
parents:
18661
diff
changeset
|
189 Word writtenValue = Word.fromWordBase(Word.fromObject(fixedValue)); |
10520
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
190 Word bufferAddress = thread.readWord(g1CardQueueBufferOffset()); |
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
191 Word indexAddress = thread.add(g1CardQueueIndexOffset()); |
590f8f159309
Static imports in write barrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10519
diff
changeset
|
192 Word indexValue = thread.readWord(g1CardQueueIndexOffset()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
193 // The result of the xor reveals whether the installed pointer crosses heap regions. |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
194 // In case it does the write barrier has to be issued. |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
195 Word xorResult = (oop.xor(writtenValue)).unsignedShiftRight(logOfHeapRegionGrainBytes()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
196 |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
197 // Calculate the address of the card to be enqueued to the |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
198 // thread local card queue. |
10950
d2187f88cdd2
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10945
diff
changeset
|
199 Word cardBase = oop.unsignedShiftRight(cardTableShift()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
200 long startAddress = cardTableStart(); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
201 int displacement = 0; |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
202 if (((int) startAddress) == startAddress) { |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
203 displacement = (int) startAddress; |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
204 } else { |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
205 cardBase = cardBase.add(Word.unsigned(cardTableStart())); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
206 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
207 Word cardAddress = cardBase.add(displacement); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
208 |
10955
769ad79b8e0c
Add G1 Barrier Counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10954
diff
changeset
|
209 g1AttemptedPostWriteBarrierCounter.inc(); |
11819
1cb614d6d25b
Readjust branch probabilities of G1 Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11817
diff
changeset
|
210 if (probability(FREQUENT_PROBABILITY, xorResult.notEqual(0))) { |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
211 g1EffectiveAfterXORPostWriteBarrierCounter.inc(); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
212 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
213 // If the written value is not null continue with the barrier addition. |
10622
844be189150c
Add probabilities in WB snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10620
diff
changeset
|
214 if (probability(FREQUENT_PROBABILITY, writtenValue.notEqual(0))) { |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
215 byte cardByte = cardAddress.readByte(0); |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
216 g1EffectiveAfterNullPostWriteBarrierCounter.inc(); |
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
217 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
218 // If the card is already dirty, (hence already enqueued) skip the insertion. |
16018
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
219 if (probability(NOT_FREQUENT_PROBABILITY, cardByte != g1YoungCardValue())) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
220 MembarNode.memoryBarrier(STORE_LOAD); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
221 byte cardByteReload = cardAddress.readByte(0); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
222 if (probability(NOT_FREQUENT_PROBABILITY, cardByteReload != dirtyCardValue())) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
223 log(trace, "[%d] G1-Post Thread: %p Card: %p \n", gcCycle, thread.rawValue(), Word.unsigned(cardByte).rawValue()); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
224 cardAddress.writeByte(0, (byte) 0, GC_CARD_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
225 g1ExecutedPostWriteBarrierCounter.inc(); |
11817
4bae51f0c888
Augment Write Barrier counters
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11776
diff
changeset
|
226 |
16018
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
227 // If the thread local card queue is full, issue a native call which will |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
228 // initialize a new one and add the card entry. |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
229 if (probability(FREQUENT_PROBABILITY, indexValue.notEqual(0))) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
230 Word nextIndex = indexValue.subtract(wordSize()); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
231 Word logAddress = bufferAddress.add(nextIndex); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
232 // Log the object to be scanned as well as update |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
233 // the card queue's next index. |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
234 logAddress.writeWord(0, cardAddress, GC_LOG_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
235 indexAddress.writeWord(0, nextIndex, GC_INDEX_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
236 } else { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
237 g1PostBarrierStub(G1WBPOSTCALL, cardAddress); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
238 } |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
239 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
240 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
241 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
242 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
243 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
244 |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
245 @Snippet |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
246 public static void g1ArrayRangePreWriteBarrier(Object object, int startIndex, int length, @ConstantParameter Register threadRegister) { |
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
247 Word thread = registerAsWord(threadRegister); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
248 byte markingValue = thread.readByte(g1SATBQueueMarkingOffset()); |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
249 // If the concurrent marker is not enabled or the vector length is zero, return. |
10951
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
250 if (markingValue == (byte) 0 || length == 0) { |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
251 return; |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
252 } |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
253 Object dest = FixedValueAnchorNode.getObject(object); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
254 Word bufferAddress = thread.readWord(g1SATBQueueBufferOffset()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
255 Word indexAddress = thread.add(g1SATBQueueIndexOffset()); |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
256 long dstAddr = GetObjectAddressNode.get(dest); |
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
257 long indexValue = indexAddress.readWord(0).rawValue(); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
258 final int scale = arrayIndexScale(Kind.Object); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
259 int header = arrayBaseOffset(Kind.Object); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
260 |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
261 for (int i = startIndex; i < length; i++) { |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
262 long address = dstAddr + header + (i * scale); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
263 Pointer oop = Word.fromObject(Word.unsigned(address).readObject(0, BarrierType.NONE)); |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
264 verifyOop(oop.toObject()); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
265 if (oop.notEqual(0)) { |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
266 if (indexValue != 0) { |
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
267 indexValue = indexValue - wordSize(); |
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
268 Word logAddress = bufferAddress.add(Word.unsigned(indexValue)); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
269 // Log the object to be marked as well as update the SATB's buffer next index. |
11774
f6eb4866d558
G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11769
diff
changeset
|
270 logAddress.writeWord(0, oop, GC_LOG_LOCATION); |
f6eb4866d558
G1 Barriers: don't use ANY_LOCATION
Bernhard Urban <bernhard.urban@jku.at>
parents:
11769
diff
changeset
|
271 indexAddress.writeWord(0, Word.unsigned(indexValue), GC_INDEX_LOCATION); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
272 } else { |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
273 g1PreBarrierStub(G1WBPRECALL, oop.toObject()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
274 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
275 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
276 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
277 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
278 |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
279 @Snippet |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
280 public static void g1ArrayRangePostWriteBarrier(Object object, int startIndex, int length, @ConstantParameter Register threadRegister) { |
10951
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
281 if (length == 0) { |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
282 return; |
e2ac8bf20433
Early exits from G1 Array Range barriers if length==0
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10950
diff
changeset
|
283 } |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
284 Object dest = FixedValueAnchorNode.getObject(object); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
285 Word thread = registerAsWord(threadRegister); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
286 Word bufferAddress = thread.readWord(g1CardQueueBufferOffset()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
287 Word indexAddress = thread.add(g1CardQueueIndexOffset()); |
10952
b43bc053ce8f
Fix bug in CardQueue/SATB buffer indexes' calculation in ArrayRangeBarrier snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10951
diff
changeset
|
288 long indexValue = thread.readWord(g1CardQueueIndexOffset()).rawValue(); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
289 |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
290 int cardShift = cardTableShift(); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
291 long cardStart = cardTableStart(); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
292 final int scale = arrayIndexScale(Kind.Object); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
293 int header = arrayBaseOffset(Kind.Object); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
294 long dstAddr = GetObjectAddressNode.get(dest); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
295 long start = (dstAddr + header + (long) startIndex * scale) >>> cardShift; |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
296 long end = (dstAddr + header + ((long) startIndex + length - 1) * scale) >>> cardShift; |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
297 long count = end - start + 1; |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
298 |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
299 while (count-- > 0) { |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
300 Word cardAddress = Word.unsigned((start + cardStart) + count); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
301 byte cardByte = cardAddress.readByte(0); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
302 // If the card is already dirty, (hence already enqueued) skip the insertion. |
16018
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
303 if (probability(NOT_FREQUENT_PROBABILITY, cardByte != g1YoungCardValue())) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
304 MembarNode.memoryBarrier(STORE_LOAD); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
305 byte cardByteReload = cardAddress.readByte(0); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
306 if (probability(NOT_FREQUENT_PROBABILITY, cardByteReload != dirtyCardValue())) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
307 cardAddress.writeByte(0, (byte) 0, GC_CARD_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
308 // If the thread local card queue is full, issue a native call which will |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
309 // initialize a new one and add the card entry. |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
310 if (indexValue != 0) { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
311 indexValue = indexValue - wordSize(); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
312 Word logAddress = bufferAddress.add(Word.unsigned(indexValue)); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
313 // Log the object to be scanned as well as update |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
314 // the card queue's next index. |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
315 logAddress.writeWord(0, cardAddress, GC_LOG_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
316 indexAddress.writeWord(0, Word.unsigned(indexValue), GC_INDEX_LOCATION); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
317 } else { |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
318 g1PostBarrierStub(G1WBPOSTCALL, cardAddress); |
6a62ccbd1658
add support for new G1 marking behavior
Tom Rodriguez <tom.rodriguez@oracle.com>
parents:
15259
diff
changeset
|
319 } |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
320 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
321 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
322 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
323 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
324 |
10517
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
325 public static final ForeignCallDescriptor G1WBPRECALL = new ForeignCallDescriptor("write_barrier_pre", void.class, Object.class); |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
326 |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
327 @NodeIntrinsic(ForeignCallNode.class) |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
328 private static native void g1PreBarrierStub(@ConstantNodeParameter ForeignCallDescriptor descriptor, Object object); |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
329 |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
330 public static final ForeignCallDescriptor G1WBPOSTCALL = new ForeignCallDescriptor("write_barrier_post", void.class, Word.class); |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
331 |
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
332 @NodeIntrinsic(ForeignCallNode.class) |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
333 public static native void g1PostBarrierStub(@ConstantNodeParameter ForeignCallDescriptor descriptor, Word card); |
10517
c2c9afee00b5
Replace G1 stub call nodes with intrinsics
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10505
diff
changeset
|
334 |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
335 public static class Templates extends AbstractTemplates { |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
336 |
20178
7ba8fb19d56d
Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
19852
diff
changeset
|
337 private final SnippetInfo serialWriteBarrier = snippet(WriteBarrierSnippets.class, "serialWriteBarrier", GC_CARD_LOCATION); |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
338 private final SnippetInfo serialArrayRangeWriteBarrier = snippet(WriteBarrierSnippets.class, "serialArrayRangeWriteBarrier"); |
20178
7ba8fb19d56d
Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
19852
diff
changeset
|
339 private final SnippetInfo g1PreWriteBarrier = snippet(WriteBarrierSnippets.class, "g1PreWriteBarrier", GC_INDEX_LOCATION, GC_LOG_LOCATION); |
7ba8fb19d56d
Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
19852
diff
changeset
|
340 private final SnippetInfo g1ReferentReadBarrier = snippet(WriteBarrierSnippets.class, "g1PreWriteBarrier", GC_INDEX_LOCATION, GC_LOG_LOCATION); |
7ba8fb19d56d
Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
19852
diff
changeset
|
341 private final SnippetInfo g1PostWriteBarrier = snippet(WriteBarrierSnippets.class, "g1PostWriteBarrier", GC_CARD_LOCATION, GC_INDEX_LOCATION, GC_LOG_LOCATION); |
7ba8fb19d56d
Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
19852
diff
changeset
|
342 private final SnippetInfo g1ArrayRangePreWriteBarrier = snippet(WriteBarrierSnippets.class, "g1ArrayRangePreWriteBarrier", GC_INDEX_LOCATION, GC_LOG_LOCATION); |
7ba8fb19d56d
Check that snippets only access locations that are either accessed by the replacee, or are private to the snippet.
Roland Schatz <roland.schatz@oracle.com>
parents:
19852
diff
changeset
|
343 private final SnippetInfo g1ArrayRangePostWriteBarrier = snippet(WriteBarrierSnippets.class, "g1ArrayRangePostWriteBarrier", GC_CARD_LOCATION, GC_INDEX_LOCATION, GC_LOG_LOCATION); |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
344 |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
345 private final CompressEncoding oopEncoding; |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
346 |
15018
db4254246f9a
Remove Constant.forObject and Constant.asObject to improve compiler/VM separation
Christian Wimmer <christian.wimmer@oracle.com>
parents:
15004
diff
changeset
|
347 public Templates(HotSpotProviders providers, TargetDescription target, CompressEncoding oopEncoding) { |
db4254246f9a
Remove Constant.forObject and Constant.asObject to improve compiler/VM separation
Christian Wimmer <christian.wimmer@oracle.com>
parents:
15004
diff
changeset
|
348 super(providers, providers.getSnippetReflection(), target); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
349 this.oopEncoding = oopEncoding; |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
350 } |
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
351 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
352 public void lower(SerialWriteBarrier writeBarrier, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
353 Arguments args = new Arguments(serialWriteBarrier, writeBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
10944
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
354 args.add("object", writeBarrier.getObject()); |
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
355 args.add("location", writeBarrier.getLocation()); |
3a9f3ec8efc6
Minor refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10942
diff
changeset
|
356 args.addConst("usePrecise", writeBarrier.usePrecise()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
357 template(args).instantiate(providers.getMetaAccess(), writeBarrier, DEFAULT_REPLACER, args); |
8571
0bacc5564d18
Revert WB refactoring
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8531
diff
changeset
|
358 } |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
359 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
360 public void lower(SerialArrayRangeWriteBarrier arrayRangeWriteBarrier, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
361 Arguments args = new Arguments(serialArrayRangeWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
9015
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
362 args.add("object", arrayRangeWriteBarrier.getObject()); |
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
363 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex()); |
92d2bedb5dfc
Change of Snippet template and instantiation process
Christian Wimmer <christian.wimmer@oracle.com>
parents:
8989
diff
changeset
|
364 args.add("length", arrayRangeWriteBarrier.getLength()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
365 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args); |
8963
63eae4723b18
Creation of snippets for ArrayCopy write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
8905
diff
changeset
|
366 } |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
367 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
368 public void lower(G1PreWriteBarrier writeBarrierPre, HotSpotRegistersProvider registers, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
369 Arguments args = new Arguments(g1PreWriteBarrier, writeBarrierPre.graph().getGuardsStage(), tool.getLoweringStage()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
370 args.add("object", writeBarrierPre.getObject()); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
371 |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
372 ValueNode expected = writeBarrierPre.getExpectedObject(); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
373 if (expected != null && expected.stamp() instanceof NarrowOopStamp) { |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
374 assert oopEncoding != null; |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
375 expected = CompressionNode.uncompress(expected, oopEncoding); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
376 } |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
377 args.add("expectedObject", expected); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
378 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
379 args.add("location", writeBarrierPre.getLocation()); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
380 args.addConst("doLoad", writeBarrierPre.doLoad()); |
10669
f11cd00d4115
Make G1PreWriteBarrier a DeoptimizingNode.
Roland Schatz <roland.schatz@oracle.com>
parents:
10631
diff
changeset
|
381 args.addConst("nullCheck", writeBarrierPre.getNullCheck()); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
382 args.addConst("threadRegister", registers.getThreadRegister()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
383 args.addConst("trace", traceBarrier()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
384 template(args).instantiate(providers.getMetaAccess(), writeBarrierPre, DEFAULT_REPLACER, args); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
385 } |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
386 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
387 public void lower(G1ReferentFieldReadBarrier readBarrier, HotSpotRegistersProvider registers, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
388 Arguments args = new Arguments(g1ReferentReadBarrier, readBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
389 args.add("object", readBarrier.getObject()); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
390 |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
391 ValueNode expected = readBarrier.getExpectedObject(); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
392 if (expected != null && expected.stamp() instanceof NarrowOopStamp) { |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
393 assert oopEncoding != null; |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
394 expected = CompressionNode.uncompress(expected, oopEncoding); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
395 } |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
396 |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
397 args.add("expectedObject", expected); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
398 args.add("location", readBarrier.getLocation()); |
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
399 args.addConst("doLoad", readBarrier.doLoad()); |
10941
b9718191858c
G1ReferentFieldReadBarrier is not a deoptimizing node
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10829
diff
changeset
|
400 args.addConst("nullCheck", false); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
401 args.addConst("threadRegister", registers.getThreadRegister()); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
402 args.addConst("trace", traceBarrier()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
403 template(args).instantiate(providers.getMetaAccess(), readBarrier, DEFAULT_REPLACER, args); |
10829
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
404 } |
761002c55728
Add new node for referent field read barrier (G1 GC)
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10828
diff
changeset
|
405 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
406 public void lower(G1PostWriteBarrier writeBarrierPost, HotSpotRegistersProvider registers, LoweringTool tool) { |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
407 StructuredGraph graph = writeBarrierPost.graph(); |
12746
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
408 if (writeBarrierPost.alwaysNull()) { |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
409 graph.removeFixed(writeBarrierPost); |
12746
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
410 return; |
a7eee87c7897
remove use of snippets for lowering nop write barriers
Doug Simon <doug.simon@oracle.com>
parents:
12660
diff
changeset
|
411 } |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
412 Arguments args = new Arguments(g1PostWriteBarrier, graph.getGuardsStage(), tool.getLoweringStage()); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
413 args.add("object", writeBarrierPost.getObject()); |
14945
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
414 |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
415 ValueNode value = writeBarrierPost.getValue(); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
416 if (value.stamp() instanceof NarrowOopStamp) { |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
417 assert oopEncoding != null; |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
418 value = CompressionNode.uncompress(value, oopEncoding); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
419 } |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
420 args.add("value", value); |
b14cb2d9253d
Make compression and uncompression explicit in the high level graph.
Roland Schatz <roland.schatz@oracle.com>
parents:
13505
diff
changeset
|
421 |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
422 args.add("location", writeBarrierPost.getLocation()); |
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
423 args.addConst("usePrecise", writeBarrierPost.usePrecise()); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
424 args.addConst("threadRegister", registers.getThreadRegister()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
425 args.addConst("trace", traceBarrier()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
426 template(args).instantiate(providers.getMetaAccess(), writeBarrierPost, DEFAULT_REPLACER, args); |
10494
6f331db530f6
Add G1 Barrier Snippets
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
9513
diff
changeset
|
427 } |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
428 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
429 public void lower(G1ArrayRangePreWriteBarrier arrayRangeWriteBarrier, HotSpotRegistersProvider registers, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
430 Arguments args = new Arguments(g1ArrayRangePreWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
431 args.add("object", arrayRangeWriteBarrier.getObject()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
432 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
433 args.add("length", arrayRangeWriteBarrier.getLength()); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
434 args.addConst("threadRegister", registers.getThreadRegister()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
435 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
436 } |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
437 |
13505
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
438 public void lower(G1ArrayRangePostWriteBarrier arrayRangeWriteBarrier, HotSpotRegistersProvider registers, LoweringTool tool) { |
75a67ebd50e8
Introduce a LoweringStage so that lowerings can be conditional on one of the three times that lowering is performed by default
Christian Wimmer <christian.wimmer@oracle.com>
parents:
12791
diff
changeset
|
439 Arguments args = new Arguments(g1ArrayRangePostWriteBarrier, arrayRangeWriteBarrier.graph().getGuardsStage(), tool.getLoweringStage()); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
440 args.add("object", arrayRangeWriteBarrier.getObject()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
441 args.add("startIndex", arrayRangeWriteBarrier.getStartIndex()); |
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
442 args.add("length", arrayRangeWriteBarrier.getLength()); |
12791
1cc0a907e1dd
pass thread register into write barrier snippets instead of getting it from the host provider
Doug Simon <doug.simon@oracle.com>
parents:
12746
diff
changeset
|
443 args.addConst("threadRegister", registers.getThreadRegister()); |
12054
bba234a1670e
grouped provider values/parameters into a Providers object (GRAAL-511)
Doug Simon <doug.simon@oracle.com>
parents:
12052
diff
changeset
|
444 template(args).instantiate(providers.getMetaAccess(), arrayRangeWriteBarrier, DEFAULT_REPLACER, args); |
10552
ca8efaf18f27
Add ArrayRange Snippets for G1
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10547
diff
changeset
|
445 } |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
446 } |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
447 |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
448 /** |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
449 * Log method of debugging purposes. |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
450 */ |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
451 public static void log(boolean enabled, String format, long value) { |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
452 if (enabled) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
453 Log.printf(format, value); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
454 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
455 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
456 |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
457 public static void log(boolean enabled, String format, long value1, long value2) { |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
458 if (enabled) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
459 Log.printf(format, value1, value2); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
460 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
461 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
462 |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
463 public static void log(boolean enabled, String format, long value1, long value2, long value3) { |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
464 if (enabled) { |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
465 Log.printf(format, value1, value2, value3); |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
466 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
467 } |
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
468 |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
469 public static boolean traceBarrier() { |
10701
f1cd428fca9f
Remove caching of gcStartCycles
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10669
diff
changeset
|
470 return GraalOptions.GCDebugStartCycle.getValue() > 0 && ((int) Word.unsigned(HotSpotReplacementsUtil.gcTotalCollectionsAddress()).readLong(0) > GraalOptions.GCDebugStartCycle.getValue()); |
10620
260e75dc4b6e
Add tracing info to G1 Write Barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10552
diff
changeset
|
471 } |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
472 |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
473 /** |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
474 * Validation helper method which performs sanity checks on write operations. The addresses of |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
475 * both the object and the value being written are checked in order to determine if they reside |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
476 * in a valid heap region. If an object is stale, an invalid access is performed in order to |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
477 * prematurely crash the VM and debug the stack trace of the faulty method. |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
478 */ |
11234
73eb4d8a4015
Change scope of logging methods
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
11221
diff
changeset
|
479 public static void validateObject(Object parent, Object child) { |
11221
8d4bd13c6983
Field renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10955
diff
changeset
|
480 if (verifyOops() && child != null && !validateOop(VALIDATE_OBJECT, parent, child)) { |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
481 log(true, "Verification ERROR, Parent: %p Child: %p\n", Word.fromObject(parent).rawValue(), Word.fromObject(child).rawValue()); |
19852
3d0116ec99c5
Create utilities LocationIdentity#isAny, LocationIdentity#isSingle, LocationIdentity#any, LocationIdentity#overlaps.
Thomas Wuerthinger <thomas.wuerthinger@oracle.com>
parents:
19775
diff
changeset
|
482 DirectObjectStoreNode.storeObject(null, 0, 0, null, LocationIdentity.any(), Kind.Object); |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
483 } |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
484 } |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
485 |
11221
8d4bd13c6983
Field renaming
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10955
diff
changeset
|
486 public static final ForeignCallDescriptor VALIDATE_OBJECT = new ForeignCallDescriptor("validate_object", boolean.class, Word.class, Word.class); |
10953
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
487 |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
488 @NodeIntrinsic(ForeignCallNode.class) |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
489 private static native boolean validateOop(@ConstantNodeParameter ForeignCallDescriptor descriptor, Object parent, Object object); |
97e282186b5b
Add heap sanity checker with premature hard crash for debugging write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
10952
diff
changeset
|
490 |
8488
225c984588ee
-Snippetization of CMS write barriers
Christos Kotselidis <christos.kotselidis@oracle.com>
parents:
diff
changeset
|
491 } |