Mercurial > hg > graal-compiler

annotate src/share/vm/gc_implementation/g1/g1OopClosures.inline.hpp @ 1705:2d160770d2e5

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
6814437: G1: remove the _new_refs array Summary: The per-worker _new_refs array is used to hold references that point into the collection set. It is populated during RSet updating and subsequently processed. In the event of an evacuation failure it processed again to recreate the RSets of regions in the collection set. Remove the per-worker _new_refs array by processing the references directly. Use a DirtyCardQueue to hold the cards containing the references so that the RSets of regions in the collection set can be recreated when handling an evacuation failure. Reviewed-by: iveresov, jmasa, tonyp
author johnc
date Mon, 02 Aug 2010 12:51:43 -0700
parents c18cbe5936b8
children f95d63e2154a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
1 /*
1705
2d160770d2e5 6814437: G1: remove the _new_refs array
johnc
parents: 1552
diff changeset
2 * Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved.
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
4 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
5 * This code is free software; you can redistribute it and/or modify it
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
6 * under the terms of the GNU General Public License version 2 only, as
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
7 * published by the Free Software Foundation.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
8 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
9 * This code is distributed in the hope that it will be useful, but WITHOUT
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
12 * version 2 for more details (a copy is included in the LICENSE file that
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
13 * accompanied this code).
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
14 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
15 * You should have received a copy of the GNU General Public License version
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
16 * 2 along with this work; if not, write to the Free Software Foundation,
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
18 *
1552
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories
trims
parents: 1261
diff changeset
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories
trims
parents: 1261
diff changeset
20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories
trims
parents: 1261
diff changeset
21 * questions.
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
22 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
23 */
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
24
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
25 /*
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
26 * This really ought to be an inline function, but apparently the C++
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
27 * compiler sometimes sees fit to ignore inline declarations. Sigh.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
28 */
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
29
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
30 // This must a ifdef'ed because the counting it controls is in a
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
31 // perf-critical inner loop.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
32 #define FILTERINTOCSCLOSURE_DOHISTOGRAMCOUNT 0
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
33
845
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
34 template <class T> inline void FilterIntoCSClosure::do_oop_nv(T* p) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
35 T heap_oop = oopDesc::load_heap_oop(p);
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
36 if (!oopDesc::is_null(heap_oop) &&
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
37 _g1->obj_in_cs(oopDesc::decode_heap_oop_not_null(heap_oop))) {
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
38 _oc->do_oop(p);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
39 #if FILTERINTOCSCLOSURE_DOHISTOGRAMCOUNT
1705
2d160770d2e5 6814437: G1: remove the _new_refs array
johnc
parents: 1552
diff changeset
40 if (_dcto_cl != NULL)
2d160770d2e5 6814437: G1: remove the _new_refs array
johnc
parents: 1552
diff changeset
41 _dcto_cl->incr_count();
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
42 #endif
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
43 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
44 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
45
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
46 #define FILTEROUTOFREGIONCLOSURE_DOHISTOGRAMCOUNT 0
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
47
845
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
48 template <class T> inline void FilterOutOfRegionClosure::do_oop_nv(T* p) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
49 T heap_oop = oopDesc::load_heap_oop(p);
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
50 if (!oopDesc::is_null(heap_oop)) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
51 HeapWord* obj_hw = (HeapWord*)oopDesc::decode_heap_oop_not_null(heap_oop);
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
52 if (obj_hw < _r_bottom || obj_hw >= _r_end) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
53 _oc->do_oop(p);
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
54 #if FILTEROUTOFREGIONCLOSURE_DOHISTOGRAMCOUNT
845
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
55 _out_of_region++;
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
56 #endif
845
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
57 }
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
58 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
59 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
60
845
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
61 template <class T> inline void FilterInHeapRegionAndIntoCSClosure::do_oop_nv(T* p) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
62 T heap_oop = oopDesc::load_heap_oop(p);
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
63 if (!oopDesc::is_null(heap_oop) &&
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
64 _g1->obj_in_cs(oopDesc::decode_heap_oop_not_null(heap_oop)))
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
65 _oc->do_oop(p);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
66 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
67
845
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
68 template <class T> inline void FilterAndMarkInHeapRegionAndIntoCSClosure::do_oop_nv(T* p) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
69 T heap_oop = oopDesc::load_heap_oop(p);
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
70 if (!oopDesc::is_null(heap_oop)) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
71 oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
72 HeapRegion* hr = _g1->heap_region_containing((HeapWord*) obj);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
73 if (hr != NULL) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
74 if (hr->in_collection_set())
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
75 _oc->do_oop(p);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
76 else if (!hr->is_young())
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
77 _cm->grayRoot(obj);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
78 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
79 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
80 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
81
845
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
82 // This closure is applied to the fields of the objects that have just been copied.
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
83 template <class T> inline void G1ParScanClosure::do_oop_nv(T* p) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
84 T heap_oop = oopDesc::load_heap_oop(p);
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
85
845
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
86 if (!oopDesc::is_null(heap_oop)) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
87 oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
88 if (_g1->in_cset_fast_test(obj)) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
89 // We're not going to even bother checking whether the object is
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
90 // already forwarded or not, as this usually causes an immediate
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
91 // stall. We'll try to prefetch the object (for write, given that
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
92 // we might need to install the forwarding reference) and we'll
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
93 // get back to it when pop it from the queue
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
94 Prefetch::write(obj->mark_addr(), 0);
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
95 Prefetch::read(obj->mark_addr(), (HeapWordSize*2));
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
96
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
97 // slightly paranoid test; I'm trying to catch potential
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
98 // problems before we go into push_on_queue to know where the
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
99 // problem is coming from
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
100 assert(obj == oopDesc::load_decode_heap_oop(p),
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
101 "p should still be pointing to obj");
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
102 _par_scan_state->push_on_queue(p);
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
103 } else {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
104 _par_scan_state->update_rs(_from, p, _par_scan_state->queue_num());
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps
ysr
parents: 342
diff changeset
105 }
342
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
106 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector
ysr
parents:
diff changeset
107 }
1261
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
108
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
109 template <class T> inline void G1ParPushHeapRSClosure::do_oop_nv(T* p) {
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
110 T heap_oop = oopDesc::load_heap_oop(p);
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
111
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
112 if (!oopDesc::is_null(heap_oop)) {
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
113 oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
114 if (_g1->in_cset_fast_test(obj)) {
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
115 Prefetch::write(obj->mark_addr(), 0);
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
116 Prefetch::read(obj->mark_addr(), (HeapWordSize*2));
1705
2d160770d2e5 6814437: G1: remove the _new_refs array
johnc
parents: 1552
diff changeset
117
2d160770d2e5 6814437: G1: remove the _new_refs array
johnc
parents: 1552
diff changeset
118 // Place on the references queue
1261
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
119 _par_scan_state->push_on_queue(p);
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
120 }
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
121 }
0414c1049f15 6923991: G1: improve scalability of RSet scanning
iveresov
parents: 845
diff changeset
122 }
1705
2d160770d2e5 6814437: G1: remove the _new_refs array
johnc
parents: 1552
diff changeset
123