graal-compiler: src/share/vm/gc_implementation/g1/satbQueue.cpp annotate

annotate src/share/vm/gc_implementation/g1/satbQueue.cpp @ 3979:4dfb2df418f2

6484982: G1: process references during evacuation pauses Summary: G1 now uses two reference processors - one is used by concurrent marking and the other is used by STW GCs (both full and incremental evacuation pauses). In an evacuation pause, the reference processor is embedded into the closures used to scan objects. Doing so causes causes reference objects to be 'discovered' by the reference processor. At the end of the evacuation pause, these discovered reference objects are processed - preserving (and copying) referent objects (and their reachable graphs) as appropriate. Reviewed-by: ysr, jwilhelm, brutisso, stefank, tonyp

author	johnc
date	Thu, 22 Sep 2011 10:57:37 -0700
parents	7e37af9d69ef
children	2ace1c4ee8da

rev	line source
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	1 /*
2149 7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	2 * Copyright (c) 2001, 2011, Oracle and/or its affiliates. All rights reserved.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	4 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	7 * published by the Free Software Foundation.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	8 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	13 * accompanied this code).
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	14 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	18 *
1552 c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1317 diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1317 diff changeset	20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1317 diff changeset	21 * questions.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	22 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	23 */
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	24
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1552 diff changeset	25 #include "precompiled.hpp"
2149 7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	26 #include "gc_implementation/g1/g1CollectedHeap.inline.hpp"
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1552 diff changeset	27 #include "gc_implementation/g1/satbQueue.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1552 diff changeset	28 #include "memory/allocation.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1552 diff changeset	29 #include "memory/sharedHeap.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1552 diff changeset	30 #include "runtime/mutexLocker.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1552 diff changeset	31 #include "runtime/thread.hpp"
3979 4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	32 #include "runtime/vmThread.hpp"
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	33
2149 7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	34 // This method removes entries from an SATB buffer that will not be
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	35 // useful to the concurrent marking threads. An entry is removed if it
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	36 // satisfies one of the following conditions:
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	37 //
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	38 // * it points to an object outside the G1 heap (G1's concurrent
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	39 // marking only visits objects inside the G1 heap),
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	40 // * it points to an object that has been allocated since marking
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	41 // started (according to SATB those objects do not need to be
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	42 // visited during marking), or
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	43 // * it points to an object that has already been marked (no need to
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	44 // process it again).
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	45 //
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	46 // The rest of the entries will be retained and are compacted towards
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	47 // the top of the buffer. If with this filtering we clear a large
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	48 // enough chunk of the buffer we can re-use it (instead of enqueueing
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	49 // it) and we can just allow the mutator to carry on executing.
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	50
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	51 bool ObjPtrQueue::should_enqueue_buffer() {
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	52 assert(_lock == NULL \|\| _lock->owned_by_self(),
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	53 "we should have taken the lock before calling this");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	54
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	55 // A value of 0 means "don't filter SATB buffers".
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	56 if (G1SATBBufferEnqueueingThresholdPercent == 0) {
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	57 return true;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	58 }
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	59
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	60 G1CollectedHeap* g1h = G1CollectedHeap::heap();
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	61
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	62 // This method should only be called if there is a non-NULL buffer
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	63 // that is full.
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	64 assert(_index == 0, "pre-condition");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	65 assert(_buf != NULL, "pre-condition");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	66
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	67 void** buf = _buf;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	68 size_t sz = _sz;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	69
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	70 // Used for sanity checking at the end of the loop.
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	71 debug_only(size_t entries = 0; size_t retained = 0;)
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	72
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	73 size_t i = sz;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	74 size_t new_index = sz;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	75
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	76 // Given that we are expecting _index == 0, we could have changed
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	77 // the loop condition to (i > 0). But we are using _index for
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	78 // generality.
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	79 while (i > _index) {
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	80 assert(i > 0, "we should have at least one more entry to process");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	81 i -= oopSize;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	82 debug_only(entries += 1;)
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	83 oop* p = (oop*) &buf[byte_index_to_index((int) i)];
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	84 oop obj = *p;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	85 // NULL the entry so that unused parts of the buffer contain NULLs
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	86 // at the end. If we are going to retain it we will copy it to its
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	87 // final place. If we have retained all entries we have visited so
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	88 // far, we'll just end up copying it to the same place.
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	89 *p = NULL;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	90
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	91 bool retain = g1h->is_obj_ill(obj);
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	92 if (retain) {
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	93 assert(new_index > 0, "we should not have already filled up the buffer");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	94 new_index -= oopSize;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	95 assert(new_index >= i,
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	96 "new_index should never be below i, as we alwaysr compact 'up'");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	97 oop* new_p = (oop*) &buf[byte_index_to_index((int) new_index)];
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	98 assert(new_p >= p, "the destination location should never be below "
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	99 "the source as we always compact 'up'");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	100 assert(*new_p == NULL,
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	101 "we should have already cleared the destination location");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	102 *new_p = obj;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	103 debug_only(retained += 1;)
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	104 }
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	105 }
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	106 size_t entries_calc = (sz - _index) / oopSize;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	107 assert(entries == entries_calc, "the number of entries we counted "
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	108 "should match the number of entries we calculated");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	109 size_t retained_calc = (sz - new_index) / oopSize;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	110 assert(retained == retained_calc, "the number of retained entries we counted "
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	111 "should match the number of retained entries we calculated");
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	112 size_t perc = retained_calc * 100 / entries_calc;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	113 bool should_enqueue = perc > (size_t) G1SATBBufferEnqueueingThresholdPercent;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	114 _index = new_index;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	115
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	116 return should_enqueue;
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	117 }
7e37af9d69ef 7011379: G1: overly long concurrent marking cycles tonyp parents: 1972 diff changeset	118
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	119 void ObjPtrQueue::apply_closure(ObjectClosure* cl) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	120 if (_buf != NULL) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	121 apply_closure_to_buffer(cl, _buf, _index, _sz);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	122 _index = _sz;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	123 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	124 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	125
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	126 void ObjPtrQueue::apply_closure_to_buffer(ObjectClosure* cl,
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	127 void** buf, size_t index, size_t sz) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	128 if (cl == NULL) return;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	129 for (size_t i = index; i < sz; i += oopSize) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	130 oop obj = (oop)buf[byte_index_to_index((int)i)];
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	131 // There can be NULL entries because of destructors.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	132 if (obj != NULL) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	133 cl->do_object(obj);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	134 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	135 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	136 }
845 df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	137
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	138 #ifdef ASSERT
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	139 void ObjPtrQueue::verify_oops_in_buffer() {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	140 if (_buf == NULL) return;
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	141 for (size_t i = _index; i < _sz; i += oopSize) {
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	142 oop obj = (oop)_buf[byte_index_to_index((int)i)];
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	143 assert(obj != NULL && obj->is_oop(true /* ignore mark word */),
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	144 "Not an oop");
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	145 }
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	146 }
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	147 #endif
df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	148
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	149 #ifdef _MSC_VER // the use of 'this' below gets a warning, make it go away
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	150 #pragma warning( disable:4355 ) // 'this' : used in base member initializer list
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	151 #endif // _MSC_VER
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	152
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	153
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	154 SATBMarkQueueSet::SATBMarkQueueSet() :
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	155 PtrQueueSet(),
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	156 _closure(NULL), _par_closures(NULL),
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	157 _shared_satb_queue(this, true /perm/)
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	158 {}
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	159
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	160 void SATBMarkQueueSet::initialize(Monitor* cbl_mon, Mutex* fl_lock,
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	161 int process_completed_threshold,
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	162 Mutex* lock) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	163 PtrQueueSet::initialize(cbl_mon, fl_lock, process_completed_threshold, -1);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	164 _shared_satb_queue.set_lock(lock);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	165 if (ParallelGCThreads > 0) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	166 _par_closures = NEW_C_HEAP_ARRAY(ObjectClosure*, ParallelGCThreads);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	167 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	168 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	169
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	170
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	171 void SATBMarkQueueSet::handle_zero_index_for_thread(JavaThread* t) {
845 df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	172 DEBUG_ONLY(t->satb_mark_queue().verify_oops_in_buffer();)
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	173 t->satb_mark_queue().handle_zero_index();
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	174 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	175
1317 d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	176 #ifdef ASSERT
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	177 void SATBMarkQueueSet::dump_active_values(JavaThread* first,
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	178 bool expected_active) {
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	179 gclog_or_tty->print_cr("SATB queue active values for Java Threads");
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	180 gclog_or_tty->print_cr(" SATB queue set: active is %s",
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	181 (is_active()) ? "TRUE" : "FALSE");
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	182 gclog_or_tty->print_cr(" expected_active is %s",
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	183 (expected_active) ? "TRUE" : "FALSE");
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	184 for (JavaThread* t = first; t; t = t->next()) {
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	185 bool active = t->satb_mark_queue().is_active();
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	186 gclog_or_tty->print_cr(" thread %s, active is %s",
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	187 t->name(), (active) ? "TRUE" : "FALSE");
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	188 }
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	189 }
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	190 #endif // ASSERT
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	191
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	192 void SATBMarkQueueSet::set_active_all_threads(bool b,
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	193 bool expected_active) {
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	194 assert(SafepointSynchronize::is_at_safepoint(), "Must be at safepoint.");
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	195 JavaThread* first = Threads::first();
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	196
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	197 #ifdef ASSERT
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	198 if (_all_active != expected_active) {
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	199 dump_active_values(first, expected_active);
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	200
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	201 // I leave this here as a guarantee, instead of an assert, so
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	202 // that it will still be compiled in if we choose to uncomment
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	203 // the #ifdef ASSERT in a product build. The whole block is
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	204 // within an #ifdef ASSERT so the guarantee will not be compiled
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	205 // in a product build anyway.
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	206 guarantee(false,
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	207 "SATB queue set has an unexpected active value");
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	208 }
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	209 #endif // ASSERT
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	210 _all_active = b;
1317 d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	211
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	212 for (JavaThread* t = first; t; t = t->next()) {
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	213 #ifdef ASSERT
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	214 bool active = t->satb_mark_queue().is_active();
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	215 if (active != expected_active) {
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	216 dump_active_values(first, expected_active);
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	217
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	218 // I leave this here as a guarantee, instead of an assert, so
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	219 // that it will still be compiled in if we choose to uncomment
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	220 // the #ifdef ASSERT in a product build. The whole block is
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	221 // within an #ifdef ASSERT so the guarantee will not be compiled
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	222 // in a product build anyway.
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	223 guarantee(false,
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	224 "thread has an unexpected active value in its SATB queue");
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	225 }
d4197f8d516a 6935821: G1: threads created during marking do not active their SATB queues tonyp parents: 1111 diff changeset	226 #endif // ASSERT
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	227 t->satb_mark_queue().set_active(b);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	228 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	229 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	230
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	231 void SATBMarkQueueSet::set_closure(ObjectClosure* closure) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	232 _closure = closure;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	233 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	234
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	235 void SATBMarkQueueSet::set_par_closure(int i, ObjectClosure* par_closure) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	236 assert(ParallelGCThreads > 0 && _par_closures != NULL, "Precondition");
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	237 _par_closures[i] = par_closure;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	238 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	239
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	240 void SATBMarkQueueSet::iterate_closure_all_threads() {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	241 for(JavaThread* t = Threads::first(); t; t = t->next()) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	242 t->satb_mark_queue().apply_closure(_closure);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	243 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	244 shared_satb_queue()->apply_closure(_closure);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	245 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	246
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	247 void SATBMarkQueueSet::par_iterate_closure_all_threads(int worker) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	248 SharedHeap* sh = SharedHeap::heap();
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	249 int parity = sh->strong_roots_parity();
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	250
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	251 for(JavaThread* t = Threads::first(); t; t = t->next()) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	252 if (t->claim_oops_do(true, parity)) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	253 t->satb_mark_queue().apply_closure(_par_closures[worker]);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	254 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	255 }
3979 4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	256
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	257 // We also need to claim the VMThread so that its parity is updated
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	258 // otherwise the next call to Thread::possibly_parallel_oops_do inside
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	259 // a StrongRootsScope might skip the VMThread because it has a stale
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	260 // parity that matches the parity set by the StrongRootsScope
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	261 //
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	262 // Whichever worker succeeds in claiming the VMThread gets to do
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	263 // the shared queue.
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	264
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	265 VMThread* vmt = VMThread::vm_thread();
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	266 if (vmt->claim_oops_do(true, parity)) {
4dfb2df418f2 6484982: G1: process references during evacuation pauses johnc parents: 2149 diff changeset	267 shared_satb_queue()->apply_closure(_par_closures[worker]);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	268 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	269 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	270
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	271 bool SATBMarkQueueSet::apply_closure_to_completed_buffer_work(bool par,
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	272 int worker) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	273 BufferNode* nd = NULL;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	274 {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	275 MutexLockerEx x(_cbl_mon, Mutex::_no_safepoint_check_flag);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	276 if (_completed_buffers_head != NULL) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	277 nd = _completed_buffers_head;
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	278 _completed_buffers_head = nd->next();
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	279 if (_completed_buffers_head == NULL) _completed_buffers_tail = NULL;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	280 _n_completed_buffers--;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	281 if (_n_completed_buffers == 0) _process_completed = false;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	282 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	283 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	284 ObjectClosure* cl = (par ? _par_closures[worker] : _closure);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	285 if (nd != NULL) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	286 void **buf = BufferNode::make_buffer_from_node(nd);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	287 ObjPtrQueue::apply_closure_to_buffer(cl, buf, 0, _sz);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	288 deallocate_buffer(buf);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	289 return true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	290 } else {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	291 return false;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	292 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	293 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	294
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	295 void SATBMarkQueueSet::abandon_partial_marking() {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	296 BufferNode* buffers_to_delete = NULL;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	297 {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	298 MutexLockerEx x(_cbl_mon, Mutex::_no_safepoint_check_flag);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	299 while (_completed_buffers_head != NULL) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	300 BufferNode* nd = _completed_buffers_head;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	301 _completed_buffers_head = nd->next();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	302 nd->set_next(buffers_to_delete);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	303 buffers_to_delete = nd;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	304 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	305 _completed_buffers_tail = NULL;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	306 _n_completed_buffers = 0;
845 df6caf649ff7 6700789: G1: Enable use of compressed oops with G1 heaps ysr parents: 342 diff changeset	307 DEBUG_ONLY(assert_completed_buffer_list_len_correct_locked());
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	308 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	309 while (buffers_to_delete != NULL) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	310 BufferNode* nd = buffers_to_delete;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	311 buffers_to_delete = nd->next();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 845 diff changeset	312 deallocate_buffer(BufferNode::make_buffer_from_node(nd));
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	313 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	314 assert(SafepointSynchronize::is_at_safepoint(), "Must be at safepoint.");
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	315 // So we can safely manipulate these queues.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	316 for (JavaThread* t = Threads::first(); t; t = t->next()) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	317 t->satb_mark_queue().reset();
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	318 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	319 shared_satb_queue()->reset();
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	320 }

Mercurial > hg > graal-compiler

annotate src/share/vm/gc_implementation/g1/satbQueue.cpp @ 3979:4dfb2df418f2