graal-compiler: src/share/vm/gc_implementation/parNew/parNewGeneration.cpp annotate

annotate src/share/vm/gc_implementation/parNew/parNewGeneration.cpp @ 10185:d50cc62e94ff

8012715: G1: GraphKit accesses PtrQueue::_index as int but is size_t Summary: In graphKit INT operations were generated to access PtrQueue::_index which has type size_t. This is 64 bit on 64-bit machines. No problems occur on little endian machines as long as the index fits into 32 bit, but on big endian machines the upper part is read, which is zero. This leads to unnecessary branches to the slow path in the runtime. Reviewed-by: twisti, johnc Contributed-by: Martin Doerr <martin.doerr@sap.com>

author	johnc
date	Wed, 24 Apr 2013 14:48:43 -0700
parents	a30e7b564541
children	f2110083203d

rev	line source
0 a61af66fc99e Initial load duke parents: diff changeset	1 /*
4911 d903bf750e9f 7129514: time warp warnings after 7117303 johnc parents: 4728 diff changeset	2 * Copyright (c) 2001, 2012, Oracle and/or its affiliates. All rights reserved.
0 a61af66fc99e Initial load duke parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
a61af66fc99e Initial load duke parents: diff changeset	4 *
a61af66fc99e Initial load duke parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
a61af66fc99e Initial load duke parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
a61af66fc99e Initial load duke parents: diff changeset	7 * published by the Free Software Foundation.
a61af66fc99e Initial load duke parents: diff changeset	8 *
a61af66fc99e Initial load duke parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
a61af66fc99e Initial load duke parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
a61af66fc99e Initial load duke parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
a61af66fc99e Initial load duke parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
a61af66fc99e Initial load duke parents: diff changeset	13 * accompanied this code).
a61af66fc99e Initial load duke parents: diff changeset	14 *
a61af66fc99e Initial load duke parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
a61af66fc99e Initial load duke parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
a61af66fc99e Initial load duke parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
a61af66fc99e Initial load duke parents: diff changeset	18 *
1552 c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1387 diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1387 diff changeset	20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1387 diff changeset	21 * questions.
0 a61af66fc99e Initial load duke parents: diff changeset	22 *
a61af66fc99e Initial load duke parents: diff changeset	23 */
a61af66fc99e Initial load duke parents: diff changeset	24
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	25 #include "precompiled.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	26 #include "gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	27 #include "gc_implementation/parNew/parNewGeneration.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	28 #include "gc_implementation/parNew/parOopClosures.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	29 #include "gc_implementation/shared/adaptiveSizePolicy.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	30 #include "gc_implementation/shared/ageTable.hpp"
6595 aaf61e68b255 6818524: G1: use ergonomic resizing of PLABs johnc parents: 6197 diff changeset	31 #include "gc_implementation/shared/parGCAllocBuffer.hpp"
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	32 #include "gc_implementation/shared/spaceDecorator.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	33 #include "memory/defNewGeneration.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	34 #include "memory/genCollectedHeap.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	35 #include "memory/genOopClosures.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	36 #include "memory/generation.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	37 #include "memory/generation.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	38 #include "memory/referencePolicy.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	39 #include "memory/resourceArea.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	40 #include "memory/sharedHeap.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	41 #include "memory/space.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	42 #include "oops/objArrayOop.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	43 #include "oops/oop.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	44 #include "oops/oop.pcgc.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	45 #include "runtime/handles.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	46 #include "runtime/handles.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	47 #include "runtime/java.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	48 #include "runtime/thread.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	49 #include "utilities/copy.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	50 #include "utilities/globalDefinitions.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1888 diff changeset	51 #include "utilities/workgroup.hpp"
0 a61af66fc99e Initial load duke parents: diff changeset	52
a61af66fc99e Initial load duke parents: diff changeset	53 #ifdef _MSC_VER
a61af66fc99e Initial load duke parents: diff changeset	54 #pragma warning( push )
a61af66fc99e Initial load duke parents: diff changeset	55 #pragma warning( disable:4355 ) // 'this' : used in base member initializer list
a61af66fc99e Initial load duke parents: diff changeset	56 #endif
a61af66fc99e Initial load duke parents: diff changeset	57 ParScanThreadState::ParScanThreadState(Space* to_space_,
a61af66fc99e Initial load duke parents: diff changeset	58 ParNewGeneration* gen_,
a61af66fc99e Initial load duke parents: diff changeset	59 Generation* old_gen_,
a61af66fc99e Initial load duke parents: diff changeset	60 int thread_num_,
a61af66fc99e Initial load duke parents: diff changeset	61 ObjToScanQueueSet* work_queue_set_,
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	62 Stack<oop, mtGC>* overflow_stacks_,
0 a61af66fc99e Initial load duke parents: diff changeset	63 size_t desired_plab_sz_,
a61af66fc99e Initial load duke parents: diff changeset	64 ParallelTaskTerminator& term_) :
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	65 _to_space(to_space_), _old_gen(old_gen_), _young_gen(gen_), _thread_num(thread_num_),
0 a61af66fc99e Initial load duke parents: diff changeset	66 _work_queue(work_queue_set_->queue(thread_num_)), _to_space_full(false),
1836 894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	67 _overflow_stack(overflow_stacks_ ? overflow_stacks_ + thread_num_ : NULL),
0 a61af66fc99e Initial load duke parents: diff changeset	68 _ageTable(false), // false ==> not the global age table, no perf data.
a61af66fc99e Initial load duke parents: diff changeset	69 _to_space_alloc_buffer(desired_plab_sz_),
a61af66fc99e Initial load duke parents: diff changeset	70 _to_space_closure(gen_, this), _old_gen_closure(gen_, this),
a61af66fc99e Initial load duke parents: diff changeset	71 _to_space_root_closure(gen_, this), _old_gen_root_closure(gen_, this),
a61af66fc99e Initial load duke parents: diff changeset	72 _older_gen_closure(gen_, this),
a61af66fc99e Initial load duke parents: diff changeset	73 _evacuate_followers(this, &_to_space_closure, &_old_gen_closure,
a61af66fc99e Initial load duke parents: diff changeset	74 &_to_space_root_closure, gen_, &_old_gen_root_closure,
a61af66fc99e Initial load duke parents: diff changeset	75 work_queue_set_, &term_),
a61af66fc99e Initial load duke parents: diff changeset	76 _is_alive_closure(gen_), _scan_weak_ref_closure(gen_, this),
a61af66fc99e Initial load duke parents: diff changeset	77 _keep_alive_closure(&_scan_weak_ref_closure),
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	78 _promotion_failure_size(0),
0 a61af66fc99e Initial load duke parents: diff changeset	79 _strong_roots_time(0.0), _term_time(0.0)
a61af66fc99e Initial load duke parents: diff changeset	80 {
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	81 #if TASKQUEUE_STATS
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	82 _term_attempts = 0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	83 _overflow_refills = 0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	84 _overflow_refill_objs = 0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	85 #endif // TASKQUEUE_STATS
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	86
0 a61af66fc99e Initial load duke parents: diff changeset	87 _survivor_chunk_array =
a61af66fc99e Initial load duke parents: diff changeset	88 (ChunkArray*) old_gen()->get_data_recorder(thread_num());
a61af66fc99e Initial load duke parents: diff changeset	89 _hash_seed = 17; // Might want to take time-based random value.
a61af66fc99e Initial load duke parents: diff changeset	90 _start = os::elapsedTime();
a61af66fc99e Initial load duke parents: diff changeset	91 _old_gen_closure.set_generation(old_gen_);
a61af66fc99e Initial load duke parents: diff changeset	92 _old_gen_root_closure.set_generation(old_gen_);
a61af66fc99e Initial load duke parents: diff changeset	93 }
a61af66fc99e Initial load duke parents: diff changeset	94 #ifdef _MSC_VER
a61af66fc99e Initial load duke parents: diff changeset	95 #pragma warning( pop )
a61af66fc99e Initial load duke parents: diff changeset	96 #endif
a61af66fc99e Initial load duke parents: diff changeset	97
a61af66fc99e Initial load duke parents: diff changeset	98 void ParScanThreadState::record_survivor_plab(HeapWord* plab_start,
a61af66fc99e Initial load duke parents: diff changeset	99 size_t plab_word_size) {
a61af66fc99e Initial load duke parents: diff changeset	100 ChunkArray* sca = survivor_chunk_array();
a61af66fc99e Initial load duke parents: diff changeset	101 if (sca != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	102 // A non-null SCA implies that we want the PLAB data recorded.
a61af66fc99e Initial load duke parents: diff changeset	103 sca->record_sample(plab_start, plab_word_size);
a61af66fc99e Initial load duke parents: diff changeset	104 }
a61af66fc99e Initial load duke parents: diff changeset	105 }
a61af66fc99e Initial load duke parents: diff changeset	106
a61af66fc99e Initial load duke parents: diff changeset	107 bool ParScanThreadState::should_be_partially_scanned(oop new_obj, oop old_obj) const {
a61af66fc99e Initial load duke parents: diff changeset	108 return new_obj->is_objArray() &&
a61af66fc99e Initial load duke parents: diff changeset	109 arrayOop(new_obj)->length() > ParGCArrayScanChunk &&
a61af66fc99e Initial load duke parents: diff changeset	110 new_obj != old_obj;
a61af66fc99e Initial load duke parents: diff changeset	111 }
a61af66fc99e Initial load duke parents: diff changeset	112
a61af66fc99e Initial load duke parents: diff changeset	113 void ParScanThreadState::scan_partial_array_and_push_remainder(oop old) {
a61af66fc99e Initial load duke parents: diff changeset	114 assert(old->is_objArray(), "must be obj array");
a61af66fc99e Initial load duke parents: diff changeset	115 assert(old->is_forwarded(), "must be forwarded");
a61af66fc99e Initial load duke parents: diff changeset	116 assert(Universe::heap()->is_in_reserved(old), "must be in heap.");
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	117 assert(!old_gen()->is_in(old), "must be in young generation.");
0 a61af66fc99e Initial load duke parents: diff changeset	118
a61af66fc99e Initial load duke parents: diff changeset	119 objArrayOop obj = objArrayOop(old->forwardee());
a61af66fc99e Initial load duke parents: diff changeset	120 // Process ParGCArrayScanChunk elements now
a61af66fc99e Initial load duke parents: diff changeset	121 // and push the remainder back onto queue
a61af66fc99e Initial load duke parents: diff changeset	122 int start = arrayOop(old)->length();
a61af66fc99e Initial load duke parents: diff changeset	123 int end = obj->length();
a61af66fc99e Initial load duke parents: diff changeset	124 int remainder = end - start;
a61af66fc99e Initial load duke parents: diff changeset	125 assert(start <= end, "just checking");
a61af66fc99e Initial load duke parents: diff changeset	126 if (remainder > 2 * ParGCArrayScanChunk) {
a61af66fc99e Initial load duke parents: diff changeset	127 // Test above combines last partial chunk with a full chunk
a61af66fc99e Initial load duke parents: diff changeset	128 end = start + ParGCArrayScanChunk;
a61af66fc99e Initial load duke parents: diff changeset	129 arrayOop(old)->set_length(end);
a61af66fc99e Initial load duke parents: diff changeset	130 // Push remainder.
a61af66fc99e Initial load duke parents: diff changeset	131 bool ok = work_queue()->push(old);
a61af66fc99e Initial load duke parents: diff changeset	132 assert(ok, "just popped, push must be okay");
a61af66fc99e Initial load duke parents: diff changeset	133 } else {
a61af66fc99e Initial load duke parents: diff changeset	134 // Restore length so that it can be used if there
a61af66fc99e Initial load duke parents: diff changeset	135 // is a promotion failure and forwarding pointers
a61af66fc99e Initial load duke parents: diff changeset	136 // must be removed.
a61af66fc99e Initial load duke parents: diff changeset	137 arrayOop(old)->set_length(end);
a61af66fc99e Initial load duke parents: diff changeset	138 }
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	139
0 a61af66fc99e Initial load duke parents: diff changeset	140 // process our set of indices (include header in first chunk)
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	141 // should make sure end is even (aligned to HeapWord in case of compressed oops)
0 a61af66fc99e Initial load duke parents: diff changeset	142 if ((HeapWord *)obj < young_old_boundary()) {
a61af66fc99e Initial load duke parents: diff changeset	143 // object is in to_space
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	144 obj->oop_iterate_range(&_to_space_closure, start, end);
0 a61af66fc99e Initial load duke parents: diff changeset	145 } else {
a61af66fc99e Initial load duke parents: diff changeset	146 // object is in old generation
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	147 obj->oop_iterate_range(&_old_gen_closure, start, end);
0 a61af66fc99e Initial load duke parents: diff changeset	148 }
a61af66fc99e Initial load duke parents: diff changeset	149 }
a61af66fc99e Initial load duke parents: diff changeset	150
a61af66fc99e Initial load duke parents: diff changeset	151
a61af66fc99e Initial load duke parents: diff changeset	152 void ParScanThreadState::trim_queues(int max_size) {
a61af66fc99e Initial load duke parents: diff changeset	153 ObjToScanQueue* queue = work_queue();
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	154 do {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	155 while (queue->size() > (juint)max_size) {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	156 oop obj_to_scan;
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	157 if (queue->pop_local(obj_to_scan)) {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	158 if ((HeapWord *)obj_to_scan < young_old_boundary()) {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	159 if (obj_to_scan->is_objArray() &&
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	160 obj_to_scan->is_forwarded() &&
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	161 obj_to_scan->forwardee() != obj_to_scan) {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	162 scan_partial_array_and_push_remainder(obj_to_scan);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	163 } else {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	164 // object is in to_space
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	165 obj_to_scan->oop_iterate(&_to_space_closure);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	166 }
0 a61af66fc99e Initial load duke parents: diff changeset	167 } else {
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	168 // object is in old generation
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	169 obj_to_scan->oop_iterate(&_old_gen_closure);
0 a61af66fc99e Initial load duke parents: diff changeset	170 }
a61af66fc99e Initial load duke parents: diff changeset	171 }
a61af66fc99e Initial load duke parents: diff changeset	172 }
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	173 // For the case of compressed oops, we have a private, non-shared
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	174 // overflow stack, so we eagerly drain it so as to more evenly
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	175 // distribute load early. Note: this may be good to do in
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	176 // general rather than delay for the final stealing phase.
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	177 // If applicable, we'll transfer a set of objects over to our
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	178 // work queue, allowing them to be stolen and draining our
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	179 // private overflow stack.
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	180 } while (ParGCTrimOverflow && young_gen()->take_from_overflow_list(this));
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	181 }
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	182
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	183 bool ParScanThreadState::take_from_overflow_stack() {
695 becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	184 assert(ParGCUseLocalOverflow, "Else should not call");
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	185 assert(young_gen()->overflow_list() == NULL, "Error");
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	186 ObjToScanQueue* queue = work_queue();
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	187 Stack<oop, mtGC>* const of_stack = overflow_stack();
1836 894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	188 const size_t num_overflow_elems = of_stack->size();
894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	189 const size_t space_available = queue->max_elems() - queue->size();
894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	190 const size_t num_take_elems = MIN3(space_available / 4,
894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	191 ParGCDesiredObjsFromOverflowList,
894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	192 num_overflow_elems);
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	193 // Transfer the most recent num_take_elems from the overflow
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	194 // stack to our work queue.
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	195 for (size_t i = 0; i != num_take_elems; i++) {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	196 oop cur = of_stack->pop();
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	197 oop obj_to_push = cur->forwardee();
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	198 assert(Universe::heap()->is_in_reserved(cur), "Should be in heap");
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	199 assert(!old_gen()->is_in_reserved(cur), "Should be in young gen");
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	200 assert(Universe::heap()->is_in_reserved(obj_to_push), "Should be in heap");
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	201 if (should_be_partially_scanned(obj_to_push, cur)) {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	202 assert(arrayOop(cur)->length() == 0, "entire array remaining to be scanned");
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	203 obj_to_push = cur;
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	204 }
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	205 bool ok = queue->push(obj_to_push);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	206 assert(ok, "Should have succeeded");
0 a61af66fc99e Initial load duke parents: diff changeset	207 }
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	208 assert(young_gen()->overflow_list() == NULL, "Error");
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	209 return num_take_elems > 0; // was something transferred?
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	210 }
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	211
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	212 void ParScanThreadState::push_on_overflow_stack(oop p) {
695 becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	213 assert(ParGCUseLocalOverflow, "Else should not call");
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	214 overflow_stack()->push(p);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	215 assert(young_gen()->overflow_list() == NULL, "Error");
0 a61af66fc99e Initial load duke parents: diff changeset	216 }
a61af66fc99e Initial load duke parents: diff changeset	217
a61af66fc99e Initial load duke parents: diff changeset	218 HeapWord* ParScanThreadState::alloc_in_to_space_slow(size_t word_sz) {
a61af66fc99e Initial load duke parents: diff changeset	219
a61af66fc99e Initial load duke parents: diff changeset	220 // Otherwise, if the object is small enough, try to reallocate the
a61af66fc99e Initial load duke parents: diff changeset	221 // buffer.
a61af66fc99e Initial load duke parents: diff changeset	222 HeapWord* obj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	223 if (!_to_space_full) {
a61af66fc99e Initial load duke parents: diff changeset	224 ParGCAllocBuffer* const plab = to_space_alloc_buffer();
a61af66fc99e Initial load duke parents: diff changeset	225 Space* const sp = to_space();
a61af66fc99e Initial load duke parents: diff changeset	226 if (word_sz * 100 <
a61af66fc99e Initial load duke parents: diff changeset	227 ParallelGCBufferWastePct * plab->word_sz()) {
a61af66fc99e Initial load duke parents: diff changeset	228 // Is small enough; abandon this buffer and start a new one.
a61af66fc99e Initial load duke parents: diff changeset	229 plab->retire(false, false);
a61af66fc99e Initial load duke parents: diff changeset	230 size_t buf_size = plab->word_sz();
a61af66fc99e Initial load duke parents: diff changeset	231 HeapWord* buf_space = sp->par_allocate(buf_size);
a61af66fc99e Initial load duke parents: diff changeset	232 if (buf_space == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	233 const size_t min_bytes =
a61af66fc99e Initial load duke parents: diff changeset	234 ParGCAllocBuffer::min_size() << LogHeapWordSize;
a61af66fc99e Initial load duke parents: diff changeset	235 size_t free_bytes = sp->free();
a61af66fc99e Initial load duke parents: diff changeset	236 while(buf_space == NULL && free_bytes >= min_bytes) {
a61af66fc99e Initial load duke parents: diff changeset	237 buf_size = free_bytes >> LogHeapWordSize;
a61af66fc99e Initial load duke parents: diff changeset	238 assert(buf_size == (size_t)align_object_size(buf_size),
a61af66fc99e Initial load duke parents: diff changeset	239 "Invariant");
a61af66fc99e Initial load duke parents: diff changeset	240 buf_space = sp->par_allocate(buf_size);
a61af66fc99e Initial load duke parents: diff changeset	241 free_bytes = sp->free();
a61af66fc99e Initial load duke parents: diff changeset	242 }
a61af66fc99e Initial load duke parents: diff changeset	243 }
a61af66fc99e Initial load duke parents: diff changeset	244 if (buf_space != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	245 plab->set_word_size(buf_size);
a61af66fc99e Initial load duke parents: diff changeset	246 plab->set_buf(buf_space);
a61af66fc99e Initial load duke parents: diff changeset	247 record_survivor_plab(buf_space, buf_size);
a61af66fc99e Initial load duke parents: diff changeset	248 obj = plab->allocate(word_sz);
a61af66fc99e Initial load duke parents: diff changeset	249 // Note that we cannot compare buf_size < word_sz below
a61af66fc99e Initial load duke parents: diff changeset	250 // because of AlignmentReserve (see ParGCAllocBuffer::allocate()).
a61af66fc99e Initial load duke parents: diff changeset	251 assert(obj != NULL \|\| plab->words_remaining() < word_sz,
a61af66fc99e Initial load duke parents: diff changeset	252 "Else should have been able to allocate");
a61af66fc99e Initial load duke parents: diff changeset	253 // It's conceivable that we may be able to use the
a61af66fc99e Initial load duke parents: diff changeset	254 // buffer we just grabbed for subsequent small requests
a61af66fc99e Initial load duke parents: diff changeset	255 // even if not for this one.
a61af66fc99e Initial load duke parents: diff changeset	256 } else {
a61af66fc99e Initial load duke parents: diff changeset	257 // We're used up.
a61af66fc99e Initial load duke parents: diff changeset	258 _to_space_full = true;
a61af66fc99e Initial load duke parents: diff changeset	259 }
a61af66fc99e Initial load duke parents: diff changeset	260
a61af66fc99e Initial load duke parents: diff changeset	261 } else {
a61af66fc99e Initial load duke parents: diff changeset	262 // Too large; allocate the object individually.
a61af66fc99e Initial load duke parents: diff changeset	263 obj = sp->par_allocate(word_sz);
a61af66fc99e Initial load duke parents: diff changeset	264 }
a61af66fc99e Initial load duke parents: diff changeset	265 }
a61af66fc99e Initial load duke parents: diff changeset	266 return obj;
a61af66fc99e Initial load duke parents: diff changeset	267 }
a61af66fc99e Initial load duke parents: diff changeset	268
a61af66fc99e Initial load duke parents: diff changeset	269
a61af66fc99e Initial load duke parents: diff changeset	270 void ParScanThreadState::undo_alloc_in_to_space(HeapWord* obj,
a61af66fc99e Initial load duke parents: diff changeset	271 size_t word_sz) {
a61af66fc99e Initial load duke parents: diff changeset	272 // Is the alloc in the current alloc buffer?
a61af66fc99e Initial load duke parents: diff changeset	273 if (to_space_alloc_buffer()->contains(obj)) {
a61af66fc99e Initial load duke parents: diff changeset	274 assert(to_space_alloc_buffer()->contains(obj + word_sz - 1),
a61af66fc99e Initial load duke parents: diff changeset	275 "Should contain whole object.");
a61af66fc99e Initial load duke parents: diff changeset	276 to_space_alloc_buffer()->undo_allocation(obj, word_sz);
a61af66fc99e Initial load duke parents: diff changeset	277 } else {
481 7d7a7c599c17 6578152: fill_region_with_object has usability and safety issues jcoomes parents: 457 diff changeset	278 CollectedHeap::fill_with_object(obj, word_sz);
0 a61af66fc99e Initial load duke parents: diff changeset	279 }
a61af66fc99e Initial load duke parents: diff changeset	280 }
a61af66fc99e Initial load duke parents: diff changeset	281
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	282 void ParScanThreadState::print_and_clear_promotion_failure_size() {
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	283 if (_promotion_failure_size != 0) {
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	284 if (PrintPromotionFailure) {
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	285 gclog_or_tty->print(" (%d: promotion failure size = " SIZE_FORMAT ") ",
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	286 _thread_num, _promotion_failure_size);
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	287 }
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	288 _promotion_failure_size = 0;
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	289 }
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	290 }
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	291
0 a61af66fc99e Initial load duke parents: diff changeset	292 class ParScanThreadStateSet: private ResourceArray {
a61af66fc99e Initial load duke parents: diff changeset	293 public:
a61af66fc99e Initial load duke parents: diff changeset	294 // Initializes states for the specified number of threads;
a61af66fc99e Initial load duke parents: diff changeset	295 ParScanThreadStateSet(int num_threads,
a61af66fc99e Initial load duke parents: diff changeset	296 Space& to_space,
a61af66fc99e Initial load duke parents: diff changeset	297 ParNewGeneration& gen,
a61af66fc99e Initial load duke parents: diff changeset	298 Generation& old_gen,
a61af66fc99e Initial load duke parents: diff changeset	299 ObjToScanQueueSet& queue_set,
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	300 Stack<oop, mtGC>* overflow_stacks_,
0 a61af66fc99e Initial load duke parents: diff changeset	301 size_t desired_plab_sz,
a61af66fc99e Initial load duke parents: diff changeset	302 ParallelTaskTerminator& term);
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	303
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	304 ~ParScanThreadStateSet() { TASKQUEUE_STATS_ONLY(reset_stats()); }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	305
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	306 inline ParScanThreadState& thread_state(int i);
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	307
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	308 void reset(int active_workers, bool promotion_failed);
0 a61af66fc99e Initial load duke parents: diff changeset	309 void flush();
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	310
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	311 #if TASKQUEUE_STATS
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	312 static void
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	313 print_termination_stats_hdr(outputStream* const st = gclog_or_tty);
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	314 void print_termination_stats(outputStream* const st = gclog_or_tty);
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	315 static void
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	316 print_taskqueue_stats_hdr(outputStream* const st = gclog_or_tty);
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	317 void print_taskqueue_stats(outputStream* const st = gclog_or_tty);
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	318 void reset_stats();
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	319 #endif // TASKQUEUE_STATS
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	320
0 a61af66fc99e Initial load duke parents: diff changeset	321 private:
a61af66fc99e Initial load duke parents: diff changeset	322 ParallelTaskTerminator& _term;
a61af66fc99e Initial load duke parents: diff changeset	323 ParNewGeneration& _gen;
a61af66fc99e Initial load duke parents: diff changeset	324 Generation& _next_gen;
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	325 public:
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	326 bool is_valid(int id) const { return id < length(); }
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	327 ParallelTaskTerminator* terminator() { return &_term; }
0 a61af66fc99e Initial load duke parents: diff changeset	328 };
a61af66fc99e Initial load duke parents: diff changeset	329
a61af66fc99e Initial load duke parents: diff changeset	330
a61af66fc99e Initial load duke parents: diff changeset	331 ParScanThreadStateSet::ParScanThreadStateSet(
a61af66fc99e Initial load duke parents: diff changeset	332 int num_threads, Space& to_space, ParNewGeneration& gen,
a61af66fc99e Initial load duke parents: diff changeset	333 Generation& old_gen, ObjToScanQueueSet& queue_set,
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	334 Stack<oop, mtGC>* overflow_stacks,
0 a61af66fc99e Initial load duke parents: diff changeset	335 size_t desired_plab_sz, ParallelTaskTerminator& term)
a61af66fc99e Initial load duke parents: diff changeset	336 : ResourceArray(sizeof(ParScanThreadState), num_threads),
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	337 _gen(gen), _next_gen(old_gen), _term(term)
0 a61af66fc99e Initial load duke parents: diff changeset	338 {
a61af66fc99e Initial load duke parents: diff changeset	339 assert(num_threads > 0, "sanity check!");
1836 894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	340 assert(ParGCUseLocalOverflow == (overflow_stacks != NULL),
894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	341 "overflow_stack allocation mismatch");
0 a61af66fc99e Initial load duke parents: diff changeset	342 // Initialize states.
a61af66fc99e Initial load duke parents: diff changeset	343 for (int i = 0; i < num_threads; ++i) {
a61af66fc99e Initial load duke parents: diff changeset	344 new ((ParScanThreadState*)_data + i)
a61af66fc99e Initial load duke parents: diff changeset	345 ParScanThreadState(&to_space, &gen, &old_gen, i, &queue_set,
1836 894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	346 overflow_stacks, desired_plab_sz, term);
0 a61af66fc99e Initial load duke parents: diff changeset	347 }
a61af66fc99e Initial load duke parents: diff changeset	348 }
a61af66fc99e Initial load duke parents: diff changeset	349
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	350 inline ParScanThreadState& ParScanThreadStateSet::thread_state(int i)
0 a61af66fc99e Initial load duke parents: diff changeset	351 {
a61af66fc99e Initial load duke parents: diff changeset	352 assert(i >= 0 && i < length(), "sanity check!");
a61af66fc99e Initial load duke parents: diff changeset	353 return ((ParScanThreadState*)_data)[i];
a61af66fc99e Initial load duke parents: diff changeset	354 }
a61af66fc99e Initial load duke parents: diff changeset	355
a61af66fc99e Initial load duke parents: diff changeset	356
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	357 void ParScanThreadStateSet::reset(int active_threads, bool promotion_failed)
0 a61af66fc99e Initial load duke parents: diff changeset	358 {
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	359 _term.reset_for_reuse(active_threads);
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	360 if (promotion_failed) {
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	361 for (int i = 0; i < length(); ++i) {
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	362 thread_state(i).print_and_clear_promotion_failure_size();
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	363 }
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	364 }
0 a61af66fc99e Initial load duke parents: diff changeset	365 }
a61af66fc99e Initial load duke parents: diff changeset	366
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	367 #if TASKQUEUE_STATS
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	368 void
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	369 ParScanThreadState::reset_stats()
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	370 {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	371 taskqueue_stats().reset();
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	372 _term_attempts = 0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	373 _overflow_refills = 0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	374 _overflow_refill_objs = 0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	375 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	376
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	377 void ParScanThreadStateSet::reset_stats()
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	378 {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	379 for (int i = 0; i < length(); ++i) {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	380 thread_state(i).reset_stats();
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	381 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	382 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	383
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	384 void
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	385 ParScanThreadStateSet::print_termination_stats_hdr(outputStream* const st)
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	386 {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	387 st->print_raw_cr("GC Termination Stats");
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	388 st->print_raw_cr(" elapsed --strong roots-- "
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	389 "-------termination-------");
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	390 st->print_raw_cr("thr ms ms % "
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	391 " ms % attempts");
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	392 st->print_raw_cr("--- --------- --------- ------ "
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	393 "--------- ------ --------");
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	394 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	395
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	396 void ParScanThreadStateSet::print_termination_stats(outputStream* const st)
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	397 {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	398 print_termination_stats_hdr(st);
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	399
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	400 for (int i = 0; i < length(); ++i) {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	401 const ParScanThreadState & pss = thread_state(i);
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	402 const double elapsed_ms = pss.elapsed_time() * 1000.0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	403 const double s_roots_ms = pss.strong_roots_time() * 1000.0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	404 const double term_ms = pss.term_time() * 1000.0;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	405 st->print_cr("%3d %9.2f %9.2f %6.2f "
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	406 "%9.2f %6.2f " SIZE_FORMAT_W(8),
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	407 i, elapsed_ms, s_roots_ms, s_roots_ms * 100 / elapsed_ms,
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	408 term_ms, term_ms * 100 / elapsed_ms, pss.term_attempts());
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	409 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	410 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	411
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	412 // Print stats related to work queue activity.
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	413 void ParScanThreadStateSet::print_taskqueue_stats_hdr(outputStream* const st)
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	414 {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	415 st->print_raw_cr("GC Task Stats");
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	416 st->print_raw("thr "); TaskQueueStats::print_header(1, st); st->cr();
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	417 st->print_raw("--- "); TaskQueueStats::print_header(2, st); st->cr();
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	418 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	419
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	420 void ParScanThreadStateSet::print_taskqueue_stats(outputStream* const st)
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	421 {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	422 print_taskqueue_stats_hdr(st);
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	423
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	424 TaskQueueStats totals;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	425 for (int i = 0; i < length(); ++i) {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	426 const ParScanThreadState & pss = thread_state(i);
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	427 const TaskQueueStats & stats = pss.taskqueue_stats();
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	428 st->print("%3d ", i); stats.print(st); st->cr();
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	429 totals += stats;
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	430
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	431 if (pss.overflow_refills() > 0) {
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	432 st->print_cr(" " SIZE_FORMAT_W(10) " overflow refills "
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	433 SIZE_FORMAT_W(10) " overflow objects",
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	434 pss.overflow_refills(), pss.overflow_refill_objs());
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	435 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	436 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	437 st->print("tot "); totals.print(st); st->cr();
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	438
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	439 DEBUG_ONLY(totals.verify());
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	440 }
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	441 #endif // TASKQUEUE_STATS
94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	442
0 a61af66fc99e Initial load duke parents: diff changeset	443 void ParScanThreadStateSet::flush()
a61af66fc99e Initial load duke parents: diff changeset	444 {
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	445 // Work in this loop should be kept as lightweight as
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	446 // possible since this might otherwise become a bottleneck
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	447 // to scaling. Should we add heavy-weight work into this
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	448 // loop, consider parallelizing the loop into the worker threads.
0 a61af66fc99e Initial load duke parents: diff changeset	449 for (int i = 0; i < length(); ++i) {
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	450 ParScanThreadState& par_scan_state = thread_state(i);
0 a61af66fc99e Initial load duke parents: diff changeset	451
a61af66fc99e Initial load duke parents: diff changeset	452 // Flush stats related to To-space PLAB activity and
a61af66fc99e Initial load duke parents: diff changeset	453 // retire the last buffer.
a61af66fc99e Initial load duke parents: diff changeset	454 par_scan_state.to_space_alloc_buffer()->
a61af66fc99e Initial load duke parents: diff changeset	455 flush_stats_and_retire(_gen.plab_stats(),
6595 aaf61e68b255 6818524: G1: use ergonomic resizing of PLABs johnc parents: 6197 diff changeset	456 true /* end_of_gc */,
aaf61e68b255 6818524: G1: use ergonomic resizing of PLABs johnc parents: 6197 diff changeset	457 false /* retain */);
0 a61af66fc99e Initial load duke parents: diff changeset	458
a61af66fc99e Initial load duke parents: diff changeset	459 // Every thread has its own age table. We need to merge
a61af66fc99e Initial load duke parents: diff changeset	460 // them all into one.
a61af66fc99e Initial load duke parents: diff changeset	461 ageTable *local_table = par_scan_state.age_table();
a61af66fc99e Initial load duke parents: diff changeset	462 _gen.age_table()->merge(local_table);
a61af66fc99e Initial load duke parents: diff changeset	463
a61af66fc99e Initial load duke parents: diff changeset	464 // Inform old gen that we're done.
a61af66fc99e Initial load duke parents: diff changeset	465 _next_gen.par_promote_alloc_done(i);
a61af66fc99e Initial load duke parents: diff changeset	466 _next_gen.par_oop_since_save_marks_iterate_done(i);
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	467 }
0 a61af66fc99e Initial load duke parents: diff changeset	468
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	469 if (UseConcMarkSweepGC && ParallelGCThreads > 0) {
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	470 // We need to call this even when ResizeOldPLAB is disabled
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	471 // so as to avoid breaking some asserts. While we may be able
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	472 // to avoid this by reorganizing the code a bit, I am loathe
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	473 // to do that unless we find cases where ergo leads to bad
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	474 // performance.
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	475 CFLS_LAB::compute_desired_plab_size();
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	476 }
0 a61af66fc99e Initial load duke parents: diff changeset	477 }
a61af66fc99e Initial load duke parents: diff changeset	478
a61af66fc99e Initial load duke parents: diff changeset	479 ParScanClosure::ParScanClosure(ParNewGeneration* g,
a61af66fc99e Initial load duke parents: diff changeset	480 ParScanThreadState* par_scan_state) :
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	481 OopsInKlassOrGenClosure(g), _par_scan_state(par_scan_state), _g(g)
0 a61af66fc99e Initial load duke parents: diff changeset	482 {
a61af66fc99e Initial load duke parents: diff changeset	483 assert(_g->level() == 0, "Optimized for youngest generation");
a61af66fc99e Initial load duke parents: diff changeset	484 _boundary = _g->reserved().end();
a61af66fc99e Initial load duke parents: diff changeset	485 }
a61af66fc99e Initial load duke parents: diff changeset	486
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	487 void ParScanWithBarrierClosure::do_oop(oop* p) { ParScanClosure::do_oop_work(p, true, false); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	488 void ParScanWithBarrierClosure::do_oop(narrowOop* p) { ParScanClosure::do_oop_work(p, true, false); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	489
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	490 void ParScanWithoutBarrierClosure::do_oop(oop* p) { ParScanClosure::do_oop_work(p, false, false); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	491 void ParScanWithoutBarrierClosure::do_oop(narrowOop* p) { ParScanClosure::do_oop_work(p, false, false); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	492
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	493 void ParRootScanWithBarrierTwoGensClosure::do_oop(oop* p) { ParScanClosure::do_oop_work(p, true, true); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	494 void ParRootScanWithBarrierTwoGensClosure::do_oop(narrowOop* p) { ParScanClosure::do_oop_work(p, true, true); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	495
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	496 void ParRootScanWithoutBarrierClosure::do_oop(oop* p) { ParScanClosure::do_oop_work(p, false, true); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	497 void ParRootScanWithoutBarrierClosure::do_oop(narrowOop* p) { ParScanClosure::do_oop_work(p, false, true); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	498
0 a61af66fc99e Initial load duke parents: diff changeset	499 ParScanWeakRefClosure::ParScanWeakRefClosure(ParNewGeneration* g,
a61af66fc99e Initial load duke parents: diff changeset	500 ParScanThreadState* par_scan_state)
a61af66fc99e Initial load duke parents: diff changeset	501 : ScanWeakRefClosure(g), _par_scan_state(par_scan_state)
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	502 {}
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	503
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	504 void ParScanWeakRefClosure::do_oop(oop* p) { ParScanWeakRefClosure::do_oop_work(p); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	505 void ParScanWeakRefClosure::do_oop(narrowOop* p) { ParScanWeakRefClosure::do_oop_work(p); }
0 a61af66fc99e Initial load duke parents: diff changeset	506
a61af66fc99e Initial load duke parents: diff changeset	507 #ifdef WIN32
a61af66fc99e Initial load duke parents: diff changeset	508 #pragma warning(disable: 4786) /* identifier was truncated to '255' characters in the browser information */
a61af66fc99e Initial load duke parents: diff changeset	509 #endif
a61af66fc99e Initial load duke parents: diff changeset	510
a61af66fc99e Initial load duke parents: diff changeset	511 ParEvacuateFollowersClosure::ParEvacuateFollowersClosure(
a61af66fc99e Initial load duke parents: diff changeset	512 ParScanThreadState* par_scan_state_,
a61af66fc99e Initial load duke parents: diff changeset	513 ParScanWithoutBarrierClosure* to_space_closure_,
a61af66fc99e Initial load duke parents: diff changeset	514 ParScanWithBarrierClosure* old_gen_closure_,
a61af66fc99e Initial load duke parents: diff changeset	515 ParRootScanWithoutBarrierClosure* to_space_root_closure_,
a61af66fc99e Initial load duke parents: diff changeset	516 ParNewGeneration* par_gen_,
a61af66fc99e Initial load duke parents: diff changeset	517 ParRootScanWithBarrierTwoGensClosure* old_gen_root_closure_,
a61af66fc99e Initial load duke parents: diff changeset	518 ObjToScanQueueSet* task_queues_,
a61af66fc99e Initial load duke parents: diff changeset	519 ParallelTaskTerminator* terminator_) :
a61af66fc99e Initial load duke parents: diff changeset	520
a61af66fc99e Initial load duke parents: diff changeset	521 _par_scan_state(par_scan_state_),
a61af66fc99e Initial load duke parents: diff changeset	522 _to_space_closure(to_space_closure_),
a61af66fc99e Initial load duke parents: diff changeset	523 _old_gen_closure(old_gen_closure_),
a61af66fc99e Initial load duke parents: diff changeset	524 _to_space_root_closure(to_space_root_closure_),
a61af66fc99e Initial load duke parents: diff changeset	525 _old_gen_root_closure(old_gen_root_closure_),
a61af66fc99e Initial load duke parents: diff changeset	526 _par_gen(par_gen_),
a61af66fc99e Initial load duke parents: diff changeset	527 _task_queues(task_queues_),
a61af66fc99e Initial load duke parents: diff changeset	528 _terminator(terminator_)
a61af66fc99e Initial load duke parents: diff changeset	529 {}
a61af66fc99e Initial load duke parents: diff changeset	530
a61af66fc99e Initial load duke parents: diff changeset	531 void ParEvacuateFollowersClosure::do_void() {
a61af66fc99e Initial load duke parents: diff changeset	532 ObjToScanQueue* work_q = par_scan_state()->work_queue();
a61af66fc99e Initial load duke parents: diff changeset	533
a61af66fc99e Initial load duke parents: diff changeset	534 while (true) {
a61af66fc99e Initial load duke parents: diff changeset	535
a61af66fc99e Initial load duke parents: diff changeset	536 // Scan to-space and old-gen objs until we run out of both.
a61af66fc99e Initial load duke parents: diff changeset	537 oop obj_to_scan;
a61af66fc99e Initial load duke parents: diff changeset	538 par_scan_state()->trim_queues(0);
a61af66fc99e Initial load duke parents: diff changeset	539
a61af66fc99e Initial load duke parents: diff changeset	540 // We have no local work, attempt to steal from other threads.
a61af66fc99e Initial load duke parents: diff changeset	541
a61af66fc99e Initial load duke parents: diff changeset	542 // attempt to steal work from promoted.
a61af66fc99e Initial load duke parents: diff changeset	543 if (task_queues()->steal(par_scan_state()->thread_num(),
a61af66fc99e Initial load duke parents: diff changeset	544 par_scan_state()->hash_seed(),
a61af66fc99e Initial load duke parents: diff changeset	545 obj_to_scan)) {
a61af66fc99e Initial load duke parents: diff changeset	546 bool res = work_q->push(obj_to_scan);
a61af66fc99e Initial load duke parents: diff changeset	547 assert(res, "Empty queue should have room for a push.");
a61af66fc99e Initial load duke parents: diff changeset	548
a61af66fc99e Initial load duke parents: diff changeset	549 // if successful, goto Start.
a61af66fc99e Initial load duke parents: diff changeset	550 continue;
a61af66fc99e Initial load duke parents: diff changeset	551
a61af66fc99e Initial load duke parents: diff changeset	552 // try global overflow list.
a61af66fc99e Initial load duke parents: diff changeset	553 } else if (par_gen()->take_from_overflow_list(par_scan_state())) {
a61af66fc99e Initial load duke parents: diff changeset	554 continue;
a61af66fc99e Initial load duke parents: diff changeset	555 }
a61af66fc99e Initial load duke parents: diff changeset	556
a61af66fc99e Initial load duke parents: diff changeset	557 // Otherwise, offer termination.
a61af66fc99e Initial load duke parents: diff changeset	558 par_scan_state()->start_term_time();
a61af66fc99e Initial load duke parents: diff changeset	559 if (terminator()->offer_termination()) break;
a61af66fc99e Initial load duke parents: diff changeset	560 par_scan_state()->end_term_time();
a61af66fc99e Initial load duke parents: diff changeset	561 }
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	562 assert(par_gen()->_overflow_list == NULL && par_gen()->_num_par_pushes == 0,
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	563 "Broken overflow list?");
0 a61af66fc99e Initial load duke parents: diff changeset	564 // Finish the last termination pause.
a61af66fc99e Initial load duke parents: diff changeset	565 par_scan_state()->end_term_time();
a61af66fc99e Initial load duke parents: diff changeset	566 }
a61af66fc99e Initial load duke parents: diff changeset	567
a61af66fc99e Initial load duke parents: diff changeset	568 ParNewGenTask::ParNewGenTask(ParNewGeneration* gen, Generation* next_gen,
a61af66fc99e Initial load duke parents: diff changeset	569 HeapWord* young_old_boundary, ParScanThreadStateSet* state_set) :
a61af66fc99e Initial load duke parents: diff changeset	570 AbstractGangTask("ParNewGeneration collection"),
a61af66fc99e Initial load duke parents: diff changeset	571 _gen(gen), _next_gen(next_gen),
a61af66fc99e Initial load duke parents: diff changeset	572 _young_old_boundary(young_old_boundary),
a61af66fc99e Initial load duke parents: diff changeset	573 _state_set(state_set)
a61af66fc99e Initial load duke parents: diff changeset	574 {}
a61af66fc99e Initial load duke parents: diff changeset	575
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	576 // Reset the terminator for the given number of
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	577 // active threads.
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	578 void ParNewGenTask::set_for_termination(int active_workers) {
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	579 _state_set->reset(active_workers, _gen->promotion_failed());
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	580 // Should the heap be passed in? There's only 1 for now so
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	581 // grab it instead.
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	582 GenCollectedHeap* gch = GenCollectedHeap::heap();
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	583 gch->set_n_termination(active_workers);
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	584 }
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	585
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	586 // The "i" passed to this method is the part of the work for
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	587 // this thread. It is not the worker ID. The "i" is derived
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	588 // from _started_workers which is incremented in internal_note_start()
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	589 // called in GangWorker loop() and which is called under the
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	590 // which is called under the protection of the gang monitor and is
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	591 // called after a task is started. So "i" is based on
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	592 // first-come-first-served.
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	593
4728 441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	594 void ParNewGenTask::work(uint worker_id) {
0 a61af66fc99e Initial load duke parents: diff changeset	595 GenCollectedHeap* gch = GenCollectedHeap::heap();
a61af66fc99e Initial load duke parents: diff changeset	596 // Since this is being done in a separate thread, need new resource
a61af66fc99e Initial load duke parents: diff changeset	597 // and handle marks.
a61af66fc99e Initial load duke parents: diff changeset	598 ResourceMark rm;
a61af66fc99e Initial load duke parents: diff changeset	599 HandleMark hm;
a61af66fc99e Initial load duke parents: diff changeset	600 // We would need multiple old-gen queues otherwise.
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	601 assert(gch->n_gens() == 2, "Par young collection currently only works with one older gen.");
0 a61af66fc99e Initial load duke parents: diff changeset	602
a61af66fc99e Initial load duke parents: diff changeset	603 Generation* old_gen = gch->next_gen(_gen);
a61af66fc99e Initial load duke parents: diff changeset	604
4728 441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	605 ParScanThreadState& par_scan_state = _state_set->thread_state(worker_id);
441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	606 assert(_state_set->is_valid(worker_id), "Should not have been called");
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	607
0 a61af66fc99e Initial load duke parents: diff changeset	608 par_scan_state.set_young_old_boundary(_young_old_boundary);
a61af66fc99e Initial load duke parents: diff changeset	609
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	610 KlassScanClosure klass_scan_closure(&par_scan_state.to_space_root_closure(),
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	611 gch->rem_set()->klass_rem_set());
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	612
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	613 int so = SharedHeap::SO_AllClasses \| SharedHeap::SO_Strings \| SharedHeap::SO_CodeCache;
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	614
0 a61af66fc99e Initial load duke parents: diff changeset	615 par_scan_state.start_strong_roots();
a61af66fc99e Initial load duke parents: diff changeset	616 gch->gen_process_strong_roots(_gen->level(),
989 148e5441d916 6863023: need non-perm oops in code cache for JSR 292 jrose parents: 695 diff changeset	617 true, // Process younger gens, if any,
148e5441d916 6863023: need non-perm oops in code cache for JSR 292 jrose parents: 695 diff changeset	618 // as strong roots.
148e5441d916 6863023: need non-perm oops in code cache for JSR 292 jrose parents: 695 diff changeset	619 false, // no scope; this is parallel code
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	620 true, // is scavenging
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	621 SharedHeap::ScanningOption(so),
989 148e5441d916 6863023: need non-perm oops in code cache for JSR 292 jrose parents: 695 diff changeset	622 &par_scan_state.to_space_root_closure(),
148e5441d916 6863023: need non-perm oops in code cache for JSR 292 jrose parents: 695 diff changeset	623 true, // walk all scavengable nmethods
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	624 &par_scan_state.older_gen_closure(),
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	625 &klass_scan_closure);
0 a61af66fc99e Initial load duke parents: diff changeset	626 par_scan_state.end_strong_roots();
a61af66fc99e Initial load duke parents: diff changeset	627
a61af66fc99e Initial load duke parents: diff changeset	628 // "evacuate followers".
a61af66fc99e Initial load duke parents: diff changeset	629 par_scan_state.evacuate_followers_closure().do_void();
a61af66fc99e Initial load duke parents: diff changeset	630 }
a61af66fc99e Initial load duke parents: diff changeset	631
a61af66fc99e Initial load duke parents: diff changeset	632 #ifdef _MSC_VER
a61af66fc99e Initial load duke parents: diff changeset	633 #pragma warning( push )
a61af66fc99e Initial load duke parents: diff changeset	634 #pragma warning( disable:4355 ) // 'this' : used in base member initializer list
a61af66fc99e Initial load duke parents: diff changeset	635 #endif
a61af66fc99e Initial load duke parents: diff changeset	636 ParNewGeneration::
a61af66fc99e Initial load duke parents: diff changeset	637 ParNewGeneration(ReservedSpace rs, size_t initial_byte_size, int level)
a61af66fc99e Initial load duke parents: diff changeset	638 : DefNewGeneration(rs, initial_byte_size, level, "PCopy"),
a61af66fc99e Initial load duke parents: diff changeset	639 _overflow_list(NULL),
a61af66fc99e Initial load duke parents: diff changeset	640 _is_alive_closure(this),
a61af66fc99e Initial load duke parents: diff changeset	641 _plab_stats(YoungPLABSize, PLABWeight)
a61af66fc99e Initial load duke parents: diff changeset	642 {
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	643 NOT_PRODUCT(_overflow_counter = ParGCWorkQueueOverflowInterval;)
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	644 NOT_PRODUCT(_num_par_pushes = 0;)
0 a61af66fc99e Initial load duke parents: diff changeset	645 _task_queues = new ObjToScanQueueSet(ParallelGCThreads);
a61af66fc99e Initial load duke parents: diff changeset	646 guarantee(_task_queues != NULL, "task_queues allocation failure.");
a61af66fc99e Initial load duke parents: diff changeset	647
a61af66fc99e Initial load duke parents: diff changeset	648 for (uint i1 = 0; i1 < ParallelGCThreads; i1++) {
1665 a93a9eda13f7 6962947: shared TaskQueue statistics jcoomes parents: 1552 diff changeset	649 ObjToScanQueue *q = new ObjToScanQueue();
a93a9eda13f7 6962947: shared TaskQueue statistics jcoomes parents: 1552 diff changeset	650 guarantee(q != NULL, "work_queue Allocation failure.");
a93a9eda13f7 6962947: shared TaskQueue statistics jcoomes parents: 1552 diff changeset	651 _task_queues->register_queue(i1, q);
0 a61af66fc99e Initial load duke parents: diff changeset	652 }
a61af66fc99e Initial load duke parents: diff changeset	653
a61af66fc99e Initial load duke parents: diff changeset	654 for (uint i2 = 0; i2 < ParallelGCThreads; i2++)
a61af66fc99e Initial load duke parents: diff changeset	655 _task_queues->queue(i2)->initialize();
a61af66fc99e Initial load duke parents: diff changeset	656
1836 894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	657 _overflow_stacks = NULL;
894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	658 if (ParGCUseLocalOverflow) {
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	659
d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	660 // typedef to workaround NEW_C_HEAP_ARRAY macro, which can not deal
d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	661 // with ','
d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	662 typedef Stack<oop, mtGC> GCOopStack;
d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	663
d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	664 _overflow_stacks = NEW_C_HEAP_ARRAY(GCOopStack, ParallelGCThreads, mtGC);
1836 894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	665 for (size_t i = 0; i < ParallelGCThreads; ++i) {
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	666 new (_overflow_stacks + i) Stack<oop, mtGC>();
695 becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	667 }
becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	668 }
becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	669
0 a61af66fc99e Initial load duke parents: diff changeset	670 if (UsePerfData) {
a61af66fc99e Initial load duke parents: diff changeset	671 EXCEPTION_MARK;
a61af66fc99e Initial load duke parents: diff changeset	672 ResourceMark rm;
a61af66fc99e Initial load duke parents: diff changeset	673
a61af66fc99e Initial load duke parents: diff changeset	674 const char* cname =
a61af66fc99e Initial load duke parents: diff changeset	675 PerfDataManager::counter_name(_gen_counters->name_space(), "threads");
a61af66fc99e Initial load duke parents: diff changeset	676 PerfDataManager::create_constant(SUN_GC, cname, PerfData::U_None,
a61af66fc99e Initial load duke parents: diff changeset	677 ParallelGCThreads, CHECK);
a61af66fc99e Initial load duke parents: diff changeset	678 }
a61af66fc99e Initial load duke parents: diff changeset	679 }
a61af66fc99e Initial load duke parents: diff changeset	680 #ifdef _MSC_VER
a61af66fc99e Initial load duke parents: diff changeset	681 #pragma warning( pop )
a61af66fc99e Initial load duke parents: diff changeset	682 #endif
a61af66fc99e Initial load duke parents: diff changeset	683
a61af66fc99e Initial load duke parents: diff changeset	684 // ParNewGeneration::
a61af66fc99e Initial load duke parents: diff changeset	685 ParKeepAliveClosure::ParKeepAliveClosure(ParScanWeakRefClosure* cl) :
a61af66fc99e Initial load duke parents: diff changeset	686 DefNewGeneration::KeepAliveClosure(cl), _par_cl(cl) {}
a61af66fc99e Initial load duke parents: diff changeset	687
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	688 template <class T>
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	689 void /ParNewGeneration::/ParKeepAliveClosure::do_oop_work(T* p) {
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	690 #ifdef ASSERT
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	691 {
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	692 assert(!oopDesc::is_null(*p), "expected non-null ref");
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	693 oop obj = oopDesc::load_decode_heap_oop_not_null(p);
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	694 // We never expect to see a null reference being processed
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	695 // as a weak reference.
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	696 assert(obj->is_oop(), "expected an oop while scanning weak refs");
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	697 }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	698 #endif // ASSERT
0 a61af66fc99e Initial load duke parents: diff changeset	699
a61af66fc99e Initial load duke parents: diff changeset	700 _par_cl->do_oop_nv(p);
a61af66fc99e Initial load duke parents: diff changeset	701
a61af66fc99e Initial load duke parents: diff changeset	702 if (Universe::heap()->is_in_reserved(p)) {
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	703 oop obj = oopDesc::load_decode_heap_oop_not_null(p);
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	704 _rs->write_ref_field_gc_par(p, obj);
0 a61af66fc99e Initial load duke parents: diff changeset	705 }
a61af66fc99e Initial load duke parents: diff changeset	706 }
a61af66fc99e Initial load duke parents: diff changeset	707
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	708 void /ParNewGeneration::/ParKeepAliveClosure::do_oop(oop* p) { ParKeepAliveClosure::do_oop_work(p); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	709 void /ParNewGeneration::/ParKeepAliveClosure::do_oop(narrowOop* p) { ParKeepAliveClosure::do_oop_work(p); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	710
0 a61af66fc99e Initial load duke parents: diff changeset	711 // ParNewGeneration::
a61af66fc99e Initial load duke parents: diff changeset	712 KeepAliveClosure::KeepAliveClosure(ScanWeakRefClosure* cl) :
a61af66fc99e Initial load duke parents: diff changeset	713 DefNewGeneration::KeepAliveClosure(cl) {}
a61af66fc99e Initial load duke parents: diff changeset	714
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	715 template <class T>
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	716 void /ParNewGeneration::/KeepAliveClosure::do_oop_work(T* p) {
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	717 #ifdef ASSERT
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	718 {
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	719 assert(!oopDesc::is_null(*p), "expected non-null ref");
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	720 oop obj = oopDesc::load_decode_heap_oop_not_null(p);
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	721 // We never expect to see a null reference being processed
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	722 // as a weak reference.
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	723 assert(obj->is_oop(), "expected an oop while scanning weak refs");
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	724 }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	725 #endif // ASSERT
0 a61af66fc99e Initial load duke parents: diff changeset	726
a61af66fc99e Initial load duke parents: diff changeset	727 _cl->do_oop_nv(p);
a61af66fc99e Initial load duke parents: diff changeset	728
a61af66fc99e Initial load duke parents: diff changeset	729 if (Universe::heap()->is_in_reserved(p)) {
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	730 oop obj = oopDesc::load_decode_heap_oop_not_null(p);
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	731 _rs->write_ref_field_gc_par(p, obj);
0 a61af66fc99e Initial load duke parents: diff changeset	732 }
a61af66fc99e Initial load duke parents: diff changeset	733 }
a61af66fc99e Initial load duke parents: diff changeset	734
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	735 void /ParNewGeneration::/KeepAliveClosure::do_oop(oop* p) { KeepAliveClosure::do_oop_work(p); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	736 void /ParNewGeneration::/KeepAliveClosure::do_oop(narrowOop* p) { KeepAliveClosure::do_oop_work(p); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	737
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	738 template <class T> void ScanClosureWithParBarrier::do_oop_work(T* p) {
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	739 T heap_oop = oopDesc::load_heap_oop(p);
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	740 if (!oopDesc::is_null(heap_oop)) {
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	741 oop obj = oopDesc::decode_heap_oop_not_null(heap_oop);
0 a61af66fc99e Initial load duke parents: diff changeset	742 if ((HeapWord*)obj < _boundary) {
a61af66fc99e Initial load duke parents: diff changeset	743 assert(!_g->to()->is_in_reserved(obj), "Scanning field twice?");
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	744 oop new_obj = obj->is_forwarded()
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	745 ? obj->forwardee()
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	746 : _g->DefNewGeneration::copy_to_survivor_space(obj);
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	747 oopDesc::encode_store_heap_oop_not_null(p, new_obj);
0 a61af66fc99e Initial load duke parents: diff changeset	748 }
a61af66fc99e Initial load duke parents: diff changeset	749 if (_gc_barrier) {
a61af66fc99e Initial load duke parents: diff changeset	750 // If p points to a younger generation, mark the card.
a61af66fc99e Initial load duke parents: diff changeset	751 if ((HeapWord*)obj < _gen_boundary) {
a61af66fc99e Initial load duke parents: diff changeset	752 _rs->write_ref_field_gc_par(p, obj);
a61af66fc99e Initial load duke parents: diff changeset	753 }
a61af66fc99e Initial load duke parents: diff changeset	754 }
a61af66fc99e Initial load duke parents: diff changeset	755 }
a61af66fc99e Initial load duke parents: diff changeset	756 }
a61af66fc99e Initial load duke parents: diff changeset	757
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	758 void ScanClosureWithParBarrier::do_oop(oop* p) { ScanClosureWithParBarrier::do_oop_work(p); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	759 void ScanClosureWithParBarrier::do_oop(narrowOop* p) { ScanClosureWithParBarrier::do_oop_work(p); }
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 6 diff changeset	760
0 a61af66fc99e Initial load duke parents: diff changeset	761 class ParNewRefProcTaskProxy: public AbstractGangTask {
a61af66fc99e Initial load duke parents: diff changeset	762 typedef AbstractRefProcTaskExecutor::ProcessTask ProcessTask;
a61af66fc99e Initial load duke parents: diff changeset	763 public:
a61af66fc99e Initial load duke parents: diff changeset	764 ParNewRefProcTaskProxy(ProcessTask& task, ParNewGeneration& gen,
a61af66fc99e Initial load duke parents: diff changeset	765 Generation& next_gen,
a61af66fc99e Initial load duke parents: diff changeset	766 HeapWord* young_old_boundary,
a61af66fc99e Initial load duke parents: diff changeset	767 ParScanThreadStateSet& state_set);
a61af66fc99e Initial load duke parents: diff changeset	768
a61af66fc99e Initial load duke parents: diff changeset	769 private:
4728 441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	770 virtual void work(uint worker_id);
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	771 virtual void set_for_termination(int active_workers) {
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	772 _state_set.terminator()->reset_for_reuse(active_workers);
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	773 }
0 a61af66fc99e Initial load duke parents: diff changeset	774 private:
a61af66fc99e Initial load duke parents: diff changeset	775 ParNewGeneration& _gen;
a61af66fc99e Initial load duke parents: diff changeset	776 ProcessTask& _task;
a61af66fc99e Initial load duke parents: diff changeset	777 Generation& _next_gen;
a61af66fc99e Initial load duke parents: diff changeset	778 HeapWord* _young_old_boundary;
a61af66fc99e Initial load duke parents: diff changeset	779 ParScanThreadStateSet& _state_set;
a61af66fc99e Initial load duke parents: diff changeset	780 };
a61af66fc99e Initial load duke parents: diff changeset	781
a61af66fc99e Initial load duke parents: diff changeset	782 ParNewRefProcTaskProxy::ParNewRefProcTaskProxy(
a61af66fc99e Initial load duke parents: diff changeset	783 ProcessTask& task, ParNewGeneration& gen,
a61af66fc99e Initial load duke parents: diff changeset	784 Generation& next_gen,
a61af66fc99e Initial load duke parents: diff changeset	785 HeapWord* young_old_boundary,
a61af66fc99e Initial load duke parents: diff changeset	786 ParScanThreadStateSet& state_set)
a61af66fc99e Initial load duke parents: diff changeset	787 : AbstractGangTask("ParNewGeneration parallel reference processing"),
a61af66fc99e Initial load duke parents: diff changeset	788 _gen(gen),
a61af66fc99e Initial load duke parents: diff changeset	789 _task(task),
a61af66fc99e Initial load duke parents: diff changeset	790 _next_gen(next_gen),
a61af66fc99e Initial load duke parents: diff changeset	791 _young_old_boundary(young_old_boundary),
a61af66fc99e Initial load duke parents: diff changeset	792 _state_set(state_set)
a61af66fc99e Initial load duke parents: diff changeset	793 {
a61af66fc99e Initial load duke parents: diff changeset	794 }
a61af66fc99e Initial load duke parents: diff changeset	795
4728 441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	796 void ParNewRefProcTaskProxy::work(uint worker_id)
0 a61af66fc99e Initial load duke parents: diff changeset	797 {
a61af66fc99e Initial load duke parents: diff changeset	798 ResourceMark rm;
a61af66fc99e Initial load duke parents: diff changeset	799 HandleMark hm;
4728 441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	800 ParScanThreadState& par_scan_state = _state_set.thread_state(worker_id);
0 a61af66fc99e Initial load duke parents: diff changeset	801 par_scan_state.set_young_old_boundary(_young_old_boundary);
4728 441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	802 _task.work(worker_id, par_scan_state.is_alive_closure(),
0 a61af66fc99e Initial load duke parents: diff changeset	803 par_scan_state.keep_alive_closure(),
a61af66fc99e Initial load duke parents: diff changeset	804 par_scan_state.evacuate_followers_closure());
a61af66fc99e Initial load duke parents: diff changeset	805 }
a61af66fc99e Initial load duke parents: diff changeset	806
a61af66fc99e Initial load duke parents: diff changeset	807 class ParNewRefEnqueueTaskProxy: public AbstractGangTask {
a61af66fc99e Initial load duke parents: diff changeset	808 typedef AbstractRefProcTaskExecutor::EnqueueTask EnqueueTask;
a61af66fc99e Initial load duke parents: diff changeset	809 EnqueueTask& _task;
a61af66fc99e Initial load duke parents: diff changeset	810
a61af66fc99e Initial load duke parents: diff changeset	811 public:
a61af66fc99e Initial load duke parents: diff changeset	812 ParNewRefEnqueueTaskProxy(EnqueueTask& task)
a61af66fc99e Initial load duke parents: diff changeset	813 : AbstractGangTask("ParNewGeneration parallel reference enqueue"),
a61af66fc99e Initial load duke parents: diff changeset	814 _task(task)
a61af66fc99e Initial load duke parents: diff changeset	815 { }
a61af66fc99e Initial load duke parents: diff changeset	816
4728 441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	817 virtual void work(uint worker_id)
0 a61af66fc99e Initial load duke parents: diff changeset	818 {
4728 441e946dc1af 7121618: Change type of number of GC workers to unsigned int. jmasa parents: 4095 diff changeset	819 _task.work(worker_id);
0 a61af66fc99e Initial load duke parents: diff changeset	820 }
a61af66fc99e Initial load duke parents: diff changeset	821 };
a61af66fc99e Initial load duke parents: diff changeset	822
a61af66fc99e Initial load duke parents: diff changeset	823
a61af66fc99e Initial load duke parents: diff changeset	824 void ParNewRefProcTaskExecutor::execute(ProcessTask& task)
a61af66fc99e Initial load duke parents: diff changeset	825 {
a61af66fc99e Initial load duke parents: diff changeset	826 GenCollectedHeap* gch = GenCollectedHeap::heap();
a61af66fc99e Initial load duke parents: diff changeset	827 assert(gch->kind() == CollectedHeap::GenCollectedHeap,
a61af66fc99e Initial load duke parents: diff changeset	828 "not a generational heap");
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	829 FlexibleWorkGang* workers = gch->workers();
0 a61af66fc99e Initial load duke parents: diff changeset	830 assert(workers != NULL, "Need parallel worker threads.");
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	831 _state_set.reset(workers->active_workers(), _generation.promotion_failed());
0 a61af66fc99e Initial load duke parents: diff changeset	832 ParNewRefProcTaskProxy rp_task(task, _generation, *_generation.next_gen(),
a61af66fc99e Initial load duke parents: diff changeset	833 _generation.reserved().end(), _state_set);
a61af66fc99e Initial load duke parents: diff changeset	834 workers->run_task(&rp_task);
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	835 _state_set.reset(0 /* bad value in debug if not reset */,
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	836 _generation.promotion_failed());
0 a61af66fc99e Initial load duke parents: diff changeset	837 }
a61af66fc99e Initial load duke parents: diff changeset	838
a61af66fc99e Initial load duke parents: diff changeset	839 void ParNewRefProcTaskExecutor::execute(EnqueueTask& task)
a61af66fc99e Initial load duke parents: diff changeset	840 {
a61af66fc99e Initial load duke parents: diff changeset	841 GenCollectedHeap* gch = GenCollectedHeap::heap();
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	842 FlexibleWorkGang* workers = gch->workers();
0 a61af66fc99e Initial load duke parents: diff changeset	843 assert(workers != NULL, "Need parallel worker threads.");
a61af66fc99e Initial load duke parents: diff changeset	844 ParNewRefEnqueueTaskProxy enq_task(task);
a61af66fc99e Initial load duke parents: diff changeset	845 workers->run_task(&enq_task);
a61af66fc99e Initial load duke parents: diff changeset	846 }
a61af66fc99e Initial load duke parents: diff changeset	847
a61af66fc99e Initial load duke parents: diff changeset	848 void ParNewRefProcTaskExecutor::set_single_threaded_mode()
a61af66fc99e Initial load duke parents: diff changeset	849 {
a61af66fc99e Initial load duke parents: diff changeset	850 _state_set.flush();
a61af66fc99e Initial load duke parents: diff changeset	851 GenCollectedHeap* gch = GenCollectedHeap::heap();
a61af66fc99e Initial load duke parents: diff changeset	852 gch->set_par_threads(0); // 0 ==> non-parallel.
a61af66fc99e Initial load duke parents: diff changeset	853 gch->save_marks();
a61af66fc99e Initial load duke parents: diff changeset	854 }
a61af66fc99e Initial load duke parents: diff changeset	855
a61af66fc99e Initial load duke parents: diff changeset	856 ScanClosureWithParBarrier::
a61af66fc99e Initial load duke parents: diff changeset	857 ScanClosureWithParBarrier(ParNewGeneration* g, bool gc_barrier) :
a61af66fc99e Initial load duke parents: diff changeset	858 ScanClosure(g, gc_barrier) {}
a61af66fc99e Initial load duke parents: diff changeset	859
a61af66fc99e Initial load duke parents: diff changeset	860 EvacuateFollowersClosureGeneral::
a61af66fc99e Initial load duke parents: diff changeset	861 EvacuateFollowersClosureGeneral(GenCollectedHeap* gch, int level,
a61af66fc99e Initial load duke parents: diff changeset	862 OopsInGenClosure* cur,
a61af66fc99e Initial load duke parents: diff changeset	863 OopsInGenClosure* older) :
a61af66fc99e Initial load duke parents: diff changeset	864 _gch(gch), _level(level),
a61af66fc99e Initial load duke parents: diff changeset	865 _scan_cur_or_nonheap(cur), _scan_older(older)
a61af66fc99e Initial load duke parents: diff changeset	866 {}
a61af66fc99e Initial load duke parents: diff changeset	867
a61af66fc99e Initial load duke parents: diff changeset	868 void EvacuateFollowersClosureGeneral::do_void() {
a61af66fc99e Initial load duke parents: diff changeset	869 do {
a61af66fc99e Initial load duke parents: diff changeset	870 // Beware: this call will lead to closure applications via virtual
a61af66fc99e Initial load duke parents: diff changeset	871 // calls.
a61af66fc99e Initial load duke parents: diff changeset	872 _gch->oop_since_save_marks_iterate(_level,
a61af66fc99e Initial load duke parents: diff changeset	873 _scan_cur_or_nonheap,
a61af66fc99e Initial load duke parents: diff changeset	874 _scan_older);
a61af66fc99e Initial load duke parents: diff changeset	875 } while (!_gch->no_allocs_since_save_marks(_level));
a61af66fc99e Initial load duke parents: diff changeset	876 }
a61af66fc99e Initial load duke parents: diff changeset	877
a61af66fc99e Initial load duke parents: diff changeset	878
a61af66fc99e Initial load duke parents: diff changeset	879 bool ParNewGeneration::_avoid_promotion_undo = false;
a61af66fc99e Initial load duke parents: diff changeset	880
a61af66fc99e Initial load duke parents: diff changeset	881 // A Generation that does parallel young-gen collection.
a61af66fc99e Initial load duke parents: diff changeset	882
a61af66fc99e Initial load duke parents: diff changeset	883 void ParNewGeneration::collect(bool full,
a61af66fc99e Initial load duke parents: diff changeset	884 bool clear_all_soft_refs,
a61af66fc99e Initial load duke parents: diff changeset	885 size_t size,
a61af66fc99e Initial load duke parents: diff changeset	886 bool is_tlab) {
a61af66fc99e Initial load duke parents: diff changeset	887 assert(full \|\| size > 0, "otherwise we don't want to collect");
a61af66fc99e Initial load duke parents: diff changeset	888 GenCollectedHeap* gch = GenCollectedHeap::heap();
a61af66fc99e Initial load duke parents: diff changeset	889 assert(gch->kind() == CollectedHeap::GenCollectedHeap,
a61af66fc99e Initial load duke parents: diff changeset	890 "not a CMS generational heap");
a61af66fc99e Initial load duke parents: diff changeset	891 AdaptiveSizePolicy* size_policy = gch->gen_policy()->size_policy();
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	892 FlexibleWorkGang* workers = gch->workers();
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	893 assert(workers != NULL, "Need workgang for parallel work");
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	894 int active_workers =
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	895 AdaptiveSizePolicy::calc_active_workers(workers->total_workers(),
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	896 workers->active_workers(),
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	897 Threads::number_of_non_daemon_threads());
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	898 workers->set_active_workers(active_workers);
0 a61af66fc99e Initial load duke parents: diff changeset	899 _next_gen = gch->next_gen(this);
a61af66fc99e Initial load duke parents: diff changeset	900 assert(_next_gen != NULL,
a61af66fc99e Initial load duke parents: diff changeset	901 "This must be the youngest gen, and not the only gen");
a61af66fc99e Initial load duke parents: diff changeset	902 assert(gch->n_gens() == 2,
a61af66fc99e Initial load duke parents: diff changeset	903 "Par collection currently only works with single older gen.");
a61af66fc99e Initial load duke parents: diff changeset	904 // Do we have to avoid promotion_undo?
a61af66fc99e Initial load duke parents: diff changeset	905 if (gch->collector_policy()->is_concurrent_mark_sweep_policy()) {
a61af66fc99e Initial load duke parents: diff changeset	906 set_avoid_promotion_undo(true);
a61af66fc99e Initial load duke parents: diff changeset	907 }
a61af66fc99e Initial load duke parents: diff changeset	908
a61af66fc99e Initial load duke parents: diff changeset	909 // If the next generation is too full to accomodate worst-case promotion
a61af66fc99e Initial load duke parents: diff changeset	910 // from this generation, pass on collection; let the next generation
a61af66fc99e Initial load duke parents: diff changeset	911 // do it.
a61af66fc99e Initial load duke parents: diff changeset	912 if (!collection_attempt_is_safe()) {
1888 a7214d79fcf1 6896603: CMS/GCH: collection_attempt_is_safe() ergo should use more recent data ysr parents: 1836 diff changeset	913 gch->set_incremental_collection_failed(); // slight lie, in that we did not even attempt one
0 a61af66fc99e Initial load duke parents: diff changeset	914 return;
a61af66fc99e Initial load duke parents: diff changeset	915 }
a61af66fc99e Initial load duke parents: diff changeset	916 assert(to()->is_empty(), "Else not collection_attempt_is_safe");
a61af66fc99e Initial load duke parents: diff changeset	917
a61af66fc99e Initial load duke parents: diff changeset	918 init_assuming_no_promotion_failure();
a61af66fc99e Initial load duke parents: diff changeset	919
a61af66fc99e Initial load duke parents: diff changeset	920 if (UseAdaptiveSizePolicy) {
a61af66fc99e Initial load duke parents: diff changeset	921 set_survivor_overflow(false);
a61af66fc99e Initial load duke parents: diff changeset	922 size_policy->minor_collection_begin();
a61af66fc99e Initial load duke parents: diff changeset	923 }
a61af66fc99e Initial load duke parents: diff changeset	924
6064 9d679effd28c 7166894: Add gc cause to GC logging for all collectors brutisso parents: 4911 diff changeset	925 TraceTime t1(GCCauseString("GC", gch->gc_cause()), PrintGC && !PrintGCDetails, true, gclog_or_tty);
0 a61af66fc99e Initial load duke parents: diff changeset	926 // Capture heap used before collection (for printing).
a61af66fc99e Initial load duke parents: diff changeset	927 size_t gch_prev_used = gch->used();
a61af66fc99e Initial load duke parents: diff changeset	928
a61af66fc99e Initial load duke parents: diff changeset	929 SpecializationStats::clear();
a61af66fc99e Initial load duke parents: diff changeset	930
a61af66fc99e Initial load duke parents: diff changeset	931 age_table()->clear();
263 12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	932 to()->clear(SpaceDecorator::Mangle);
0 a61af66fc99e Initial load duke parents: diff changeset	933
a61af66fc99e Initial load duke parents: diff changeset	934 gch->save_marks();
a61af66fc99e Initial load duke parents: diff changeset	935 assert(workers != NULL, "Need parallel worker threads.");
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	936 int n_workers = active_workers;
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	937
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	938 // Set the correct parallelism (number of queues) in the reference processor
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	939 ref_processor()->set_active_mt_degree(n_workers);
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	940
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	941 // Always set the terminator for the active number of workers
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	942 // because only those workers go through the termination protocol.
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	943 ParallelTaskTerminator _term(n_workers, task_queues());
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	944 ParScanThreadStateSet thread_state_set(workers->active_workers(),
0 a61af66fc99e Initial load duke parents: diff changeset	945 to(), this, _next_gen, task_queues(),
695 becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	946 _overflow_stacks, desired_plab_sz(), _term);
0 a61af66fc99e Initial load duke parents: diff changeset	947
a61af66fc99e Initial load duke parents: diff changeset	948 ParNewGenTask tsk(this, _next_gen, reserved().end(), &thread_state_set);
a61af66fc99e Initial load duke parents: diff changeset	949 gch->set_par_threads(n_workers);
a61af66fc99e Initial load duke parents: diff changeset	950 gch->rem_set()->prepare_for_younger_refs_iterate(true);
a61af66fc99e Initial load duke parents: diff changeset	951 // It turns out that even when we're using 1 thread, doing the work in a
a61af66fc99e Initial load duke parents: diff changeset	952 // separate thread causes wide variance in run times. We can't help this
a61af66fc99e Initial load duke parents: diff changeset	953 // in the multi-threaded case, but we special-case n=1 here to get
a61af66fc99e Initial load duke parents: diff changeset	954 // repeatable measurements of the 1-thread overhead of the parallel code.
a61af66fc99e Initial load duke parents: diff changeset	955 if (n_workers > 1) {
989 148e5441d916 6863023: need non-perm oops in code cache for JSR 292 jrose parents: 695 diff changeset	956 GenCollectedHeap::StrongRootsScope srs(gch);
0 a61af66fc99e Initial load duke parents: diff changeset	957 workers->run_task(&tsk);
a61af66fc99e Initial load duke parents: diff changeset	958 } else {
989 148e5441d916 6863023: need non-perm oops in code cache for JSR 292 jrose parents: 695 diff changeset	959 GenCollectedHeap::StrongRootsScope srs(gch);
0 a61af66fc99e Initial load duke parents: diff changeset	960 tsk.work(0);
a61af66fc99e Initial load duke parents: diff changeset	961 }
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	962 thread_state_set.reset(0 /* Bad value in debug if not reset */,
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	963 promotion_failed());
0 a61af66fc99e Initial load duke parents: diff changeset	964
a61af66fc99e Initial load duke parents: diff changeset	965 // Process (weak) reference objects found during scavenge.
453 c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	966 ReferenceProcessor* rp = ref_processor();
0 a61af66fc99e Initial load duke parents: diff changeset	967 IsAliveClosure is_alive(this);
a61af66fc99e Initial load duke parents: diff changeset	968 ScanWeakRefClosure scan_weak_ref(this);
a61af66fc99e Initial load duke parents: diff changeset	969 KeepAliveClosure keep_alive(&scan_weak_ref);
a61af66fc99e Initial load duke parents: diff changeset	970 ScanClosure scan_without_gc_barrier(this, false);
a61af66fc99e Initial load duke parents: diff changeset	971 ScanClosureWithParBarrier scan_with_gc_barrier(this, true);
a61af66fc99e Initial load duke parents: diff changeset	972 set_promo_failure_scan_stack_closure(&scan_without_gc_barrier);
a61af66fc99e Initial load duke parents: diff changeset	973 EvacuateFollowersClosureGeneral evacuate_followers(gch, _level,
a61af66fc99e Initial load duke parents: diff changeset	974 &scan_without_gc_barrier, &scan_with_gc_barrier);
457 27a80744a83b 6778647: snap(), snap_policy() should be renamed setup(), setup_policy() ysr parents: 454 diff changeset	975 rp->setup_policy(clear_all_soft_refs);
4095 bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	976 // Can the mt_degree be set later (at run_task() time would be best)?
bca17e38de00 6593758: RFE: Enhance GC ergonomics to dynamically choose ParallelGCThreads jmasa parents: 2369 diff changeset	977 rp->set_active_mt_degree(active_workers);
453 c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	978 if (rp->processing_is_mt()) {
0 a61af66fc99e Initial load duke parents: diff changeset	979 ParNewRefProcTaskExecutor task_executor(*this, thread_state_set);
453 c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	980 rp->process_discovered_references(&is_alive, &keep_alive,
c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	981 &evacuate_followers, &task_executor);
0 a61af66fc99e Initial load duke parents: diff changeset	982 } else {
a61af66fc99e Initial load duke parents: diff changeset	983 thread_state_set.flush();
a61af66fc99e Initial load duke parents: diff changeset	984 gch->set_par_threads(0); // 0 ==> non-parallel.
a61af66fc99e Initial load duke parents: diff changeset	985 gch->save_marks();
453 c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	986 rp->process_discovered_references(&is_alive, &keep_alive,
c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	987 &evacuate_followers, NULL);
0 a61af66fc99e Initial load duke parents: diff changeset	988 }
a61af66fc99e Initial load duke parents: diff changeset	989 if (!promotion_failed()) {
a61af66fc99e Initial load duke parents: diff changeset	990 // Swap the survivor spaces.
263 12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	991 eden()->clear(SpaceDecorator::Mangle);
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	992 from()->clear(SpaceDecorator::Mangle);
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	993 if (ZapUnusedHeapArea) {
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	994 // This is now done here because of the piece-meal mangling which
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	995 // can check for valid mangling at intermediate points in the
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	996 // collection(s). When a minor collection fails to collect
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	997 // sufficient space resizing of the young generation can occur
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	998 // an redistribute the spaces in the young generation. Mangle
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	999 // here so that unzapped regions don't get distributed to
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	1000 // other spaces.
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	1001 to()->mangle_unused_area();
12eea04c8b06 6672698: mangle_unused_area() should not remangle the entire heap at each collection. jmasa parents: 167 diff changeset	1002 }
0 a61af66fc99e Initial load duke parents: diff changeset	1003 swap_spaces();
a61af66fc99e Initial load duke parents: diff changeset	1004
1387 0bfd3fb24150 6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit. jmasa parents: 1145 diff changeset	1005 // A successful scavenge should restart the GC time limit count which is
0bfd3fb24150 6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit. jmasa parents: 1145 diff changeset	1006 // for full GC's.
0bfd3fb24150 6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit. jmasa parents: 1145 diff changeset	1007 size_policy->reset_gc_overhead_limit_count();
0bfd3fb24150 6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit. jmasa parents: 1145 diff changeset	1008
0 a61af66fc99e Initial load duke parents: diff changeset	1009 assert(to()->is_empty(), "to space should be empty now");
7609 a30e7b564541 8005972: ParNew should not update the tenuring threshold when promotion failed has occurred brutisso parents: 7451 diff changeset	1010
a30e7b564541 8005972: ParNew should not update the tenuring threshold when promotion failed has occurred brutisso parents: 7451 diff changeset	1011 adjust_desired_tenuring_threshold();
0 a61af66fc99e Initial load duke parents: diff changeset	1012 } else {
1836 894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	1013 assert(_promo_failure_scan_stack.is_empty(), "post condition");
894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	1014 _promo_failure_scan_stack.clear(true); // Clear cached segments.
894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	1015
0 a61af66fc99e Initial load duke parents: diff changeset	1016 remove_forwarding_pointers();
a61af66fc99e Initial load duke parents: diff changeset	1017 if (PrintGCDetails) {
a61af66fc99e Initial load duke parents: diff changeset	1018 gclog_or_tty->print(" (promotion failed)");
a61af66fc99e Initial load duke parents: diff changeset	1019 }
a61af66fc99e Initial load duke parents: diff changeset	1020 // All the spaces are in play for mark-sweep.
a61af66fc99e Initial load duke parents: diff changeset	1021 swap_spaces(); // Make life simpler for CMS \|\| rescan; see 6483690.
a61af66fc99e Initial load duke parents: diff changeset	1022 from()->set_next_compaction_space(to());
1888 a7214d79fcf1 6896603: CMS/GCH: collection_attempt_is_safe() ergo should use more recent data ysr parents: 1836 diff changeset	1023 gch->set_incremental_collection_failed();
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	1024 // Inform the next generation that a promotion failure occurred.
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	1025 _next_gen->promotion_failure_occurred();
6 73e96e5c30df 6624765: Guarantee failure "Unexpected dirty card found" jmasa parents: 0 diff changeset	1026
73e96e5c30df 6624765: Guarantee failure "Unexpected dirty card found" jmasa parents: 0 diff changeset	1027 // Reset the PromotionFailureALot counters.
73e96e5c30df 6624765: Guarantee failure "Unexpected dirty card found" jmasa parents: 0 diff changeset	1028 NOT_PRODUCT(Universe::heap()->reset_promotion_should_fail();)
0 a61af66fc99e Initial load duke parents: diff changeset	1029 }
a61af66fc99e Initial load duke parents: diff changeset	1030 // set new iteration safe limit for the survivor spaces
a61af66fc99e Initial load duke parents: diff changeset	1031 from()->set_concurrent_iteration_safe_limit(from()->top());
a61af66fc99e Initial load duke parents: diff changeset	1032 to()->set_concurrent_iteration_safe_limit(to()->top());
a61af66fc99e Initial load duke parents: diff changeset	1033
a61af66fc99e Initial load duke parents: diff changeset	1034 if (ResizePLAB) {
6819 2e6857353b2c 8000311: G1: ParallelGCThreads==0 broken johnc parents: 6725 diff changeset	1035 plab_stats()->adjust_desired_plab_sz(n_workers);
0 a61af66fc99e Initial load duke parents: diff changeset	1036 }
a61af66fc99e Initial load duke parents: diff changeset	1037
a61af66fc99e Initial load duke parents: diff changeset	1038 if (PrintGC && !PrintGCDetails) {
a61af66fc99e Initial load duke parents: diff changeset	1039 gch->print_heap_change(gch_prev_used);
a61af66fc99e Initial load duke parents: diff changeset	1040 }
a61af66fc99e Initial load duke parents: diff changeset	1041
1712 2d6b74c9a797 6976378: ParNew: stats are printed unconditionally in debug builds jcoomes parents: 1710 diff changeset	1042 if (PrintGCDetails && ParallelGCVerbose) {
2d6b74c9a797 6976378: ParNew: stats are printed unconditionally in debug builds jcoomes parents: 1710 diff changeset	1043 TASKQUEUE_STATS_ONLY(thread_state_set.print_termination_stats());
2d6b74c9a797 6976378: ParNew: stats are printed unconditionally in debug builds jcoomes parents: 1710 diff changeset	1044 TASKQUEUE_STATS_ONLY(thread_state_set.print_taskqueue_stats());
2d6b74c9a797 6976378: ParNew: stats are printed unconditionally in debug builds jcoomes parents: 1710 diff changeset	1045 }
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	1046
0 a61af66fc99e Initial load duke parents: diff changeset	1047 if (UseAdaptiveSizePolicy) {
a61af66fc99e Initial load duke parents: diff changeset	1048 size_policy->minor_collection_end(gch->gc_cause());
a61af66fc99e Initial load duke parents: diff changeset	1049 size_policy->avg_survived()->sample(from()->used());
a61af66fc99e Initial load duke parents: diff changeset	1050 }
a61af66fc99e Initial load duke parents: diff changeset	1051
4911 d903bf750e9f 7129514: time warp warnings after 7117303 johnc parents: 4728 diff changeset	1052 // We need to use a monotonically non-deccreasing time in ms
d903bf750e9f 7129514: time warp warnings after 7117303 johnc parents: 4728 diff changeset	1053 // or we will see time-warp warnings and os::javaTimeMillis()
d903bf750e9f 7129514: time warp warnings after 7117303 johnc parents: 4728 diff changeset	1054 // does not guarantee monotonicity.
d903bf750e9f 7129514: time warp warnings after 7117303 johnc parents: 4728 diff changeset	1055 jlong now = os::javaTimeNanos() / NANOSECS_PER_MILLISEC;
d903bf750e9f 7129514: time warp warnings after 7117303 johnc parents: 4728 diff changeset	1056 update_time_of_last_gc(now);
0 a61af66fc99e Initial load duke parents: diff changeset	1057
a61af66fc99e Initial load duke parents: diff changeset	1058 SpecializationStats::print();
a61af66fc99e Initial load duke parents: diff changeset	1059
453 c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	1060 rp->set_enqueuing_is_done(true);
c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	1061 if (rp->processing_is_mt()) {
0 a61af66fc99e Initial load duke parents: diff changeset	1062 ParNewRefProcTaskExecutor task_executor(*this, thread_state_set);
453 c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	1063 rp->enqueue_discovered_references(&task_executor);
0 a61af66fc99e Initial load duke parents: diff changeset	1064 } else {
453 c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	1065 rp->enqueue_discovered_references(NULL);
0 a61af66fc99e Initial load duke parents: diff changeset	1066 }
453 c96030fff130 6684579: SoftReference processing can be made more efficient ysr parents: 269 diff changeset	1067 rp->verify_no_references_recorded();
0 a61af66fc99e Initial load duke parents: diff changeset	1068 }
a61af66fc99e Initial load duke parents: diff changeset	1069
a61af66fc99e Initial load duke parents: diff changeset	1070 static int sum;
a61af66fc99e Initial load duke parents: diff changeset	1071 void ParNewGeneration::waste_some_time() {
a61af66fc99e Initial load duke parents: diff changeset	1072 for (int i = 0; i < 100; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1073 sum += i;
a61af66fc99e Initial load duke parents: diff changeset	1074 }
a61af66fc99e Initial load duke parents: diff changeset	1075 }
a61af66fc99e Initial load duke parents: diff changeset	1076
a61af66fc99e Initial load duke parents: diff changeset	1077 static const oop ClaimedForwardPtr = oop(0x4);
a61af66fc99e Initial load duke parents: diff changeset	1078
a61af66fc99e Initial load duke parents: diff changeset	1079 // Because of concurrency, there are times where an object for which
a61af66fc99e Initial load duke parents: diff changeset	1080 // "is_forwarded()" is true contains an "interim" forwarding pointer
a61af66fc99e Initial load duke parents: diff changeset	1081 // value. Such a value will soon be overwritten with a real value.
a61af66fc99e Initial load duke parents: diff changeset	1082 // This method requires "obj" to have a forwarding pointer, and waits, if
a61af66fc99e Initial load duke parents: diff changeset	1083 // necessary for a real one to be inserted, and returns it.
a61af66fc99e Initial load duke parents: diff changeset	1084
a61af66fc99e Initial load duke parents: diff changeset	1085 oop ParNewGeneration::real_forwardee(oop obj) {
a61af66fc99e Initial load duke parents: diff changeset	1086 oop forward_ptr = obj->forwardee();
a61af66fc99e Initial load duke parents: diff changeset	1087 if (forward_ptr != ClaimedForwardPtr) {
a61af66fc99e Initial load duke parents: diff changeset	1088 return forward_ptr;
a61af66fc99e Initial load duke parents: diff changeset	1089 } else {
a61af66fc99e Initial load duke parents: diff changeset	1090 return real_forwardee_slow(obj);
a61af66fc99e Initial load duke parents: diff changeset	1091 }
a61af66fc99e Initial load duke parents: diff changeset	1092 }
a61af66fc99e Initial load duke parents: diff changeset	1093
a61af66fc99e Initial load duke parents: diff changeset	1094 oop ParNewGeneration::real_forwardee_slow(oop obj) {
a61af66fc99e Initial load duke parents: diff changeset	1095 // Spin-read if it is claimed but not yet written by another thread.
a61af66fc99e Initial load duke parents: diff changeset	1096 oop forward_ptr = obj->forwardee();
a61af66fc99e Initial load duke parents: diff changeset	1097 while (forward_ptr == ClaimedForwardPtr) {
a61af66fc99e Initial load duke parents: diff changeset	1098 waste_some_time();
a61af66fc99e Initial load duke parents: diff changeset	1099 assert(obj->is_forwarded(), "precondition");
a61af66fc99e Initial load duke parents: diff changeset	1100 forward_ptr = obj->forwardee();
a61af66fc99e Initial load duke parents: diff changeset	1101 }
a61af66fc99e Initial load duke parents: diff changeset	1102 return forward_ptr;
a61af66fc99e Initial load duke parents: diff changeset	1103 }
a61af66fc99e Initial load duke parents: diff changeset	1104
a61af66fc99e Initial load duke parents: diff changeset	1105 #ifdef ASSERT
a61af66fc99e Initial load duke parents: diff changeset	1106 bool ParNewGeneration::is_legal_forward_ptr(oop p) {
a61af66fc99e Initial load duke parents: diff changeset	1107 return
a61af66fc99e Initial load duke parents: diff changeset	1108 (_avoid_promotion_undo && p == ClaimedForwardPtr)
a61af66fc99e Initial load duke parents: diff changeset	1109 \|\| Universe::heap()->is_in_reserved(p);
a61af66fc99e Initial load duke parents: diff changeset	1110 }
a61af66fc99e Initial load duke parents: diff changeset	1111 #endif
a61af66fc99e Initial load duke parents: diff changeset	1112
a61af66fc99e Initial load duke parents: diff changeset	1113 void ParNewGeneration::preserve_mark_if_necessary(oop obj, markOop m) {
2038 74ee0db180fa 6807801: CMS: could save/restore fewer header words during scavenge ysr parents: 1972 diff changeset	1114 if (m->must_be_preserved_for_promotion_failure(obj)) {
74ee0db180fa 6807801: CMS: could save/restore fewer header words during scavenge ysr parents: 1972 diff changeset	1115 // We should really have separate per-worker stacks, rather
74ee0db180fa 6807801: CMS: could save/restore fewer header words during scavenge ysr parents: 1972 diff changeset	1116 // than use locking of a common pair of stacks.
0 a61af66fc99e Initial load duke parents: diff changeset	1117 MutexLocker ml(ParGCRareEvent_lock);
2038 74ee0db180fa 6807801: CMS: could save/restore fewer header words during scavenge ysr parents: 1972 diff changeset	1118 preserve_mark(obj, m);
0 a61af66fc99e Initial load duke parents: diff changeset	1119 }
a61af66fc99e Initial load duke parents: diff changeset	1120 }
a61af66fc99e Initial load duke parents: diff changeset	1121
a61af66fc99e Initial load duke parents: diff changeset	1122 // Multiple GC threads may try to promote an object. If the object
a61af66fc99e Initial load duke parents: diff changeset	1123 // is successfully promoted, a forwarding pointer will be installed in
a61af66fc99e Initial load duke parents: diff changeset	1124 // the object in the young generation. This method claims the right
a61af66fc99e Initial load duke parents: diff changeset	1125 // to install the forwarding pointer before it copies the object,
a61af66fc99e Initial load duke parents: diff changeset	1126 // thus avoiding the need to undo the copy as in
a61af66fc99e Initial load duke parents: diff changeset	1127 // copy_to_survivor_space_avoiding_with_undo.
a61af66fc99e Initial load duke parents: diff changeset	1128
a61af66fc99e Initial load duke parents: diff changeset	1129 oop ParNewGeneration::copy_to_survivor_space_avoiding_promotion_undo(
a61af66fc99e Initial load duke parents: diff changeset	1130 ParScanThreadState* par_scan_state, oop old, size_t sz, markOop m) {
a61af66fc99e Initial load duke parents: diff changeset	1131 // In the sequential version, this assert also says that the object is
a61af66fc99e Initial load duke parents: diff changeset	1132 // not forwarded. That might not be the case here. It is the case that
a61af66fc99e Initial load duke parents: diff changeset	1133 // the caller observed it to be not forwarded at some time in the past.
a61af66fc99e Initial load duke parents: diff changeset	1134 assert(is_in_reserved(old), "shouldn't be scavenging this oop");
a61af66fc99e Initial load duke parents: diff changeset	1135
a61af66fc99e Initial load duke parents: diff changeset	1136 // The sequential code read "old->age()" below. That doesn't work here,
a61af66fc99e Initial load duke parents: diff changeset	1137 // since the age is in the mark word, and that might be overwritten with
a61af66fc99e Initial load duke parents: diff changeset	1138 // a forwarding pointer by a parallel thread. So we must save the mark
a61af66fc99e Initial load duke parents: diff changeset	1139 // word in a local and then analyze it.
a61af66fc99e Initial load duke parents: diff changeset	1140 oopDesc dummyOld;
a61af66fc99e Initial load duke parents: diff changeset	1141 dummyOld.set_mark(m);
a61af66fc99e Initial load duke parents: diff changeset	1142 assert(!dummyOld.is_forwarded(),
a61af66fc99e Initial load duke parents: diff changeset	1143 "should not be called with forwarding pointer mark word.");
a61af66fc99e Initial load duke parents: diff changeset	1144
a61af66fc99e Initial load duke parents: diff changeset	1145 oop new_obj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1146 oop forward_ptr;
a61af66fc99e Initial load duke parents: diff changeset	1147
a61af66fc99e Initial load duke parents: diff changeset	1148 // Try allocating obj in to-space (unless too old)
a61af66fc99e Initial load duke parents: diff changeset	1149 if (dummyOld.age() < tenuring_threshold()) {
a61af66fc99e Initial load duke parents: diff changeset	1150 new_obj = (oop)par_scan_state->alloc_in_to_space(sz);
a61af66fc99e Initial load duke parents: diff changeset	1151 if (new_obj == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1152 set_survivor_overflow(true);
a61af66fc99e Initial load duke parents: diff changeset	1153 }
a61af66fc99e Initial load duke parents: diff changeset	1154 }
a61af66fc99e Initial load duke parents: diff changeset	1155
a61af66fc99e Initial load duke parents: diff changeset	1156 if (new_obj == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1157 // Either to-space is full or we decided to promote
a61af66fc99e Initial load duke parents: diff changeset	1158 // try allocating obj tenured
a61af66fc99e Initial load duke parents: diff changeset	1159
a61af66fc99e Initial load duke parents: diff changeset	1160 // Attempt to install a null forwarding pointer (atomically),
a61af66fc99e Initial load duke parents: diff changeset	1161 // to claim the right to install the real forwarding pointer.
a61af66fc99e Initial load duke parents: diff changeset	1162 forward_ptr = old->forward_to_atomic(ClaimedForwardPtr);
a61af66fc99e Initial load duke parents: diff changeset	1163 if (forward_ptr != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1164 // someone else beat us to it.
a61af66fc99e Initial load duke parents: diff changeset	1165 return real_forwardee(old);
a61af66fc99e Initial load duke parents: diff changeset	1166 }
a61af66fc99e Initial load duke parents: diff changeset	1167
a61af66fc99e Initial load duke parents: diff changeset	1168 new_obj = _next_gen->par_promote(par_scan_state->thread_num(),
a61af66fc99e Initial load duke parents: diff changeset	1169 old, m, sz);
a61af66fc99e Initial load duke parents: diff changeset	1170
a61af66fc99e Initial load duke parents: diff changeset	1171 if (new_obj == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1172 // promotion failed, forward to self
a61af66fc99e Initial load duke parents: diff changeset	1173 _promotion_failed = true;
a61af66fc99e Initial load duke parents: diff changeset	1174 new_obj = old;
a61af66fc99e Initial load duke parents: diff changeset	1175
a61af66fc99e Initial load duke parents: diff changeset	1176 preserve_mark_if_necessary(old, m);
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	1177 // Log the size of the maiden promotion failure
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	1178 par_scan_state->log_promotion_failure(sz);
0 a61af66fc99e Initial load duke parents: diff changeset	1179 }
a61af66fc99e Initial load duke parents: diff changeset	1180
a61af66fc99e Initial load duke parents: diff changeset	1181 old->forward_to(new_obj);
a61af66fc99e Initial load duke parents: diff changeset	1182 forward_ptr = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1183 } else {
a61af66fc99e Initial load duke parents: diff changeset	1184 // Is in to-space; do copying ourselves.
a61af66fc99e Initial load duke parents: diff changeset	1185 Copy::aligned_disjoint_words((HeapWord)old, (HeapWord)new_obj, sz);
a61af66fc99e Initial load duke parents: diff changeset	1186 forward_ptr = old->forward_to_atomic(new_obj);
a61af66fc99e Initial load duke parents: diff changeset	1187 // Restore the mark word copied above.
a61af66fc99e Initial load duke parents: diff changeset	1188 new_obj->set_mark(m);
a61af66fc99e Initial load duke parents: diff changeset	1189 // Increment age if obj still in new generation
a61af66fc99e Initial load duke parents: diff changeset	1190 new_obj->incr_age();
a61af66fc99e Initial load duke parents: diff changeset	1191 par_scan_state->age_table()->add(new_obj, sz);
a61af66fc99e Initial load duke parents: diff changeset	1192 }
a61af66fc99e Initial load duke parents: diff changeset	1193 assert(new_obj != NULL, "just checking");
a61af66fc99e Initial load duke parents: diff changeset	1194
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1195 #ifndef PRODUCT
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1196 // This code must come after the CAS test, or it will print incorrect
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1197 // information.
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1198 if (TraceScavenge) {
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1199 gclog_or_tty->print_cr("{%s %s " PTR_FORMAT " -> " PTR_FORMAT " (%d)}",
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1200 is_in_reserved(new_obj) ? "copying" : "tenuring",
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1201 new_obj->klass()->internal_name(), old, new_obj, new_obj->size());
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1202 }
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1203 #endif
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1204
0 a61af66fc99e Initial load duke parents: diff changeset	1205 if (forward_ptr == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1206 oop obj_to_push = new_obj;
a61af66fc99e Initial load duke parents: diff changeset	1207 if (par_scan_state->should_be_partially_scanned(obj_to_push, old)) {
a61af66fc99e Initial load duke parents: diff changeset	1208 // Length field used as index of next element to be scanned.
a61af66fc99e Initial load duke parents: diff changeset	1209 // Real length can be obtained from real_forwardee()
a61af66fc99e Initial load duke parents: diff changeset	1210 arrayOop(old)->set_length(0);
a61af66fc99e Initial load duke parents: diff changeset	1211 obj_to_push = old;
a61af66fc99e Initial load duke parents: diff changeset	1212 assert(obj_to_push->is_forwarded() && obj_to_push->forwardee() != obj_to_push,
a61af66fc99e Initial load duke parents: diff changeset	1213 "push forwarded object");
a61af66fc99e Initial load duke parents: diff changeset	1214 }
a61af66fc99e Initial load duke parents: diff changeset	1215 // Push it on one of the queues of to-be-scanned objects.
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1216 bool simulate_overflow = false;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1217 NOT_PRODUCT(
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1218 if (ParGCWorkQueueOverflowALot && should_simulate_overflow()) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1219 // simulate a stack overflow
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1220 simulate_overflow = true;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1221 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1222 )
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1223 if (simulate_overflow \|\| !par_scan_state->work_queue()->push(obj_to_push)) {
0 a61af66fc99e Initial load duke parents: diff changeset	1224 // Add stats for overflow pushes.
a61af66fc99e Initial load duke parents: diff changeset	1225 if (Verbose && PrintGCDetails) {
a61af66fc99e Initial load duke parents: diff changeset	1226 gclog_or_tty->print("queue overflow!\n");
a61af66fc99e Initial load duke parents: diff changeset	1227 }
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1228 push_on_overflow_list(old, par_scan_state);
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	1229 TASKQUEUE_STATS_ONLY(par_scan_state->taskqueue_stats().record_overflow(0));
0 a61af66fc99e Initial load duke parents: diff changeset	1230 }
a61af66fc99e Initial load duke parents: diff changeset	1231
a61af66fc99e Initial load duke parents: diff changeset	1232 return new_obj;
a61af66fc99e Initial load duke parents: diff changeset	1233 }
a61af66fc99e Initial load duke parents: diff changeset	1234
a61af66fc99e Initial load duke parents: diff changeset	1235 // Oops. Someone beat us to it. Undo the allocation. Where did we
a61af66fc99e Initial load duke parents: diff changeset	1236 // allocate it?
a61af66fc99e Initial load duke parents: diff changeset	1237 if (is_in_reserved(new_obj)) {
a61af66fc99e Initial load duke parents: diff changeset	1238 // Must be in to_space.
a61af66fc99e Initial load duke parents: diff changeset	1239 assert(to()->is_in_reserved(new_obj), "Checking");
a61af66fc99e Initial load duke parents: diff changeset	1240 if (forward_ptr == ClaimedForwardPtr) {
a61af66fc99e Initial load duke parents: diff changeset	1241 // Wait to get the real forwarding pointer value.
a61af66fc99e Initial load duke parents: diff changeset	1242 forward_ptr = real_forwardee(old);
a61af66fc99e Initial load duke parents: diff changeset	1243 }
a61af66fc99e Initial load duke parents: diff changeset	1244 par_scan_state->undo_alloc_in_to_space((HeapWord*)new_obj, sz);
a61af66fc99e Initial load duke parents: diff changeset	1245 }
a61af66fc99e Initial load duke parents: diff changeset	1246
a61af66fc99e Initial load duke parents: diff changeset	1247 return forward_ptr;
a61af66fc99e Initial load duke parents: diff changeset	1248 }
a61af66fc99e Initial load duke parents: diff changeset	1249
a61af66fc99e Initial load duke parents: diff changeset	1250
a61af66fc99e Initial load duke parents: diff changeset	1251 // Multiple GC threads may try to promote the same object. If two
a61af66fc99e Initial load duke parents: diff changeset	1252 // or more GC threads copy the object, only one wins the race to install
a61af66fc99e Initial load duke parents: diff changeset	1253 // the forwarding pointer. The other threads have to undo their copy.
a61af66fc99e Initial load duke parents: diff changeset	1254
a61af66fc99e Initial load duke parents: diff changeset	1255 oop ParNewGeneration::copy_to_survivor_space_with_undo(
a61af66fc99e Initial load duke parents: diff changeset	1256 ParScanThreadState* par_scan_state, oop old, size_t sz, markOop m) {
a61af66fc99e Initial load duke parents: diff changeset	1257
a61af66fc99e Initial load duke parents: diff changeset	1258 // In the sequential version, this assert also says that the object is
a61af66fc99e Initial load duke parents: diff changeset	1259 // not forwarded. That might not be the case here. It is the case that
a61af66fc99e Initial load duke parents: diff changeset	1260 // the caller observed it to be not forwarded at some time in the past.
a61af66fc99e Initial load duke parents: diff changeset	1261 assert(is_in_reserved(old), "shouldn't be scavenging this oop");
a61af66fc99e Initial load duke parents: diff changeset	1262
a61af66fc99e Initial load duke parents: diff changeset	1263 // The sequential code read "old->age()" below. That doesn't work here,
a61af66fc99e Initial load duke parents: diff changeset	1264 // since the age is in the mark word, and that might be overwritten with
a61af66fc99e Initial load duke parents: diff changeset	1265 // a forwarding pointer by a parallel thread. So we must save the mark
a61af66fc99e Initial load duke parents: diff changeset	1266 // word here, install it in a local oopDesc, and then analyze it.
a61af66fc99e Initial load duke parents: diff changeset	1267 oopDesc dummyOld;
a61af66fc99e Initial load duke parents: diff changeset	1268 dummyOld.set_mark(m);
a61af66fc99e Initial load duke parents: diff changeset	1269 assert(!dummyOld.is_forwarded(),
a61af66fc99e Initial load duke parents: diff changeset	1270 "should not be called with forwarding pointer mark word.");
a61af66fc99e Initial load duke parents: diff changeset	1271
a61af66fc99e Initial load duke parents: diff changeset	1272 bool failed_to_promote = false;
a61af66fc99e Initial load duke parents: diff changeset	1273 oop new_obj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1274 oop forward_ptr;
a61af66fc99e Initial load duke parents: diff changeset	1275
a61af66fc99e Initial load duke parents: diff changeset	1276 // Try allocating obj in to-space (unless too old)
a61af66fc99e Initial load duke parents: diff changeset	1277 if (dummyOld.age() < tenuring_threshold()) {
a61af66fc99e Initial load duke parents: diff changeset	1278 new_obj = (oop)par_scan_state->alloc_in_to_space(sz);
a61af66fc99e Initial load duke parents: diff changeset	1279 if (new_obj == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1280 set_survivor_overflow(true);
a61af66fc99e Initial load duke parents: diff changeset	1281 }
a61af66fc99e Initial load duke parents: diff changeset	1282 }
a61af66fc99e Initial load duke parents: diff changeset	1283
a61af66fc99e Initial load duke parents: diff changeset	1284 if (new_obj == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1285 // Either to-space is full or we decided to promote
a61af66fc99e Initial load duke parents: diff changeset	1286 // try allocating obj tenured
a61af66fc99e Initial load duke parents: diff changeset	1287 new_obj = _next_gen->par_promote(par_scan_state->thread_num(),
a61af66fc99e Initial load duke parents: diff changeset	1288 old, m, sz);
a61af66fc99e Initial load duke parents: diff changeset	1289
a61af66fc99e Initial load duke parents: diff changeset	1290 if (new_obj == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1291 // promotion failed, forward to self
a61af66fc99e Initial load duke parents: diff changeset	1292 forward_ptr = old->forward_to_atomic(old);
a61af66fc99e Initial load duke parents: diff changeset	1293 new_obj = old;
a61af66fc99e Initial load duke parents: diff changeset	1294
a61af66fc99e Initial load duke parents: diff changeset	1295 if (forward_ptr != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1296 return forward_ptr; // someone else succeeded
a61af66fc99e Initial load duke parents: diff changeset	1297 }
a61af66fc99e Initial load duke parents: diff changeset	1298
a61af66fc99e Initial load duke parents: diff changeset	1299 _promotion_failed = true;
a61af66fc99e Initial load duke parents: diff changeset	1300 failed_to_promote = true;
a61af66fc99e Initial load duke parents: diff changeset	1301
a61af66fc99e Initial load duke parents: diff changeset	1302 preserve_mark_if_necessary(old, m);
1145 e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	1303 // Log the size of the maiden promotion failure
e018e6884bd8 6631166: CMS: better heuristics when combatting fragmentation ysr parents: 989 diff changeset	1304 par_scan_state->log_promotion_failure(sz);
0 a61af66fc99e Initial load duke parents: diff changeset	1305 }
a61af66fc99e Initial load duke parents: diff changeset	1306 } else {
a61af66fc99e Initial load duke parents: diff changeset	1307 // Is in to-space; do copying ourselves.
a61af66fc99e Initial load duke parents: diff changeset	1308 Copy::aligned_disjoint_words((HeapWord)old, (HeapWord)new_obj, sz);
a61af66fc99e Initial load duke parents: diff changeset	1309 // Restore the mark word copied above.
a61af66fc99e Initial load duke parents: diff changeset	1310 new_obj->set_mark(m);
a61af66fc99e Initial load duke parents: diff changeset	1311 // Increment age if new_obj still in new generation
a61af66fc99e Initial load duke parents: diff changeset	1312 new_obj->incr_age();
a61af66fc99e Initial load duke parents: diff changeset	1313 par_scan_state->age_table()->add(new_obj, sz);
a61af66fc99e Initial load duke parents: diff changeset	1314 }
a61af66fc99e Initial load duke parents: diff changeset	1315 assert(new_obj != NULL, "just checking");
a61af66fc99e Initial load duke parents: diff changeset	1316
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1317 #ifndef PRODUCT
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1318 // This code must come after the CAS test, or it will print incorrect
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1319 // information.
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1320 if (TraceScavenge) {
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1321 gclog_or_tty->print_cr("{%s %s " PTR_FORMAT " -> " PTR_FORMAT " (%d)}",
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1322 is_in_reserved(new_obj) ? "copying" : "tenuring",
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1323 new_obj->klass()->internal_name(), old, new_obj, new_obj->size());
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1324 }
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1325 #endif
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1326
0 a61af66fc99e Initial load duke parents: diff changeset	1327 // Now attempt to install the forwarding pointer (atomically).
a61af66fc99e Initial load duke parents: diff changeset	1328 // We have to copy the mark word before overwriting with forwarding
a61af66fc99e Initial load duke parents: diff changeset	1329 // ptr, so we can restore it below in the copy.
a61af66fc99e Initial load duke parents: diff changeset	1330 if (!failed_to_promote) {
a61af66fc99e Initial load duke parents: diff changeset	1331 forward_ptr = old->forward_to_atomic(new_obj);
a61af66fc99e Initial load duke parents: diff changeset	1332 }
a61af66fc99e Initial load duke parents: diff changeset	1333
a61af66fc99e Initial load duke parents: diff changeset	1334 if (forward_ptr == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1335 oop obj_to_push = new_obj;
a61af66fc99e Initial load duke parents: diff changeset	1336 if (par_scan_state->should_be_partially_scanned(obj_to_push, old)) {
a61af66fc99e Initial load duke parents: diff changeset	1337 // Length field used as index of next element to be scanned.
a61af66fc99e Initial load duke parents: diff changeset	1338 // Real length can be obtained from real_forwardee()
a61af66fc99e Initial load duke parents: diff changeset	1339 arrayOop(old)->set_length(0);
a61af66fc99e Initial load duke parents: diff changeset	1340 obj_to_push = old;
a61af66fc99e Initial load duke parents: diff changeset	1341 assert(obj_to_push->is_forwarded() && obj_to_push->forwardee() != obj_to_push,
a61af66fc99e Initial load duke parents: diff changeset	1342 "push forwarded object");
a61af66fc99e Initial load duke parents: diff changeset	1343 }
a61af66fc99e Initial load duke parents: diff changeset	1344 // Push it on one of the queues of to-be-scanned objects.
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1345 bool simulate_overflow = false;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1346 NOT_PRODUCT(
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1347 if (ParGCWorkQueueOverflowALot && should_simulate_overflow()) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1348 // simulate a stack overflow
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1349 simulate_overflow = true;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1350 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1351 )
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1352 if (simulate_overflow \|\| !par_scan_state->work_queue()->push(obj_to_push)) {
0 a61af66fc99e Initial load duke parents: diff changeset	1353 // Add stats for overflow pushes.
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1354 push_on_overflow_list(old, par_scan_state);
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	1355 TASKQUEUE_STATS_ONLY(par_scan_state->taskqueue_stats().record_overflow(0));
0 a61af66fc99e Initial load duke parents: diff changeset	1356 }
a61af66fc99e Initial load duke parents: diff changeset	1357
a61af66fc99e Initial load duke parents: diff changeset	1358 return new_obj;
a61af66fc99e Initial load duke parents: diff changeset	1359 }
a61af66fc99e Initial load duke parents: diff changeset	1360
a61af66fc99e Initial load duke parents: diff changeset	1361 // Oops. Someone beat us to it. Undo the allocation. Where did we
a61af66fc99e Initial load duke parents: diff changeset	1362 // allocate it?
a61af66fc99e Initial load duke parents: diff changeset	1363 if (is_in_reserved(new_obj)) {
a61af66fc99e Initial load duke parents: diff changeset	1364 // Must be in to_space.
a61af66fc99e Initial load duke parents: diff changeset	1365 assert(to()->is_in_reserved(new_obj), "Checking");
a61af66fc99e Initial load duke parents: diff changeset	1366 par_scan_state->undo_alloc_in_to_space((HeapWord*)new_obj, sz);
a61af66fc99e Initial load duke parents: diff changeset	1367 } else {
a61af66fc99e Initial load duke parents: diff changeset	1368 assert(!_avoid_promotion_undo, "Should not be here if avoiding.");
a61af66fc99e Initial load duke parents: diff changeset	1369 _next_gen->par_promote_alloc_undo(par_scan_state->thread_num(),
a61af66fc99e Initial load duke parents: diff changeset	1370 (HeapWord*)new_obj, sz);
a61af66fc99e Initial load duke parents: diff changeset	1371 }
a61af66fc99e Initial load duke parents: diff changeset	1372
a61af66fc99e Initial load duke parents: diff changeset	1373 return forward_ptr;
a61af66fc99e Initial load duke parents: diff changeset	1374 }
a61af66fc99e Initial load duke parents: diff changeset	1375
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1376 #ifndef PRODUCT
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1377 // It's OK to call this multi-threaded; the worst thing
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1378 // that can happen is that we'll get a bunch of closely
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1379 // spaced simulated oveflows, but that's OK, in fact
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1380 // probably good as it would exercise the overflow code
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1381 // under contention.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1382 bool ParNewGeneration::should_simulate_overflow() {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1383 if (_overflow_counter-- <= 0) { // just being defensive
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1384 _overflow_counter = ParGCWorkQueueOverflowInterval;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1385 return true;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1386 } else {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1387 return false;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1388 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1389 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1390 #endif
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1391
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1392 // In case we are using compressed oops, we need to be careful.
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1393 // If the object being pushed is an object array, then its length
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1394 // field keeps track of the "grey boundary" at which the next
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1395 // incremental scan will be done (see ParGCArrayScanChunk).
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1396 // When using compressed oops, this length field is kept in the
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1397 // lower 32 bits of the erstwhile klass word and cannot be used
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1398 // for the overflow chaining pointer (OCP below). As such the OCP
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1399 // would itself need to be compressed into the top 32-bits in this
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1400 // case. Unfortunately, see below, in the event that we have a
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1401 // promotion failure, the node to be pushed on the list can be
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1402 // outside of the Java heap, so the heap-based pointer compression
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1403 // would not work (we would have potential aliasing between C-heap
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1404 // and Java-heap pointers). For this reason, when using compressed
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1405 // oops, we simply use a worker-thread-local, non-shared overflow
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1406 // list in the form of a growable array, with a slightly different
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1407 // overflow stack draining strategy. If/when we start using fat
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1408 // stacks here, we can go back to using (fat) pointer chains
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1409 // (although some performance comparisons would be useful since
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1410 // single global lists have their own performance disadvantages
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1411 // as we were made painfully aware not long ago, see 6786503).
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1412 #define BUSY (oop(0x1aff1aff))
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1413 void ParNewGeneration::push_on_overflow_list(oop from_space_obj, ParScanThreadState* par_scan_state) {
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1414 assert(is_in_reserved(from_space_obj), "Should be from this generation");
695 becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	1415 if (ParGCUseLocalOverflow) {
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1416 // In the case of compressed oops, we use a private, not-shared
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1417 // overflow stack.
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1418 par_scan_state->push_on_overflow_stack(from_space_obj);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1419 } else {
695 becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	1420 assert(!UseCompressedOops, "Error");
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1421 // if the object has been forwarded to itself, then we cannot
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1422 // use the klass pointer for the linked list. Instead we have
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1423 // to allocate an oopDesc in the C-Heap and use that for the linked list.
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1424 // XXX This is horribly inefficient when a promotion failure occurs
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1425 // and should be fixed. XXX FIX ME !!!
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1426 #ifndef PRODUCT
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1427 Atomic::inc_ptr(&_num_par_pushes);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1428 assert(_num_par_pushes > 0, "Tautology");
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1429 #endif
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1430 if (from_space_obj->forwardee() == from_space_obj) {
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	1431 oopDesc* listhead = NEW_C_HEAP_ARRAY(oopDesc, 1, mtGC);
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1432 listhead->forward_to(from_space_obj);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1433 from_space_obj = listhead;
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1434 }
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1435 oop observed_overflow_list = _overflow_list;
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1436 oop cur_overflow_list;
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1437 do {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1438 cur_overflow_list = observed_overflow_list;
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1439 if (cur_overflow_list != BUSY) {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1440 from_space_obj->set_klass_to_list_ptr(cur_overflow_list);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1441 } else {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1442 from_space_obj->set_klass_to_list_ptr(NULL);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1443 }
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1444 observed_overflow_list =
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1445 (oop)Atomic::cmpxchg_ptr(from_space_obj, &_overflow_list, cur_overflow_list);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1446 } while (cur_overflow_list != observed_overflow_list);
0 a61af66fc99e Initial load duke parents: diff changeset	1447 }
a61af66fc99e Initial load duke parents: diff changeset	1448 }
a61af66fc99e Initial load duke parents: diff changeset	1449
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1450 bool ParNewGeneration::take_from_overflow_list(ParScanThreadState* par_scan_state) {
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1451 bool res;
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1452
695 becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	1453 if (ParGCUseLocalOverflow) {
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1454 res = par_scan_state->take_from_overflow_stack();
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1455 } else {
695 becb17ad5e51 6824570: ParNew: Fix memory leak introduced in 6819891 ysr parents: 679 diff changeset	1456 assert(!UseCompressedOops, "Error");
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1457 res = take_from_overflow_list_work(par_scan_state);
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1458 }
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1459 return res;
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1460 }
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1461
cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1462
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1463 // NOTE: The overflow list manipulation code here and
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1464 // in CMSCollector:: are very similar in shape,
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1465 // except that in the CMS case we thread the objects
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1466 // directly into the list via their mark word, and do
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1467 // not need to deal with special cases below related
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1468 // to chunking of object arrays and promotion failure
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1469 // handling.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1470 // CR 6797058 has been filed to attempt consolidation of
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1471 // the common code.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1472 // Because of the common code, if you make any changes in
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1473 // the code below, please check the CMS version to see if
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1474 // similar changes might be needed.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1475 // See CMSCollector::par_take_from_overflow_list() for
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1476 // more extensive documentation comments.
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1477 bool ParNewGeneration::take_from_overflow_list_work(ParScanThreadState* par_scan_state) {
0 a61af66fc99e Initial load duke parents: diff changeset	1478 ObjToScanQueue* work_q = par_scan_state->work_queue();
a61af66fc99e Initial load duke parents: diff changeset	1479 // How many to take?
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1480 size_t objsFromOverflow = MIN2((size_t)(work_q->max_elems() - work_q->size())/4,
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1481 (size_t)ParGCDesiredObjsFromOverflowList);
0 a61af66fc99e Initial load duke parents: diff changeset	1482
1836 894b1d7c7e01 6423256: GC stacks should use a better data structure jcoomes parents: 1833 diff changeset	1483 assert(!UseCompressedOops, "Error");
679 cea947c8a988 6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops ysr parents: 579 diff changeset	1484 assert(par_scan_state->overflow_stack() == NULL, "Error");
0 a61af66fc99e Initial load duke parents: diff changeset	1485 if (_overflow_list == NULL) return false;
a61af66fc99e Initial load duke parents: diff changeset	1486
a61af66fc99e Initial load duke parents: diff changeset	1487 // Otherwise, there was something there; try claiming the list.
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1488 oop prefix = (oop)Atomic::xchg_ptr(BUSY, &_overflow_list);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1489 // Trim off a prefix of at most objsFromOverflow items
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1490 Thread* tid = Thread::current();
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1491 size_t spin_count = (size_t)ParallelGCThreads;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1492 size_t sleep_time_millis = MAX2((size_t)1, objsFromOverflow/100);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1493 for (size_t spin = 0; prefix == BUSY && spin < spin_count; spin++) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1494 // someone grabbed it before we did ...
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1495 // ... we spin for a short while...
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1496 os::sleep(tid, sleep_time_millis, false);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1497 if (_overflow_list == NULL) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1498 // nothing left to take
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1499 return false;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1500 } else if (_overflow_list != BUSY) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1501 // try and grab the prefix
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1502 prefix = (oop)Atomic::xchg_ptr(BUSY, &_overflow_list);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1503 }
0 a61af66fc99e Initial load duke parents: diff changeset	1504 }
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1505 if (prefix == NULL \|\| prefix == BUSY) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1506 // Nothing to take or waited long enough
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1507 if (prefix == NULL) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1508 // Write back the NULL in case we overwrote it with BUSY above
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1509 // and it is still the same value.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1510 (void) Atomic::cmpxchg_ptr(NULL, &_overflow_list, BUSY);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1511 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1512 return false;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1513 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1514 assert(prefix != NULL && prefix != BUSY, "Error");
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1515 size_t i = 1;
0 a61af66fc99e Initial load duke parents: diff changeset	1516 oop cur = prefix;
167 feeb96a45707 6696264: assert("narrow oop can never be zero") for GCBasher & ParNewGC coleenp parents: 113 diff changeset	1517 while (i < objsFromOverflow && cur->klass_or_null() != NULL) {
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1518 i++; cur = cur->list_ptr_from_klass();
0 a61af66fc99e Initial load duke parents: diff changeset	1519 }
a61af66fc99e Initial load duke parents: diff changeset	1520
a61af66fc99e Initial load duke parents: diff changeset	1521 // Reattach remaining (suffix) to overflow list
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1522 if (cur->klass_or_null() == NULL) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1523 // Write back the NULL in lieu of the BUSY we wrote
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1524 // above and it is still the same value.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1525 if (_overflow_list == BUSY) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1526 (void) Atomic::cmpxchg_ptr(NULL, &_overflow_list, BUSY);
0 a61af66fc99e Initial load duke parents: diff changeset	1527 }
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1528 } else {
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1529 assert(cur->klass_or_null() != (Klass*)(address)BUSY, "Error");
da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1530 oop suffix = cur->list_ptr_from_klass(); // suffix will be put back on global list
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1531 cur->set_klass_to_list_ptr(NULL); // break off suffix
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1532 // It's possible that the list is still in the empty(busy) state
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1533 // we left it in a short while ago; in that case we may be
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1534 // able to place back the suffix.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1535 oop observed_overflow_list = _overflow_list;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1536 oop cur_overflow_list = observed_overflow_list;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1537 bool attached = false;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1538 while (observed_overflow_list == BUSY \|\| observed_overflow_list == NULL) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1539 observed_overflow_list =
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1540 (oop) Atomic::cmpxchg_ptr(suffix, &_overflow_list, cur_overflow_list);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1541 if (cur_overflow_list == observed_overflow_list) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1542 attached = true;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1543 break;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1544 } else cur_overflow_list = observed_overflow_list;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1545 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1546 if (!attached) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1547 // Too bad, someone else got in in between; we'll need to do a splice.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1548 // Find the last item of suffix list
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1549 oop last = suffix;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1550 while (last->klass_or_null() != NULL) {
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1551 last = last->list_ptr_from_klass();
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1552 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1553 // Atomically prepend suffix to current overflow list
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1554 observed_overflow_list = _overflow_list;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1555 do {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1556 cur_overflow_list = observed_overflow_list;
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1557 if (cur_overflow_list != BUSY) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1558 // Do the splice ...
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1559 last->set_klass_to_list_ptr(cur_overflow_list);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1560 } else { // cur_overflow_list == BUSY
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1561 last->set_klass_to_list_ptr(NULL);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1562 }
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1563 observed_overflow_list =
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1564 (oop)Atomic::cmpxchg_ptr(suffix, &_overflow_list, cur_overflow_list);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1565 } while (cur_overflow_list != observed_overflow_list);
0 a61af66fc99e Initial load duke parents: diff changeset	1566 }
a61af66fc99e Initial load duke parents: diff changeset	1567 }
a61af66fc99e Initial load duke parents: diff changeset	1568
a61af66fc99e Initial load duke parents: diff changeset	1569 // Push objects on prefix list onto this thread's work queue
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1570 assert(prefix != NULL && prefix != BUSY, "program logic");
0 a61af66fc99e Initial load duke parents: diff changeset	1571 cur = prefix;
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1572 ssize_t n = 0;
0 a61af66fc99e Initial load duke parents: diff changeset	1573 while (cur != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1574 oop obj_to_push = cur->forwardee();
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6595 diff changeset	1575 oop next = cur->list_ptr_from_klass();
0 a61af66fc99e Initial load duke parents: diff changeset	1576 cur->set_klass(obj_to_push->klass());
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1577 // This may be an array object that is self-forwarded. In that case, the list pointer
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1578 // space, cur, is not in the Java heap, but rather in the C-heap and should be freed.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1579 if (!is_in_reserved(cur)) {
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1580 // This can become a scaling bottleneck when there is work queue overflow coincident
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1581 // with promotion failure.
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1582 oopDesc* f = cur;
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6064 diff changeset	1583 FREE_C_HEAP_ARRAY(oopDesc, f, mtGC);
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1584 } else if (par_scan_state->should_be_partially_scanned(obj_to_push, cur)) {
0 a61af66fc99e Initial load duke parents: diff changeset	1585 assert(arrayOop(cur)->length() == 0, "entire array remaining to be scanned");
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1586 obj_to_push = cur;
0 a61af66fc99e Initial load duke parents: diff changeset	1587 }
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1588 bool ok = work_q->push(obj_to_push);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1589 assert(ok, "Should have succeeded");
0 a61af66fc99e Initial load duke parents: diff changeset	1590 cur = next;
a61af66fc99e Initial load duke parents: diff changeset	1591 n++;
a61af66fc99e Initial load duke parents: diff changeset	1592 }
1710 94251661de76 6970376: ParNew: shared TaskQueue statistics jcoomes parents: 1665 diff changeset	1593 TASKQUEUE_STATS_ONLY(par_scan_state->note_overflow_refill(n));
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1594 #ifndef PRODUCT
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1595 assert(_num_par_pushes >= n, "Too many pops?");
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1596 Atomic::add_ptr(-(intptr_t)n, &_num_par_pushes);
5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1597 #endif
0 a61af66fc99e Initial load duke parents: diff changeset	1598 return true;
a61af66fc99e Initial load duke parents: diff changeset	1599 }
534 5cfd8d19e546 6786503: Overflow list performance can be improved ysr parents: 481 diff changeset	1600 #undef BUSY
0 a61af66fc99e Initial load duke parents: diff changeset	1601
a61af66fc99e Initial load duke parents: diff changeset	1602 void ParNewGeneration::ref_processor_init()
a61af66fc99e Initial load duke parents: diff changeset	1603 {
a61af66fc99e Initial load duke parents: diff changeset	1604 if (_ref_processor == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1605 // Allocate and initialize a reference processor
2369 92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1606 _ref_processor =
92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1607 new ReferenceProcessor(_reserved, // span
92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1608 ParallelRefProcEnabled && (ParallelGCThreads > 1), // mt processing
92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1609 (int) ParallelGCThreads, // mt processing degree
92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1610 refs_discovery_is_mt(), // mt discovery
92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1611 (int) ParallelGCThreads, // mt discovery degree
92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1612 refs_discovery_is_atomic(), // atomic_discovery
92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1613 NULL, // is_alive_non_header
92da084fefc9 6668573: CMS: reference processing crash if ParallelCMSThreads > ParallelGCThreads ysr parents: 2038 diff changeset	1614 false); // write barrier for next field updates
0 a61af66fc99e Initial load duke parents: diff changeset	1615 }
a61af66fc99e Initial load duke parents: diff changeset	1616 }
a61af66fc99e Initial load duke parents: diff changeset	1617
a61af66fc99e Initial load duke parents: diff changeset	1618 const char* ParNewGeneration::name() const {
a61af66fc99e Initial load duke parents: diff changeset	1619 return "par new generation";
a61af66fc99e Initial load duke parents: diff changeset	1620 }

Mercurial > hg > graal-compiler

annotate src/share/vm/gc_implementation/parNew/parNewGeneration.cpp @ 10185:d50cc62e94ff